BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20211207T054813Z
LOCATION:240-241-242
DTSTART;TZID=America/Chicago:20211118T143000
DTEND;TZID=America/Chicago:20211118T150000
UID:submissions.supercomputing.org_SC21_sess155_pap322@linklings.com
SUMMARY:Clairvoyant Prefetching for Distributed Machine Learning I/O
DESCRIPTION:Paper\n\nClairvoyant Prefetching for Distributed Machine Learn
 ing I/O\n\nDryden, Böhringer, Ben-Nun, Hoefler\n\nI/O is emerging as a maj
 or bottleneck for machine learning training, especially in distributed env
 ironments. Indeed, at large scale, I/O takes as much as 85% of training ti
 me. Addressing this I/O bottleneck necessitates careful optimization, as o
 ptimal data ingestion pipelines differ between systems, and require a deli
 cate balance among access to local storage, external filesystems and remot
 e nodes. We introduce NoPFS, a machine learning I/O middleware, which prov
 ides a scalable, flexible and easy-to-use solution to the I/O bottleneck. 
 NoPFS uses clairvoyance: Given the seed generating the random access patte
 rn for training with SGD, it can exactly predict when and where a sample w
 ill be accessed. We combine this with an analysis of access patterns and a
  performance model to provide distributed caching policies that adapt to d
 ifferent datasets and storage hierarchies. NoPFS reduces I/O times and imp
 roves end-to-end training by up to 5.4x on the ImageNet-1k, ImageNet-22k a
 nd CosmoFlow datasets.\n\nTag: Reproducibility Badge, Memory Systems, Netw
 orks\n\nRegistration Category: Tech Program Reg Pass\n\nReproducibility Ba
 dges: Artifact Available
END:VEVENT
END:VCALENDAR