BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20211207T054758Z
LOCATION:230-231-232
DTSTART;TZID=America/Chicago:20211116T113000
DTEND;TZID=America/Chicago:20211116T120000
UID:submissions.supercomputing.org_SC21_sess177_pap133@linklings.com
SUMMARY:Enable Simultaneous DNN Services Based on Deterministic Operator O
 verlap and Precise Latency Prediction
DESCRIPTION:Paper\n\nEnable Simultaneous DNN Services Based on Determinist
 ic Operator Overlap and Precise Latency Prediction\n\nCui, Zhao, Chen, Zhe
 ng, Leng...\n\nWhile user-facing services experience diurnal load patterns
 , co-locating services improve the hardware utilization. Prior work on co-
 locating services on GPUs run queries sequentially, as the latencies of th
 e queries are neither stable nor predictable when running simultaneously. 
 The input sensitiveness and the non-deterministic operator overlap are two
  primary factors of the latency unpredictability. Hence, We propose Abacus
 , a runtime system that runs multiple services simultaneously. Abacus enab
 les deterministic operator overlap to enforce latency predictability. Abac
 us consists of an overlap-aware latency predictor, a headroom-based query 
 controller, and segmental model executors. The predictor predicts the late
 ncies of the deterministic operator overlap. The controller determines the
  appropriate operator overlap for QoS guarantee of all the services. The e
 xecutors run the operators as needed to support the deterministic operator
  overlap. Our evaluation shows that Abacus reduces 51.3% of the QoS violat
 ion and improves the throughput by 29.8% on average compared with state-of
 -the-art solutions.\n\nTag: Reproducibility Badge, Machine Learning and Ar
 tificial Intelligence\n\nRegistration Category: Tech Program Reg Pass\n\nA
 ward Finalist: Best Reproducibility Advancement Finalist\n\nReproducibilit
 y Badges: Artifact Available, Artifact Functional, Results Reproduced
END:VEVENT
END:VCALENDAR