BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20211207T054759Z
LOCATION:230-231-232
DTSTART;TZID=America/Chicago:20211116T133000
DTEND;TZID=America/Chicago:20211116T140000
UID:submissions.supercomputing.org_SC21_sess172_pap123@linklings.com
SUMMARY:ET: Re-Thinking Self-Attention for Transformer Models on GPUs
DESCRIPTION:Paper\n\nET: Re-Thinking Self-Attention for Transformer Models
  on GPUs\n\nChen, Huang, Pandey, Li, Gao...\n\nTransformer-based deep lear
 ning models have become a ubiquitous vehicle driving a variety of natural 
 language processing (NLP) -related tasks beyond their accuracy ceiling. Th
 ese models, however, also suffer from two pronounced challenges, that is, 
 gigantic model size and prolonged turnaround time. To this end, we introdu
 ce E.T., which re-thinks self-attention computation transformer models on 
 GPUs with the following contributions: First, we introduce a novel self-at
 tention architecture, which encompasses two tailored self-attention operat
 ors with corresponding sequence length-aware optimization, as well as oper
 ation reordering optimizations. Second, we achieve tensor core aware weigh
 t pruning by revamping the existing pruning algorithms, as well as designi
 ng new ones for transformers. This work goes further by introducing an att
 ention-aware adaptive pruning design. Taken together, we evaluate E.T. acr
 oss a variety of benchmarks for Transformer, BERTBASE and DistillBERT, whe
 re E.T. presents superior performance over the mainstream projects, includ
 ing the popular Nvidia Enterprise solutions; i.e., TensorRT and FasterTran
 sformer.\n\nTag: Machine Learning and Artificial Intelligence\n\nRegistrat
 ion Category: Tech Program Reg Pass
END:VEVENT
END:VCALENDAR