The 32nd Annual Conference on Neural Information Processing Systems (NeurIPS 2018) was held in Montreal between December 2 and 8. The proceedings brought together 8000 attendees and 1011 papers. It also included posters and workshops covering an array of algorithms, theories, experiments, and ideas presented by the crème de la crème of researchers on machine learning. Sieving through this massive database, the insightful platform Medium has shortlisted its influential list of papers and poster presentations. In the latter list is “Training DNNs with Hybrid Block Floating Point,” which was presented by EPFL researchers Mario Drumond, Tao Lin, Martin Jaggi, and Babak Falsafi.
Based on the NeurIPS 2018 poster sessions, Medium has prepared its list of influential contributions under five broad research categories: Understanding, Essentials, Progress, Big problems, and Future. The EPFL quartet is included under “Essentials” because their research deals with the critical aspect of training DNNs. While some data center operators employ densely packed full-precision floating-point arithmetic, others opt for fixed-point arithmetic to maximize performance density. Conversely, block floating point (BFP) could be a viable alternative since it exhibits a dynamic range and enables most DNN operations to be performed with fixed-point logic. However, BFP is hindered by its limited direct applicability. To address this problem, the EPFL researchers have proposed a hybrid BFP-FP approach, which delivers the best of both worlds: high accuracy of floating point at the superior hardware density of fixed point.
Mario Drumond is Doctoral Assistant, Tao Lin is pursuing his doctoral program, Martin Jaggi is Tenure Track Assistant Professor, and Babak Falsafi is Full Professor. They are attached to EPFL’s School of Computer and Communication Sciences.
NeurIPS is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia, and oral and poster presentations of refereed papers. NeurIPS 2019 will be held in Vancouver.