Seminar by Pr. Dongsuk JEON (Seoul National University): "Designing an optimal hardware solution for deep neural network training"

"Designing an optimal hardware solution for deep neural network training"
By Pr. Dongsuk JEON (Seoul National University)

Abstract: The size and complexity of recent deep learning models continue to increase exponentially, causing a serious amount of hardware overheads for training those models. Contrary to inference-only hardware, neural network training is very sensitive to computation errors; hence, training processors must support high-precision computation to avoid a large performance drop, severely limiting their processing efficiency. This talk will introduce a comprehensive design approach to arrive at an optimal training processor design. More specifically, the talk will discuss how we should make important design decisions for training processors in more depth, including i) hardware-friendly training algorithms, ii) optimal data formats, and iii) processor architecture for high precision and utilization.