Nathan BAIN

Abstract: According to recent analyzes from Facebook, artificial intelligence tasks will have to undergo a reduction in their power consumption by two orders of magnitude to be sustainable. However, this decrease in consumption should not be accompanied by a decrease in the quality of the results, nor in the speed at which they are obtained. The importance of the subject is major, and all players in the field (Facebook, Google, Alibaba, Apple, etc.) have embarked on the design of specialized integrated circuits to drastically lower their energy needs. This thesis subject focuses on the integration of high energy efficiency hardware / software architectures for deep learning. This thesis is being done in collaboration with STMicroelectronics and will be based on their specific needs.
The main challenges to be overcome concern the close integration of AI accelerators in systems with strong software components, taking into account the non-functional requirements that justify designing and implementing ad-hoc hardware architectures : low to very low energy consumption, easy use of inference coprocessors at the system level, reproducibility of results, real-time and low latency calculations.
The avenues that would make it possible to advance towards obtaining energy efficiency two orders of magnitude greater than what is done on CPU for AI tasks, while maintaining intact precision, are as follows:
- Simplification of the arithmetic representation of weights and activations. Part of the thesis work will focus on the analysis of the cost of networks according to the quantification of weights and activations.
- STMicroelectronics is interested in different classes of applications for which reducing the size of networks, both from a computational point of view and from a storage point of view, is essential. Part of the study will focus on compressing networks that make sense for STM applications. The objective is to reduce the silicon footprint, while remaining within the precision constraints adapted to the application.
- The significant reduction in the size of the numbers in the network (quantification) requires new learning techniques, and recent publications show that good results are possible. The study of teacher / student or generative antagonist-type approaches, for example, could be done on networks of interest to ST, with the objective of deriving general methods.
- Finally, one avenue that could prove to be promising is the exploitation of these techniques within regular architectures allowing to do, to a certain extent, calculation inside the memory, in order to reduce energy costs.