Phd contract in the field of Engineering financed for three years by the University of Clermont Auvergne

Distributed deep learning over a network of dynamically reconfigurable wireless smart cameras.

This thesis aims to address various issues related to the wireless networking of smart cameras. However, as opposed to a "classical" wireless sensor network, we aim to use a technology based on reconfigurable electronics in order to dynamically modify the computing architecture within each camera. Indeed, the use of cameras in a wireless sensor network can prove to be paradoxical: the wireless network tolerates only a low bandwidth while the cameras produce a large amount of data. Part of our approach therefore consists in trying to solve this lock by proposing a methodology based on "real-time customization" of (mote)cameras according to the context. However, one of the main originalities of this thesis is to consider each mote (node) as a computationalelement and thus potentially part of a deep learning network. The great novelty here is not to see the learning network as a simple pipeline (path from input (image) to output (class)) but rather as a collaborative multi-input network. Since the motes aremade of dynamically reconfigurable circuits, it would then be possible to build the hardware architecture of computation according to the perceptual context.

Presentation:

Recent advances in wireless technologies allow for the easy and low-cost implementation of distributed intelligent systems. In this context, this thesis aims to address various issues related to the wireless networking of smart cameras. A networked smart camera has three main functions:-Capturing the exteroceptive information of a scene-Calculate data using these collected values to derive semantic information and minimize the amount of information to be transmitted-Communicate them through the networkHowever, as opposed to a "classic" wireless sensor network, we aim to use a technology based on reconfigurable electronics in order to dynamically modify the computing architecture of each camera. Indeed, the use of cameras in a wireless sensor network can prove to be paradoxical: the wireless network tolerates only a low bandwidthwhile the cameras produce a large amount of data. Part of our approach therefore consists in trying to solve this lock by proposing a methodology based on "context-driven use". In other words, we aim to be able to adapt each perception node (smart camera)according to its context (environment and events).To do this, we are interested in simple architectures based on a heterogeneous structure (Processor and FPGA). Associated with these systems, low-cost CMOS imagers and a wireless interface have enabled the development of the first fully connected cameras in a previousthesis.

This IoT-type approach led us to identify 3 clear issues:

-Objects (smart cameras) are capable of producing information according to requests made by other cameras. If the requests require a new hardware computing architecture, then a computing server (cloud computing) will generate the new computing architecture (reconfiguration data). In an ideal but currently unrealistic approach,the camera would be able to generate its computing configuration.

-Wireless systems can interoperate horizontally in a collaborative way: Several smart cameras can be associated in two ways:oBy aggregating data from several cameras to provide more robust or complementary data. A trivial example is the tracking of people in buildings where the cameras will "pass the hand" in order to keep the target in sight. oBy pooling computing cores. In this context, the computing cores of the different cameras can be combined to provide more computing power.

-The notion of information is linked to the notion of service, just like peripherals and the network: with such an approach (IoT), smart cameras become real Internet resources by producing data and also being a vectorfor distributed computing.

A first thesis has already led to the design of perceptual nodes (see figure on the right) and also proposes mathematical models for the interpretation of perceived information. In this approach, we have motivated the use of very simple nodes (low resolution and low computing capacity) but with a strong redundancy allowing a high robustness. In this way, the proposed system is based on an ontology (knowledge model) and sparse and ultra-distributed information.Through this new thesis, we aim to address issues of distributed learning. Motivation consists in considering each mote (node) as a computational element, thus potentially part of a deep learning network. The big novelty here is not to see the learning network as a simple pipeline (path from input (image) to output (class)) but rather as a collaborative multi-input network. Since the motes are made of dynamically reconfigurable circuits, it would then be possible to build the hardware architecture of computation according to the perceptual context.

In terms of novelty, to our knowledge there is no such approach reconciling smart camera network, contextual dynamic reconfiguration and distributed learning. In this respect, this work represents a break with research that only considers "simple" proprioceptive sensors without any real computational capabilities.

Publications:

oL. Ben Khelifa, J.-C. Quinton, and F. Berry, "LobNet: A low specification camera network platform using Ant-Cam," in International Conference on Distributed Smart Cameras, (Best PhD Paper),Stanford, United States, Sep. 2017, pp. 208-209.oL. Ben Khelifa, L. Maggiani, J.-C. Quinton, and F. Berry, "Ant-Cams Network: a cooperative network model for silly cameras," in 10th International Conference on Distributed Smart Camera (ICDSC'16), Paris, France, Sep. 2016, pp. 104-109oEmbedded System Interfacing: Design for the Internet-of-Things (IoT) and Cyber-Physical Systems (CPS) 1st Edition, by Marilyn Wolf, 2019