Research Scientist on Human Action Recognition

Introduction:

The Centre for Robotics of MINES ParisTech, PSL Université Paris, is involved in several research projects on human motion pattern recognition applied to the Factory of the Future, the Creative and Cultural Industries and the Autonomous Vehicles. The main objective of these projects is the development of novel methodologies and technological paradigms that improve the perception of the machine and allows for natural body interactions in human-machine partnerships.

Topic:

MINES ParisTech is opening a short-term position for a research scientist on 'Development of an end-to-end embedding of a CNN into a HMM', which is horizontal on various H2020 and industrial projects. The most recent advances of Convolutional Neural Networks  (CNNs) in computer vision, have also shown promising results in human action recognition. Nevertheless, in most previous CNN-based approaches the stochasticity of the human movement, which can also be seen as a temporal evolution of video data, is not properly taken into account. The majority of the studies make use of a simple sliding window while evaluate the output as a per-frame overlap with the ground truth. Furthermore, very often CNNs are trained on a frame-level while only a very few datasets provide frame labels. In practice, this is very rarely the case, especially for real-time human action recognition in professional environments or other real-life data scenarios. Stochastic models, such as Hidden Markov Models (HMMs), manage well tasks where the inputs have a variable length. The objective of this short-term recent position is to model the emission probability of the HMM by an embedded CNN, which has more powerful image modelling capabilities than generative models such as Gaussian Mixture Models, in a bayesian framework.

The candidate will have to: 1. perform the appropriate training of the CNN (using also transfer learning when needed), 2. propose a method to convert the posterior probability of the CNN to class-conditional likelihoods, 3. add a number of hyper-parameters to mesure the effect of both the CNN structure and the states of the HMMs to the hybrid approach, 4. deliver an implementation of the approach for professional action recognition, which will be tested in various use-cases, such as the LCD TV assembly, the riveting of aircrafts parts, etc. and 5. compare the results with the so-called "tandem approach" where the CNN is not used as a classifier but to extract features that are then modelled by a GMM.

This position will give the possibility to the candidate to work with other European researchers both in the project and in the wider academic community, as well as opportunities to work with industrial partners. Finally, the candidate will be autonomous and concentrated on his/her work while some assistance in the related teaching duties of the Post-Master’s Degree AIMove is also expected.