Scene Understanding and Movement-based Human-Robot Collaboration in Industry 4.0

Introduction:

The Centre for Robotics of MINES ParisTech, PSL Université Paris, is involved in several research projects on human motion pattern recognition applied to the Factory of the Future, the Creative and Cultural Industries and the Autonomous Vehicles. The main objective of these projects is the development of novel methodologies and technological paradigms that improve the perception of the machine and allows for natural body interactions in human-machine partnerships.

Topic:

MINES ParisTech is opening a position on Scene Understanding and Movement-based Human-Robot Collaboration (HRC) in Industry 4.0 in the context of the H2020 Collaborate project. HRC requires a smooth, natural and efficient coordination between robots and human operators. During collaborative tasks, it is crucial to establish a safe framework for the human worker by improving the robot’s perception. The machine, whether a Humanoid Robot or an AGV, must be able to detect: 1/ not only the human presence (e.g. operator or visitor) but also 2/ specific actions/intentions and/or gestures. The operators perform these gestures in production lines (e.g. to screw, to assembly, etc.) or generally in space (e.g. when a human tends to cut the motion trajectory of the AGV or is giving to it basic gestural commands, e.g. to charge or not a palette, to go back to the starting point, etc.). To achieve this goal, the candidate will develop tools and methods for both scene understanding and human tracking by using OpenPose and TensorFlow, whether the camera is fixed (e.g. on a Humanoid Robot) or in motion (e.g. on an AGV). With regards to the scene analysis, deep and machine learning will be used in order to extract meaningful features from RGB-D sensing (e.g. Random Decision Forests, Geodesic distances etc.) and segment the scene into objects, workbench etc.. Moreover, the tools of the worker will be augmented by using inertial sensors and detect whether they are used or not, when needed. For the gesture/action recognition itself, it shall be based on spatial trajectories of tracked joints and other time-series. Since the ultimate goal of this research is a safe Human-Robot Collaboration framework, early recognition and prediction techniques will be used, such as Hidden Markov Models, Gaussian Mixture Models, Deep Learning, etc. In particular, the use of time-series specific methods of Deep-Learning, recently proposed by the Centre for Robotics, shall be investigated. The candidate will deliver a generic methodology and a technological prototype for professional gesture and intentions recognition, which will be tested in 4 use-cases: a. car starter assembly, b. windshield visual quality control, c. LCD TV assembly and d. aircrafts parts manufacturing.

This position will give the possibility to the candidate to work with other European researchers both in the project and in the wider academic community, as well as opportunities to work directly with industrial partners. Moreover, the candidate will acquire transferable skills that will enhance future employability through leading and contributing to highly interactive and collaborative work. Finally, the candidate will be autonomous and concentrated on his/her work and will contribute to the project management tasks, such as preparation of the project meetings (distance calls or physical meetings in different European countries – 3 per year), reports and deliverables. Active assistance in the teaching duties of the Post-Master’s Degree AIMove is also expected.