Stage : Autonomous object recognition

Autonomous object recognition in videos using Deep learning and Developmental learning

Key-words: Autonomous systems, Deep learning, Developmental learning, Unsupervised learning, Siamese Neural Networks, Similarity learning, Object discovery, Saliency, Spatio-temporal coherence

Period: 5 months, starting from February/March (subject to negotiation)

Suited skills for the candidate:

– Master in artificial intelligence or computer vision. Previous experience in neural networks would be appreciated, particularly for image recognition. Interested in developmental learning. Scientific curiosity. Ability to read/write scientific articles. Good autonomy.

– Good programming skills required (C++, python, opencv, tensorflow, git)

Frédéric Armetta and Mathieu Lefort (SMA Team) – Stefan Duffner (Imagine Team)

Localisation: LIRIS Laboratory, Lyon, France

Send applications (curriculums and motivation letters) and any request to,,

Financial reward:  500 € per month

This internship aims to contribute to the development of an autonomous object recognition system for videos. In this context, the system is exposed to a visual flow (videos) and has to extract (proto-)objects to iteratively refine its internal representation for them. The purpose of this work is to develop such a system that is able to autonomously recognize and differentiate objects thanks to the building of an internal representation for these objects. Taking inspiration from human perception and following the constructivist learning paradigm [1], we want to get rid of the use of a large labeled database, prior knowledge or sophisticated object detectors, but instead provide an autonomous development. The problem and associated objectives differ from the general way to address the learning using supervised reinforcement methods like deep learning. Indeed, no large dataset would ideally be available for the system (extraction of visual identified objects should be part of the result and not provided).

This internship will capitalize on previous promising preliminary results. The current system can first extract global shapes to catch candidate objects from the video, using simple temporal filters (for instance, a Kalman filter) and the spatio-temporal coherence of objects (movement and spatial overlap can help to define instances of objects as similar). It then uses Siamese Neural Networks [2][3] to learn a similarity metric by providing pairs of examples marked as coming from the same or from different classes. This model constructs a manifold that can be used to classify examples of unknown classes. Following these guidelines, preliminary results show that the system is able to classify new instances of objects with a good accuracy. Nevertheless, the way to maintain and make evolve this representation raises many questions that can be deepened on a short or long term according to the analysis of needs in the course of the project (catastrophic forgetting [4], active learning [5], overfitting, ability to generalize, little data, etc.).

A challenging topic that we would like to deepen during this internship relies in the possibility to use the so built internal representation to facilitate the object extraction from the videos. Indeed, without any knowledge of the objects and due to the relatively simple temporal filtering to detect candidate objects, the first extraction is coarse and highly sensitive to environmental noise. The internal representation could then be used to validate and outline the candidate objects. In this case, the object catching and the internal representation for objects evolve together. The process we want to elaborate is a self-starting one operating without external input. In other words, the so form system has to learn how to perceive efficiently in order to be able to learn more, and reciprocally. We face here a chicken-and-egg cognitive problem, also known as a representation bootstrapping problem [6].

The project could lead to a PhD position in case of financial acceptance of the associated submitted project.

Bibliography :

[1] Piaget. J. (1948), « La naissance de l’intelligence chez l’enfant »

[2] Zheng, L., Duffner, S., Idrissi, K., Garcia, C., Baskurt, A. (2016). « Pairwise Identity Verification via Linear Concentrative Metric Learning ». IEEE Transactions on Cybernetics

[3] Berlemont, S., Lefebvre, G., Duffner, S., Garcia, C. (2017). « Class-Balanced Siamese Neural Networks », Neurocomputing

[4] Goodfellow, J., Mirza, M., Xiao, D., Courville, A., Bengio, Y. (2015). « An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks », CoRR

[5] Lefort, M., Gepperth, A. (2015). « Active learning of local predictable representations with artificial curiosity ». International Conference on Development and Learning and Epigenetic Robotics (ICDL-Epirob), Providence (USA)

[6] Mazac, S., Armetta, F., Hassas, S. (2014). «On bootstrapping sensori-motor patterns for a constructivist learning system in continuous environments. In Alife 14


stage de M2 Recherche : Stratégies de contrôle décentralisées pour la régulation coopérative du trafic routier

Offre de stage de M2 Recherche (2017-2018)

Stratégies de contrôle décentralisées pour la régulation coopérative du trafic routier

Les paradigmes distribués (systèmes multi-agents) et les techniques d’Intelligence artificielle et de science des données (apprentissage) permettent d’élaborer de nouvelles stratégies de contrôle qui s’adaptent à différents horizons et/ou à différents problèmes applicatifs.

Ce type d’approche fait référence aux techniques d’apprentissage développemental (Mazac, Armetta, & Hassas, 2014), qui permettent la construction de représentation (perception-action) et son enrichissement (montée en abstraction) au cours du cycle de vie du système (méthodes en ligne). L’objectif de ce stage est de proposer de nouveaux mécanismes permettant d’étendre un modèle constructiviste existant (Guériau, Armetta, Hassas, Billot, & El Faouzi, 2016). Le modèle développé a pour vocation d’être générique, pouvant être implémenté dans différents contextes applicatifs innovants comme les systèmes coopératifs de transport intelligents (CITS) ou la robotique.

Profil attendu :
Master 2 Informatique, idéalement avec un cursus orienté Intelligence Artificielle. Un goût pour les approches multidisciplinaires est attendu. Une motivation pour les systèmes multiagents et l’IA développementale est fortement appréciée. Bonne connaissance et mise en pratique des méthodes de conception (UML) et de la programmation orientée objet.
Curiosité, autonomie et force de proposition complèteront très positivement le profil.

Durée du stage : 5 mois de février à juin 2018.

Encadrement :
– Salima Hassas (Laboratoire LIRIS, Université Claude Bernard Lyon 1,
– Nour-Eddin El Faouzi (Laboratoire LICIT, IFSTTAR, Université de Lyon,
– Maxime Guériau (LIRIS & LICIT).

Organisme d’accueil :
Ce sujet est proposé dans le cadre d’une collaboration entre un laboratoire d’informatique et un laboratoire d’ingénierie du trafic. L’équipe SMA du LIRIS est spécialisée dans les systèmes distribués à base de systèmes multi-agents et les techniques d’Intelligence Artificielle et d’apprentissage développemental. L’équipe MOMI du LICIT est à la pointe des techniques de modélisation et de régulation du trafic routier. Les deux équipes participeront à l’encadrement du stage, qui se déroulera au Laboratoire LIRIS à Villeurbanne

Description complète ici : PropositionStage_VehiculeAutonome