Cognitive and Flexible Learning System Operating Robust Interpretation
of Extended Real Scenes by Multi-sensors Datafusion

Project Data

2008 - 2011
Supported by the European Community under grant IST 214975

AKKA, Toulouse - David Cher (coordinator)
University of Hamburg - Bernd Neumann
University of Leeds - Tony Cohn
The University of Reading - James Ferryman
INRIA, Sophia Antipolis - Francois Bremont
Toulouse-Blagnac Airport - Lionel Bousquet

Co-Friend home page

The Co-Friend Team in Hamburg:

Bernd Neumann
Wilfried Bohlken
Roberto Fraile
Johannes Hartz
Kasim Terzic
Department of Informatics, University of Hamburg
Lothar Hotz
HITeC Hamburg

Co-Friend Research in Hamburg

Toulouse Airport High-level Scene Interpretation
A major goal of Co-Friend is to recognise aircraft service operations based on video streams from several cameras. The work in Hamburg is concerned with high-level interpretation based on primitive events such as "tanker enters zone1" or "loader in zone2". A scene interpretation system called SCENIOR (SCENe Interpretation with Ontology-based Rules) has been developed which generates interpretations according to a compositional hierarchy of activity concepts defined in an OWL ontology. Activity concepts are transformed into JESS rules which give rise to an efficient interpretation process. Common-sense rules fill in for missing observations if the context is conclusive.
Constraint Nets
Model Adaptation
A supervised learning approach has been developed for adapting activity models in case of model failure due to inadequate temporal constraints. Given a temporal constraint net in terms of inequalities and offsets between starting or ending time points of activities, an example of activities violating the constraint net can be used to modify the constraint net in a unique and minimal way such that the example becomes consistent with the modified net.
SCENIOR Components
Probabilistic Control of Incremental Real-time Interpretation
In order to achieve correct real-time interpretation of an evolving scene, several hypotheses about possible interpretations have to be entertained and rated according to their posterior probability. A probabilistic model for the temporal structure of aircraft servicing has been developed in terms of Bayesian Compositional Hierarchies (BCHs). Each BCH encodes the durations and temporal relations of a set of coherent activities, such as refuelling or loading.  Furthermore, a probabilistic clutter model has been designed for activities not related to the servicing or resulting from erroneous low-level processing. Based on these probabilistic models, alternative interpretations of incremental evidence can be rated and weak interpretations can be discarded. The interpretation system SCENIOR may keep track of up to 50 alternative interpretations in parallel.  

Bohlken, W.; Koopmann, P.; Neumann, B.: SCENIOR: Ontology-based Interpretation of Aircraft Service Activities. Technical Report FBI-HH-B‐297/11, Department of Informatics, University of Hamburg, 2011   PDF
Zimmer, F.-M.; Neumann, B.: Incremental Recognition of Multi-object Behaviour Using Hierarchical Probabilistic Models. Proc. of the 5th Workshop on Behaviour Monitoring and Interpretation, BMI'10,, issn 1613-0073, 2010, 8-21   PDF
Bohlken,W., Neumann, B.: Generation of Rules from Ontologies for High-level Scene Interpretation. In: G. Governatori et al. (eds.): Rule Interchange and Applications, Proc. International Symposium RuleML 2009, Springer LNCS 5858, 2009, 93-107   PDF


background picture