eTRIMS-Hamburg

eTraining for the Interpretation
of Man-made Scenes (eTRIMS)

Project Data

2006 - 2009
Supported by the European Community under grant IST 027113

Consortium:
University of Bonn, Institute for Photogrammetry - Wolfgang Förstner (coordinator)
University of Hamburg - Bernd Neumann
Czech Technical University in Prague, Center for Machine Perception - Radim Sára
Imperial College London, Department of Electrical and Electronic Engineering - Maria Petrou
HITeC, Hamburg Informatics Technology Center - Lothar Hotz

eTRIMS home page

The eTRIMS Team in Hamburg:

Bernd Neumann Johannes Hartz Arne Kreutzmann Kasim Terzic	Department of Informatics, University of Hamburg
Lothar Hotz	HITeC Hamburg

eTRIMS Research in Hamburg

	Scene interpretation using learnt concepts The system SCENIC ("SCENe Interpretation by Configuration") has been extended in several ways to handle learnt concepts in low-level and high-level scene interpretation processes. A middle layer has been developed which mediates between low-level image analysis and high-level interpretation. It maps low-level data to specific object views and allows high-level hypotheses to control low-level processing. A standardised interpretation process has been defined which combines bottom-up part-whole reasoning with goal-dependent top-down processing and can cope with incomplete evidence. Learnt concepts can be integrated in the SCENIC knowledge base using the Semantic Web language OWL. A GUI has been implemented to control and visualise the interpretation process.

	Version Space Learning of spatial structures Conceptual descriptions of aggregates play an essential role in model-based scene interpretation. An aggregate specifies a set of objects with certain properties and relations which together constitute a meaningful scene entity. Aggregate concepts for spatially related objects can be learnt from positive and negative examples using Version Space Learning, introduced by Mitchell. Our approach features a rich representation language encompassing quantitative and qualitative spatial attributes and relations. Using examples from the buildings domain, we have shown that aggregate concepts for window arrays, balconies and other structures can in fact be learnt from annotated images and successfully employed in the conceptual knowledge base of a scene interpretation system.

	Probabilistic models for compositional hierarchies In the scene interpretation system SCENIC, high-level knowledge about visual scenes is currently represented by means of a logic-based knowledge representation language, using taxonomical and compositional hierarchies. We are developing a probabilistic framework which can be combined with our hierarchical knowledge structures. It will support probabilistic learning methods for high-level structures such as building facades, and provide probabilistic guidance for stepwise scene interpretation. By imposing intuitive abstraction properties on compositional hierarchies, evidence propagation during the interpretation process may become computationally feasible even in large knowledge bases.

	Context-aware Classification Appearance-based classification is a difficult task in many domains due to ambiguous evidence. Knowledge about the relationships between objects in the scene can help to resolve this problem. We have developed a new probabilistic classification framework based on the cooperation of decision trees and Bayesian Compositional Hierarchies, and have shown that introducing contextual knowledge in the form of dynamic priors significantly improves classification performance in the facade domain. For example, in storey images (left), balcony doors and windows could be much better distinguished once a probabilistic context was established.

	Learning Probabilistic Structure Graphs A novel domain-independent method for learning the probabilistic structure of 2D object configuration has been developed. The approach is graph-based and uses Maximum Common Subgraph Isomorphism to generalise a model graph over a set of training examples. A full probabilistic model, called a Probabilistic Structure Graph (PSG), is obtained by generalising the structural probabilities learnt from the examples. Models learnt from the eTRIMS image database have been used for classifying unknown object configurations, and an average classification rate of ca. 80% has been obtained. Additionally, bottom-up scene interpretation based solely on learnt models has been performed with promising results.

Publications

Terziç, K.; Hotz, L.; Sochman, J.: Interpreting Structures in Man-Made Scenes: Combining Low-Level and High-Level Structure Sources. In: Proc. International Conference on Agents and Artificial Intelligence (ICAART 2010), Valencia, January 2010 PDF

Terziç, K.; Neumann, B.: Integrating Context Priors into a Decision Tree Classification Scheme. In: Proc. International Conference on Machine Vision, Image Processing, and Pattern Analysis (MVIPPA 2009), Bangkok, December 2009 PDF

Hartz, J.: Learning Probabilistic Structure Graphs for Classification and Detection of Object Structures. To appear in: IEEE Proceedings of the International Conference on Machine Learning and Applications, Miami (Florida, USA), December 2009 PDF

Terzic, K., Neumann, B.: Decision Trees for Probabilistic Top-down and Bottom-up Integration. Report FBI-HH-B-288/09, Department of Informatics, University of Hamburg, 2009 PDF

Kreutzmann, A.; Terziç, K.; Neumann, B.: Context-aware Classification for Incremental Scene Interpretation. In: Proc. Workshop on Use of Context in Vision Processing (UCVP 2009), Boston, Nov 2009 PDF

Hartz, J.; Hotz, L.; Neumann, B.; Terzic, K.: Automatic Incremental Model Learning for Scene Interpretation. Proceedings of the International Conference on Computational Intelligence (IASTED CI-2009), Honolulu (Hawaii, USA), August 2009 PDF

Hotz, L.; Neumann, B.; Terzic, K.: High-level Expectations for Low-level Image Processing. Proc. KI-2008, Springer, 2008 PDF

Neumann, B.: Bayesian Compositional Hierarchies - A Probabilistic Structure for Scene Interpretation. Report FBI-HH-B-282/08, Department of Informatics, Hamburg University, 2008. Revised (Section 6) Dec. 2008 PDF

Möller, R.; Neumann, B.: Ontology-Based Reasoning Techniques for Multimedia Interpretation and Retrieval. In: Y. Kompatsiaris, P. Hobson (Eds.): Semantic Multimedia and Ontologies: Theory and Applications, Springer 2008, 55-98
PDF (final draft)

Hartz, J.; Neumann, B.: Learning a Knowledge Base of Ontological Concepts for High-Level Scene Interpretation. International Conference on Machine Learning and Applications, Cincinnati (Ohio, USA), December 2007 PDF

Terzic, K.; Hotz, L.; Neumann, B. : Division of Work During Behaviour Recognition - The SCENIC Approach. Workshop on Behaviour Monitoring and Interpretation, KI-2007 PDF

Hartz, J.; Neumann, B.: Version Space Learning of Spatial Structures for High-Level Scene Interpretation. TR FBI-B-277/07, Department of Informatics, University of Hamburg, 2007 PDF

Hotz, L.; Neumann, B.; Terzic, K.; Sochman, J.: Feedback between Low-Level and High-Level Image Processing. TR FBI-B-278/07, Department of Informatics, University of Hamburg, 2007 PDF