Fusion 2013 Special Session

Paulo Costa's picture

Fusion 2013 Special Session on
Evaluation of Technologies for Uncertainty Reasoning

On this page:



The ETUR Session is intended to leverage on the latest results of the ISIF’s ETURWG, which aims to bring together advances and developments in the area of evaluation of uncertainty representation. The session will be attended by ETURWG participants, as well as other researchers and practitioners interested in uncertainty evaluation. The session will summarize the state of the art in uncertainty analysis, representation, and evaluation. By having a special session, the community can collectively address a common need for the ISIF community, coordinate with researchers in the area, and jointly assess perspectives in various evaluation techniques of uncertainty assessment and key to fusion, reduction of uncertainty.


One of the main goals of information fusion is uncertainty reduction, which is dependent on the representation chosen. Uncertainty representation differs across the various levels of Information Fusion (as defined by the JDL/DFIG models). Given the advances in information fusion systems, there is a need to determine how to represent and evaluate situational (Level 2 Fusion), impact (Level 3 Fusion) and process refinement (Level 5 Fusion), which is not well standardized for the information fusion community. The goal of the ETUR Session is bring together advances and developments in the area of evaluation of uncertainty representation. The session will leverage on the current work of the ISIF’s ETURWG, a working group devoted to the topic, and bring together researchers in the area to summarize the state of the art in uncertainty analysis, representation, and evaluation.

Anticipated Impact

The impact to the ISIF community would be an organized session with a series of methods in uncertainty representation as coordinated with evaluation. The techniques discussed and questions/answers would be important for the researchers in the ISIF community; however, the bigger impact would be for the customers of information fusion systems to determine how measure, evaluate, and approve systems that assess the situation beyond Level 1 fusion. The customers of information fusion products would have some guidelines to draft requirements documentation, the gain of fusion systems over current techniques, as well as issues that important in information fusion systems designs.

Panel: Construction of a unified evaluation framework: lessons learned and future steps


Current advances in operational information fusion systems (IFSs) require common semantic ontologies for collection, storage, and access to various data, sensor, and information. One of the major contributions of information fusion is to reduce uncertainty. With an enormous amount of sensors, measurements, and systems; it is not always easy to determine the uncertainty reduction.  Thus, the Evaluation of Technologies for Uncertainty Representation Working Group (ETURWG) has formed under ISIF. Over the first two years, some lessons learned are: need to (1) define different types of uncertainty, (2) use cases for discussion, (3) multiple perspectives on a topic, and (4) metrics.  The future requires (1) standard data sets, (2) metric standards, and (3) comprehensive terminology. A use case is presented from one of the use cases on the ETURWG for Wide-Area Motion Imagery (WAMI) simultaneous tracking and identification.


Paulo Costa / Anne-Laure Jousselme


  • Valentina Dragos
  • Gavin Powell
  • Paulo Costa
  • Anne-Laure Jousselme
  • Juergen Ziegler

Reference Questions/Topics

  • What are the main lessons learned of the effort?
  • What has been accomplished and how difficult was it?
  • Is the current URREF useful? What is missing?
  • When one would use the URREF work?
  • How to better explain the URREF for people outside the group that built it?
  • Should the ontology and associate proceedings be "frozen"?
  • What is next for ETUR?

Special Session Papers

A total of 9 papers were accepted for presentation at Fusion 2013 ETUR Special Session. The following presentation schedule was defined in the FUSION 2013 program:

Thursday, July 11th, Room Sakarya A
Session Chair: Paulo Costa

Time Paper Title Authors (presenter is highlighted)
ETUR Special Session: Thursday Schedule
15:20 - 15:40 An Ontological Analysis of Uncertainties in Soft Data Valentina Dragos
15:40 - 16:00 Measures of Conflicting Evidence in Bayesian Networks for Classification Max Kruger
16:00 - 16:20 Traceable Uncertainty H. Joe Steinhauer, Alexander Karlsson, Sten F. Andler
16:20 - 16:40 Evaluating Complex Fusion Systems Based on Casual Probabilistic Models

Frank Mignet, Gregor Pavlin, Patrick de Oude, Paulo C. G. Costa

16:40 - 17:00 URREF Reliability Versus Credibility in Information Fusion (STANAG 2511) Erik Blasch, Valentina Dragos, Kathryn Laskey, Paulo Costa, Anne-Laure Jousselme, Jean Dezert













Friday, July 12th, Room Fevzi Cakmak
Session Chair: Anne-Laure Jousselme

Time Paper Title Authors (presenter is highlighted)
ETUR Special Session: Friday Schedule
13:10 - 13:30 Application of Empirical Methodology to Evaluate Information Fusion Approaches Jürgen Ziegler, Frank Detje
13:30 - 13:50 Determining Model Correctness for Situations of Belief Fusion Audun Jøsang, Paulo C.G. Costa, Erik Blasch
13:50 - 14:10 Multi-Entity Bayesian Networks Learning for Hybrid Variables in Situation Awareness Cheol Young Park, Kathryn Blackmond Laskey, Paulo C. G. Costa, Shou Matsumoto
14:10 - 14:30 Comparison of Uncertainty Representations for Missing Data in Information Retrieval Anne-Laure Jousselme, Patrick Maupin
14:30 - 14:50 Reasoning Under Uncertainty: Variations of Subjective Logic Deduction Lance M. Kaplan, Murat Sensoy, Yuqing Tang, Supriyo Chakraborty, Chatschik Bisdikian, Geeth de Mel