Making Sense : The Vision
We envisage a systems architecture which has four main components:
Collection of data encompasses automated approaches to gist-ing multimedia content (extraction of the gist of the information -- a process of abstraction which might, for example, summarise a phone message by the caller phone number, the phone number of the recipient, its duration, key words and phrases which stand out as most significant, and potentially suspicious words and phrases) and data management and resource allocation issues.
Fusion and inference involves the integration of different modalities of data of variable reliability, the estimation of missing data for use in scenario development, methods for the resolution of contradictions, and psychological studies of how analysts relate information in this setting. We will investigate whether work on GIS which has developed meta-languages such as UnCertML can be used in this process.
Analysis involves further summarizing of the fused data; we will build on existing machine learning techniques but anticipate that the characteristics of the data (which include temporal and spatial information as well as the uncertainty aspects discussed above) will pose significant new challenges. A particular challenge which will require psychological input is the drawing of relevant connections that have arisen in the fused data.
Visualization must be informed by the operational model(s) of the data analyst(s), risk assessment and by legal considerations. The key challenge is to find a flexible, interactive way of visualising the data that allows the analyst to query the data and focus attention in a natural way. A key aspect of the visualization system is that it forms both the input and output of the system allowing the complexity of the data and underlying system to be hidden beneath an intuitive interface.