US-Army funded researchers have developed a video surveillance system that aims to use artificial intelligence to reason about human activity and flag anomalous behaviour.

The aim of the research is to interpret data from a video-feed, and eventually to predict the actions of people being observed, through semantic analysis and reasoning, Carnegie Mellon researchers said in a research paper presented last week.

"The goal is to approximate human visual intelligence in making effective and consistent detections," researchers Alessandro Oltramari and Christian Lebiere said in the paper.

The research builds on a Carnegie Mellon cognitive model  and software, both called 'ACT-R'. Oltramari and Lebiere developed a 'cognitive engine' to recognise activity using parts of the ACT-R  cognitive model.

"We devised the cognitive engine to work in a human-like fashion... trying to disambiguate the scene in terms of the most reliable conceptual structures," said the paper.

The cognitive engine uses 'partial matching' to, for example, recognise that an object such as a bag is more like a basket than a dog; and 'spreading of activation' to recognise that a bag can be associated with activities such as travel.

For example, with video footage of a person getting into a truck and driving away, the cognition engine can move from 'machine vision', which detects a truck, a person and the person and the truck disappearing from the picture, to 'a person gets into a truck and drives off'.

The cognitive engine is designed to be used as a module in the DARPA-funded Mind's Eye architecture. Mind's Eye is a project to develop a smart camera that uses machine-based visual intelligence.

The researchers have a working prototype of this technology, Oltramari told SC Magazine UK on Monday. The research is for both commercial and military use.

"Our study, which leverages on ACT-R cognitive architecture, is open source," said Oltramari.

The researchers presented the paper, 'Using Ontologies in a Cognitive-Grounded System: Automatic Action Recognition in Video Surveillance', at the Semantic Technology for Intelligence, Defense and Security 2012 conference on Wednesday.

Research into predictive video analysis is also being carried out In Europe. The EU-funded Indect project is developing algorithms for use with existing video monitoring systems. Indect is also developing technologies for threat detection, such as child pornography, in computer networks.