printer friendly version

Intelligent image analysis

Modern video sensor technology is the technology of choice for security surveillance, given the fact that human observers are hardly able to monitor the increasing number of video channels without technical assistance.

New digital product and system design concepts that use intelligent video codecs, intelligent IP cameras and network video systems offer high-quality, reliable image analysis in potentially dangerous situations, thus optimising security system effectiveness.

Viewing two monitors with automatic image switchover soon becomes overtaxing for human eyes. Experiments in the US have shown that a human observer will miss up to 45% of all activity in scenes after only 12 minutes, and up to 95% after 22 minutes. For security applications, that could have disastrous consequences. Demand is growing for intelligent video analysis methods that continuously examine available camera signals for relevant objects, thereby relieving the observer of information overload. Manufacturers are scrambling to continuously come up with new solutions to meet this boost in demand and trend toward completely digitised video systems.

Video technology at the crossroads

Video technology is currently undergoing a major shift. With the still widely used analog technology, the video sensor is connected upstream as an autonomous unit that analyses the video signal and displays the analysis results in the image. The video matrix is the central control unit, which not only handles the system topology management, but also the alarm processing.

In the digital world of video-over-IP, the previously separately implemented device concepts are increasingly being merged into 'intelligent video codecs' such as the Sistore CX series. These digitise the incoming video signals in realtime (40 milliseconds per frame) and ideally compress them into an MPEG-4 format. Based on the digitised data, the tasks of sensing, storing and subsequent processing are integrated into a single unit. By adjusting the priority given to each of these tasks, the processing power of the unit can be optimised for its application, thereby also achieving an optimised price/performance ratio.

As this shift progresses, digitised technology is extending further and further into the field level. IP cameras are becoming more powerful, for example, and can perform ever more complicated image processing routines. Network video recorder (NVR) will also be more powerful in the future, and soon be capable not only of intelligently scanning stored image data, but also evaluating data streams from IP cameras (NOOSE - network of optical sensors) in realtime. Modern video sensors were not only enhanced in terms of their operation capabilities. They are now also easier to configure. All the user needs to do to improve detection reliability in a few easy steps, for example, is to set the scene geometry along with a few basic parameters.

Unlike analog sensors with special hardware add-ons, digital sensors incorporate powerful digital signal processors (DSPs) that are capable of running much more sophisticated software algorithms. These DSPs now form the processing core of intelligent video codecs, which shares the general management, storage and network tasks with an embedded host processor. There are also PC-based video sensor systems that perform security-relevant tasks via image processing utilising frame grabber cards, or directly on the IP streaming signal. In the future, chips with ready-made algorithms for image analysis will also be available.

The right surveillance system for every situation

Today, there is an impressive range of video sensors suitable for many different video analysis applications, including such applications as licence plate identification, face recognition, object tracking and smoke detection. The application scenarios are roughly divided into inanimate and animate scenes. Inanimate scenes include conventional fence, terrain or façade surveillance for perimeter protection, for example, in prisons, power plants, refineries as well as factory grounds.

As it can be assumed that the object under surveillance would rather prefer to remain undetected, it is especially important to avoid false alarms. Frequent false alarms reduce trust in the system while jeopardising overall security. Such applications require a high degree of sensitivity to recognise camouflaged objects as well as the fast detection of an alarm situation (under one second). The surveillance system must also be able to recognise moving objects in front of a known background, distinguish objects based on their size and speed, classify them according to movement patterns and pick up any attempts to sabotage the camera.

In animated scenes such as at train stations or in crowded visitor halls, the main task is to reliably detect a change in the background (abandoned objects or objects removed without permission), as well as to deduce statistical indicators or behaviour patterns from a mass of moving objects (person density, numbers of people, behaviour of people).

Identifying people and objects quickly and reliably

In the case of unequivocal object detection for the professional security business, the statistical method such as the one employed by Sistore CX EDS has proven very successful. It analyses the complete frame sequence over a prolonged period of time and, rather than comparing one frame with the next, as in the case of the differential frame method, compares the current frame with the background. This method offers a number of advantages over the differential frame method. Reliable detection is possible irrespective of the contrast conditions and the brightness distribution over the entire image. Moreover, the statistical analysis is continuously upheld during operation, because the algorithm permanently optimises its operating point, adjusting itself to the conditions in the scene. Objects are always recognised as whole objects, not simply as outlines, and the sensor adjusts itself to the contrast conditions. Interference is thereby largely avoided.

This robust object detection method is particularly useful for security as a result of the fact that it enables applications such as motion detection and object tracking, detection of abandoned objects and detection of sabotage attempts. Typical movement patterns of objects (congregating of people, birds, flying leaves, falling snow, etc,) can also be deduced from the information provided by object tracking (trajectory).

A complete view in one sweep

Security applications employ stationary cameras or cameras with a pan-tilt-zoom (PTZ) feature. These are mainly used for automatic tracking of objects that are either selected manually by the user (click-and-track) or that are detected by a second sensor camera. Communication of the object coordinates to the 'tracking sensor system' connected upstream of the PTZ camera for further object tracking streamlines surveillance tasks for the operator. But also for detection purposes, PTZ cameras combined with digitised technology permit entirely new applications. A PTZ camera can, for example, scan its whole pan range and then piece together a panorama image of the entire environment. If this operation is repeated regularly, changes to the image can be detected as well as objects tracked.

Prepared for future requirements

The future development of video sensors will see further integration and miniaturisation. Concurrently with image generation, users will also be able to preprocess the video data. A further challenge will involve the ability to track the trajectory of an object on a site plan. Intelligent management systems will be capable of displaying similar objects with the same icons on the site plan. Not to be overlooked, image sensors of the future will no longer compose an object from single pixels, but rather generate the object with the help of 'feature vectors'. This means that typical objects such as a car, a bicycle or a dog will be learned as a whole pattern during the development phase to be identified later.

Share this article:

Further reading: