Intelligent image analysis

April 2007 CCTV, Surveillance & Remote Monitoring

Modern video sensor technology is the technology of choice for security surveillance, given the fact that human observers are hardly able to monitor the increasing number of video channels without technical assistance.

New digital product and system design concepts that use intelligent video codecs, intelligent IP cameras and network video systems offer high-quality, reliable image analysis in potentially dangerous situations, thus optimising security system effectiveness.

Viewing two monitors with automatic image switchover soon becomes overtaxing for human eyes. Experiments in the US have shown that a human observer will miss up to 45% of all activity in scenes after only 12 minutes, and up to 95% after 22 minutes. For security applications, that could have disastrous consequences. Demand is growing for intelligent video analysis methods that continuously examine available camera signals for relevant objects, thereby relieving the observer of information overload. Manufacturers are scrambling to continuously come up with new solutions to meet this boost in demand and trend toward completely digitised video systems.

Video technology at the crossroads

Video technology is currently undergoing a major shift. With the still widely used analog technology, the video sensor is connected upstream as an autonomous unit that analyses the video signal and displays the analysis results in the image. The video matrix is the central control unit, which not only handles the system topology management, but also the alarm processing.

In the digital world of video-over-IP, the previously separately implemented device concepts are increasingly being merged into 'intelligent video codecs' such as the Sistore CX series. These digitise the incoming video signals in realtime (40 milliseconds per frame) and ideally compress them into an MPEG-4 format. Based on the digitised data, the tasks of sensing, storing and subsequent processing are integrated into a single unit. By adjusting the priority given to each of these tasks, the processing power of the unit can be optimised for its application, thereby also achieving an optimised price/performance ratio.

As this shift progresses, digitised technology is extending further and further into the field level. IP cameras are becoming more powerful, for example, and can perform ever more complicated image processing routines. Network video recorder (NVR) will also be more powerful in the future, and soon be capable not only of intelligently scanning stored image data, but also evaluating data streams from IP cameras (NOOSE - network of optical sensors) in realtime. Modern video sensors were not only enhanced in terms of their operation capabilities. They are now also easier to configure. All the user needs to do to improve detection reliability in a few easy steps, for example, is to set the scene geometry along with a few basic parameters.

Unlike analog sensors with special hardware add-ons, digital sensors incorporate powerful digital signal processors (DSPs) that are capable of running much more sophisticated software algorithms. These DSPs now form the processing core of intelligent video codecs, which shares the general management, storage and network tasks with an embedded host processor. There are also PC-based video sensor systems that perform security-relevant tasks via image processing utilising frame grabber cards, or directly on the IP streaming signal. In the future, chips with ready-made algorithms for image analysis will also be available.

The right surveillance system for every situation

Today, there is an impressive range of video sensors suitable for many different video analysis applications, including such applications as licence plate identification, face recognition, object tracking and smoke detection. The application scenarios are roughly divided into inanimate and animate scenes. Inanimate scenes include conventional fence, terrain or façade surveillance for perimeter protection, for example, in prisons, power plants, refineries as well as factory grounds.

As it can be assumed that the object under surveillance would rather prefer to remain undetected, it is especially important to avoid false alarms. Frequent false alarms reduce trust in the system while jeopardising overall security. Such applications require a high degree of sensitivity to recognise camouflaged objects as well as the fast detection of an alarm situation (under one second). The surveillance system must also be able to recognise moving objects in front of a known background, distinguish objects based on their size and speed, classify them according to movement patterns and pick up any attempts to sabotage the camera.

In animated scenes such as at train stations or in crowded visitor halls, the main task is to reliably detect a change in the background (abandoned objects or objects removed without permission), as well as to deduce statistical indicators or behaviour patterns from a mass of moving objects (person density, numbers of people, behaviour of people).

Identifying people and objects quickly and reliably

In the case of unequivocal object detection for the professional security business, the statistical method such as the one employed by Sistore CX EDS has proven very successful. It analyses the complete frame sequence over a prolonged period of time and, rather than comparing one frame with the next, as in the case of the differential frame method, compares the current frame with the background. This method offers a number of advantages over the differential frame method. Reliable detection is possible irrespective of the contrast conditions and the brightness distribution over the entire image. Moreover, the statistical analysis is continuously upheld during operation, because the algorithm permanently optimises its operating point, adjusting itself to the conditions in the scene. Objects are always recognised as whole objects, not simply as outlines, and the sensor adjusts itself to the contrast conditions. Interference is thereby largely avoided.

This robust object detection method is particularly useful for security as a result of the fact that it enables applications such as motion detection and object tracking, detection of abandoned objects and detection of sabotage attempts. Typical movement patterns of objects (congregating of people, birds, flying leaves, falling snow, etc,) can also be deduced from the information provided by object tracking (trajectory).

A complete view in one sweep

Security applications employ stationary cameras or cameras with a pan-tilt-zoom (PTZ) feature. These are mainly used for automatic tracking of objects that are either selected manually by the user (click-and-track) or that are detected by a second sensor camera. Communication of the object coordinates to the 'tracking sensor system' connected upstream of the PTZ camera for further object tracking streamlines surveillance tasks for the operator. But also for detection purposes, PTZ cameras combined with digitised technology permit entirely new applications. A PTZ camera can, for example, scan its whole pan range and then piece together a panorama image of the entire environment. If this operation is repeated regularly, changes to the image can be detected as well as objects tracked.

Prepared for future requirements

The future development of video sensors will see further integration and miniaturisation. Concurrently with image generation, users will also be able to preprocess the video data. A further challenge will involve the ability to track the trajectory of an object on a site plan. Intelligent management systems will be capable of displaying similar objects with the same icons on the site plan. Not to be overlooked, image sensors of the future will no longer compose an object from single pixels, but rather generate the object with the help of 'feature vectors'. This means that typical objects such as a car, a bicycle or a dog will be learned as a whole pattern during the development phase to be identified later.

For more information contact Jose Machado, Siemens Southern Africa, +27 0)11 652 2149,,

Share this article:
Share via emailShare via LinkedInPrint this page

Further reading:

Why rehearsals and drills for crises and evacuation are so important
August 2019, Leaderware , CCTV, Surveillance & Remote Monitoring, Security Services & Risk Management
The rehearsal of concept (ROC) drill has been used extensively in the military, but is just as important when facing risks in organisations and institutions to protect people on the ground.

Milestone partners prove their skills
August 2019, Milestone Systems , News, CCTV, Surveillance & Remote Monitoring, Training & Education
Within the span of one week in mid-May, the Milestone Learning & Performance group celebrated important benchmarks: 200 000 course registrations and tutorial views, and 10 000 certifications.

ONVIF Hosts 20th Developers’ Plugfest
August 2019 , News, CCTV, Surveillance & Remote Monitoring
ONVIF, the global standardisation initiative for IP-based physical security products, hosted its twentieth ONVIF Developers’ Plugfest in early June in Tokyo.

Licence plate recognition camera
August 2019 , Products, CCTV, Surveillance & Remote Monitoring
VIVOTEK has introduced a new licence plate recognition camera featuring built-in licence plate recognition software and edge-computing capability.

Securing Savannah Hills Estate
August 2019, Rhyco Risk Projects , CCTV, Surveillance & Remote Monitoring, Perimeter Security, Alarms & Intruder Detection
Savannah Hills Estate in Midrand planned a security upgrade which was awarded to Rhyco Risk Projects. The project started in March 2019 and was completed in May 2019.

10 things to consider when shopping for a VMS
August 2019, Genetec , CCTV, Surveillance & Remote Monitoring, Integrated Solutions
Today’s video management systems (VMS) provide a wide range of tools and capabilities that help make security personnel more efficient by allowing them to focus on what really matters.

How far are we really at with artificial intelligence?
August 2019, Axis Communications SA , Editor's Choice, CCTV, Surveillance & Remote Monitoring, IT infrastructure, Residential Estate (Industry)
Justin Ludik unpacks exactly how far AI has come and what it potentially can do for society and more importantly, surveillance.

Enhancing security with AI
August 2019, Hikvision South Africa , CCTV, Surveillance & Remote Monitoring
Hikvision explains how residential estates can revolutionise their security strategies and outcomes with artificial intelligence.

More than simply a camera
August 2019, Forbatt SA, Secutel Technologies , CCTV, Surveillance & Remote Monitoring, Residential Estate (Industry)
With the human element being the weakest link in all security solutions, Hi-Tech Security Solutions looks at the pros and cons of using body-worn cameras in estates.

The importance of effective perimeter security
August 2019, Elf Rentals - Electronic Security Solutions, Stafix , Editor's Choice, CCTV, Surveillance & Remote Monitoring, Residential Estate (Industry)
Protecting the perimeter is critical for any residential estate; how does one go about making sure your perimeter is as secure as possible?