printer friendly version

Video metadata tagging as a search management tool

About 18 months ago I was sitting in one of the oldest pubs in London discussing video and image meta tagging with one of the UK police representatives and a technology development specialist. It was memorable because of the contrast in the age and history of the pub versus the very modern technology we were talking about.

We were discussing the options around an image database that we developed called ImageIn, that facilitated incident descriptions and tagging and the discussion turned to automated video metadata tagging. The technology specialist claimed that within six months they would be able to provide an automated system to provide meta tagging of video including clothing, colours and other details such as number plates. I told him at the time I thought this was wildly optimistic and 18 months later we are still waiting for this automated system to occur.

Video metadata tagging has the potential to be one of the most useful concepts in CCTV, providing descriptive and search tags to allow one to search and go to a point in any video that matches the search criterion or criteria. This could potentially allow you to search for clothing descriptions, cars, colours, behaviours and a range of other criteria. The technology has already been developed to some extent for things like the searching of brand names for marketing purposes. The concept is a potentially huge application for video management and research is being sponsored internationally by various government agencies, major research organisations as well as commercial companies. However, the technology at this stage is largely inadequate for most practical purposes.

Analytics limited

Much of the automated metadata tagging concepts are based on video analytics. A number of companies also claim some capabilities in this respect through cameras or computer-based analysis. Sadly, despite claims of visual analytics of reportedly pinpointing threats and either reducing the operator need or input, most of these claims are limited at best. Video analytics are best at seeing obvious movement where there should not be, or in directions that are unusual or erratic. Recognition of subtle behavioural cues or detailed descriptions is usually difficult at the best of times for trained operators, and video analytic capabilities simply fall short in most subtle detection, never mind tagging these characteristics.

Another option is for an operator to input data into a field when material is seen in the video stream that can then be associated with that particular part of the video. Searching then allows the retrieval of the section of the video based on the input criteria. A more basic way of tagging is simply to classify video with associated data in a separate database. One would then go to the separate database and search the criteria, and then retrieve the relevant video. In my experience this is still relatively uncommon though.

The Google of video?

It is clear that there is a definite practical need for this. I find myself regularly searching through folders for particular video footage and I feel I am relatively organised in this respect. I regularly find that when viewing video at a client, one of the biggest time wasters is retrieving relevant video. One has to spend time going through a whole number of folders trying to find relevant video with obscure names that people can only remember vaguely.

Electronic occurrence books should provide some basic tagging opportunities and linking these to video material through file names. Most proprietary video systems display very few of these capabilities though, and it is at the system level where this kind of facility would be easiest to use. The added complication is that when exporting video to other formats such as avi files, the links are often lost.

Most image management systems, including consumer ones have the ability to tag photos or images with relevant key words. Something like Google Desktop provides an automated method of instantly searching your document or spreadsheet data through indexing all content. However, it looks like a while before one will be able to have any such systematic capacity for retrieving that obscure piece of video through a metadata-based system. Whether they will be compatible with other systems when they do emerge in strength remains to be seen. In the meantime, we are limited to largely manual input-based systems, although incident management databases do have a strong capacity to manage images and tags which can link to video already.

Dr Craig Donald is a human factors specialist in security and CCTV. He is a director of Leaderware which provides instruments for the selection of CCTV operators, X-ray screeners and other security personnel in major operations around the world. He also runs CCTV Surveillance Skills and Body Language, and Advanced Surveillance Body Language courses for CCTV operators, supervisors and managers internationally, and con-sults on CCTV management. He can be contacted on +27 (0)11 787 7811 or craig.donald@leaderware.com

Share this article:

Further reading: