See evil, hear evil, raise the alarm
April 2018, CCTV, Surveillance & Remote Monitoring, Integrated Solutions

Surveillance is not only about video anymore. More cameras today are including audio capabilities to allow operators to hear what’s going on and to offer a voice deterrent as a first line of defence.

In many cases, when the subjects of your surveillance hear someone talking to them, they realise they are being watched and will move along. While this isn’t always the case, audio cuts down on the number of times response personnel need to be dispatched, further lowering the waste of resources for no reason.

For those who use Android or iOS smartphones, we know that voice technology has advanced much further than simply using a camera as an intercom of sorts. While voice recognition may not be a function in the physical security world (it is, however, becoming more popular as a biometric authentication mechanism), allowing systems to ‘hear’ what is happening in their surroundings adds another layer to your physical security posture.

In a prison, for example, audio analytics would be able to pick up aggressive interactions before they turn violent, potentially allowing the guards to intervene before anyone gets hurt. Someone screaming or calling for help in a subway, even if not on camera, can create an alert in a black-screen control room and PTZ cameras or staff can be guided to the source of the sound. In Cape Town we have seen the successful rollout of gunshot recognition in certain violent areas that enables the police to accurately triangulate the source of the shot and be on the scene in minutes.

So, while the market is focused on video images and analytics, as well as artificial intelligence and deep learning, audio and audio analytics is quietly creeping into the surveillance control room as a key aspect of a security surveillance operation. In the public security space, audio analytics is also a good way to make authorities aware of a situation immediately. Instead of waiting to be called and told there is gunfire at a location, surveillance cameras with audio can pick up the sound and alert operators within seconds.

At home, audio analytics can also have as big an impact. We already have some home security kits that can be used to switch on lights and so forth, but what about listening for sounds as well through your camera’s microphone. Breaking glass could raise an alarm, alerting the owners to the fact that a sound designated as abnormal has occurred. Using the homes surveillance cameras, the owner can view the home from their smartphone and see if it’s someone coming in the window or if the kids have kicked the soccer ball through the window. Whatever the cause, the owner can then dispatch their armed response company or call the kids (or speak to them through the camera’s speaker) and give them some choice vocabulary.

Challenges remain

It’s not that simple, however. When talking to your assistant, you start by saying a key word the system is trained to recognise – such as “OK Google”, or “Alexa”. Adding audio into an uncontrolled environment where there are many sounds and voices and noises all happening at once is not that simple. The system needs to be able to sort through all the sounds and pick up those that are deemed relevant – a scream for help or breaking glass etc.

Fortunately, just as many video surveillance companies are adopting AI and ‘training’ their systems to recognise faces, for example, a similar principle applies to audio analytics. By taking huge data sets of sound – a shopping mall, for example – companies are able to train their systems to distinguish noise from relevant sounds. As the systems learn, their algorithms are designed to compare new sounds with what they have already been trained on in order to recognise real alerts. Naturally, the acoustics of the environment also play a role in the success or failure of audio analytics.

Therefore, while audio has been around for a long time, just like video analytics, users should temper their expectations of what this technology can do. Research has advanced tremendously over the years, but we are still not at the point where audio delivers the perfect science-fiction performance. Using audio as an integrated solution is the best solution right now to avoid large numbers of false alarms as well as the problem of missing real events. Here, the idea of visual verification plays a role, alerting the operator to a possible security event and allowing them to check it out over video before taking action.

The privacy bug

Another issue to consider when using audio in your security operation is that of privacy. Someone sitting at a coffee shop in a mall, or a worker on their lunch break may know they are under video surveillance, but not think much of it as they are not doing anything wrong. However, when they find out that the cameras are listening to their conversations while capturing video there may be a problem.

Video capture in public or at a workplace is generally acceptable because you are in an area where you have no expectation of privacy. When it comes to listening or recording people’s private conversations in those same locations, that is another matter. If the security system records the audio for later analysis there will be an even greater problem, especially with laws like PoPI around the corner.

Of course, you may argue that their conversation is not being recorded in the audio, but simply captured and analysed, but it’s best to obtain legal advice before you get into trouble. Most people won’t mind an additional layer of security if their audio is properly secured while it is held, but there are always the few…

Audio analytics has come a long way from simply being a way to hear sounds and hopefully detect something that is worth raising the alarm to. There are companies that offer audio surveillance without being part of a surveillance solution, offering alerts and warnings based on sound alone. Most will say you need a camera in support of the audio to visually verify the alert, which makes the current trend of including audio in IP surveillance cameras a big deal. Now we just need to be able to select and install the audio analytics solution of our choice on a camera or plug it into a VMS to enhance the security of our environment, whether a home, office or open public areas.