printer friendly version

Practical testing issues

I remember some time ago when helping to design a control room for one of the major SA cities, I insisted on building a simple mock-up using cardboard and existing furniture, which allowed us to measure important things like sitting positions, line of sight issues, desk heights for wheelchair access, monitor positioning and numbers etc.

The control room worked extremely well, although after looking at careful design principles such as console material and individually controlled workstation lighting levels, the impact was reduced by the console constructor using a high gloss surface varnish on the finished console desks – something we hadn’t anticipated in our mock-up testing. What shocked me as a human factors specialist at the time was that the installation company remarked that they had never done something like a mock-up simulation before.

There are probably many control rooms that would have benefited from a test of the design concepts beforehand. Since then, I’ve seen some where the whole purpose of the control room was being subverted by the control room design and the way camera views were being displayed.

Similarly, when viewing a demonstration of a low-light camera some years ago, I remember being stunned by the clarity despite the poor lighting. However, when I took the cellphone that was being observed by the camera off the table it was on, there was no view of the action of taking it. A few seconds later, there was just no view of a cellphone on the camera.

The camera had been using slow frame rates which allowed more light exposure but virtually no recording of any action. So disappointing that something that looked so good initially didn’t meet the test of practical action.

A third example was looking at an estate fence line which was protected by thermal cameras and video analytics, which created an alarm condition if anybody approached the fence from the outside. However, when reviewing this example, one could see a suspect looking for ways to get out of the estate and trying various efforts for 15 minutes without the control room being aware of it.

The importance of testing

Technology has improved significantly over the years, but the principle of testing technology application is just as important as ever. This is especially the case where AI or video analytics are represented in so much of the technology out there in sites where protection is seen as a critical factor. For users, the key issue is not its impressive sounding name, what they say it does, or even how it looks in demos or other sites, it is a case of how well it works for me. It is not only a case of testing out the technology, but also the feasibility of your ideas of what you want to do with the technology.

Yet there are few users who insist on an on-site demonstration using the same kind of conditions that would be required for live performance. The use of sophisticated software in combination with hardware technology with up-to-date enhancements can create a wonderful impression just through the marketing. The words used to label the technology may sound amazing and be incredibly seductive. However, the overriding concern for use should be that the solution is useable, practical and it delivers a workable result without too many distractions like false alarms.

Suited to purpose

From an end user point of view, the first question that one should be asking is whether the technology solution really applies to the concerns that the client has? In many cases, a cool-looking technology may really provide little deliverable value, or it may be amazing. If a product has multiple benefits, can you quantify the core benefits in terms of monetary value, rather than side value benefits?

Does it then fit into your current systems, or is it going to lead to running two or more systems in parallel, or even a major cost revamp to existing infrastructure? Ease of integration is a major concern. While we have to move on from legacy systems sometime, even more recent or proprietary technology may show incompatibilities.

When evaluating a technology for your security systems and control room, both the initial and follow up setting up as well as the ease-of-use of the end user experience are important. I have seen systems that are running and people working there have no idea how to modify it, or how to customise or enhance its use.

Of course, there are restrictions by equipment providers to avoid people from messing up their settings, but the capacity to initiate, modify and improve on things like virtual trip wires on perimeters and resolving of false alarms is something that should be readily available to clients. This is to ensure the solution provided can be used to its best effect. Therefore, a client needs to check how easy is it to set up and how easy is it to change the parameters or introduce new zones or areas. It has additional impact in that if there is a handover to a new appointee, the new person should also be able to pick up easily how to work with and adjust the system.

Does the technology detect or recognise what you want to and does it ignore things that shouldn’t trigger a response? For example, does it accurately pick up a human figure and ignore a dog or a buck in the same area? Given that many criminals hunch down or even crawl to access an area after hours, can these kinds of shapes still be recognised as human? Do practical tests to see if these can be detected.

On-site testing at different times

Testing at different times of the day may also be a part of this trial evaluation as shadows and different lighting may impact on effectiveness. If you have a number of commuters walking along your fence line and you still want detection capabilities, what kind of options are there to deal with your specific concerns? Can algorithms be tweaked to suit the client needs and tolerances, how easy is it to do this and does the system store defaults and a history of changes or versions?

Do you have local support for the systems and equipment and what are the lead times in parts availability? Delays that are going to mean your system is down for a few days pose a major potential risk on your site. Technical support from Europe or the US is easy to arrange nowadays, but its pricing is usually a lot higher.

If there are remote diagnostics and updates, who is going to be doing this and what protection do you have. Ask current users what they have experienced and see if there is a user forumsomewhere on social media where people are exchanging views on the systems you are interested in. This also enables an informal user support forum.

I believe strongly in getting a maintenance contract with the supplier when they install. It keeps the sales people more honest if the company is going to pick up the tab for what happens afterwards, makes sure it is suited to the environmental conditions it may face and I’ve seen too many examples of nearly new systems failing and suppliers walking away from the problem saying it is now an end user problem.

What are the proprietary requirements for the systems hardware and software to run and how does that compare with what you have at present? Ask what can’t the system do, as well as working out what it can do. What other systems can it interface with? Are there differences in requirements if upgrading to a higher capacity system? What it can’t do may be on your site development horizon and you don’t want to limit yourself.

Demonstrations are typically done by those marketing systems (or car fuel consumption) using the best conditions. How do the systems work in worst-case scenarios, and are there any examples of this? Check out comparative field trials and demonstrations of which there are some available in the country on different types of equipment. This gives you a comparison with technical functioning of equipment in the conditions you may be faced with.

Most importantly, do a review of how well your current management, supervisory and operating personnel can work with the interface and operational requirements for the new systems. If they can’t, what is needed for upgrading people, or alternatively what kind of new people are going to be required and what is the cost implication? The best system is no use if you haven’t got the capacity to operate it. Finally, does it tick all the legal checkboxes to ensure that you as the user won’t be liable for anything you weren’t expecting?

About Craig Donald

Dr Craig Donald.

Dr Craig Donald is a human factors specialist in security and CCTV. He is a director of Leaderware which provides instruments for the selection of CCTV operators, X-ray screeners and other security personnel in major operations around the world. He also runs CCTV Surveillance Skills and Body Language, and Advanced Surveillance Body Language courses for CCTV operators, supervisors and managers internationally, and consults on CCTV management. He can be contacted on +27 11 787 7811 or craig.donald@leaderware.com

Share this article:

Further reading: