Big data operational and management requirements

November 2019 Editor's Choice

Big data offers huge advantages in enhancing or ‘augmenting’ data, giving it added meaning and context, creating a broader perspective in which to view things, and adding to the value of data and enhancing the potential depth of evaluation. We are increasingly seeing the implementation of big data integration and synthesis to assist in decision making in a range of control room environments, and this is just the start of a process that is likely to be much more sophisticated in the future. Processes such as face recognition, number plate recognition, access and personal information, movement tracking, and various dimensions of video processing can all be combined to assist us to make decisions in the security area.

The foundation of big data is, by nature, shared and linked data. In many cases this may be live data from sources such as weather, traffic, number plate recognition, positioning or status from electronic tags, etc. In others, you may access short- to long-term stored data where delays of data transfer and processing can lead to a few seconds of latency in the process of the transmission and integration.

For example, scams using delays measured in seconds in live feeds for betting abuse have been found in a number of crime syndicate related activities. However, users are also likely to access systems that contain days, weeks or even years of recorded data. In all of these, there is a quality factor that should be evaluated very carefully. Risk implications include having inaccurate information, out of date information, faked information, or delayed updates that can have a major impact on the integrity of judgement or conclusions that arise from the data.

Data quality

Data is only as good as the capture process and ongoing management of the information in the database. One would consider the data inflow process, how consistently is data treated across the capture process, and the considerations regarding accuracy and possible deviations or inconsistencies. In addition, how timeous is the data or does it reflect conditions that may have already changed or become obsolete, how regularly is it updated and is there a way to establish the most recent update, and what happens with redundant data – is it cleaned, does it clog up the systems, when does it become redundant, and who has responsibility for this?

Obviously the design of the big data system interaction and integration would consider some of these factors, but the danger is in taking these kinds of conditions for granted and ending up with a domino effect where small errors can lead to major implications for valid decision making. Where parties responsible for data management of some of the systems are not efficient in their capture, management and maintenance it has implications for the validity of the data.

Before you start using big data sources, you need to have your own evaluations of the risk implications of using the various sources. This should include trustworthiness, confidence levels, and measures that are built in for verification. Further, it is not just the data itself. There may be various algorithms that have been set up to process the data which may have their own implications. The issue of algorithm bias and discrimination that may result from perspectives of those setting up the system came up as a prominent issue in a behaviour analysis conference in Minneapolis recently, with comments from an ACLU representative commenting about the potential of this.

Protecting yourself

Ultimately the use of big data is a social process and not just an IT one. Where data can get captured, criminals may be working actively to combat identification or labelling by systems such as number plate recognition, or face recognition. This may go to active strategies such as cloning number plates and even vehicles, not just hiding identifiers. Hacking and manipulation of such systems, or paying to have data compromised or changed need to be taking into consideration.

Users should conduct a thorough risk analysis of the systems they may be drawing from, and have procedures in place to ensure verification, legal accountability, and be in a position where they can evaluate the cost of working decisions or outcomes on the business or organisation. Use of powerful tools needs to be balanced with appropriate caution.


Craig Donald.

Relying on AI for decision making should also be part of any risk evaluation. Thinking and verifying before one acts on data should ultimately be a human decision which incorporates verification and situational awareness. I continually am surprised by the amount of social and psychological consideration that needs to go into security systems that are supposedly ‘intelligent’ and can think and act on their own. How much do you trust the people behind the data?

Craig Donald

Dr Craig Donald is a human factors specialist in security and CCTV. He is a director of Leaderware which provides instruments for the selection of CCTV operators, X-ray screeners and other security personnel in major operations around the world. He also runs CCTV Surveillance Skills and Body Language, and Advanced Surveillance Body Language courses for CCTV operators, supervisors and managers internationally, and consults on CCTV management. He can be contacted on +27 11 787 7811 or [email protected]


Credit(s)




Share this article:
Share via emailShare via LinkedInPrint this page



Further reading:

AI-enabled tools reducing time to value and enhancing application security
Editor's Choice
Next-generation AI tools are adding new layers of intelligent testing, audit, security, and assurance to the application development lifecycle, reducing risk, and improving time to value while augmenting the overall security posture.

Read more...
Perspectives on personal care monitoring and smart surveillance
Leaderware Editor's Choice Surveillance Smart Home Automation IoT & Automation
Dr Craig Donald believes smart surveillance offers a range of options for monitoring loved ones, but making the right choice is not always as simple as selecting the latest technology.

Read more...
AI enables security solutions to define business strategies
Regal Distributors SA Editor's Choice
While allowing technologies to do exactly what they should do with even more efficiency and precision, AI is also empowering these same technologies to break through their traditional boundaries and create an ecosystem where one interface delivers outcomes across highly segmented verticals.

Read more...
Putting cyber into surveillance
Dallmeier Electronic Southern Africa Cathexis Technologies Technews Publishing Editor's Choice
Cybersecurity has become an essential part of the physical security industry. However, unlike other IoT technologies, of which security products are a part, surveillance technologies have more to protect.

Read more...
2024 State of Security Report
Editor's Choice
Mobile IDs, MFA and sustainability emerge as top trends in HID Global’s 2024 State of Security Report, with artificial intelligence appearing in the conversation for the first time.

Read more...
Cyberthreats facing SMBs
Editor's Choice
Data and credential theft malware were the top two threats against SMBs in 2023, accounting for nearly 50% of all malware targeting this market segment. Ransomware is still the biggest threat.

Read more...
Are we our own worst enemy?
Editor's Choice
Sonja de Klerk believes the day-to-day issues we face can serve as opportunities for personal growth and empowerment, enabling us to contribute to creating a better and safer environment for ourselves and South Africa.

Read more...
How to spot a cyberattack if you are not a security pro
Editor's Choice
Cybersecurity awareness is straightforward if you know what to look for; vigilance and knowledge are our most potent weapons and the good news is that anyone can grasp the basics and spot suspicious activities.

Read more...
Protecting IP and secret data in the age of AI
Editor's Choice
The promise of artificial intelligence (AI) is a source of near-continuous hype for South Africans. However, for enterprises implementing AI solutions, there are some important considerations regarding their intellectual property (IP) and secret data.

Read more...
Super election year increases risks of political violence
Editor's Choice
Widening polarisation is expected in many elections, with terrorism, civil unrest, and environmental activism risks intensifying in a volatile geopolitical environment. Multinational businesses show an increasing interest in political violence insurance coverage in mitigation.

Read more...