Storage: what is the right choice?

CCTV Handbook 2016 Surveillance, Infrastructure

Everyone talks about it, everyone knows it: The requirements for video surveillance are continually on the rise. Evermore, cameras, higher resolutions and longer retention times are increasing the demand for storage systems.

When you need a new storage system, it is important to first be clear about your specific requirements. Put another way: You must define your needs in terms of reliability,

capacity, cost and performance. Of course these requirements are interdependent, making the decision that much more difficult. This article provides an overview of common storage systems and their pros and cons to help you make the right decision.

Reliability = Availability

Reliability is a great virtue; for video systems as well. In this case, it can even be defined: Reliability means availability. How much you need depends on the application. The following question may be able to help you assess your needs: How critical is it if individual sequences of images are missing in the recording? What would happen if one in ten incidents were not documented? Or one in a hundred?

Of course, everyone would prefer absolute reliability and thus 100% availability. Technically speaking, this level is almost impossible to realise. And of course, more security costs more money. The level of availability is therefore always a compromise between requirements and the available budget.

But even here the situation is not black or white – the possibilities are vast.

The basis: RAID

RAID stands for Redundant Array of Independent Disks. It is a method of storing data on multiple individual disks that are combined into a single logical unit. A RAID controller controls the write and read operations, so that the entire system achieves a higher level of reliability or a higher data throughput than the individual physical storage medium itself. RAID is therefore more than just data backup. When one hard disk fails, system functionality remains intact (exception: RAID 0). After replacing the failed disk, the original state of the storage system is restored by a rebuild.

Depending on the objective, different

RAID configurations, also called levels, are used. In all, there are about a dozen different combinations of configurations. Four types used most often are:

RAID 0 or striping

• High data throughput: Data is distributed across multiple disks with the aim to achieve a high data transfer rate. The bottleneck in storage access is the mechanical limitation of the hard disk (the speed of read/write unit). By controlling multiple disks in parallel, it is like their speeds are “added together”.

• No data security: On the contrary, because when one disk fails, not only is all the data on that disk lost, the data on the other disks are at risk as well due to the fact that the information is distributed across them. If one disk fails, all data records are affected whose fragments were stored on this disk – in most cases this means the entire array of disks. Recovering the data records that were not affected is only possible using time consuming methods.

• Not a realistic option for video security:

Due to the lack of reliability, RAID 0 is not recommended for use in video surveillance systems.

RAID 1 or mirroring

• Duplication of data: All data is mirrored, that is, it is stored twice.

• Maximum reliability: This method provides a very high level of reliability as well as high performance.

• High costs: Doubling the number of hard drives is usually quite expensive.

• The net capacity is cut in half: Only half of the total capacity of all physically present hard drives is available for storing data.

• Special area of use: The method is particularly useful when the entire dataset must be removed for external use without interrupting system operation, for example for use as evidence.

RAID 5 and RAID 6 – the standards

• Reliability: User data and parity data (generated during data storage) are stored across all disks. Using the parity data, all data can be completely recovered, even if one disk fails (RAID 5). In a RAID 6 configuration, it is even possible for two hard drives to fail simultaneously, as in this case the parity data has itself been backed up by a duplicate.

• Continuous data access: Once a defective disk is replaced, the system automatically starts with the rebuild (recovery process) of the parity data so that once this procedure is complete the original level of reliability is again provided. During this period, it is still possible to access all user data.

• Limited costs: The net usable capacity for

storing the user data depends on the selected RAID level: In RAID 5 it is n-1, for RAID 6 it is n-2, where n is the total number of disks. In a RAID system with a total of 16 hard drives, 2 disks are used to ensure RAID Level 6, leaving 14 hard drives to store the user data.

• Balance: Users for whom reliability, data throughput and cost efficiency are equally important often use RAID 5 or RAID 6.

Since the performance of RAID systems depend on the proper interaction of hard disks, it is advisable to only use identical hard disk drives. Identical means: Same manufacturer, same capacity, same type number and, if possible, the same firmware version. Moreover, in RAID systems we exclusively use so-called “enterprise'' HDDs that are certified for this use. They usually are characterised by a longer service life and reliability than desktop drives.

Which RAID level should be used for video security?

Although RAID is a complex topic, this question can be answered simply: The trend is moving toward RAID level 6; in particular as hard drives continue to increase in capacity.

With double parity, RAID 6 provides protection against the simultaneous failure of two drives. This means that even when the system is performing a rebuild, the RAID array is still protected against the failure of another hard drive. The intense read and write operations during the rebuild place increased demands on the drives. In addition, hard drive sizes continue to grow unabated, meaning the recovery process takes longer as well. The risk that a drive fails during a rebuild thus increases with the hard drive capacity. RAID 6 ensures the reliability of the system even during rebuilds – and at the price of only one additional disk!

An alternative is RAID 5 with an integrated 'hot spare' disk: A reserve disk that is incorporated in the system but remains inoperative until one active disk fails. Only then is it activated by the RAID controller and the rebuild process starts immediately. The advantage: The 'hot spare' disk hardly ages for as long as it is not in operation. The disadvantage: During the rebuild there is no security against failure of an additional disk. However, the rebuild time is reduced. This makes sense if a system cannot be easily accessed by the system administrator, and as a result it can take some time until a defective disk can be replaced. In principle, 'hot spare' can also be integrated into a RAID 6 system.

Be sure to enable Controller Messaging. Ensure that your administrator takes the message seriously and keeps spare disks ready on site. The administrator can thus react quickly and minimise the risk of data loss.

Central or local

It is and remains a matter of priority: Do you need maximum storage capacity of hard disks, optimum data throughput or maximum reliability? Not just the RAID level within a system counts, the selection of the storage concept and the connection with the data-generating devices also have their influence. In general, a distinction is made between network attached (central) and directly attached (local) storage. These are the differences:

iSCSI Storage – central storage over longer distances

iSCSI (Internet Small Computer System Interface) is a standard for storage networks based on IP technology, and it is used for centralised management of storage within a LAN. It is used in particular when multiple devices generate data that are stored on a shared storage system. It can be used to save space, for improved administration or to combine the (individually small) storage requirements of multiple units into a larger array. Assuming RAID 6 is used by default, the following variants exist:

For maximum storage capacity, all available disks (n) are combined to form a large storage system, a 'volume group'. The net capacity (RAID 6 = n-2) is then divided into individual partitions (Virtual Disks). Each partition contains a corresponding portion of the available storage capacity. The data generating computers (in our application, DVRs or NVRs, i.e. recorders) now use logical, virtual locations instead of local storage media. A potential bottleneck is the centralised storage management, which receives and distributes the data from all recorders – at a limited data throughput. The advantage, however, is that the RAID 6 reliability costs only two hard disks for the total capacity of all recorders. (See Figure 1.)

Figure 1. All recorders share a volume group with RAID 6. The maximum data throughput is shared between the devices. The access speed for each recorder is limited. © Geutebrück
Figure 1. All recorders share a volume group with RAID 6. The maximum data throughput is shared between the devices. The access speed for each recorder is limited. © Geutebrück

For even more data security, multiple volume groups within a RAID system are each configured with a RAID 6 system. Each volume group is a separate partition (Virtual Disk). Each recorder uses a separate volume group for its data. Each volume group is controlled separately. Benefits are even greater reliability and more efficient management by the RAID controller with a correspondingly higher data throughput per recorder. However, this concept costs significantly more hard drives – namely exactly twice the number of volume groups – that are not available for user data. (See Figure 2.)

Figure 2. Management of the RAID system is simplified. The performance of the individual units increases. Writing, reading and access are faster. When a hard drive fails within a volume group, only one recorder is affected. © Geutebrück
Figure 2. Management of the RAID system is simplified. The performance of the individual units increases. Writing, reading and access are faster. When a hard drive fails within a volume group, only one recorder is affected. © Geutebrück

SAS storage – the direct link

SAS (Serial Attached SCSI) is a storage system directly attached to the computer (also known as: Direct Attached Storage (DAS) or Server Attached Storage). There is a dedicated, direct cable connection between the computer and storage unit via a SCSI cable. The distance between the two is limited to a few metres.

Figure 3. A G-Scope/8000 with integrated RAID system can be expanded using JBODs to a database size of 256 TB. © Geutebrück
Figure 3. A G-Scope/8000 with integrated RAID system can be expanded using JBODs to a database size of 256 TB. © Geutebrück

The overall performance of the RAID system is 100% available for the connected recorder, but only for this recorder. Each additional recorder needs its own storage system. The advantage is very high data throughput and therefore optimum disk access. This approach is recommended if the storage requirements for an individual recorder are high, for example when the images from a large number of high-resolution cameras need to be stored over a long period of time.

Use recorders with an integrated RAID system such as the G-Scope/8000 and expand the storage up to the capacity you need using SAS-connected JBODs (Just a Bunch Of Disks). They are cost-effective expansion units, which are controlled by the RAID controller of the recorder.

Summary

A storage system is used to improve performance in at least one major aspect compared to operation of one or more individual hard drives. Depending on the configuration, a RAID system offers the following advantages:

• Improved reliability of data storage.

• Guaranteed high availability.

• Improved data throughput rate.

For more information contact Geutebrück, +27(0)11 867 6585, bernard@geutebruck.co.za, www.geutebrueck.com





Share this article:
Share via emailShare via LinkedInPrint this page



Further reading:

Pentagon appointed as Milestone distributor
Elvey Security Technologies News & Events Surveillance
Milestone Systems appointed Pentagon Distribution (an Elvey Group company within the Hudaco Group of Companies) as a distributor. XProtect’s open architecture means no lock-in and the ability to customise the connected video solution that will accomplish the job.

Read more...
Embracing next-generation surveillance for safer cities
Surveillance Integrated Solutions AI & Data Analytics
With the South African government highlighting the importance of building smart cities by integrating advanced technologies to make them more resilient and liveable, the role of next-generation network video and surveillance technologies cannot be ignored.

Read more...
Hik-Connect for Teams released
Hikvision South Africa Products & Solutions Surveillance
Hikvision has released an update for its security management platform; Hik-Connect for Teams not only meets the specific needs of individual users, but also caters to various medium-sized businesses (SMBs) and multi-site management scenarios.

Read more...
The human factor side of video management systems
Leaderware Editor's Choice Surveillance Risk Management & Resilience
A video management system (VMS) is central to, and the most vital element to any control room operation using CCTV as part of its service delivery, however, all too often, it is seen as a technical solution rather than an operational solution.

Read more...
A simple system for complex protection
Surveillance AI & Data Analytics IoT & Automation
Reconeyez is a company that provides autonomous wireless visual verification systems with built-in artificial intelligence. Running on batteries, each device includes communication capabilities, creating a mesh network between the various devices, linking them to the control room.

Read more...
Revolutionise your security with integrated solutions
Secutel Technologies Products & Solutions Surveillance
By merging the capabilities of SecuVue and Secutraq, Secutel has forged a robust system empowering users to fine-tune analytics and harness facial recognition, pose detection, and XFS, among other features.

Read more...
All aspects of data protection
Technews Publishing Editor's Choice Information Security Infrastructure AI & Data Analytics
SMART Security Solutions spoke to Kate Mollett, Senior Director, Commvault Africa, about the company and its evolution from a backup specialist to a full data protection specialist, as well as the latest announcements from the company.

Read more...
Safer roads to a sustainable future
Dahua Technology South Africa Surveillance
With the ever-increasing number of vehicles on the road, it is imperative to constantly improve the quality of transportation monitoring and traffic management in urban and rural areas.

Read more...
From damage management to false alarm prevention
Dallmeier Electronic Southern Africa Transport (Industry) Surveillance Logistics (Industry)
Video surveillance is undergoing an exciting evolution; today’s IP-based solutions can be fully integrated into digital systems. With the increasing integration of artificial intelligence (AI); logistics, in particular, can make processes more transparent and exploit previously untapped potential.

Read more...
Majority of South African companies concerned about cloud security
Information Security Infrastructure
Global and local businesses share a common concern when it comes to cloud security. 95% of global businesses and 89% of local businesses are concerned about the security of public clouds.

Read more...