Storage: what is the right choice?

May 2016 CCTV, Surveillance & Remote Monitoring, IT infrastructure

Everyone talks about it, everyone knows it: The requirements for video surveillance are continually on the rise. Evermore, cameras, higher resolutions and longer retention times are increasing the demand for storage systems.

When you need a new storage system, it is important to first be clear about your specific requirements. Put another way: You must define your needs in terms of reliability,

capacity, cost and performance. Of course these requirements are interdependent, making the decision that much more difficult. This article provides an overview of common storage systems and their pros and cons to help you make the right decision.

Reliability = Availability

Reliability is a great virtue; for video systems as well. In this case, it can even be defined: Reliability means availability. How much you need depends on the application. The following question may be able to help you assess your needs: How critical is it if individual sequences of images are missing in the recording? What would happen if one in ten incidents were not documented? Or one in a hundred?

Of course, everyone would prefer absolute reliability and thus 100% availability. Technically speaking, this level is almost impossible to realise. And of course, more security costs more money. The level of availability is therefore always a compromise between requirements and the available budget.

But even here the situation is not black or white – the possibilities are vast.

The basis: RAID

RAID stands for Redundant Array of Independent Disks. It is a method of storing data on multiple individual disks that are combined into a single logical unit. A RAID controller controls the write and read operations, so that the entire system achieves a higher level of reliability or a higher data throughput than the individual physical storage medium itself. RAID is therefore more than just data backup. When one hard disk fails, system functionality remains intact (exception: RAID 0). After replacing the failed disk, the original state of the storage system is restored by a rebuild.

Depending on the objective, different

RAID configurations, also called levels, are used. In all, there are about a dozen different combinations of configurations. Four types used most often are:

RAID 0 or striping

• High data throughput: Data is distributed across multiple disks with the aim to achieve a high data transfer rate. The bottleneck in storage access is the mechanical limitation of the hard disk (the speed of read/write unit). By controlling multiple disks in parallel, it is like their speeds are “added together”.

• No data security: On the contrary, because when one disk fails, not only is all the data on that disk lost, the data on the other disks are at risk as well due to the fact that the information is distributed across them. If one disk fails, all data records are affected whose fragments were stored on this disk – in most cases this means the entire array of disks. Recovering the data records that were not affected is only possible using time consuming methods.

• Not a realistic option for video security:

Due to the lack of reliability, RAID 0 is not recommended for use in video surveillance systems.

RAID 1 or mirroring

• Duplication of data: All data is mirrored, that is, it is stored twice.

• Maximum reliability: This method provides a very high level of reliability as well as high performance.

• High costs: Doubling the number of hard drives is usually quite expensive.

• The net capacity is cut in half: Only half of the total capacity of all physically present hard drives is available for storing data.

• Special area of use: The method is particularly useful when the entire dataset must be removed for external use without interrupting system operation, for example for use as evidence.

RAID 5 and RAID 6 – the standards

• Reliability: User data and parity data (generated during data storage) are stored across all disks. Using the parity data, all data can be completely recovered, even if one disk fails (RAID 5). In a RAID 6 configuration, it is even possible for two hard drives to fail simultaneously, as in this case the parity data has itself been backed up by a duplicate.

• Continuous data access: Once a defective disk is replaced, the system automatically starts with the rebuild (recovery process) of the parity data so that once this procedure is complete the original level of reliability is again provided. During this period, it is still possible to access all user data.

• Limited costs: The net usable capacity for

storing the user data depends on the selected RAID level: In RAID 5 it is n-1, for RAID 6 it is n-2, where n is the total number of disks. In a RAID system with a total of 16 hard drives, 2 disks are used to ensure RAID Level 6, leaving 14 hard drives to store the user data.

• Balance: Users for whom reliability, data throughput and cost efficiency are equally important often use RAID 5 or RAID 6.

Since the performance of RAID systems depend on the proper interaction of hard disks, it is advisable to only use identical hard disk drives. Identical means: Same manufacturer, same capacity, same type number and, if possible, the same firmware version. Moreover, in RAID systems we exclusively use so-called “enterprise'' HDDs that are certified for this use. They usually are characterised by a longer service life and reliability than desktop drives.

Which RAID level should be used for video security?

Although RAID is a complex topic, this question can be answered simply: The trend is moving toward RAID level 6; in particular as hard drives continue to increase in capacity.

With double parity, RAID 6 provides protection against the simultaneous failure of two drives. This means that even when the system is performing a rebuild, the RAID array is still protected against the failure of another hard drive. The intense read and write operations during the rebuild place increased demands on the drives. In addition, hard drive sizes continue to grow unabated, meaning the recovery process takes longer as well. The risk that a drive fails during a rebuild thus increases with the hard drive capacity. RAID 6 ensures the reliability of the system even during rebuilds – and at the price of only one additional disk!

An alternative is RAID 5 with an integrated 'hot spare' disk: A reserve disk that is incorporated in the system but remains inoperative until one active disk fails. Only then is it activated by the RAID controller and the rebuild process starts immediately. The advantage: The 'hot spare' disk hardly ages for as long as it is not in operation. The disadvantage: During the rebuild there is no security against failure of an additional disk. However, the rebuild time is reduced. This makes sense if a system cannot be easily accessed by the system administrator, and as a result it can take some time until a defective disk can be replaced. In principle, 'hot spare' can also be integrated into a RAID 6 system.

Be sure to enable Controller Messaging. Ensure that your administrator takes the message seriously and keeps spare disks ready on site. The administrator can thus react quickly and minimise the risk of data loss.

Central or local

It is and remains a matter of priority: Do you need maximum storage capacity of hard disks, optimum data throughput or maximum reliability? Not just the RAID level within a system counts, the selection of the storage concept and the connection with the data-generating devices also have their influence. In general, a distinction is made between network attached (central) and directly attached (local) storage. These are the differences:

iSCSI Storage – central storage over longer distances

iSCSI (Internet Small Computer System Interface) is a standard for storage networks based on IP technology, and it is used for centralised management of storage within a LAN. It is used in particular when multiple devices generate data that are stored on a shared storage system. It can be used to save space, for improved administration or to combine the (individually small) storage requirements of multiple units into a larger array. Assuming RAID 6 is used by default, the following variants exist:

For maximum storage capacity, all available disks (n) are combined to form a large storage system, a 'volume group'. The net capacity (RAID 6 = n-2) is then divided into individual partitions (Virtual Disks). Each partition contains a corresponding portion of the available storage capacity. The data generating computers (in our application, DVRs or NVRs, i.e. recorders) now use logical, virtual locations instead of local storage media. A potential bottleneck is the centralised storage management, which receives and distributes the data from all recorders – at a limited data throughput. The advantage, however, is that the RAID 6 reliability costs only two hard disks for the total capacity of all recorders. (See Figure 1.)

Figure 1. All recorders share a volume group with RAID 6. The maximum data throughput is shared between the devices. The access speed for each recorder is limited. © Geutebrück
Figure 1. All recorders share a volume group with RAID 6. The maximum data throughput is shared between the devices. The access speed for each recorder is limited. © Geutebrück

For even more data security, multiple volume groups within a RAID system are each configured with a RAID 6 system. Each volume group is a separate partition (Virtual Disk). Each recorder uses a separate volume group for its data. Each volume group is controlled separately. Benefits are even greater reliability and more efficient management by the RAID controller with a correspondingly higher data throughput per recorder. However, this concept costs significantly more hard drives – namely exactly twice the number of volume groups – that are not available for user data. (See Figure 2.)

Figure 2. Management of the RAID system is simplified. The performance of the individual units increases. Writing, reading and access are faster. When a hard drive fails within a volume group, only one recorder is affected. © Geutebrück
Figure 2. Management of the RAID system is simplified. The performance of the individual units increases. Writing, reading and access are faster. When a hard drive fails within a volume group, only one recorder is affected. © Geutebrück

SAS storage – the direct link

SAS (Serial Attached SCSI) is a storage system directly attached to the computer (also known as: Direct Attached Storage (DAS) or Server Attached Storage). There is a dedicated, direct cable connection between the computer and storage unit via a SCSI cable. The distance between the two is limited to a few metres.

Figure 3. A G-Scope/8000 with integrated RAID system can be expanded using JBODs to a database size of 256 TB. © Geutebrück
Figure 3. A G-Scope/8000 with integrated RAID system can be expanded using JBODs to a database size of 256 TB. © Geutebrück

The overall performance of the RAID system is 100% available for the connected recorder, but only for this recorder. Each additional recorder needs its own storage system. The advantage is very high data throughput and therefore optimum disk access. This approach is recommended if the storage requirements for an individual recorder are high, for example when the images from a large number of high-resolution cameras need to be stored over a long period of time.

Use recorders with an integrated RAID system such as the G-Scope/8000 and expand the storage up to the capacity you need using SAS-connected JBODs (Just a Bunch Of Disks). They are cost-effective expansion units, which are controlled by the RAID controller of the recorder.

Summary

A storage system is used to improve performance in at least one major aspect compared to operation of one or more individual hard drives. Depending on the configuration, a RAID system offers the following advantages:

• Improved reliability of data storage.

• Guaranteed high availability.

• Improved data throughput rate.

For more information contact Geutebrück, +27(0)11 867 6585, bernard@geutebruck.co.za, www.geutebrueck.com




Share this article:
Share via emailShare via LinkedInPrint this page



Further reading:

Why rehearsals and drills for crises and evacuation are so important
August 2019, Leaderware , CCTV, Surveillance & Remote Monitoring, Security Services & Risk Management
The rehearsal of concept (ROC) drill has been used extensively in the military, but is just as important when facing risks in organisations and institutions to protect people on the ground.

Read more...
Milestone partners prove their skills
August 2019, Milestone Systems , News, CCTV, Surveillance & Remote Monitoring, Training & Education
Within the span of one week in mid-May, the Milestone Learning & Performance group celebrated important benchmarks: 200 000 course registrations and tutorial views, and 10 000 certifications.

Read more...
ONVIF Hosts 20th Developers’ Plugfest
August 2019 , News, CCTV, Surveillance & Remote Monitoring
ONVIF, the global standardisation initiative for IP-based physical security products, hosted its twentieth ONVIF Developers’ Plugfest in early June in Tokyo.

Read more...
Licence plate recognition camera
August 2019 , Products, CCTV, Surveillance & Remote Monitoring
VIVOTEK has introduced a new licence plate recognition camera featuring built-in licence plate recognition software and edge-computing capability.

Read more...
Double dose of storage security
August 2019 , Products, IT infrastructure
The integration of InfiniBox and SafeNet KeySecure platforms brings data-at-rest encryption together with centralised logging, auditing capability.

Read more...
Securing Savannah Hills Estate
August 2019, Rhyco Risk Projects , CCTV, Surveillance & Remote Monitoring, Perimeter Security, Alarms & Intruder Detection
Savannah Hills Estate in Midrand planned a security upgrade which was awarded to Rhyco Risk Projects. The project started in March 2019 and was completed in May 2019.

Read more...
10 things to consider when shopping for a VMS
August 2019, Genetec , CCTV, Surveillance & Remote Monitoring, Integrated Solutions
Today’s video management systems (VMS) provide a wide range of tools and capabilities that help make security personnel more efficient by allowing them to focus on what really matters.

Read more...
How far are we really at with artificial intelligence?
August 2019, Axis Communications SA , Editor's Choice, CCTV, Surveillance & Remote Monitoring, IT infrastructure, Residential Estate (Industry)
Justin Ludik unpacks exactly how far AI has come and what it potentially can do for society and more importantly, surveillance.

Read more...
Enhancing security with AI
August 2019, Hikvision South Africa , CCTV, Surveillance & Remote Monitoring
Hikvision explains how residential estates can revolutionise their security strategies and outcomes with artificial intelligence.

Read more...
More than simply a camera
August 2019, Forbatt SA, Secutel Technologies , CCTV, Surveillance & Remote Monitoring, Residential Estate (Industry)
With the human element being the weakest link in all security solutions, Hi-Tech Security Solutions looks at the pros and cons of using body-worn cameras in estates.

Read more...