printer friendly version

Purpose-built storage

The increasing adoption of HD and megapixel cameras is placing new and challenging demands on video storage systems. The exponential growth in data storage requirements is forcing IT and video security vendors to develop new solutions to keep up with all that video data.

Some video surveillance applications may be well served by traditional IT storage architectures. In actual practice, however, digital video storage is unique and differs from traditional data in the following ways:

* Database size.

* Write-intensive applications.

* Intolerant of system latencies.

* Constant first-in, first-out (FIFO) of recorded footage with sustained bit-rate streaming.

Database size

Network administrators are experts at managing storage farms. However, most IT professionals are not accustomed to dealing with the massive capacities that video storage requires. For example, a 100-camera system with seven days of storage, recording 24 hours per day at 15 images per second, using MJPEG compression could require more than 30 terabytes (TB) of storage.

To put this figure into perspective, the entire amount of storage used at the Pelco by Schneider Electric Clovis, California facility – with 1500 PC users; software and databases required to sustain the needs of full-blown fabrication and assembly factories; inventory management and logistics systems to support five warehouses around the world; and storage capacity needed for sales, accounting, and CRM systems – is only 32 TB. In fact, most enterprise-class IT storage systems are well under 100 TB of total capacity.

Now consider installations that have hundreds or thousands of cameras, or installations that require 365 days of online video storage. Well-designed video storage systems must be capable of efficiently scaling from a few gigabytes to petabytes. What’s more, storage systems designed for video must offer the same degree of data protection and safeguards as enterprise-class IT storage systems, but at a fraction of the price.

Write-intensive operations

Surveillance cameras never cease streaming content – and when doing so, nearly 95% of the operations executed on a hard disk drive are write operations. However, most applications in an enterprise work on the principle of 'write once, read many', meaning a file is written once, and then accessed hundreds of times. For this reason, most storage systems are designed to support fast access times for read operations while compromising access time allocated for write operations. Consequently, the write intensive operations of video comprise a complete reversal of what most enterprise-class storage solutions are designed to do.

If the storage sub-system is not tuned for massive amounts of write operations, the cost of deploying a storage solution increases dramatically. To accommodate a lack of write operation throughput, network video recorder (NVR) servers need to buffer incoming streams. Not only do more servers increase capital expenditures, but they also consume additional power, cooling, network ports, and rack space that increase the ongoing operational budget of the system.

Intolerance of system latency

Video surveillance systems are designed to reduce end-to-end latency as much as possible. As such, components including the camera, recorder, and decoder have very little buffer space available. Conversely, buffer and retransmit schemes are used widely in enterprise storage systems to accommodate asynchronous requests from various servers. Therefore, while enterprise storage systems have very impressive throughput numbers, the ability to service an incoming request from a given server is not a deterministic activity. The asynchronous manner in which storage systems service incoming requests can wreak havoc on a video surveillance system.

Averaging is what makes this approach dangerous for video surveillance storage. Server number one’s allocation of 100 megabits per second (Mbps) can easily take the form of 1 gigabyte per second (GBps) for a second and nothing for the next 9 seconds (1000 Mbps/10 seconds = 100 Mbps on average). In this example, the storage controller is well within its specified operational parameters. But from a camera’s perspective, something must now buffer nine seconds worth of video or the result will be dropped video frames.

The server running the NVR software can typically buffer some of this video traffic, but as different cameras come into play and different storage topologies are introduced, the NVR server must either be significantly over-resourced (read higher cost) to accommodate the variation in load or have a strict limit as to the number and type of cameras that each server can support. As one of the elements of a well-designed storage solution is to reduce cost, requiring more servers to accommodate a given number of cameras is again in direct opposition to the effort to minimise total cost of ownership.

Constant FIFO of recorded footage with sustained bit-rate streaming

Most surveillance systems configure storage arrays based on set parameters such as camera bit-rate, recording schedule, number of cameras, and retention period needed. Once the system is commissioned, video is recorded and the hard drives begin to fill. If initial storage calculations were done correctly, the drives will begin to overwrite once the retention period is reached, deleting the oldest video files to make room for new ones – e.g. first-in, first out (FIFO).

When PC users delete information from hard drives, fragments are left behind, and so PC users must occasionally run a defragmentation utility to optimise their drive. Now imagine a video storage array whose steady state is a constant overwrite scenario. As fragmentation builds, it will eventually choke off the remaining capacity of the drives. Ultimately, if no additional storage capacity remains, the NVR will cease to write new files, in an effort to protect the data that is already on the drives.

While fragmentation cannot easily be avoided, its effects can be mitigated. One such way is to leverage a file system that is less prone to fragmentation and can recover faster. XFS is an example of a file system built for performance and equipped with tools to easily recover from fragmentation issues. As video systems are constantly overwriting data, having a robust file system is critical to a properly designed video storage solution.

Source: Pelco Press Summer 2013. Used with permission.

For more information contact Pelco, +27 (0)12 254 6400, malcom.govender@schneider-electric.com, www.pelco.com

Share this article:

Further reading: