printer friendly version

Exploration of panoramic surveillance

Securing public areas with video surveillance can be challenging. Specifically, blind spots cause problems when tracking people passing through the area. Whether it is an airport, a bank building or a retail outlet, there are lots of places where it is important to monitor people from the moment they enter the

building and follow them as they move around. All of their movements may be crucial in terms of monitoring behaviour or forensic search.

Even though today’s conventional IP cameras can deliver perfect images of what they view, they simply cannot see everything – there will always be blind spots. However, a panoramic camera can eliminate blind spots that would otherwise impact the effectiveness of video surveillance. But how do panoramic cameras work and in what kind of applications can they be beneficial?

Panoramic surveillance

Following the transition from analogue to IP video surveillance over recent years, multi-megapixel (MMP) cameras are now grabbing all the attention. While the dramatic increase in resolution benefits any surveillance application, it has also spurred development of high resolution panoramic cameras.

Panoramic cameras usually come in two different types: a 360° or a 180° field of view. The extraordinary 360° field of view of these cameras, coupled with exceptionally high resolution sensors of 12 megapixels and even beyond, results in detailed overview images in one single view. Usually installed in a ceiling, a 360° camera covers the entire area below the camera, providing a full surround view without any blind spots. A 180° panoramic camera provides full situational awareness when installed on a wall, e.g. in corridors or reception areas.

Panoramic cameras start to prove themselves as a critical component when building a video surveillance system that needs to meet the highest security standards as their ability to avoid blind spots enables operators to minimise the risk of missing important information. There are two panoramic camera designs: multi-lens panoramic cameras and single-lens panoramic cameras.

Multi-lens panoramic cameras generally have three or four lenses/sensors. The camera stitches together the separate video streams from each sensor to form a single image.

Single-lens panoramic cameras use an extremely wide angle lens of 180° or 360°.

A multi-lens camera provides four images from 90° lenses. The images are rectangular in shape and are not optically distorted. However, to create full situational awareness the images need to be stitched together using specific software.

A single-lens panoramic camera uses a fisheye lens that creates a hemispherical image of the full region located below the lens. These lenses form a circular field that ranges 180° horizontally and 180° vertically. Optical distortion correction technology is needed to create useful images, this is known as dewarping.

Single-lens panoramic cameras are generally available in two versions: 180° and 360°. The difference is in the size of the projected image circle. With a 360° camera, the sensor captures the full hemispheric region. The circular image fits the size of the sensor and has a 180° field of view horizontally and 180° vertically. With a 180° camera, the circular image fits within the vertical field of view of the sensor, but not within the horizontal field of view. The camera therefore has a 180° field of view horizontally, but less than 180° vertically.

This paper will continue with the focus on single-lens panoramic cameras.

Multi-lens panoramic camera versus single-lens panoramic camera.

Panoramic versus fixed cameras

Panoramic cameras are used to gain full situational awareness with a single camera. Fixed cameras are a great complement to camera systems when identification and details in a more restricted scene are needed on short and long distances – for instance, when monitoring a local grocery store or overseeing an entire queuing area at an airport. However, a panoramic camera can be used to provide a complete overview of a certain area.

Panoramic versus moving camera (PTZ)

A PTZ camera provides identification possibilities on short and long distances thanks to its optical zoom, which has a larger range than the panoramic camera. PTZ cameras can display and record only one part of the scene and are sometimes pointing in the wrong direction when an event occurs. This is where panoramic cameras can complement PTZ cameras by monitoring and recording all events and activities in the full area simultaneously.

This means that operators have full-area access to both live and recorded images. The panoramic camera monitors, analyses and continuously records everything. What’s more, panoramic cameras can effectively pan, tilt and zoom into a specific region without losing sight of other regions of interest. However, a panoramic camera can zoom in only digitally and is therefore not very suitable for identification as resolution drops due to loss of pixel density.

Image performance

The image performance or quality of a panoramic camera depends on the sensor and the image processing algorithms applied. When assessing image performance, users typically look at four areas: resolution, dynamic range, sensitivity and bitrate.

Resolution

Resolution is the number of pixels that can be displayed. If the number of pixels on the sensor is low, the image resolution will be low, and details will not be visible. Low resolution results in either pixilated or blurred images, especially in places where panoramic lens distortion needs to be heavily corrected and the correction algorithms need to expand images. A camera containing a sensor with more pixels will require significantly greater computing power.

The resolution of a panoramic camera is not as straightforward as conventional security cameras. A panoramic camera using a 12 megapixel (MP) sensor will not provide full 12 MP resolution. The optical circle of the lens is smaller than the active image area of the sensor and therefore does not cover all active pixels. The effective resolution of the image is the amount of pixels that will fit within the optical circle.

As an example, take a 12 MP sensor with 4000x3000 pixels and measuring 6.20x4.65 mm. The 360° optical circle of the lens has a diameter of 4.1 mm. This would result in square with a resolution of 7.0 MP. So when selecting a panoramic camera it is important to know whether it will really provide the advertised resolution or whether this is just the sensor’s resolution.

Dynamic range and sensitivity

Besides capturing details, the sensor is also responsible for the dynamic range and low light sensitivity of the camera. The usability of a panoramic image is highly dependent on the dynamic range of the sensor (i.e. the strength of its ability to capture detail in both bright and dark areas of the scene – the higher dynamic range the more details the sensor will capture in bright and dark areas). With a panoramic camera, there is a far greater likelihood of bright and dark areas in the field of view. A sensor with a low dynamic range will then not be able to show details in every area of the scene.

Bitrate

All IP networks have a limit to the traffic they can carry, this limit is known as bandwidth. Increasing resolutions means increasing amounts of data and this drives bitrates up. This calls for tools and technologies to help managing video data efficiency. Innovative data compression technologies such as H.265 reduce the amount of data provided by a high resolution camera such as a panoramic camera.

Intelligent Dynamic Noise Reduction (IDNR)

To reduce bitrates after encoding and further minimise strain on bandwidth and storage requirements, several forms of noise reduction can be applied. ‘Intelligent Dynamic Noise Reduction’ (IDNR) is an innovative technology that uses a combination of two noise reduction techniques: one that is best suited for scenes with motion (spatial noise reduction) and another that is best suited for still scenes (temporal noise reduction).

In this way, IDNR reduces bitrates and required storage capacity without affecting video quality. By combining these processes, bitrate increases when an event occurs and is minimised when there is no motion and thereby lowers the total required bitrate.

Encoder regions

Area-based encoding (encoder regions) can lower bitrates even further. This feature allows the image to be divided into zones of high importance and less compression to show more detail, and those of low importance and therefore high compression and less detail. For example, if the top of the image is always looking at the sky, this area can be marked for high compression. By selecting important, unimportant and normal regions in a scene, and adapting the compression ratios accordingly, a lower average bitrate can be achieved.

Dewarping for distortion-free viewing and recording

The fisheye lens of a panoramic camera produces optically distorted images that are circular in shape. Image correction technology is needed to create an optimised and useful image without distortion, this is known as dewarping. The dewarping algorithm remaps the pixels in the scene to optimise the image and remove distortion. The algorithm can offer various dewarped view mode options such as panorama, double panorama and surround.

Dewarping can in some cases provide real-time views across multiple streams. The various video viewing modes allow the user to control the 180° or 360° images and guarantee distortion-free video. It is also possible to define customised Regions of Interests (ROI) by adding presets. This way, dewarping can also help to make video data more manageable.

Edge versus client-side dewarping

Dewarping video images can be done either by dewarping algorithm on the camera before sending the video data to the Video Management System, this is known as edge dewarping, or on PC via a specific Video Software Development Kit (VSDK), this is called client-side dewarping.

Edge dewarping has some advantages over client-side dewarping:

• Edge dewarping makes the camera’s integration into a system easier. Most panoramic cameras require a specific VSDK for dewarping, which makes integration into VMS challenging. The VMS will need to support the VSDK, which can result in poor or no integration with third-party software. This can, in turn, limit the wide implementation and use of panoramic cameras. Edge dewarping allows the user to see a corrected image directly from the camera without the need of any special integration.

• Edge dewarping sends undistorted virtual camera images to the VMS. If the camera architecture allows it, the camera can create multiple virtual cameras, and edge dewarping will help send undistorted virtual camera streams to the VMS. Access rights to the virtual cameras can then be managed by the VMS and so providing different video streams to different users.

• Lower costs. Dewarping on the client-side requires a lot of computing power. Edge dewarping is done by the camera rather than using the computer’s CPU, which significantly reduces the processing power needed on a PC.

• Lower bitrates. Edge dewarping enables the user to monitor only those areas that are relevant for the user when there is no need to monitor the entire image circle. Relevant areas can be selected by the user, further lowering bitrates.

Two new panoramic cameras

In 2015 Bosch Security Systems launched its first panoramic cameras, the FLEXIDOME IP panoramic 7000 MP and the FLEXIDOME IP panoramic 5000 MP. They also offer the same intelligent data minimisation for efficient transmission and storage and are half the size of multi-lens panoramic cameras.

The FLEXIDOME IP panoramic 7000 MP uses a 12 MP sensor resulting in effectively 7 MP resolution at 30 fps The FLEXIDOME IP panoramic 5000 MP uses a 5 MP sensor at 15 fps. The 360° lens on the 5 MP sensor results in a 3.2 MP effective resolution. Both models include built-in Intelligent Dynamic Noise Reduction (IDNR), which reduces bitrates by up to 50% at the source. This significantly reduces storage costs and network strain without compromising video quality.

The FLEXIDOME IP panoramic 5000 MP includes Content Based Imaging Technology (CBIT), which ensures the highest quality of relevant images by tailoring the captured images to the content of the scene. The FLEXIDOME IP panoramic 7000 MP also features CBIT in which Intelligent Auto Exposure automatically adjusts the exposure settings of the camera when the lighting conditions change. The camera also comes with Bosch’s VCA, Intelligent Video Analysis (IVA), so you are alerted when needed and can trace back important events from hours of video in seconds. You can choose between a 180° lens with effectively 8 MP resolution or a 360° lens with effectively 7 MP resolution.

They both offer edge and client-side dewarping to correct image distortion.

For more information contact Bosch Security Systems - South Africa & sub-Saharan Africa, +27 (0)11 651 9600, charles.coetzee@za.bosch.com, africa.boschsecurity.com

[footer] This white paper has been shortened. The full article can be downloaded at africa.boschsecurity.com

Share this article:

Further reading: