A UNIFIED SMART CAMERA STANDARD: TECHNICAL- BUSINESS-ETHICAL

The contribution CCTV has made in protecting the public and assisting the police to investigate crime has occurred despite CCTV systems being developed in a piecemeal fashion with little strategic direction, control or regulation. This approach has failed to maximize the potential of urban CCTV infrastructures and many involved in its operation and management felt there remained a pressing need to examine existing standards, procedures, training and methods of operation. In addition, as local authorities, the police and criminal justice agencies face the challenges associated with the move from VHS technology to digitally recorded images, the lack of a coordinated approach to CCTV development poses significant risks in terms of compatibility of systems, cost of accessing the images and the potential loss of operational effectiveness.


Research has shown a vast range in the quality and usefulness of the business processes employed across all of those involved in CCTV. This includes standard working practices; standard job profiles; standard interfaces / protocols with other agencies; the use of standard performance indicators; evidence to support business cases, and indications of success. This was particularly noticeable in the differences between police forces, local government, and the interfaces between them. This has led to a fragmented approach across the country. Best
practices have not been built upon and there have been strained or less effective interfaces.


In my long discussions with business partners, integrators, end user, law enforcement organizations I have sensed the need of improvement in the technical aspects of CCTV deployment, the potential benefits and impact of digital recording systems, rapidly evolving technology, and the legal background to the operation of CCTV. There is also a constant need to observe data protection, privacy and human rights legislation. Recognizing data protection and privacy rights in the operation of all systems is not only important because the law requires us to do so, but also that it is right that we should do so, with concerns about surveillance so apparent in our everyday lives.


This is mainly what motivates my team at DigiSensory. The Technical, the Business and the Ethical values.

I had long meetings with my R&D group last week in relation to our latest Smart Agent products line, and our effort towards a unified standard. Let me tell you more about it, the Avista Smart Agent; is an XML- Metadata based intelligent video network protocol that standardizes AVISTA™ solution analytic rule, alert and configuration communications. Based on metadata-XML and open communication/web services standards, ASA enables seamless interoperability between any AVISTA™ smart camera device and any video management system to deliver a high level of intelligence and efficient operations. This protocol provides system integrators universal compatibility among any video management system with simple, feature rich API. Similarly, NVR manufacturers, can create analytic rule configuration and data output interfaces into their own software, and immediately communicate with and manage any number of AVISTA™ intelligent edge devices.

While most of the vendors today provide image processing based analytic DVRs and NVRs, it is clear that scalability is one of the main issues. It is impossible to drive more than 16 cameras without a powerful CPU to process images/video.Another issue is compromising the imaging quality by reducing the resolution to allow processing.

This week we have demonstrated at ASIAL 2008 in a life demo, over 100 cameras in a single IT infrastructure and network, delivering analitycs capability (tracking and counting) and an outstanding megapixel imaging quality.
We have also demonstrated that a smart CCTV system based on the ASA protocol and architecture is more than a security and surveillance tool, it is a business tool that drive performance.

As I said before data protection and privacy rights in the operation of all systems is not only important because the law requires us to do so, but also that it is right that we should do so, with concerns about surveillance so apparent in our everyday lives. Let me now talk about the technical link/the innovation and how we can maintain our integrity and ethical values.

Let me give you an example. There are 3 levels in our architecture:

The smart sensor layer
Smart cameras stream video and xml metadata. The xml metadata summarize the image "feature extraction" as specific entities like faces, motion, shapes, colors.

The smart agent layer
Analyze the metadata and communicate with different cameras. When a face is detected a filter is placed on the face in order to hide it. Once there is a specific threat the filter is removed. The smart Agent layer perform a number of services and build a high level of intelligence. And maintain integrity.

The application layer
Recieve video from cameras for view and storage and intelligence from ASA to build a high level applications.

The type of network we are discussing today will enable an extraodinary progress in the world wide web, what we know as "semantic web" where smart machines will drive web content.
But as I stated the main challenge will be: Technical, Business and Ethical for the future.

This is in general the passion driving our R&D team at DigiSensory Technologies. Congratulations for the ASA product launch.

A UNIFIED SMART CAMERA STANDARD: OUR ROLE AS A GLOBAL INNOVATOR. PART1

I have been asked this question many times: How can we effectively use analytics and smart sensors? How can we make them work?

Let me first share with you some good news, in this quarter we have just delivered one of the most advanced network of distributed smart cameras to the Australian Jockey Club as a first stage of a contract to deliver over 300 distributed smart cameras for autonomous Video Surveillance and Monitoring. The technical approach uses multiple, cooperative video smart sensors to provide continuous coverage of people and vehicles in a cluttered environment.

HOW DID WE GET THEIR?
A Homeland Security objective:
In the last 3 years we have developed an active video understanding technology using distributed XML metadata that enables a single human operator to monitor activities over a complex area using a distributed network of active smart cameras over a web based control and management platform. The goal is to automatically collect and disseminate real-time information ( XML metadata, image descriptors) from the battlefield to improve the situational awareness of commanders and staff. Other military and federal law enforcement applications include providing perimeter security for troops, monitoring peace treaties or refugee movements from unmanned airvehicles, providing security for embassies or airports, and staking out suspected drug or terrorist hide-outs by collecting time-stamped pictures and event description of everyone entering and exiting the area.

A Commercial Application:
Mounting cameras to capture video imagery is cheap, but finding available human resources to sit and watch that imagery is expensive. Surveillance cameras are already prevalent in commercial establishments, with camera output being recorded to tapes that are either rewritten periodically or stored in video archives. After a crime occurs – a store is robbed or a car is stolen – investigators can go back after the fact to see what happened, but of course by then it is too late. What is needed is continuous 24-hour monitoring and analysis of video surveillance data to alert security officers to a burglary in progress, or to a suspicious individual loitering in the parking lot, while options are still open for avoiding the crime.

Keeping track of people, vehicles, and their interactions in an urban or battlefield environment is a difficult task. The role of the AVISTA video technology platform in achieving this goal is to automatically describe people and vehicles from sensor raw data, determine their geolocation, and insert them into a dynamic scene visualization. We have developed camera embedded technology for detecting and tracking moving objects at the camera level. The Camera delivers continuous stream of images with embedded descriptors for moving objects by shape, color, movement pattern... Detected objects are classified into semantic categories such as human, human group, car, and truck using shape and color analysis, and these labels are used to improve tracking using temporal consistency constraints.

Further classification of human activity, such as walking and running, has also been achieved. Geolocations of labeled entities are determined from their image coordinates using two overlapping camera views. These computed locations feed into a higher level tracking module that tasks multiple sensors with variable pan, tilt and zoom to cooperatively and continuously track an object through the scene. All resulting object hypotheses from all smart cameras are transmitted as symbolic data (XML Metadata) packets back to a central operator control unit, where they are displayed on a web based graphical user interface to give a broad overview of scene activities.

The Australian Jockey Club netwotk represent a real example of how the AVISTA smart camera technology platform works.
The AVISTA smart camera acts as an intelligent processor. Its function is to capture and analyze video imagery for the presence of significant entities or events, and to transmit that information symbolically as XML metadata to the management platform. This arrangement allows for many different sensor modalities to be seamlessly integrated into the system. Furthermore, performing as much video processing as possible on each camera reduces the bandwidth requirements of the AVISTA network. Full video signals do not need to be transmitted; only symbolic XML metadata extracted from video signals is distributed.



The AVISTA smart camera has an open platform architecture around one module named the Morpheus processor. The Morpheus is a System on Chip (SoC) solution.
The open architecture allows camera manufacturers, system integrators, IT department, and end users to build advanced and robust cost effective solutions.

I have mentioned in many occasions that video analytics should be Customer driven not vendor driven. Open architecture solutions are the path towards product maturity and SHIFTING THE SECURITY PARADIGM. In the last few years we have witnessed many analytics companies driving applications like Motion detection, left object...Unfortunately we have witnessed a large disappointment from end users and the statement was clear "the technology is not their yet". In my next post I will elaborate more on the methodology to use in order to build advanced smart sensor networks using the AVISTA open architecture. I will also share with you our effort towards a smart camera unified standard and our position as a global innovator.

PERCEPTUALLY AWARE INTELLIGENT SYSTEMS.

As human we have the ability to sense, analyze and make judgment, abnormal odor, abnormal sound, abnormal action, someone running/screaming, some one you never saw in the building (suspicious) , someone walking in the opposite direction.... That's what we call human intelligence.

How can we build automated systems that will help us to find "THE BAD GUYS", systems, networks and sensors that will effectively contribute to our security and safety? Systems that can make sense and easily contribute to human intelligence.

We spent last week working with one of our partners in the retail industry, setting up a new AVISTA network, it was a very challenging week and in the same time rewarding,. The power of our devices is based on the capability of detection/processing and then distributing sensor metadata. Metadata is used to facilitate the understanding, use and management of data sensors. Let me put this in simple words,

1. A smart camera internally classify what they are seeing against a model set of parameters defining normal behavior, deviation from that norm trigers an alarm, the alarm can be predefined, as visual, message(email,mms, sms), XML message.

2. Each individual smart cameras is networked, each information obtained from each individual camera can be related to information picked up from every other camera in the network, in real time (XML), and in image database and archive. If a camera A indicate abnormal behavior at location A1, that information is potentially meaningful in isolation, but if 3 cameras C, D, and E at location A2,A3, A4, note suspicious loitering activity, it indicates a wider potential pattern of threat.

3. Web based command and management platform, enabling sharing sensory information and human intelligence.This is what I call "PERCEPTUALLY AWARE INTELLIGENT SYSTEMS". Perceptually aware intelligent systems are capable of finding things that are interesting in a manner similar to the way that an analyst might find things that are interesting.

To maintain our commitment towards advances in vision algorithms and massive distributed parallel processors and distributed smart sensors, we released last month our OEM platform that will allow camera manufacturers to build advanced systems and take smart camera technology to the next step of evolution. First quarter 2008 we will be distributing our camera development board in the US, Europe and China, with the latest innovation a web based SDK reducing dramatically time to market and building a strong efficient support and partnership from product specs to first manufacturing run.

The security and surveillance industry has been and is a conservative industry. However, since the 9/11 we are witnessing a gradual change, private businesses, government organization and law enforcement agencies are working together towards building efficient secure cities...For the first time we are talking about "STRATEGIES". This is what I call "VISION" . This is what makes my job at DigiSensory Technologies exciting, listening to private business operators, government leaders, and law enforcement professionals, understanding their strategies, their vision towards: PERCEPTUALLY AWARE INTELLIGENT SYSTEMS.

HOMELAND SECURITY

I spent the month of September in Washington DC talking with different companies, governement organizations, professionals about one thing: smart camera technology for homeland security and law enforcement.

As the number of installed surveillance cameras increases, and the cost of cameras decreases the CCTV/security industry is facing the prospect of large multimedia archives where it may be very difficult to locate specific content. To be able to get the full benefit of this wealth of multimedia data, based on my discussions with Law Enforcement agencies security and home land security organizations it was clear that the main requirement today in modern surveillance is the need to be able to automatically highlight events of interest to the operator in real-time. Furthermore, past experiences such as the Australian embassy bombing in Jakarta, the London subway bombing showed the urgent need to make it possible to quickly identify and retrieve content which meets particular criteria.

My last stop was Las Vegas to attend ASIS2007. At the expo we have demonstrated and comunicated how advances in distributed smart camera technology can be used to effectively analyze, tag, store, search, alert, distribute and retrieve proactively multimedia content in surveillance systems. AVISTA smart IP cameras are utilized for multimedia compression, analyze in real time, and metadata are automatically generated to describe the multimedia content and distribute intelligence. A simple still effective Description Definition Language (DDL) standard was demonstrated to define application-specific multimedia descriptors and description schemes for multimedia management.


I had a good discussion with our partners from d3data(http://www.d3data.com/), and exciting evenings of ideas...A graphical multimedia retrieval application is needed to provide content-based searching, browsing, retrieval and playback over a web based command & control platform is the "DREAM PRODUCT", the "KILLER APPLICATION".

We are talking TECHNOLOGY let me convert that to Security/homelandsecurity/law enforcement the result is: "Critical Infrastructure Protection".
The Australian governement define "Critical Infrastructure Protections" as: "those physical facilities, supply chains, information technologies and communication networks which, if destroyed, degraded or rendered unavailable for an extended period, would adversely impact on the social or economic well-being of the nation or affect Australia’s ability to ensure national security".

The purpose is clear, I can envision a future where the smart sensors networks and smart cameras will be the driving force of our protection and homeland security. This statement has been reinforced when I attended a security breakfast series sponsored by Crowel Moring and Legend Merchant Group. Let me share with you the important points of the PARADIGM SHIFT:

1. The convergence of Physical and IT security



2. The convergence of private sector and gov security solutions



3. The convergence of technologies to serve both security and business needs



4. Incorporation of intelligent sensors and analytics into traditional security



5. Required compliance-driven security software



6. Security solutions producing positive ROI through operational efficiency



7. Transfer of technology and services to state, local, and private sector
   

We will dedicate our effort in the next few years to the smart sensor revolution, this year we have been active and effective in supporting our local police and we have been committed to the private sector in providing real value.
I said it before and said it in Vegas: "Our objective is to think beyond new products. Our aim is to imagine, innovate and create products and services that have the power to profoundly change customers’ expectations"