[go: up one dir, main page]

WO2006068463A1 - Intelligent distributed image processing - Google Patents

Intelligent distributed image processing Download PDF

Info

Publication number
WO2006068463A1
WO2006068463A1 PCT/NL2004/000912 NL2004000912W WO2006068463A1 WO 2006068463 A1 WO2006068463 A1 WO 2006068463A1 NL 2004000912 W NL2004000912 W NL 2004000912W WO 2006068463 A1 WO2006068463 A1 WO 2006068463A1
Authority
WO
WIPO (PCT)
Prior art keywords
slave
unit
processing unit
data
network
Prior art date
Application number
PCT/NL2004/000912
Other languages
French (fr)
Inventor
Mark Bloemendaal
Original Assignee
Ultrawaves Design Holding B.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ultrawaves Design Holding B.V. filed Critical Ultrawaves Design Holding B.V.
Priority to EP04808828A priority Critical patent/EP1834486A1/en
Priority to PCT/NL2004/000912 priority patent/WO2006068463A1/en
Publication of WO2006068463A1 publication Critical patent/WO2006068463A1/en

Links

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19639—Details of the system layout
    • G08B13/19641—Multiple cameras having overlapping views on a single scene
    • G08B13/19643—Multiple cameras having overlapping views on a single scene wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
    • G08B13/19608—Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19654—Details concerning communication with a camera
    • G08B13/19656—Network used to communicate with a camera, e.g. WAN, LAN, Internet
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B25/00—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems
    • G08B25/009—Signalling of the alarm condition to a substation whose identity is signalled to a central station, e.g. relaying alarm signals in order to extend communication range
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources

Definitions

  • the present invention relates to a surveillance system, e.g. for surveillance of an area to be monitored, comprising a main camera unit provided with an imaging sensor, a processing unit connected to the imaging sensor, and a network communication unit connecting the processing unit to a data network, and at least one slave camera unit provided with a slave imaging sensor, a slave processing unit connected to the slave imaging sensor, and a slave network communication unit connecting the slave processing unit to the data network.
  • Such a camera network is e.g. known from American patent US-B-6,724,421.
  • This publication discloses a video surveillance system with pilot and slave cameras.
  • the pilot camera is a stationary camera with a field of view covering an area under surveillance.
  • Each slave camera is arranged to have a field of view covering only part of the area under surveillance.
  • An object in the area may be tracked using the slave cameras under control of a microprocessor.
  • the present invention seeks to provide a surveillance system capable of providing a flexible surveillance system which may be used and put in operation under various circumstances and for various surveillance functions.
  • a surveillance system according to the preamble defined above is provided, in which the processing unit is configured to provide an image processing function on data from the main camera unit and the at least one slave camera unit (e.g. for object tracking), in which the image processing function comprises controlling data flow in the data network and distributing the image processing function operations over the main processing unit and the at least one slave processing unit.
  • the image processing function operations are distributed such that use of processing power of the main processing unit and the at least one slave processing unit is optimized.
  • the image processing function may in a further embodiment comprise minimizing data flow in the data network. This allows to make the most efficient use of the data network, and also makes the surveillance system more robust and fault tolerant.
  • the image processing function comprises controlling the actual bandwidth use in the data network. Branches of the data network which allow a lot of data traffic are used in favor of branches with a lower data transmission capacity.
  • the surveillance system may further comprise further network units which are connected to the data network, in which the further units are provided with a data processing unit, which may be used by the main processing unit to allocate image processing function operations to. Further network units may include, but are not limited to relay units, controller units, operator consoles, etc.
  • the main processing unit controls the field of view (zoom, pan) of the main camera unit and the at least one slave camera unit, such that an object to be monitored is always in view of at least the main camera unit or the at least one slave camera unit.
  • Such a control allows tracking of objects or persons in an area under surveillance.
  • the required processing for the tracking function may be executed in the main processing unit, in the slave processing unit associated with the slave camera unit providing images, or in any other processing unit in the surveillance system.
  • the main processing unit is in a further embodiment configured to provide the image processing function on the basis of system parameters, the system parameters comprising network parameters, such as network topology, (actual) available bandwidths in branches of the data network, capabilities of processing units connected to the data network.
  • network parameters such as network topology, (actual) available bandwidths in branches of the data network, capabilities of processing units connected to the data network.
  • Fig. 1 shows a schematic diagram of a number of camera units in a network configuration
  • Fig. 2 shows a schematic diagram of a surveillance area including a number of cameras
  • Fig. 3 shows a schematic diagram of a complete surveillance network in which the present invention may be utilized.
  • a possible configuration of camera units 10, 20, in which the present invention may be used, is shown schematically in Fig. 1.
  • a main camera unit 10 comprises an imaging sensor 11, such as a camera, and a main processing unit 12 connected to the camera 11.
  • the imaging sensor 11 may comprise multiple, independently controllable cameras as well.
  • the main camera unit 20 is connected to a data network 15.
  • Further camera units 20 (or slave camera units) are also connected to the data network 15.
  • Each camera unit 20 comprises a slave camera 21, a slave processing unit 22 connected to the slave camera 21, and a network communication unit or interface 23.
  • the main camera unit 10 and slave camera units 20 may be identical types of units with the same components.
  • the designation of 'main' and 'slave' in that case only relates to the functional role of the respective camera unit in the present invention, as explained below.
  • the data network 15 may be of a simple form, merely connecting the main camera unit 10 and slave camera units 20 for transmission of data, or alternatively, the data network is more complex.
  • a first embodiment of such a data network is shown schematically in Fig. 2.
  • An area controller unit 30a is envisaged to control a plurality of camera units 10, 20 in order to be able to observe an area to be monitored (as indicated by 'area 1 ').
  • Such a controller unit 30a may be equipped with observer monitors (not shown), or with further equipment (e.g. a console) for controlling the data network equipment and for processing and storing data, such as image or video data from the camera units 10, 20.
  • observer monitors not shown
  • further equipment e.g. a console
  • the area controller unit 30a is further connected to an interface 31 , which in turn is connected to a number of antennas 3.
  • the camera units 10, 20 are also provided with antennas 3, as a result of which the control data and the data from the camera units 10, 20 may be transmitted in a wireless manner (indicated by broken lines between the antennas 3). This wireless transmission may be accomplished e.g. using Wireless LAN technology, or other wireless techniques, such as UMTS.
  • further devices may be used, such as the relay unit 32 as indicated in Fig. 2, which allows to span a larger distance between area controller unit 30a and camera units 20.
  • the further device 32 may also be equipped with processing means (not shown), which allow to process image/video data as will be explained further below.
  • the area controller unit 30a may be part of a larger surveillance system 7, as e.g. shown schematically in Fig. 3.
  • the area controller unit 30a is equipped with a system antenna 5, in order to be able to communicate in a wireless manner with a central office unit 40, which is also equipped with a system antenna 5.
  • the wireless communication may in this embodiment be implemented as a wide area network, e.g. using a Wireless Local Loop (WLL) standard.
  • WLL Wireless Local Loop
  • the central office unit 40 can then monitor and control several area controller units 30a, 30b, 30n, including data flow in each of the respective area systems.
  • the central office unit 40 may be connected to a further network 41, e.g. an IP network, in order to allow access to the central office unit 40 from other sites (remote access).
  • a further network 41 e.g. an IP network
  • the processing power of each of the components of an entire surveillance system 7, the bandwidth of the respective network branches, and other possible resources, such as storage, are optimally used.
  • Intelligent image analysis functions may be assigned to predetermined ones of the camera units 10, 20 in the surveillance system 7.
  • the main camera unit 10 is arranged to control one or more other (slave) camera units 20, e.g. to obtain images of the same specific object from a different angle.
  • the monitoring function can use the redundant images of the object originating from different cameras 11, 21, in order to better implement the monitoring function (e.g. by being resistant against failure or sabotage of one of the cameras 11, 21).
  • the main processing unit 12 is, in this embodiment, arranged to control the data which is needed from the slave camera units 20. E.g., it may be sufficient to receive images with a predetermined frequency, or with a predetermined resolution. These parameters may be further restricted by bandwidth limitations in the network 15 between the master camera unit 10 and slave camera unit 20. E.g. the main processing unit 12 may send instructions to the slave processing unit 22 to first compress images obtained by the slave camera before transmission to the master camera unit 10 in order to comply with bandwidth constraints.
  • the main processing unit 12 may be arranged to control the viewing angle and zoom angle of other slave camera units 20, in order to be able to track an object, e.g. in case of the function of tracking a moving object in a surveillance area.
  • Prior knowledge of the area, field of view of the various slave camera units 20, and position of the slave camera units 20 (or rather slave cameras 21) is then required to correctly execute the object tracking function.
  • the main processing unit 12 is arranged to make an optimum use of available processing capacity in the surveillance system 7. It may assign specific operations on data to another slave camera unit 10, which is known to have spare capacity. Likewise, specific operations on data may be assigned to other components in the surveillance system 7 having spare processing capacity, such as the relay unit 32, one of the area controller units 30, or even the central office unit 40. Preferably, the main processing unit 12 assigns the specific operations to components as close as possible to the origin of the data to be processed, in order to lower the bandwidth requirements for the data network 15.
  • bandwidth restrictions of the data paths in the data network 15 are taken into account in a further embodiment. This may be accomplished by keeping the data flow as close as possible to either the origin of the data, or the destination of the
  • the main processing unit 12 has knowledge of the storage capacity of each component in the surveillance system 7 (e.g. in the form of a dynamic table), and, for the function of (temporary) storage of image data, assigns specific slave camera units 20 (or other system components 30, 32, 40) for storage of image data, possibly form another source of origin than the assigned unit.
  • the main processing unit 12 may be arranged to perform other specific functions in the surveillance system, e.g. intelligent image processing. For each function, usually a specific executable may be sent to the main processing unit 12 by the respective area controller unit 30, or even by the central office unit 40.
  • the executable associated with a function may comprise sub-executables with operations on image data. These sub- executables may be transmitted to a slave processing unit 22 for execution of the associated operations.
  • the main processing unit 12 also controls the (timely) flow of data to the assigned slave processing unit, and the flow of result data to the eventual source destination (e.g. the main processing unit 12 itself).
  • Some functions related to a surveillance system may be event driven. This means that upon detection of a certain event, a number of further operations are executed on image data in the system. As a result, it is impossible to predict exactly which resource (processing power, storage space, bandwidth) is needed at what time.
  • the main processing unit 12 is arranged to dynamically adapt the resources in the surveillance system 7 depending on actual use of the surveillance system 7. This allows to perform load balancing of the resources available in the surveillance system 7.
  • image processing software which allows to detect specific events (e.g., theft detection, behavior characterization of individuals or groups of individuals). These image processing software allow to implement complex image processing functions, such as multi channel movement detection, correlation of images from different cameras 11, 21, etc.
  • these complex and often time and resource consuming image processing software may be utilized to improve the performance of the surveillance system 7, without the necessity to enlarge the processing and storage capacity of individual components of the surveillance system 7, as part of the image processing functions may be performed in system components other than the component from which the data originates or the component for which the data is destined.
  • the main processing unit 12 is capable of assigning the appropriate resources depending on the circumstances, without the need to centrally monitor the resource usage of individual components or elements of the surveillance system 7.
  • the main processing unit 12 can take its role as main processing unit 12 after detection of a predetermined event. Up until detection of the event, the specific main camera unit 10 acts as a normal slave camera unit 20. Having taken the role of main camera unit 10, the main processing unit 12 then establishes which further data it needs to execute the assigned function, and which other resources in the surveillance system 7 are needed. Also, it is possible that a main camera unit 10 is assigned from the onset to execute a predetermined function, e.g. to monitor an object such as a painting in a museum.
  • the main processing unit 12 can then determine which image data (at which resolution, at which update rate, etc.) it needs from the slave camera units 20 in the same area, to optimally perform its function.
  • other slave camera units 20 may also be able to capture the object in its field of view, and the main processing unit 12 may revert to data from these slave camera units 20 in case its own view on the object disappears (e.g. caused by malfunction of sabotage of the main camera 11).
  • Camera units 10, 20 in a surveillance system network 7 are provided with processing and storage capability.
  • the processing capability is e.g. used in a decentralised manner to determine where a predetermined event takes place.
  • One of the camera units 10, 20 takes the role of master camera unit 10, and controls at least one other camera unit as a slave camera unit 20 for processing images of a scene.
  • Processing capability as present in the surveillance system network 7, as well as available network bandwidth and other system resources are monitored and adapted as necessary in the given circumstances.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Studio Devices (AREA)

Abstract

Surveillance system, e.g. for surveillance of an area (30) to be monitored. The surveillance system comprises a main camera unit (10) with an imaging sensor (11), a processing unit (12), and a network communication unit (13). Furthermore, one or more slave camera units (20) are present with a slave imaging sensor (21), a slave processing unit (22), and a slave network communication unit (23). The processing unit (12) is configured to provide an image processing function on data from the main camera unit (10) and the slave camera unit(s) (20). The image processing function comprises controlling data flow in the data network (15) and distributing the image processing function operations over the main processing unit (12) and the slave processing unit(s) (22).

Description

Intelligent distributed image processing
Field of the invention
The present invention relates to a surveillance system, e.g. for surveillance of an area to be monitored, comprising a main camera unit provided with an imaging sensor, a processing unit connected to the imaging sensor, and a network communication unit connecting the processing unit to a data network, and at least one slave camera unit provided with a slave imaging sensor, a slave processing unit connected to the slave imaging sensor, and a slave network communication unit connecting the slave processing unit to the data network.
Prior art
Such a camera network is e.g. known from American patent US-B-6,724,421. This publication discloses a video surveillance system with pilot and slave cameras. The pilot camera is a stationary camera with a field of view covering an area under surveillance. Each slave camera is arranged to have a field of view covering only part of the area under surveillance. An object in the area may be tracked using the slave cameras under control of a microprocessor.
American patent US-B-6,359,647 described an automated camera handoff system in a multiple camera system. A number of camera's positioned in a secured area, are controlled by a single camera handoff system controller, e.g. to switch to another camera when an object is at the edge of a camera's field of view. All data from all camera's in the multiple camera system has to be transported to the handoff system controller, which requires extensive cabling or wideband wireless communication channels.
International patent application WO00/75859 discloses system with a digital camera using a method for distributed image processing and wireless transmission of image data. The system is arranged to reduce the amount of processing necessary for the digital camera and for reducing the required bandwidth necessary for transmitting the images to a target platform. Processing of image data is deferred from the digital camera to a target platform, and data compression is applied to lower the (wireless) bandwidth requirements. Summary of the invention
The present invention seeks to provide a surveillance system capable of providing a flexible surveillance system which may be used and put in operation under various circumstances and for various surveillance functions. According to the present invention, a surveillance system according to the preamble defined above is provided, in which the processing unit is configured to provide an image processing function on data from the main camera unit and the at least one slave camera unit (e.g. for object tracking), in which the image processing function comprises controlling data flow in the data network and distributing the image processing function operations over the main processing unit and the at least one slave processing unit. This provides a very efficient and flexible use of the surveillance system resources. System capacities no longer have to be designed for maximum peak usage, as the peak usage may be distributed over multiple resources available in the surveillance system. It is noted that American patent application US-A-2004/0131254 discloses a system and method for object identification and behavior characterization using video analysis. A computer coupled to a camera is configured to analyze a received video stream, in order to identify, track and classify objects, and to perform behavior identification. However, only a single camera is used, and a dedicated image processing device which is connected to the camera.
In a further embodiment, the image processing function operations are distributed such that use of processing power of the main processing unit and the at least one slave processing unit is optimized. By having an optimal distribution of processing power in use over various components in the surveillance system, it is possible to maintain a certain level of reserve processing power which allows to efficiently react to increase in demand of processing power.
The image processing function may in a further embodiment comprise minimizing data flow in the data network. This allows to make the most efficient use of the data network, and also makes the surveillance system more robust and fault tolerant. Advantageously, the image processing function comprises controlling the actual bandwidth use in the data network. Branches of the data network which allow a lot of data traffic are used in favor of branches with a lower data transmission capacity. The surveillance system may further comprise further network units which are connected to the data network, in which the further units are provided with a data processing unit, which may be used by the main processing unit to allocate image processing function operations to. Further network units may include, but are not limited to relay units, controller units, operator consoles, etc.
In a further embodiment, the main processing unit controls the field of view (zoom, pan) of the main camera unit and the at least one slave camera unit, such that an object to be monitored is always in view of at least the main camera unit or the at least one slave camera unit. Such a control allows tracking of objects or persons in an area under surveillance. The required processing for the tracking function may be executed in the main processing unit, in the slave processing unit associated with the slave camera unit providing images, or in any other processing unit in the surveillance system.
The main processing unit is in a further embodiment configured to provide the image processing function on the basis of system parameters, the system parameters comprising network parameters, such as network topology, (actual) available bandwidths in branches of the data network, capabilities of processing units connected to the data network. By using knowledge of the surveillance system, and adapting usage of resources to actual circumstances, a very flexible and efficiently operating surveillance system is provided.
Short description of drawings
The present invention will be discussed in more detail below, using a number of exemplary embodiments, with reference to the attached drawings, in which
Fig. 1 shows a schematic diagram of a number of camera units in a network configuration;
Fig. 2 shows a schematic diagram of a surveillance area including a number of cameras; Fig. 3 shows a schematic diagram of a complete surveillance network in which the present invention may be utilized. Detailed description of exemplary embodiments
A possible configuration of camera units 10, 20, in which the present invention may be used, is shown schematically in Fig. 1. A main camera unit 10 comprises an imaging sensor 11, such as a camera, and a main processing unit 12 connected to the camera 11. The imaging sensor 11 may comprise multiple, independently controllable cameras as well. Via a network communication unit or interface 13, the main camera unit 20 is connected to a data network 15. Further camera units 20 (or slave camera units) are also connected to the data network 15. Each camera unit 20 comprises a slave camera 21, a slave processing unit 22 connected to the slave camera 21, and a network communication unit or interface 23. In fact, the main camera unit 10 and slave camera units 20 may be identical types of units with the same components. The designation of 'main' and 'slave' in that case only relates to the functional role of the respective camera unit in the present invention, as explained below.
The data network 15 may be of a simple form, merely connecting the main camera unit 10 and slave camera units 20 for transmission of data, or alternatively, the data network is more complex. A first embodiment of such a data network is shown schematically in Fig. 2. An area controller unit 30a is envisaged to control a plurality of camera units 10, 20 in order to be able to observe an area to be monitored (as indicated by 'area 1 '). Such a controller unit 30a may be equipped with observer monitors (not shown), or with further equipment (e.g. a console) for controlling the data network equipment and for processing and storing data, such as image or video data from the camera units 10, 20. As shown in the embodiment of Fig. 2, the area controller unit 30a is further connected to an interface 31 , which in turn is connected to a number of antennas 3. The camera units 10, 20 are also provided with antennas 3, as a result of which the control data and the data from the camera units 10, 20 may be transmitted in a wireless manner (indicated by broken lines between the antennas 3). This wireless transmission may be accomplished e.g. using Wireless LAN technology, or other wireless techniques, such as UMTS. In the network 15, further devices may be used, such as the relay unit 32 as indicated in Fig. 2, which allows to span a larger distance between area controller unit 30a and camera units 20. The further device 32 may also be equipped with processing means (not shown), which allow to process image/video data as will be explained further below. The area controller unit 30a may be part of a larger surveillance system 7, as e.g. shown schematically in Fig. 3. For this, the area controller unit 30a is equipped with a system antenna 5, in order to be able to communicate in a wireless manner with a central office unit 40, which is also equipped with a system antenna 5. The wireless communication may in this embodiment be implemented as a wide area network, e.g. using a Wireless Local Loop (WLL) standard. The central office unit 40 can then monitor and control several area controller units 30a, 30b, 30n, including data flow in each of the respective area systems. Furthermore, the central office unit 40 may be connected to a further network 41, e.g. an IP network, in order to allow access to the central office unit 40 from other sites (remote access).
According to an embodiment of the present invention, the processing power of each of the components of an entire surveillance system 7, the bandwidth of the respective network branches, and other possible resources, such as storage, are optimally used. This is accomplished by having a single camera unit 10, 20 being designated as main camera unit 10 for a specific aspect or function of the surveillance system 7, e.g. monitoring a specific object (e.g. a painting in a museum). Intelligent image analysis functions may be assigned to predetermined ones of the camera units 10, 20 in the surveillance system 7. Depending on the specific function assigned to the main camera unit 10, e.g. monitoring a stationary object, the main camera unit 10 is arranged to control one or more other (slave) camera units 20, e.g. to obtain images of the same specific object from a different angle. In this manner, the monitoring function can use the redundant images of the object originating from different cameras 11, 21, in order to better implement the monitoring function (e.g. by being resistant against failure or sabotage of one of the cameras 11, 21).
The main processing unit 12 is, in this embodiment, arranged to control the data which is needed from the slave camera units 20. E.g., it may be sufficient to receive images with a predetermined frequency, or with a predetermined resolution. These parameters may be further restricted by bandwidth limitations in the network 15 between the master camera unit 10 and slave camera unit 20. E.g. the main processing unit 12 may send instructions to the slave processing unit 22 to first compress images obtained by the slave camera before transmission to the master camera unit 10 in order to comply with bandwidth constraints.
Also, the main processing unit 12 may be arranged to control the viewing angle and zoom angle of other slave camera units 20, in order to be able to track an object, e.g. in case of the function of tracking a moving object in a surveillance area. Prior knowledge of the area, field of view of the various slave camera units 20, and position of the slave camera units 20 (or rather slave cameras 21) is then required to correctly execute the object tracking function.
Furthermore, the main processing unit 12 is arranged to make an optimum use of available processing capacity in the surveillance system 7. It may assign specific operations on data to another slave camera unit 10, which is known to have spare capacity. Likewise, specific operations on data may be assigned to other components in the surveillance system 7 having spare processing capacity, such as the relay unit 32, one of the area controller units 30, or even the central office unit 40. Preferably, the main processing unit 12 assigns the specific operations to components as close as possible to the origin of the data to be processed, in order to lower the bandwidth requirements for the data network 15.
When assigning specific operations associated with a certain function (such as object tracking), bandwidth restrictions of the data paths in the data network 15 are taken into account in a further embodiment. This may be accomplished by keeping the data flow as close as possible to either the origin of the data, or the destination of the
(processed) data.
In a further embodiment, the main processing unit 12 has knowledge of the storage capacity of each component in the surveillance system 7 (e.g. in the form of a dynamic table), and, for the function of (temporary) storage of image data, assigns specific slave camera units 20 (or other system components 30, 32, 40) for storage of image data, possibly form another source of origin than the assigned unit.
The main processing unit 12 may be arranged to perform other specific functions in the surveillance system, e.g. intelligent image processing. For each function, usually a specific executable may be sent to the main processing unit 12 by the respective area controller unit 30, or even by the central office unit 40. The executable associated with a function may comprise sub-executables with operations on image data. These sub- executables may be transmitted to a slave processing unit 22 for execution of the associated operations. In this case, the main processing unit 12 also controls the (timely) flow of data to the assigned slave processing unit, and the flow of result data to the eventual source destination (e.g. the main processing unit 12 itself).
Some functions related to a surveillance system may be event driven. This means that upon detection of a certain event, a number of further operations are executed on image data in the system. As a result, it is impossible to predict exactly which resource (processing power, storage space, bandwidth) is needed at what time. In a further embodiment, the main processing unit 12 is arranged to dynamically adapt the resources in the surveillance system 7 depending on actual use of the surveillance system 7. This allows to perform load balancing of the resources available in the surveillance system 7.
The person skilled in the art is aware of image processing software which allows to detect specific events (e.g., theft detection, behavior characterization of individuals or groups of individuals). These image processing software allow to implement complex image processing functions, such as multi channel movement detection, correlation of images from different cameras 11, 21, etc.
Using the present invention, these complex and often time and resource consuming image processing software may be utilized to improve the performance of the surveillance system 7, without the necessity to enlarge the processing and storage capacity of individual components of the surveillance system 7, as part of the image processing functions may be performed in system components other than the component from which the data originates or the component for which the data is destined.
The main processing unit 12 is capable of assigning the appropriate resources depending on the circumstances, without the need to centrally monitor the resource usage of individual components or elements of the surveillance system 7.
In a further embodiment, the main processing unit 12 can take its role as main processing unit 12 after detection of a predetermined event. Up until detection of the event, the specific main camera unit 10 acts as a normal slave camera unit 20. Having taken the role of main camera unit 10, the main processing unit 12 then establishes which further data it needs to execute the assigned function, and which other resources in the surveillance system 7 are needed. Also, it is possible that a main camera unit 10 is assigned from the onset to execute a predetermined function, e.g. to monitor an object such as a painting in a museum. From the knowledge of the system parameters, such the other slave camera unit parameters (position, field of view), the main processing unit 12 can then determine which image data (at which resolution, at which update rate, etc.) it needs from the slave camera units 20 in the same area, to optimally perform its function. E.g. other slave camera units 20 may also be able to capture the object in its field of view, and the main processing unit 12 may revert to data from these slave camera units 20 in case its own view on the object disappears (e.g. caused by malfunction of sabotage of the main camera 11).
Camera units 10, 20 in a surveillance system network 7 are provided with processing and storage capability. The processing capability is e.g. used in a decentralised manner to determine where a predetermined event takes place. One of the camera units 10, 20 takes the role of master camera unit 10, and controls at least one other camera unit as a slave camera unit 20 for processing images of a scene. Processing capability as present in the surveillance system network 7, as well as available network bandwidth and other system resources are monitored and adapted as necessary in the given circumstances.
In the above description, examples have been described of embodiments of the present invention, relating to an exemplary surveillance system 7. For the person skilled in the art it will be apparent that various modifications and amendments can be made to the surveillance system configuration without departing from the scope of the present invention, as described in the appended claims.

Claims

1. Surveillance system, e.g. for surveillance of an area (30) to be monitored, comprising a main camera unit (10) provided with an imaging sensor (11), a processing unit (12) connected to the imaging sensor (11), and a network communication unit (13) connecting the processing unit (12) to a data network (15), and at least one slave camera unit (20) provided with a slave imaging sensor (21), a slave processing unit (22) connected to the slave imaging sensor (21), and a slave network communication unit (23) connecting the slave processing unit (22) to the data network (15), in which the processing unit (12) is configured to provide an image processing function on data from the main camera unit (10) and the at least one slave camera unit (20), in which the image processing function comprises controlling data flow in the data network (15) and distributing the image processing function operations over the main processing unit (12) and the at least one slave processing unit (22).
2. Surveillance system according to claim 1, in which the image processing function operations are distributed such that use of processing power of the main processing unit (12) and the at least one slave processing unit (22) is optimized.
3. Surveillance system according to claim 1 or 2, in which the image processing function comprises minimizing data flow in the data network (15).
4. Surveillance system according to claim 1, 2 or 3, in which the image processing function comprises controlling the actual bandwidth use in the data network (15).
5. Surveillance system according to any one of claims 1 to 4, further comprising further network units (32) which are connected to the data network, in which the further network units (32) are provided with a data processing unit, which may be used by the main processing unit (12) to allocate image processing function operations to.
6. Surveillance system according to claim any one of claims 1 to 5, in which the main processing unit (12) controls the field of view of the main camera unit (11) and the at least one slave camera unit (21), such that an object to be monitored is always in view of at least the main camera unit (10) or the at least one slave camera unit (20).
7. Surveillance system according to any one of claims 1 to 6, in which the main processing unit (12) is configured to provide the image processing function on the basis of system parameters, the system parameters comprising network parameters, such as network topology, available bandwidths in branches of the data network (15), capabilities of processing units (12, 22) connected to the data network (15).
**************
PCT/NL2004/000912 2004-12-24 2004-12-24 Intelligent distributed image processing WO2006068463A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP04808828A EP1834486A1 (en) 2004-12-24 2004-12-24 Intelligent distributed image processing
PCT/NL2004/000912 WO2006068463A1 (en) 2004-12-24 2004-12-24 Intelligent distributed image processing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/NL2004/000912 WO2006068463A1 (en) 2004-12-24 2004-12-24 Intelligent distributed image processing

Publications (1)

Publication Number Publication Date
WO2006068463A1 true WO2006068463A1 (en) 2006-06-29

Family

ID=35033636

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/NL2004/000912 WO2006068463A1 (en) 2004-12-24 2004-12-24 Intelligent distributed image processing

Country Status (2)

Country Link
EP (1) EP1834486A1 (en)
WO (1) WO2006068463A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008100359A1 (en) * 2007-02-16 2008-08-21 Panasonic Corporation Threat detection in a distributed multi-camera surveillance system
EP2196972A1 (en) * 2008-12-12 2010-06-16 Nederlandse Organisatie voor toegepast-natuurwetenschappelijk Onderzoek TNO Object tracking system, object infrastructure provided with object tracking system and method for tracking objects
EP2629517A3 (en) * 2012-02-15 2014-01-22 Hitachi Ltd. Image monitoring apparatus, image monitoring system, and image monitoring system configuration method
CN111405189A (en) * 2020-04-17 2020-07-10 维沃移动通信有限公司 Shooting control method, electronic device, and shooting device

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2000075859A1 (en) * 1999-06-08 2000-12-14 Lightsurf Improved digital camera device and methodology for distributed processing and wireless transmission of digital images
EP1128676A2 (en) * 2000-02-28 2001-08-29 Hitachi Kokusai Electric Inc. Intruding object monitoring method and intruding object monitoring system
US6359647B1 (en) * 1998-08-07 2002-03-19 Philips Electronics North America Corporation Automated camera handoff system for figure tracking in a multiple camera system
US20030025800A1 (en) * 2001-07-31 2003-02-06 Hunter Andrew Arthur Control of multiple image capture devices
US20030163289A1 (en) * 2000-04-11 2003-08-28 Whelan Michael David Clive Object monitoring system
WO2004006184A2 (en) * 2002-07-05 2004-01-15 Aspectus Ltd. A method and system for effectively performing event detection in a large number of concurrent image sequences
US6724421B1 (en) * 1994-11-22 2004-04-20 Sensormatic Electronics Corporation Video surveillance system with pilot and slave cameras
US20040131254A1 (en) * 2000-11-24 2004-07-08 Yiqing Liang System and method for object identification and behavior characterization using video analysis
WO2004068855A1 (en) * 2003-01-20 2004-08-12 Futuretel Digital Imaging, Llc Mpeg adaptive motion digital video (scss) security system

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6724421B1 (en) * 1994-11-22 2004-04-20 Sensormatic Electronics Corporation Video surveillance system with pilot and slave cameras
US6359647B1 (en) * 1998-08-07 2002-03-19 Philips Electronics North America Corporation Automated camera handoff system for figure tracking in a multiple camera system
WO2000075859A1 (en) * 1999-06-08 2000-12-14 Lightsurf Improved digital camera device and methodology for distributed processing and wireless transmission of digital images
EP1128676A2 (en) * 2000-02-28 2001-08-29 Hitachi Kokusai Electric Inc. Intruding object monitoring method and intruding object monitoring system
US20030163289A1 (en) * 2000-04-11 2003-08-28 Whelan Michael David Clive Object monitoring system
US20040131254A1 (en) * 2000-11-24 2004-07-08 Yiqing Liang System and method for object identification and behavior characterization using video analysis
US20030025800A1 (en) * 2001-07-31 2003-02-06 Hunter Andrew Arthur Control of multiple image capture devices
WO2004006184A2 (en) * 2002-07-05 2004-01-15 Aspectus Ltd. A method and system for effectively performing event detection in a large number of concurrent image sequences
WO2004068855A1 (en) * 2003-01-20 2004-08-12 Futuretel Digital Imaging, Llc Mpeg adaptive motion digital video (scss) security system

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008100359A1 (en) * 2007-02-16 2008-08-21 Panasonic Corporation Threat detection in a distributed multi-camera surveillance system
EP2196972A1 (en) * 2008-12-12 2010-06-16 Nederlandse Organisatie voor toegepast-natuurwetenschappelijk Onderzoek TNO Object tracking system, object infrastructure provided with object tracking system and method for tracking objects
WO2010068106A1 (en) * 2008-12-12 2010-06-17 Nederlandse Organisatie Voor Toegepast-Natuurwetenschappelijk Onderzoek Tno Vehicle tracking system, vehicle infrastructure provided with vehicle tracking system and method for tracking vehicles
EP2629517A3 (en) * 2012-02-15 2014-01-22 Hitachi Ltd. Image monitoring apparatus, image monitoring system, and image monitoring system configuration method
CN111405189A (en) * 2020-04-17 2020-07-10 维沃移动通信有限公司 Shooting control method, electronic device, and shooting device
CN111405189B (en) * 2020-04-17 2021-08-31 维沃移动通信有限公司 Shooting control method, electronic device, and shooting device

Also Published As

Publication number Publication date
EP1834486A1 (en) 2007-09-19

Similar Documents

Publication Publication Date Title
CA2941428C (en) Aircraft interior monitoring
US6671589B2 (en) Method and apparatus to support remote and automatically initiated data loading and data acquisition of airborne computers using a wireless spread spectrum aircraft data services link
US8284254B2 (en) Methods and apparatus for a wide area coordinated surveillance system
US7633520B2 (en) Method and apparatus for providing a scalable multi-camera distributed video processing and visualization surveillance system
US8451329B2 (en) PTZ presets control analytics configuration
US20070035627A1 (en) Methods and apparatus for providing fault tolerance in a surveillance system
KR20090064675A (en) Surveillance camera and method using camera-to-camera collaboration
KR101961258B1 (en) Multi-channel network camera monitoring system and constructing method thereof
EP4075796B1 (en) Video surveillance system with drone support
KR102592291B1 (en) System and method for monitoring unmanned-office using mobile robot
WO2015047136A1 (en) Distributed architecture for forest video monitoring system
KR102423415B1 (en) System and method for managing network of control robot
US10021609B2 (en) Method and system for selecting a communication interface
WO2006068463A1 (en) Intelligent distributed image processing
JP3656823B2 (en) Automatic monitoring method and automatic monitoring device
HK1066447A2 (en) Backup system for digital surveillance system
US20080026713A1 (en) Methods and apparatus for a wireless surveillance system
KR20120025217A (en) System for remote controlled rtgc using wireless network
KR101883671B1 (en) Method and management server for dtitributing node
WO2018188251A1 (en) Method and system for remote control of switcher
WO2003041026A1 (en) System for audiovisual monitoring of certain areas
JP2004128659A (en) Wireless monitoring system
JP2010232826A (en) Video display system
JP3713526B2 (en) Driver's cab monitor backup system
JP2006180026A (en) Surveillance video recording system

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2004808828

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 2004808828

Country of ref document: EP