JP2005534205A - Method and apparatus for storing digital video content provided from multiple cameras - Google Patents

Abstract

The present invention provides a distributed monitoring system that allows digital storage of data and recognition of external patterns obtained in parallel operation. Digital data is stored with different levels of compression, and pattern recognition is realized while the digital data is in its compressed form. The size of the digital data frame is reduced when no movement across sequential frames is observed so that data storage is minimized. Also disclosed is a method for detecting the occurrence of an event, which includes comparing an initial compressed frame size with a later compressed frame size to determine the occurrence of the event. The method of monitoring a chance game operates a video camera to obtain a data stream that contains multiple repetitive actions stored there regarding the chance game, and automates the data stream to count multiple repetitive actions. The resulting count provides an indicator that can be used to monitor the chance game.

Description

  The present invention relates to a method and apparatus for storing digital video content provided by a plurality of cameras.

  Surveillance cameras are very well known and are used to help prevent crimes such as theft.

  It is also well known to store events recorded by surveillance cameras as a sequence of images on several types of media. Typically, such storage is done using a VCR and storing the image resulting from the event on tape. However, such a storage medium has the disadvantage that it requires frequent removal of old tapes every few hours and insertion of new tapes, which consumes time when multiple cameras are used, It's a tedious process. Furthermore, disadvantages arise because it is difficult to authenticate the records stored on the tape. Furthermore, the resulting sequence of images may not be clear as a result of the tape being played over time and then later on a player that is typically different from the player that recorded the tape.

  Storage of images obtained from such events in digital form is also considered. In such systems, images from events recorded by the camera are converted to digital form and compressed in several ways, such as using a known MPEG format. To date, however, such systems have not proved feasible for a variety of reasons. In situations where a large number of cameras are required at the same time, the bandwidth required as a result of the use of all cameras prevents the image from being stored for a sufficiently long period of time, which reduces the total amount of digital storage to digital. Sufficient measures are needed to reduce memory to the point of being economical.

  One common way to this problem is to start storing when a certain type of motion is detected, so that still images are not recorded continuously and are therefore necessary for system operation. Reduce memory requirements. The use of such an event start indicator by this method introduces its own problems such as not having a continuous record.

  Applications such as video surveillance monitor the environment and often the activities of individuals within these environments. In video surveillance, data is sent from a transfer device, such as a digital or analog camera, to a repository where the data is stored, typically between a number of processing steps. Analog systems typically store data on video cassette recorder tapes in analog form, which is bulky and cumbersome. In contrast, digital systems store data in a digital format.

  Analog systems are now widely used. This is mainly due to the low cost of analog devices with respect to the overall price per frame of camera and image data. Therefore, most surveillance systems currently in use, even with a digital transfer device such as a digital camera, require their analog form for real-time observation or analog storage on an analog monitor. However, it converts digital information to analog form in several ways. Thus, in typical systems, there are typically one or more analog switches that provide data received from a number of different cameras to respective monitors and / or video cassette recorder devices for viewing.

While all digital systems are available in prototype form, practical pricing considerations regarding both the digital transfer device and the price of the memory required to store digital data related to digital images and sound It is not widely configured. These prototypes have great limitations,
Usually, the activity captured by a video surveillance system depends on the environment being monitored. There are often areas that require monitoring of activity over a long period of time, but do not show a lot of activity over a long period of time. For example, the camera is focused on the door to the bank vault 24 hours a day, but only captures a relatively small number of people entering the vault or simply walking near the vault door. Under normal equipment, monitoring data for monitoring is typically stored in multiple ways. In analog systems, as described above, surveillance data is typically stored in analog form on a video cassette recorder.

  In digital systems, in order to reduce memory requirements, proposals have been made for the system to send data to the memory that occurs when an operation begins. At the beginning of operation, the start of data storage has the effect of reducing memory requirements, but has the undesirable effect of not continuously capturing events recorded by a particular digital camera.

  Accordingly, it is desirable to have the ability to reduce the amount of storage required to accommodate monitoring data without compromising the integrity of the monitoring system.

  In its simplest form, the surveillance system basically consists of an analog video camera fastened to a remote video monitor or an audio device fastened to a speaker as shown in FIG. May also include an audio device. In the example using a camera, the camera is directed to a spot of interest, such as a front door, an automatic teller machine, etc., and provides an image of the scene to the monitor. The operator looks at the monitor to look for suspicious or unauthorized actions in the scene. If such activity is perceived, the operator takes appropriate measures, identifies the person, and notifies the security police or the like.

  The system has one or multiple cameras, each of which can be displayed in a predetermined area of the monitor. Alternatively, the operator may switch scenes. Further, instead of one or more analog cameras, the system may use a digital camera such as a CCD camera or the like. Such a digital camera has the advantage of providing high quality and low noise compared to an analog camera.

  Another possible video surveillance structure is shown in FIG. The system uses a number of cameras connected to a monitor via a controller. The controller multiplexes several camera signals and provides them to the monitor. In addition, the position of the camera can be controlled. Operators use input devices such as keyboards, joysticks, etc. to command the controller to control the movement of the camera, so that they are directed to specific areas within those ranges and are image issues Track characteristics. Also, an input device for controlling the control device is used to instruct the control device to give a specific camera signal to the monitor.

  FIG. 11C shows another configuration of the video surveillance system. In this example, the video recording device is connected to the camera output, the monitor input, or both. Video recording devices, such as video cassette recorders for analog cameras, can record camera signals for archiving, later investigation, etc. In addition, images displayed on the monitor as evidence of activity that occurred in the environment being inspected can be recorded. In a digital system, the video storage device may be a digital storage device, a mass storage device such as a hard disk drive, or the like. When a hard disk drive is used, it may be a separate device from the user control device and camera control device, or it may be part of an integrated system.

  When the camera is an analog model, those signals are stored in analog or digital storage. With one or more analog storage devices, such as a video cassette recorder, the camera signal is stored on a video tape just like a television signal. For example, in a digital surveillance system that digitizes camera signals or a system that uses a digital storage device of an analog camera system, the camera image is pixilated and stored as a data file in the digital storage device. Files may be decompressed or they may be compressed using a compression algorithm to make maximum use of storage space.

  If the camera image is stored continuously in the digital storage without being deleted, the storage (or portion allocated for storage of the camera image) will be full as a result. At that point, the stored data and incoming data must be made to accommodate new data.

  Therefore, there is a general need for an improved monitoring system.

  The present invention provides a distributed monitoring system that allows digital storage of data and recognition of external patterns obtained in parallel operation. Furthermore, the digital data is stored at different compression levels, and pattern recognition is realized while the digital data is in its compressed form.

  The present invention described herein provides an effective technique for data frame adaptation to minimize storage size, source noise cancellation, and source authentication of a data frame transfer device, for example, in a surveillance system.

  The present invention discloses a method and system for adapting the size of a digital data frame to minimize data storage, canceling source noise present in the digital data frame, and authenticating the digital data frame.

  One embodiment of the present invention provides a control program for controlling a digital storage device. The control program monitors the status of the digital storage device. When the storage device (or the portion of it allocated to the image storage device) is full and new information needs to be added, the control program will erase the information to make room for new information in the storage device. To order. This is done based on various data parameters such as the priority of individual messages or data units, the age of each message or data unit, etc. For example, when information needs to be erased from the digital storage to make room for new information, the oldest data of the high priority camera is saved in place of the new data of the low priority camera. In this way, the digital storage device can be used efficiently.

  The present invention also includes a method for detecting the occurrence of an event that exists within a stored sequence of frames for the scene being described, and a method for operating on data about the scene once the event is detected.

  The present invention also includes a method for detecting the occurrence of an event, which follows an initially stored frame to determine whether the magnitude of change between frames is greater than a predetermined threshold. Includes comparing to stored frames. In the preferred embodiment, the first and later stored frames are compressed and operated when compressed to determine the occurrence of an event.

  Also provided is a method that operates on data relating to a scene where an event is detected, which provides a still image at intervals of, for example, every 5-10 frames using at least data from the storage device that detected the event. Is included. In yet another embodiment, still images from other storage devices associated with the storage device that detected the event in a predetermined manner are also obtained at certain time intervals.

  A monitor program is also provided that monitors images coming from one or more surveillance cameras. When an image or set of images meets certain conditions, the monitor program performs the appropriate action.

  A method for automatically monitoring chance games is described. This method operates the video camera to obtain a data stream containing multiple repetitive actions stored there on a chance game and automatically parses the data stream to count multiple repetitive actions Steps are included and the resulting count provides an indicator that can be used to monitor the chance game.

The foregoing and other objects, features, and advantages of the present invention will be further described in the following details by way of example, and not by way of limitation, with reference to the drawings. In the description, like reference numerals designate like parts of the invention throughout the several views.
The present invention provides a distributed monitoring system and a method of using the system. Included features are digital storage of both visible and audible monitoring information, pattern recognition of external patterns in streams of monitoring information obtained from different sources, and automatic detection of repetitive activity patterns such as these external patterns Distribute external patterns to various computer locations that can be used to generate general interpretations and reports therefrom, and distribute individual images that are automatically determined to contain the particular image in question Setting pattern recognition rules to detect the activity in question, and others described herein.

  The present invention is implemented using a combination of hardware and software elements. Although FIG. 1 illustrates an exemplary system 100 according to the present invention and a variety of different devices that allow the various substitutions described herein to be understood, the exemplary system 100 is not limited to the present invention. Should not be construed as limitations. FIG. 1 shows a plurality of conventional analog or digital cameras 110-1 to 110-n, each connected to a computer 120-1 to 120-m, preferably a system for detecting both images and sound. Is included. There are numerous connections between the camera 110 and the computer 120, both a one-to-one correspondence connection and a connection between multiple cameras 110 and a single computer 120 are shown. Further, the computer 120 is also shown as not connected to any camera, whereby digital data stored in any digital format can be operated according to the present invention, such as stored in the computer.

  Each computer 120 is preferably coupled to the central server 130 using several types of networks 150, such as the Internet or a private network. Each computer 120 and the camera 110 connected to it can be placed in any number of different locations. The computer 120 connected to the camera 110 is preferably located in or close to the same building to minimize the distance that the signal from the camera 110 must propagate until it reaches the computer 120. It may be on different floors of the same building, and the computer 120 may be in different buildings. Although the system is shown here for convenience as having a single central server 130, it will be appreciated that any or all of the computers 120 can be configured to act as the central server described herein. Let's go. However, it should be understood that it would be advantageous to have the central server 130 at a location remote from the location where the monitoring takes place, and more preferably having a remote location in a physically separate building. In typical operation, a particular company has its own server 130, and if desired, each different server 130 can be connected together via a network 150 to allow sharing of data between them. . However, in the following description, the system 100 will be described with reference to a single server 130, which can serve a number of different companies to illustrate the most flexible features of the system 100 described herein.

  In addition, various other computers 140-1 through 140-n each including a monitor 142 are also shown coupled to the server 130 via the network 150. Data received by the computer 140 is decrypted and decrypted for subsequent use so that it can be perceived visually using the monitor 142 and audibly using the speaker 144 shown. The

  This particular exemplary system 100 thus allows the camera 110 to provide at least an image, if desired, sound, which can be encoded and stored for monitoring purposes. This data can be stored in digital form at computer 120 or computer 130, as described below. In addition, the image obtained from the camera 110 compared to the external pattern or the pattern in the sound, as further described herein, can be obtained using a computer 120 and / or server 130 provided with image data from the camera 110, Can be recognized.

  The transmission of data between each computer 120 or 140 and server 130 is preferably encrypted as described further below. For data sent to the computer 120, 140 or server 130, the data is decrypted to operate there.

  Although the foregoing network is given to illustrate one way in which the advantages of the present invention described herein can be used, this is merely exemplary. For example, each different computer 120, 130, 140 can be configured as a single computer, and devices other than computers that include a processor capable of carrying out the invention described herein can be used instead. . Many other variations are possible. Devices such as those described above typically include several types of processors, such as an Intel Pentium 4 microprocessor or DSP, with program instructions written based on the considerations herein, but field programmable gates. Other hardware that implements the present invention, such as an array, can also be used. The program instructions are preferably written in C ++ or some other computer programming language. In repetitive routines used repeatedly, these can be written in assembler for fast processing.

  FIG. 1 described above illustrates the hardware elements of the monitoring system 100 in accordance with the present invention. The present invention also includes software 200 residing on each computer 120 and 140 and server 130 to provide the various functions described herein. It will be appreciated that the various modules 210 of the software 200 reside on different computers, thereby allowing the functionality to be implemented as described herein.

One feature of the software 200 is that the modules are preferably composed of functions that allow collaborative operation between them, as described below. It will be apparent from the following description that this joint operation is possible. In this regard, as shown in FIG. 2, a module 210 typically included within the computer 120 and operating with it is
A local front-end processing module 210-1;
A local pattern recognition module 210-2 that allows both internal pattern recognition and external pattern recognition, described below, as described below;
-Local multipath module 210-3;
A local encryption / decryption module 210-4;
A local user interface module 210-5;
A local priority data storage module 210-6.

Typically, a module 210 included in a computer and operating with the server 130 is
-Enables both internal pattern recognition and external pattern recognition as described below, including network priority information, event alert generation, repetitive behavior pattern analysis and associated report generation as described below. A network pattern recognition module 210-12;
-Network multipath module 213-13;
-Network encryption / decryption module 210-14;
-Network interface module 210-15;
-Network priority data storage module 210-16.

  Each of the aforementioned modules will be further described so that the operation of the system 100 can be understood.

  Local front end processing module 210-1, local user interface module 210-5, network interface module 210-15.

  The local front end processing module 210-1, the local user interface module 210-5, and the network interface module 210-15 will be described first. As generally described above, in accordance with the present invention, the surveillance system 100 allows the recording device, typically the camera 110, to be placed at various locations associated with the building, so that remotely located buildings are served. Enable. Thus, in the preferred embodiment, each building includes at least one local computer 120, but the required number of local computers depends on the amount of processing power desired with respect to the number of recording devices and other at the location itself. It changes based on the processing operation. In this regard, the local user interface module 210-5 is used to set up and track connections between cameras included in the local system. Thus, a single local interface module typically associated with a single company tracks company information, camera information, and user information.

  The tracked company information typically includes the company name and address information for each building that needs to be monitored.

  The camera information associated with a particular building includes the camera model, camera priority, camera identifiers that can be used by both local and network systems, as well as the viewing location as well as one for use at the casino. Embodiments may include a reference to a specific activity, such as a specific gaming table number, a gaming type identifier if the camera is used in a gaming table, an operational state identifier (functional or out of service), at a casino Examples of contexts used include camera purpose identifiers for whether the camera is used for games or non-game purposes. These methods of use are further described below.

  User information identifies the users who can access the system and the degree to which they should access the system. Some users have observation access, which is observation of only one part or all observations, while other users have access to make changes, action reports, etc. as further described herein.

  For each camera configuration, individual cameras 110 need to be configured to operate properly as well as need to configure and connect the system using local user interface module 210-5. In this regard, the local front end processing module 210-1 is preferably used to properly configure the camera to operate as efficiently as possible. The configuration of individual cameras 110 using the local user interface module 210-5 is described below with respect to FIGS. 3-6 and is filed on the same day as the present application and assigned to the assignee of the present invention. It is described in US Patent Application No. 042503/0273340 (Title of Invention “Method And System For Size Adaptation And Storage Minimization, Source Noise Correction, And Source Watermarking Of Digital Data Frames”). Here it is taken as a reference.

  The local user interface module 210-5 is also configured to send information about the local system to the network interface module 210-15. It will be appreciated that the network interface module 210-15 preferably includes the same information that exists for each different local user interface module 210-5. In addition, because the network interface module 210-15 receives information from a number of different computers 120, the network interface module 210-15 also provides network settings for the network external pattern desired in pattern recognition, as described further herein. Includes network-related features such as camera priority rules, network backup procedures, and network report generation. The manner in which these various local and network features are implemented is described below, but both the local user interface module 210-5 and the network interface module 210-15 are connected to the operational results performed by the other modules 210 described below. It will be appreciated that it allows access.

  As mentioned above, for convenience, with respect to the local and network camera priority rules described with the local user interface module 210-5 and the network interface module 210-15, the local user interface module 210-5 configures the camera. Camera priorities are set, and this priority scheme takes a number of different forms, with 0-10 priorities, where 0 is the highest priority and 10 is the lowest priority. In addition to the local priority setting, there is another network priority setting, thus ensuring that certain types of data are transmitted from the computer 120 to the server 130. Thus, there is also a network camera priority setting, which causes the network interface module 210-15 to start transferring data from the camera 110 that has a higher priority than the other cameras 110.

[Local pattern recognition module 210-2 and network pattern recognition module 210-12]
Local pattern recognition module 210-2 and network pattern recognition module 210-12 each provide the ability to recognize patterns in the data. The data typically includes image data that is formatted in frames. Recognized patterns include patterns in the data and external patterns generated externally and are searched in the data as further described below.

  Both the local pattern recognition module 210-2 and the network pattern recognition module 210-12 can use various pattern recognition techniques, but are preferably filed on Oct. 31, 2001, assigned to the applicant of the present invention and represented. Using pattern recognition techniques such as those described in U.S. Patent Application No. 042503/0259665 (name of the invention “Method And Apparatus For Determining Patterns Within Adjacent Blocks Of Data”), pattern recognition and compression are performed. The contents of this specification are hereby incorporated by reference.

  There is a difference between the local pattern recognition module 210-2 and the network pattern recognition module 210-12. One major difference is typically that the local pattern recognition module 210-2 is typically associated with US patent application no. Operating on uncompressed data as described in Data "), the network pattern recognition module 210-12 operates on data in a compressed form.

  In particular, the network pattern recognition module 210-12 can be operated with data that is consistently compressed. Compression is preferably the technique described in US patent application Ser. No. 09 / 727,096 filed Nov. 29, 2000 (Title “Method And Apparatus For Encoding Information Using Multiple Passes And Decoding In A Single Pass”). Realized using. In this compression scheme, the recorded data undergoes multiple passes to achieve further compression, as described in the following section.

  Both the local pattern recognition module 210-2 and the network pattern recognition module 210-12 also provide recognition of external patterns in the externally generated image. The external pattern is stored with each different level of compression present so that the external pattern can be detected regardless of the amount of compression used, as further described below.

  The network interface module 210-15, in cooperation with the local pattern recognition module 210-2 and the network pattern recognition module 210-12, identifies and tracks pattern recognition of external patterns and desired external patterns for priority. Used to do. In particular, an external pattern represents an object, which is a specific item such as a person's face, wallet, card, vehicle or vehicle license plate. Whatever the object, the present invention stores the representation of the object and uses that representation for comparison with the pattern present in the recorded image.

  A limiting element of the system 100's ability to track external patterns is that the system 100 has already acquired data from the various cameras 110 and compressed the data as described above. Performing that task alone requires substantial processing power, leaving only a few percent based on the computer power available to track the external pattern of the object. Accordingly, the network interface module 210-15 maintains a tracking of the priority of each external pattern that is retrieved based on the input from each camera 110. Some highest priority external patterns are distributed to the computer 120 in an uncompressed form using a sufficient number of points, such as 25, which, as is known, uses an (x, y) offset. In the U.S. patent application specification (Method And Apparatus For Determining Patterns Within Adjacent Blocks Of Data) with the above-referenced agent reference number 0425503/0259665 As described, it allows pattern recognition to be performed using a separate processor thread for the purpose of searching for special patterns. The low-priority external pattern is held in the server 130 in a compressed form and is searched in the server 130 in the manner described above.

  Since the computer 120 is typically operating on the received real-time image, it is clear that the external pattern located by the computer 120 is obtained more quickly than the external pattern discovered by the server 130. Need not be searched in real time. However, if the computer 120 cannot complete the search with an external pattern, it notifies the server 130 and then preferably searches for all desired external patterns and assumes that the computer 120 did not find any.

  Use pattern detection using US Patent Application No. 042503/0259665 referenced above (Title of Invention “Method And Apparatus For Determining Patterns Within Adjacent Blocks Of Data”) or other conventional techniques The operation is further described below.

  Each pattern recognition operation described can be performed by the system 100 described herein, if desired.

[Local path module 210-3 and network multipath module 210-13]
Local path module 210-3 and network multipath module 210-13 operate to provide further compression from the compression obtained by local pattern recognition module 210-2 and network pattern recognition module 210-12. The further compression is preferably filed on November 29, 2000, assigned to the same application as the present invention, US patent application Ser. No. 09 / 727,096 (invention name “Method And Apparatus For Encoding Information Using Multiple Passes” And Decoding In A Single Pass ") or other encoding / decoding process, the contents of which are hereby incorporated by reference.

  Each local path module 210-3 and network multipath module 210-13 are configured to operate in the same manner, but this is typically not the case. Rather, the local path module 210-3 is typically configured to perform some predetermined number of passes of the data it receives, thereby using further passes to further compress the data. The data is partially compressed before being sent by the computer 120 to the server 130 for further operations, including further compression operations performed by the network multipath module 210-13. In this regard, each local multipath module 210-3 that uses the same number of passes and the same compression routine is preferred, whereby data received by the server and further operated using the network multipath module 210-13. Are already compressed between various sources that receive data consistently. Thus, as will be apparent, the network multipath module 210-13 preferably includes additional multipath compression routines not found in the local multipath module 210-3 that allow further passage to take place. Typically, passes 1 and 2 are made by computer 120, while further passes are made by server 130. Even more compression requires more passes and slows down the system so that the data can be stored with user-specified fishing compressions. Since images are recorded with different levels of compression, there must be an associated compression of all external patterns at each different compression level. Thus, if the image is recorded in one of 1, 2, 5, 10, 15, or 20 passes, the external pattern is obtained in each 1, 2, 5, 10, 15, and 20 passes. Must be applied so that a properly compressed external pattern can be used during the comparison operation based on the compression of the image.

[Local encryption / decryption module 210-4 and network encryption / decryption module 210-14]
Each of the local encryption / decryption module 210-4 and the network encryption / decryption module 210-14 has the same function: data encryption and transmission to the source destination, and data encrypted and transmitted previously. Receive and decode. A number of encryption / decryption techniques exist, but one technique that is effectively implemented is US patent application Ser. No. 09 / 832,278, filed Mar. 29, 2001 (named “Method And Apparatus For Streaming Data”). Using Rotating Cryptographic Keys "), which is assigned to the same applicant as the present invention, the contents of which are hereby incorporated by reference.

[Local Priority Data Storage Module 210-6 and Network Priority Data Storage Module 210-16]
The local priority data storage module 210-6 and the network priority data storage module 210-16 maintain tracking of data stored in each computer 120 and server 130, respectively. These priority data storage modules, unlike data backup, assume the worst case scenario where no data backup is performed. Basically, both the local priority data storage module 210-6 and the network priority data storage module 210-16 operate in the same way to maintain the most important data. The identification of how the data is differentiated and how these modules work is the United States application filed on the same day as this application and assigned to the same applicant as the present invention, at agent number 042503/0273342 It is described in a patent application specification (invention name “System And Method For Managing Memory In surveillance System”), the contents of which are hereby incorporated by reference.

  Also, each local user interface module 210-5 can operate with resident software that continues to perform functions typically associated with the computer 120 if not connected to a network. The data to be stored is in accordance with the patent specification referenced in the immediately preceding paragraph if prioritization is required.

[Other environment]
As previously mentioned, the surveillance system describes security in the context of a casino, which includes a gaming table, a safe area, and other common areas that require surveillance. However, it will be appreciated that the present invention can be implemented in any environment requiring security, including financial institutions, large commercial buildings, jewelry stores, airports, and the like.

  In certain buildings, airports, etc., various levels of security are provided at various locations. At an airport, for example, the first gate entrance area may represent one level of security and the actual gate may represent another level of security. In a casino, jewelry store or financial institution, entrance 1 represents one level of security and its internal safe represents another level of security. Comparison of images of these related areas and the occurrence of alarms and other information based thereon can be used in the environment described further below. Certain characteristics of each environment can be used to make comparisons. For example, in a jewelry store, the comparison can be made between unmasked glass display case masks so that an alarm is sounded if the case is broken.

[Adaptation for wireless environment]
The present invention can also be adapted for monitoring in an environment such as an aircraft where a wire connection between the computer 120 and the server 130 is not possible. In such an environment, the available transmission bandwidth becomes a more limited factor than a wire network. Thus, in such an environment, the computer 120 is typically limited to performing the aforementioned compression operations, wireless transmission of compressed information, and pattern recognition, and other operations are adapted to receive wirelessly transmitted data. Is done on the server 130.

  The following embodiments described herein are useful for data frame adaptation to minimize storage size, data frame noise correction to aid pattern recognition, eg for source authentication of a data frame transfer device in a surveillance system Technology.

  This embodiment describes a method and system for adapting the dimensions of a digital data frame to minimize data storage, correcting source noise present in the digital data frame, and authenticating the source of the digital data frame.

  Referring initially to FIG. 3, a block diagram illustrates an exemplary transmission system 300 in accordance with the present invention and a variety of different devices that allow the various exchanges described herein to be understood. It will be appreciated that this exemplary transmission system 300 should not be construed as limiting the invention. The system 300 includes a source data transfer device 310, such as a conventional camera 310-1 to 310-N, which is connected to a computer device 320 at a data interface 318 via a respective transmission device 316-1 to 316-N. Is done.

  Source data transfer devices 310-1 to 310-N preferably include a system that detects both image and sound, but devices that can reproduce either image or sound, but not both, are also techniques of the present invention. Is within the scope. The source data transfer device 310 is analog or digital.

  In addition, the source transfer device 310 generates noise superimposed on the recorded data. The transfer device that is most likely to generate a large amount of noise is the device 310 for recording an image, in other words, a camera. While there are high quality cameras that generate only a small amount of such noise, cameras used in many surveillance environments are often of poor quality. Thus, cameras often generate a significant amount of noise that is superimposed on the actual image being recorded.

  In the image recording device 310, this noise results from the combination of internal elements used to record the image, including optical systems, transducers, digital circuits, power supplies, AC / DC converters, and the like. However, once the camera is switched on in time, it has been observed that steady state operation is reached such that the noise repeats with a periodic noise pattern. The present invention uses this property to remove periodic noise from the recorded image, as will be described hereinafter. Thus, one feature of the present invention is to correct the noise signature of a device 310 such as a camera.

  It has been discovered that the environment in which the device 310 is located is also important. If the device 310 is an analog device, each transmitting device 316 is typically analog, and if the device 310 is a digital device, each transmitting device is typically digital. A typical configuration of the analog camera device 310 includes an analog transmission device 316A that includes an analog transmission line and an amplifier positioned at a length along the analog transmission line to refresh the analog signal as appropriate. It is out. The typical configuration of the digital camera device 310 is, as is known, typically only an optical transmission line since a digital signal can propagate along a much longer optical transmission line distance than an analog signal can propagate. Includes digital transmission device 316D.

  However, in most systems, an analog switch is located in the data interface 318 that allows the data flow from various cameras to various monitors and / or recording devices to be switched. Thus, at this point, conversion of digital data to analog form is still required in many instances if previously obtained. Thus, regardless of the type of camera used, ie, analog or digital, the data interface 318 in many cases includes an analog-to-digital (A / D) converter after the analog switch, and therefore the analog output from the switch. The signal can be converted to digital form for input to computer 320. In such systems that include a digital recording device 310 in front of an analog switch, there are digital-to-analog (D / A) converters that convert the digital signal to analog form so that they can be operated by the analog switch. it can. Therefore, in addition to source noise generated from recording device 310, transmission medium 316 also affects signal degradation and amplifier distortion, especially in the analog context, and noise from digital-to-analog and analog-to-digital conversion in the digital context. It is also obvious.

  Although the foregoing systems are used in the remaining description herein, it is to be understood that these are exemplary and many other configurations are possible.

  As previously mentioned, the device 310, particularly a low quality camera, generates a periodic noise pattern that is further altered as a result of the transmission medium 316. One obvious component of this noise is likely from the power used to drive the electrical component. A DC voltage is typically used to drive circuit components, but this DC voltage is typically obtained as a result of conversion from an AC power source, which oscillates at 60 Hz / second in the United States. Therefore, this AC noise becomes a component of power supply noise, and most image devices 310 have a particularly serious effect because they record an image at a frequency relatively close to the vibration frequency of the AC power signal at 30 frames / second.

  As described above, the present invention uses the presence of this periodic noise characteristic to remove periodic noise from the recorded image, and the manner in which this is done will be described with respect to the flowchart of FIG. As indicated by step 310, the initial setup is preferably done first, thereby placing the recording device 310 and the transmission medium 316 associated with the device in place. This ensures the stability of the start routine. When step 510 is complete, a start step is initiated, and the first start step 520 switches the device 310 on after the computer 320 is configured to record the output of the device 310. Note that in this starting configuration, the amount of time required for device 310 to heat up before showing a periodic noise pattern is not known.

  When first switched on, the camera records an image taken from a known color pattern, such as a known white blotter. Initially, as indicated by step 530, the image is recorded for a number of frames, typically in the range of 200-300 frames. Each of these frames is then compared against a stored “white” image containing pixel representations corresponding to actual known color patterns to obtain a different frame, as indicated by step 540. In the next step 550, these different frames are compared with each other to determine if there are any pattern repetitions between them. A conventional pattern recognition algorithm can be used, preferably US Patent Application No. 042503/0259665, filed Oct. 31, 2001, assigned to the same applicant as the present invention. The pattern recognition algorithm described in the title “Method And Apparatus For Determining Patterns Within Adjacent Blocks Of Data”) is used. In order to use this U.S. patent application specification with agent reference number 042503/0259665 and the nomenclature described therein in pattern recognition, each frame is a reference frame, compared to each other frame, Each frame is a target frame for comparison purposes. In order to maximize the comparison, each frame can be designated as a reference frame and other frames can be designated as target frames, but such multiple comparisons result in redundant and redundant comparisons and are therefore required. It will be appreciated that the number of comparisons made is small.

  Following step 550, if the periodic noise pattern is not covered there, the periodic noise pattern can be stored at 560.

  However, if the periodic noise pattern is not covered, the recording device 310 is operated at step 570 for a period of time longer than the previous period, and the recording is recorded. Thereafter, step 550 is repeated using multiple recorded frames to eliminate periodic noise pattern coverage. Steps 560, 570, 550 are then repeated until a periodic noise pattern is found.

  With respect to the typical length of time it takes to remove the periodic noise pattern cover, in modern digital cameras such as the Fujitsu Series XV, the periodic noise pattern appears after a heat-up time of about 2 minutes. It has been determined that the periodic noise pattern typically repeats every 250-350 frame range. However, in the old analog model, the required heat-up time is about several days, but the periodic noise pattern once provided is about 2,000-4,000 frames.

  Once a periodic noise pattern is obtained, it can be used to remove noise from the recorded data, as shown in FIG. 6, so if device 310 is a camera, As in the case of an image, the detected good display can be obtained. As shown in step 610, once the recording device 310 is switched on, a start period corresponding to the previously determined heat-up period may occur so that the device 310 enters steady state operation. preferable. When this period of time has elapsed, recording of the desired scene can begin as indicated by step 620. Once recording begins, each recorded frame is input to computer 320 and synchronized with the corresponding frame from the periodic noise pattern to remove periodic noise therefrom, as indicated by step 630. Is done. Thus, as shown by step 640, each frame from which periodic noise has been removed is obtained. The frame can then be used as desired in subsequent monitoring operations.

  It should also be noted that this periodic noise pattern has been determined to be a substantially independent frequency. That is, the known white blotter is shown as described above, and any suitable color of the known color, as long as the known color is identical to the color of the benchmark frame that mathematically represented the data used for comparison. Solid materials can be used.

  In another aspect of the invention, the invention uses the resulting periodic noise pattern. As mentioned above, the periodic noise pattern is preferably detected within each frame and removed or minimized. According to another feature of the invention, the digital watermark of a particular frame generated by the source recording device 310 is performed with a noise signature. In the preferred embodiment of the present invention, camera noise is not removed every nth frame to obtain a detectable watermark indicating that the frame is actually coming from that particular source recording device 310. If a different source recorder 310 'is used instead, there will be a different noise pattern and no expected noise pattern will be found. Thus, this noise generates a digital signature that identifies the frame as coming from a particular recording device 310 rather than from a different recording device 310 ', thus preventing attempts to introduce a surrogate for the data stream. In this regard, when storing a special sequence, a periodic noise pattern is also stored in the sequence so that a special camera recording the special sequence can be proved at a later time, thereby Can be done at a later time.

  It should also be noted that although periodic noise removal of the present invention has been described with respect to real-time removal of periodic noise patterns, periodic noise removal can also be performed on previously stored data. Although an actual recording device used for recording data is desired, the noise pattern can be detected in the stored data without an actual camera.

  As described above, if the monitoring system is to digitally store the recorded data, the memory requirement becomes quite large and expensive. Minimizing the storage space required to store data, for example digital data frames, is a common goal of data transfer systems. As previously mentioned, the fact that there are systems that do not store data during periods when no motion is detected, while the fact that no continuous recording is available is undesirable.

  Although there are compression routines that can operate to minimize the amount of recorded data that needs to be stored, the amount of data is still very large. Usually, the amount of data recorded by the video surveillance system 300 depends on the monitored environment. In many environments, there are often areas that require activity monitoring over a long period of time that does not show a lot of activity over a long period of time. For example, the camera 310 focuses on the door to the bank safe 24 hours a day, but only captures a relatively small number of people entering the safe or simply walking near the door of the safe. This is easily contrasted with the case of motion pictures or frames from video cameras operated in very crowded areas.

  An exemplary feature of the present invention uses environmental monitoring that does not exhibit a large number of activities over an extended monitoring period. Rather than recording all recorded surveillance data, another feature of the present invention reduces storage by reducing the resolution of the recorded image of the data frame corresponding to no detected motion.

  A frame of digital image data typically consists of pixels having a RGB display of, for example, 16, 24, or 32 bits, respectively. Specific to save data storage space occupied by filming those environments that show no activity in the extended period after a predetermined time of storing full size frames during periods when no motion is observed The frame resolution is increased as the number of pixels used to represent the frame is increased, and the stored frame resolution is reduced to some percentage, such as a quarter of the full size frame size. A small frame size is used until a frame with motion appears. Thereafter, the stored frame size is increased to a larger frame size. It should be understood that the lower the percentage, the greater the reduction in storage space typically required for data recording. Although resolutions below or above 25% can be stored, this amount has been found and understood to be a good compromise between maintaining image clarity and reducing stored data. Is a conflicting request.

  4A-4D illustrate the various operations necessary to perform reduced resolution frame storage. In FIGS. 4A-D, if an exemplary frame storage size of 640 × 480 pixels (before any compression is performed) is used and no difference is observed indicating the motion or activity occurring in the environment or region, It has a reduced resolution frame storage size of 320 × 240 pixels (before any compression is performed). Preferably, the computing device 320 performs a frame-by-frame comparison of a particular camera of the camera 310. It will be appreciated that even if the periodic noise pattern is removed, there will be a difference between the images if the actual recorded scene is the same. Thus, differences between frames that exceed some predetermined threshold, such as an allowed loss of 3-5%, are used to indicate the introduction of motion into the scene. Note that the predetermined threshold between adjacent frames containing motion is exceeded because a new object contained in the frame significantly changes certain bits in the frame. Furthermore, if the comparison operation operates at the full resolution frame size and there is no motion between adjacent frames, the reduced frame size is preferably stored.

  Whether or not the adjacent frame is within the threshold is determined using pattern recognition technology, preferably using the pattern recognition technology described in the aforementioned US Patent Application No. 042503/0259665. Can be determined. Usually, particularly in FIGS. 4A-4D, the reference frame is initially set to the start frame of the sequence of frames, and the target frame is initially set to the next frame of the sequence of frames. Once the reference and target frames are compared to each other, the next frame that was the target frame is designated again as a new reference frame, and another next frame following the next frame is designated again as a new target frame. The The process is preferably repeated with each successive frame in turn.

  It should be noted that according to the preferred embodiment, the recording device 310 is fixed in position and does not zoom and always records the same background scene. Therefore, the processing can be simplified from the state where the recording device 310 is not fixed. If not fixed, an inactive reference frame 414 cannot be obtained and sequential comparisons of the frames are required. However, since a sequential comparison of frames has already been obtained, if compression and the reduction in frame size shown here are used, the comparison is not always performed using the same non-operational reference frame. Note that it can be used.

  In FIG. 4A, a 640 × 480 reference frame 402 of digital data previously recorded as a 640 × 480 size frame capturing scene A is compared to the next 640 × 480 target frame 404 of digital data. The As shown, this next frame includes scene B, which is different from scene A, thus causing an activity or movement that causes a difference between frames 402 and 404, exceeding a predetermined threshold. Let Since the predetermined threshold is exceeded, scene B is recorded at a larger 640 × 480 size. The next frame 406 is recorded in a larger 640 × 480 frame size until the predetermined threshold is not exceeded in several time windows, typically 200-300 frames with no activity.

  In FIG. 4B, the digital data 640 × 480 reference frame 408 representing scene A previously recorded as a 320 × 480 reduced frame is compared to the digital data 640 × 480 target frame 410. This captures a subsequent scene A that falls within a predetermined threshold. Since subsequent scene A falls within a predetermined threshold, it is also recorded as a reduced 320 × 480 frame size, indicating that no identified activity or action has occurred. Preferably, the next frame 412 continues to be dimensioned to a small 320 × 240 frame size until a difference between frames exceeding a predetermined threshold is recognized.

  In FIG. 4C, the 640 × 480 reference frame 414 of the digital data recorded at 640 × 480 of scene A is compared with the next 640 × 480 target frame 416 of the digital data, which is a predetermined threshold value. The next scene A, which is different by a smaller value, is captured. Since the first scene A and the next scene A are within the threshold, it is concluded that no identified activity or action has occurred. Accordingly, the recorded frame size is adjusted to a small 320 × 240 frame size once the above time window has elapsed. If the time field has not elapsed, the next scene A is stored as a 640 × 480 frame, but the counter corresponding to the time window is incremented. The next frame 418 that is still within the predetermined threshold even after the time window has been exceeded is thus dimensioned to a small 320 × 240 frame size until a difference that exceeds the predetermined threshold is recognized. The

  In FIG. 4D, a 640 × 480 reference frame 420 of digital data capturing the scene A is recorded at 320 × 240. This reference frame is compared to a 640 × 480 target frame 422 of digital data, which captures the next frame of scene B, which differs from scene A by a value greater than a predetermined threshold, and frame 420 Indicates that an activity or action that produces a difference between 422 and 422 is occurring. Since the predetermined threshold is exceeded, the subsequent frame size is adjusted to a large 640 × 480 frame size. Preferably, the next frame 406 is dimensioned with a large 640 × 480 frame size until the predetermined threshold is no longer exceeded, and the time window elapses.

  In a variation of the previous embodiment, if the last recorded frame was recorded with a smaller 320 × 240 frame size, the comparison operation would instead replace two 320 × 240 frames instead of comparing two different 640 × 480 frames. Compare 240 frames, which reduces the number of comparison operations required and, if a predetermined threshold is exceeded, a frame of size 640 × 480 that is obtained but not required by the comparison operations The whole is memorized.

  Other variations are within the scope of the present invention. For example, the order in which the steps are performed can be varied.

  Furthermore, the detected periodic noise can be used for other purposes. For example, in a typical configuration, the camera, amplifier, etc. are all turned on and used continuously 24 hours a day. As a result, they tend to operate in a stable manner, so the periodic noise pattern can be reduced. However, if the camera, amplifier or other component begins to move away from its stable operating characteristics, a new periodic noise pattern that is different from the initially obtained periodic noise pattern will drift. As a result, watermarks that are often passed are different as described above. When this occurs, the difference changes as described above. Although this change indicates a suspicious condition, this may also cause one of the components, such as a camera or amplifier, to fail in the near future, as the early indicator that the device will fail is an unstable operation and therefore causes drift. Is shown. Thus, the present invention can be used as an early warning system that can indicate that a particular device will soon fail completely. If it is discovered that a particular device is unstable and needs to be replaced, the new device will produce a different periodic noise pattern, so the initial setup needs to be performed again as described above. Note that.

  Furthermore, a further reduction in the size of the stored frame can be made. One example is that some predetermined percentage of consecutive frames are generally black, such as 98% overall, indicating that the illumination is extinguished and there is no detectable image. In such a state, it may be further reduced to 1/8 of the original frame size.

  One pattern recognition embodiment discloses a method for detecting the occurrence of an event that exists within a stored sequence of frames for a scene, and describes a method that operates on data about a scene once an event is detected. .

  The event detection method compares the first stored frame with a later stored frame to determine if there is a size change between frames that is greater than a predetermined threshold. Is included. In the preferred embodiment, the first and later stored frames are compressed and operate with compression to determine the occurrence of the event.

  A method of operating on data relating to scene 1 in which an event has been detected includes providing still images, typically at intervals of every 5-10 frames, using at least data from the recording device that detected the event. . In yet another embodiment, still images from other recording devices associated with the recording device that detected the event in a predetermined manner are also obtained at certain intervals.

  A method and apparatus for detecting and responding to changes in the field of view of a video camera whose output is digitized and typically compressed is described. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced with various compressed data systems in which the occurrence of events is indicated by the size of the compressed data. In other instances, well-known acts, steps, functions, or elements have not been shown in order to avoid obscuring the present invention.

  A portion of the description is made using terminology commonly used by those skilled in the art to convey the content of those operations such as compression, pattern recognition, frames, images, fields of view, etc. to those skilled in the art. The various operations are described as a number of discrete steps that are performed in the most convenient way to understand the invention. However, the order of description should not be construed to imply that these operations are necessarily performed in the order in which they are given or dependent. Finally, repeated use of “one embodiment”, “another embodiment”, or “alternative embodiment” may refer to the same embodiment, but not necessarily the same embodiment. .

  FIG. 7 shows an exemplary system 700 in accordance with the present invention, which has a computer 720 that compresses and operates digitized data using the features of the present invention described herein. Is shown as being. The computer 720 also operates to compress the digitized data and send it to another device, shown as the server 730, so that the server 730 takes advantage of the inventive features described herein. Uses and operates on digitized data. The compression is done by computer 730, but in practice this is not efficient. A plurality of computers 720 are shown as providing digitized data to server 730, a feature that various associated digital data streams are operated in accordance with one embodiment of the present invention, as described below. It is shown to further illustrate the manner in which

  The foregoing system illustrates one way in which the advantages of the invention described herein can be used, but this is exemplary. For example, computers 720 and 730 can be configured as a network of computers or as a device other than a computer that includes a processor capable of carrying out the invention described herein. Many other variations are possible. A device such as that described typically includes several types of processors, such as an Intel Pentium 4 microprocessor or DSP, with program instructions written based on considerations herein, but field programmable gates. Other hardware that implements the present invention, such as an array, can also be used. The program instructions are preferably written in C ++ or some other computer programming language. In repetitive routines used repeatedly, these can be written in assembler for fast processing. As mentioned above, the present invention preferably operates on data formatted into a matrix array within a frame, as further described below. For data that is not temporally related, blocks can be formatted into frames that may or may not have the ability to store the same amount of data. For temporally related data, such as image and audio data, each image and its associated audio data preferably has its own frame, but the image and audio data are stored in separate tracks and analyzed independently. It is not necessary because it can be done.

  According to one embodiment, it is assumed that the computer 720 or server 730 is receiving a digital image / audio frame associated with a sequence, which is digitized into frames and compressed in several ways. These frames are basically compressed and operated in real time or compressed and transmitted for storage at another location. In addition, there exists a situation when compression is not necessary as if pattern recognition between frames is performed substantially during the real-time operation period of the frame.

  In some examples, the data obtained corresponds to locations where monitored and stored scenes often do not change. For example, the camera is placed outside the safe or on the stairwell door, records the scene, and is then received by the computer 700. Only when someone enters the safe or crosses the stairwell doors, the image frame changes substantially (even at successive frames recording the same scene, the change in the data representing the frame is at least noise. Because it exists for the effect).

  In the image frame corresponding to the scene to be compressed, it is desirable to be able to operate further on the compressed image frame, for example to determine whether an event has occurred in the image frame. For example, it may be desirable to be able to detect that an image frame has changed due to a change in field of view (eg, someone has crossed the field of view) without having to decompress the image. The present invention provides a mechanism for detecting that a change is occurring in the field of view without having to decompress the image frame, and also provides several mechanisms for responding to the detection of the change.

  FIG. 8 shows a compressed frame being generated by a camera having an object passing through the field of view. The camera 802 is directed to the stairwell doorway 806, and computers (not shown) attached to the camera 802, such as the computer 720 of FIG. 7, are digitized and compressed frames 808a-n, 810a-n. 812a-n and 814a-n are generated. The compression was filed on Oct. 31, 2001, assigned to the assignee of the present invention, and assigned to US patent application No. 042503/0259665 (title of the invention “Method And Apparatus For Determining Patterns Within Adjacent Blocks Of Data ”) and US patent application Ser. No. 09 / 727,096 filed Nov. 29, 2000, assigned to the assignee of the present invention (invention name“ Method And Apparatus For Encoding Information Using Multiple Passes And Decoding In ”). A Single Pass ”), both of which are hereby incorporated by reference. Note that when using these techniques, particularly pattern recognition, rotation of the search block and diagonal polygon traversal are typically not needed, especially when the camera is fixed in position.

  Frames 808a-n, 810a-n, 812a-n, 814a-n are basically operated and stored in real time, compressed, then operated, compressed, stored, then operated, or at another location Even if compressed, transmitted, stored and operated, the present invention described herein is equally applicable. In still scenes with a sequence of frames, the same number of bits are used to represent the frame, so the decompressed frame may or may not be compressed, or may or may not have motion or motion Have a relatively constant dimension. However, once compressed, the amount of data required to represent each frame in the same scene is substantially less than the same scene with other objects superimposed on it, and the compressed scene Are compressed to approximately the same dimensions, but there are some non-uniform positives due to noise and other factors. Thus, as shown in FIG. 8, the compressed frame of 812a-n is larger than the compressed frames 808a-n and 810a-n because person 804 enters doorway 806. When the person 804 exits the field of view of the camera 802, the compressed frame size returns to the previous relatively small size as indicated by frames 814a-n.

  FIG. 9A shows a process 900 for detecting and responding to changes in the dimensions of a compressed frame obtained from the same source. It should be appreciated that the present invention is not limited to compressed video frames. Although the foregoing frame is described in the context of video data, a frame is audio data or a mixture of audio and video data, or another form of data that provides an indication of the occurrence of an event. According to one embodiment, the process 900 compresses from the camera's field of view having a substantial activity-free start period of the field of view followed by an activity period caused by objects in the field of view that result in a change in the size of the compressed video frame. Works with selected video frames. Each frame is preferably stored in compressed form as a block of data having a size associated with it. In process 900, each frame determines whether a frame to be further processed in step 920 follows. If no, the process 900 stops at step 922. If yes, in another frame, the dimensions of adjacent compressed frames are compared in step 902. If at step 904 the next frame is greater than the previous frame by a threshold, a response follows at step 905 to indicate the detection of the occurrence of the event.

  FIG. 9B illustrates some possible examples of actions that can be performed in response to detect the occurrence of an event in accordance with one embodiment of the present invention. In particular, as shown in FIG. 9B, one or more of the three possible operations 906, 910, 916 can be performed as a result of an event that occurs, as determined by step 905.

  If there is an event to be determined, operation 906 can be performed, which causes a sequence of still images obtained at some interval from each other following the start of the event. These still frames represent a significant decrease from the total number of frames that a particular camera acquires. By spacing every fourth to tenth frame, preferably every sixth frame, one of the resulting still images is the “best field of view of the object that increases the size of the compressed frame. "including. With each obtained still frame, an operation such as transmission of each still image to a predetermined position by e-mail or other transmission mechanism shown in step 908 can be executed. If desired, image recognition can be performed for each still image as indicated by step 909, and a still frame is performed and shown at step 910 described in detail below. Indicates that it should be processed with external pattern recognition of objects such as people or objects.

  The external pattern recognition in step 910 aims to look for previously obtained external patterns in either compressed or decompressed data representing frames. This pattern recognition is shown to be initiated by step 905 after the compressed frame has been operated, and external pattern recognition need not be performed on frames of approximately the same size indicating that no operation has been performed. So this is preferred with the previous compressed data. For uncompressed data frames, external pattern recognition is described in the aforementioned US patent application filed October 31, 2001 (Title “Method And Apparatus For Determining Patterns Within Adjacent Blocks Of Data”). Can be performed on the decompressed data being retrieved to determine whether a significant external pattern is present there, and at least one separate thread is preferred Starts with each different outer pattern to be searched and is used to obtain a search block in which the outer pattern is searched in the target frame instead of the reference frame used to obtain the search block. Of course, other conventional pattern recognition techniques can be used.

  The external pattern in question is preferably included in both the uncompressed and compressed file tables, and which file is used depends on whether the pattern recognition is uncompressed or compressed data, respectively. According to what is done. The compressed object in question is stored using the same compression technique used to obtain the compression of the frame, thus allowing more efficient pattern recognition.

  If a match is found to exist as a result of the external pattern match, the match indication causes several types of alerts to be generated at step 914, as shown in step 912. This can be generated at computer 720 or server 730. An alert is, for example, an indication on the security guard or other authorities' monitor that indicates the identity of an individual or event location identified by external pattern recognition, or to make someone face a description of a location. This is an audible alarm to the wireless security guard. An example of an alarm is described with reference to FIG. 10 below.

  Additionally or alternatively, when the security guard observing the camera field of view on the monitor navigates in front of the camera and appears on the monitor, as shown in step 916, the process 900 Allows to tag or mark an object / person causing a change in the size of the frame to allow easy tracking of the person. Different shapes and colors of tags can be used to distinguish between different levels of security to be applied to each object. For example, one shape object such as a triangle can be used to show an external pattern with a high priority while another shape object such as a circle has an external pattern with a low priority. Can be used to show. Similarly, or in combination with the shape used, one color such as red can be used to indicate a different high priority and another color such as green indicates a different low priority. Can be used for.

  Once the object in question is located, tagging of that object can be used to cause the marker to appear adjacent to the image. Of course, this feature can be switched off if desired. FIG. 10 illustrates in detail a technique that facilitates tracing an image of an object, such as a person, between groups of objects displayed on the monitor by marking the image of the object on the monitor with a tag or mark. Yes. FIG. 10 shows four screen displays 1002, 1004, 1006, 1008 in which the object has changed in frame size. In the screen display 1004, the tag 1001a is attached to a human image 1001. When the person image 1001 moves from the display 1004 to the display 1006 and then 1008, the tag 1001a facilitates viewing the location on the display where the person image 1001 is located.

  The displays 1002, 1004, 1006, 1008 also show alarm areas in the corners of the display. As shown with FIG. 9B, according to one embodiment, the visible alarm generates an indication that a match has occurred in the external pattern recognition process 910, and the matching name of both, in this case the person's Describes the name and the identity and location of the identified object that occurred and thus identified where it occurred.

  FIG. 12 illustrates a video surveillance system according to one embodiment of the present invention. This system is similar to that shown in FIG. 11C, except that in addition to camera position control, the camera signal is supplied to the monitor, the controller also manages information in a digital storage device. To do.

  Assuming that the digital storage device is a hard drive system, a number of techniques are known for storing data therein. For illustrative purposes, all data units such as files stored on the hard drive, for example, the size of each file, the creation date, the date of the last access, the sector from which the data unit starts to be stored (or appropriate) Suppose we remember a table listing other units). Each segment of the data unit includes a link to the next sector of the data unit. Perhaps it includes a linkback to the previous sector. The final sector of the data unit points to a zero value as the next sector. When linkback is included, the linkback of the first sector is similarly directed to a zero value.

The controller receives the image stored on the hard disk drive and assumes that the disk drive is full or otherwise has a smaller amount of free space than is required to store the image data. Some data must be erased to make room for new image data. One embodiment of the invention scores individual data units based on priority and age, with low priority, old data; low priority, new data; high priority, new data; high priority Data unit is selected for erasing in order of old data; low priority. In other words, this image table entry is associated with an image priority parameter of 1 or 0, 1 being a high priority, 0 being a low priority, and an image age parameter to an age scale from 0 to 255. Assuming that 0 is old and 255 is new, the controller can construct the data unit score as follows:
Score = 256 * priority + oldness This is from 0 (low priority, old data) to 255 (low priority, new data) and 256 (high priority, old data) to 511 (high priority, new data) Provide a score that is in the range of. This groups the data units into four non-overlapping groups: high priority, new data; high priority, old data; low priority, new data; low priority, old data, in decreasing order To do. The controller then selects a sufficiently low scoring data unit to erase based on the file size associated with the image, so that there is enough room for new data. The controller can then instruct the hard disk drive to erase the selected file and store new data there.

In this desired order, ie, high priority, new data; high priority, old data; low priority, new data; low priority, old data may hold image data having a high priority. Effective in the most important situations. Other configurations may be used in other states, such as scores.
Score = 256 * priority + age 1 is high priority, 0 is low priority, 0-255 age scale is associated with the image, 0 is new, 255 is old, low priority, New data; low priority, old data; high priority, new data; high priority, generate a desirability score in the order of old data, the last being most desirable. This ordering is useful, for example, in data archiving situations when old data is more important than new data. Instead, the score is
Score = 256 * priority + age, where 1 is low priority, 0 is high priority, 0-255 age scale is associated with the image, 0 is new, 255 is old, low priority Old data; low priority, new data; high priority, old data; high priority, order of decreasing desirability of new data.

Furthermore, the score is
If score = 256 * priority + age, 1 is low priority, 0 is high priority, age scale is 0-255, 0 is old, and 255 is new, low priority, New data; low priority, old data; high priority, new data; high priority, order that reduces the desirability of old data.

  Other configurations are possible. For example, instead of the age parameter indicating the age of creation of the image file as described above, the time since the last access of the image can be represented instead.

Furthermore, both the age of creation and the age of access can be used. In addition, other parameters can be used. For example,
Score = 512 * priority + 256 * subject + stale score is used when the subject is 1 in the safe, 0 in the stairwell, the priority is 1 for the high priority, 0 for the low priority The oldness is 0 when old, and 255 when new. This is the most desirable to undesired order: high priority, safe, new; high priority, safe, old; high priority, stairwell, new; high priority Low priority, safe and new; Low priority, safe and old; Low priority, stairwell, new; Low priority, stairwell, Order scores in chronological order. This scoring system appreciates images from safe cameras rather than images from stairwells.

  Other numbering systems are of course possible. Further, data units that are potentially subject to erasure need not be limited to data units that are already stored, but may include units intended to be additionally stored. In that case, the new data unit is instructed to be erased, in which case it is not necessary to erase the stored information. Also, instead of the two value parameters (0 and 1), the system may use more than two value parameters. For example, the priority parameter may have a high, medium, low value, or a range such as 0-10, with 0 being the highest priority and 10 being the lowest priority.

  FIG. 13 illustrates a video surveillance system according to one embodiment of the present invention. In addition to controlling the position of the camera and providing the camera signal to the monitor, the system monitors the image produced by the camera under certain conditions as specified by rules set by the video surveillance system operator. , Similar to the system shown in FIG. 11C, except that when one of these conditions is met, an alarm, also called an alarm, is generated. Alternatively, the monitoring program need not be in the control unit, but may be separate and may monitor the image of the digital storage device after storage.

  Regardless of where the monitoring takes place, the base of the monitoring program is pattern recognition of image features. Typically pattern recognition, as used herein, can identify a person based on a facial shot of an image. In addition, the pattern recognition system can analyze objects such as wallets, briefcases, individual cards, and wagers. The degree of analysis is of course based on a number of factors, as is known. All such items that are pattern recognition objects are sometimes referred to as entries in the following description and claims.

  Pattern recognition is based on a single image, eg “If the guard is in a safe shot, please notify the system operator” or a number of images, eg “Jon Doe and John Smith (2 are bank robbers) Can be based on “notify the system operator and start the search for Joe Smith” if only the user is in the lobby shot at time T1 and only John Doe is in the lobby shot at a later time T2.

The basic pattern recognition expression is
If P1, P2 and P3 are ... Q1, Q2 and Q3 are ... (1)
If P1 or P2 or P3 is ... Q1, Q2 and Q3 are ... (2)
For example, in a casino setting, the monitor program has the following rules.
If (the image shows a person on the list of known card counters), (3)
Then (notify the system operator)
When there is one P and one Q. Instead, rules are connected.
If (the image at time T1 shows card counter A) and (the image at time T1 shows card counter B) and (the image at time T2 shows card counter A) and (the image at time T2 shows card counter B (Not shown) (notify system operator).

And (notify casino security) (4)
Instead, the rules are disjunctive as follows:
If (the image shows a game dealer in the cashier area) or (the image shows a game dealer in the safe area) (notify the system operator) (5)
Some other alert generation recognition rules that may occur in these situations include:
(At time T1, object or person A is present in the first image shot) and (At time T1, object or person B is present in the first image shot) and (At time T2, object or person A is second (Is present in the image shot) and (the object or person B is not present in the second image shot at time T2) (generates an alarm) (6)
(At time T1, object A is present in the first image shot) and (At time T1, object B is not present in the first image shot) and (At time T1, object A is present in the second image shot) and (At time T2, object B exists in the second image shot) (Alarm is generated) (7)
One example of this rule is at various airport security check-in locations. At the first entry position, the person A has taken a picture that does not carry the object. In another location, such as the entrance passage to the airport, another picture shows person A with object B, which can be used to generate an alarm and shows the changed condition.

If (the object or person A is present in the image shot where the object or person is not allowed to enter) (generate an alarm) (8)
An example of this rule is
(If a 100 chip bet exists in an area other than the high betting area) (Alarm is generated) (9)
Or
If (the person working at the kitchen is in an area other than the kitchen), (generates an alarm)
(10)
If (person or object A is present in the image of the scene) and (person or object A is present in the list of people or objects of a certain type) and (person or object B is a list of people or objects of a certain type (Alarm) and (search for person or object B in the image) (11)
With respect to the previous rule (11), rule modifications are also made to include other or false information about a particular person or object. Thus, the group information can also include alternative or fake information. That is, if there is a picture of the same person, one with a heel or sunglasses and one with no heels or sunglasses, both pictures are associated with that person when searching for that person. Can be explored. The same alternative information can also be stored for the object. For example, changing the purpose of this rule, which uses only one person's identity,
If (person A is in the scene image) and (person A is on the list of known or suspected terrorists) and (person B is also on the list of known or suspected terrorists) (Search for person B in the image) (12)
This pattern recognition process can be performed on the video surveillance system image at a single time. Instead, this can be done on a regular or continuous basis. Furthermore, rules can have time limits. For example, the rule is that if person A is recognized in the image, the system will search for person B in the image for 15 minutes, and if person B is found within that time, specify that an action be taken Is done.

  FIG. 14 illustrates a data collection system according to one embodiment of the present invention. The system 1400 includes player location settings 1410a-g, dealer settings 1412, a camera 1414, a computer 1415, a network 1416, and a terminal 1417. Camera 1414 shoots table 1400, player location settings 1410a-g, and dealer settings 1412 to obtain a digital data stream that includes the repetitive motion that occurs. The repetitive action is the activity that occurs at the location setting 1410a-g and the dealer setting 1412. The camera 1414 is preferably fixed and is preferably set to the same zoom position for all comparison operations performed as described herein, thereby obtaining as much consistency as possible between adjacent frames of the digital data stream. It is done.

  As shown in FIG. 15B, the player location settings 1410a-g have a betting area 1502 and a play area 1504. During a card game, such as blackjack, for example, a player chips or A wager such as jet-on is placed, and a card of the player's hand is placed in the play area 1504. As the card game progresses and the cards added to one or more players are dispensed, the activity takes place in play area 1504, possibly betting area 1502. Similarly, the dealer's hand is placed in the dealer's hand area 1506. As the card game progresses, one or more cards are added to the dealer's hand area 1506. FIG. 16 shows a sequence of repetitive actions possible in a game performed in accordance with an embodiment of the present invention. At 1602a, each player's location setting 1410a-g has an arbitrary card and wager removed, and at 1602b the dealer's setting 1412 has an arbitrary card removed. When the card game progresses from 1602a to 1612a and progresses from 1602b to 1612b, the card is added to the player area 1504 and the dealer's hand area 1506, and the wager is placed in the betting area 1502. A sequence similar to the sequence shown in FIG. 16 may be performed at other player location settings, indicating that the sequence of repetitive actions 1602a-312a occurs at one of the player location settings 1410a-g. it can.

  By taking the data stream appearing from the camera and analyzing it with the computer 1415 to determine the resulting displacement, the number of players playing at the table can be determined. This is possible through the use of a mask according to one embodiment of the invention described further below to detect transitions between competitors (ie, the end of a competitor or the start of a new player). According to one embodiment, the mask indicates a place setting where no cards and wagers are present. However, other indications of competitor start or competitor end can be used. FIG. 15A shows a mask of the gaming table according to one embodiment of the present invention. Mask 1500 includes a mask for player location settings 1510a-g and a mask for dealer position settings 1512.

  The computer 1415 stores the mask 1500 and uses it to detect displacement between competitors. FIG. 17 shows the mask for setting the player's location and the dealer's position in more detail. Mask 1702 is for a player location setting 1510a-g where no cards and stakes are present, and mask 1704 is for a dealer setting 1512 where no cards are present. By comparing each repetitive action on the corresponding mask at computer 1415, it can be determined whether the competitor is ending and / or a new player is about to start. Between competitors, successive frames representing masks are not counted as separate competitors.

  For example, by comparing the mask 1702 with 1602a, it is clear that the game is about to begin. If the game continues and the data stream from the camera is analyzed and the mask is compared to the next frame, 1610a indicates that the game is complete only after masks 1702 and 1610a are identical. At that point, a counter that tracks the game playing at table 1400 can be incremented. The comparison process continues with iterative actions 1612a and 1612b.

  Technically, the pattern comparison described above requires that a pattern matching operation be performed between the mask 1702 and the portion of the digital data stream corresponding to the position of the mask 1702 during the play of the chance game. In such a comparison operation, the mask 1702 is an external pattern that is basically searched at a specific position of each frame of a digital data stream representing an image. A typical pattern recognition system operates on a digital data stream and can be used to obtain an indication of the mask 1702 that is within the resulting digital data stream. In addition, the mask area is further required to recognize at least the objects that are important to it, such as cards or chips, so that erroneous objects such as players appearing in the mask area may be in progress or completed. To prevent it from being shown incorrectly.

  While the described mask is a preferred comparison method for pattern recognition purposes, it is not the only method by which comparisons can be made. Comparisons between frames can also be made so that consecutive periods of activity can generate counts. For example, a white space in a dealer card area that exists longer than a predetermined time period is used to generate a count, and another count places the card there in another predetermined period. Not generated until later.

  The above description was in the context of a single player playing with a dealer. It should be appreciated that more than one player can play at the same time as the dealer. If a large number of players play at the table and the player's hand ends before the other player's hand, the winning player's game is detected when the player clears the player's betting and play areas . Each time the end of a player's game is detected, a game counter (not shown, a software register) can be incremented by the computer 1415. Information about the game played at the gaming table and other information that can be gathered in accordance with the present invention is provided to terminal 1417 or other terminal (not shown) via network 1416. Network 1416 may be the Internet, another distributed Internet network, or a dedicated network.

  FIG. 18A shows a roulette layout. Layout 1800 is divided into 180 areas for betting. The basic area of the layout 1800 is the red and black alternating areas numbered 1-36 and the green digits 0 and 00. The remaining areas are basic areas: even number, odd number, red number, black number, first 12 number, second 12 number, third 12 number, first 18 number, last number This is a permutation of the 18th area. You can bet on any single number (straight up), number combination, red, black, odd or even. Each player from 1 to 6 players in the roulette table is given a chip of a different color, thereby allowing tracking of the numbers on the layout betting using a reference to the color.

  FIG. 18B shows a roulette wheel. The wheel 1810 is divided into 38 slots 512 on which the balls land and are numbered 1 to 36, 0 and 00. Each roulette game begins when the dealer rotates the wheel in one direction, and then rolls a small ball along the inner edge 1814 of the wheel 1810 in the opposite direction. The ball then falls into one of the numbered slots 1812. That number declares the winner of the game.

  FIG. 18C shows a mask 1820 for a roulette wheel that is as simple as tracking the slot area 1812 where the ball has rolled. The mask 1820 is stored in a computer like the computer 1415 in FIG. 14, and is used to detect displacement between roulette games. During a single game, the ball rolls, the wheel is rotated, and the ball hits one of the slots 1812. A camera, such as camera 1414, is positioned to observe wheel 1810 and is used to capture the repetitive motion of the roulette wheel and ball. In particular, each time the ball rolls into a slot, this indicates that the game has been completed and can be recorded as a repetitive sequence. That camera or another camera can also be used to capture the repetitive movement of the chips played on the table, each separate betting area has its own mask area and uses the techniques described above. And recurring activity queries. Similarly, the action of taking a chip by the dealer due to losing a bet and the action of giving a chip from the dealer to the winner can be used to count the number of games played in a given time period. Activity.

  FIG. 19 shows a sequence of repetitive movements of the roulette wheel and the ball. By comparing mask 1820 with repetitive actions 1902-1908 with a computer such as computer 1415, it can be determined that two games will be played. When the ball enters any slot 1812, it can be used as an indication that the game has ended, and this action is indicated by actions 1902 and 1908, so a sequence of four frames (no other frames in between are shown) Is known. Instead, each time the ball appears at the inner edge 1814 of the wheel 1810 can be used to indicate that a new game is being played. By maintaining time tracking, the efficiency of the roulette dealer can be tracked. In addition, by tracking both the mask 1820 and the mask associated with each separate betting area, the winner declared at the table becomes the actual winner as determined by the slot 1812 of the number that the ball actually landed. Corresponding can be determined.

  The present invention can be adapted to other repetitive games such as poker, 3 card poker, pai gou, caribbean stud, baccarat and other games.

  In addition, reports can be generated based on statistics obtained by the present invention. By tracking a particular dealer in each table during a predetermined period, the number of matches processed during that period is obtained. By combining periods for a particular dealer, the average efficiency of that dealer can be determined. In addition, statistics can be maintained based on, for example, table position, thereby determining which table is the most crowded at various time periods, which is the most efficient, for example, the most crowded table It is possible to make the dealer work. FIGS. 20A and 20B show two different reports regarding dealers and table positions, respectively showing statistics obtained from a single shift over a predetermined period of time. Since it is possible to confirm that payment to the actual winner has been made, security can be added.

  Other repetitive activities can also be monitored using the techniques of the present invention.

  For example, in casinos, money is always counted in the same way, and in the US, money is typically spread to one table in exactly the same way in increments of $ 20,000. Each movement can be tracked if desired, so that at each position where the next amount is spread, it is tracked, and then the amount is tracked there at the next predicted position. Is done. If there is a deviation from this, this is also time-based and an alarm can be triggered.

  In another example, the camera at the entrance may be used to track the time period when the laundry cart is in front of a special room, using a mask containing a photo of the room without the cart in front. it can. When an object appears for a time period longer than 3 minutes, it can be interpreted as a cart. The time period during which the object is moved from the scene can be used to manage the amount of time that the cart was in front of the room, thus obtaining an estimate of the time required to clean the room.

  In yet another example, iterative operation of dealer payment of money can be used to count the amount paid. Typically, since the camera is above the table, a perspective image of the rack containing the chips used for payment cannot be obtained. However, since it is typical to put silver coins between every 5 chips, each time a silver coin found in the area corresponding to a particular row of chips to be paid appears 5 chips x chips paid Can be used to evaluate the amount of money. Thus, by counting instances recognizing silver coins in the area corresponding to the row of chips, an overall assessment of the amount paid is obtained. Thus, the iterative operation is looking for instances where silver appears in the mask region corresponding to that chip row.

  Of course, other repetitive activities can be automatically monitored using the techniques described herein.

  The foregoing embodiments are provided for purposes of illustration only, and the present invention should not be construed as limited thereby. Variations of the present invention will be readily apparent to those skilled in the art after reading this description, and the present invention and claims are intended to include such variations as well.

1 is a block diagram of a digital video content storage system according to at least one of the embodiments of the present invention. FIG. 4 is a block diagram of software modules used by different processors in accordance with at least one of the embodiments of the invention. 1 is a block diagram illustrating a transmission system according to at least one of the embodiments of the present invention. FIG. 4 illustrates an example size adjustment for a frame based on whether there is motion in the monitored area. FIG. 4 is a flow diagram illustrating an exemplary noise pattern discovery process in accordance with at least one embodiment of the invention. FIG. 4 is a flow diagram illustrating an exemplary noise correction process in accordance with at least one embodiment of the invention. 1 illustrates an example system in accordance with at least one embodiment of the invention. FIG. FIG. 6 shows a compressed frame generated by a camera having an object passing through a field of view. FIG. 4 shows a process for detecting and reacting to changes in the frame size of a source. FIG. 4 illustrates some possible examples of actions that can be performed in response to detect the occurrence of an event in accordance with at least one embodiment of the invention. The figure which shows four screen displays in which the person is changing the dimension of a flame | frame. The block diagram of various video surveillance systems. 1 illustrates a video surveillance system according to one embodiment of the present invention. FIG. 1 illustrates a video surveillance system according to one embodiment of the present invention. FIG. FIG. 10 is a diagram illustrating a gaming table according to at least one embodiment of the present invention. The figure which shows the setting of the place of the player which has a betting area and a play area. The figure which shows the sequence of the repetitive operation | movement which can be performed in the game performed according to embodiment of this invention. The figure which shows the mask of the place setting of the player where a card | curd and a bet do not exist, and the mask of the dealer setting where a card | curd does not exist. The figure which shows the layout of a roulette, the figure of the mask used with a roulette layout, the roulette wheel, and the mask used with a roulette wheel. The figure which shows the sequence of the repetitive operation | movement of a roulette wheel and a ball | bowl. FIG. 4 is a diagram illustrating an exemplary report arising from a repetitive operation being monitored.

Claims (79)

In a method for digital data storage of an image data stream and pattern recognition of at least one external pattern in the image data stream,
Compress the digital image data stream using at least a first thread to obtain a first compressed digital image data stream and at least a second thread to attempt to include a match And pattern recognition using the first and second threads are processed using the processing system,
Send the first compressed digital image data stream to another processing system,
Using at least a third thread to obtain a further compressed digital image data stream, and further compressing the initially compressed digital image data stream to include a match, and at least a fourth thread; Performing pattern recognition of at least one external pattern using a first compressed external pattern corresponding to the external pattern, wherein the third and fourth threads are processed using another processing system Including methods. A video surveillance system used for region monitoring, which preserves data storage by storing each of a plurality of sequential frames of data consisting of a plurality of bits at different resolutions according to received motion instructions In the method
Compare the reference frame against the frame to get the difference,
Determine whether the difference exceeds a predetermined threshold,
If a predetermined threshold is exceeded, operate on a frame at regular resolution, and the predetermined threshold is not exceeded for that frame and a predetermined number of previous frames Works with reduced resolution less than regular resolution,
A storage method in which the comparison, determination, and operation steps are repeated for a plurality of frames following the frame.   The method of claim 2, wherein the predetermined number of previous frames includes a time window.   The method of claim 2, wherein the predetermined number of previous frames is zero.   The method of claim 2, wherein the reduced resolution is less than half of the regular resolution.   3. The method of claim 2, further comprising the step of setting a reference frame relative to a first frame of the plurality of sequential frames prior to the comparing step.   7. The method of claim 6, further comprising the step of setting the target to the next frame of the plurality of sequential frames prior to the comparing step.   Further comprising the step of re-designating the next frame that was the target frame as the reference frame and re-designating another next frame following the next frame as the target frame before the iteration step. Item 8. The method according to Item 7.   The method of claim 2, wherein the operating step compresses the frame using one of regular resolution and reduced resolution.   The method of claim 2, wherein the operating step stores the frame using one of regular resolution and reduced resolution. In a video surveillance system that uses a data transfer device to monitor a region, a method for correcting periodic noise present in a plurality of sequential frames of a plurality of bits,
Install a data delivery device at a location to monitor the area,
Acquire a periodic noise pattern corresponding to the data transfer device,
To obtain multiple sequential frames, monitor the area using a data transfer device,
Removing the periodic noise present in at least one of the plurality of sequential frames to obtain a corrected plurality of sequential frames.   Further, authenticating at least one frame of the plurality of sequential frames by either not removing periodic noise from one frame or reintroducing periodic noise removed from one frame; The method of claim 11 including the step of thereby using periodic noise as the watermark. Further, one frame of the plurality of sequential frames is compared with another frame of the plurality of sequential frames to obtain a difference,
Determine whether the difference exceeds a predetermined threshold,
If a predetermined threshold is exceeded, operate on another frame at regular resolution, and if the predetermined threshold is not exceeded for a predetermined number of previous frames in the frame 12. The method of claim 11 including the step of operating at a reduced resolution that is less than regular resolution.   14. The method of claim 13, wherein the predetermined number of previous frames includes a time window.   14. The method of claim 13, wherein the predetermined number of previous frames is zero.   14. The method of claim 13, wherein the reduced resolution is less than half of the regular resolution.   The method of claim 2, wherein the operating step stores the frame using one of regular resolution and reduced resolution.   The method of claim 17, further comprising storing periodic noise having a plurality of sequential frames. The installing step further includes installing at least one component, and
Determine that the periodic noise has changed,
Replace one of the data delivery device and one component,
The method of claim 11, wherein the stub for obtaining a periodic noise pattern is repeated to obtain a new periodic noise pattern.   12. The method of claim 11, further comprising an initial warm-up period during which the data transfer device is switched on before the monitoring step. The step of obtaining a periodic noise pattern is:
During a certain period, record a test pattern of a predetermined frame,
Obtain the difference between each of the predetermined test patterns of the frames obtained during that period and the reference pattern,
Use the difference to search for periodic noise,
If periodic noise is detected, remember that periodic noise,
If no periodic noise is detected, set the period to a longer period,
The method of claim 11 including the step of repeating the recording, acquiring, using and setting steps until periodic noise is detected. In a video surveillance system with a fixed-position camera, data storage by adjusting the frame storage size stored according to the motion instructions in which individual frames of a plurality of sequential frames each consisting of a plurality of bits are received In a way to maintain
Monitor frames of multiple sequential frames for differences exceeding a predetermined threshold;
When the predetermined threshold is not exceeded, the frame storage size for the first group of the next frame is reduced from a large frame storage size to a small frame storage size;
A method comprising increasing a frame storage size for a second group of next frames from a small frame storage size to a large frame storage size when a predetermined threshold is exceeded.   To determine whether there is a size change greater than a predetermined amount, the first compressed digital frame size and the later compressed digital frame size in a sequence of compressed digital image frames An event occurrence detection method comprising the steps of comparing and using the determination to indicate the occurrence of an event if the resizing is greater than a predetermined amount.   24. The method of claim 23, wherein the event occurrence is the appearance of a new object in a later compressed frame.   25. The method of claim 24, further comprising the step of periodically selecting frames in the sequence of compressed frames after the event occurs and transmitting each selected frame.   26. The method of claim 25, wherein the periodic selection step selects at least one frame for every 10 adjacent frames during the event occurrence.   27. The method of claim 26, further comprising performing external pattern recognition on the selected frame using the external pattern.   27. The method further comprising the step of performing external pattern recognition on a frame that is later compressed using the external pattern and further generating an alert when the external pattern recognition matches the external pattern to a new object. the method of.   30. The method of claim 28, wherein the alert includes a visual alert indicating the identity of the new object.   30. The method of claim 28, wherein the alert comprises an audible alert.   25. The method of claim 24, further comprising positioning the tag on the image of the new object displayed on the monitor so that the tag follows the image on the monitor as the new object moves on the monitor.   24. The first compressed digital frame provides an image of a still scene that does not include movement there, and the later compressed digital frame includes a new object that indicates that motion has occurred in the scene. Method.   35. The method of claim 32, further comprising the step of periodically selecting a frame in the sequence of compressed frames after the event occurs and transmitting each selected frame.   34. The method of claim 33, wherein the periodic selection step selects at least one frame for every ten adjacent frames during the event occurrence. In a system for managing data units stored in a storage device,
Receiving means for receiving a new data unit to be stored in the monitoring means for monitoring the storage state;
Monitoring means for monitoring the status of the storage device and generating an indication whether there is sufficient free space in the storage device to store the new data unit;
Weighting means for weighting the data units stored in the storage device according to a predetermined weighting system when the monitoring means indicates that there is not enough free space in the storage device to store the new data unit;
Erasing means for erasing data units in a weighted order until there is sufficient free space in the storage device to accommodate the new data units.   36. The system of claim 35, wherein the weighting system weights the data units based on priority and age.   The weighting system weights data units with high priority, new data; high priority, old data; low priority, new data; low priority, old data order with respect to retention desire. 36. The system according to 36.   36. The system of claim 35, wherein the weighting system weights the data units based on priority, key items, and age. In a method for managing data units stored in a storage device,
Receiving a new data unit stored in the storage device,
Monitor the status of the storage device and generate an indication whether there is sufficient free space in the storage device to store the new data unit;
When there is not enough free space in the storage device to store the new data unit, the data unit stored in the storage device is weighted according to a pre-determined weighting system, the weighting system regarding the retention desirability Weight the data units in high priority, new data; high priority, old data; low priority, new data; low priority, old data in order
A method comprising erasing data units in a weighted order until there is sufficient free space in the storage device to accommodate the new data units.   40. The method of claim 39, wherein the weighting system weights the data units based on priority and age.   40. The method of claim 39, wherein the weighting system weights the data units based on priority, key items, and age. In systems that generate alarms in video surveillance systems,
An access means for accessing a sequence of digital images;
Pattern recognition means for recognizing entities from at least one digital image from a sequence of digital images;
A rule applying means for applying a rule for at least one entity with respect to entities potentially appearing in the at least one digital image;
Means for generating an alarm in response to at least one digital image satisfying one or more rules.   43. The system of claim 42, wherein the rules include a rule that identifies the occurrence of an alarm when an entity in the at least one digital image is recognized by the pattern recognition means as corresponding to a tracked entity.   A rule is recognized by the pattern recognition means that two entities in the first digital image are tracked entities, and only one of these entities is recognized as being in the next second digital image 43. The system of claim 42, further comprising a rule that identifies the occurrence of an alarm.   43. The rule includes a rule that identifies the occurrence of an alarm when the entity is recognized by the pattern recognition means as being in at least one digital image corresponding to an area that is tracked and assumed not to be an entity. The described system.   A rule is recognized by the pattern recognition means as the first entity is in the first digital image, and recognized by the pattern recognition means as the first and second entities are in the next second digital image. 43. The system of claim 42, further comprising a rule that identifies the occurrence of an alarm.   The rules include rules where the pattern recognition means recognizes a first entity in at least one digital image, and then a search is initiated to look for a second entry in the other digital image. Item 43. The system according to Item 42.   48. The system of claim 47, wherein the rule further identifies the occurrence of an alarm if the second entity is recognized by the pattern recognition means as being in at least one other digital image.   The rule includes a rule in which an alert is identified when the first condition is met at a first time and the second condition is met at a second time different from the first time. 42. The system according to 42. In a method of monitoring a chance game that includes repetitive actions,
Acquire a digital image data stream containing multiple repetitive actions related to the chance game stored there,
A monitoring method that automatically parses the digital image data stream to count multiple repetitive actions, and the resulting count provides an indicator that can be used to monitor the chance game.   51. The method of claim 50, wherein the step of automatically parsing the stream performs pattern recognition at a fixed location.   51. The method of claim 50, wherein the step of obtaining the digital image data stream uses a video camera fixed in a non-zoom position.   51. The method of claim 50, wherein the repetitive actions include dealer actions.   54. The method of claim 53, wherein the dealer's repetitive motion corresponds to a table area where the dealer trades.   51. The method of claim 50, wherein the repetitive actions include player actions.   56. The method of claim 55, wherein the repetitive action of the player corresponds to a table area where the player places a bet.   57. The method of claim 56, wherein the repetitive action of the player corresponds to a table area that the player plays.   51. The method of claim 50, wherein the repetitive actions include both dealer and player actions.   51. The method of claim 50, wherein the repetitive actions include object actions.   60. The method of claim 59, wherein the repetitive movement of the object comprises a marble landing of a wheel slot in a roulette game.   61. The method of claim 60 further comprising the step of comparing the slot where the marble fell to the positioning of the marker on the table position.   62. The method of claim 61, further comprising triggering an event alert if the slot where the marble fell and the table position where the marker is located do not correspond to each other.   61. The method of claim 60, wherein the repetitive motion of the object is that the marble is located at an inner edge portion of the wheel of the roulette game.   51. The method of claim 50, wherein the chance game is a card game.   68. The method of claim 64, wherein the repetitive action is no card at a predetermined position of the table.   The method of claim 64, wherein the repetitive action is the presence of an object at a predetermined position of the table.   The method of claim 64, wherein the card game is one of blackjack, pai go, 3 card poker, and baccarat.   68. The method of claim 64, further comprising the step of comparing the card color against the deck color to help verify that the card is valid.   51. The method of claim 50, wherein the automatic parsing step uses a mask to compare a scene corresponding to a position on the frame with the mask.   69. The method of claim 68, wherein the scene corresponds to an area of the table on which the card is placed.   69. The method of claim 68, wherein the scene corresponds to an area of the table on which bets are placed. Acquire a digital image data stream containing multiple repetitive actions stored there,
A repetitive motion monitoring method comprising automatically parsing a digital image data stream to count a plurality of repetitive motions and the resulting count provides an indicator that can be used to monitor the repetitive motion.   51. The method of claim 50, wherein the step of automatically parsing the stream performs pattern recognition at a fixed position in the frame.   72. The method of claim 71, wherein the digital image data stream includes a plurality of frames, and a portion of each frame is compared to the mask.   72. The method of claim 71, wherein adjacent frames that both include portions corresponding to the mask do not exhibit repetitive motion. Obtaining a digital image data stream containing a repeating sequence that exists for multiple objects;
Automatically parse the digital image data stream to count the repetitive sequences present,
A method of counting a plurality of objects comprising the step of using a counted repeated sequence that exists to evaluate the number of objects.   76. The method of claim 75, wherein the plurality of objects includes a plurality of vertically stacked chips separated by spacers at repeated intervals, and the step of automatically parsing counts each different spacer.   The plurality of objects includes a plurality of object positions, a predetermined amount of value is placed at each object position, and the automatic parsing step is another predetermined value amount at another sequential object position. 76. The method of claim 75, wherein each position of is counted.   79. The method of claim 78, further comprising the step of triggering an event alert if the step of automatically parsing detects the position of the value quantity at a position that does not correspond to the next sequential object position.

Images