DE10261501A1 - Data reduction procedures - Google Patents

Abstract

The invention relates to a method for image sequence data reduction. It is provided here that image area prioritization information is provided, image area prioritization is carried out on the basis of the image area prioritization information, and sufficiently prioritized image areas are transmitted.

Description

The present invention relates to what is claimed in the generic term and is therefore concerned with reduction of data in image data sequences.

If images are to be transferred, kick Problems and a. with regard to the available data rates of the transmission channels or the Extent of available available storage volume, because image data is typically very high extensive, especially when high-resolution images are available. This applies especially to The supervision of objects where it is common is about a longer one Periodically write away image data and after elapse cyclically during this period of, for example, 48 or 72 hours by overwriting older data to begin, if no need has been identified until then, the data to secure. If, however, a relevant event occurs, it lies this in a typically short period of time and then it is desired that indeed reproduce relevant details with high accuracy.

It is therefore common for the Reduce amount of image data by compression. A well known Procedure (JEPG compression) divides the image into a number of areas of equal size ("Tiles") and perform a discrete cosine transformation (DCT) with subsequent lossy ones in each of the areas Run-length coding of the coefficients (Huffmann encoding). By determination and compression of the data to be transmitted Coefficients also determine the quality of the image to be transmitted. At the recipient is for every image tile a reverse transformation with the transferred Coefficients performed and then put the picture back together like a tile. It is in this context, by the way mentioned, that the Terms "sender" and "receiver" and "transmission" only for the purpose of illustration chosen are. It should be noted that data is always sent can also be done locally to a particular unit, such as a hard drive, and that the Receiver for example can be the same PC that was previously sending data to a hard disk saved and now reads again from there to display them. Also in cameras it is possible to perform such a compression and images on internal storage file without deviating from the concept of the terms "transmitter" and "receiver" in the sense used here.

Compression with JPEG allowed a data reduction with at least acceptable leaving the image quality in single images. For moving Other methods are known in image sequences. That's the MPEG process spreads, in which areas of an image unchanged block by block be moved to typical events in moving images like moving objects, people, etc., or panning the camera to meet. Only the block and its are to be specified here Movement in a so-called key picture. It is also possible, for this Coding to isolate several separate image areas, separate them Determine displacement and then reassemble it in the picture. For example, a first movement for a person's head and a second movement for them body be determined. The differently moved objects are then at the receiver again composed. However, use is made in any case from a key frame in which background information is reproduced and which therefore is imperative to reconstruct a particular scenery to be able to. But because the key picture mandatory for the reconstruction is required can not with the display a video stream can be started before not this key frame is present.

A disadvantage of the known methods is among other things that the Coding is often very complex, that a keyframe is necessary, that the complex Coding and waiting for a keyframe transmission a remarkable one delay between those recorded on the transmitter Scene and the one at the receiver reproduced scene and what is special for certain applications what is significant is that important Details that can change from picture to picture are not recognizable, because only an unchanged block is moved in the displayed image. This applies in particular to security systems; like they're monitoring used by banks or certain security areas are. It would be desirable, alleviate at least partially some of the above problems.

The aim of the present invention is new for to provide the commercial application.

The goal is achieved with the in independent Form stress. Preferred embodiments can be found in the subclaims.

The present invention thus proposes in a first basic idea a method for image sequence data reduction in which it is provided that image area prioritization information is provided, image area prioritization based on the image area prioritization information carried out and in particular changing from picture to picture and / or with time, sufficiently prioritized image areas are transmitted.

A first essential aspect of the present invention is that the image data compression is carried out by selection, which is carried out on the basis of specific selection information. This allows only those image areas to be extracted which are particularly important from the aspects used for the selection, which lowers the data rate during the transmission from the outset and, moreover, enables the receiver to easily receive images from the data by prioritization with regard to the Priority information selected image areas can be reconstructed by linking. All that is necessary is to store a corresponding amount of image areas with their respective position within an image and the time at which they were transmitted or recorded. By selecting taking into account the prioritization, a first, massive reduction in the amount of data is achieved.

In a particularly preferred variant on the side of the recipient a memory for that different image areas and then the Images from the most recent, for disposal standing image areas composed. It was on it again noted that in Context of the present application, a recipient can also be a memory can, like a RAM or a hard drive, on which the image data is stored become. Unless otherwise apparent, it should also be understood that that then if from a reduced transfer rate we are talking about the temporal integral of the same reference could be taken that is, the total amount of data in an image sequence.

It is particularly preferred if within the image sequence from image to image, at least for a large number same areas of images are provided. Particularly easy this can be done by tiling of the image can always be achieved with the same tiles. by virtue of the brevity and descriptiveness of this term is often used below used in places where just as well on non-tiled ones Image areas are referenced.

This also allows one more Compression of the data stream using conventional ones Procedures like JPEG compression. It should be mentioned in this context that it is entirely possible different compression ratios for different tiles provided. So for a Refreshing image areas without further change another compression to get voted than with tiles in which big changes in the Picture available. If necessary, this can further increase the transmission rate be lowered.

Various can be used as image area prioritization information Sizes evaluated become. Mentioned be especially for the cyclical refresh of the picture the time since the last transmission; This is also important because it is taken into account the time since the last transmission recent, very strong changes can be used to prioritize an image area. This is advantageous, for example, when a person steps into a picture, which is very much on the edge at first big changes has the consequence. A user will then typically want to know what can be seen after the person has entered the picture. An edge pass will typically result in that he observed the corresponding image area for some time longer. It is further possible, Differences between an image area and a previous image area in a previous image to be determined, in particular the immediately preceding picture, or a middle picture, this mean being weighted can be determined, for example, to simulate a gradual "decay" of a previous image there may also be significant changes in neighboring fields considered become. Here it is particularly possible that such neighboring areas with reduced resolution, so higher Compression of their tiles compared to those areas transferred be where the change indeed has taken place.

It is possible when forming differences Brightness and / or Evaluate color differences in at least one color channel. It is also mentioned that this Taking differences into account of threshold values and / or typical, average changes of all tiles for one current prioritization can be used. When prioritizing can both local and global differences to one immediately or long-term Image can be evaluated. Take global differences into account making changes in the overall picture, such as by switching on lighting occur, consider local rather the actual changes in a picture tile or a differently defined picture area. Regarding the Image tiles should once again be emphasized that obviously overlapping Image areas also for the prioritization can be used and a tiling with Adjacent tiles are not mandatory, but they are advantageous is.

The different influencing factors, as exemplified for preferred forms of the invention have been enumerated linked different ways be, such as by additive linking such that at the same time meets a number of conditions have to be, thus an image area for transmission selected and / or a weighting or a determination can be made by means of Procedures of fuzzy logic are done.

It is particularly preferred if the data stream of the data to be transmitted Image areas Information about it enclosed which indicates the position of the prioritized image areas in the image. Alternatively, it would be possible, this information at the recipient to reconstruct, because even with strong changes in the picture it is still typical those tiles or image areas that can be transferred are identifiable were. The typical overhead due to the transmission of area information is so small, however, that the additional Computational effort is preferably avoided.

That in addition to tiling other types of image area determination such as vectorization, Polygon determination etc., bitmap transmission etc. is possible, be mentioned.

It should be mentioned that the methods can be implemented in video cameras or the like, ge just like in conventional computer systems. Protection is therefore also claimed both for a video camera which is intended and suitable for carrying out one of the methods of the invention and for software which, when loaded onto a device which manipulates image data, such as a computer system, makes it possible to carry out a method according to the invention.

Protection is also sought both regarding Media, which include such software as well as a method of manufacture such a disk.

The invention is hereinafter and described for example with reference to the drawings. In this shows:

1 an arrangement for transmitting data-compressed images from a scene,

2 an illustration of the image area prioritization,

3 an illustration of an image reconstruction at the receiver.

To 1 includes a general with 1 designated arrangement 1 for data compression or reduction during the transmission of images from a sequence, which here is a scene 2 represents at a bank counter, an imager 3 , a downstream image data reduction and transmission unit 4 , in which, as will be described in more detail below, image area prioritization information is provided, image area prioritization is carried out on the basis of the image area prioritization information, and sufficiently prioritized image areas via a transmission line 5 to a recipient 6 are transmitted, which, as will be described, is designed to temporarily store transferred image areas of a plurality of images of a sequence and to link them to display individual images and / or the entire image sequence.

The picture scene 2 in the present example shows a bank counter observed from a given direction 2a with public traffic, represented by one person 2 B , and bank equipment in the picture area, shown here as a plant 2c , It will be appreciated that when monitoring the scene 2 special emphasis on the observation of customer traffic, such as by the person 2 B symbolizes, and less weight on the furniture, as by the switch 2a and the plant 2c symbolizes to be laid.

The imager 3 is designed in the present case as a camera with a CCD field, which images from the in 2a (I) - 2a (V) shown perspective of the scene 2 supplies. As can be seen, this perspective is unchangeable, so the camera is permanently mounted; However, this is not absolutely the case, but was chosen only for the sake of easier explanation.

In the image data reduction and transmission unit 4 is an entrance 4a for the image signals from the camera 3 intended. It is internal storage 4b with the entrance 4a connected, which is designed to initially record several images completely and repeatedly in whole or in part in a prioritization level 4c issue. The memory 4b has storage capacity for at least two images. The prioritization level 4c is designed to subdivide individual images into several image areas and to prioritize the respective areas in one image, the prioritization level 4c for receiving prioritization information from a prioritization information store 4d is trained. The prioritization level 4c is at an output with an image area compression and transfer stage 4e linked, which is designed to transmit data to image areas in compressed form on the transmission line 5 to dine.

The transmission line 5 is limited in its bandwidth, in such a way that image sequences cannot be transmitted permanently and completely uncompressed.

The recipient 6 points behind the outlet for the transmission line 5 an entrance stage 6a with a memory for the transmitted image areas and the additionally transmitted information about their position in the respective image. This input stage is with the logic stage 6b connected, which has read access to the memory of the input stage and links the data read from the memory in a manner yet to be described. The link level 6b is an image output unit 6c downstream to output the areas linked to an image to a display device.

It should be noted that the image data reduction and transmission unit through programmable microelectronics such as a PC or a dedicated signal processing electronics can be implemented, just as as possible is the recipient and the steps described, for example, by software on a computer system to realize. Furthermore, it is obviously possible to receive and Summarize transmitters, possibly on one and the same System, such as a PC, simultaneously image processing, storage and playback can be done.

With the system described, an image sequence data reduction now takes place, that is to say a compression of the data of an image sequence, for example a continuous stream of video data with 15-40 images per second, as described below with reference to FIG 2 and 3 explained. Based on 2 the image area prioritization is explained using 3 the image reconstruction of an image of an image sequence.

The 2a (I - V) shows successively recorded images of the scene 2 , In pictures (I) and (II) a person is shown at the bank counter who is standing there immobile for a longer period of time, so that changes from picture (I) to picture (II) are only caused by noise. In the picture (III) the person draws a gun. In picture (IV) the person begins to speak, which leads to the fact that the area around the mouth between picture (III) and picture (IV) changes significantly. At the same time, a random change in the background has occurred due to the falling off of a leaf of the plant shown.

It will be appreciated that this change in Background for the viewer of the raid scene from is of minor interest. In picture (V) the person turns with the face of the camera too.

This scene, which was chosen only for reasons of easier explanation and regardless of the typically smaller changes from picture to picture in actual video sequences with higher frame rates, can now be carried out as follows with reduced data by using the device 1 be transmitted.

The pictures (I) - (V) are initially in divided a large number of image areas, namely in the present case into 6 × 8 tiles, which are identical from picture to picture. This is through the grid shown in the pictures.

The more different Importance of the image areas taken into account.

2 B) shows areas on the far left that are of less interest to the viewer. On the one hand, there is the area on the left in the first picture of the row near the switch, which is completely unchangeable, and the area on the far right in this picture, which also shows the almost unchangeable bench furniture, ie a plant. These areas of lesser interest can be specified by the user and as prioritization information in the image data reduction and transmission unit 4 , especially in the prioritization information store 4d same, can be filed or automatically identified through long-term observation of changes.

From image (I) to image (II), changes as described above only occur due to noise. The tiles with the greatest changes in the noise value are in the first picture of 2 B (far left in the row) marked. The markings take into account the strength of the noise. A serpentine line indicates that the noise is below the threshold justifying a transmission, while the change in the cross was so large that an actually relevant change should not initially be excluded. It is clear that noise will typically appear in every tile. The serpentine lines are therefore only those tiles in which the noise between image (I) and image (II) caused the greatest differences.

Now is in 2 B In the picture on the far left it can be seen that two tiles, which, as can be seen from the crosses, have a particularly large noise difference between the two pictures, are located in the area of the plant that is determined to be uninteresting and thus prioritized. This is in 2c indicated in the picture on the far left that there is a zero instead of the two crosses, which means that after linking the information about the most changed area with the information about the typical relevance of the same areas in which this change occurs , a transmission can be dispensed with. The pictures of 2c The far left are therefore obtained by determining the differences between two images, taking into account their typical relevance based on the respective area.

2d shows that in addition to the image areas determined to be transmitted by forming the difference and comparing them with particularly relevant areas, other areas are also added cyclically. In the example shown, this is a block of three tiles, which moves from image to image in a meandering manner, as indicated by the serpentine arrow and is clearly visible when viewing the images from left to right. This block is transmitted regardless of whether or not an image tile is in an area that is of no interest per se. In the example shown, the 6 × 8 tiles, into which each individual image is broken down, are completely transferred with groups of three after the transmission of 16 partial images. At the latest after 16 drawing files, each tile was retransmitted once, even if there was no change in it. 2e shows on the far left that only the three image tiles from the cyclical transmission and a tile with high noise have to be taken into account for the transmission.

In comparison between image (II) and image (III) in 2a the biggest changes in the area of the freshly drawn weapon are now apparent. These changes are very big. In particular, they are significantly larger than the changes between the images (I) and (II), which are marked with crosses and come just above the noise threshold. For this reason, the two tiles with the largest changes are marked with an exclamation mark and it is found that, due to the very large change, neighboring tiles have to be transferred as prioritized. This is indicated by the framing of the two tiles with exclamation marks, cf. 2 B , 2nd picture left. The area around the two most important tiles is 3 × 4 tiles in total. Of these twelve tiles are, however, as in 2c recognizable in the second field from the left, two tiles in the area that is not marked as interesting in principle. These tiles are again hidden.

2c shows where tiles for cyclical refreshing are to be provided in the current image and 2e shows which tiles are actually to be transferred after taking the cyclical refresh into account. As can be seen, a total of thirteen tiles are transferred.

The pictures (III) and (IV) differ from each other in those tiles in which the mouth of the Person lies and in which the fallen leaf of the plant was. Due to the large change, a frame is again drawn around the tile with the moving mouth, since changes can typically also be relevant in adjacent tiles and, moreover, a complete image section with the surroundings of the change can be provided for the transmission. On the other hand, no frame is drawn around the strong change on the tile of the fallen plant leaf, because it is determined that this tile lies within an area of no interest per se. Furthermore, when prioritizing areas to be selected for transmission, it is found that there have previously been major changes in the area of the tiles in which the weapon has appeared. It cannot therefore be ruled out that relevant details can still be recognized there, even if the overall changes may be small compared to other changes in the image. This is indicated by a question mark in the respective tiles, cf. 2 B , 2c shows that priority is thus given to those image areas for the transmission which are in the vicinity of the image tile provided with an exclamation mark and the two tiles in which the greatest change was previously observed. In addition, as in 2d indicated, three tiles from the cyclical run of the picture. 2e shows where the fourteen tiles to be transferred are.

In the comparison of the pictures (IV) and (V), the greatest change is in the area of the head, whereby a part of the tiles with strong changes was previously considered due to the movement of the mouth. 2c shows on the far right those twelve tiles that were selected accordingly, 2d shows on the far right which three tiles are added by the cyclical refresh and 2e shows which tiles are actually made available for transmission.

From the above it can be seen that a Prioritization and a. takes place with regard to the general meaning certain image areas, which are characterized by user specifications and / or from the image analysis itself reveals that and how noise thresholds, sizes of the change also and especially compared to other tiles in the current Image, the short-term history of differences and the consideration of changes in nearby, in the present case only immediately adjacent Image tiles can be used. It should be mentioned that more Criteria can be added and / or parts or all of the above criteria no longer apply can and still get a remarkable image sequence data reduction becomes. It should also be mentioned that others possibilities the link the prioritization information is possible as the above, a link using methods of fuzzy logic, weighting, etc. can be done.

3 now shows how the image areas transmitted at certain times lie within the image. It can be seen that these areas are in a memory in the receiver, in the present case in the memory 6a , stored and read out for the reconstruction of a given image. The pictures of the sequence are in 3 characterized by the times (t-0, t-1, t-2 ....) at which they were recorded.

In order to construct an image that is as current as possible for the time t = 0, all those tiles that were transmitted for this time t-0 are first used. They are on the bottom line of 3 marked with 0 in the picture. Then all those tiles are taken into account that were transferred for image t-1 and could not yet be found in image t-0. In the last line they are marked with a 1 in the picture. Then it is determined which tiles do not appear in either picture t-0 or picture t-1, but can be found in picture t-2. These receive a 2 in the corresponding line in the last line. This continues up to Figure t-15. At the latest now, due to the cyclical run, all tiles are available once. The image to be displayed at time t-0 thus has a small number of tiles that are at most sixteen images in the sequence behind and a large number of tiles from images that are younger. In particular, in the image that is output for the time t = 0, there are primarily tiles in the relevant area that have very fresh changes.

It should be noted that the above description exclusively quick understanding serves the invention and that variations are easily recognizable. So it is not imperative the transfer rate vary from picture to picture. It can also be a fixed transfer of n tiles of an image. It is possible from the Way in which the image is used for cyclical refreshing other priorities etc. are possible. The "decay" of images can longer than just take a picture and some of the change significance and / or if necessary, further changes depend. It can Differences in brightness and / or color channel differences individually and / or together be evaluated. It should be noted that each one Tile before its transfer own compression such as wavelet compression or the like can be subjected. In particular, this enables one and the same transmitter a different line, for example, a typical JPEG still picture to output and at the same time to produce moving picture sequences and / or by simple linking create a still image of tiles in standard format.

Claims (18)

Method for image sequence data reduction, characterized in that image area prioritization information is provided, image area prioritization is carried out on the basis of the image area prioritization information and sufficiently prioritized image areas are transmitted. Method according to the preceding claim, characterized characterized that for Playback transmitted Image areas of a plurality of images of a sequence are stored and images to be displayed are reconstructed by linking the image areas become. Method according to one of the preceding claims, characterized characterized that the Image into a variety of at least for a variety of images same areas, especially for all images of a sequence have the same areas, especially tiles is divided. Method according to one of the preceding claims, characterized characterized that as Image area prioritization information Information regarding at least a size of the following Group is provided, namely Time since the last transmission of a certain, in particular of the particular image area under consideration or one or a group of neighboring and / or otherwise of related Image areas, difference between an image area and a previous image area in a previous picture, especially the immediately preceding picture, the presence of serious changes at a previous point in time, especially at short notice past Time and / or the repeated occurrence of relevant changes in a given area, a user definition, the existence of serious changes in one or more neighboring image areas and / or a specification from the transmitter and / or another controlling or regulating unit. A method according to the preceding claim, wherein Image area prioritization information by comparison, in particular Formation of a difference to one or a middle previous picture is determined and in which brightness and / or in at least one color channel Color differences can be evaluated. A method according to any one of the preceding claims, wherein local and / or global differences from an immediate or long term past, especially long-term middle image are evaluated and image area prioritization information taking into account a noise threshold is determined, in particular an image area prioritization taking into account noise thresholds he follows. A method according to any one of the preceding claims, wherein a variety of influencing factors the prioritization of the image area by weighting, the process of blurred Logic considered become. A method according to any one of the preceding claims, wherein Image differences from previous ones Images, in particular typical and / or average images and image areas with compared to other image areas particularly high change be prioritized. Method according to one of the preceding claims, characterized characterized that only sufficient prioritized image area transmitted become. Method according to one of the preceding claims, characterized characterized that the to be transferred Image areas are subjected to a compression process, in particular an approximately DCT and / or wavelet-based compression method, especially in JPEG standards compliant Wise. Method according to one of the preceding claims, characterized characterized that the Image areas are defined by tiles. Method according to one of the preceding claims, characterized characterized that area information also transmitted which indicates the position of the prioritized image areas in the image. Method according to one of the preceding claims, characterized characterized that a Prioritization is carried out in such a way that a regular, in particular a cyclical transmission each image area is provided. Method according to one of the preceding claims, characterized characterized that a Vectorization, determination of polygons, bitmap transmission and / or tiling for image area definition. Method according to one of the preceding claims, characterized characterized that a Tiling is used to define the image area and square tiles be provided. Method according to one of the preceding claims, characterized characterized that the Reconstruction with evaluation of area information, which is the location of the image-transmitting prioritized image area. Video camera with a means for reducing the size of image sequence data before storage and / or transmission. Software that loads the Reduction of image sequence data according to one of the preceding method claims and / or the Reconstruction of such data-reduced image sequences made possible.