JPH09219836A - Image information recording method and image synthesizing apparatus - Google Patents

Abstract

(57) [Abstract] [PROBLEMS] In order to continuously screen-combine a moving specific portion in a reproduced digital moving image with another image, another image needs to be calculated three-dimensionally in accordance with the reproduced moving image. The purpose is to facilitate the generation of this calculation data. SOLUTION: A key that is used to detect the calculation data at the same time without delay from a reproduction signal of a medium on which the calculation data is pre-recorded and to calculate another image data in a reproduced moving image in synchronization with each scene in the same manner. Highly accurate image composition is performed by determining a specific part to be composed by data.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION In a video CD or a digital video disc (hereinafter referred to as a DVD) for converting a moving image into a digital signal and recording / reproducing it, another image arbitrarily input by the user is synthesized with the reproduced image. The present invention relates to an image information recording method and an image synthesizing apparatus for creating a new moving image.

[0002]

2. Description of the Related Art Recently, a recording / reproducing apparatus which digitally processes a moving image and records / reproduces it as a digital signal has attracted attention. These have advantages of high recording density, excellent signal reproducibility, and easy signal processing by digital processing.

Conventionally, mainly in analog signals, 2
There was a method called Superimpose or Chromakey as a method to combine two images. However, although these means can combine images of fixed places and backgrounds, they cannot continuously replace or combine a specific part of a moving image such as a moving hero's face. This is because the means for continuously detecting the specific portion of the moving image and the second image creation corresponding to the specific portion are complicated and very difficult in real time. These are also becoming possible by digitalizing moving images.

[0004]

However, in order to continuously replace or change a specific part of a moving image as a digital signal with another moving image, the specific part of the image is detected and the image to be replaced is specified as the moving image. These processes were very complicated because it was necessary to perform three-dimensional conversion and image synthesis according to the orientation of the parts. Further, no information expressing a specific part of the image was recorded.

In view of this point, the present invention, in a device for recording / reproducing an image by a digital signal, for example, a DVD, information for extracting a specific portion of a moving image that continuously changes at the time of recording is preliminarily sub-digitalized on the same recording medium. An object of the present invention is to easily compose an image by recording as a data together with a digital moving image signal, performing signal processing on the basis of the information at the time of reproduction, and composing the reproduced moving image signal.

[0006]

To solve this problem, the present invention provides key data for extracting a specific portion of a digital moving image signal and three-dimensional data representing the specific portion for each video frame of the digital moving image signal. Is an image synthesizing device for reproducing a recording medium recorded by an image information recording method in which a digital moving image signal is inserted as sub-digital data in each predetermined section of the digital moving image signal. Means, a second restoring means for extracting the sub-digital data to restore the key data and the three-dimensional data, a memory for inputting and storing a digital video signal, and a two-dimensional operation for calculating the digital video signal of the memory by the three-dimensional data. It is provided with a video calculation means for creating a video and a synthesizing means for extracting the output of the first restoring means with key data and synthesizing the two-dimensional video signal.

As a result, since the key data for extracting a specific portion and its three-dimensional data are recorded together with the moving image in the recording process, the key data and the three-dimensional data necessary for the image synthesis on the reproducing device side are created. Without doing so, the image composition can be simplified during the reproduction process.

Furthermore, with respect to the time lag between the moving image and the second two-dimensional image generated in the process of image combination on the reproducing device side,
This can be solved by inserting timing data for correcting the deviation in advance during recording.

According to the present invention having the above-mentioned structure, the second image data is three-dimensionally processed by the three-dimensional data read from the recorded medium on the reproducing device side, and the specific position on the moving image is determined by the same key data, The second image after the three-dimensional processing is combined with the moving image.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The invention according to claim 1 of the present invention is an image information recording method, wherein key data and a specific portion for extracting a specific portion of a digital moving image signal for each video frame of the digital moving image signal. And three-dimensional data expressing the above are inserted as sub-digital data into a predetermined section of a digital moving picture signal and recorded on a recording medium, and information on a specific portion of the digital moving picture signal is a predetermined section of the digital moving picture signal. It has an effect of being able to detect each.

A second aspect of the present invention is an image synthesizing device for reproducing a recording medium recorded by the image information recording method according to the first aspect, wherein a digital moving image signal is converted into an original video signal. A first restoring means for restoring and a second restoring means for extracting the sub-digital data to restore the key data and the three-dimensional data
Restoring means, a memory for inputting and storing a digital video signal, a video computing means for computing a digital video signal in the memory with three-dimensional data to create a two-dimensional video, and an output of the first restoring means with key data, 2 The digital data is provided by including a synthesizing means for synthesizing the three-dimensional video signal, and restoring key data and three-dimensional data from information of a specific portion of the digital moving image signal which can be detected by the image information recording method of claim 1. A specific part is extracted from the video signal restored from the video signal, and the separately input digital video signal is
It has an effect of synthesizing into a three-dimensional video signal.

[0012]

An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the circuit arrangement of a first embodiment of the image synthesizing apparatus of the present invention. Further, FIG. 2 is an explanatory diagram of the relationship between images in the image information recording method in the first embodiment of the present invention, and FIG. 3 is a structural diagram of a recording signal in the image information recording method.

First, the image information recording method of the present invention will be described. In FIG. 2, a is a scene forming a moving image, b is key data indicating a specific portion of the scene a, c is three-dimensional data such as a line drawing of the specific portion, and d is three-dimensional data c.
Which are enlarged images of the above, e and f relate to an image synthesizing device described later, e is an input video, and f is a final synthesized output image.

In FIG. 3, a, b, and c are respectively shown in FIG.
The one scene a, the key data b, and the three-dimensional data c shown in FIG. 2 are treated as a group of a plurality of frames, and this unit is called GOP (Group Of Picture). This G
As shown in g in FIG. 3, the data string in the unit of OP is M in which a plurality of frames of moving image data are grouped together, similarly, the same number of frames of key data and three-dimensional data is grouped in D, and the header H is also recorded. , The character data P, the voice data A, etc. form a GOP. Further DVD
In such a case, interleaving is applied to compensate for signal loss in the transmission system and recorded on the medium. In digital recording / reproduction of a moving image, image data is divided into certain fixed units to form a stream together with other data. As an example thereof, FIG. 3 shows a data structure in the case of forming one GOP for every 15 frames of an image.

The moving image of the scene a shown in FIG. 2 is digitized into V1 to Vn shown in FIG. 3, and encoded as a standard data stream in which a header H, character data P, voice data A, etc. are added to this. It is continuously recorded on the recording medium. As will be described later, a decoder that performs this reverse operation at the time of reproduction (first in FIG. 1 showing the image forming apparatus of the present invention)
The original moving image is restored by the restoring means 1).

In the present invention, key data represents the location of a specific portion of a moving image, and three-dimensional data represents a so-called line image in which the three-dimensional coordinates of the specific portion are connected by a line. These two pieces of data are created as sub-digital data for each frame, and 15 frames are collected and inserted for each GOP unit as shown in the data string g of FIG. 3 and recorded. For example, as a means for creating sub-digital data, a sensor is attached to a specific portion of a moving image and a plurality of image pickup devices pick up images at different angles, and the angle of the sensor received by the image pickup device is detected and imaged. A method of computing and generating a plurality of image data, a method of analyzing captured image data for each frame and artificially computing and generating the above-mentioned data of a specific portion, and a method of generating three-dimensional data from the beginning to use computer graphics However, any of these methods can be performed by the conventional technique.

That is, according to the present invention, a new function can be provided at the time of reproduction by recording the information of the specific portion of the moving picture signal as sub digital data at the same time as the digital moving picture signal.

Next, the structure and operation of the image synthesizing device for reproducing the recording medium recorded by such an image information recording method will be described.

In FIG. 1, a signal in which sub-digital data is inserted in a digital moving picture signal is input to an input terminal 6, and the first restoring means 1 is means for restoring the digital moving picture signal to the original reproduced moving image. The 2 restoration means 2 extracts the sub-digital data and restores it to the key data representing the location of the specific portion of the moving image and the three-dimensional data representing a so-called line image in which the three-dimensional coordinates of the specific portion are connected by a line. On the other hand, the second image is input to the input terminal 7 and the video data is stored in the memory 3. The image calculation means 4 calculates the direction and expression of the image data in the memory 3 based on the three-dimensional data to form a two-dimensional image, and the synthesizing means 5 extracts the necessary part from the reproduced moving image according to the key data and the two-dimensional image. Combine with image.

FIG. 4 shows an example of the second two-dimensional image and its digital data in the image synthesizing apparatus. The second image can be easily generated by, for example, a digital camera, or a plurality of images can be pseudo-composited by a computer.

In this way, the second image is digitized as two-dimensional data and stored in the memory 3. The memory 3 is connected to the video calculation means 4 of the reproducing device, and the two-dimensional video signal is three-dimensionally calculated according to the sub-digital data, and the angle, size, facial expression and peripheral parts corresponding to a specific part of the moving image are displayed. A new two-dimensional image that covers the effects of the above is generated.
This new two-dimensional video signal is sent to the synthesizing means 5 together with the key data of the sub-digital data and reproduced moving image (f in FIG. 2).
Is synthesized.

In the first embodiment, the case where the face of a person is replaced with the second image has been described. However, if data of a portion other than the face is newly added and recorded in FIGS. 2 and 3,
It goes without saying that parts other than the face of the person can be replaced at the time of reproduction by utilizing similar technology.

Further, in the embodiment, a video CD or a DV
Although the example of D has been given, the present invention is not limited to this, and for example, even in karaoke, games, etc., it is possible to provide a moving image which has not been possible by the same means and principle.

[0025]

As described above, according to the present invention, since sub-digital data for reproducing a specific portion of a moving image is recorded in advance, a reproducing device can easily generate another image data for replacing the specific portion. Since images can be combined,
It is possible to provide an image that is close to the user, such as replacing the face of the main character of the movie with the face of the user, or replacing the background image with another background, and the effect of implementation is great.

[Brief description of drawings]

FIG. 1 is a block diagram of an image composition device according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram of image relationships in the image information recording method according to the first embodiment of the present invention.

FIG. 3 is a structural diagram of a recording signal of the same.

FIG. 4 is an explanatory diagram of a second image and data structure of the same.

[Explanation of symbols]

 1 First Restoration Means 2 Second Restoration Means 3 Memory 4 Video Calculation Means 5 Combining Means A Audio Data D Sub Digital Data H Header M Moving Image Data O1 Front Image Data O2 Top Image Data O3 Side Image Data P Character Data Q Header a A person image including a specific portion b A specific portion of a moving image c A coordinate axis line drawing of a specific portion d An enlarged line drawing of c An image of a specific portion f Composite output image

Claims (2)

[Claims] 1. A predetermined section of the digital moving image signal, in which key data for extracting a specific portion of the digital moving image signal and three-dimensional data expressing the specific portion are used as sub-digital data for each video frame of the digital moving image signal. An image information recording method in which each image is inserted and recorded on a recording medium. 2. An image synthesizing device for reproducing a recording medium recorded by the image information recording method according to claim 1, wherein the first reproducing device restores a digital moving image signal to an original video signal, and a sub-digital device. Second restoring means for extracting data and restoring it to key data and three-dimensional data, a memory for inputting and storing a digital video signal, and a digital video signal of the memory is operated by the three-dimensional data to create a two-dimensional video. An image synthesizing apparatus comprising: a video calculating unit for performing the above-described operation;