JP2001145112A - Moving image synthesizing apparatus, moving image synthesizing method, and recording medium - Google Patents

Abstract

(57) [Problem] To provide a moving image synthesizing apparatus, a moving image synthesizing method, and a recording medium capable of synthesizing frames of different encoding types at high speed. SOLUTION: An image encoding means 101 compression-encodes an input video signal 151 into an MPEG format on a frame-by-frame basis. The image accumulating unit 102 includes the image encoding unit 10
1 is temporarily stored. The image synthesizing unit 103 extracts one frame of data of each moving image data to be synthesized from the image storage unit 102, and if all the frames of the extracted moving image data have the same encoding type, synthesizes the data as it is. Perform processing.
When frames of the I-picture format and the P-picture format are included, the image synthesizing unit 103 converts the I-picture into a P-picture by the frame type conversion unit 104, and sets the encoding type to the P-picture format. The moving image data is synthesized.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image synthesizing apparatus for synthesizing compression-encoded moving image data, a moving image synthesizing method, and a recording medium storing a program for executing the moving image synthesizing method. , More specifically, MPEG
Encoding type (I
The present invention relates to a moving image synthesizing apparatus, a moving image synthesizing method, and a recording medium capable of synthesizing moving image data of different encoding types at high speed irrespective of a picture format and a P picture format.

[0002]

2. Description of the Related Art MPEG is a method of compressing and encoding a video signal of a moving image as digital data (moving image data).
There are international standards called methods. The MPEG system is
A high compression rate is realized by encoding only information that has changed from the preceding and succeeding frames using the correlation between consecutive frames. That is, in the MPEG system, a frame in which encoding is completed in one frame without referring to another frame is changed from I-picture (Intra-picture) format data (hereinafter referred to as I-picture) from the immediately preceding frame. Frame that encodes only the extracted information is a P-picture (Pred
ictive picture) format data (hereinafter referred to as a P picture), and a frame obtained by encoding only information that has changed from two frames before and after the frame is converted into a B picture (Bidirectionally predicate).
tive picture) format data (hereinafter referred to as B picture)
And these three types of frames are I, B,
Encoding is performed in a mixed format such as B, P, B, B, P,.

[0003] Since the MPEG system can realize a high compression rate, it is widely used when moving image data is transmitted via a network or stored in a storage device. Then, a moving image list system and a video conference system can be realized by generating a multi-screen image by synthesizing a plurality of MPEG moving image data by the moving image synthesizing device.

In a system in which a video taken by a video camera is transmitted via a network, and each video is synthesized and displayed on a multi-screen, when a B picture is inserted, a delay occurs during reproduction. Often, only P-pictures and P-pictures are used.

FIG. 25 shows a configuration diagram of a conventional moving image synthesizing apparatus which generates a multi-screen image by synthesizing a plurality of MPEG moving image data. Here, the moving image synthesizing apparatus of the first conventional example in FIG. 25 is used when the original moving image data to be synthesized is of the same type of encoding format (that is, when frames of the same picture format are synthesized). Used.

In FIG. 25, the moving image synthesizing apparatus of the first conventional example changes the display position information of the frame in the moving image data inputted by the synthesizing apparatus 500 in accordance with the arrangement of the synthesizing screen, and generates a multi-screen image ( (Combined moving image). Thus, the input moving image data can be quickly synthesized with the synthesized moving image without decoding.

[0007]

However, in the moving image synthesizing apparatus of the first conventional example, it is not possible to synthesize frames of different encoding formats, and it is not possible to synthesize an image captured by a video camera or a received image. In the case of configuring a system that combines broadcasts and displays a list on a multi-screen, there is no guarantee that the encoding format types of the frames of each input moving image data are the same. There was a problem.

Another moving image synthesizing apparatus for synthesizing a plurality of MPEG moving image data to generate a multi-screen video is to decode input moving image data to generate synthesized moving image data. There is a moving image synthesizing apparatus that generates a synthesized moving image by encoding, regardless of the encoding format of a frame in input moving image data. FIG.
FIG. 6 shows a configuration diagram of the moving image synthesizing apparatus of the second conventional example.

Referring to FIG. 1, a moving image synthesizing apparatus according to a second conventional example temporarily decodes moving image data input by decoding apparatuses 600a and 600b into moving image data in a spatial area such as a bit map, and encodes the data. A combined moving image data is generated by generating the combined moving image data by 601 and re-encoding. This makes it possible to generate a multi-screen video (synthesized moving image) irrespective of the encoding format type of the frame of the moving image data to be synthesized.

However, in the moving picture synthesizing apparatus of the second conventional example, the decoding apparatuses 600a and 600b and the coding apparatus 601 are necessary to decode and re-encode the moving picture data. That is, in order to perform complicated calculations of decoding and encoding at high speed, an expensive device configuration with higher performance is required, and there is a problem that the device cost increases. In addition, since the encoded moving image data is decoded, synthesized, and then encoded again, there is a problem in that the quality of the video is degraded.

The present invention has been made in view of the above-mentioned conventional circumstances and problems, and synthesizes moving image data in which, for example, I pictures and P pictures conforming to the MPEG standard are mixed in an arbitrary order. At this time, by converting the encoding type of the I picture into the P picture format, a moving image synthesizing apparatus capable of synthesizing moving image data including different encoding types at a high speed with a minimum decoding process, It is intended to provide a method and a recording medium.

[0012]

According to a first aspect of the present invention, there is provided a moving picture synthesizing apparatus for converting an input video signal into an I picture format and a P picture format conforming to the MPEG standard. Image encoding means for encoding as moving image data including, image accumulating means for accumulating the moving image data, frame type converting means for converting I picture format data of the moving image data to the P picture format data, Image synthesizing means for extracting moving image data from the image accumulating means and performing synthesizing processing, and when synthesizing moving image data including different encoding types, the image synthesizing means includes an encoded or converted P-picture. This is to synthesize moving image data including format data.

Further, a moving image synthesizing apparatus according to claim 2 is
An image encoding unit that encodes an input video signal as moving image data including an I-picture format and a P-picture format based on the MPEG standard; Marking code insertion means for inserting a marking code capable of identifying the boundary of the macroblock data; image storage means for storing the moving image data; and detecting the marking code of the moving image data to obtain the I picture format data. Frame type conversion means for converting the coding type of the macro block into a code word of a P-picture format, and image synthesizing means for extracting moving image data from the image storage means and performing a synthesizing process. When synthesizing the moving image data including the encoded or converted P-picture, It is intended to synthesis processing moving image data containing the expression data.

Further, a moving image synthesizing apparatus according to claim 3 is
3. The moving picture synthesizing apparatus according to claim 2, wherein the marking code inserted by the marking code inserting unit is a data of a processing system in which the amount of decrease in code length when the coding type of the macroblock is converted by the frame type converting unit. A code word having a code length that is a multiple length of the processing unit,
The frame type conversion means deletes the marking code when converting the coding type of the macroblock.

Further, the moving picture synthesizing apparatus according to claim 4 is
4. The moving picture synthesizing apparatus according to claim 2, wherein when the image coding means performs variable length coding on the macroblock data, the code length of the macroblock data is a data processing unit of a processing system. It is provided with code length adjusting means for changing the code word of the high frequency component of the macro block.

Further, a moving image synthesizing apparatus according to claim 5 is
A plurality of N frames (N is a positive integer) of a maximum of N frames per second of moving image data including I picture format data and P picture format data encoded in I picture format and P picture format conforming to the MPEG standard are input. An image buffer; skip frame generating means for generating the P-picture skip frame when there is no moving image data when extracting moving image data from the plurality of image buffers; and I-picture format data of the moving image data. And a frame type conversion unit for converting video data into the P picture format data, and an image synthesis unit for extracting moving image data from the plurality of image buffers and performing a synthesis process once every Nth of a second. When synthesizing the moving image data including the type, the image synthesizing means includes an encoded or converted P-picture format. The moving image data including over data is to synthesis processing.

Further, the moving picture synthesizing apparatus according to claim 6 is
6. The moving picture synthesizing apparatus according to claim 5, further comprising a code amount adjusting means for adjusting the coding amount of the moving image data synthesized by said image synthesizing means to be constant.

Further, a moving image synthesizing apparatus according to claim 7 is
7. The moving picture synthesizing apparatus according to claim 6, wherein the code amount adjusting unit converts the I picture format data into the skip frame when the data amount of the moving image data synthesized by the image synthesizing unit is larger than a predetermined range. With
If it is smaller than the predetermined range, predetermined supplementary data is added after the moving image data.

Further, a moving image synthesizing apparatus according to claim 8 is
8. The moving picture synthesizing apparatus according to claim 1, wherein said frame type conversion means comprises:
The encoding type of the frame of the picture format data is converted into a code word of the P picture format.

Further, a moving image synthesizing apparatus according to claim 9 is
9. The moving picture synthesizing apparatus according to claim 1, wherein said frame type conversion means sets a coding type of a macroblock of said I picture format data to a P picture format code. It is converted into words.

According to a tenth aspect of the present invention, in the moving image synthesizing apparatus according to the first, second, third, fourth, fifth, sixth, seventh, eighth, or ninth aspect, the frame type conversion means includes: A macroblock stuffing (Macroblock Stuffing) code is inserted so that the increase in the code length of the I picture format data is a multiple of the data processing unit of the processing system.

According to a eleventh aspect of the present invention, in the moving image synthesizing apparatus according to the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth or tenth aspect, the frame type conversion means is provided. Rewrites the code word of the high-frequency component of the immediately preceding macroblock or the low-frequency component of the immediately following macroblock so that the amount of change in the code length of the I-picture format data matches the data processing unit of the processing system. .

The moving picture synthesizing apparatus according to the twelfth aspect is characterized in that the moving image synthesizing apparatus according to the twelfth aspect has the following features.
12. The moving picture synthesizing apparatus according to claim 11, further comprising a noise reduction unit that converts a high frequency component of a quantization matrix of the moving image data generated by the image synthesizing unit into a predetermined small value.

The moving picture synthesizing method according to claim 13 is a picture coding step of coding an input video signal as moving picture data including an I picture format and a P picture format conforming to the MPEG standard. An image storing step of storing moving image data; a frame type converting step of converting I-picture format data of the moving image data into the P-picture format data; and an image for extracting moving image data from the image storing step and performing a synthesizing process. And a synthesizing step. When synthesizing moving image data including different encoding types, the image synthesizing step is for synthesizing moving image data including encoded or converted P-picture format data.

Further, the moving picture synthesizing method according to claim 14 is a picture coding step of coding an input video signal as moving picture data including an I picture format and a P picture format conforming to the MPEG standard. When the image encoding step performs variable-length encoding of macroblock data, a marking code insertion step of inserting a marking code capable of identifying a boundary of the macroblock data, and an image accumulation step of accumulating the moving image data,
Detecting the marking code of the moving image data,
Set the encoding type of the macroblock of the picture format data to P
A frame type conversion step of converting the image data into a code word of a picture format, and an image synthesis step of taking out moving image data from the image storage step and performing a synthesis process, when synthesizing moving image data including different coding types. The image synthesizing step is for synthesizing moving image data including encoded or converted P-picture format data.

According to a fifteenth aspect of the present invention, in the moving image composing method according to the fourteenth aspect, the marking code inserted in the marking code inserting step is a macroblock encoding performed by the frame type converting step. A code word having a code length such that the amount of reduction in the code length when converting the type is a multiple of the data processing unit of the processing system, and the frame type conversion step converts the coding type of the macroblock. In doing so, the marking code is deleted.

[0027] In the moving picture synthesizing method according to the present invention, when the image coding step performs variable length coding of the macroblock data, the macroblock data may be decoded. The method includes a code length adjusting step of changing a code word of a high frequency component of the macro block so that the code length of the data is a data processing unit of the processing system.

The moving picture synthesizing method according to the seventeenth aspect of the present invention provides a method for synthesizing moving picture data including I picture format data and P picture format data encoded in I picture format and P picture format conforming to the MPEG standard. N
An image input step of inputting a number of frames (N is a positive integer) to a plurality of image buffers; and, when moving image data is extracted from the plurality of image buffers, a skip frame of the P picture format when there is no moving image data , A frame type conversion step of converting I-picture format data of the video data into the P-picture format data, and extracting video data from the plurality of image buffers, for 1 / N second 1 in
And synthesizing moving image data including the encoding type, the image synthesizing step includes converting the moving image data including the encoded or converted P-picture format data. This is a process for combining.

In the moving picture synthesizing method according to the eighteenth aspect of the present invention, the coding amount of the moving picture data synthesized in the image synthesizing step is adjusted to be constant. It comprises a code amount adjusting step.

According to a nineteenth aspect of the present invention, in the moving image synthesizing method according to the eighteenth aspect, the code amount adjusting step includes a step of setting a data amount of the moving image data synthesized in the image synthesizing step to a predetermined amount. If it is larger than the range, the I-picture format data is replaced with the skip frame. If it is smaller than the predetermined range, predetermined supplementary data is added after the moving image data.

According to a twentieth aspect of the present invention, in the moving image synthesizing method according to the thirteenth, fourteenth, sixteenth, seventeenth, eighteenth, or nineteenth aspect, the frame type conversion step is performed in the I picture format It converts the encoding type of the data frame into a code word of the P picture format.

Further, the moving picture synthesizing method according to claim 21 is based on claims 13, 14, 15, 16, 17, 18, 19.
21. In the moving picture synthesizing method according to 20, the step of converting the frame type converts a coding type of a macroblock of the I-picture format data into a code word of a P-picture format.

Further, the moving picture synthesizing method according to claim 22 is based on claims 13, 14, 15, 16, 17, 18, 1
22. The moving picture synthesizing method according to 9, 20, or 21, wherein the frame type conversion step includes the step of converting the macroblock stuffing (macroblock stuffing) so that the amount of increase in the code length of the I picture format data is a multiple of the data processing unit of the processing system. Macr
oblock stuffing) code.

Further, the moving image synthesizing method according to claim 23 is the method according to claims 13, 14, 15, 16, 17, 18, and 1.
23. The moving picture synthesizing method according to 9, 20, 21, or 22, wherein the frame type conversion step includes: changing a code length of the I-picture format data to a data processing unit of a processing system of a preceding macroblock. Of the high frequency component or the code word of the low frequency component of the immediately following macro block.

The moving image synthesizing method according to claim 24 is the method according to claims 13, 14, 15, 16, 17, 18, 1
24. The moving image synthesizing method according to 9, 20, 21, 22, or 23, further comprising a noise reduction step of converting a high frequency component of a quantization matrix of the moving image data generated in the image synthesizing step into a predetermined small value. is there.

A computer-readable recording medium according to claim 25 is a computer-readable recording medium.
5,16,17,18,19,20,21,22,23
Alternatively, it is recorded as a program for causing a computer to execute the moving image synthesizing method described in 24.

A moving picture synthesizing apparatus according to claims 1, 8, 9, 10, 11 and 12 of the present invention.
In the moving image combining method according to any one of the first, second, third, and fourth aspects, and the recording medium according to the twenty-fifth aspect, when moving image data to be combined includes different coding types of an I picture format and a P picture format, The I-picture format data is converted to P-picture format data by the frame type conversion means (frame type conversion step), and the moving image data whose encoding type is aligned to the P-picture format is synthesized by the image synthesis means (image synthesis step). Like that.

In particular, in the moving picture synthesizing apparatus according to the eighth and ninth aspects and the moving picture synthesizing method according to the twentieth and twenty-first aspects, the conversion from I-picture format data to P-picture format data is performed by a frame type conversion means. In the (frame type conversion step), the coding type of the frame of the I picture format data is converted into a code word of the P picture format,
It is desirable to perform the conversion by converting the coding type of each macroblock of the I picture format data into a codeword indicating an intra macroblock of the P picture format data.

As a result, there is no need to perform variable-length decoding and re-encoding processing as in the prior art, and an excellent moving picture synthesizing apparatus capable of generating a synthesized moving picture at high speed without deterioration in image quality. A synthesizing method and a recording medium can be realized.

In the moving picture synthesizing apparatus according to the tenth aspect and the moving picture synthesizing method according to the twenty-second aspect, when the coding type of the macroblock is converted by the frame type converting means (frame type converting step), A plurality of macroblock stuffing codes are inserted at the beginning of the macroblock so that the increase in the code length of the I picture format data is a multiple of the data processing unit of the processing system.

For example, in a processing system in which the data processing unit is 8 bits, if the code length is increased by 4 bits due to the conversion of the coding type of the macroblock, by inserting four macroblock stuffing codes,
Although the total number of bits increases by 48 bits, the amount of increase in the code length becomes a multiple of the data processing unit of the processing system, and the bit shift processing or the like accompanying the conversion becomes unnecessary. In other words, when converting the coding type of a macroblock using a processing system such as a computer, data processing for codewords other than the head part in which coding information of the macroblock is described becomes unnecessary, and image quality is degraded. Therefore, it is possible to generate a synthesized moving image at high speed without any problem.

In the moving image synthesizing apparatus according to the eleventh aspect and the moving image synthesizing method according to the twenty-third aspect, when the encoding type of the macroblock is converted by the frame type converting means (frame type converting step), The code word of the high frequency component of the immediately preceding macro block or the low frequency component of the immediately following macro block is rewritten.

For example, when the data processing unit is an 8-bit processing system, conversion of macroblock coding type
If the code length increases by bits, reduce the high-frequency component of the immediately preceding macroblock by 4 bits so that the code length does not change as a whole, or increase the high-frequency component of the immediately preceding macroblock by 4 bits. Thus, the code length is increased by 8 bits as a whole, so that the amount of change in the code length matches the data processing unit of the processing system. Thus, when the coding type of the macroblock is converted using a processing system such as a computer, the amount of change in the code length of the I-picture format data matches the data processing unit of the processing system, and the coding information of the macroblock is converted. Data processing for codewords other than the first part where
It is possible to generate a synthesized moving image at high speed without deterioration in image quality.

In the moving picture synthesizing apparatus according to the twelfth aspect and the moving picture synthesizing method according to the twenty-fourth aspect, when the coding type conversion of the macroblock is performed by the frame type conversion means (frame type conversion step), rewriting is performed. The high frequency component of the quantization matrix of the obtained moving image data is converted into a predetermined small value by a noise reduction unit (noise reduction step). As a result, it is possible to suppress deterioration in image quality caused by high-frequency components rewritten for high speed during reproduction.

Further, according to the present invention,
A moving picture synthesizing apparatus according to any one of claims 9, 10, 11, and 12, wherein
In the moving image combining method according to the fourth aspect and the recording medium according to the twenty-fifth aspect, when the moving image data to be combined includes different encoding types of the I picture format and the P picture format, the image encoding means (image When the coding step) performs variable-length coding on the macroblock data, the marking code insertion means (marking insertion step) uses a marking code capable of identifying a macroblock data boundary as a codeword of a high-frequency component of each macroblock data. At the time of synthesis, the I-picture format data is converted into P-picture format data by the frame type conversion means (frame type conversion step), and the encoding type is P
The moving image data arranged in the picture format is synthesized by the image synthesizing means (image synthesizing step).

For example, if the marking code is “00000”
If the number is 1000000010101010 ”,“ 00000010000000010 combined with an EOB (End Of Code) code “10” indicating the end of the macroblock data
The code word 10101010 ″ exists only at the final position in the macroblock data, and by detecting this codeword, the boundary of the macroblock can be found. Therefore, the coding type of the macroblock is converted. In this case, it is possible to specify the conversion point without performing variable-length decoding, and there is no need to perform variable-length decoding and re-encoding processing as in the past, so that there is no deterioration in image quality and higher speed. Thus, it is possible to realize an excellent moving image synthesizing apparatus, a moving image synthesizing method, and a recording medium capable of generating a synthesized moving image.

In particular, in the moving image synthesizing apparatus according to the third aspect and the moving image synthesizing method according to the fifteenth aspect, a code for converting a coding type of a macroblock is converted by a marking code inserting means (marking code inserting step). Insert a marking code with a code length such that the amount of decrease in length is a multiple of the data processing unit of the processing system, and delete the inserted marking code when converting the coding type of the macroblock. I have.

Thus, when converting the coding type of the macroblock, it is possible to specify the conversion location without performing variable length decoding, and by deleting the inserted marking code, the conversion of the coding type can be performed. Increase or decrease of the code length coincides with the data processing unit of the processing system, so that data processing for codewords other than the head part in which coding information of the macroblock is described becomes unnecessary, and the synthesized moving image can be more quickly processed. Can be generated.

In the moving picture synthesizing apparatus according to the fourth aspect and the moving picture synthesizing method according to the sixteenth aspect, when the macroblock data is subjected to variable length coding, the code length adjusting means (code length adjusting step) is used. For example, the code word of the high frequency component of the macro block is changed by rewriting the code word to a code length adjustment code, for example.

For example, in a processing system in which the data processing unit is 8 bits, it is calculated how many bits the code length of the entire macroblock data is short of a multiple length of 8 bits including the marking code. Is inserted according to the code length adjustment code. As a result, even if the marking code is inserted, the code length of the macroblock data becomes the data processing unit of the processing system, and when the coding type of the macroblock is converted, the comparison operation in the data processing unit without variable length decoding is performed. It is possible to specify the conversion point only by
Data processing for codewords other than the head part in which the coding information of the macroblock is described becomes unnecessary, and it is possible to generate a synthesized moving image at higher speed.

In the moving image synthesizing apparatus according to the twelfth aspect and the moving image synthesizing method according to the twenty-fourth aspect, the moving image data in which the code word is added to the high frequency component by the marking code inserting means and the code length adjusting means. The noise reduction means (noise reduction step) converts the high frequency component of the quantization matrix into a predetermined small value. As a result, it is possible to reduce a deterioration in image quality caused by a marking code or a code length adjustment code added to a high-frequency component for speeding up during reproduction, and to generate a synthesized moving image at high speed.

Further, according to the present invention, claims 5, 6, 7, 8,
A moving picture synthesizing apparatus according to any one of claims 9, 10, 11, and 12, and claims 17, 18, 19, 20, 21, 22, 23, and 2.
In the moving image synthesizing method according to the fourth aspect and the recording medium according to the twenty-fifth aspect, a plurality of image buffers (image input step) into which a maximum of N frames of moving image data are input per second
And a skip frame generating unit (skip frame generating step) for generating a skip frame in a P picture format. When the moving image data of each image buffer is synthesized by the image synthesizing unit (image synthesizing step), If there is no, a skip frame is inserted and N
The synthesizing process is performed once every 1 / second. Further, when the moving image data to be synthesized includes an encoding type different from the I picture format and the P picture format, the I picture format data is converted into the P picture format data by the frame type conversion means (frame type conversion step). The moving image data whose encoding type is arranged in the P picture format is synthesized by the image synthesizing means (image synthesizing step).

As a result, it is not necessary to perform variable length decoding and re-encoding processing as in the prior art, and it is possible to generate a synthesized moving image at high speed without deterioration in image quality.
An excellent moving picture synthesizing apparatus capable of generating a synthesized moving picture for N frames per second at a high speed in accordance with the MPEG standard,
A moving image combining method and a recording medium can be realized.

In particular, in the moving image synthesizing apparatus according to the sixth and seventh aspects and the moving image synthesizing method according to the eighteenth and nineteenth aspects, the code for adjusting the code amount of the synthesized moving image data to be constant. An amount adjusting means (code amount adjusting step) is provided. If the data amount of the combined moving image data is larger than a predetermined range, the I-picture format data is replaced with a skip frame. In addition, predetermined supplementary data is added after the moving image data.

For example, if the combined moving image data exceeds the set code amount, the I-picture format data is replaced with a skip frame, and if insufficient, a supplementary bit is inserted, thereby causing overflow and underflow during reproduction. Thus, a smooth synthesized moving image free of delay and jitter can be generated at a high speed without causing a problem.

[0056]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a first embodiment, a second embodiment, and a third embodiment of the moving image synthesizing apparatus and the moving image synthesizing method according to the present invention will be described. ,
[Fourth embodiment], [Fifth embodiment], and [Sixth embodiment] will be described in detail in this order with reference to the drawings.

In the description of each embodiment, a moving image synthesizing apparatus and a moving image synthesizing method according to the present invention will be described in detail. However, a recording medium according to the present invention is used to execute the moving image synthesizing method. Since the recording medium is a recording medium on which the program is recorded, the description thereof is included in the following description of the moving image combining method. In addition, the moving image synthesizing apparatus and the moving image synthesizing method according to the present invention include “H.
"H.263", "MPEG1", "MPEG2",
The present invention can be applied to each compression encoding method of “MPEG4”.

[First Embodiment] First, a moving image synthesizing apparatus and a moving image synthesizing method according to a first embodiment of the present invention will be described. Here, the first embodiment is a moving image synthesizing apparatus according to claims 1, 8 and 9 of the present invention, and
This corresponds to the moving image composing method according to claims 13, 20 and 21.

FIG. 1 is a block diagram of a moving image synthesizing apparatus according to the first embodiment. The moving image synthesizing apparatus according to the present embodiment performs a synthesizing process without decoding by converting an I picture into a P picture by the frame type converting means 104. In FIG. 1, the moving image synthesizing apparatus of the present embodiment includes an image encoding unit 101, an image storage unit 102, an image synthesizing unit 103, and a frame type conversion unit 104.

Here, the image encoding means 101 generates moving image data 152 including an I picture and a P picture obtained by encoding an input video signal 151 into an I picture format and a P picture format conforming to the MPEG standard. .
That is, the image encoding unit 101 receives the video signal 151 captured by a video camera or the like, compresses and encodes the video signal into an MPEG format (I-picture format and P-picture format) on a frame basis, and generates an I picture and a P picture. I do.

For example, the image encoding means 101
It is a G encoder, and its compression encoding method is performed in the process shown in FIG. Here, FIG. 2 is an explanatory diagram for explaining a compression encoding method in the MPEG encoder 101.

In the figure, the input video signal 151
Is first converted into pixel data 151 indicating the content of an image of one frame, and input to the MPEG encoder 101. Each pixel in the pixel data is represented by luminance Y, color difference U, and color difference V. Next, the MPEG encoder 1
In step 01, the input pixel data is divided into small regions of 16 × 16 pixels called macroblocks, and motion prediction and DCT (discrete cosine) transform are performed for each macroblock.

The motion prediction is performed by comparing with the pixel data of the immediately preceding frame when compressing as a P picture. Then, when an area with little change is found, a motion vector for that area is recorded, the amount of change in each pixel data is calculated as difference data, and encoded as a forward prediction macroblock. If an area with little change is not found, the macroblock is directly coded as an intra macroblock.

In the DCT transform, the difference data or pixel data is further divided into four regions of 8 × 8 pixels for luminance Y, and thinned to 8 × 8 pixels for color difference U and color difference V. Then, each DCT coefficient data is generated by performing an 8 × 8 two-dimensional DCT transform. Next, the DCT coefficient data is quantized and subjected to variable-length coding in accordance with the MPEG format codebook, so that the moving image data 152 in the MPEG format is obtained.
(MPEG bit stream) is generated. MPEG
The data of one frame of the moving image data encoded in the format has a hierarchical structure, as shown in FIG.
Here, FIG. 3 is an explanatory diagram illustrating a data configuration of one frame in the MPEG format.

In FIG. 3, one frame in the MPEG format has a picture header code 1 indicating information of the entire frame.
One and several slice data 12;
The slice data 12 is composed of a slice header code 17 indicating information on the entire slice and some macroblock data 18 constituting the slice. The macroblock data 18 is a macroblock header 19 indicating information on the entire macroblock. And the DC of the luminance component
Four luminance YDCT coefficient data 20 indicating a T coefficient value;
It is composed of color difference UDCT coefficient data 21 and color difference VDCT coefficient data 22 indicating the DCT coefficient value of the color difference component.

The picture header code 11 includes a picture start (PST) code 13 indicating the start of the frame, a temporal reference (TR) code 14 indicating the display order of the frame, and a picture coding indicating the encoding type of the frame. Type (PCT) code 15
And a video buffer delay (VD) code 16 indicating the capacity of the buffer when the frame is reproduced.

The macro block header code 19
Is a macroblock address increment (MBAI) code 23 indicating the display position of the macroblock, a macroblock type (MBTYPE) code 24 indicating the encoding type of the macroblock, and a code indicating the quantization scale (QS) of the macroblock. 25.

Next, the image storage means 102 stores the moving image data 152. That is, the moving image data 152 encoded in the I picture format and the P picture format by the image encoding unit 101 is temporarily stored until it is extracted by the image synthesizing unit 103.

The image synthesizing unit 103 extracts moving image data from the image storage unit 102 and performs a synthesizing process.
Further, the frame type conversion means 104 converts the I picture into P
Convert to picture. That is, the image combining means 103
Extracts the moving image data for one frame of the combined moving image from the image storage unit 102 and combines the extracted moving image data.

At this time, if the moving image data has a different encoding format, the image synthesizing means 103
2. The I picture extracted from the frame type conversion means 1
04, and the frame classification unit 104 converts the I picture received from the image combining unit 103 into a P picture. That is, the PCT code 15 of the picture header code 11 and the MBTYPE code 24 of the macroblock header code 19 in the I picture are converted into the P picture format. Then, the image synthesizing unit 103 receives the moving image data converted by the frame type converting unit 104 and performs a synthesizing process to generate a synthesized moving image 155.

Next, the operation of the moving picture synthesizing apparatus of the present embodiment having the above-described configuration, that is, the moving picture synthesizing method will be described in detail.

First, the image encoding means 101 converts the input video signal 151 into MPEG format I
Compression encoding into picture format or P picture format,
Output to the image storage unit 202. Image storage means 102
Temporarily stores the moving image data 152 output by the image encoding unit 101 until the moving image data 152 is designated and extracted by the image combining unit 103.

The image synthesizing means 103 takes in one frame of data of each moving image data to be synthesized from the image storage means 102 and generates data of one frame of the synthesized moving image. Here, an image combining processing method of the image combining unit 103 will be described with reference to FIG. FIG. 4 is a flowchart illustrating an image combining processing method of the image combining unit 103 according to the first embodiment.

In FIG. 4, first, in step S40, the image synthesizing unit 103 extracts one frame of each moving image data used for synthesizing from the image storage unit 102. Then, in step S41, when all the moving image data to be combined are extracted, in step S42, the image combining unit 103 determines whether all the extracted frames of the moving image data are of the same encoding type. Is determined, and if they are the same, the process proceeds to step S44, and the combining process is performed as it is. If both frames of the I-picture format and the P-picture format are included, in step S43, the frame type conversion means 1
04 converts an I picture into a P picture by a frame type conversion method described below.

Then, in step S44, the image synthesizing unit 103 synthesizes the moving image data with the encoding types all set to the P picture format by the frame type converting unit 104. This synthesizing process is performed according to the layout of slice data 12
Are rearranged, and the display position information of the slice header code 19 is corrected. This makes it possible to perform the combining process without decoding and re-encoding processes even between pictures of different encoding formats.

Next, a frame type conversion method in the frame type conversion means 104 will be described with reference to FIG. FIG.
Is the frame type conversion means 104 in the first embodiment.
5 is a flowchart illustrating a frame type conversion method of FIG.

First, in step S50, the frame type conversion means 104 sets the PCT of the picture header code 11
Code 15 is rewritten from a coding type indicating an I picture to a code word indicating a P picture. Next, in step S51, for each macroblock data 18, the MBTYPE code 24 of the macroblock header code 19 is set to I
The coding type indicating the intra macroblock of the picture is rewritten to a codeword indicating the intra macroblock of the P picture. Specifically, this code word is represented as shown in FIG. Here, FIG. 6 is an explanatory diagram illustrating code words of the coding type of the macroblocks of the I picture and the P picture.

As shown in FIG. 6, the MBTYPE code 2
No. 4 has codewords for specifying a quantization scale for the coding type of each macroblock and for using a standard quantization scale. For example, when the coding type of a macroblock of an I picture is rewritten to a codeword indicating an intra macroblock of a P picture, when the MBTYPE code 24 of the original I picture is “01”, the MBTYPE code 24 is “ 00000
Is converted to 1 ".

Similarly, when the standard quantization scale is used, the MBTYPE code 24 is converted from “1” to “00011” when the coding type of the macroblock of the I picture is converted. Regardless of which codeword is used, the amount of increase in the converted codeword is the same.

The processing of step S51 is performed on the MBTYPE codes 24 of all the macroblocks (step S52), and the coding type of the macroblock of the I picture is changed to a codeword indicating the intra macroblock of the P picture. rewrite.

As described above, even when the moving image data to be synthesized has a different encoding format, the PCT code 15 of the picture header code 11 and the MBTYPE code of the macro block header code 19 in the I picture are obtained by the frame type conversion means 104. 24 can be converted to a P-picture format to convert an I-picture into a P-picture, so that moving picture data can be synthesized without decoding and re-encoding.

As described above, the moving picture synthesizing apparatus and the moving picture synthesizing method of the first embodiment are provided with the frame type converting means 104 (frame type converting step) for converting an I picture into a P picture, and are going to synthesize. When the coding type of the moving image data includes the I picture format and the P picture format, the PCT code 15 and the MBTYPE code 24 are respectively rewritten from the coding type indicating the I picture to the code word indicating the P picture, and the I picture is changed. The image is converted into a P-picture and the image synthesizing means 103 (image synthesizing step)
, The moving image data whose encoding type is aligned in the P picture format is synthesized. This eliminates the need for variable-length decoding and re-encoding processing as in the related art, and can generate a synthesized moving image at high speed without deteriorating image quality.

[Second Embodiment] Next, a moving image combining apparatus and a moving image combining method according to a second embodiment of the present invention will be described. Here, the second embodiment corresponds to a moving image synthesizing apparatus according to claims 1, 8, 9 and 10 and a moving image synthesizing method according to claims 13, 20, 21 and 22 of the present invention. It is.

In the moving picture synthesizing apparatus of the present embodiment, the frame type converting means 104 of the moving picture synthesizing apparatus (first embodiment) shown in FIG. When converting to a codeword indicating an intra macroblock of a macroblock, a codeword (macroblock stuffing code (Macrob
lock Stuffing code).

The functions and operations of the components other than the frame type conversion means 104 in the present embodiment are the same as those in the first embodiment, so that detailed description will be omitted.

In the moving picture synthesizing apparatus shown in FIG. 1, the video signal 151 is transmitted to the image encoding means 1 as in the first embodiment.
01, and is temporarily stored in the image storage means 102. Next, image synthesizing means 1
Reference numeral 03 fetches one frame of data of each moving image data to be synthesized from the image storage unit 102 in order to generate data of one frame of the synthesized moving image.

At this time, if the coding type of the moving image data is different, the frame type conversion means 104 converts the coding type of the PCT code 15 into the P picture format, and sets the coding type of the macroblock of the I picture. Is rewritten to a codeword indicating an intra macroblock of a P picture, and an I picture is converted to a P picture. For example, each code word of the macroblock coding type has a bit configuration as shown in FIG. 6, and the data configuration after conversion into a codeword indicating an intra macroblock of a P picture is the original I word. This is a configuration in which the number of bits of the picture is increased by 4 bits.

At this time, the increase in the code length of 4 bits is 1
There is no particular problem in a processing system that processes data in units of bits. However, in a processing system that processes data in units of 8 bits, which is performed in many computers, a part in which the code length has changed All subsequent macroblock data must be subjected to a 4-bit bit shift process.

In order to solve such a problem, in the present embodiment,
The macroblock stuffing code (hereinafter referred to as MBSTUFFF) is set by the frame type conversion means 104 such that the increase in the code length of the I picture is a multiple of the data processing unit.
Code). That is, when the coding type of a macroblock of an I picture is rewritten to a codeword indicating an intra macroblock of a P picture, an MBSTUFF code, which is a codeword, is inserted at the head of the macroblock to increase the code length of the I picture. Since the amount becomes a multiple length of the data processing unit and the coding type of each macroblock can be converted according to the data processing unit, the coding information of the macroblock is described when the coding type of the macroblock is converted. Bit shift processing for codewords other than the first part is not required.

Hereinafter, a frame type conversion method of the frame type conversion means 104 according to the second embodiment will be described with reference to FIG.

In FIG. 7, first, in step S70, as in the frame type conversion means 104 of the first embodiment, the PCT code 1 indicating the coding type of the entire frame
5 is rewritten from an I picture to a codeword indicating a P picture. Next, in step S71, the coding type of each macroblock is rewritten from a codeword indicating an intra macroblock of an I picture to a codeword indicating an intra macroblock of a P picture. For example, when the code word in FIG. 6 is used, the code length is increased by 4 bits by rewriting the code word.

In this case, if the processing system processes data in units of 1 bit, such a change in the code length in units of bits does not cause any particular problem.
Since data is processed in units of bits, a 4-bit bit shift process must be performed on all macroblock data after the portion where the code length has changed.

Therefore, in step S73, when rewriting the MBTYPE code 24 indicating the encoding type of the macroblock, the frame type conversion means 104 of this embodiment adds the MBSTUFFF to the beginning of the macroblock data.
Insert four codes. This MBSTUF code is
In the MPEG standard, when the data amount is insufficient for a predetermined bit rate, it is a codeword that can be inserted in 11-bit units at the head of macroblock data. Therefore, the MBSTUFF code is changed to 4 in step S72.
44 bits of data are inserted by the individual insertion. Here, FIG. 8 is an explanatory diagram illustrating the data configuration of the macroblock data before and after the MBSTUFF code is inserted. FIG. 8A shows the data structure before the MBSTUFF code is inserted, and FIG. 8B shows the data structure after the MBSTUFF code is inserted.

As shown in FIG. 8, the code length of the macroblock is determined by the MB when converting the coding type of the macroblock.
By inserting the STFF code, the number of bits increased by 4
Due to the increase of 44 bits due to the insertion of the UFF code, the code length of the macroblock header code 19 increases to 48 bits as a whole. Since the increase amount of the entire code length is a multiple of 8 bits, the bit shift processing is unnecessary for the macroblock data 18 after the macroblock header code 19.

As described above, in the moving picture synthesizing apparatus and the moving picture synthesizing method according to the second embodiment, the frame type converting means 104 (frame type converting step)
When converting the codeword of each macroblock of an I picture into a codeword indicating an intra macroblock of a P picture,
A plurality of MBSTUFF codes are inserted at the head of the macro block. Thereby, the amount of increase in the code length of the I-picture format data can be a multiple of the data processing unit, and when converting the coding type of the macroblock, the head of the coding information of the macroblock is described. Data processing for codewords other than the part becomes unnecessary, and a synthesized moving image can be generated at high speed without deterioration in image quality.

In this embodiment, the processing system in which the data processing unit is 8 bits has been described. However, by inserting the MSBTUFF code into the processing system having any processing unit, the shift bit processing becomes unnecessary. It is possible to In this case, it is necessary to change the number of MBSTUF codes to be inserted according to each processing unit.

[Third Embodiment] Next, a moving image synthesizing apparatus and a moving image synthesizing method according to a third embodiment of the present invention will be described. Here, the third embodiment corresponds to a moving image synthesizing apparatus according to claims 1, 8, 9 and 11 of the present invention, and a moving image synthesizing method according to claims 13, 20, 21 and 23 of the present invention. It is.

In the moving picture synthesizing apparatus of the present embodiment, the frame type converting means 104 of the moving picture synthesizing apparatus (first embodiment) shown in FIG. When converting to a codeword indicating an intra macroblock, the codeword of the high-frequency component of the immediately preceding macroblock is converted. It should be noted that compared to the moving image synthesizing apparatus of the second embodiment, data processing can be performed at higher speed because the amount of added data is small.

The functions and operations of the components other than the frame type conversion means 104 in the present embodiment are the same as those in the first embodiment, so that detailed description will be omitted.

In the moving picture synthesizing apparatus shown in FIG. 1, as in the first embodiment, the video signal 151
01, and is temporarily stored in the image storage means 102. Next, image synthesizing means 1
Numeral 03 fetches data of one frame of each moving image data to be synthesized in order to generate data of one frame of the synthesized moving image. At this time, if the encoding type of the moving image data is different, the frame type conversion unit 104
The coding type of the PCT code 15 of the picture is converted into the P picture format, and the coding type of the macro block is rewritten to a code word indicating an intra macro block of the P picture.

The frame type conversion means 104 of this embodiment
Further, in order to eliminate the need for a bit shift process using the bits increased by the rewriting of the codeword, the high-frequency component of the immediately preceding macroblock is changed so that the amount of change in the code length of the I picture data matches the data processing unit. Replace codeword. That is, when converting the coding type of a macroblock of an I picture, a macroblock stuffing code, which is a codeword, is inserted at the head of the macroblock at the head of each slice data, and the other macroblocks are inserted. As for the block, the code word of the high-frequency component of the immediately preceding macroblock is replaced. As a result, the frame type conversion processing can be performed according to the data processing unit, and the bit shift processing for codewords other than the head part in which the coding information of the macroblock is described becomes unnecessary.

Here, the coding order of the macro blocks in the MPEG standard will be described with reference to FIG. FIG.
FIG. 9A is an explanatory diagram showing an encoding order of a code string of macroblock data, FIG. 9B is an explanatory diagram showing a DCT coefficient matrix generated by two-dimensional DCT transform, and FIG.
FIG. 9 is an explanatory diagram showing an encoding order in a code sequence of T coefficients.

In the MPEG standard, as shown in FIG. 9 (a), the macroblock data 18 has the above-mentioned MBAI 17 and MBTYPE at the beginning of the macroblock data.
A macroblock header code 19 indicating information of the entire macroblock such as 18, the luminance information of the macroblock of 16 × 16 pixels is divided into four 8 × 8 pixel blocks,
Four DCT transformed luminance YDCT coefficients 20, 16 × 1
The color difference information U and the color difference information V of the six pixels are converted into 8 × 8 pixel blocks by thinning processing or the like, and are subjected to DCT conversion.
Color difference UDCT coefficient 21 and color difference VDCT coefficient 22
Are encoded.

DCT obtained by each two-dimensional DCT transform
The coefficient is a DC of 8 × 8 as shown in FIG.
It is a T coefficient matrix, and is encoded in zigzag scan order as shown in FIG. The DCT coefficient matrix is scanned such that the high-frequency components (62, 63, 64 in the figure) are located last, so that a code string as shown in FIG. 9C is generated. In addition, as a scanning method of the DCT coefficient matrix, there is an alternate scanning method and the like.
Any scanning method may be used.

The high-frequency component located at the end of the code string of the macro block differs depending on the data, but rewriting the code word does not significantly affect the image quality. For this reason, in the present embodiment, the frame type conversion means 104
The code word of the high frequency component (62, 63, 64) shown in (c) is rewritten. This makes it possible to adjust the length of the macroblock data to a data processing unit length and perform data processing at a higher speed.

Hereinafter, a frame type conversion method in the frame type conversion means 104 will be described with reference to FIG.
Here, FIG. 10 is a flowchart illustrating a frame type conversion method of the frame type conversion unit 104 according to the third embodiment.

In the figure, first, in step S100, similarly to the frame type conversion means 104 of the first embodiment, a PCT code indicating the coding type of the entire frame is changed from an I-picture format to a code word indicating a P-picture format. Rewrite to In step S101, the coding type of the first macroblock of the slice data is rewritten from a codeword indicating an intra macroblock of an I picture to a codeword indicating an intra macroblock of a P picture.
02, in the same manner as in the moving picture synthesizing apparatus of the second embodiment, M
Insert four BSTUFF codes.

Next, in step S103, MBTYP is applied to macroblocks other than the head macroblock.
The E code is rewritten, and in step S104, the codeword of the high-frequency component of the immediately preceding macroblock is converted into a 4-bit shorter codeword or a 4-bit longer codeword.

FIG. 11 is an explanatory diagram illustrating the data configuration of macroblock data before and after rewriting the MBTYPE code and the code word of the high-frequency component. FIG. 11A shows the data structure before converting the coding type of the macroblock of the I picture.
1 (b) and 11 (c) are frames obtained by converting the coding type of the macroblock of the I picture and then converting the codeword of the high-frequency component of the immediately preceding macroblock into a 4-bit short codeword or a 4-bit long codeword. 1 shows the configuration.

As shown in FIG. 11B, when converting a high-frequency component codeword into a short 4-bit codeword, the code length of macroblock data is increased by 4 bits due to rewriting of the coding type. Since the code word of the high-frequency component of the immediately preceding macroblock is reduced by 4 bits, there is no change in the code length as a whole, and no bit shift processing occurs before and after the rewrite location.

As shown in FIG. 11 (c), when the code word of the high frequency component is converted into a code word longer by 4 bits, the code length of the macroblock data is increased by 4 bits by rewriting the coding type. However, since the code word of the high-frequency component of the immediately preceding macroblock increases by 4 bits, the code length increases by 8 bits as a whole, and no bit shift processing occurs before and after the rewrite location.

As described above, steps S103 and S1 are performed.
04 is performed on all the macroblocks in the slice (step S105), and when the coding type and the high-frequency component of all the macroblocks in the slice are rewritten, next, in step S106, all the slices in the picture are processed. It is determined whether or not processing has been performed on. If the processing has not been performed for all slices, the process returns to step S101, and the processing returns to step S101 to step S1 for all slices in the picture.
The processing in 05 is repeated until all slices in the picture have been processed.

As described above, in the moving picture synthesizing apparatus and the moving picture synthesizing method according to the third embodiment, the frame type conversion means 104 (frame type conversion step) sets the coding type of each macroblock of an I picture to P. When converting to a codeword indicating an intra macroblock of a picture,
The codeword of the high frequency component of the immediately preceding macroblock is converted. As a result, the amount of change in the code length of the I picture matches the data processing unit, and when converting the coding type of the macroblock, the data for the codeword other than the head portion in which the coding information of the macroblock is described. Processing becomes unnecessary, and a synthesized moving image can be generated at high speed without deterioration in image quality.

In the present embodiment, the code word of the high-frequency component of the immediately preceding macroblock is converted. However, the code word of the low-frequency component of the immediately following macroblock may be converted, and the present invention can be similarly implemented.

Further, in the present embodiment, the processing system in which the data processing unit is 8 bits has been described. However, the processing system having any processing unit converts the code word of the high-frequency component according to the processing unit. By doing so, it can be similarly implemented.

[Fourth Embodiment] Next, a moving image synthesizing apparatus and a moving image synthesizing method according to a fourth embodiment of the present invention will be described. Here, the fourth embodiment relates to a moving picture synthesizing apparatus according to claims 1, 2, 3, 4, 8, 9 and 12 of the present invention, and claims 13, 14, 15, 16, and 2 of the present invention.
This corresponds to the moving image synthesizing methods according to 0, 21 and 24.

FIG. 12 is a configuration diagram of a moving image synthesizing apparatus according to the fourth embodiment. The moving image synthesizing apparatus according to the present embodiment includes:
This is a configuration in which a marking code insertion unit 35, a code length adjustment unit 206, and a noise reduction unit 207 are further added to the moving picture synthesizing apparatus shown in FIG. 1, and a conversion process of a frame encoding type without performing variable length decoding. Can be specified, and the bit shift process for the data after the conversion position is not required, so that higher-speed processing can be performed.

In FIG. 12, a moving picture synthesizing apparatus according to the present embodiment comprises an image encoding means 201, an image storing means 202,
Image synthesis means 203 and frame type conversion means 204
And a marking code insertion means 2
05, code length adjusting means 206 and noise reducing means 20
7 is provided.

Note that the image storage means 2 in this embodiment is
02 and the image synthesizing means 203 are the same as those in the first, second and third embodiments, and therefore, detailed description will be omitted.

The marking code insertion means 205 inserts a marking code capable of identifying the boundary of macroblock data when the image coding means 201 performs variable length coding of the macroblock data. This marking code is one of the variable-length codes constituting the macroblock data, and is a code word longer by 4 bits than the multiple length of the data processing unit. The marking code insertion means 205 converts the marking code into an EOB indicating the end of the macroblock.
(End Of Block)
Enables detection of macroblock boundaries.

Further, the frame type conversion means 204 detects the marking code and converts the coding type of the macroblock of the I picture into a code word of the P picture format. That is, the marking code inserted into the macroblock by the marking code insertion means 205 is detected to detect the boundary of the macroblock, and the coding type of the macroblock of the I picture is converted into a codeword indicating the intra macroblock of the P picture. I do. As described above, by detecting the marking code, the position of the MBTYPE code of the I picture can be quickly and accurately found, so that the data processing speed when converting the coding type of the macroblock is increased.

Further, the code length adjusting means 206 controls the coding of the high frequency component of the macroblock so that the code length of the macroblock data becomes a data processing unit when the image coding means 201 performs variable length coding of the macroblock. Change words. That is, when the coding type of the macroblock is changed by changing the codeword of the high-frequency component of the macroblock so that the code length of the macroblock increased by the insertion of the marking code becomes a data processing unit. Is unnecessary.

The noise reduction unit 207 converts the high-frequency component of the quantization matrix of the moving image data generated by the image synthesis unit 203 into a predetermined small value. That is, in order to increase the data processing speed, video noise generated by the codeword added to the high-frequency component of the macroblock by the marking code insertion unit 205 and the code length adjustment unit 206 is reduced, and the image is degraded. Is preventing.

The following describes the image encoding method of the image encoding means 201, the marking code insertion method of the marking code insertion means 205, and the code length adjustment method of the code length adjustment means 206 in the moving picture synthesizing apparatus of this embodiment.
This will be described with reference to FIGS. Here, FIG.
14 is a flowchart showing an image encoding method of the image encoding unit 201 in the embodiment, and FIG.
06 is an explanatory view exemplifying a code length adjustment table used for adjusting the code length, and FIG. 15 is an explanatory view exemplifying the data configuration of macroblock data on which a marking code has been inserted and the code length has been adjusted.

In FIG. 13, first, at step S130
Then, the image encoding unit 201 divides the video signal 251 into macroblock units, and encodes the generated luminance YDCT coefficient and color difference UDCT coefficient after DCT transform.
Next, in step S131, the first to 60th components of the chrominance VDCT coefficients are variable-length coded, and
In 132, the level of the 61st component of the chrominance VDCT coefficient is set to "1" to perform variable length coding.

In the step S133, the code length adjusting means 2
06 checks the code length N of the macroblock data at this point, and determines the code length N of the entire macroblock data, including the marking code added last, with respect to the multiple length of 8 bits which is the data processing unit. Ask if bits are missing. That is, this shortage length is obtained by calculating 8- (code length N + marking code length + EOB code) mod8. Here "MmodN"
Is an operation for calculating a remainder obtained by dividing a natural number M by a natural number N.

Next, in step S134, referring to the code length adjustment table shown in FIG.
Insert a code length adjustment code. As shown in FIG. 14, the code length adjustment code is generated by combining at most two variable length codes whose run is 0 or 1 and whose level is relatively small. As a result, even if a marking code is inserted later, a bit shift process due to an increase in the number of bits becomes unnecessary, and a process of converting the coding type of a macroblock can be performed at high speed.

In step S135, the marking code is inserted by the marking code insertion means 205. Finally, in step S136, the EOB code indicating the end of the macro block is inserted.

The marking code is one of variable length codes constituting the macroblock data, and is arranged immediately before the EOE code indicating the end of the macroblock as shown in FIG. For example, "0000010000001
“0101010” can be used as a marking code, and is “00” obtained by combining the EOB code “10”.
Since the code word 00000100001010101010 ″ exists only at the last position of the macro block in the macro block data, the boundary of the macro block can be found by detecting this code word.

The marking code is configured as a code word that is 4 bits longer than the multiple length of the data processing unit of the computer. In the above example, the macro block data has a code length of 2 bits.
It is 0 bit, and is a code that is 4 bits longer than twice the 8 bits that are the data processing unit. Accordingly, even if the coding type of the macroblock of the I picture is converted into a codeword indicating an intra macroblock of the P picture, even if the code length of 4 bits is increased, by removing this marking code, 16 bits as a whole can be obtained. And the bit shift processing for codewords other than the head part in which the coding information of the macroblock is described becomes unnecessary.

FIG. 15 shows a data configuration in which code words (code length adjustment codes and marking codes) have been inserted into the high frequency components of the macroblock data in steps S133 to S135. As shown in FIG. 15, in the macroblock data, the marking code is inserted before the EOB so that the boundary of the macroblock can be identified, and the EOB is positioned at the byte boundary by rewriting the code length adjustment code. Has been adjusted.

Conventionally, since macroblock data has been variable-length coded, it has been necessary to sequentially perform variable-length decoding from the beginning in order to detect macroblock boundaries.
In the present embodiment, as described above, by inserting the marking code in step S135, it is possible to detect a macroblock boundary only by a simple process of comparing codes. Steps S133 and S1
By inserting the code length adjustment code of No. 34, the bit shift process by inserting the marking code is not required, and the process of converting the coding type of the macroblock can be performed at high speed.

Next, a description will be given of a frame type conversion method of the frame type conversion means 204 of the present embodiment. FIG.
6 is a frame type converting means 20 in the fourth embodiment.
9 is a flowchart illustrating a frame type conversion method of No. 4;

In FIG. 16, first, at step S160
Then, as in the first embodiment, the PCT code is converted by the frame type conversion means 204 into a codeword indicating a P picture. Next, in step S161, four MBSTUFF codes are inserted into the head macroblock of each slice, as in the second embodiment.

For other macroblocks, in step S163, a marking code is detected to detect a macroblock boundary. Since the tail of the code word obtained by combining the marking code and the EOB code exists on the byte boundary as shown in FIG. 15, it can be easily detected by the byte-by-byte comparison processing. Then, in step S164, the MBTYPE code indicating the coding type of the macroblock is rewritten, and in step S165, the marking code is deleted.

Steps S163 to S165
Is performed on all the macroblocks in the slice (step S166), and subsequently, step S161
Steps S166 to S166 are performed on all slices in the picture (step S167).

FIG. 17 is an explanatory diagram illustrating the data structure of the macroblock data before and after the conversion of the MBTYPE code and the deletion of the marking code. FIG.
FIG. 7A shows macroblock data before MBTYPE code conversion, and FIG. 17B shows macroblock data after MBTYPE code conversion and marking code deletion. As shown in FIG. 17B, the code length of the macroblock data is obtained by converting the MBTYPE code (that is, from “01” to “000001”).
And the marking code is deleted, so that the entire code word of the macroblock is shortened by 2 bytes. As a result, even if the marking code is deleted, a bit unit shift does not occur before and after the rewriting position of the coding type of the macroblock, so that the coding type of the macroblock can be converted without performing a bit shift process. become.

As described above, the marking code insertion means 2
05 and the code length adjusting means 206 make it possible to detect the macroblock boundary without performing variable-length decoding when rewriting the coding type of the macroblock, thereby making the necessary bit shift processing unnecessary. In addition, the rewriting of the high-frequency component of the macroblock data causes a problem that video noise is generated in the moving image data synthesized by the image synthesizing unit 203, and the image is deteriorated.

In order to deal with such a problem, in this embodiment, the noise reduction unit 207 uses the image synthesizing unit 2.
03 converts the high frequency component of the quantization matrix of the moving image data generated into a predetermined small value, and generates an image generated by a code word added to the high frequency component of the macroblock by the marking code insertion unit 205 and the code length adjustment unit 206. Noise is reduced to prevent image degradation.

Hereinafter, the noise reduction means 20 of this embodiment will be described.
7 will be described with reference to FIG. FIG. 18 is a flowchart illustrating a noise reduction method of the noise reduction unit 207 according to the fourth embodiment. FIG. 19 is an explanatory diagram illustrating the code configuration of the quantization matrix.

In FIG. 18, first, the noise reduction unit 2
07 is the MPEG synthesized by the image synthesis means 203
In step S180, a sequence header code is detected from the input code sequence, and in step S181, it is checked whether the detected code includes a quantization matrix code.

If the quantization matrix code is included, the flow advances to step S183. If no quantization matrix code is included, the process proceeds to step S182, where a standard quantization matrix code defined by the MPEG standard is generated and inserted into the sequence header. In this quantization matrix, each matrix component shown in FIG. 19A is encoded in zigzag scan order, and as shown in FIG. 19B, each component is encoded with a fixed length of 8 bits.

In step S183, the codeword corresponding to the 61st to 64th components of the quantization matrix code is set to “0”.
Or, rewrite it to “1”. By adding such a quantization matrix code to the code string, the high frequency component of each macroblock is inversely quantized as a very small value during reproduction, so that the code length adjusting means 206 and the marking code inserting means 205 It is possible to prevent the image quality from deteriorating due to the high-frequency component codeword (code length adjustment code or good marking code) added in (1). Here, the high frequency component is from the 61st component to the 64th component, but is not limited to this.

As described above, in the moving picture synthesizing apparatus and the moving picture synthesizing method of the fourth embodiment, the code length adjusting means 206 (code length adjusting step) for changing the code word of the high frequency component of the macro block, Marking code insertion means 205 (marking insertion step) for inserting a marking code capable of identifying a boundary of block data, and noise reduction means 207 for converting a high-frequency component of a quantization matrix of generated moving image data into a predetermined small value. (Noise reduction step), and when the macroblock data is subjected to variable-length coding, a marking code is used as a code word of a high-frequency component of each macroblock data immediately before an EOB (End Of Block) code indicating the end of the macroblock. And insert
The code length adjusting means 206 (code length adjusting step) rewrites the code word of the high frequency component of the macroblock into a code length adjusting code, and the marking code inserting means 205 (marking inserting step) and the code length adjusting means 206 (code length adjusting step). In this way, the high frequency component of the quantization matrix is converted into a predetermined small value in the moving image data in which the code word is added to the high frequency component.

Thus, when converting the coding type of the macroblock, it is possible to specify the conversion part without performing variable length decoding, and even if a marking code is inserted, the code length of the macroblock data is not changed. It becomes a processing unit, and when converting the coding type of a macroblock, it is possible to specify the conversion part only by comparison operation in the data processing unit without performing variable length decoding, A composite moving image can be generated at high speed without deterioration in image quality caused by a marking code or a code length adjustment code added to a component.

Also, the marking code insertion means 205
In the (marking code insertion step), a marking code having a code length such that the reduction amount of the code length when converting the coding type of the macroblock is a multiple of the data processing unit is inserted, and the coding of the macroblock is performed. When converting the type, the inserted marking code is deleted. As a result, even if the marking code is deleted, the increase or decrease in the code length due to the coding type conversion matches the data processing unit, and the data processing for code words other than the head part in which the coding information of the macroblock is described. Is unnecessary, and a synthesized moving image can be generated at higher speed.

In the present embodiment, a marking code that is 4 bits longer than the multiple of the data processing unit of the computer is used. However, it is assumed that only the rewrite position is detected using a code word of an arbitrary length. It is also possible to adopt a mode in combination with the second embodiment.

In this embodiment, the code length adjusting means 2
In step 06, the code length of the macroblock data is adjusted in units of data processing. However, this code length adjusting means is omitted, and a marking code is inserted at the end. Even if the comparison is performed by using the unit comparison processing, the present invention can be similarly implemented.

In this embodiment, one variable-length code is used as a marking code. However, even if a plurality of variable-length codes in a codebook are combined to generate a marking code, the same applies to the present embodiment. It is possible.

Further, in the present embodiment, the description has been given of the processing system in which the data processing unit is 8-bit. By converting using a marking code, the present invention can be similarly implemented.

In this embodiment, the noise reduction means 2
07, the deterioration of the image quality is suppressed by the code words added by the code length adjusting means 206 and the marking code inserting means 205. However, a code word having a low level is selected as the code length adjusting code and the marking code. For example, the noise reduction unit 207 can be omitted.

[Fifth Embodiment] Next, a moving picture composing apparatus and a moving picture composing method according to a fifth embodiment of the present invention will be described. Here, the fifth embodiment corresponds to the moving image synthesizing apparatus according to claims 1 and 5 and the moving image synthesizing method according to claims 13 and 17 of the present invention.

FIG. 20 is a configuration diagram of a moving image synthesizing apparatus according to the fifth embodiment. The present embodiment receives frame data encoded in the MPEG format transmitted at a maximum of 30 frames per second (N = 30), and inserts a skip frame when there is no data to perform a combining process. Thus, a moving image of 30 frames per second conforming to the MPEG standard is synthesized at high speed in real time.

In the figure, the moving picture synthesizing apparatus according to the present embodiment includes image buffers 302a, 302b, 302c, 3
02d, a skip frame generating unit 308, an image synthesizing unit 303, and a frame type converting unit 304.

Here, the image buffers 302a, 302
b, 302c and 302d are I-codes conforming to the MPEG standard.
A plurality of image buffers to which a maximum of 30 frames per second of moving image data including an I picture and a P picture encoded in a picture format and a P picture format are input. In other words, up to 30 I-pictures encoded in MPEG format or P-pictures encoding difference information from I-pictures are transmitted to each image buffer at a maximum of 30 pictures per second.
The image is temporarily stored until the image synthesizing unit 303 takes out the image.

The skip frame generating means 308
Generates a skip frame in the P-picture format when there is no moving image data when extracting moving image data from a plurality of image buffers. That is, when there is no moving image data when moving image data is taken out from the image buffer,
A motion vector is "0", P-picture format moving image data without difference information is generated, and it is possible to combine moving image data in real time.

Also, the frame type conversion means 304
Convert a picture to a P picture. That is, similarly to the frame type conversion unit 104 according to the first embodiment, the frame type conversion unit 304 combines the coding types of the PCT code of the I picture and the MBTYPE code when combining moving image data of different coding types. To the P picture format.

The image synthesizing means 303 extracts moving image data from the image buffers 302a, 302b, 302c, and 302d and performs a synthesizing process once every 1/30 second. That is, when combining moving image data of different encoding types, the frame type conversion means 304
The picture is converted to a P picture, and the moving picture data arranged in the P picture format is synthesized. At this time, if there is no moving image data from which the moving image data is to be extracted from the image buffers 302a, 302b, 302c, 302d, a skip frame is generated by the skip frame generating means 308, and the generated skip frame is combined. By performing the processing, a synthesized moving image 355 for 30 frames per second conforming to the MPEG standard can be generated at high speed in real time.

The following describes the image synthesizing means 303 of the present embodiment.
Will be described in detail with reference to FIG. Here, FIG. 21 is a flowchart illustrating the image combining process of the image combining unit 303 according to the fifth embodiment.

In FIG. 21, first, the image synthesizing means 30
3 is that each image buffer 302
It is checked whether moving image data exists in a, 302b, 302c, and 302d. If there is moving image data, the moving image data is extracted from the image buffer in step S211. If there is no moving image data in the image buffer, the skip frame generating means 30
8 generates data (skip frame) for displaying the same image as the previous frame during reproduction. The skip frame data generated by the skip frame generation means 308 is a P picture having a motion prediction value of “0” and no difference information. Then, in step S213, the above steps S210 to S212 are repeated until all the buffers are processed.

Next, in step S214, the coding type of each frame is determined, and if a different coding type is included, the I-picture is converted to a P-picture in step S215 to change the coding type of all frames. After the alignment, the combining process is performed in step S216. Also,
In step S214, if all the frames of the moving image data have the same encoding type, the process proceeds to step S216, and the combining process is performed. In the synthesizing process, the image synthesizing unit 303 performs the processing in step S210 within 1/30 second.
To S215 are completed, and the above-described step S216 is performed once every 1/30 second, so that the moving image data can be synthesized in real time.

As described above, in the moving image synthesizing apparatus and the moving image synthesizing method according to the fifth embodiment, a plurality of image buffers 302a, 302b, 302c, and 302c to which a maximum of 30 frames per second of moving image data are input. 302d (image input step), and a skip frame generation means 308 (skip frame generation step) for generating a P-picture format skip frame, and the image synthesis means 303 (image synthesis step) converts the moving image data of each image buffer. If there is no moving image data at the time of combining, a skip frame is inserted and the combining process is performed only once every 1/30 second. As a result, it is possible to generate a synthesized moving image for 30 frames per second at high speed in real time in accordance with the MPEG standard.

In this embodiment, the frame type conversion means which operates in the same manner as in the first embodiment is used. However, the present embodiment is constructed in combination with the frame type conversion means of the second, third and fourth embodiments. Can be similarly implemented.

The number of input data is not limited to four, and the number of inputs can be changed as long as the synthesis processing can be performed in real time.

[Sixth Embodiment] Next, a moving image synthesizing apparatus and a moving image synthesizing method according to a sixth embodiment of the present invention will be described. Here, the sixth embodiment is directed to a moving image synthesizing apparatus according to claims 1, 6 and 7 of the present invention, and
This corresponds to the moving image composing method according to claims 13, 18 and 19.

FIG. 22 is a configuration diagram of a moving image synthesizing apparatus according to the sixth embodiment. This embodiment further includes a code amount adjusting unit 409 as compared with the fifth embodiment (see FIG. 20), and when the combined moving image data exceeds the set code amount, the I picture is set to a skip frame. Replacement,
If it is insufficient, a supplementary bit is inserted. The functions and operations of the components other than the code amount adjusting unit 409 in the present embodiment are the same as those in the fifth embodiment, and thus detailed description will be omitted.

In FIG. 22, code amount adjusting means 409
Is adjusted so that the coding amount of the moving image data synthesized by the image synthesizing unit 403 is constant. That is, the data amount of all frames to be combined is calculated, and the average data amount after combination exceeds a prescribed upper limit, that is, the data amount of combined moving image data combined by the image combining unit 403 is larger than a predetermined range. In this case, some data is replaced with skip frames. Also, if the value is less than the specified lower limit, that is, if the value is smaller than the predetermined range, meaningless supplementary data is added to the end of the combined moving image data,
A composite moving image 455 having a predetermined code amount is generated.

FIG. 23 is a flowchart illustrating a code amount adjusting method of the code amount adjusting unit 409 according to the sixth embodiment. In the figure, first, step S2
In steps 30 to S233, as in the fifth embodiment, each of the image buffers 302a, 302b, 302c, 3
02d, data for one frame of the synthesized moving image is extracted. That is, each of the image buffers 302a, 302b, 30
It is checked whether or not there is moving image data in 2c and 302, and if so, the moving image data is extracted from the image buffer. If there is no moving image data, skip frame generation means 308 generates a skip frame. This skip frame is a P picture having a motion vector of “0” and no difference information.

Here, the MPEG standard (in particular, MPEG1
Standard), the average data amount of a moving image must not fluctuate greatly. If the average data amount exceeds a specified range, a buffer overflow occurs in a decoder, and the average data amount is reduced to a specified value. If the number is smaller than the range, buffer underflow occurs in the decoder, and normal reproduction of a moving image cannot be performed.

In the present embodiment, in step S235, the entire data amount of the input moving image data is checked by the code amount adjusting means 409, and it is checked whether the data amount exceeds a predetermined range. If it is over, any one of the I pictures input in step S234 is replaced with a skip frame, and the process returns to step S235. The processing operations in steps S234 and S235 are repeated until the data amount falls within a predetermined range. Therefore, it is possible to prevent buffer underflow in the decoder that occurs when the average data amount is smaller than the specified range.

Steps S236 to S23
In step 8, as in the fifth embodiment, the coding types of all frames are checked, and if frames of different coding types are included, the I picture is converted to a P picture. Then, all frames having the same encoding type are synthesized.

Next, in step S239, the code amount adjusting means 409 checks the data amount of the frame of the synthesized moving image data. If the data amount is less than the predetermined range, the code amount adjusting means 409 returns to the end of the frame data in step S240. The supplementary data is added, and the data amount is increased so as to fall within a predetermined range. Here, according to the MPEG standard, a codeword for compensating for the data amount can be inserted between frames in order to compensate for the lack of the data amount. Therefore, this codeword is used as supplementary data. As a result, it is possible to prevent a buffer overflow in the decoder that occurs when the average data amount exceeds a specified range.

FIG. 24 exemplifies a frame configuration before and after the adjustment of the code amount. 24A shows the frame configuration before adjustment, FIG. 24B shows the frame configuration after adjustment, and FIG. 24C shows the transition of the code amount before and after adjustment. Note that the upper limit and the lower limit in FIG. 24C mean that the data amount of each frame must fall within the range. In the figure, I indicates a frame in the I picture format, and S indicates a skip frame.

In the moving image data before adjustment shown in FIG. 24A, the data amount is within a predetermined range from the first frame to the third frame, and can be normally reproduced. But 4
In the frame, as shown in FIG. 24 (c), the data amount before adjustment is large, and cannot be reproduced because an overflow occurs. In the fifth frame, since the data amount is too small, an underflow occurs and the reproduction cannot be performed.

Therefore, in this embodiment, as shown in FIG. 24B, the lower left I-picture of the fourth frame is converted into a skip frame, and supplementary data is added to the end of the fifth frame. As a result, as shown in the adjusted graph of FIG. 24C, the data amount of all the frames falls within a predetermined range, so that the data can be normally reproduced by the decoder. As a result, moving images can be synthesized at high speed without causing overflow and underflow at the time of reproduction, and a smooth moving image without delay or jitter can be generated at the time of reproduction due to a constant code amount. it can.

As described above, in the moving image synthesizing apparatus and the moving image synthesizing method according to the sixth embodiment, the code amount adjusting means 409 (the code amount adjusting means) for adjusting the code amount of the synthesized moving image data to be constant. Adjusting step) and a skip frame generating means 308 (skip frame generating step) for generating a skip frame in a P picture format. If the data amount of the synthesized moving image data is larger than a predetermined range, the I picture Is replaced with a skip frame, and if it is smaller than a predetermined range, predetermined supplementary data is added after the moving image data. As a result, a smooth synthesized moving image without delay or jitter can be generated at high speed without causing overflow and underflow during reproduction.

[0177]

As described above, according to the moving picture synthesizing apparatus, the moving picture synthesizing method and the recording medium of the present invention, the moving picture data to be synthesized has different coding types of I picture format and P picture format. Is included, the I-picture format data is converted to P-picture format data by the frame type conversion means (frame type conversion step), and the moving image data whose encoding type is aligned to the P-picture format is converted to the image synthesis means (image synthesis means). Step), there is no need to perform variable-length decoding and re-encoding processing as in the prior art, and an excellent moving image synthesis capable of generating a synthesized moving image at high speed without deterioration in image quality. An apparatus, a moving image combining method, and a recording medium can be realized.

According to the present invention, when converting the coding type of a macroblock by the frame type conversion means (frame type conversion step), a plurality of macroblock stuffing codes are inserted at the head of the macroblock. Since the amount of increase in the code length of the I-picture format data is set to be a multiple of the data processing unit of the processing system, when converting the coding type of the macro block using a processing system such as a computer, This eliminates the need for data processing for codewords other than the head part in which the coding information is described, and can generate a synthesized moving image at high speed without deterioration in image quality.

According to the present invention, when the coding type of a macroblock is converted by the frame type converting means (frame type converting step), the high-frequency component of the immediately preceding macroblock or the low-frequency component of the immediately following macroblock is converted. When the coding type of the macroblock is converted using a processing system such as a computer, the amount of change in the code length of the I-picture format data matches the data processing unit of the processing system. This eliminates the need for data processing for codewords other than the head part in which the coding information of the macroblock is described, and can generate a synthesized moving image at high speed without deterioration in image quality.

Further, according to the present invention, when converting the encoding type of a macroblock by the frame type conversion means (frame type conversion step), the high frequency component of the rewritten moving image data quantization matrix is reduced by the noise reduction means. Since the value is converted to a predetermined small value by the (noise reduction step), it is possible to suppress deterioration in image quality caused by a high-frequency component rewritten for high speed during reproduction.

Further, according to the moving picture synthesizing apparatus, the moving picture synthesizing method and the recording medium of the present invention, when moving picture data to be synthesized includes different coding types of the I picture format and the P picture format, When the coding means (image coding step) performs variable length coding of the macroblock data, the marking code insertion means (marking insertion step) identifies a macroblock data boundary as a high-frequency component codeword of each macroblock data. A possible marking code is inserted, and at the time of synthesis, the I-picture format data is converted into P-picture format data by a frame type conversion means (frame type conversion step), and the encoding type is aligned with the P-picture format. Moving image data is synthesized by the image synthesizing means (image synthesizing step). When converting the coding type of a black block, it is possible to specify the conversion point without performing variable length decoding, and there is no need to perform variable length decoding and re-encoding processing as in the past, so image quality It is possible to realize an excellent moving image synthesizing apparatus, a moving image synthesizing method, and a recording medium that can generate a synthesized moving image at higher speed without deterioration of the moving image.

Further, according to the present invention, the amount of reduction in the code length when converting the coding type of the macroblock by the marking code insertion means (marking code insertion step) is set to a multiple of the data processing unit of the processing system. Inserting a marking code having a code length such that the marking code inserted is deleted when converting the coding type of the macroblock. It is possible to specify the conversion part without performing long decoding, and by deleting the inserted marking code, the increase or decrease of the code length due to the conversion of the coding type matches the data processing unit of the processing system. Data processing is not required for codewords other than the head part where the block coding information is described, and a synthesized moving image is generated more quickly Rukoto can.

According to the present invention, when macroblock data is subjected to variable length encoding, the codeword of the high frequency component of the macroblock is changed by the code length adjusting means (code length adjusting step). Even if a marking code is inserted, the code length of the macroblock data becomes the data processing unit of the processing system.When converting the encoding type of the macroblock, only the comparison operation in the data processing unit is performed without performing variable length decoding. It is possible to specify the conversion part, and it becomes unnecessary to perform data processing on codewords other than the head part in which the coding information of the macroblock is described, and it is possible to generate a synthesized moving image at higher speed.

Further, according to the present invention, with respect to the moving image data in which the code word has been added to the high frequency component by the marking code inserting means and the code length adjusting means, the noise reducing means (noise reducing step) uses the high frequency component of the quantization matrix. Is converted to a predetermined small value, so that the deterioration of the image quality caused by the marking code or code length adjustment code added to the high-frequency component for high speed during reproduction is reduced,
A composite moving image can be generated at high speed.

According to the moving picture synthesizing apparatus, the moving picture synthesizing method and the recording medium of the present invention, a plurality of image buffers (image input steps) into which a maximum of N frames of moving image data are input per second, A skip frame generating means (skip frame generating step) for generating a skip frame in a P-picture format, wherein there is no moving image data when the moving image data of each image buffer is synthesized by the image synthesizing means (image synthesizing step). In such a case, a skipping frame is inserted to perform the synthesizing process once every Nth of a second, and the moving image data to be synthesized includes different coding types of the I picture format and the P picture format. The I-picture format data is converted to P-picture format data by frame type conversion means (frame type conversion step) Since the moving image data whose type is arranged in the P picture format is synthesized by the image synthesizing means (image synthesizing step), it is not necessary to perform variable-length decoding and re-encoding processing as in the related art, and the image quality deteriorates. An excellent moving picture synthesizing apparatus and a moving picture synthesizing apparatus capable of generating a synthesized moving picture at high speed without any trouble and capable of generating a synthesized moving picture for N frames per second at a high speed in accordance with the MPEG standard. A method and a recording medium can be realized.

Further, according to the present invention, there is provided a code amount adjusting means (code amount adjusting step) for adjusting the code amount of the combined moving image data to be constant, and the data of the combined moving image data is provided. If the amount is larger than the predetermined range, the I-picture format data is replaced with a skip frame.If the amount is smaller than the predetermined range, predetermined supplementary data is added after the moving image data. It is possible to generate a smooth synthesized moving image without delay or jitter at high speed without causing overflow and underflow.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a moving image synthesizing apparatus according to a first embodiment.

FIG. 2 is an explanatory diagram illustrating a compression encoding method in an MPEG encoder.

FIG. 3 is an explanatory diagram illustrating a data structure of one frame in the MPEG format.

FIG. 4 is a flowchart illustrating an image combining process of an image combining unit according to the first embodiment.

FIG. 5 is a flowchart illustrating a frame type conversion method of a frame type conversion unit according to the first embodiment.

FIG. 6 is an explanatory diagram exemplifying a code word of a coding type of a macroblock of an I picture and a P picture.

FIG. 7 is a flowchart illustrating a frame type conversion method of a frame type conversion unit according to the second embodiment.

FIG. 8 is an explanatory diagram exemplifying a data configuration of macroblock data before and after an MBSTUFF code is inserted;

FIG. 9 is an explanatory diagram illustrating an encoding order of macroblocks in the MPEG standard.

FIG. 10 is a flowchart illustrating a frame type conversion method of a frame type conversion unit according to the third embodiment.

FIG. 11 is an explanatory diagram illustrating a data configuration of macroblock data before and after rewriting an MBTYPE code and a code word of a high-frequency component.

FIG. 12 is a configuration diagram of a moving image synthesizing apparatus according to a fourth embodiment.

FIG. 13 is a flowchart illustrating an image encoding method of an image encoding unit according to a fourth embodiment.

FIG. 14 is an explanatory diagram exemplifying a code length adjustment table used by a code length adjusting unit for adjusting a code length;

FIG. 15 is an explanatory diagram illustrating a data configuration of macroblock data in which a marking code is inserted and a code length is adjusted in a fourth embodiment.

FIG. 16 is a flowchart illustrating a frame type conversion method of a frame type conversion unit according to the fourth embodiment.

FIG. 17 is an explanatory diagram illustrating a data configuration of macroblock data before and after MBTYPE code conversion and marking code deletion.

FIG. 18 is a flowchart illustrating a noise reduction method of a noise reduction unit according to the fourth embodiment.

FIG. 19 is an explanatory diagram illustrating a code configuration of a quantization matrix.

FIG. 20 is a configuration diagram of a moving image synthesizing apparatus according to a fifth embodiment.

FIG. 21 is a flowchart illustrating an image combining process of an image combining unit according to the fifth embodiment.

FIG. 22 is a configuration diagram of a moving image synthesizing apparatus according to a sixth embodiment.

FIG. 23 is a flowchart illustrating a code amount adjusting method of a code amount adjusting unit according to the sixth embodiment.

FIG. 24 is an explanatory diagram illustrating a frame configuration before and after a code amount adjustment is performed.

FIG. 25 is an explanatory diagram of a moving image synthesizing apparatus according to a first conventional example.

FIG. 26 is an explanatory diagram of a moving image synthesizing apparatus according to a second conventional example.

[Explanation of symbols]

Reference Signs List 11 picture header code 12 slice data 13 PSC code 14 TR code 15 PCT code 16 VD code 17 slice header coder 18 macro block data 19 macro block header code 20 luminance YDCT coefficient code 21 color difference UDCT coefficient code 22 color difference VDCT coefficient code 23 MBAI code 24 MBTYPE code 25 QS code 101, 201 Image coding means 102, 202 Image storage means 103, 203, 303, 403 Image synthesis means 104, 204, 304 Frame type conversion means 151, 251 Video signal 153 Moving picture data 155, 256 , 355, 455 Synthetic moving image 205 Marking code insertion means 206 Code length adjustment means 207 Noise reduction means 255 Code string in MPEG format 302 a, 302b, 302c, 302d Image buffer 308 Skip frame generating means 409 Code amount adjusting means

 ──────────────────────────────────────────────────続 き Continued on front page F term (reference) 5B057 BA02 CA16 CB18 CE08 CG05 CG07 5C053 FA14 FA27 GB07 GB23 GB29 GB32 GB38 KA04 KA24 LA01 5C059 KK37 KK39 MA00 MA05 MA23 MC11 ME01 PP04 RB01 RB12 RC09 SS06 SS11 SS07 TA27 TA43 UA43 UA05 UA34 UA38 UA39

Claims (25)

[Claims] 1. An image encoding means for encoding an input video signal as moving picture data including an I picture format and a P picture format conforming to the MPEG standard; an image storing means for storing the moving picture data; Frame type conversion means for converting the I picture format data of the video data into the P picture format data; and image synthesis means for taking out the video data from the image storage means and performing synthesis processing. When synthesizing moving image data containing
The moving image synthesizing device, wherein the image synthesizing unit synthesizes moving image data including encoded or converted P-picture format data. 2. An image encoding means for encoding an input video signal as moving image data including an I picture format and a P picture format conforming to the MPEG standard, and wherein said image encoding means converts macroblock data into variable length data. At the time of encoding, a marking code insertion unit that inserts a marking code capable of identifying a boundary of the macroblock data, an image storage unit that stores the moving image data, and a marking code of the moving image data is detected. Said I
Set the encoding type of the macroblock of the picture format data to P
When synthesizing moving image data including different coding types, the frame type converting unit converts the image data into a code word in a picture format, and an image synthesizing unit that extracts moving image data from the image storage unit and performs a synthesizing process. To
The moving image synthesizing device, wherein the image synthesizing unit synthesizes moving image data including encoded or converted P-picture format data. 3. A marking code inserted by the marking code inserting means, wherein the amount of reduction in code length when the coding type of a macroblock is converted by the frame type converting means is a multiple length of a data processing unit of a processing system. The moving image according to claim 2, wherein the frame type converting unit deletes the marking code when converting the coding type of the macroblock. Synthesizer. 4. A codeword for a high-frequency component of a macroblock such that when the image coding means performs variable-length coding on the macroblock data, the code length of the macroblock data is a data processing unit of a processing system. 4. The moving picture synthesizing apparatus according to claim 2, further comprising a code length adjusting means for changing the length of the moving image. 5. A maximum of N frames (N is a positive integer) of moving image data including I picture format data and P picture format data encoded in an I picture format and a P picture format conforming to the MPEG standard. A plurality of input image buffers; a skip frame generating means for generating the P-picture format skip frame when there is no moving image data when extracting moving image data from the plurality of image buffers; Frame type conversion means for converting the I picture format data into the P picture format data; and extracting moving image data from the plurality of image buffers,
Image synthesizing means for performing a synthesizing process once every Nth of a second;
Wherein, when synthesizing moving image data including different encoding types, the image synthesizing means synthesizes moving image data including coded or converted P-picture format data. apparatus. 6. The moving picture synthesizing apparatus according to claim 5, further comprising code amount adjusting means for adjusting the coding amount of the moving image data synthesized by said image synthesizing means to be constant. 7. The code amount adjusting unit, when the data amount of the moving image data synthesized by the image synthesizing unit is larger than a predetermined range, replaces the I-picture format data with the skip frame, and 7. The moving picture synthesizing apparatus according to claim 6, wherein when the size is smaller than the moving picture data, predetermined supplementary data is added after the moving picture data. 8. The apparatus according to claim 1, wherein said frame type conversion means converts the coding type of the frame of the I picture format data into a code word of a P picture format.
8. The moving picture synthesizing apparatus according to 2, 3, 4, 5, 6, or 7. 9. The method according to claim 1, wherein said frame type conversion means converts a coding type of a macroblock of said I picture format data into a code word of P picture format. , 6, 7 or 8. 10. The frame type conversion means according to claim 1, wherein
5. A macroblock stuffing code is inserted so that the amount of increase in the code length of picture format data is a multiple of the data processing unit of the processing system. 5,6,7,8
Or the moving picture synthesizing apparatus according to 9. 11. The frame type conversion means according to claim 1, wherein
The codeword of the high-frequency component of the immediately preceding macroblock or the low-frequency component of the immediately succeeding macroblock such that the amount of change in the code length of the picture format data matches the data processing unit of the processing system. 1,2,3,
The moving picture synthesizing apparatus according to 4, 5, 6, 7, 8, 9, or 10. 12. The apparatus according to claim 1, further comprising noise reduction means for converting a high frequency component of a quantization matrix of the moving image data generated by said image synthesis means into a predetermined small value. , 5, 6, 7, 8, 9, 10, or 11. 13. An image encoding step of encoding an input video signal as moving image data including an I picture format and a P picture format conforming to the MPEG standard; an image storing step of storing the moving image data; A frame type conversion step of converting the I picture format data of the video data into the P picture format data; and an image synthesis step of taking out the video data from the image storage step and performing a synthesis process. When synthesizing moving image data containing
The moving image combining method is characterized in that the moving image combining step combines moving image data including encoded or converted P-picture format data. 14. An image encoding step of encoding an input video signal as moving image data including an I picture format and a P picture format conforming to the MPEG standard; At the time of encoding, a marking code insertion step of inserting a marking code capable of identifying a boundary of the macroblock data, an image accumulation step of accumulating the moving image data, and detecting a marking code of the moving image data Said I
Set the encoding type of the macroblock of the picture format data to P
A frame type conversion step of converting the image data into a code word in a picture format; and an image synthesis step of extracting moving image data from the image storage step and performing a synthesizing process, when synthesizing moving image data including different encoding types. To
The moving image combining method is characterized in that the moving image combining step combines moving image data including encoded or converted P-picture format data. 15. The marking code to be inserted in the marking code inserting step is such that the amount of reduction in code length when converting the coding type of a macroblock in the frame type conversion step is a multiple length of a data processing unit of a processing system. The moving image according to claim 14, wherein the frame type converting step deletes the marking code when converting the coding type of the macroblock. Synthesis method. 16. A high-frequency component codeword of a macroblock such that when the image encoding step performs variable-length encoding of macroblock data, the code length of the macroblock data is a data processing unit of a processing system. 15. A code length adjusting step of changing the value
Or the moving image synthesizing method according to 15. 17. A maximum of N (N is a positive integer) frames of moving image data including I picture format data and P picture format data encoded in I picture format and P picture format conforming to the MPEG standard. An image input step of inputting to a plurality of image buffers; and a skip frame generating step of generating the P-picture format skip frame when there is no moving image data when extracting moving image data from the plurality of image buffers; A frame type conversion step of converting I picture format data of the moving image data into the P picture format data; and extracting moving image data from the plurality of image buffers,
An image synthesizing step of performing a synthesizing process once every Nth of a second. When synthesizing moving image data including different encoding types,
The moving image combining method is characterized in that the moving image combining step combines moving image data including encoded or converted P-picture format data. 18. The method according to claim 1, further comprising a code amount adjusting step of adjusting the coding amount of the moving image data synthesized in the image synthesizing step to be constant.
8. The moving picture synthesizing method according to 7. 19. The code amount adjusting step includes, if the data amount of the moving image data synthesized in the image synthesizing step is larger than a predetermined range, replacing I-picture format data with the skip frame. 19. The moving image composing method according to claim 18, wherein when the value is smaller, predetermined supplementary data is added after the moving image data. 20. The apparatus according to claim 13, wherein said frame type conversion step converts the coding type of the frame of said I picture format data into a code word of P picture format. 20. The moving picture synthesizing method according to 18 or 19. 21. The frame type conversion step, wherein the coding type of a macroblock of the I-picture format data is converted into a P-picture format codeword. , 18, 19 or 20. 22. In the frame type conversion step, a macroblock stuffing (Macroblock Stuffing) code is inserted such that an increase in code length of the I-picture format data is a multiple of a data processing unit of a processing system. 13. The method according to claim 13, wherein:
22. The moving picture synthesizing method according to 7, 18, 19, 20 or 21. 23. The frame type conversion step, wherein the high-frequency component of the immediately preceding macroblock or the low-frequency component of the immediately following macroblock is processed so that the amount of change in the code length of the I-picture format data matches the data processing unit of the processing system. 13. The method according to claim 13, wherein the code word of the frequency component is rewritten.
The moving image composing method according to 4, 15, 16, 17, 18, 19, 20, 21, or 22. 24. The apparatus according to claim 13, further comprising a noise reduction step of converting a high frequency component of a quantization matrix of the moving image data generated in the image synthesis step into a predetermined small value. , 17,18,1
24. The moving image composing method according to 9, 20, 21, 22, or 23. 25. The method of claim 13, 14, 15, 16, 1
A computer-readable recording medium recorded as a program for causing a computer to execute the moving image combining method according to 7, 18, 19, 20, 21, 22, 23, or 24.