JP2001238169A - Image editing device - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To provide an image editing apparatus capable of dividing and editing a moving image file at a free time position by relatively simple processing without rewriting large data packets. SOLUTION: An image editing apparatus according to the present invention, when a screen after divided editing is encoded using inter-picture prediction, performs decoding once, and then encodes in a mode not using inter-picture prediction. . The time stamp value of the data packet in which the re-encoded moving image encoded data is stored is updated by the moving image division control unit 33. The update of the time stamp value is executed in accordance with the increase in the data code amount due to the re-encoding. For data packets that do not include the re-encoded moving picture encoded data, the content is not rewritten.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image file editing apparatus in which multimedia information such as sound and moving image is recorded, and more particularly to a moving image file in which moving image data is encoded and stored in a recording medium. The present invention relates to a temporal editing process for an image file, and more particularly, to an image editing apparatus for performing file division editing.

[0002]

2. Description of the Related Art With the development of image processing technology and the improvement of the performance of personal computers (hereinafter, also simply referred to as PCs), multimedia information can be easily handled even by a home PC. For this reason, it is required that multimedia information such as audio and moving images can be recorded and used as a file on a normal PC according to, for example, a DOS file format. By recording and managing the multimedia information as a file on the PC, it becomes possible to handle the information as easily as a file of another application such as a word processor or a spreadsheet.

[0003] When a file in which a moving image is recorded can be easily used, it becomes necessary to provide an editing function for the moving image file. Editing of a moving image mainly includes temporal editing such as file division, file combination, and trimming for extracting only a part of a file. Among these, the division editing for cutting a part recorded extra when a moving image is captured or extracting only a part to be focused on is a particularly important function. Note that trimming can be realized by combining two file divisions, and thus can be functionally executed by processing similar to file division.

However, since moving image data has a large data amount, it is necessary to compress the amount of information by encoding when recording the moving image data in a file. For this reason, the file division editing process cannot be simply executed. Generally, in coding moving image data, only information that has changed between screens is extracted and used by using inter-frame / field prediction. Therefore, for a certain screen, only the difference information from the previous screen referred at the time of prediction is recorded in the file as encoded data.
For this reason, if editing such as division is performed on a moving image file, there is a possibility that a case where a reference image for prediction for a screen included in the latter half of the divided file disappears and a normal image cannot be restored may occur. .

FIG. 8 is a diagram for explaining a prediction process between screens in coding moving image data.

Referring to FIG. 8, each screen of the coded moving image data is an intra-screen coded screen that uses no prediction from other screens (indicated by I in the figure, and hereinafter referred to as an intra-coded screen). ) And an inter-coded screen (noted by P in the figure) that is predictively coded with reference to the immediately preceding screen. Here, for example, as shown in (a),
A case where moving image data is divided immediately before a screen 62 which is an inter-screen coding screen will be described. Screen 62 is screen 6
The encoding is performed using the prediction from 1.

At this time, in the moving image data after the division shown in (b), since the screen 62 is positioned at the head, the screen 61 referred to at the time of the prediction of the screen 62 does not exist. The screen 62 cannot be correctly decoded.

In the conventional image editing apparatus, in order to avoid such a problem, division editing is performed only immediately before an intra-coded screen, which is coded without using prediction from a previous screen. Was to be done. That is, in the example of FIG. 8, division is possible only immediately before the screen 63 which is an intra-coded screen.

FIG. 9 is a diagram for explaining rewriting of a data packet in a file when moving image data is divided in a conventional image editing apparatus.

Although the details will be described later, the moving picture coded data (VOP) and the voice coded data corresponding to one screen are divided into a plurality of pieces, and each of the divided data forms a data packet. In FIG. 9, moving image encoded data I-VOP6 of the intra-encoded screen is I-VOP6.
IVOP6-1, I-VOP6-2 and I-VOP6
-3 and stored in the data packet.

Here, when a moving image file is divided immediately before an intra-coded screen corresponding to I-VOP 6, the preceding moving image encoded data and audio encoded data are unnecessary. Therefore, the first encoded data I
-Discard the data located before the I-VOP 6-1 in the data packet storing the VOP 6-1 and pack the encoded data of the I-VOP 6-1 forward.

Therefore, the moving picture data P-VOP 5-2 corresponding to the immediately preceding inter-coded picture stored in the data packet 201 is discarded, and the divided video data P-VOP 5-2 corresponding to the data packet 201 before division is discarded. Data packet 20
In 2, the coded data of the I-VOP 6-1 is stuffed to the front, and padding information, which is a dummy data byte for keeping the packet data length constant, is filled in the vacant portion.

As described above, although the first packet 202 after division is rewritten, the subsequent data packet 2
For 03 and 205, rewriting of packet data is unnecessary.

As described above, as an image editing apparatus for limiting the division position of a moving image file immediately before an intra-encoded screen and operating the moving image encoded data itself to perform division processing, for example, Japanese Patent Laid-Open No. 8-149408 There is a moving image editing method and apparatus disclosed in Japanese Patent Application Laid-Open Publication No. H10-26095. In the editing apparatus, G used in MPEG-1, MPEG-2, etc.
Divided editing is performed in OP (group of picture) units.

In general, the first picture of a GOP is configured to be an intra-coded picture. Therefore, if divisional editing is to be performed in GOP units, correct decoded data cannot be obtained in the first picture after division. It is possible to get.

FIG. 10 shows a conventional image editing apparatus 20.
FIG. 2 is a schematic block diagram illustrating a configuration of a zero.

Referring to FIG. 10, image editing apparatus 200
A packet decomposer 51 for decomposing a video file data packet to be edited into video encoded data and audio encoded data; and a packet decomposer 51 for editing the video encoded data among the decomposed packet data. Coded data division unit 52, coded data reconstruction unit 55, coded data analysis control unit 57, dummy coded data creation unit 53 and stuffing coded data creation unit 54, A packet reconstructing unit 5 for reconstructing audio encoded data into a moving image file data packet
6 is provided.

The packet decomposing unit 51 divides a packet file of moving picture data, extracts voice encoded data and moving picture encoded data, and outputs them.

The coded data decomposing unit 52 divides the coded moving picture data so that the dividing position is at the head of the GOP.

However, in the case of MPEG-1 or MPEG-2, the GOP includes a GOP located temporally ahead such as a bidirectional prediction screen immediately after the first intra-coded screen.
Some screens refer to the screens in the list. in this way,
If the encoded data of the screen in which the prediction reference screen disappears after the division is included, the dummy encoded data creation unit 5
In step 3, dummy encoded data indicating that there is no change between screens is created. Then, the coded data reconstruction unit 55 replaces the coded data with dummy coded data. Even if such replacement is performed, the only effect is that the same screen as the GOP top screen is displayed continuously for several screens, and no visually abnormal screen is displayed. .

Further, in the conventional image editing apparatus 200, when decoding the divided and encoded moving image data, the buffer state of the decoder is the same as the buffer state for the moving image encoded data before division. The stuffing data generation unit 54 generates the stuffing data so that the code generation amount is smaller than expected, and the reconstruction processing is performed so that the stuffing data is inserted into the moving image coded data in order to prevent underflow. Do.

The encoded data analysis control unit 57 determines whether the divided position of the encoded moving image data is a GOP according to the edit control information.
Of the encoded data dividing unit 52
Specify the division position in.

The packet reconstructing unit 56 reconstructs a moving image file data packet by combining the divided moving image encoded data output from the encoded data reconstructing unit 55 and the corresponding audio encoded data. I do.

The packet reconfiguration unit 56 outputs the divided data packet. The encoded data analysis control unit 57
For example, it receives the editing control information including the dividing position indication information from the outside and controls the dividing process of the moving picture encoded data.
The coded data division unit 52 includes a coded data analysis control unit 5
7 to divide the moving image encoded data at the designated position.

The encoded data analysis control unit 57 controls the division position in the encoded data division unit 52, replaces the encoded data with the dummy encoded data in the encoded data reconstruction unit 55, and inserts the stuffing data in accordance with the editing control information. And so on. The coded moving picture data thus reconstructed is subjected to multiplexing processing with the coded sound data to be stored in the moving picture file in the packet reconstructing unit 56 to form a data packet after division. Output.

Japanese Patent Laid-Open No. 7-184176 discloses a method for adjusting the buffer state of a decoder when a moving image file is read and reproduced, instead of embedding stuffing data in the moving image file in advance. Some technologies have been developed. In this publication, when the recorded moving picture encoded data is read out and transmitted to the receiving terminal, the buffer is monitored while monitoring the information amount of the image data so that the decoder buffer of the receiving terminal does not overflow or underflow. A stored image reproducing apparatus that adjusts the amount of data by inserting bits is disclosed.

Further, as another prior art, Japanese Patent Application Laid-Open
Japanese Patent Application Publication No. 184189 discloses a video information accumulation control device in which, when recording moving image encoded data received from a terminal, an arbitrary screen can be recorded as the top by intra-coding the top screen. ing.

[0028]

However, in these prior arts, since the division position is limited to only the boundary between the intra-coded screen and the GOP, the user can freely select the division position of the file. In addition, it is not possible to perform free division editing according to the scene of the recorded moving image.

The intervals and positions at which the intra-coded screens are inserted are determined by the encoder appropriately selecting the intervals and positions reflecting the contents of the moving image such as the start position of the scene desired by the user. Not.

In particular, when the moving image is several kbit / s to several hundred kb
In the case of encoding at a low rate such as it / s, the frame drop processing for reducing the amount of information and the insertion interval in the intra-encoded screen in order to avoid an increase in the amount of data due to intra-encoding. Is required to be as long as several seconds to several tens of seconds, so that the allowable position of division is extremely limited. Therefore, there is a problem that the user cannot divide the moving image file at a desired screen position.

Further, according to the structure of the coded moving picture data, it is possible to forcibly divide the data at the position of the inter prediction coding screen. There is no reference image in inter prediction. For this reason, a normal decoded image cannot be obtained until a correct prediction reference screen is obtained by decoding the next intra-coded screen. Therefore, in order to obtain a normal decoded image, it is necessary to wait until the next intra-coded screen, so that the moving image to be displayed is the same as that when the image is divided at the position of the intra-coded screen as described above. I will.

Furthermore, simply re-encoding the divided top screen into an intra-coded screen increases the data amount as compared with the inter-picture prediction coding before the division processing.
The buffer may overflow or underflow, and the decoder may not operate properly.

Therefore, when the encoded data is directly manipulated as in the conventional image editing apparatus 200 in order to perform a stuffing insertion process or the like, the reconstructed moving image encoded data is again formatted so as to conform to the file structure. Then, a process of writing to the file is required, and a process of rewriting the entire file is required. This makes it possible to read / write files when editing on a small portable device such as a moving image camera, which has large restrictions on processing operations, or when processing a very large file such as a long moving image file. Since the load is large, the processing time becomes very long. Therefore, it is very difficult to realize such a dividing method. In particular, as described above, when the data amount increases due to re-encoding, it is impossible to cope with the stuffing only.

Further, in the insertion processing such as stuffing and fill bit, processing for estimating the state of the decoder buffer is required in order to determine the amount of data to be inserted. In this estimation process, it is necessary to assume the behavior of the decoder in consideration of all the conditions such as the bit frame, frame rate, and encoding method of the encoded data in the moving image file. Detailed analysis is required. Such an estimation process is not only complicated, but also cannot cope with an arbitrary coding system other than a known coding system, and thus has a problem that the result lacks flexibility.

The present invention has been made to solve such a problem, and an object of the present invention is to provide a free time position by relatively simple processing without rewriting large data packets. The present invention provides an image editing apparatus capable of dividing and editing a moving image file.

[0036]

According to a first aspect of the present invention, there is provided an image editing apparatus for dividing and editing a moving image file including a plurality of data packets storing moving image encoded data. A decoding processing unit for decoding moving image encoded data and outputting moving image data, a division editing control unit for controlling a division editing process of a data packet, and a moving image screen to be the first after the division editing. A divided screen re-encoding unit that receives moving image data corresponding to the first screen from the decoding processing unit, re-encodes in a mode that does not use prediction from a temporally previous screen, and outputs moving image encoded data, Packet reconstructing means for combining the encoded video data and a time stamp value indicating the transmission time of the data packet at the time of reproduction to generate a data packet after the division and editing processing; Obtain.

The image editing apparatus according to the second aspect is the first aspect of the invention.
The image editing device according to the above, further comprising a data switching circuit disposed between the divided screen re-encoding means and the packet reconstructing means, wherein the data switching circuit is controlled by the divided editing control means, One of the encoded video data output from the re-encoding unit and the encoded video data before being decoded by the decoding processing unit is provided to the packet reconstructing unit.

The image editing apparatus according to the third aspect is the second aspect.
The image editing apparatus according to the above, wherein the packet reconstructing unit comprises:
If the moving picture encoded data corresponding to the first screen is encoded in a mode using prediction from a temporally previous screen before the divided editing processing, the first screen of the data packet after the divided editing processing is used. Is generated using the encoded video data output from the divided screen re-encoding means.

The image editing apparatus according to the fourth aspect is the second aspect of the invention.
The image editing apparatus according to the above, wherein the packet reconstructing unit comprises:
If the encoded video data corresponding to the first screen is coded in a mode that does not use prediction from the temporally previous screen before the divided editing processing, the leading part of the data packet after the divided editing processing is used. The encoded video data corresponding to the screen is generated using the encoded video data before being decoded by the decoding processing unit as it is.

An image editing apparatus according to a fifth aspect of the present invention provides the image editing apparatus according to the first aspect.
In the image editing apparatus described above, the divided editing control unit does not instruct the rewriting of the contents of the data packet that does not include the moving picture encoded data corresponding to the first screen before the divided editing processing.

According to a sixth aspect of the present invention, there is provided an image editing apparatus.
In the image editing apparatus described above, the divided editing control unit sets a time stamp value of a data packet after the divided editing process in which the moving image encoded data re-encoded by the divided screen re-encoding unit is stored. The time stamp value updating means for updating the time stamp value based on the data amount of the re-encoded moving image encoded data and the transmission bit rate of the encoded moving image data. Set.

According to a seventh aspect of the present invention, there is provided the image editing apparatus according to the sixth aspect.
The timestamp value updating means, based on a data amount of the re-encoded moving image encoded data and a transmission bit rate of the encoded moving image data, a buffer of a decoder at the time of reproducing the first screen. Estimate the amount,
Based on the estimated buffer amount, a time stamp value is set so that overflow or underflow does not occur in the decoder.

[0043] The image editing apparatus according to claim 8 provides the image editing apparatus according to claim 6.
The time stamp value updating means includes a moving image corresponding to the first screen for a time corresponding to a data amount increase accompanying re-encoding of the moving image encoded data corresponding to the first screen. The time stamp value is set so that the encoded data is transmitted first.

An image editing apparatus according to a ninth aspect is the eighth aspect of the invention.
The timestamp value updating means is configured to reproduce the data amount of the re-encoded moving image encoded data and the first screen of the moving image file before the divided editing process. It is calculated from the difference from the buffer amount of the decoder at the time and the transmission bit rate.

An image editing apparatus according to a tenth aspect is the image editing apparatus according to the sixth aspect, wherein the packet reconstructing means is configured to perform processing on a data packet that does not include encoded moving image data corresponding to the first screen. Then, the data packet after the division editing process is generated using the time stamp value before the division editing process as it is.

An image editing apparatus according to an eleventh aspect is the image editing apparatus according to the first aspect, wherein a file storage unit having a storage medium divided into a plurality of sectors and a data stored in the plurality of sectors are provided. It further includes a file management unit for controlling management information for composing a moving image file, and a file writing unit for writing data to a file storage unit. Is performed for each sector unit.

An image editing apparatus according to a twelfth aspect is the image editing apparatus according to the eleventh aspect, wherein:
When the configuration of the sector of the moving image file changes due to the division editing process, the management information is updated.

[0048]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a diagram for explaining an outline of a moving image data dividing process in the image editing apparatus according to the present invention.

Referring to FIG. 1, as in the case of FIG. 8, an intra-coded screen is denoted by I, and an inter-coded screen accompanied by predictive coding is denoted by P. As described above, when a moving image file is divided just before an inter-picture coding screen 62 that is predictively coded with reference to the intra-coding screen 61 as shown in FIG. Since the prediction reference screen of the screen 62 no longer exists, the screen 62 cannot be correctly decoded.

Therefore, in the image editing apparatus according to the present invention, as shown in (c), in the dividing process, the screen 62 at the beginning of the division is temporarily decoded and the decoded image data 6 is decoded.
2d is obtained. Here, in order to decode the screen 62, the moving picture coded data is sequentially decoded from the head, or an independently coded intra coded screen immediately before the screen 62 (screen 59 in FIG. 1). ), Decoding up to the screen 62.

Next, the screen 62d restored to the image data
Is re-encoded into an intra-coded screen so as not to use the prediction from the previous screen. Through such a series of operations, the moving picture encoded data of the intra-coded screen 64 that has been intra-coded is obtained. For a screen 65 subsequent to the screen 64, prediction is performed with reference to the re-encoded screen 64, so that the encoded data reflecting the original inter-picture prediction may be used.

FIG. 2 shows an image editing apparatus 100 according to the present invention.
FIG. 2 is a schematic block diagram showing the configuration of FIG.

Referring to FIG. 2, image editing apparatus 100 includes:
A file storage unit 21 for storing a moving image file, and a file management unit 22 for managing a storage area of the file storage unit 21 and controlling access to the moving image file and the like.
A file reading unit 27 and a file decrypting unit 28 that execute reading of the moving image file and decrypting the contents of the moving image file when reproducing the moving image file;
A divided editing processing block 40 for performing divided editing of moving image data, and a user input processing unit 26 for processing a user instruction such as divided editing processing for a moving image file
And a sound output unit 52 and a moving image output unit 55 for outputting a sound and a moving image, respectively, based on decoding results of the sound coded data and the moving image coded data.

The division / edit processing block 40 comprises a packet decomposing unit 29 for dividing the data packet output from the file decoding unit 28 into audio encoded data and moving image encoded data, and a moving image output from the packet decomposing unit 29. Moving image data decoding unit 30 for decoding image encoded data
And a file division processing unit 25 that receives the edit control information such as the designation of the division position from the user input to the user input processing unit 26 and the output of the moving image data decoding unit 30 and performs the division editing process.

When reproducing a moving image file, the moving image file stored in the file storage unit 21 is read by the file reading unit 27 under the control of the file management unit 22. The file decryption unit 28 decrypts the content of the moving image file to read the attribute / control information and the like, and extracts and outputs the data packet which is the main body of the multimedia information. The structure of the moving image file will be described later in detail.

Further, the packet decomposing unit 29 receives the data packet from the file deciphering unit 28 and separates and outputs the multiplexed audio encoded data and moving image encoded data. The encoded audio data is sent to the audio data decoding unit 50 and subjected to a decoding process. The audio data is restored and output from the audio output unit 52. The encoded moving image data is sent to the moving image data decoding unit 30 and subjected to a decoding process. The moving image data is restored, output from the moving image output unit 55, and displayed.

The image editing apparatus 100 further executes the divided editing process, and thereafter, generates a divided and edited moving image file, and the divided and edited moving image file generated by the file generating unit 24. And a file writing unit 23 for writing to the file storage unit.

In the case of performing a division editing process on a moving image file, an instruction from a user such as designation of a division position is processed in a user input processing unit 26 and transmitted to the file division processing unit 25 as edit control information. Is done.

The file division processing section 25 receives the edit control information, the encoded video data, and the encoded audio data, and outputs a data packet reconstructed by the division editing operation. The edited data packet output from the file division processing unit 25 is
, Necessary information such as header information and footer control information is added. The moving image file constructed in accordance with the predetermined format as described above is stored in the file storage unit 21 via the file management unit 22 by the file writing unit 23.
Is stored in

FIG. 3 is a block diagram for explaining in detail the configuration of the division edit processing unit 25. Referring to FIG. 3, a divided screen re-encoding unit 31 for re-encoding the moving image data decoded by the moving image data decoding unit 30, and a control unit for controlling the divided editing process based on the edit control information. The moving image division control unit 33 and the moving image data decoding unit 30 and the divided screen re-encoding unit 31 controlled by the moving image division control unit 33
35 for setting a path for bypassing the video data, and moving picture coded data from one of the split screen re-encoding unit 31 and the packet decomposing unit 29 via the multiplexer 35,
And a packet reconstructing unit 32 that receives the audio coded data from 9 and generates a divided and edited data packet.

The packet decomposing unit 29 receives a data packet of a moving image file, decomposes the data packet into audio encoded data and moving image encoded data, and outputs them. The voice coded data is output to the packet reconstruction unit 32. The moving image data is sent to the file division processing unit 25 and the moving image data decoding unit 30. Further, the packet decomposing unit 29 extracts the time stamp value TS from the data packet and supplies the extracted time stamp value TS to the moving image division control unit 33. The time stamp value TS will be described later in detail.

The moving image division control section 33 specifies a screen to be re-encoded by the divided screen re-encoding section 31 and a screen to be decoded by the moving image data decoding section 30 according to the input editing control information. Execute the control of.

The moving image division control unit 33 determines whether or not each picture needs to be re-encoded.
5 is switched.

That is, when re-encoding is necessary for the divided top screen as described in (c) of FIG. 1, the multiplexer 35 converts the moving picture coded data corresponding to the top screen into a moving picture. After being decoded by the image data decoding unit 30, the image data is re-encoded by the divided screen re-encoding unit 31 and transmitted to the packet reconstruction unit 32.

On the other hand, when it is not necessary to re-encode the first screen after division, the content does not need to be rewritten for the data packet corresponding to the first screen. The moving picture data obtained by the moving picture data decoding section 30 and the divided picture re-encoding section 31 are directly bypassed by the packet reconstructing section 3.
Send to 2. At this time, if at least one of the moving image data decoding unit 30 and the split screen re-encoding unit 31 is inactive, power consumption can be reduced.

Since data packets corresponding to screens other than the first screen after division do not need to be rewritten, the multiplexer 35 converts the moving image data output from the packet decomposing unit 29 into moving image data decoding data. The packet is directly transmitted to the packet reconstruction unit 32 by bypassing the encoding unit 30 and the divided screen re-encoding unit 31. Similarly, at this time, if at least one of the moving image data decoding unit 30 and the divided screen re-encoding unit 31 is set to the non-operation state, power consumption can be reduced.

With such a configuration, at the time of divided editing, the moving image division control unit 33 determines whether or not re-encoding is necessary based on the edit control information and the like. Can be appropriately switched.

The moving image division control unit 33 updates the time stamp value of the data packet storing the re-encoded moving image encoded data when the divided first screen is re-encoded. A value updating circuit 34 is included.
The time stamp value updating circuit 34 calculates the encoded data amount B of the re-encoded screen obtained by the divided-screen re-encoding unit 31.
L, the time stamp value TS before the divided editing, the transmission bit rate BR of the moving picture encoded data, and the time stamp value updating necessity determination FL based on the type of the first screen after the division FL. The time stamp value is updated and sent to the packet reconstruction unit 32.

The packet reconstructing section 32 includes a time stamp value output from the time stamp value updating circuit 34, encoded voice data transmitted from the packet decomposing section 29, and moving picture encoding output through the multiplexer 35. Upon receiving the data, the data packet after the division and editing is reconstructed and transmitted to the file generation unit 24.

Next, a series of processes performed when the image editing apparatus 100 performs divided editing will be described. The user determines and designates a position to be divided for the selected moving image file by using an appropriate user interface. The information at the time of division is processed by the user input processing unit 26 and sent to the moving image division control unit 33 in the file division processing unit 25 as edit control information.

If the screen at the division position is encoded using inter-picture prediction, the moving image division control unit 33 determines that re-encoding is necessary, and the moving image data decoding unit 30 decodes the image. Instructs the divided screen re-encoding unit 31 to re-encode the moving image data in the intra mode. The multiplexer 35 sends the output of the split screen re-encoding unit 31 to the packet re-constructing unit 32.

Here, for example, when the user designates the display of the division position while reproducing the moving image file and displaying the restored moving image data on the user interface, the user is prompted to give such a message. Since the moving image data of the screen at the division position has already been obtained in order to present the information, the moving image data may be input by the divided screen re-encoding unit 31, and it is not necessary to decode such a screen again. Absent.

On the other hand, when the division position of the moving image file is specified by the time code, the screen number, the data position in the moving image file, or the like to the image editing apparatus 100, the moving image data decoding unit 30 In, it is necessary to decode the screen at the division position. When decoding the screen that is coded using inter-screen prediction, this screen is decoded by sequentially decoding each screen from a screen that can be decoded independently such as the immediately previous intra-screen. can do.

The packet reconstructing unit 32 performs a process of replacing the original encoded video data for the re-encoded screen with the encoded video data after re-encoding, a process of discarding the encoded data before the division process, and the like. A line is created to create a data packet of the moving image file after the divided editing. In addition, the content is not rewritten for a data packet storing moving image encoded data corresponding to a screen other than the screen on which re-encoding has been performed.

For the divided and edited data packet created by the re-encoding, the transmission time stamp value set by the time stamp value updating circuit 34 is added to each data packet. The time stamp value is not updated for a data packet composed only of moving image encoded data not related to re-encoding.

FIG. 4 is a view for explaining a configuration example of a moving image file handled by the image editing apparatus 100 of the present invention.

Multimedia information such as audio and video is
Each file is stored independently as one complete file. As shown in FIG. 4D, the moving image file includes header control information and footer control information indicating the attributes and properties of the entire multimedia information in the file, and a multimedia data body. , Are classified and stored independently in one file. The header control information includes, for example, control information relating to the entire multimedia information included in a moving image file as described below.

File identifier (file ID) for specifying a file; length of multimedia data body; time required for overall reproduction; average / maximum bit rate; type of media included in the data body In addition, the header control information also includes the following control information on a moving image.

Screen reproduction / resolution of video; coding method (for example, ITU-T H.261,
H. 262, H .; 263, H .; 26L, ISO standard 11
172 (MPEG-1), 13818 (MPEG-
2), 14696 (MPEG-4) identification, etc. ・ Parameters in the encoding method (selection of profile, level, option mode, etc.), and a position at which random access is possible as header control information or footer control information Index information to specify,
Information for recording the position of multimedia information, such as marker information for designating and storing an appropriate position of temporally continuous multimedia information, is also included.

The index information designates an intra-screen coded screen when referring to coded video data, and determines the access position and randomly reproduces multimedia information such as high-speed reproduction at random access. It is used for determining the information to be reproduced in the case where the information is reproduced.

The data body of the moving image file is shown in FIG.
As shown in (c), the packet is divided into packets of an appropriate size called data packets and stored as packetized information. Therefore, the data body of the moving image file is composed of a sequence of data packets.

Here, a case will be described in which the encoded video data shown in FIG. 4A and the encoded audio data shown in FIG. 4B are multiplexed to form the data packet sequence shown in FIG. 4C. .

FIG. 4C shows an example in which a plurality of media such as voices and moving images are multiplexed and stored in one data packet.

As shown in FIG. 4B, the speech coded data is one continuous data string. actually,
Speech coded data is composed of coded data for each coding processing unit called a voice frame, but since its length is generally much shorter than the data packet length,
It can be considered as a continuous data sequence.

[0086] The encoded voice data is divided into appropriate values called voice packets and stored in data packets. The length of the voice packet is determined in consideration of the packetization delay time, transmission delay time, and packetization efficiency.
For voice encoded data at a fixed bit rate, voice packets of a fixed length are usually used in order to keep the delay time constant.

On the other hand, the moving picture coded data of each screen in FIG. 4A is further divided and stored in data packets. It should be noted that these encoded video data are MPEG-
When using the 4-video encoding method, VOP (Video
Object Plane).

As a result, as shown in FIG. 4C, coded audio data and coded video data are multiplexed and stored in the data packet. Here, the length of the data packet may be either a fixed length or a variable length. For fixed length, the length of the packet is, for example,
When the system conditions such as the sector length of the recording medium, the decode length, the track length, the PDU (SDU) length of the lower protocol, the transmission / reception buffer size, or the data amount per time (bit rate) are constant, the data packet It is determined according to time conditions such as a transmission interval. If the data packet length is variable,
The unit of packetization can be determined according to the nature of the encoded data of each medium.

A data packet has a packet header indicating control information such as a packet length and the number of data multiplexed therein. When reading and reproducing a moving image file, a time stamp value TS describing the time at which the data packet should be transmitted is added to the packet header. By recording the transmission time in this way, when the server storing the moving image file transmits the data of the moving image file via the network by access from the user, the server can transmit the time of each data packet. Since it is only necessary to refer to the stamp value and send it out at the designated time, the load on the server can be reduced, which is very effective. Such a transmission form of multimedia information is called streaming transfer (streaming tab), and is generally adopted.

In the above example, the case where a plurality of coded data are multiplexed in one data packet has been described. However, one media packet is shown in FIG. 4 in one data packet. The data packet may be configured so as to include only a part obtained by dividing the audio packet or the moving image encoded data. Also in this case, if a time stamp value indicating the transmission time is added to the packet header of the data packet, the server can realize the streaming transfer only by the same processing as described above.

FIG. 5 is a diagram for explaining the packet reconstruction at the time of the division editing process. As for header control information and footer control information other than data packets, since information about the entire moving image file, such as the length of the moving image file, the number of data packets, and the reproduction time, is written, the contents are changed by division. It is necessary to do. For this reason, in the file generation unit 24 shown in FIG. 2, a process of newly generating header control information and footer control information or duplicating the information of the moving image file before division and changing necessary parts is performed. Done.

Here, in a system in which the initial value of the time stamp value (for example, 0 second) is specified in advance, when reproducing a moving image file, the time stamp value of the first data packet from the time of the initial value is reproduced. Will wait until the time indicated by. In order to avoid this waiting time, a time stamp offset value may be added to the header control information of the divided moving image file. Specifically, the transmission time stamp value of the leading data packet of the divided moving image file may be specified as the offset value of the time stamp value.

As described above, the image editing apparatus 100 of the present invention
Even if the transmission time stamp value of a data packet that does not need to be rewritten during the division editing process is left as it is, the offset specified by the header control information when playing back the divided moving image file It is possible to start reproduction without waiting by subtracting the value from all the time stamp values or adjusting the initial value of the time to the offset value.

As described above, in the data packet of the moving image file, the coded audio data and the coded moving image data are multiplexed and stored, and the packet header 16 including the transmission time stamp is placed at the head. Has been added.

Here, the division position shown in FIG. 5, that is, the case where the image is divided immediately before the inter-picture coded screen corresponding to the moving picture coded data P-VOP6 will be described.

In the original video file, the video coded data P-VOP6 is composed of P-VOP6-1 and P-VOP6-1.
-VOP 6-2, divided into two data packets 2
11 and 212 respectively. here,
Since the screen corresponding to the moving picture coded data P-VOP6 has been subjected to the inter-picture prediction coding, the moving picture coded data P-VOP6 is temporarily decoded when the moving picture file is divided,
Re-encoding is performed in the intra mode to obtain encoded video data I-VOP6 corresponding to the intra-coded screen.

For the data packet 212 in which the coded video data P-VOP 6-2 has been stored, the coded video data I-
From the end of VOP6, moving picture encoded data P-VOP6
-2 encoded video data I-VOP6 having a length corresponding to -2
-3 (symbol 1c), and the above P-VOP6-2
To form a data packet.

Moving picture coded data P corresponding to the next screen
-Data packets subsequent to the data packet 213 storing the VOP 7-1 are divided as they are because there is no need to change moving picture encoded data and audio encoded data, and it is not necessary to rewrite the data packets themselves. It can be a data packet of a moving image file.

Next, the moving picture coded data P-VOP6-
1 is stored in the data packet 211.
Moving picture coded data P corresponding to the screen immediately before the division position
-Discard the P-VOP 5-2 which is a part of the VOP 5 and the encoded audio data 12 preceding the encoded video data P-VOP 6. As a result, the data packet 211 becomes empty. Therefore, of the moving image encoded data I-VOP6 of the intra-coded screen, the moving image encoded data before the portion divided as I-VOP6-3 (1c) is converted into I-VOP6-3 (1c). -VOP
As 6-2, it is stored in the data packet 211.

Furthermore, the moving picture coded data I-VOP6
Are stored in the new data packet 210 as I-VOP 6-1.

At this time, padding data 19 is buried in the vacant portion of the data packet 210 that has not been buried. Here, an example in which padding 19 is packed after I-VOP 6-1 has been described, but the present invention is not limited to this.
The entire data packet may be occupied by the I-VOP 6-1 so that padding is buried after the encoded data of the I-VOP 6-2, or both the I-VOP 6-1 and the I-VOP 6-2 The padding may be packed later.

In the data packet 212, P
Even when the moving picture coded data P-VOP 7-1 corresponding to the next screen is included after the -VOP 6-2, the P-VOP 6-2 is similarly set in the data packet 212.
The data packet can be reconstructed by replacing only the portion corresponding to the above with the encoded data divided from the screen P-VOP6.

Transmission time stamp value T specified in packet headers 1a and 1b of each data packet
S takes into account the amount of data that has increased due to re-encoding in the intra-encoding mode (the amount of increase from P-VOP-6 to I-VOP-6). It is determined so that underflow does not occur.

In FIG. 5, the time stamp value of the data packet 212 storing the P-VOP 6-3 is T6 which is the value before the division, and the packet header 1a of the data packet 211 has the time preceding the T6. T5 'is added, and the packet header 1b of the data packet 210 has a time T4' further preceding the time T5 '.
Is added.

The setting of such a time stamp value is shown in FIG.
This is executed by the inside time stamp value update circuit 34.

FIG. 6 is a diagram for explaining a method of setting a time stamp value in the time stamp value updating circuit 34.

FIG. 6A shows the transition of the decoder buffer amount accompanying the processing of the screens P0 to P5 before dividing the moving image file. Referring to FIG. 6 (a), the horizontal axis indicates time, and the upper half (positive direction) of the vertical axis indicates the buffer amount of the decoder. In the lower half (negative direction) of the vertical axis, the buffer amount of the corresponding encoder is shown for reference. The slope of the graph is the transmission bit rate BR of the encoded video data.
Is equivalent to

Here, since a fixed bit rate is assumed, the slope of the graph is constant. In the decoder, moving image encoded data is input at a constant bit rate, so that the buffer amount increases accordingly.
For example, at the decoding time t0 of the screen P0, the screen P0
Since the decoded data of the screen P0 is read from the buffer to decode the data, the buffer amount is reduced by the encoded data amount of the screen P0. The buffer amount of the decoder is a repetition of an increase at a fixed rate and a decrease in coded data for one screen at the decoding timing of the screen.

Here, paying attention to the screen P2, the screen P2
Is input to the decoder buffer at a time t1 at which the slope of the graph is extended as shown in FIG. The buffer amount of the decoder at the time when the screen P2 is decoded is indicated by Be.

That is, when the decoded data is input at a constant bit rate, the time t1 when the head of the encoded data of the screen P2 is input and the time t1 when the screen P2 is decoded
2, the buffer amount Be after the P2 decoding is determined. Since the buffer amount Be is included in the coded data as information in advance depending on the coding standard,
This information can be used. Further, even when the information is not included in the encoded data, the prediction calculation can be performed from the transmission bit rate BR and the display time of the screen.

FIG. 6B shows the amount of decoder buffer and the corresponding amount of encoder buffer after re-encoding the screen P2 at the division position in the image editing apparatus of the present invention. The display of the horizontal axis and the vertical axis is the same as that of FIG.

Referring to FIG. 6B, screen P at the division position
2 is re-encoded in the intra mode, so that the amount of encoded data increases. In the image editing apparatus of the present invention, the decoder buffer amount after decoding the screen P2 (time t4) is equal to the buffer amount before executing the dividing process, that is, the buffer amount at time t2 in FIG. The transmission start time t3 of the encoded data of the re-encoded screen P2 is adjusted to be equal to Be.

Thus, the transition of the buffer amount of the decoder is made equal to that before the division processing is executed. That is, from the position obtained by adding the encoded data amount B2 after the re-encoding of the screen P2 to the buffer amount Be, the intersection with the horizontal axis when a straight line is drawn according to the gradient determined by the transmission bit rate BR,
That is, the transmission time stamp value is specified so that the head of the decoded data of the screen P2 is transmitted at time t3.

This is because the buffer amount Be after the decoding of the picture P2 and the buffer quantity Be after the decoding of the picture P2,
The time t3 is determined by calculating the difference between the time at which the screen P2 is decoded and the input time at the head of the encoded data of the screen P2 from the slope of the graph determined by the transmission bit rate BR. That is, the time stamp value of the data packet storing the re-encoded moving image encoded data is the time stamp value of the data packet for the time set according to the increase in the amount of encoded data accompanying the re-encoding. Updated so that the sending time is earlier.

For example, when rewriting a data packet as shown in FIG. 5 (screen P2 in FIG. 6 (b) corresponds to screen I-VOP6 in FIG. 5), the first data packet 210 after division is As the transmission time stamp value T4 'of the packet header 1b, a value corresponding to the above time t3 is specified.

If the transmission time stamp value for the leading data packet of the divided moving picture file is determined in this way, the moving picture coded data re-encoded by means other than the leading data packet is stored. For a data packet, the time stamp value may be determined so that the moving picture encoded data is transmitted at a constant bit rate BR.

That is, the head of the coded video data I-VOP 6-2 stored in the data packet 211 is transmitted as the transmission time stamp value T5 'stored in the packet header 1a of the next data packet 211. A power time is similarly obtained from the graph of FIG. 6B, and a corresponding value is designated.

Next, an example of the relationship between the moving image file and the physical data arrangement on the storage medium will be described. In the division processing of the present invention, it is unnecessary to rewrite the physical data except at the division position. This will be described.

FIG. 7 shows an image editing apparatus 100 according to the present invention.
FIG. 2 is a diagram showing an example of physical arrangement and management of moving image file data handled in the storage medium on a storage medium.

Referring to FIG. 7, moving image file data itself is stored in, for example, storage medium 12 in file storage unit 21.
1, the information is stored and managed in units of areas (reference numerals 1 to 20 in FIG. 7) called sectors according to the physical format.

Data recorded in separate sectors is
Management information called FAT (File Allocation Table) indicates in what order these sectors are linked to form a file. In the image editing apparatus 100 of the present invention, when performing the dividing process of a moving image file, the contents of the sector storing the reconstructed data packet at the dividing position are rewritten.
Other sectors in which data packets that do not require rewriting are stored can be used as they are even in the file after division.

Also, in the case where the re-encoding associated with the divisional editing process of the present invention increases the amount of encoded video data and necessitates the creation of a new data packet, the data cannot be accommodated in the original sector. , A data packet is written in the empty sector using a new empty sector.

The sector rewritten in this way is linked to the sector in which data has been newly written and the sector in which data has not been rewritten, and the like, so that a FAT is formed so as to constitute a divided moving image file.
Is changed. Therefore, physical rewriting is completely unnecessary for a sector in which a data packet that has not been changed during the division editing process is stored.

In the above description, the case where the original moving image file itself is divided and edited and a new moving image file is created after the division has been described, but the original moving image file is left as it is. The same applies to the case where a moving image file in which a part thereof is divided is newly created.
In this case, a data packet that does not require rewriting may be simply copied for each data packet or each sector in which the data packet is stored. The process of copying such data is performed by the file management unit 22 or the file reading unit 27, the file writing unit 23 shown in FIG.
Can easily be executed by the processing in.

The embodiments disclosed this time are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

[0126]

According to the image editing apparatus of the present invention, the first screen after the divided editing is re-encoded in the intra mode.
The user can divide the moving image file at any position. As a result, since the division position can be determined according to the content of the moving image, the division editing can be performed at a position desired by the user, such as at the moment when a scene desired by the user starts, and the degree of freedom of editing can be improved. . As described above, when the degree of freedom of editing is increased, it is not necessary to consider the convenience at the time of editing at the time of encoding. Therefore, the degree of freedom of selection of encoding parameters such as an insertion position and an interval of an intra-encoded screen is also increased. improves. Further, in the case of performing divisional editing to delete an unnecessary part of a moving image file, the file can be divided and deleted in small units, so that the use efficiency of the recording medium can be improved. The amount of processing required to enjoy these effects is re-encoding only the first screen, which is very small.

According to the image editing apparatus of the present invention, the dividing process of the moving image file is not performed directly on the coded moving image data, but is performed on a data packet according to a predetermined file format. You. Therefore, since only the data packet in which the coded data of the re-encoding target screen located at the division edit position is stored needs to be rewritten, the whole file does not need to be rewritten, and the processing can be executed simply and at high speed. As a result, for example, when a simple small device such as a camera is used, when a long-time moving image file is edited, or when a storage medium with a low access speed is used, the split editing process can be easily performed in a short time. It is possible to do. This effect can be enjoyed even when a new moving image file is created by partially dividing the original moving image file while leaving the original moving image file. In other words, only a simple copy process need be performed for a data packet that does not need to be rewritten, so that a divided moving image file can be newly created by a very simple process.

Furthermore, since the time stamp value indicating the transmission time of the data packet is controlled in response to the increase in the data amount accompanying the re-encoding of the top screen, stuffing is inserted while monitoring the information amount of the image data. It is possible to control the amount of the decoder buffer at the time of reproduction without performing a troublesome data operation such as that described above, thereby avoiding an overflow or an underflow. The processing required for controlling the time stamp value is very simple, and flexible control can be performed.

Further, since the time stamp value is recorded in advance in the divided moving image file, it is not necessary to insert a fill bit or the like when reading and transmitting the moving image file, and the processing can be executed easily. it can. As a result, it is possible to realize an increase in the number of accommodated users who can simultaneously receive moving image files from the server and a reduction in the processing amount required for the moving image distribution server.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating an outline of a process of dividing moving image data in an image editing apparatus according to the present invention.

FIG. 2 is an image editing apparatus 10 according to the embodiment of the present invention.
FIG. 3 is a schematic block diagram illustrating a configuration of a zero.

FIG. 3 is a block diagram for explaining a configuration of a file division processing unit 25 in detail.

FIG. 4 is a diagram showing a configuration example of a moving image file handled by the image editing device according to the present invention.

FIG. 5 is a diagram illustrating packet reconfiguration at the time of a division editing process.

FIG. 6 is a diagram for explaining a method of setting a time stamp value by a time stamp value updating circuit 34;

FIG. 7 is a diagram showing an example of physical arrangement and management on a storage medium of moving image file data handled by the image editing apparatus of the present invention.

FIG. 8 is a diagram illustrating a prediction process between screens in encoding of moving image data.

FIG. 9 is a diagram illustrating rewriting of a data packet in a file when moving image data is divided in a conventional image editing apparatus.

FIG. 10 is a schematic block diagram illustrating a configuration of a conventional image editing device 200.

[Explanation of symbols]

25 file division processing unit, 29 packet decomposition unit, 3
0 moving image data decoding unit, 31 divided screen re-encoding unit, 32 packet reconstructing unit, 33 moving image division control unit, 34 time stamp value configuration circuit.

Claims (12)

[Claims] An image editing apparatus for dividing and editing a moving image file composed of a plurality of data packets storing moving image encoded data, wherein the moving image data is decoded by decoding the moving image file. Decoding processing means for outputting the data packet; divided editing control means for controlling the divided editing processing of the data packet; and the moving image data corresponding to the top screen which is the top moving image screen after the divided editing. Divided screen re-encoding means that receives from the decoding processing means and re-encodes in a mode that does not use prediction from a temporally previous screen to output the encoded video data, and the encoded video data, By combining with the time stamp value indicating the transmission time of the data packet at the time of reproduction,
An image editing apparatus comprising: a packet reconstructing unit configured to generate a data packet after the division editing process. 2. The apparatus according to claim 1, further comprising a data switching circuit disposed between said divided screen re-encoding means and said packet reconstructing means, wherein said data switching circuit is controlled by said division editing control means, 2. The image editing apparatus according to claim 1, wherein one of the encoded video data output from the screen re-encoding unit and the encoded video data before being decoded by the decoding processing unit is provided to the packet reconstructing unit. . 3. The packet reconstructing means according to claim 1, wherein said moving picture encoded data corresponding to said first picture is encoded in a mode using prediction from said temporally previous picture before said divided editing processing. Wherein the encoded video data corresponding to the first screen in the data packet after the divided editing processing is generated using the encoded video data output from the divided screen re-encoding means. The image editing device according to the above. 4. The packet reconstructing means, wherein the moving image encoded data corresponding to the first screen is encoded in a mode that does not use prediction from the temporally previous screen before the divided editing processing. In the case of, the encoded video data corresponding to the first screen of the data packet after the divided editing process is generated using the encoded video data before being decoded by the decoding processing means as it is,
The image editing device according to claim 2. 5. The image according to claim 1, wherein the divided editing control unit does not instruct rewriting of the content of the data packet not including the moving picture encoded data corresponding to the first screen before the divided editing processing. Editing device. 6. The divided editing control means sets the time stamp value of the data packet after the divided editing processing in which the moving picture encoded data re-encoded by the divided screen re-encoding means is stored. The time stamp value updating means, based on the data amount of the re-encoded moving image encoded data and the transmission bit rate of the moving image encoded data, The image editing apparatus according to claim 1, wherein the time stamp value is set. 7. The decoder according to claim 1, wherein the time stamp value updating unit is configured to determine a buffer of a decoder when reproducing the first screen based on a data amount of the re-encoded moving image encoded data and a transmission bit rate of the encoded moving image data. 7. The method according to claim 6, further comprising: estimating an amount of data, and setting the time stamp value based on the estimated amount of the buffer so that overflow or underflow does not occur in the decoder.
An image editing apparatus as described in the above. 8. The moving picture corresponding to the first screen for a time corresponding to the data amount increase accompanying the re-encoding of the moving picture encoded data corresponding to the first screen. 7. The image editing apparatus according to claim 6, wherein the time stamp value is set so that encoded data is transmitted earlier. 9. The time stamp value updating means may be configured to determine the amount of increase in the data amount by comparing the data amount of the re-encoded moving image encoded data with the data amount of the re-encoded moving image file before the divided editing process. The image editing apparatus according to claim 8, wherein the image editing apparatus calculates the difference based on a difference from a buffer amount of a decoder during reproduction and the transmission bit rate. 10. The data processing apparatus according to claim 1, wherein the packet reconfiguring unit performs the division editing process on the data packet not including the moving picture encoded data corresponding to the first screen by using a time stamp value before the division editing process as it is. 7. The image editing apparatus according to claim 6, wherein a subsequent data packet is generated. 11. A file storage unit having a storage medium divided into a plurality of sectors, a file management unit for controlling management information for composing the moving image file from data stored in the plurality of sectors, A file writing unit for writing data to the file storage unit, wherein the file writing unit executes data rewriting necessary for the division editing process for each sector unit. 2. The image editing device according to 1. 12. The image editing apparatus according to claim 11, wherein the file management unit updates the management information when a configuration of the sector of the moving image file changes according to the division editing process. .