JP2011077564A - Encoder and encoding method of video and voice data, and video editing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an encoder capable of accelerating the encoding processing of video data. <P>SOLUTION: A hardware encoder 151 is composed of hardware for encoding processing and encodes a part of AV data. A software encoder 152 utilizes a CPU 10 to encodes the other part of the AV data in parallel to the encoding processing by the hardware encoder 151. A data assignment portion 14 assigns the AV data to both of the encoders 151 and 152. A synthetic portion 16 arranges the video data encoded with each encoder in a predetermined order, and synthesizes the video data into a series of encoded video data. An output portion 17 outputs a series of encoded video data. The encoder enables encoding at higher encoding processing speed than the encoding processing speed attained only by the hardware encoder 151. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to encoding processing of video data and audio data.

  A video editing system is a system for assisting a user in editing video data, and is generally realized by causing a computer terminal such as a personal computer equipped with a general-purpose CPU to execute a predetermined application program. The video editing system takes in one or more video data and audio data from a video camera or the like to an HDD in advance, and edits one video data stream therefrom. The edited audio / video data stream is compressed by the system, or is recorded as uncompressed data in an arbitrary recording medium such as an HDD or a DVD in the same system, or sent for broadcasting, or the like. The data is output to the outside through an interface such as IEEE1394.

  Video editing systems use various types of encoding to compress audio and video data streams. In recent years, the amount of video data per stream is enormous as seen in the HDV format, for example. Therefore, in recent video editing systems, the time required for encoding processing tends to increase.

As a conventional technique for speeding up the encoding process of audio / video data, there is a technique of increasing the number of CPUs that perform the encoding process and executing the encoding process in parallel. In addition, there is a technique in which a part of a series of encoding processes is processed by dedicated hardware and the rest is processed by a CPU. In addition, there is a technique in which a chip or a circuit dedicated to encoding processing is mounted on a system and the entire encoding processing of video data is performed on them.
Japanese Patent Laid-Open No. 2004-356851 Japanese Patent Application Laid-Open No. 06-339018

  There is an upper limit of the encoding processing speed for the chip and circuit dedicated to the encoding process, and when the speed desired by the user is not reached, the processing speed of the chip or circuit dedicated to the encoding process can be further improved, It is conceivable that the encoding processing is performed in parallel by using a plurality of chips dedicated to the encoding processing.

  However, in order to improve the processing speed of a single chip or a circuit dedicated to encoding processing, the circuit design must be changed. This increases the manufacturing cost of a single chip or circuit dedicated to the encoding process. In particular, a single chip dedicated to the encoding process is generally large and expensive, and adding this chip increases the scale and significantly increases the manufacturing cost. As described above, it is difficult for the chip or circuit dedicated to the encoding process to further increase the encoding process while suppressing an increase in manufacturing cost.

  An object of the present invention is to provide a novel and useful encoding apparatus, an encoding method thereof, and a video editing system that solve the above problems. A specific object of the present invention is to provide an encoding device, an encoding method thereof, and a video editing system capable of speeding up encoding processing of video data and audio data.

  According to one aspect of the present invention, an encoding device that encodes AV data including audio data and / or video data, which includes a CPU (Central Processing Unit) and hardware for encoding processing, A hardware encoder that encodes part of the AV data; and software that encodes another part of the AV data in parallel with the encoding process by the hardware encoder using the CPU. An encoder, a data allocator for allocating the AV data to the hardware encoder and the software encoder, and AV data encoded by each of the hardware encoder and the software encoder in a predetermined order A synthesizing unit for arranging and synthesizing into a series of encoded AV data; An encoding unit that outputs the series of encoded AV data, and encodes the AV data at an encoding processing speed larger than an encoding processing speed by only the hardware encoder. Provided.

  According to the present invention, the AV data including audio data and / or video data is encoded in parallel by the hardware encoder and the software encoder, so that a code larger than the encoding processing speed by only the hardware encoder is encoded. AV data can be encoded at the encoding processing speed, and the encoding processing speed can be increased.

  In the present specification and claims, “hardware encoder” means (i) an IC or module designed exclusively for executing part or all of the encoding processing of video data and audio data. A circuit, or (ii) an arithmetic circuit such as a CPU or DSP (Digital Signal Processor) capable of executing part or all of the encoding process of video data or audio data and processes other than the encoding process, While the arithmetic circuit is executing part or all of the encoding processing of video data and audio data, the arithmetic circuit exclusively performs part or all of the encoding processing.

  In the present specification and claims, the “software encoder” is a video data encoding process using an arithmetic circuit such as a general-purpose CPU or DSP that is also used for processing other than video data encoding processing. It is an encoder that executes video data encoding processing in parallel with other processing by executing software.

  According to another aspect of the present invention, the CPU includes an editing unit that edits AV data including audio data and / or video data, and a circuit for encoding processing, and a part of the edited AV data. A hardware encoder that encodes the CPU, and a software encoder that encodes another part of the edited AV data in parallel with the encoding process by the hardware encoder using the CPU A data allocator for allocating the edited AV data to the hardware encoder and the software encoder, and the AV data encoded by the hardware encoder and the software encoder in a predetermined order. And a synthesizing unit that synthesizes the data into a series of encoded AV data, and the series of encoded AV data. And an output unit for outputting the video editing system for encoding AV data in a large encoding processing speed than encoding processing speed only by the hardware encoder is provided.

  According to the present invention, the AV data including audio data and / or video data is encoded in parallel by the hardware encoder and the software encoder, so that a code larger than the encoding processing speed by only the hardware encoder is encoded. AV data can be encoded at the encoding processing speed, and the encoding processing speed can be increased.

  According to another aspect of the present invention, a hardware encoder that is configured by hardware for encoding processing and encodes part of AV data, and another part of the AV data using a CPU. A method of encoding video data using a software encoder for encoding the AV data, receiving the AV data, and allocating the AV data to the hardware encoder and the software encoder A step of encoding AV data respectively allocated by the hardware encoder and the software encoder; and AV data encoded by each of the hardware encoder and the software encoder Synthesizing into a series of encoded AV data arranged in order Outputting the series of encoded AV data, and encoding the AV data at an encoding processing speed larger than the encoding processing speed only by the hardware encoder. .

  According to the present invention, the AV data including audio data and / or video data is encoded in parallel by the hardware encoder and the software encoder, so that a code larger than the encoding processing speed by only the hardware encoder is encoded. AV data can be encoded at the encoding processing speed, and the encoding processing speed can be increased.

  ADVANTAGE OF THE INVENTION According to this invention, the encoding apparatus which can speed-up the encoding process of video data and audio | voice data, its encoding method, and a video editing system can be provided.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. In the present invention, both video data and audio data can be encoded. For convenience of explanation, a case where video data is encoded will be described.

Embodiment 1
FIG. 1 is a block diagram showing configurations of an encoding device 100 and a video editing system 200 according to Embodiment 1 of the present invention. Referring to FIG. 1, an encoding device 100 according to Embodiment 1 of the present invention is incorporated in a video editing system 200. The video editing system 200 according to the first embodiment is a non-linear video editing system, and is realized using a computer terminal such as a personal computer. The video editing system 200 includes a CPU 10, a memory unit 20, a built-in HDD 30A, a built-in drive 40A, an input / output interface 50, a user interface 60, an AV unit 70, a hardware encoder 80, and an internal bus 90 that connects these elements. . Among the elements included in the video editing system 200, the CPU 10 and the hardware encoder 80 are included in the encoding device 100. The video editing system 200 may further include a network interface that can be connected to an external LAN or the Internet.

  The CPU 10 executes a program stored in the memory unit 20 and functions as an editing unit 13, a data allocation unit 14, a software encoder 152, a synthesis unit 16, and an output unit 17 which will be described later. Details of these units will be described later. The CPU 10 is not limited to one chip, and may be a plurality of chips, and each chip may have a plurality of cores. Such a plurality of CPUs operate in parallel. Thereby, the encoding capability of the software encoder 152 is further improved.

  The memory unit 20 stores programs and data that cause the CPU 10 to execute processes described later. In the memory unit 20, an input buffer area BI, a first output buffer area BO1, and a second output buffer area BO2 are secured. Details of these buffer areas will be described later.

  The built-in HDD 30A is built in a computer terminal that implements the video editing system 200. Instead of the internal HDD 30A or in addition to the internal HDD 30A as shown in FIG. 1, an external HDD 30B may be connected to the internal bus 90 through the input / output interface 50.

  The built-in drive 40A is built in a computer terminal that implements the video editing system 200. An external drive 40B may be connected to the internal bus 90 through the input / output interface 50 instead of the internal drive 40A or in addition to the internal drive 40A as shown in FIG.

  The internal drive 40A and the external drive 40B input and output video data and audio data to and from a removable medium such as the DVD 102. Examples of the removable medium include a magnetic disk, a magneto-optical disk, a Blu-ray disk, and a semiconductor memory in addition to the optical disk.

  In addition to the external HDD 30B and the external drive 40B, the input / output interface 50 can connect a storage medium built in an external device such as the user interface 60 or the second camera 101B to the internal bus 90. For example, the input / output interface 50 includes an IEEE 1394 interface or the like, and inputs and outputs video data and audio data to and from the second camera 101B. In addition to the second camera 101B, the input / output interface 50 can input and output video data and audio data to and from various devices that handle video data and audio data, such as a VTR, switcher, and transmission server. it can.

  The user interface 60 is connected to the internal bus 90 through the input / output interface 50. The user interface 60 includes, for example, a mouse 61, a keyboard 62, a display 63, and a speaker 64. The user interface 60 may include a touch panel (not shown) as another input device.

  The AV unit 70 includes a video interface and an audio interface. The AV unit 70 inputs / outputs video data to / from an external device such as the first camera 101A through these interfaces. In addition to the first camera 101A, the AV unit 70 can input and output video data and audio data with various devices that handle video data and audio data, such as a VTR, a switcher, and a transmission server. .

  The hardware encoder 80 is either (i) a circuit such as an IC or module designed exclusively for executing part or all of the encoding process of video data or audio data, or (ii) video data or audio data. An arithmetic circuit such as a CPU or a DSP capable of executing a part or all of the encoding process and a process other than the encoding process, wherein the arithmetic circuit is a part of the encoding process of video data or audio data or This is an arithmetic circuit that exclusively performs part or all of the encoding process while the whole is executed.

The hardware encoder 80 compresses uncompressed video data by encoding. For example, in the case of the lossy encoding method, the video data encoding process executed by the hardware encoder 80 includes the following steps (i) to (iv).
(I) division step into blocks, tiles, etc.
(Ii) orthogonal transform steps such as DCT transform and wavelet transform;
(Iii) a quantization step; and
(Iv) A lossless encoding step such as entropy encoding.

  The video data encoding process by the hardware encoder 80 may include, for example, a sub-sampling step, a shuffling step, an intra-frame prediction step, or an inter-frame prediction step. The encoding process of video data by the hardware encoder 80 may be a compression process using a lossless encoding method or an encoding process not intended for compression.

  The hardware encoder 80 incorporates a frame buffer and encodes video data stored in the frame buffer. In addition, a predetermined storage area of the memory unit 20 may be used as a similar frame buffer. The encoding method by the hardware encoder 80 is, for example, the DV method, the MPEG method, the JPEG2000 method, the intra frame method (for example, the AVC-Intra method), or the closed GOP method in the MPEG method, but is not particularly limited. The encoding method by the hardware encoder 80 is selected in advance according to the output destination device of the encoded video data. The hardware encoder 80 encodes each data unit based on information contained therein. For example, in the intra frame method, the data unit is one frame. In addition, in the closed GOP system in the MPEG system, the data unit of encoding may be 1 GOP.

  FIG. 2 is an example of a schematic plan view of a board 800 on which the hardware encoder 80 shown in FIG. 1 is mounted. The board 800 is mounted on a computer terminal that implements the video editing system 200.

  Referring to FIG. 2, the board 800 includes a connector 81, a hardware encoder 82, a frame buffer 83, and an FPGA (Field Programmable Gate Array) 84. The AV unit 70 may be further mounted on the board 800.

  The connector 81 conforms to PCI-Express and is inserted into a PCI-Express slot in a computer terminal, thereby connecting a circuit on the board 800 to a PCI-Express bus of the computer terminal. This bus is included in the internal bus 90.

  The frame buffer 83 is an independent element of DDR2-SDRAM (Double-Data-Rate2-Synchronous-DRAM) and can store a predetermined number, for example, four frames. The video data to be encoded is transferred from the input buffer area BI of the memory unit 20 to the frame buffer 83 through the connector 81 and the FPGA 84 for each frame. The hardware encoder 82 encodes the video data sequentially from the head address of the frame buffer 83 through the FPGA 84. The video data encoded by the hardware encoder 82 is written into the frame buffer 83 through the FPGA 84 for each frame, and further transferred from the frame buffer 83 to the first output buffer area BO1 of the memory unit 20 through the FPGA 84 and the connector 81. Is done.

  The FPGA 84 is a single chip and includes a PCI-Express interface 85, a memory interface 86, a DMA engine 87, and an encoder interface 88. The PCI-Express interface 85 connects the other modules in the FPGA 84 through the connector 81 to the PCI-Express bus of the computer terminal constituting the video editing system 200. The memory interface 86 controls access to the frame buffer 83 by other modules in the FPGA 84. The DMA engine 87 uses the PCI-Express interface 85 and the memory interface 86 to control transfer of video data between the memory unit 20 and the frame buffer 83 through the PCI-Express bus in the computer terminal. . The encoder interface 88 connects other modules in the FPGA 84 to the hardware encoder 82.

  FIG. 3 is a block diagram showing a main functional configuration of the video editing system 200 of FIG. Referring to FIG. 3, the video editing system 200 includes an editing unit 13 and an encoding unit 11 as functional configurations. The encoding unit 11 includes a data allocation unit 14, a hardware encoder 151, a software encoder 152, a synthesis unit 16, and an output unit 17. The editing unit 13, the data allocating unit 14, the software encoder 152, the synthesizing unit 16, and the output unit 17 are realized by the CPU 10 of the video editing system 200 in FIG.

  The editing unit 13 accepts selection of video data to be edited, reads the selected video data, performs editing processing based on a user's editing operation, and edits the video data stream.

  Specifically, the editing unit 13 first displays a list of files stored in a resource such as the DVD 102, the internal HDD 30 </ b> A, or the external HDD 30 </ b> B on the display 63 included in the user interface 60. These files include video data, audio data, still images, text data, or the like as material data. The user operates the mouse 61 and the keyboard 62 to select a file to be edited from the list. The editing unit 13 accepts selection of a file by the user, and displays a clip corresponding to the selected file on the display 63. The clip is information that refers to part or all of the material data having a time axis. The clip includes time information representing each of the start position and end position of the data portion referenced thereby.

  FIG. 4 is an example of the edit window EW. The editing unit 13 displays an editing window EW on the display 63 and receives an editing operation from the user. Referring to FIG. 4, the editing window EW includes, for example, a material window BW, a timeline window TW, and a preview window PW.

  The editing unit 13 displays the clip IC1 corresponding to the selected file to be edited in the material window BW.

  The editing unit 13 displays a plurality of tracks TR in the timeline window TW, and accepts the arrangement of the clips CL on each track TR. In the example of FIG. 4, each track TR is an elongated strip-like region extending in the horizontal direction of the screen. Each track TR indicates position information on a timeline that is a time axis of a video data stream generated by the editing unit 13. In the example of FIG. 4, the position in the horizontal direction on each track is associated with the position on the timeline so that the position on the timeline advances as the track TR moves in the horizontal direction from left to right on the screen. It has been. The editing unit 13 accepts the placement of the clip CL on each track TR from the material window BW, for example, through the operation of the mouse 61 by the user.

  The editing unit 13 may also display the timeline cursor TLC and the time axis scale on the timeline window TW. In the example of FIG. 4, the timeline cursor TLC is a straight line that extends from the time axis scale in the vertical direction of the screen and intersects each track TR. The timeline cursor TLC can be moved in parallel to each track TR. The value of the time axis scale pointed to by the end of the timeline cursor TLC represents the position on the timeline at the intersection of the timeline cursor TLC and each track TR.

  The editing unit 13 sets the in point IP that is the start position and the out point OP that is the end position of the clip CL arranged on each track TR, and is arranged on each track TR. A later change of the in-point IP and the out-point OP of the clip CL is accepted.

  The editing unit 13 further adjusts the color and brightness of the corresponding video for the clips CL arranged on each track TR, special effects on the entire corresponding video, or between a plurality of clips CL arranged on different tracks TR. The setting of effect processing such as video composition in can be received from the user.

  The editing unit 13 displays an image corresponding to the clip CL arranged at the position on the timeline indicated by the timeline cursor TLC in the preview window PW. The editing unit 13 also displays, in the preview window PW, a moving image corresponding to a specified range among the clips CL arranged in the timeline window TW. The user can confirm the result of the editing process accepted by the editing unit 13 from the video displayed in the preview window PW.

  The editing unit 13 generates editing information for a series of video data streams to be edited based on the arrangement of the clips CL on the track TR in the timeline window TW and the effect processing set for each clip CL. In the editing information, the contents of the editing process by the editing unit 13 are defined together with the time code. The time code represents the position on the timeline of the video data subjected to the corresponding editing process. The editing unit 13 generates a series of video data streams by connecting the video data referred to by each clip CL in order on the timeline in accordance with the editing information. The editing unit 13 further outputs the video data stream as video data to be encoded to the data allocating unit 14 and writes it in the input buffer area BI in the memory unit 20.

  Returning to FIG. 3, the data allocator 14 allocates the video data to be encoded written in the input buffer area BI in the memory 20 to the hardware encoder 151 and the software encoder 152. Specifically, the data allocator 14 first secures output buffer areas BO1 and BO2 for writing encoded video data for each of the encoders 151 and 152 in the memory unit 20. Next, the data allocator 14 selects the video data written in the input buffer area BI as a data unit to be encoded by the encoders 151 and 152 in a predetermined data unit, for example, one frame at a time. Subsequently, the data allocator 14 sequentially transfers the data units selected for the hardware encoder 151 to the frame buffer of the hardware encoder 151, encodes each transferred data unit, and writes the data units to be written. An address in one output buffer area BO1 is designated to the hardware encoder 151. On the other hand, for the data unit selected for the software encoder 152, the data allocator 14 writes the address of the data unit in the input buffer area BI after the data unit is encoded and the second output to be written. The software encoder 152 is sequentially designated together with the address in the buffer area BO2. The data allocator 14 further assigns serial numbers, for example, frame numbers, to the data units allocated to the encoders 151 and 152 in order, and designates the encoders 151 and 152 as write destinations after encoding the data units of the respective numbers. The address is passed to the synthesis unit 16 together with the number.

  If there are two or more hardware encoders and they perform the encoding process independently, the data allocator 14 sends the video data to be encoded to be processed by each hardware encoder to each hardware encoder. Allocate to the wear encoder.

  The hardware encoder 151 is a device driver of the hardware encoder 80 shown in FIG. The hardware encoder 151 sequentially encodes the encoding target video data transferred from the input buffer area BI in the memory unit 20 to the frame buffer by the data allocator 14. The encoding method is set by the editing unit 13. The hardware encoder 151 encodes each data unit of the video data based on information included therein. For example, when the encoding method is an intra frame method, the encoding data unit is one frame, and when the encoding method is a closed GOP method, the encoding data unit is 1 GOP. The hardware encoder 151 transfers the encoded video data to the address in the first output buffer area BO1 designated by the data allocator 14. The hardware encoder 151 further notifies the synthesizing unit 16 of the transfer destination address each time the encoded video data is transferred to the first output buffer area BO1.

  The software encoder 152 is an encoding processing module executed by the CPU 10 and encodes video data to be encoded stored in the input buffer area BI in the memory unit 20 in the order of addresses designated by the data allocating unit 14. To do. The software encoder 152 executes an encoding process independently of the hardware encoder 151 and in parallel. As with the hardware encoder 151, the software encoder 152 encodes each data unit based on the information contained therein.

  The data unit and encoding method of the software encoder 152 are the same as the data unit and encoding method of the hardware encoder 151. That is, if the data unit encoded by the hardware encoder 151 is 1 frame, the data unit encoded by the software encoder 152 is also 1 frame, and the data unit encoded by the hardware encoder 151 is 1 GOP. For example, the data unit encoded by the software encoder 152 is also 1 GOP.

  Further, if the encoding method by the hardware encoder 151 is an intra frame method, the encoding method by the software encoder 152 is also an intra frame method, and if the encoding method by the hardware encoder 151 is a closed GOP method, software is used. The encoding method by the encoder 152 is also a closed GOP method.

  The software encoder 152 writes the encoded video data at the address in the second output buffer area BO2 specified by the data allocator 14. The software encoder 152 further notifies the synthesizing unit 16 of the write destination address each time the encoded video data is written to the second output buffer area BO2.

  The synthesizing unit 16 arranges the data units encoded by the hardware encoder 151 and the software encoder 152 in the order of the numbers allocated by the data allocating unit 14, and synthesizes them into one encoded video data stream. To do. Specifically, the combining unit 16 first specifies an address corresponding to the number of the data unit to be combined next from the addresses received from the data allocating unit 14. Next, when the same address as the identified address is notified from any of the encoders 151 and 152, the synthesizer 16 reads the encoded data unit from the address, and reads other previously read data units. Is synthesized at the end of the array of data units. Thus, the order of the combined encoded data unit matches the order written by the editing unit 13 to the input buffer area BI in the memory unit 20. Furthermore, the synthesizing unit 16 outputs the synthesized video data stream to the output unit 17.

  When the hardware encoder 151 and the software encoder 152 encode the video data in GOP units using the closed GOP method as the encoding method, the synthesis unit 16 adds the number received from the data allocation unit 14. The encoded video data stream can also be arranged based on information such as a GOP header included in the encoded video data. In addition, it is preferable that the data units encoded by the hardware encoder 151 and the software encoder 152 are not predictable from each other in terms of facilitating the processing of the synthesis unit.

  The output unit 17 shapes the synthesized video data stream into a predetermined file format or transmission format. The output unit 17 adds information, parameters, and other specified information necessary for decoding the encoded video data to the video data stream, and arranges the entire data stream into a specified format. The file format or transmission format used by the output unit 17 is set by the editing unit 13. The output unit 17 writes the synthesized video data stream to an arbitrary recording medium, for example, the internal HDD 30A, the external HDD 30B, or the DVD 102 mounted in the internal drive 40A or the external drive 40B via the internal bus 90. . The output unit 17 can also transmit the combined video data stream to a database or information terminal connected via a network interface. The output unit 17 can also output the synthesized video data stream from the AV unit 70 or the input / output interface 50 to an external device. The output unit 17 can generate and output not only the video data stream but also the audio content.

  FIG. 5 is a block diagram of the encoding unit 11 according to Embodiment 1 of the present invention. Referring to FIG. 5, the encoding unit 11 includes a data allocating unit 14, a hardware encoder 151, a software encoder 152, a combining unit 16, an output unit 17, and a buffer 21. The buffer 21 is disposed between the data allocator 14 and the hardware encoder 151. The buffer 21 represents a storage area of a frame buffer built in the hardware encoder 80, for example, the frame buffer 83 shown in FIG. In that case, the capacity of the buffer 21 is, for example, several frames. In addition, the buffer 21 may be a storage area in the memory unit 20. In that case, the capacity of the buffer 21 is, for example, several tens of frames.

  The data allocator 14 sequentially transfers the video data allocated to the hardware encoder 151 to the buffer 21. The hardware encoder 151 encodes video data in units of a predetermined data unit, for example, one frame in order from the head address of the buffer 21.

  The data allocator 14 reads the empty information from the buffer 21 and confirms the size of the empty area of the buffer 21, for example, the number of empty frame buffers, from the empty information. When the size exceeds a predetermined threshold, for example, one frame, the data allocator 14 allocates video data to be encoded to the hardware encoder 80. On the other hand, when the size of the free area of the buffer 21 is reduced to a predetermined threshold value or less, for example, 1 frame or less, the data allocator 14 determines the frame following the frame last allocated to the hardware encoder 151 as the software encoder. Allocate to 152. That is, the data allocator 14 does not write the next frame to the buffer 21 and passes the address in the input buffer area BI where the frame is stored to the software encoder 152. Thereafter, the data allocator 14 continues to allocate the video data to be encoded to the software encoder 152 until the size of the free area in the buffer 21 exceeds the threshold again.

  FIG. 6 is a flowchart of video data allocation processing by the encoding unit 11 shown in FIG. The video data allocation process will be described below with reference to FIG. In the following description, for the sake of convenience, the encoding data unit is one frame. For example, when a video data stream to be encoded is output from the editing unit 13, the following processing is started.

  First, in step S <b> 11, the data allocator 14 allocates a predetermined number of frames, for example, one frame from the head of the video data stream to be encoded to the hardware encoder 151 and transfers it to the buffer 21. The hardware encoder 151 starts to encode the video data in order from the head address of the buffer 21.

  Next, in step S12, the data allocator 14 determines whether or not the number of free frame buffers has decreased below a predetermined threshold from the free information in the buffer 21. If the number of free frame buffers is greater than the threshold (if “Yes” in step S12), the process proceeds to step S13. If the number of free frame buffers is equal to or less than the threshold (if “No” in step S12), the process proceeds to step S14.

  In step S 13, the data allocator 14 allocates the next frame of the video data stream to be encoded to the hardware encoder 151, and transfers the frame to the buffer 21. Then, the process proceeds to step S15.

  In step S14, the data allocator 14 allocates the next frame of the video data stream to be encoded to the software encoder 152. Then, the process proceeds to step S15.

  Next, in step S15, the data allocator 14 determines whether or not the allocated frame is the last frame of the video data stream to be encoded. This determination can be made by referring to a frame header, for example. If the frame is not the last (if “No” in step S15), the process returns to step S12. If the frame is the last (“Yes” in step S15), the process ends.

  In the encoding apparatus 100 of the first embodiment, when the free area of the buffer 21 of the encoding unit 11 is reduced to a threshold value or less, the data allocating unit 14 allocates a part of video data to be encoded to the software encoder 152. Thereby, since the encoding process by the hardware encoder 151 and the software encoder 152 is performed in parallel, the encoding process by the entire encoding apparatus 100 is faster than the case where the encoding process is performed only by the hardware encoder 151. Can do. In particular, the data allocator 14 obtains free information in the buffer 21 that temporarily stores frames waiting for encoding processing by the hardware encoder 151, and determines the next frame allocation destination based on the free information. Since the encoding process by the software encoder 152 is performed in parallel while always operating the hardware encoder 151, the encoding apparatus 100 performs the encoding process at a higher speed than the encoding process speed of the hardware encoder 151. Can be done. Thereby, the encoding processing speed of the video editing system 200 can be improved.

<< Embodiment 2 >>
The encoding apparatus 100 according to the second embodiment of the present invention is incorporated in the non-linear video editing system 200 as in the first embodiment. The video editing system 200 is the same as that according to the first embodiment except for the encoding unit. Therefore, the description about Embodiment 1 is used for details of these similar elements.

  FIG. 7 is a block diagram of an encoding unit 11A according to Embodiment 2 of the present invention. Referring to FIG. 7, the encoding unit 11A includes a data allocating unit 14, a hardware encoder 151, a software encoder 152, a combining unit 16, an output unit 17, and an allocation ratio setting unit 22A. 11 A of encoding parts contain the element similar to the element shown by FIG. 3 except the allocation ratio setting part 22A. In FIG. 7, the same reference numerals as those shown in FIG. 3 are given to the similar elements, and the detailed description thereof is used.

  Referring to FIG. 7, the allocation ratio setting unit 22A functions when the CPU 10 controls other elements of the computer terminal according to a predetermined program. The allocation ratio setting unit 22A sets the ratio between the number of data units allocated to the hardware encoder 151 and the number of data units allocated to the software encoder 152 in the video data stream to be encoded. The allocation ratio setting unit 22A may further cause the user to select the ratio value. That is, prior to the encoding process by the encoding unit 11A, the allocation ratio setting unit 22A displays a predetermined input screen on the display of the computer terminal, and receives the value of the above ratio from the user through a mouse or a keyboard. The value of the ratio selected by the user is passed from the allocation ratio setting unit 22A to the data allocating unit 14. The data allocator 14 allocates the video data stream to be encoded to the hardware encoder 151 and the software encoder 152 at that ratio.

  For example, it is assumed that the encoding data unit is one frame and the user sets the ratio of the number of frames allocated to the software encoder 152 to ½ with respect to the number of frames allocated to the hardware encoder 151. . The data allocator 14 first allocates the first frame of the video data stream to be encoded to the software encoder 152. Next, the data allocator 14 allocates the second and third frames of the video data stream to the hardware encoder 151. The data allocator 14 further allocates the fourth frame to the software encoder 152 and allocates the fifth and sixth frames to the hardware encoder 151. Similarly, the data allocator 14 allocates the next two frames to the hardware encoder 151 each time one frame is allocated to the software encoder 152.

  In addition, the data allocating unit 14 divides the video data to be encoded into a predetermined number of frames, for example, 15 frames, for example, every 15 frames into the memory unit 20 by the editing unit 13 and divides them into three equal parts, for example, the first 1/3, for example The fifth frame is allocated to the software encoder 152, and the remaining 2/3, for example, the sixth to fifteenth frames are allocated to the hardware encoder 151. While the software encoder 152 encodes the first to fifth frames, the hardware encoder 151 encodes the sixth to fifteenth frames.

  The encoding apparatus 100 according to the second embodiment uses the allocation ratio setting unit 22A to determine the ratio between the number of data units allocated to the hardware encoder 151 and the number of data units allocated to the software encoder 152 in the video data to be encoded. Can be selected by the user. Thereby, the user can adjust the ratio of the number of data units to be allocated to each encoder 151, 152 per unit time while watching the time required for the entire series of processing by the video editing system 200 incorporating the encoding unit 11A. As a result, the encoding apparatus 100 can make the overall encoding processing speed higher than the encoding processing speed of only the hardware encoder 151. Thereby, the encoding processing speed of the video editing system 200 can be improved.

<< Embodiment 3 >>
The encoding apparatus 100 according to the third embodiment of the present invention is incorporated in the non-linear video editing system 200 as in the first embodiment. The video editing system 200 is the same as that according to the first embodiment except for the encoding unit. Therefore, the description about Embodiment 1 is used for details of these similar elements.

  FIG. 8 is a block diagram of the encoding unit 11B according to Embodiment 3 of the present invention. Referring to FIG. 8, the encoding unit 11B includes a data allocation unit 14, a hardware encoder 151, a software encoder 152, a synthesis unit 16, an output unit 17, a CPU load estimation unit 23, an allocation ratio setting unit 22B, and A first database 24 is included. The encoding unit 11B includes the same elements as those shown in FIG. 3 except for the CPU load estimating unit 23, the allocation ratio setting unit 22B, and the first database 24. In FIG. 8, the same reference numerals as those shown in FIG. 3 are given to the similar elements, and the detailed description thereof is cited.

  Referring to FIG. 8, the CPU load estimator 23 functions when the CPU 10 controls other elements of the computer terminal according to a predetermined program. The CPU load estimation unit 23 predicts the load of the CPU 10 in the editing process by the editing unit 13 from editing information that defines the contents of the editing process.

  The CPU load estimation unit 23 receives editing information from the editing unit 13 prior to editing processing by the editing unit 13. The editing information includes a time range and time order of video data, effect contents, information on resolution conversion, information on frame rate conversion, and the like. Based on the editing information, the CPU load estimating unit 23 predicts the transition of the timeline processing index TE during the editing processing period by the editing unit 13 as follows. Here, the timeline processing index TE is defined by a processing amount of the CPU 10 required for the editing unit 13 to edit video data of one data unit, that is, a CPU load index. Hereinafter, for convenience of explanation, the data unit is one frame.

  First, the CPU load estimator 23 divides the entire editing processing period by the editing unit 13 into periods including a plurality of editing processes based on the editing information. Here, when the number of frames edited in one divided period is smaller than a predetermined threshold value, the frames are integrated in either period before or after that. Thereby, in each divided period, a frame that is equal to or greater than the threshold is edited.

  Next, the CPU load estimator 23 identifies the type of editing process in parallel for each divided period from the editing information, and obtains a CPU load index corresponding to each identified type. Here, for each type of editing process, the processing amount of the CPU 10 required for the editing process, that is, the CPU load index is evaluated in advance and stored in the memory unit 20 as the first database 24.

  FIG. 9 is a table showing an example of the CPU load index. Referring to FIG. 9, the CPU load index related to various other processes of the CPU 10 is evaluated with the CPU load index in the AVC-Intra video data decoding process being 100. The CPU load estimation unit 23 searches the first database 24 for the CPU load index corresponding to each type of the specified editing process. The CPU load estimation unit 23 further adds the retrieved CPU load indexes for each divided period, and determines each sum as a timeline processing index TE for the divided period.

  For example, it is assumed that the editing information expresses an editing process performed in a certain period as “compositing two AVC-Intra50 materials with picture-in-picture and inserting one title”. First, the CPU load estimation unit 23 specifies “decoding of AVC-Intra50 material”, “effect processing by picture-in-picture”, and “title addition” as types of editing processing performed in parallel during the period. Next, the CPU load estimating unit 23 searches the first database 24 for 100, 5, and 5 as CPU load indexes corresponding to the specified types of editing processing. The CPU load estimator 23 further determines the timeline processing index TE from the retrieved CPU load indexes 100, 5, and 5 by the following formula: TE = 100 × 2 + 5 + 5 = 210. Here, since the material of the AVC-Intra 50 is 2 frames, the CPU load index for these decoding processes is evaluated at twice the value 100 per frame.

  The CPU load estimator 23 reads from the editing information a time code indicating the start position of the time range of the video data stream edited in each divided period, and associates it with the timeline processing index TE in that period. In this way, the transition of the timeline processing index TE during the editing processing period by the editing unit 13 is predicted. The CPU load estimating unit 23 further gives a combination of the timeline processing index TE and the time code to the allocation ratio setting unit 22B.

  The allocation ratio setting unit 22B functions by the CPU 10 controlling other elements of the computer terminal according to a predetermined program. The allocation ratio setting unit 22B is allocated to both encoders 151 and 152 per unit time by the data allocation unit 14 based on the transition of the load of the CPU 10 predicted by the CPU load estimation unit 23, that is, the transition of the timeline processing index TE. Set the video data ratio. For example, when the encoding data unit is one frame, the allocation ratio setting unit 22B determines the number of frames HF allocated to the hardware encoder 151 per unit time and the software encoder 152 according to each timeline processing index TE. The ratio with the number of allocated frames SF is calculated as follows.

  The allocation ratio setting unit 22B first receives the combination of the timeline processing index TE and the time code from the CPU load estimation unit 23. Next, the allocation ratio setting unit 22B calculates a ratio CE / TE of the CPU processing capacity index CE to each timeline processing index TE. The CPU processing capability index CE is defined as a processing amount per unit time that can be provided by the CPU 10 for editing processing by the editing unit 13 or encoding processing by the software encoder 152, for example, processing amount per frame period. Here, the CPU 10 generally also performs processing related to other modules such as the OS of the computer terminal, the data allocator 14, and the synthesizer 16. The CPU processing capability index CE is obtained by subtracting the processing amount related to the processing from the total processing amount that can be provided by the CPU 10. Therefore, the ratio CE / TE is the maximum value of the editing processing speed obtained when the processing amount that can be provided by the CPU 10 is all provided to the same editing processing during the period evaluated by the timeline processing index TE, that is, the unit. This represents the maximum number MF of frames that can execute the same editing process per time.

  The allocation ratio setting unit 22B further compares each maximum value MF with the encoding processing speed by the hardware encoder 151, that is, the number of frames HF that the hardware encoder 151 can encode per unit time.

  When the maximum value MF of the editing processing speed is equal to or lower than the encoding processing speed HF by the hardware encoder 151, the editing processing evaluated at the maximum value MF can be regarded as being slower than the encoding processing by the hardware encoder 151. In that case, the allocation ratio setting unit 22B sets the number of frames SF allocated to the software encoder 152 to “0” for the editing processing period, thereby preventing an increase in the burden on the CPU 10.

  When the maximum value MF of the editing processing speed exceeds the encoding processing speed HF by the hardware encoder 151, it can be considered that the editing processing evaluated at the maximum value MF is faster than the encoding processing by the hardware encoder 151. In that case, the allocation ratio setting unit 22B sets the number of frames SF allocated to the software encoder 152 to a value larger than “0” as follows for the editing processing period. Thereby, the encoding process time in the whole encoding part 11B by the parallel processing of both the encoders 151 and 152 can be aimed at.

  The encoding processing speed HF by the hardware encoder 151 is constant regardless of the load fluctuation of the CPU 10. Therefore, the allocation ratio setting unit 22B first sets the number of frames allocated to the hardware encoder 151 per unit time as the encoding processing speed HF. In this case, in order to operate the hardware encoder 151 in parallel with the editing unit 13 without delay, the processing amount per unit time of the CPU 10 with respect to the editing unit 13 is at least the timeline processing index TE and the code by the hardware encoder 151. It is preferable to maintain the product TE × HF with the conversion processing rate HF. Therefore, the remaining RE = CE−TE × HF obtained by removing the above-mentioned product TE × HF from the CPU processing power index CE is regarded as the upper limit of the processing amount of the CPU 10 that can be provided to the software encoder 152 per unit time. Can do. The allocation ratio setting unit 22B calculates the upper limit RE for each timeline processing index TE. The upper limit RE is equal to a value obtained by multiplying the difference between the maximum value MF of the editing processing speed and the encoding processing speed HF by the hardware encoder 151 by the timeline processing index TE: RE = TE × (MF -HF). Accordingly, when the maximum value MF of the editing processing speed exceeds the encoding processing speed HF by the hardware encoder 151, the upper limit RE is larger than “0”.

  On the other hand, the allocation ratio setting unit 22B adds the processing amount per unit time of the CPU 10 required for the encoding process by the software encoder 152, that is, the software encoding processing index SE to each timeline processing index TE and adds the sum QE = SE + TE. Ask. Here, the software encoding processing index SE differs for each encoding method. The first database 24 stores a software encoding processing index SE evaluated for each type of encoding method. The table of FIG. 9 shows an example of the CPU load index for each type of encoding process. These CPU load indices are used as the software encoding process index SE. The allocation ratio setting unit 22B searches the first database 24 in advance for the software encoding index SE corresponding to the type of encoding method that is actually used.

The above sum QE represents the processing amount per unit time of the CPU 10 required for the parallel processing of the editing process corresponding to the timeline processing index TE and the encoding process corresponding to the software encoding process index SE. Therefore, the allocation ratio setting unit 22B divides the upper limit RE by the sum QE obtained from each timeline processing index TE, and the quotient RE / QE is an editing processing period indicated by the time code corresponding to the timeline processing index TE. Among these, the number of frames SF allocated to the software encoder 152 per unit time may be set. Thereby, it can be expected that the software encoder 152 operates in parallel with the editing unit 13 without delay.
Since the upper limit RE is greater than “0”, the number of frames SF allocated to the software encoder 152 is also greater than “0”.

  Here, even when the unit time is equal to the frame period, the number of frames SF and the encoding processing speed HF by the hardware encoder 151 are not necessarily integers. Therefore, the allocation ratio setting unit 22B approximates the ratio SF: HF between the number of frames SF obtained from each timeline processing index TE and the encoding processing speed HF by the hardware encoder 151 by the integer ratio HFI: SFI. For example, the sum HFI + SFI of two integers HFI and SFI representing the integer ratio HFI: SFI is set smaller than the total number of frames edited in the editing processing period indicated by the time code corresponding to the timeline processing index TE.

  The allocation ratio setting unit 22B sets the number of frames SF = 0 or the integer ratio HFI: SFI to the data allocating unit 14 as follows. First, every time one frame to be encoded is written from the editing unit 13 to the memory unit 20, the data allocating unit 14 reads the time code from the frame and passes it to the allocation ratio setting unit 22B. The allocation ratio setting unit 22B compares the time code with each time code received from the CPU load estimation unit 23. When the time code from the data allocator 14 reaches a value smaller than the time code from the CPU load estimator 23 by a predetermined amount, the allocation ratio setting unit 22B determines the time corresponding to the time code from the CPU load estimator 23. The number of frames SF = 0 or the integer ratio HFI: SFI obtained from the line processing index TE is given to the data allocator 14. When the number of frames SF = 0 is given, the data allocator 14 thereafter allocates all the encoding target frames to the hardware encoder 151. On the other hand, when the integer ratio HFI: SFI is given, the data allocator 14 continuously allocates the following HFI frames to the hardware encoder every time SFI frames are continuously allocated to the software encoder 152. Assign to encoder 151. The data allocator 14 continues the original allocation operation until the number of frames SF = 0 or the integer ratio HFI: SFI is newly input from the allocation ratio setting unit 22B.

  Here, the above-mentioned predetermined amount is a general value between the time code of the encoding target frame written in the memory unit 20 by the editing unit 13 and the time code of the frame edited in parallel with the writing. It depends on the difference. That is, the predetermined amount is adjusted so that the latter time code is estimated from the former time code.

  Apart from the above, the allocation ratio setting unit 22B uses the ratio α = HF: SF between the number of frames SF obtained from each timeline processing index TE and the encoding processing speed HF by the hardware encoder 151 as a data allocating unit. 14 may be given. In this case, the data allocator 14 writes the frame to the hardware encoder 151 with a probability α / (1 + α) every time one frame to be encoded is written from the editing unit 13 to the input buffer area BI in the memory unit 20. To the software encoder 152 with probability 1 / (1 + α).

  In this way, the allocation ratio setting unit 22B changes the ratio HF: SF of the number of frames per unit time allocated to the encoders 151 and 152 by the data allocation unit 14 for each timeline processing index TE. As a result, even if the load on the CPU 10 fluctuates according to the content of the editing process by the editing unit 13, the encoding processing speed of the entire encoding unit 11B is maintained while the waiting time of each of the encoders 151 and 152 is kept sufficiently short. Increases up to the maximum (HF + SF) / HF times from the encoding processing speed HF only by the hardware encoder 151. Therefore, processing by the entire video editing system 200 is accelerated.

  For example, when the CPU processing capability index CE is “400” and the software encoding processing index SE is “200” when one frame period is a unit time, in the editing processing period in which the timeline processing index TE is 200, The encoding processing speed CE / QE by only the software encoder 152 is equal to “1”. On the other hand, the maximum value MF = CE / TE of the editing processing speed is equal to “2”. Accordingly, when the encoding processing speed HF by the hardware encoder 151 is “1”, the value is smaller than the maximum value MF, so that the data allocator 14 sends the video data to be encoded to both the encoders 151 and 152 as frames. Number ratio HF: SF = 1: 0.5 = 2: 1. As a result, the encoding processing speed of the entire encoding unit 11B is increased up to 1.5 times the encoding processing speed of only the hardware encoder 151.

  If the encoding processing speed HF by the hardware encoder 151 is “2”, the value is equal to the maximum value MF of the editing processing speed. Therefore, the data allocator 14 may allocate video data to be encoded only to the hardware encoder 151. The entire video editing system 200 can be efficiently operated while maintaining a balance between the editing processing speed and the encoding processing speed.

  On the other hand, in the editing processing period in which the timeline processing index TE is “100”, the encoding processing speed CE / QE by only the software encoder 152 is “1.33”. At this time, since the maximum value MF = CE / TE of the editing processing speed is equal to “4”, when the encoding processing speed HF by the hardware encoder 151 is “1”, the value is smaller than the maximum value MF. Accordingly, the data allocator 14 allocates the video data to be encoded to both encoders 151 and 152 at a frame number ratio HF: SF = 1: 1. As a result, the encoding processing speed of the entire encoding unit 11B is increased up to twice as much as the encoding processing speed of only the hardware encoder 151, and is up to 1.5 times higher than the encoding processing speed of only the software encoder 152. Go up to double.

  As another example, when the CPU processing power index CE is “400” and the software encoding processing index SE is “1000”, the software encoder 152 is used in the editing processing period in which the timeline processing index TE is “200”. The encoding processing speed CE / QE based only on “0.33”. At that time, since the maximum value MF = CE / TE of the editing processing speed is equal to “2”, when the encoding processing speed HF by the hardware encoder 151 is “1”, the value is smaller than the maximum value MF. Therefore, the data allocator 14 allocates the video data to be encoded to the encoders 151 and 152 at a frame number ratio HF: SF = 1: 0.17 = 6: 1. As a result, the encoding processing speed of the entire encoding unit 11B is increased up to 1.17 times higher than the encoding processing speed of only the hardware encoder 151, and up to 3 times higher than the encoding processing speed of only the software encoder 152. .Up to 5 times.

  FIG. 10 is a flowchart of processing for setting the ratio of video data allocation by the encoding unit 11B. According to the flowchart of FIG. 10, the CPU load estimation unit 23 predicts the transition of the timeline processing index TE based on the editing information, and the allocation ratio setting unit 22B determines the number of frames allocated to both encoders 151 and 152 based on the transition. Set the ratio. Hereinafter, the prediction process and the setting process will be described with reference to FIG. When receiving the editing information from the editing unit 13, the encoding unit 11B starts the following processing.

  First, in step S21, the CPU load estimating unit 23 receives the editing information from the editing unit 13, and based on that, divides the entire editing process period by the editing unit 13 into periods in which the types of parallel editing processes are common.

  Next, in step S22, the CPU load estimation unit 23 first specifies the type of editing process from the editing information for each divided period. Next, the CPU load estimator 23 searches the first database 24 for the CPU load index corresponding to each specified type. Subsequently, the CPU load estimator 23 adds the retrieved CPU load indexes for each divided period, and determines the sum as the timeline processing index TE for the divided period. Finally, the CPU load estimator 23 reads the time code indicating the start position of each divided period from the editing information, and associates the time code with the timeline processing index TE in that period. The combination is passed to the allocation ratio setting unit 22B.

  Next, in step S23, the allocation ratio setting unit 22B obtains the maximum value MF = CE / TE of the editing processing speed for each timeline processing index TE received from the CPU load estimating unit 23.

  Next, in step S24, the allocation ratio setting unit 22B compares each maximum value MF with the encoding processing speed HF by the hardware encoder 151. If the maximum value MF is equal to or less than the encoding processing speed HF (“Yes” in step S24), the process proceeds to step S25. When the maximum value MF is larger than the encoding processing speed HF (“No” in step S24), the process proceeds to step S26.

  In step S25, the allocation ratio setting unit 22B sets the number of frames SF allocated to the software encoder 152 to “0”. Then, the process proceeds to step S28.

  In step S26, the allocation ratio setting unit 22B first calculates the upper limit RE = CE−TE × HF of the processing amount of the CPU 10 that can be provided to the software encoder 152 per unit time. Next, the allocation ratio setting unit 22B calculates a value RE / QE obtained by dividing the upper limit RE by the sum QE = SE + TE of the software encoding processing index SE and the timeline processing index TE, and allocates it to the software encoder 152 per unit time. The number of frames to be set SF = RE / QE.

  Next, in step S27, the allocation ratio setting unit 22B obtains a ratio HF: SF between the set number of frames SF and the encoding processing speed HF by the hardware encoder 151. Generally, since the ratio HF: SF is not an integer ratio, the allocation ratio setting unit 22B approximates the ratio HF: SF with the integer ratio HFI: SFI. Then, the process proceeds to step S28.

  In step S28, the allocation ratio setting unit 22B uses the number of frames SF = 0 set in step S25, or the integer ratio HFI: SFI set in step S27, and the timeline processing index used for calculating the maximum value MF. Correspond to the time code corresponding to TE.

  Next, in step S29, the allocation ratio setting unit 22B checks whether or not the timeline processing index TE received from the CPU load estimation unit 23 has not undergone the processing of steps S23 to S28. When the unprocessed timeline processing index TE remains (in the case of “Yes” in step S29), the processing returns to step S23. When there is no unprocessed timeline processing index TE remaining (in the case of “No” in step S29), the processing ends.

  FIG. 11 is a flowchart of video data allocation processing by the encoding unit 11B. The allocation ratio setting unit 22B and the data allocator 14 allocate the video data streams to be encoded to both encoders 151 and 152 according to the flowchart of FIG. Hereinafter, the allocation process will be described with reference to FIG. The following processing can be started, for example, when the editing unit 13 starts outputting a video data stream to be encoded.

  In the first step S31, the allocation ratio setting unit 22B sets the earliest playback time among the time codes received from the CPU load estimation unit 23 as the target time code Tr.

  Next, in step S32, when one frame to be encoded is written from the editing unit 13 to the input buffer area BI in the memory unit 20, the data allocating unit 14 reads the time code Tc from the frame and allocates an allocation ratio setting unit. Pass to 22B.

  Next, in step S33, the allocation ratio setting unit 22B compares the time code Tc from the data allocating unit 14 with a value smaller than the target time code Tr by a predetermined amount. If the time code Tc from the data allocator 14 is equal to or greater than the value (Tr−predetermined amount) (“Yes” in step S33), the process proceeds to step S34. If the time code Tc from the data allocator 14 is smaller than the value (Tr−predetermined amount) (“No” in step S33), the process returns to step S32.

  Next, in step S34, the allocation ratio setting unit 22B provides the data allocation unit 14 with the number of frames SF = 0 corresponding to the target time code Tr or the integer ratio HFI: SFI. If the number of frames SF = 0 is given, the process proceeds to step S35, and if the integer ratio HFI: SFI is given, the process proceeds to step S36.

  In step S35, the data allocating unit 14 performs the encoding stored in the input buffer area BI in the memory unit 20 until the number of frames SF = 0 or the integer ratio HFI: SFI is newly input from the allocation ratio setting unit 22B. All target frames are supplied to the hardware encoder 151. Then, the process proceeds to step S37.

  In step S36, the data allocator 14 is given an integer ratio HFI: SFI. Therefore, each time the data allocator 14 continuously allocates the SFI frames to the software encoder 152, the data allocator 14 continuously allocates the subsequent HFI frames to the hardware encoder 151. The data allocating unit 14 continues the allocation operation until the number of frames SF = 0 or the integer ratio HFI: SFI is newly input from the allocation ratio setting unit 22B. Then, the process proceeds to step S37.

  In step S37, the data allocator 14 determines whether or not the allocated frame is the last frame of the video data stream to be encoded. If the frame is not the last ("No" in step S37), the process returns to step S31. If the frame is the last ("Yes" in step S37), the process ends. When the process is repeated from step S31, the time code received from the CPU load estimation unit 23 by the allocation ratio setting unit 22B is set as the target time code Tr in order from the earliest playback time for each repetition.

  In the encoding device 100 of the third embodiment, the CPU load estimation unit 23 predicts the transition of the timeline processing index TE based on the editing information, and the allocation ratio setting unit 22B per unit time allocated to both the encoders 151 and 152. The ratio HF: SF of the number of frames is set based on the transition of the predicted timeline processing index TE. Thereby, even if the load on the CPU 10 fluctuates in accordance with the content of the editing process by the editing unit 13, the encoding device 100 reduces the time that each encoder 151, 152 waits for input of a frame to be encoded, Alternatively, the encoding processing by both encoders 151 and 152 can be performed in parallel while suppressing the occurrence of the time. As a result, the encoding apparatus 100 can perform the overall encoding process faster than the encoding process using only the hardware encoder 151. Thereby, the encoding processing speed of the video editing system 200 can be improved.

<< Embodiment 4 >>
The encoding apparatus 100 according to the fourth embodiment of the present invention is incorporated in the non-linear video editing system 200 as in the first embodiment. The video editing system 200 is the same as that according to the first embodiment except for the encoding unit. Therefore, the description about Embodiment 1 is used for details of these similar elements.

  FIG. 12 is a block diagram of an encoding unit 11C according to Embodiment 4 of the present invention. Referring to FIG. 12, the encoding unit 11C includes a data allocating unit 14, a hardware encoder 151, a software encoder 152, a combining unit 16, an output unit 17, a CPU load detecting unit 25, an allocation ratio setting unit 22C, and A second database 26 is included. The encoding unit 11C includes the same elements as those shown in FIG. 3 except for the CPU load detection unit 25, the allocation ratio setting unit 22C, and the second database 26. In FIG. 12, the same reference numerals as those shown in FIG. 3 are given to the similar elements, and the detailed description thereof is used.

  The CPU load detection unit 25 functions when the CPU 10 controls other elements of the computer terminal according to a predetermined program. The CPU load detection unit 25 measures the usage rate of the CPU 10 at predetermined time intervals using an API provided by the OS 27 of the computer terminal. The CPU load detection unit 25 further evaluates the load from the usage rate of the CPU 10, and notifies the allocation ratio setting unit 22C in real time. For example, the CPU load detection unit 25 divides the entire range of 0 to 100% that can be taken by the usage rate into several stages, and evaluates the load on the CPU 10 according to which stage the measured usage rate belongs. Further, when the stage to which the measured usage rate of the CPU 10 belongs is changed, the CPU load detection unit 25 notifies the allocation ratio setting unit 22C of the change together with the type of the stage after the change. For example, let us consider a case where the usage rate of the CPU 10 is divided into three stages of 0 to 20%, 20 to 80%, and 80 to 100%. When the measured usage rate of the CPU 10 changes from the first stage 0 to 20% to the second stage 20 to 80%, the CPU load detection unit 25 determines that the change has occurred and the stage after the change. Both are notified to the allocation ratio setting unit 22C that the second stage is 20 to 80%.

  The allocation ratio setting unit 22C determines the ratio HF: SF of the number of frames to be allocated to both encoders 151 and 152 per unit time according to the stage after change notified from the CPU load detection unit 25. The ratio HF: SF of the number of frames is evaluated in advance at each stage of the usage rate, and is stored as the second database 26 in the memory unit 20. In the second database 26, an integer ratio HFI: SFI is stored as the frame number ratio HF: SF. For example, the sum HFI + SFI of two integers constituting each integer ratio HFI: SFI is smaller than the total number of frames edited in a time interval for measuring the usage rate of the CPU 10. The higher the usage rate of the CPU 10, the higher the ratio of the number of frames HF: SF is set. That is, as the load on the CPU 10 is heavier, the number of frames SF allocated to the software encoder 152 per unit time is reduced. The allocation ratio setting unit 22 </ b> C searches the second database 26 for the ratio of the number of frames corresponding to the use rate stage notified from the CPU load detection unit 25. Further, the allocation ratio setting unit 22C passes the ratio of the searched number of frames to the data allocating unit 14. Thereby, the data allocator 14 allocates video data to be encoded to both encoders 151 and 152 according to the ratio of the new number of frames.

  FIG. 13 is a flowchart of video data allocation processing by the encoding unit 11C. The CPU load detection unit 25, the allocation ratio setting unit 22C, and the data allocation unit 14 change the ratio of the number of frames allocated to both encoders 151 and 152 according to the usage rate of the CPU 10 according to the flowchart of FIG. Let Hereinafter, the setting process of the ratio of the number of frames will be described with reference to FIG. The following processing can be started, for example, when the editing unit 13 starts outputting a video data stream to be encoded.

  First, in step S41, the CPU load detection unit 25 measures the usage rate of the CPU 10 using the API of the OS 27.

  Next, in step S42, the CPU load detection unit 25 identifies the stage to which the measured usage rate of the CPU 10 belongs, and determines whether or not it is the same as the previous stage. If the stage has changed (“Yes” in step S42), the process proceeds to step S43. If the stage is the same (in the case of “No” in step S42), the process returns to step S41.

  Next, in step S43, the CPU load detection unit 25 notifies the allocation ratio setting unit 22C that the stage to which the usage rate of the CPU 10 belongs is changed along with the type of the stage after the change. The allocation ratio setting unit 22C searches the second database 26 for the ratio of the number of frames corresponding to the notified use rate stage. Further, the allocation ratio setting unit 22C passes the ratio of the searched number of frames to the data allocating unit 14.

  Next, in step S44, the data allocator 14 allocates video data to be encoded to both encoders 151 and 152 in accordance with the ratio of the new number of frames.

  Next, in step S45, the data allocator 14 determines whether or not the allocated frame is the last frame of the video data stream to be encoded. If the frame is not the last (“No” in step S45), the process returns to step S41. If the frame is the last (“Yes” in step S45), the process ends.

  The loop of steps S41 to S45 is repeated at a predetermined time interval. Thereby, the ratio of the video data allocated to both the encoders 151 and 152 is dynamically adjusted according to the actual load fluctuation of the CPU 10.

  In the encoding device 100 of the fourth embodiment, the CPU load detection unit 25 measures the usage rate of the CPU 10 in real time, and the allocation ratio setting unit 22C adjusts both encoders according to the measured usage rate of the CPU 10. The ratio HF: SF of the number of frames per unit time allocated to 151 and 152 is set. As a result, the encoding apparatus 100 shortens the time for which the encoders 151 and 152 wait for the input of the video data to be encoded regardless of the usage rate of the CPU 10, that is, the load variation, or generates the time. Encoding processing by both encoders 151 and 152 can be performed in parallel while being suppressed. Therefore, the encoding apparatus 100 can perform the overall encoding process faster than the encoding process using only the hardware encoder 151. Thereby, the encoding processing speed of the video editing system 200 can be improved.

<< Embodiment 5 >>
The encoding apparatus 100 according to the fifth embodiment of the present invention is incorporated in the non-linear video editing system 200 as in the first embodiment. The video editing system 200 is the same as that according to the first embodiment except for the encoding unit. Therefore, the description about Embodiment 1 is used for details of these similar elements.

  FIG. 14 is a block diagram of an encoding unit 11D according to Embodiment 5 of the present invention. Referring to FIG. 14, the encoding unit 11D includes a data allocator 14, a hardware encoder 151, a software encoder 152, a synthesizer 16, an output unit 17, a CPU load estimator 23, an allocation ratio setting unit 22D, 1 database 24, CPU load detection unit 25, and second database 26 are included. The encoding unit 11D substantially has the elements of the third embodiment shown in FIG. 8 and the elements of the fourth embodiment shown in FIG. In FIG. 14, the same reference numerals as those shown in FIGS. 8 and 12 are given to the similar elements, and the description of FIGS. 3, 8, and 12 is used for the details thereof.

  Similar to the allocation ratio setting unit 22B according to the third embodiment, the allocation ratio setting unit 22D uses the data allocating unit 14 to change both the load of the CPU 10 predicted by the CPU load estimation unit 23, that is, the transition of the timeline processing index TE. A ratio of video data allocated to the encoders 151 and 152 per unit time is set.

  Similar to the allocation ratio setting unit 22B according to the third embodiment, the allocation ratio setting unit 22D compares the time code with each time code received from the CPU load estimation unit 23 every time the time code is passed from the data allocation unit 14. . On the other hand, the allocation ratio setting unit 22D monitors the usage rate of the CPU 10 through the CPU load detection unit 25. Each time the usage rate stage of the CPU 10 changes, the allocation ratio setting unit 22D determines the CPU load index corresponding to the stage after the change. Here, the CPU load index is evaluated in advance for each stage of the usage rate and recorded in the second database 26. The allocation ratio setting unit 22D searches the second database 26 for a CPU load index corresponding to the changed usage rate stage.

  When the time code from the data allocator 14 reaches a value smaller than the time code from the CPU load estimator 23 by a predetermined amount, the allocation ratio setting unit 22D determines the time corresponding to the time code from the CPU load estimator 23. The line processing index TE is compared with the CPU load index LE corresponding to the stage of utilization of the CPU 10 at that time. When the timeline processing index TE is larger than the CPU load index LE, the allocation ratio setting unit 22D, like the allocation ratio setting unit 22B according to the third embodiment, the number of frames SF = 0 or the integer ratio obtained from the timeline processing index TE HFI: SFI is given to the data allocator 14. On the other hand, when the timeline processing index TE is equal to or less than the CPU load index LE, the allocation ratio setting unit 22D sets the ratio of the number of frames corresponding to the usage rate stage of the CPU 10 like the allocation ratio setting unit 22D according to the fourth embodiment. 2 Retrieve from the database 26 and pass to the data allocator 14. Thus, when the actual load of the CPU 10 is equal to or higher than the load predicted from the editing information, the video data is not a ratio of the number of frames set based on the editing information, but a frame suitable for the actual load. The encoders 151 and 152 are allocated in a ratio of numbers. Thereby, a decrease in the processing speed of the entire video editing system 200 due to an error in predicting the load on the CPU 10 is prevented.

  FIG. 15 is a flowchart of video data allocation processing by the encoding unit 11D. The CPU load estimation unit 23, the CPU load detection unit 25, the allocation ratio setting unit 22D, and the data allocation unit 14 change the ratio of the number of frames allocated per unit time to both encoders 151 and 152 according to the flowchart of FIG. . Hereinafter, the processing for setting the ratio of the number of frames will be described with reference to FIG. FIG. 15 includes the same steps as those shown in FIGS. 11 and 13. Therefore, in FIG. 15, the same reference numerals as those shown in FIG. 11 and FIG. Furthermore, for the details of these similar steps, the description of FIG. 11 and FIG. 13 is incorporated. The following processing can be started, for example, when the editing unit 13 starts outputting a video data stream to be encoded.

  First, in step S31, the allocation ratio setting unit 22D sets the time code received from the CPU load estimation unit 23 as the target time code Tr in order of the reproduction time.

  Next, in step S <b> 32, the allocation ratio setting unit 22 </ b> D reads the time code Tc from the data allocation unit 14.

  In the next step S33, the allocation ratio setting unit 22D compares the time code Tc with a value smaller than the target time code Tr by a predetermined amount. If the time code Tc is equal to or greater than the value (Tr−predetermined amount) (“Yes” in step S33), the process proceeds to step S51. If the time code Tc is smaller than the value (Tr−predetermined amount) (“No” in step S33), the process is repeated from step S32.

  In step S51, the allocation ratio setting unit 22D obtains a timeline processing index TE corresponding to the target time code Tr.

  Next, in step S52, the CPU load detection unit 25 measures the usage rate of the CPU 10 at predetermined time intervals using the API of the OS 27. The CPU load detection unit 25 further identifies the stage to which the measured usage rate of the CPU 10 belongs, and determines whether or not it is the same as the previous stage. When the stage changes, the CPU load detection unit 25 notifies the allocation ratio setting unit 22D that the stage to which the usage rate of the CPU 10 belongs is changed along with the type of the stage after the change. The allocation ratio setting unit 22D searches the second database 26 for the CPU load index LE corresponding to the notified usage rate stage.

  Next, in step S53, the allocation ratio setting unit 22D compares the timeline processing index TE with the CPU load index LE. If the timeline processing index TE is larger than the CPU load index LE (“Yes” in step S53), the process proceeds to step S34. Thereby, similarly to steps S34 to S37 according to the third embodiment, the video data is allocated to both encoders 151 and 152 with the number of frames SF = 0 or the integer ratio HFI: SFI obtained from the timeline processing index TE. On the other hand, if the timeline processing index TE is equal to or less than the CPU load index LE (“No” in step S53), the process proceeds to step S43. As a result, as in steps S43 to S45 according to the fourth embodiment, the video data is allocated to both encoders 151 and 152 at the ratio of the number of frames corresponding to the stage of utilization of the CPU 10.

  First, the encoding apparatus 100 according to the fifth embodiment predicts the transition of the timeline processing index TE based on the editing information by the CPU load estimation unit 23, and the unit time allocated to both encoders 151 and 152 by the allocation ratio setting unit 22D. The ratio HF: SF of the number of hit frames is set based on the transition of the predicted timeline processing index TE. Next, the encoding device 100 measures the usage rate stage of the CPU 10 in real time by the CPU load detecting unit 25, and calculates the actual load LE of the CPU 10 from the measured usage rate stage of the CPU 10 by the allocation ratio setting unit 22D. Evaluate and compare with the timeline processing index TE. When the actual load LE of the CPU 10 is smaller than the timeline processing index TE, the encoding apparatus 100 sets both the video data to be encoded at the frame number ratio HF: SF set based on the timeline processing index TE. The encoders 151 and 152 are assigned. On the other hand, when the actual load LE of the CPU 10 is equal to or higher than the timeline processing index TE, the encoding apparatus 100 determines the number of frames in which the video data to be encoded is associated with the actual load LE of the CPU 10 in advance. Are assigned to both encoders 151 and 152. As a result, even if the load on the CPU 10 fluctuates more than the load predicted from the content of the editing process by the editing unit 13, the encoding apparatus 100 waits for the encoders 151 and 152 to wait for the input of the encoding target frame. While shortening or suppressing the occurrence of the time, the encoding processing by both encoders 151 and 152 can be performed in parallel. As a result, the encoding apparatus 100 can perform the overall encoding process faster than the encoding process using only the hardware encoder 151. Thereby, the encoding processing speed of the video editing system 200 can be improved.

Embodiment 6
The encoding apparatus 100 according to the sixth embodiment of the present invention is incorporated in the non-linear video editing system 200 as in the first embodiment. The video editing system 200 is the same as that according to the first embodiment except for the encoding unit. Therefore, the description about Embodiment 1 is used for details of these similar elements.

  FIG. 16 is a block diagram of an encoding unit 11E according to Embodiment 6 of the present invention. Referring to FIG. 16, the encoding unit 11E includes a data allocating unit 14, a hardware encoder 151, a software encoder 152, a synthesizing unit 16, an output unit 17, a processing speed measuring unit 28, and an allocation ratio setting unit 22E. Including. The encoding unit 11E includes the same elements as those shown in FIG. 3 except for the processing speed measurement unit 28 and the allocation ratio setting unit 22E. In FIG. 16, the same reference numerals as those shown in FIG. 3 are given to the similar elements, and the description of FIG.

  The processing speed measuring unit 28 functions when the CPU 10 controls other elements of the computer terminal according to a predetermined program. The processing speed measurement unit 28 encodes the data units to be encoded from the encoders 151 and 152 to the synthesis unit 16 from the time when the data units to be encoded are allocated to the encoders 151 and 152 by the data allocation unit 14. Measure the time to the point. Specifically, the processing speed measuring unit 28 first measures the time from the data allocating unit 14 to the synthesizing unit 16, the number of the data unit allocated to each encoder 151, 152, and the encoding of the data unit. It starts from the time when the address of the write destination is passed. Next, the processing speed measuring unit 28 ends the measurement of the time when the same address as that address is output from one of the encoders 151 and 152 to the combining unit 16. The processing speed measurement unit 28 further calculates the encoding processing speed of each of the encoders 151 and 152 from the measured time and passes it to the allocation ratio setting unit 22E. In addition, the processing speed measurement unit 28 may measure the bit rate of the video data output from the encoders 151 and 152, and calculate the encoding processing speed of the encoders 151 and 152 from the obtained bit rate.

  The allocation ratio setting unit 22E, based on the encoding processing speed HF1 by the hardware encoder 151 and the encoding processing speed SF1 by the software encoder 152, is a ratio HF1 / SF1 of the encoding processing speed between the encoders 151 and 152. And the ratio HF2 / SF2 of the number of frames allocated to the hardware encoder 151 and the software encoder 152 per unit time at that time. When the ratio HF1 / SF1 of the encoding processing speed is different from the ratio HF2 / SF2 of the number of frames, the allocation ratio setting unit 22E uses the ratio of the encoding processing speed as the ratio of the number of frames allocated to both encoders 151 and 152 per unit time. HF1 / SF1 is passed to the data allocator 14. Here, since the ratio of the encoding processing speed is generally not an integer ratio, the allocation ratio setting unit 22E approximates the ratio of the encoding processing speed with an integer ratio in advance.

  FIG. 17 is a flowchart of video data allocation processing by the encoding unit 11E. The processing speed measurement unit 28, the allocation ratio setting unit 22E, and the data allocation unit 14 encode the ratio of the number of frames allocated to each encoder 151, 152 per unit time according to the flowchart of FIG. The ratio is adjusted to the ratio of the processing speed. Hereinafter, the adjustment process will be described with reference to FIG. The following processing can be started, for example, when the editing unit 13 starts outputting a video data stream to be encoded.

  First, in step S61, the processing speed measurement unit 28 encodes each encoder 151, 152 from the time when the data unit to be encoded is allocated to each encoder 151, 152 by the data allocator 14. The time from the time 152 is output to the combining unit 16 is measured. The processing speed measurement unit 28 further calculates the encoding processing speed of each of the encoders 151 and 152 from the measured time and passes it to the allocation ratio setting unit 22E.

  Next, in step S62, the allocation ratio setting unit 22E obtains a ratio of the encoding processing speed between the encoders 151 and 152 based on the encoding processing speed received from the processing speed measuring unit 28, and calculates the ratio as an integer ratio. Approximate. The allocation ratio setting unit 22E further compares the ratio of the encoding processing speed with the ratio of the number of frames allocated to the encoders 151 and 152 per unit time at that time. If the ratio HF1 / SF1 of the encoding processing speed is different from the ratio HF2 / SF2 of the number of frames (“Yes” in step S62), the process proceeds to step S63. If the ratio HF1 / SF1 of the encoding processing speed is equal to the ratio HF2 / SF2 of the number of frames (“No” in step S62), the process returns to step S61.

  In step S63, the allocation ratio setting unit 22E passes the encoding processing speed ratio to the data allocating unit 14 as a new value of the ratio of the number of frames to be allocated to both encoders 151 and 152 per unit time.

  Next, in step S64, the data allocator 14 allocates video data to be encoded to both encoders 151 and 152 at the newly set ratio of the number of frames.

  Next, in step S65, the data allocator 14 determines whether the allocated frame is the last frame of the video data stream to be encoded. If the frame is not the last (“No” in step S65), the process returns to step S61. If the frame is the last (“Yes” in step S65), the process ends. The loop of steps S <b> 61 to S <b> 65 is repeated at predetermined time intervals each time video data is output from the encoders 151 and 152 to the combining unit 16.

  In the encoding apparatus 100 of the sixth embodiment, the processing speed measurement unit 28 measures the actual encoding processing speed by the encoders 151 and 152, and the number of frames allocated to both encoders 151 and 152 by the allocation ratio setting unit 22E. Is adjusted to the ratio of the actual encoding processing speed between the encoders 151 and 152. As a result, the encoding apparatus 100 waits for the input of the frames encoded by the encoders 151 and 152 regardless of the variation in the encoding processing speed by the software encoder 152 due to the load variation of the CPU 10. The encoding processing by both encoders 151 and 152 can be performed in parallel while shortening the time or suppressing the occurrence of the time. As a result, the encoding apparatus 100 can perform the overall encoding process faster than the encoding process using only the hardware encoder 151. Thereby, the encoding processing speed of the video editing system 200 can be improved.

<< Embodiment 7 >>
The encoding apparatus 100 according to the seventh embodiment of the present invention is incorporated in the non-linear video editing system 200 as in the first embodiment. The video editing system 200 is the same as that according to the first embodiment except for the encoding unit. Therefore, the description about Embodiment 1 is used for details of these similar elements.

  FIG. 18 is a block diagram of an encoding unit 11F according to Embodiment 7 of the present invention. Referring to FIG. 18, the encoding unit 11F includes a data allocator 14, a hardware encoder 151, a software encoder 152, a synthesizer 16, an output unit 17, a CPU load detector 25, an allocation ratio setting unit 22F, and a process. A speed measurement unit 28 and a second database 26 are included. The encoding unit 11F substantially has the elements according to the fourth embodiment in FIG. 12 and the elements according to the sixth embodiment in FIG. In FIG. 18, the same reference numerals as those shown in FIGS. 12 and 16 are given to the similar elements, and the description of FIGS. 3, 12, and 16 is used for the details.

  The allocation ratio setting unit 22F, like the allocation ratio setting unit 22E according to the sixth embodiment, is based on the encoding processing speed received from the processing speed measuring unit 28, and the ratio HF1 / of the encoding processing speed between the encoders 151 and 152. SF1 is obtained and compared with the ratio HF2 / SF2 of the number of frames allocated per unit time to both encoders 151 and 152 at that time. On the other hand, the allocation ratio setting unit 22F monitors the usage rate of the CPU 10 through the CPU load detection unit 25.

  When the ratio HF1 / SF1 of the encoding processing speed is different from the ratio HF2 / SF2 of the number of frames, the allocation ratio setting unit 22F sets the ratio HF3 / SF3 of the number of frames corresponding to the use rate stage of the CPU 10 to the second database 26. Search from. The allocation ratio setting unit 22F further compares the ratio HF3 / SF3 of the searched frame numbers with the ratio HF1 / SF1 of the encoding processing speed. When the ratio HF1 / SF1 of the encoding processing speed is larger than the ratio HF3 / SF3 of the number of searched frames, the allocation ratio setting unit 22F uses both encoders 151 and 152 in unit time as in the allocation ratio setting unit 22E according to the sixth embodiment. The ratio HF1 / SF1 of the encoding processing speed is passed to the data allocator 14 as the ratio of the number of frames allocated per hit. On the other hand, when the ratio HF1 / SF1 of the encoding processing speed is equal to or less than the ratio HF3 / SF3 of the number of searched frames, the allocation ratio setting unit 22F is similar to the allocation ratio setting unit 22D according to the fourth embodiment, and both encoders 151 and 152 are used. As the ratio of the number of frames to be allocated per unit time, the ratio HF3 / SF3 of the number of frames corresponding to the stage of usage rate of the CPU 10 is passed to the data allocator 14.

  In this way, when the load indicated by the usage rate of the CPU 10 is equal to or greater than the load indicated by the ratio HF1 / SF1 of the encoding processing speed between the encoders 151 and 152, the allocation ratio setting unit 22F performs the video data to be encoded. Are assigned to both encoders 151 and 152 not at the ratio HF1 / SF1 of the encoding processing speed but at the ratio HF3 / SF3 of the number of frames suitable for the load indicated by the usage rate.

  FIG. 19 is a flowchart of video data allocation processing by the encoding unit 11F. The processing speed measurement unit 28, the CPU load detection unit 25, the allocation ratio setting unit 22F, and the data allocation unit 14 change the ratio of the number of frames allocated per unit time to both encoders 151 and 152 according to the flowchart of FIG. . Hereinafter, the setting process of the number of frames will be described with reference to FIG. Note that FIG. 19 includes the same steps as those shown in FIG. Accordingly, in FIG. 19, the same reference numerals as those shown in FIG. Furthermore, for the details of these similar steps, the description of FIG. 17 is incorporated. The following processing can be started, for example, when the editing unit 13 starts outputting a video data stream to be encoded.

  First, in step S61, the processing speed measurement unit 28 starts from the time when the data unit to be encoded is allocated to each of the encoders 151 and 152 until the time when the encoded data unit is output to the synthesis unit 16. The encoding processing speed of each of the encoders 151 and 152 is calculated based on the above time, and passed to the allocation ratio setting unit 22F.

  Next, in step S62, the allocation ratio setting unit 22F obtains the ratio HF1 / SF1 of the encoding processing speed between the encoders 151 and 152 based on the encoding processing speed of each encoder 151 and 152, and the ratio is an integer. Approximate by ratio. The allocation ratio setting unit 22F further determines that the ratio HF1 / SF1 of the encoding processing speed is different from the ratio HF2 / SF2 of the number of frames allocated to both encoders 151 and 152 per unit time at that time (step S62). If “Yes” in step S71, the process proceeds to step S71. If the ratio HF1 / SF1 of the encoding processing speed is equal to the ratio HF2 / SF2 of the number of frames (“No” in step S62), the process proceeds to step S61. Return to.

  Next, in step S71, the CPU load detection unit 25 measures the usage rate of the CPU 10 at predetermined time intervals using the API of the OS 27. The CPU load detection unit 25 further identifies the stage to which the measured usage rate of the CPU 10 belongs, and determines whether or not it is the same as the previous stage. When the stage changes, the CPU load detection unit 25 notifies the allocation ratio setting unit 22F that the stage to which the usage rate of the CPU 10 belongs is changed along with the type of the stage after the change. The allocation ratio setting unit 22F searches the second database 26 for the ratio HF3 / SF3 of the number of frames corresponding to the notified use rate stage.

  Next, in step S72, the allocation ratio setting unit 22F compares the ratio HF3 / SF3 of the searched frame numbers with the ratio HF1 / SF1 of the encoding processing speed. If the ratio HF1 / SF1 of the encoding processing speed is larger than the ratio HF3 / SF3 of the searched frame numbers (“Yes” in step S72), the process proceeds to step S63. Thereby, the ratio HF1 / SF1 of the encoding processing speed is passed to the data allocator 14. On the other hand, when the ratio HF1 / SF1 of the encoding processing speed is equal to or less than the ratio HF3 / SF3 of the number of searched frames (in the case of “No” in step S72), the process proceeds to step S73. As a result, the ratio HF3 / SF3 of the number of retrieved frames is passed to the data allocator 14. Accordingly, in step S64, as in the sixth embodiment, the data allocator 14 allocates video data to both encoders 151 and 152 at the newly set ratio of the number of frames.

  In the encoding apparatus 100 according to the seventh embodiment, first, the processing speed measurement unit 28 measures the actual encoding processing speeds of the encoders 151 and 152, and the allocation ratio setting unit 22F allocates the encoders 151 and 152 to both. The ratio of the number of frames is set to the ratio HF1 / SF1 of the actual encoding processing speed between both encoders 151 and 152. Next, the encoding apparatus 100 measures the CPU 10 usage rate step in real time by the CPU load detection unit 25, and the allocation ratio setting unit 22F measures the frame number ratio HF3 corresponding to the measured CPU 10 usage rate step. / SF3 is compared with the ratio HF1 / SF1 of the encoding processing speed. When the ratio HF3 / SF3 of the number of frames corresponding to the stage of utilization of the CPU 10 is smaller than the ratio HF1 / SF1 of the encoding processing speed, the encoding apparatus 100 converts the video data to be encoded into the ratio HF1 of the encoding processing speed. / SF1 assigns to both encoders 151 and 152. On the other hand, when the ratio HF3 / SF3 of the number of frames corresponding to the use rate stage of the CPU 10 is equal to or higher than the ratio HF1 / SF1 of the encoding processing speed, the encoding apparatus 100 uses the video data to be encoded. The encoders 151 and 152 are allocated with the ratio HF3 / SF3 of the number of frames corresponding to the rate stage. As a result, the encoding apparatus 100 waits for the input of the frames encoded by the encoders 151 and 152 regardless of the variation in the encoding processing speed by the software encoder 152 due to the load variation of the CPU 10. The encoding processing by both encoders 151 and 152 can be performed in parallel while shortening the time or suppressing the occurrence of the time. As a result, the encoding apparatus 100 can perform the overall encoding process faster than the encoding process using only the hardware encoder 151. Thereby, the encoding processing speed of the video editing system 200 can be improved.

Embodiment 8
The encoding apparatus 100 according to the eighth embodiment of the present invention is incorporated in the non-linear video editing system 200 as in the first embodiment. The video editing system 200 is the same as that according to the first embodiment except for the encoding unit. Therefore, the description about Embodiment 1 is used for details of these similar elements.

  FIG. 20 is a block diagram of an encoding unit 11G according to Embodiment 8 of the present invention. Referring to FIG. 20, the encoding unit 11G includes a data allocating unit 14, a hardware encoder 151, a software encoder 152, a combining unit 16, an output unit 17, and a switching position detecting unit 29. The encoding unit 11G includes the same elements as those shown in FIG. 3 except for the switching position detection unit 29. In FIG. 20, the same reference numerals as those shown in FIG. 3 are given to the same elements, and the description of FIG. 3 is used for the details thereof.

  The switching position detector 29 functions when the CPU 10 controls other elements of the computer terminal according to a predetermined program. The switching position detection unit 29 detects a scene switching position or a change position of the type of editing processing from the video data stream to be encoded written from the editing unit 13 to the memory unit 20, and detects it according to the detection. Generate a signal.

  Prior to the editing process by the editing unit 13, the switching position detection unit 29 receives editing information from the editing unit 13. Based on the editing information, the switching position detection unit 29 sets the time code of the frame at which the scene switches from the video data stream edited by the editing unit 13 as a switching position, and decodes the video data to be edited and effects The time code of the frame where the content of the editing process such as the content of the process is switched is set as the change position.

  Each time one frame to be encoded is written from the editing unit 13 to the memory unit 20, the data allocating unit 14 reads the time code from the frame and passes it to the switching position detecting unit 29. The switching position detector 29 compares the time code with a time code indicating each switching position and each change position. When the time code from the data allocator 14 matches a time code indicating any switching position or change position (hereinafter abbreviated as a switch / change position), the switch position detector 29 sends a detection signal to the data allocator 14. Output to. The data allocator 14 switches the output destination of the frames subsequent to that frame to an encoder different from the encoder that outputs the immediately preceding frame in accordance with the detection signal.

  When a series of video data included between two successive switching / change positions is large, the switching position detecting unit 29 sets a boundary of any frame as a new switching position from the video data. May be. Thereby, the buffer area for video data allocated to the encoders 151 and 152 can be kept small. Furthermore, in order to increase the reliability of speeding up the encoding process by using both encoders 151 and 152 in parallel, the switching position detector 29 may adjust the ratio of video data allocated to both encoders 151 and 152.

  FIG. 21 is a flowchart of video data allocation processing by the encoding unit 11G. The switching position detector 29 and the data allocator 14 allocate the video data to be encoded to both encoders 151 and 152 at the switching / change position according to the flowchart of FIG. Hereinafter, the allocation process will be described with reference to FIG. The following processing can be started, for example, when the editing unit 13 starts outputting a video data stream to be encoded.

  First, in step S81, the switching position detection unit 29 receives editing information from the editing unit 13, and sets a time code indicating a switching / change position for the video data stream edited by the editing unit 13 based on the editing information. To do. Hereinafter, numbers sw = 0, 1, 2,... Are assigned to each time code Tc in the order of the reproduction time.

  Next, in step S82, the switching position detector 29 initializes the number sw to “0”.

  Next, in step S83, the data allocator 14 initializes the output destination of the video data to be encoded to the hardware encoder 151.

  Next, in step S84, each time the editing unit 13 writes one frame to be encoded in the memory unit 20, the data allocating unit 14 reads the time code Tc from the frame and passes it to the switching position detecting unit 29.

  Next, in step S85, the switching position detector 29 compares the time code Tc received from the data allocator 14 with the time code T (sw) of the sw-th switching / change position. If the time code Tc from the data allocator 14 is equal to or greater than the time code T (sw) at the sw-th switching / changing position (“Yes” in step S85), the process proceeds to step S86. If the time code Tc is smaller than the time code T (sw) at the sw-th change / change position (“No” in step S85), the process returns to step S84. In that case, the data allocator 14 allocates the frame of the time code Tc to the hardware encoder 151.

  Next, in step S86, the switching position detector 29 generates a detection signal. The data allocator 14 allocates the frames after the time code Tc to the software encoder 152 according to the detection signal.

  Next, in step S87, each time the editing unit 13 writes one frame to be encoded in the memory unit 20, the data allocating unit 14 reads the time code Tc from the frame and passes it to the switching position detecting unit 29.

  Next, in step S88, the switching position detector 29 compares the time code Tc received from the data allocator 14 with the time code T (sw + 1) of the sw + 1th switching / changing position. When the time code Tc from the data allocator 14 is equal to or greater than the time code T (sw + 1) at the sw + 1th switching / changing position (“Yes” in step S88), the process proceeds to step S89. If the time code Tc is smaller than the time code T (sw + 1) at the sw + 1-th switching / change position (“No” in step S88), the process returns to step S87. In that case, the data allocator 14 allocates the frame of the time code Tc to the software encoder 152.

  Next, in step S89, the switching position detector 29 generates a detection signal. The data allocator 14 allocates the frames after the time code Tc to the hardware encoder 151 according to the detection signal.

  Next, in step S90, the switching position detector 29 increments the number sw by “2”.

  Next, in step S91, the data allocator 14 determines whether the allocated frame is the last frame of the video data stream to be encoded. If the frame is not the last (“No” in step S91), the process is repeated from step S84, and if it is the last (“Yes” in step S91), the process ends.

  The encoding apparatus 100 according to the eighth embodiment sets the time code T (sw) indicating the switching / change position for the video data stream to be encoded based on the editing information by the switching position detection unit 29, and sets the encoding target frame. Each time the time code Tc reaches the time code T (sw) at the switching / change position, the output destination of the frame is alternately switched between the encoders 151 and 152. That is, the encoding apparatus 100 according to the eighth embodiment allocates the video data stream to be encoded to both encoders 151 and 152 at the switching / change position. Thereby, since the encoding apparatus 100 encodes a series of scenes or the entire video data on which the same editing process is continued with the same encoder, the image quality can be made substantially uniform. In addition, even if there is any change in the image quality due to the switching of the encoders 151 and 152, it is limited to the switching / change position, so it is difficult for the viewer to detect. As a result, the encoding device 100 performs the encoding processing by the entire encoding device 100 in parallel by performing the encoding processing by both the encoders 151 and 152 while keeping the image quality of the video data stream felt by the viewer uniform. This can be faster than the encoding process using only the wear encoder 151. Thereby, the encoding processing speed of the video editing system 200 can be improved.

  In addition to the above-described first to eighth embodiments, the encoding unit according to the embodiment of the present invention may be a combination of a plurality of the first to eighth embodiments. For example, Embodiment 5 and Embodiment 7 may be combined, or Embodiment 8 and any of Embodiments 2 to 7 may be combined.

  The encoding apparatus 100 according to the embodiment of the present invention may be implemented as a real-time encoder in a video capture or digital video camera in addition to the video editing system 200 described above. In that case, the encoding apparatus includes an input unit instead of the editing unit 13 in the configuration shown in FIG. The input unit takes in video data from a removable disk such as a DVD or a camera. The input unit may capture video data from resources on the network through the network interface. The input unit passes the captured video data to the data allocator 14. The CPU of the computer terminal may be mounted on the same board as the hardware encoder.

In addition, the following items are further disclosed regarding the above description.
(Item 1)
An encoding device that encodes AV data including audio data and / or video data,
CPU,
A hardware encoder that is configured by hardware for encoding processing and encodes a part of the AV data;
A software encoder that encodes another part of the AV data in parallel with the encoding process by the hardware encoder using the CPU;
A data allocator for allocating the AV data to the hardware encoder and the software encoder;
A synthesizing unit that arranges AV data encoded by each of the hardware encoder and the software encoder in a predetermined order and combines them into a series of encoded AV data;
An output unit for outputting the series of encoded AV data;
With
The encoding apparatus, which encodes AV data at an encoding processing speed larger than an encoding processing speed only by the hardware encoder.
(Item 2)
The encoding apparatus according to item 1, wherein encoding methods of the hardware encoder and the software encoder are common.
(Item 3)
The encoding apparatus according to item 1, wherein AV data encoded by each of the hardware encoder and the software encoder is not in a predictive relationship with each other.
(Item 4)
A buffer connected between the data allocator and the hardware encoder, storing AV data allocated to the hardware encoder by the data allocator, and supplying AV data to the hardware encoder; Prepared,
The data allocator obtains free information of the buffer and allocates the AV data to the software encoder based on the free information;
Item 4. The encoding device according to Item 1.
(Item 5)
Data representing a ratio of the number of data units per unit time between AV data allocated to the hardware encoder and AV data allocated to the software encoder is supplied to the data allocator.
The data allocator allocates AV data to the hardware encoder and the software encoder based on the data;
Item 4. The encoding device according to Item 1.
(Item 6)
The CPU also performs processing other than encoding processing on the AV data,
A CPU load estimator for predicting the CPU load caused by the other processing in correspondence with the AV data;
According to the predicted CPU load, a ratio of the number of data units per unit time between AV data allocated to the hardware encoder and AV data allocated to the software encoder is calculated, and the ratio is calculated. An allocation ratio setting unit for supplying data to the data allocating unit;
Further comprising
The data allocator allocates AV data to the hardware encoder and the software encoder based on the data;
Item 4. The encoding device according to Item 1.
(Item 7)
7. The encoding apparatus according to item 6, wherein the allocation ratio setting unit detects a load on the CPU and changes the ratio based on the detected load.
(Item 8)
The CPU load is detected, and the ratio of the number of data units per unit time between the AV data allocated to the hardware encoder and the AV data allocated to the software encoder is calculated according to the detected load. An allocation ratio setting unit that supplies data representing the ratio to the data allocation unit;
The data allocator allocates AV data to the hardware encoder and the software encoder based on the data;
Item 4. The encoding device according to Item 1.
(Item 9)
A processing speed measuring unit for measuring the encoding processing speed by the hardware encoder and the encoding processing speed by the software encoder;
According to the measured encoding processing speed, a ratio of the number of data units per unit time between AV data allocated to the hardware encoder and AV data allocated to the software encoder is calculated, and the ratio is calculated. An allocation ratio setting unit for supplying data to the data allocating unit;
Further comprising
The data allocator allocates AV data to the hardware encoder and the software encoder based on the data;
Item 4. The encoding device according to Item 1.
(Item 10)
The encoding device according to item 9, wherein the allocation ratio setting unit detects a load of the CPU and changes the ratio according to the detected load.
(Item 11)
A switching position detector for detecting a scene switching position included in the AV data or a change position of the type of editing processing and generating a detection signal in response to the detection;
The data allocator switches the output destination of AV data between the hardware encoder and the software encoder according to the detection signal.
Item 4. The encoding device according to Item 1.
(Item 12)
CPU,
An editing unit for editing AV data including audio data and / or video data;
A hardware encoder configured with a circuit for encoding processing and encoding a part of the edited AV data;
A software encoder that encodes another part of the edited AV data in parallel with the encoding process by the hardware encoder using the CPU;
A data allocator for allocating the edited AV data to the hardware encoder and the software encoder;
A synthesizing unit that arranges AV data encoded by each of the hardware encoder and the software encoder in a predetermined order and combines them into a series of encoded AV data;
An output unit for outputting the series of encoded AV data;
With
A video editing system for encoding AV data at an encoding processing speed larger than an encoding processing speed by only the hardware encoder.
(Item 13)
Item 13. The video editing system according to Item 12, wherein the encoding method by the hardware encoder and the software encoder is common.
(Item 14)
Item 13. The video editing system according to Item 12, wherein AV data encoded by each of the hardware encoder and the software encoder is not in a predictive relationship with each other.
(Item 15)
A buffer connected between the data allocator and the hardware encoder, storing AV data allocated to the hardware encoder by the data allocator, and supplying AV data to the hardware encoder; Prepared,
The data allocator obtains free information of the buffer and allocates the AV data to the software encoder based on the free information;
Item 13. The video editing system according to Item 12.
(Item 16)
Data representing a ratio of the number of data units per unit time between AV data allocated to the hardware encoder and AV data allocated to the software encoder is supplied to the data allocator.
13. The video editing system according to item 12, wherein the data allocator allocates AV data to the hardware encoder and the software encoder based on the data.
(Item 17)
The editing unit has editing information set by the user for the AV data to be edited,
A CPU load estimator for predicting a load on the CPU when the editing unit executes an editing process based on editing information;
According to the predicted CPU load, a ratio of the number of data units per unit time between AV data allocated to the hardware encoder and AV data allocated to the software encoder is calculated, and the ratio is calculated. An allocation ratio setting unit for supplying data to the data allocating unit;
Further comprising
The data allocator allocates the AV data to the hardware encoder and the software encoder based on the data;
Item 13. The video editing system according to Item 12.
(Item 18)
18. The video editing system according to item 17, wherein the allocation ratio setting unit detects a load on the CPU and changes the ratio based on the detected load.
(Item 19)
The CPU load is detected, and the ratio of the number of data units per unit time between the AV data allocated to the hardware encoder and the AV data allocated to the software encoder is calculated according to the detected load. , An allocation ratio setting unit that supplies data representing the ratio to the data allocation unit,
Further comprising
The data allocator allocates the AV data to the hardware encoder and the software encoder based on the data;
Item 13. The video editing system according to Item 12.
(Item 20)
A processing speed measuring unit for measuring the encoding processing speed by the hardware encoder and the encoding processing speed by the software encoder;
According to the measured encoding processing speed, a ratio of the number of data units per unit time between AV data allocated to the hardware encoder and AV data allocated to the software encoder is calculated, and the ratio is calculated. An allocation ratio setting unit for supplying data to the data allocating unit;
Further comprising
The data allocator allocates the AV data to the hardware encoder and the software encoder based on the data;
Item 13. The video editing system according to Item 12.
(Item 21)
21. The video editing system according to item 20, wherein the allocation ratio setting unit detects a load on the CPU and changes the ratio according to the detected load.
(Item 22)
The editing unit detects a scene switching position included in AV data obtained by executing editing processing based on editing information or a change position of the type of editing processing from the editing information, and generates a predetermined detection signal. A frame switching position detection unit for performing,
The data allocator switches the output destination of AV data between the hardware encoder and the software encoder according to the detection signal.
Item 13. The video editing system according to Item 12.
(Item 23)
A hardware encoder that encodes a part of AV data, and a software encoder that encodes another part of the AV data by using a CPU. A method for encoding video data using:
Receiving the AV data;
Allocating the AV data to the hardware encoder and the software encoder;
Encoding the AV data respectively allocated by the hardware encoder and the software encoder;
Arranging the AV data encoded by each of the hardware encoder and the software encoder in a predetermined order and synthesizing the AV data into a series of encoded AV data;
Outputting the series of encoded AV data;
Including
The method, wherein the AV data is encoded at an encoding processing speed larger than an encoding processing speed only by the hardware encoder.
(Item 24)
The step of allocating the AV data includes
Storing data allocated to the hardware encoder in a buffer;
Supplying AV data from the buffer to the hardware encoder;
Obtaining free information in the buffer;
Allocating the AV data to the software encoder based on free space information of the buffer;
24. The method according to item 23, comprising:
(Item 25)
The step of allocating the AV data includes
Receiving data representing a ratio of the number of data units per unit time between AV data allocated to the hardware encoder and AV data allocated to the software encoder;
Allocating AV data to the hardware encoder and the software encoder based on the data;
24. The method according to item 23, comprising:
(Item 26)
During the encoding step, the CPU also performs other processing associated with the AV data other than encoding by the software encoder,
Predicting the load of the CPU by the other processing in correspondence with the AV data;
According to the predicted CPU load, a ratio of the number of data units per unit time between AV data allocated to the hardware encoder and AV data allocated to the software encoder is calculated, and the ratio is calculated. Outputting data to represent;
Further including
24. The method according to item 23, wherein the step of allocating the AV data allocates AV data to the hardware encoder and the software encoder based on the data.
(Item 27)
27. The method according to item 26, wherein the step of allocating the AV data further includes a step of detecting a load on the CPU and changing the ratio based on the detected load.
(Item 28)
Editing the AV data into a series of AV data to be encoded based on editing information set by a user, the step being performed in parallel with the encoding step;
Predicting the CPU load when executing the editing process based on the editing information of the editing step in correspondence with a series of AV data to be encoded;
According to the predicted CPU load, a ratio of the number of data units per unit time between AV data allocated to the hardware encoder and AV data allocated to the software encoder is calculated, and the ratio is calculated. Outputting data to represent;
Further including
24. The method according to item 23, wherein the step of allocating the AV data allocates AV data to the hardware encoder and the software encoder based on the data.
(Item 29)
The CPU load is detected, and the ratio of the number of data units per unit time between the AV data allocated to the hardware encoder and the AV data allocated to the software encoder is calculated according to the detected load. Further comprising providing data representing the percentage,
24. The method according to item 23, wherein the step of allocating the AV data allocates AV data to the hardware encoder and the software encoder based on the data.
(Item 30)
Measuring the encoding processing speed by the hardware encoder and the encoding processing speed by the software encoder;
Data representing the ratio of the number of data units per unit time calculated between the AV data allocated to the hardware encoder and the AV data allocated to the software encoder according to the measured code processing speed Supplying a step;
Further comprising
24. The method of item 23, wherein allocating the AV data includes allocating AV data to the hardware encoder and the software encoder based on the data.
(Item 31)
The method according to item 30, wherein the step of allocating the AV data further includes a step of detecting a load of the CPU and changing the ratio according to the detected load.
(Item 32)
Detecting a scene switching position included in the AV data or a change position of the type of editing processing, and further generating a predetermined detection signal;
24. The method according to item 23, wherein the step of allocating the AV data includes a step of switching an output destination of AV data between the hardware encoder and the software encoder in accordance with the detection signal.
(Item 33)
Editing the AV data into a series of AV data to be encoded based on editing information set by a user;
Detecting a scene switching position included in the series of AV data to be encoded or a change position of the type of editing process, and generating a predetermined detection signal;
Further comprising
24. The method according to item 23, wherein the step of allocating the AV data includes a step of switching an output destination of AV data between the hardware encoder and the software encoder in accordance with the detection signal.
(Item 34)
A program for encoding AV data including audio data and / or video data,
CPU,
A hardware encoder configured with hardware dedicated to encoding processing and encoding a part of AV data;
A software encoder that encodes another part of the AV data using the CPU;
In a device with
Receiving the AV data;
Allocating the AV data to the hardware encoder and the software encoder;
Parallelizing the encoding process for the AV data by the hardware encoder and the software encoder;
Arranging AV data encoded by each of the hardware encoder and the software encoder in a predetermined order and synthesizing the AV data into one encoded AV data;
Outputting the encoded AV data;
A program for executing
The program for encoding AV data at an encoding processing speed larger than an encoding processing speed only by the hardware encoder.

It is a block diagram which shows the structure of the video editing system and encoding apparatus by Embodiment 1 of this invention. FIG. 2 is a schematic plan view of a board on which the hardware encoder shown in FIG. 1 is mounted. It is a block diagram which shows the software structure of the video editing system by embodiment of this invention. It is a figure which shows an example of the edit window displayed on the video editing system by Embodiment 1 of this invention. It is a block diagram of the encoding part by Embodiment 1 of this invention. 6 is a flowchart of video data allocation processing by the encoding unit shown in FIG. 5. It is a block diagram of the encoding part by Embodiment 2 of this invention. It is a block diagram of the encoding part by Embodiment 3 of this invention. FIG. 9 is an example of a CPU load index indicated by the first database shown in FIG. 8. FIG. It is a flowchart of the setting process of the ratio which allocates the video data by the encoding part shown by FIG. 9 is a flowchart of video data allocation processing by the encoding unit shown in FIG. 8. It is a block diagram of the encoding part by Embodiment 4 of this invention. 13 is a flowchart of video data allocation processing by the encoding unit shown in FIG. 12. It is a block diagram of the encoding part by Embodiment 5 of this invention. It is a flowchart of the allocation process of the video data by the encoding part shown by FIG. It is a block diagram of the encoding part by Embodiment 6 of this invention. FIG. 17 is a flowchart of video data allocation processing by the encoding unit shown in FIG. 16. FIG. It is a block diagram of the encoding part by Embodiment 7 of this invention. 19 is a flowchart of video data allocation processing by the encoding unit shown in FIG. 18. It is a block diagram of the encoding part by Embodiment 8 of this invention. FIG. 21 is a flowchart of video data allocation processing by the encoding unit illustrated in FIG. 20. FIG.

Explanation of symbols

10 CPU
11, 11A to 11G Encoding unit 13 Editing unit 14 Data allocation unit 151 Hardware encoder 152 Software encoder 16 Synthesis unit 17 Output units 22A to 22F Allocation ratio setting unit 23 CPU load estimation unit 25 CPU load detection unit 28 Processing speed Measuring unit 100 Encoding device 200 Video editing system

Claims (16)

An encoding device that encodes AV data including audio data and / or video data,
CPU,
A hardware encoder that is configured by hardware for encoding processing and encodes a part of the AV data;
A software encoder that encodes another part of the AV data in parallel with the encoding process by the hardware encoder using the CPU;
A data allocator for allocating the AV data to the hardware encoder and the software encoder;
A synthesizing unit that arranges AV data encoded by each of the hardware encoder and the software encoder in a predetermined order and combines them into a series of encoded AV data;
An output unit for outputting the series of encoded AV data;
With
The encoding apparatus, which encodes AV data at an encoding processing speed larger than an encoding processing speed only by the hardware encoder. A buffer connected between the data allocator and the hardware encoder, storing AV data allocated to the hardware encoder by the data allocator, and supplying AV data to the hardware encoder; Prepared,
The data allocator obtains free information of the buffer and allocates the AV data to the software encoder based on the free information;
The encoding device according to claim 1. Data representing a ratio of the number of data units per unit time between AV data allocated to the hardware encoder and AV data allocated to the software encoder is supplied to the data allocator.
The data allocator allocates AV data to the hardware encoder and the software encoder based on the data;
The encoding device according to claim 1. The CPU also performs processing other than encoding processing on the AV data,
A CPU load estimator for predicting the CPU load caused by the other processing in correspondence with the AV data;
According to the predicted CPU load, a ratio of the number of data units per unit time between AV data allocated to the hardware encoder and AV data allocated to the software encoder is calculated, and the ratio is calculated. An allocation ratio setting unit for supplying data to the data allocating unit;
Further comprising
The data allocator allocates AV data to the hardware encoder and the software encoder based on the data;
The encoding device according to claim 1.   The encoding apparatus according to claim 4, wherein the allocation ratio setting unit detects a load of the CPU and changes the ratio based on the detected load. The CPU load is detected, and the ratio of the number of data units per unit time between the AV data allocated to the hardware encoder and the AV data allocated to the software encoder is calculated according to the detected load. An allocation ratio setting unit that supplies data representing the ratio to the data allocation unit;
The data allocator allocates AV data to the hardware encoder and the software encoder based on the data;
The encoding device according to claim 1. A processing speed measuring unit for measuring the encoding processing speed by the hardware encoder and the encoding processing speed by the software encoder;
According to the measured encoding processing speed, a ratio of the number of data units per unit time between AV data allocated to the hardware encoder and AV data allocated to the software encoder is calculated, and the ratio is calculated. An allocation ratio setting unit for supplying data to the data allocating unit;
Further comprising
The data allocator allocates AV data to the hardware encoder and the software encoder based on the data;
The encoding device according to claim 1.   The encoding apparatus according to claim 7, wherein the allocation ratio setting unit detects a load on the CPU and changes the ratio according to the detected load. A switching position detector for detecting a scene switching position included in the AV data or a change position of the type of editing processing and generating a detection signal in response to the detection;
The data allocator switches the output destination of AV data between the hardware encoder and the software encoder according to the detection signal.
The encoding device according to claim 1. CPU,
An editing unit for editing AV data including audio data and / or video data;
A hardware encoder configured with a circuit for encoding processing and encoding a part of the edited AV data;
A software encoder that encodes another part of the edited AV data in parallel with the encoding process by the hardware encoder using the CPU;
A data allocator for allocating the edited AV data to the hardware encoder and the software encoder;
A synthesizing unit that arranges AV data encoded by each of the hardware encoder and the software encoder in a predetermined order and combines them into a series of encoded AV data;
An output unit for outputting the series of encoded AV data;
With
A video editing system for encoding AV data at an encoding processing speed larger than an encoding processing speed by only the hardware encoder. The editing unit has editing information set by the user for the AV data to be edited,
A CPU load estimator for predicting a load on the CPU when the editing unit executes an editing process based on editing information;
According to the predicted CPU load, a ratio of the number of data units per unit time between AV data allocated to the hardware encoder and AV data allocated to the software encoder is calculated, and the ratio is calculated. An allocation ratio setting unit for supplying data to the data allocating unit;
Further comprising
The data allocator allocates the AV data to the hardware encoder and the software encoder based on the data;
The video editing system according to claim 10.   The video editing system according to claim 10, wherein the allocation ratio setting unit detects a load on the CPU and changes the ratio according to the detected load. The editing unit detects a scene switching position included in AV data obtained by executing editing processing based on editing information or a change position of the type of editing processing from the editing information, and generates a predetermined detection signal. A frame switching position detection unit for performing,
The data allocator switches the output destination of AV data between the hardware encoder and the software encoder according to the detection signal.
The video editing system according to claim 10. A hardware encoder that encodes a part of AV data, and a software encoder that encodes another part of the AV data by using a CPU. A method for encoding video data using:
Receiving the AV data;
Allocating the AV data to the hardware encoder and the software encoder;
Encoding the AV data respectively allocated by the hardware encoder and the software encoder;
Arranging the AV data encoded by each of the hardware encoder and the software encoder in a predetermined order and synthesizing the AV data into a series of encoded AV data;
Outputting the series of encoded AV data;
Including
The method, wherein the AV data is encoded at an encoding processing speed larger than an encoding processing speed only by the hardware encoder. Editing the AV data into a series of AV data to be encoded based on editing information set by a user, the step being performed in parallel with the encoding step;
Predicting the CPU load when executing the editing process based on the editing information of the editing step in correspondence with a series of AV data to be encoded;
According to the predicted CPU load, a ratio of the number of data units per unit time between AV data allocated to the hardware encoder and AV data allocated to the software encoder is calculated, and the ratio is calculated. Outputting data to represent;
Further including
15. The method according to claim 14, wherein the step of allocating AV data allocates AV data to the hardware encoder and the software encoder based on the data. Editing the AV data into a series of AV data to be encoded based on editing information set by a user;
Detecting a scene switching position or a change position of the type of editing process included in the series of AV data to be encoded, and generating a predetermined detection signal;
Further comprising
15. The method according to claim 14, wherein allocating the AV data includes switching an output destination of AV data between the hardware encoder and the software encoder in accordance with the detection signal.

Images