JP2002199370A - Video / audio distribution apparatus and video / audio file analysis method - Google Patents

Abstract

(57) [Summary] [PROBLEMS] A video capable of shortening a response time from receiving a video / audio transmission request to transmitting a video / audio stream, and reducing a main memory usage of the video / audio distribution apparatus. To provide an audio distribution device. A video / audio file analysis unit analyzes a video / audio file stored in a video / audio file storage unit, and stores the result in a video / audio file analysis result storage unit. If there is not enough main memory to save,
Delete existing analysis results. Video / audio file analyzer 6
When the information on the video / audio file is requested from the video / audio data acquisition unit 5, if the requested information is in the video / audio file analysis result storage unit 7, it is returned. As a result, it is possible to shorten the response time from receiving the video / audio transmission request to transmitting the video / audio, and to reduce the amount of main memory used in the video / audio distribution device.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video and audio distribution apparatus for distributing recorded video and audio, and a video and audio file analysis method.

[0002]

2. Description of the Related Art In recent years, devices for distributing video and audio as digital signals in response to user requests have been developed.
For example, a video / audio receiving device (or software for causing a computer to function as a video / audio receiving device) capable of receiving and reproducing video / audio by connecting to the Internet has been developed, and based on a request transmitted from the receiving device. A device for distributing a video / audio stream has been developed.

In addition, with the development of low bit rate video / audio coding technology such as MPEG-4, video / audio distribution devices have the ability to process video / audio distribution required independently from a large number of receiving devices. is needed.

In a video / audio distribution apparatus, a video / audio stream to be distributed is encoded as a digital signal of an MPEG format or the like and stored in a recording medium such as a hard disk in advance. When the receiving device requests transmission of the video and audio, the video and audio stream is transmitted from the beginning of the video and audio or from the time specified by the receiving device.

[0005] As a method of transmitting video and audio data, there is a method in which video and audio are packetized and sequentially transmitted. For example, using a communication protocol such as RTP, the video / audio stream is divided into packets containing video / audio at fixed time intervals, and information such as reproduction time is added and transmitted. By not performing retransmission even if a packet is lost or data error occurs in transmission, it is possible to place importance on performing reproduction processing interactively and in real time.

Although this method has been used particularly for the distribution of video and audio such as live broadcasts not stored in a recording medium, it can also be applied to the distribution of a video and audio stream previously stored in a recording medium.

[0007]

In the prior art, when a video / audio distribution device receives a video / audio transmission request, it first analyzes a video / audio file and then transmits a video / audio stream. As a result, there is a problem that the time required for performing the analysis process is added to the time required from the transmission request of the video and audio to the transmission of the packet, and the response time to the receiving device is extended.

[0008] In a video / audio distribution apparatus which receives independent video / audio transmission requests from a plurality of receiving apparatuses, different transmission requests are generally received for the same video / audio file. Therefore, it is necessary to individually deal with transmission requests. In such a situation, if the analysis processing of the video / audio file is performed for each transmission request as described above, a load is imposed on the processing capability of the distribution device.

In order to packetize video and audio data, it is necessary to know which part of the video and audio file corresponds to the time at which the video and audio should be transmitted. It is desirable that such information be in a high-speed accessible main memory such as a semiconductor memory. However, the conventional method lacks a method for effectively storing such information in a main memory having a limited capacity. In particular, if such information is provided for each means for processing a transmission request, the same data exists in a plurality of main storage areas of the apparatus, and there is a problem that the use efficiency of the main storage is reduced.

[0010] The present invention has been made in view of the above circumstances, and can reduce the time from transmission request to response to video and audio, and can reduce the amount of main memory used in the video and audio distribution device. It is an object to provide an audio distribution device and a video / audio file analysis method.

[0011]

SUMMARY OF THE INVENTION According to the present invention, there is provided a video / audio transmission apparatus which generates a packet including a video / audio frame corresponding to a transmission request of a video / audio file received from an external request source, and transmits the packet to the request source. A distribution apparatus, comprising: a first stream for storing a stream including video and audio divided for each frame, and a video and audio file including meta information regarding each frame of the stream and the entire file;
Storage means, acquisition means for acquiring a video and audio frame corresponding to the transmission request from the first storage means when generating the packet, and the video and audio file stored in the first storage means Analysis means for analyzing the video / audio file as needed to respond to the acquisition means with analysis information on the frame and analysis information on the entire video / audio file, and the analysis created by the analysis means A second storage unit for storing all or a part of the information, wherein when the analysis information created by the analysis unit is stored in the second storage unit, analysis of the entire video / audio file is performed. The information is stored in its entirety, and the analysis information on the frame of the video / audio file is stored in at least a predetermined part of the frame. Is stored, and when the analysis information is requested by the acquisition means, if the corresponding analysis information is stored in the second storage means, the stored information is stored. And if it is not stored, it responds with analysis information created by analyzing the video / audio file stored in the first storage means.

This makes it possible to shorten the response time from receiving a video / audio transmission request to transmitting the video / audio. Further, by sharing the analysis result of the video / audio file with a plurality of transmission requests, it is possible to reduce the amount of main memory used in the video / audio distribution device.

Preferably, when the recording area for storing the analysis information on the frame of the video / audio file cannot be secured in the second storage means, the analysis means preferably selects a frame to be deleted and assigns it to the frame. Related analysis information may be deleted from the second storage unit.

As described above, when it is not possible to secure a recording area for storing information related to a frame, the main unit of the video / audio distribution apparatus for storing the analysis result of the video / audio file by selecting the frame and deleting the related information. It is possible to reduce the amount of storage used and use it efficiently.

Preferably, when the contents of the video / audio file stored in the first storage unit are changed, the analysis unit stores the video / audio file stored in the second storage unit. The analysis information related to the file may be deleted and updated immediately when the transmission request is received immediately or after the change.

In this way, when the video / audio file is changed, the analysis means updates the information on the video / audio file stored in the second storage means. It is possible to keep the contents of the stream consistent.

Preferably, when storing the analysis information on the frame of the video / audio file in the second storage means,
For a frame that can be reproduced without decoding other frames, the recording position of the frame in the video / audio file and the length of the frame according to the number of bits are stored, and for a frame that requires decoding of another frame, The recording position of the frame in the video / audio file may not be stored, but the length of the frame according to the number of bits may be stored.

As described above, the analyzing means converts the information on the frame recorded in the video / audio file into an expression format including the recording position and the size of the frame, and omits saving the recording position in the video / audio file. Accordingly, it is possible to reduce the amount of main storage used by the second storage unit.

The present invention also provides a first storage means for storing a stream composed of video and audio divided for each frame, and a video and audio file including meta information on each frame of the stream and the entire file. In response to a transmission request for a video / audio file received from an external request source, a corresponding video / audio frame is obtained from the first storage unit, a packet including the video / audio frame is generated, and transmitted to the request source. Processing means for transmitting, to the processing means, analysis information relating to frames of the video / audio file and analysis information relating to the entire video / audio file stored in the first storage means, as necessary. Analysis means for analyzing the audio file; and all or part of the analysis information created by the analysis means. A video / audio file analysis method in a video / audio distribution device comprising: a second storage unit for performing the analysis information when the analysis information is requested by the processing unit. If the information is stored in the second storage means, the stored information is returned. If the information is not stored, the video / audio file stored in the first storage means is analyzed and created. When the analysis information is created and the analysis information is created because the corresponding analysis information is not stored in the second storage unit, the analysis information relating to the entire video / audio file is included in the created analysis information. The information is stored in its entirety, and the analysis information on the frames of the video / audio file is stored in at least a part of predetermined frames. To.

It should be noted that the present invention relating to the apparatus is also realized as an invention relating to a method, and the present invention relating to a method is also realized as an invention relating to an apparatus. Further, the present invention according to an apparatus or a method has a function of causing a computer to execute a procedure corresponding to the present invention (or causing a computer to function as means corresponding to the present invention, or a computer having a function corresponding to the present invention). The present invention is also realized as a program (for realizing), and is also realized as a computer-readable recording medium on which the program is recorded.

According to the present invention, in response to a plurality of independent transmission requests for video and audio, analysis of the video and audio file is performed as required, and the analysis result is shared, thereby enabling the transmission of the video and audio transmission request. From the time of the response to this, and the amount of use of the main memory of the video and audio distribution apparatus can be suppressed.

[0022]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a configuration example of a video and audio distribution apparatus according to an embodiment of the present invention. As shown in FIG. 1, the video / audio distribution device 10 includes a video / audio transmission request input unit 1, one or more video / audio packet generation units 2, a video / audio packet output unit 3, a video / audio file storage unit 4, It has a video / audio data acquisition unit 5, a video / audio file analysis unit 6, and a video / audio file analysis result storage unit 7. In FIG. 1, the video / audio packet generation unit includes 2a, 2b
Are present (or two are activated).

In FIG. 1, reference numeral 20 denotes a video / audio receiving apparatus. In FIG. 1, the description of a network (for example, the Internet, a LAN, or the like) between the video / audio distribution device and the video / audio reception device is omitted. Although FIG. 1 shows only one video / audio receiving device, a plurality of video / audio receiving devices may exist (they may issue requests independently).

The video / audio transmission request input unit 1 receives a video / audio file transmission request from the video / audio receiving device 20. The video / audio transmission request input unit 1 includes communication hardware such as an Ethernet (registered trademark) adapter, hardware including components such as a processor and a memory, and software executed on the hardware. This software is compatible with RTSP (real time stream).
A request is processed in accordance with a communication protocol such as an A.A. protocol and the video / audio receiving apparatus 20 is responded to, and instructs the video / audio packet generation unit 2 to transmit a video / audio.

Of the hardware constituting the video / audio transmission request input unit 1, a device for executing software such as a processor and a memory can be shared with other components of the video / audio distribution device.

In accordance with the request received by the video / audio transmission request input unit 1, the video / audio packet generation unit 2 converts the video / audio encoded in the MPEG-4 format or the like into a video / audio data acquisition unit 5.
, And sequentially generates a video / audio packet, and outputs the generated video / audio packet to the video / audio packet output unit 3. As described above, the video and audio distribution device can include (or start up) a plurality of video and audio packet generation units. The video and audio data are in RTP (thermal-time trans).
It is packetized according to a communication protocol such as port protocol. The video and audio packet generation unit 2
It can be implemented as software, and each transmission request can be processed in a separate process or thread.

The video / audio packet output unit 3 converts the video / audio packet generated by the video / audio packet generation unit 2 into an RTP
According to a communication protocol such as the above, the video / audio file is transmitted to the receiving device 20 which has requested transmission. The video / audio packet output unit 3 includes communication hardware and software, similarly to the video / audio transmission request input unit 1. The communication hardware can be shared with the video / audio transmission request input unit 1.

The video / audio file storage unit 4 stores the MPEG-
A video and audio coded in four formats or the like are converted to a predetermined file format such as MP4, and are stored in a recording medium as a video and audio file. In response to a request from the video / audio data acquisition unit 5 or the video / audio file analysis unit 6, the contents of the specified portion of the video / audio file are returned. The video / audio file storage unit 4 includes a recording medium such as a hard disk or a DVD-ROM and a reading device. Further, a cache mechanism may be used in combination to improve the transfer speed and response time.

The video / audio data acquisition section 5 returns data of a video / audio file in response to a request from the video / audio packet generation section 2. The recording position of the stream in the video / audio file corresponding to the time zone of the video / audio is obtained from the video / audio file analysis unit 6, and the data of the video / audio file is obtained from the video / audio file storage unit 4 based on the obtained position. The video / audio data acquisition unit 5 is realized as hardware and software.

The video / audio file analysis unit 6 analyzes the content of the video / audio file stored in the video / audio file storage unit 4. A video / audio file conforming to a standard such as MP4 includes, as meta information of a video / audio stream, information on a video / audio frame and other information on the entire video / audio file. The video / audio file analysis unit 6 analyzes these and stores them in the video / audio file analysis result storage unit 7. The video / audio file analysis unit 6 is realized as hardware and software.

The video / audio file analysis result storage unit 7 stores a video / audio file analysis result using a recording medium that can be randomly accessed at high speed such as a semiconductor memory.

Hereinafter, a case where video and audio distribution is performed using the video and audio distribution apparatus of the present embodiment will be described.

In the following, M and M are used as the video / audio coding method.
PEG-4 is used, and the format of the video / audio file is M
Although the case of using the MP4 file format which is a part of the PEG-4 standard will be described as an example, the present invention is of course applicable to video / audio and video / audio files encoded by other methods. .

FIG. 2 and FIG. 3 show examples of the configuration of the analysis result of the video and audio file recorded in the video and audio file analysis result storage unit 7.

FIG. 2 shows a list of video / audio files stored in the video / audio distribution apparatus and whether or not the analysis result of each video / audio file is stored in the video / audio file analysis result storage unit 7. In addition, for a file that has been analyzed, it indicates whether the file can be correctly reproduced according to a predetermined format. In the example of FIG. 2, the video and audio files f1 and f2 have been analyzed as described below, f3 has not been analyzed, and as a result of the analysis, f4 has been found to be unplayable because the file is damaged. .

In addition to this method, the video / audio file analysis result storage unit 7 manages only the video / audio file name in which the analysis result of the video / audio file is stored. It can also be checked when transmission is requested by the device 20.

FIG. 3 shows an analysis result for the entire video / audio file. As the analysis result, data for analyzing the time length of the video / audio stream and the information of each frame is stored in the video / audio file analysis result storage unit 7. In addition to the above, other attributes of the video / audio file, a description about the video / audio file, and the like may be stored.

In order to analyze a frame, it is necessary to know in which part of the video / audio file information such as the reproduction time of the frame, the size, whether synchronization can be achieved, and the data recording position, etc. These values may be common to all frames depending on the video / audio file.

In the MP4 file, information on frames is stored collectively for each type of information. Therefore, the recording position of such information in the MP4 file is stored in the video / audio file analysis result storage unit 7. And frame information can be analyzed. Also, since the reproduction time is stored instead of the reproduction time of the frame, it is necessary to obtain the reproduction time from the reproduction time.

FIG. 4 is an example of information on frames of a video / audio file including a video stream and an audio stream of a variable bit rate.

In the video / audio file f1 shown in the upper part of FIG. 4A, a video stream, an audio stream, and meta information relating to both streams are sequentially recorded. FIG.
(A) In the upper video stream, for example, each frame is recorded as shown in the middle, and in the audio stream, for example, each frame is recorded as shown in the lower.

FIG. 4B shows information about each frame of the video / audio file f1 recorded in the video / audio file analysis result storage unit 7. In the information of FIG.
The item of “frame number” is 1 at the beginning of the video stream.
The number is a number assigned in the order of reproducing frames.
The item of “reproduction time” is the reproduction time of a frame with the start of the stream being 0 second, and means that the frame is displayed until the reproduction time of the next frame in the video stream. Display until the end of the entire playback time). In the audio stream, it indicates that audio is output from the reproduction time. "Sync"
Indicates whether the frame can be reproduced without reproducing another frame, and indicates that the frame is an Iframe in the MPEG-4 video stream.

The "offset" item and the "size" item respectively describe the "head" and "size" of the recording position where the data of the frame are stored in the video / audio file in byte units. For the sake of explanation, in the figure, instead of specific values, they are indicated by variable names consisting of offset (offset) or size (size), frame number, v (video) or a (audio). If there is a missing or abnormal frame information, it can be described that the frame cannot be reproduced instead of such information.

In the example shown in FIG.
Since the information regarding the third and subsequent frames and the 53rd and subsequent frames of the audio stream has not been analyzed, the items of “synchronization”, “offset”, and “size” are unknown.
When the playback time of a frame is stored in a video / audio file such as an MP4 file, the item of “playback time” is calculated from the sum of the playback time of the last frame analyzed and the playback time. Can be obtained. If the reproduction time is stored in the video / audio file instead of the reproduction time of the frame, the reproduction time of the frame that has not been analyzed cannot be obtained.

In the MP4 file format, a continuous sample of data (corresponding to a frame) is defined as a chunk, and the data recording position is recorded not for each sample but for each chunk. As for the stream stored in the continuous area as shown in FIG. 4A, the offset of the frame at the head of the chunk may be stored, but the offset of other frames may not be stored. The offset of the frame in which the offset is not stored can be sequentially obtained by the sum of the offset of the previous frame and the size of the previous frame. In addition,
In each figure, the offset value obtained by the calculation is shown in parentheses ().

In a video / audio file format in which offsets are recorded for all frames, the size of a frame can be obtained by subtracting the offset value of the frame whose size is to be obtained from the offset of the next frame.

As described above, since the offset can be obtained from the size, it is sufficient to store only one of the offset and the size as the analysis result of the video / audio file. Since the number of bits required to represent the size is generally smaller than the number of bits required to represent the offset, recording the size can reduce the main storage area required to store data, rather than recording the offset. .

When the frames are sequentially reproduced, the offset of the frame reproduced immediately before has been obtained.
By adding the size of the frame reproduced immediately before, the offset of the frame reproduced this time can be obtained. When frames are played back discontinuously by fast-forwarding or rewinding video and audio, the size of each frame from the frame whose playback time at which the offset is found is closest to the previous one to the frame whose offset is to be found may be added. .

When the number of frames in which the offset is stored in the video / audio file analysis result storage unit 7 is small, the number of frames to be added increases, and this process takes time. Especially when the frame information is not stored, it is necessary to acquire and analyze the video and audio files,
The processing time is added to the processing time for obtaining the offset.

Therefore, by storing a part of the offset obtained by the calculation in the video / audio file analysis result storage unit 7, the calculation of the offset can be speeded up. As a frame for storing the offset, for example, only a frame that can be synchronized can be selected. In each of the drawings, the offset stored in the video / audio file analysis result storage unit 7 among the offsets obtained by the calculation is shown in parentheses [].

As described above, it is possible not to store the frame offset in the video / audio file analysis result storage unit 7, and the main storage area of the video / audio file analysis result storage unit 7 for storing the frame information is reduced. can do.

FIG. 5 shows an example of information on each frame of a video / audio file including a video stream of a variable bit rate and an audio stream of a fixed bit rate.

In the video / audio file f2 shown in the upper part of FIG. 5A, video / audio streams and meta information relating to both streams are recorded in order. In the video / audio stream in the upper part of FIG. 5A, for example, as shown in the lower part, the video stream and the audio stream are respectively recorded as a group of streams at regular time intervals.

FIG. 5B shows information about each frame of the video / audio file f2 recorded in the video / audio file analysis result storage unit 7. FIG.
This is the same format as (b).

In the example of FIG. 5, the video stream 6
An abnormal value is set for the size of the 0th frame. As described above, when an abnormality is found for each frame during the analysis of the video / audio file, the reproduction cannot be performed. When the transmission of the subsequent frame and the reproduction by the video / audio receiving device 20 are possible, , Can continue.

In FIG. 5B, the audio stream is
The playback time and size of each frame are all the same, and synchronization can be achieved for all frames. for that reason,
The playback time can be obtained from the frame number using the following equation. Conversely, the frame number can be obtained from the reproduction time. Note that the first frame number is 1. Reproduction time = (frame number−first frame number) × reproduction time per frame That is, it is not necessary to store the reproduction time of each frame in the video / audio file analysis result storage unit 7.

When frame data is recorded discontinuously with the previous frame in the video / audio file,
You need to specify the frame offset. When it is continuous with the previous frame, the offset can be obtained from the offset and size of the previous frame, as in the example of FIG.

In the example shown in FIG. 5, since the 61st frame of the video stream and the 101st frame of the audio stream are not recorded at positions that are continuous with the previous frame, the offset of the frame recorded in the video / audio file is obtained. There is a need to.

Next, a processing procedure for transmitting a video / audio file in the video / audio distribution apparatus of the present embodiment will be described with reference to FIGS.

In step S 11, the video / audio transmission request input unit 1 receives a processing request for a video / audio file from the video / audio receiving device 20. For example, when a transmission request and a response are made based on RTSP, software for receiving the RTSP request is always executed.

When a new transmission request for a video / audio file is received, the process proceeds to step S12, where the video / audio transmission request input unit 1 activates the video / audio packet generation unit 2 and instructs preparation for video / audio transmission. The video / audio transmission request input unit 1 receives the transmission instruction for this transmission request from step S13, and at the same time, accepts the start of a new transmission request in step S11.

In step S13, if a transmission instruction for a video / audio file requested to be transmitted is received from the video / audio receiving device 20, the process proceeds to steps S14 and S1.
5, the process proceeds to S16, where processing according to the transmission instruction is performed. In the case of the "reproduction instruction", the process proceeds to step S18, instructs the video / audio packet generation unit 2 to start transmission, and returns to step S13. In the case of "pause instruction", step S
The program proceeds to step S19, instructing the video / audio packet generator 2 to temporarily stop transmission, and returns to step S13. In the case of another instruction, an appropriate process is performed according to the content of the instruction, and the process returns to step S13.

In the case of the "end instruction", the flow advances to step S17 to instruct the video / audio packet generator 2 to stop generating the video / audio packet. If the video / audio packet generation unit 2 is a process or a thread, the processing is terminated and resources are released, or the process is made to wait until another transmission request comes. Then, the video / audio transmission request input unit 1 ends receiving the transmission instruction regarding the video / audio file for which the transmission request has been received.

Next, the processing procedure of the video / audio packet generator 2 of the video / audio distribution apparatus of the present embodiment will be described with reference to FIGS.

First, in step S31, preparation is made for transmission of a video / audio file instructed to be transmitted. At this time, by analyzing the video / audio file in the video / audio file analysis unit 6, it is possible to reduce the response time from when a reproduction instruction is received later to when transmission is started.

If it is determined in step S32 that an operation instruction for a video / audio file has been received, the process proceeds to steps S41 and S41.
Proceeding to steps S42 and S43, a process corresponding to the transmission instruction is performed. When the reproduction is instructed, the reproduction is started in step S44, and the process proceeds to step S34. When the pause of reproduction is instructed, the reproduction is paused in step S45, and the process returns to step S32 to wait for the next instruction. When the end of reproduction is instructed, the transmission processing ends. In the case of another instruction, an appropriate process is performed, and the process returns to step S32.

If no operation instruction for the video / audio file is received in step S32, the process proceeds to step S33.
If the video / audio packet generation unit 2 is not in the state of being reproduced in step, the process returns to step S32 and waits for an operation instruction.

If no operation instruction for the video / audio file is received in step S32, the process proceeds to step S33.
In step S34, S35, S36, S37,
A series of processing of S38 is sequentially executed.

In this series of processing, the contents of the video / audio file are sequentially acquired for each video stream of a fixed time, and packetized for transmission.

Here, the first transmission of a video / audio file will be described by taking as an example a case where the first 0.1 second is packetized and transmitted.

First, in step S 34, the video / audio packet generator 2 sends a video / audio data acquisition unit 5
Instruct the acquisition of the contents for 0.1 second from the beginning of the video / audio file.

In step S35, the video / audio data acquisition unit 5 inquires of the video / audio file analysis unit 6 in which part of the video / audio file the data for the designated time is stored. Here, the video / audio file analysis unit 6 analyzes and returns a range in which data corresponding to 0.1 second from the beginning is stored for each of the video stream and the audio stream. The processing performed by the video / audio file analysis unit 6 will be described later.

In step S 36, the video / audio data acquiring unit 5 acquires the data of the video / audio file in the range specified by the video / audio file analyzing unit 6 from the video / audio file storage unit 4.

In step S 37, the video / audio packet generator 2 generates a packet based on the data based on a predetermined protocol such as the RTP protocol, and sends the video / audio packet output unit 3 to the video / audio receiver 20. Send. The video / audio packet output unit 3 transmits a packet to the video / audio receiving device 20.

Then, in step S38, the video / audio packet generator 2 updates the internal time, and returns to step S32. If the reproduction is continued, steps S34 and subsequent steps are repeatedly executed to generate and transmit a packet corresponding to the next time of the previously transmitted video and audio data.

Next, the processing procedure of the video / audio file analysis unit 6 and the video / audio file analysis result storage unit 7 of the video / audio distribution device of the present embodiment will be described with reference to FIGS.

Here, the video / audio file f1 is transmitted for 0.1 second from the beginning with no video / audio file analysis result stored in the video / audio file analysis result storage unit 7. An example will be described.

This video / audio file f1 is shown in FIGS.
The video stream has a variable bit rate of 30 frames per second, and the audio has a variable bit rate of 50 frames per second.

When the transmission request for the video / audio file f1 is received from the video / audio receiving device 20, the video / audio file analysis unit 6 receives the inquiry from the video / audio data acquisition unit 5 in step S35 as described above. .

In this case, first, in step S 51, the video / audio file analysis unit 6 stores the data analysis result of the video / audio file f 1 requested by the video / audio data acquisition unit 5 in the video / audio file analysis result storage unit 7. Check if it is saved.

If not stored (for example, if the video / audio file f1 has never been transmitted before, if it has been transmitted but the analysis result has already been deleted,
Etc.), the process proceeds to step S52, the video / audio file f1 is read from the video / audio file storage unit 4, the contents are analyzed, and the process proceeds to step S53. Information on the entire video / audio file is recorded in the video / audio file analysis result storage unit 7, for example, in the format shown in FIG. The analysis of the video / audio file f1 can be performed at the time when the reproduction of the transmission of the video / audio file f1 is requested, or can be performed before the transmission request is received.

If the data relating to the entire video / audio file is present in the video / audio file analysis result storage unit 7 in step S51, the process proceeds to step S55, in which all the frames included in the time zone requested by the video / audio data acquisition unit 4 Check for information. The video / audio file analysis unit 6 refers to the information in the format shown in FIGS. 3 and 4 recorded in the video / audio file analysis result storage unit 7.

If it is determined in step S55 that all or part of the information of the frame included in the requested time zone is not present, in step S53, the information of the frame in which the analysis result is not stored is obtained. And the video / audio file f1 is stored in the video / audio file storage unit 4.
And analyze it. If there is a main storage area for storing the analysis result in the video / audio file analysis result storage unit 7 in step S54, the analysis result is stored in the video / audio file analysis result storage unit 7 in step S57.
As shown in FIG. From this information, it is possible to know in which position of the video / audio file f1 the video data for 0.1 second from the beginning of the video / audio is stored.

If it is determined in step S51 that there is no analysis result of the data related to the video / audio file f1, and the analysis is performed in step S52, the process proceeds to step S53, and the above analysis and storage are performed.

The case where there is no main storage area for storing the analysis result in the video / audio file analysis result storage unit 7 in step S54 will be described later.

When it is determined in step S55 that there is all the information of the frame included in the requested time zone, or when the analysis and storage in steps S53 and S57 are completed (in this example, the video / audio file analysis result storage unit) 7
, It is assumed that no analysis result of the video / audio file has been stored yet, which corresponds to the latter). In step S56, the video / audio file analysis unit 6
The information is returned to the video / audio data acquisition unit 5 with reference to the data shown in FIG. In the case of this example, the video stream is size1 from offset1v byte of the video / audio file f1.
v + size2v + size3v bytes are obtained, and the audio stream is offset1a of the video / audio file f1.
From the byte, size1a + size2a + size3
It returns to the video / audio data acquisition unit 4 to acquire a + size4a + size5a bytes.

Subsequently, in this example, it is assumed that the video / audio stream of the video / audio file f1 is transmitted with respect to the content from 0.1 seconds after the beginning to 0.2 seconds after the head. In this case, the analysis of the entire video / audio file is performed at the time of transmission for 0.1 second from the last time, and the result is stored in the video / audio file analysis result storage unit 7, so the analysis in step S52 is performed. No need. However, there is no information on the frames included in the requested time zone. Therefore, steps S51, S55, S5
Step S57 is executed to perform analysis from 0.1 seconds to 0.2 seconds later, and the information shown in FIG. Further, since the offset information relating to the frame in this time zone is not recorded in the video / audio file, the acquisition position of the video / audio data is from the position next to the previously acquired data. That is, offset4v = offset1v + size1v +
size2v + size3v, offset6a = offset1a + size1a +
size2a + size3a + size4a + size
5a.

As a result, the video / audio data acquisition unit 4 outputs the video stream to the offset4 of the video / audio file f1.
From the v-th byte, size4v + size5v + size
6v bytes, the audio stream is offset
From size 6a byte size6a + size7a + siz
Returns to acquire e8a + size9a + size10a bytes.

Here, the frame information of the video / audio file f1 was sequentially analyzed as required. However, in the reproduction of video and audio, a frame subsequent to the currently transmitted frame is considered to have a higher probability of being reproduced next than other frames. Therefore, the analysis of the frame can be performed before the requested frame. by this,
The number of accesses to the video / audio file storage unit 4 can be reduced as compared with the sequential analysis of the video / audio file, and the load can be reduced.

Next, it is assumed that the video / audio receiving device 20 different from the current video / audio transmission destination issues a transmission request for the same video / audio file f1 as the currently transmitted video / audio file.

In step S51, since the result regarding the entire analysis video / audio file f1 is stored in the video / audio file analysis result storage unit 7, it is not necessary to analyze the video / audio file f1. That is, the time for analyzing the video / audio file f1 can be reduced, and the load for reading the video / audio file storage unit 4 from the video / audio file f1 can be reduced.

The analysis result of the frame information necessary for generating the packet is also stored in the video / audio file analysis result storage unit 7. Therefore, the recording range of the video and audio data can be obtained without analyzing the frame information of the video and audio file f1.

As described above, the response time to the transmission request of the video / audio file f1 can be shortened. Also,
As described above, since the analysis result of the video / audio file f1 is shared with another transmission request, it is possible to suppress the use amount of the main memory of the video / audio file analysis result storage unit 7.

When the video / audio file storage unit 4 has a data cache, the process of reading the video / audio stream from the recording medium can be reduced.

Next, consider the case where the video / audio receiving apparatus 20 instructs fast-forward of video / audio reproduction. For example, consider the case where reproduction is performed from a position 5.1 seconds ahead of the head.

The video / audio file analyzer 6 analyzes the meta information of the video / audio file f1, and searches for a frame number corresponding to the 5.1 second. In the MP4 file format, since the reproduction time is recorded instead of the reproduction time of the frame, the frame time may be obtained by adding the reproduction times of all the frames from the beginning.

As a result, as shown in FIG. 14, it is detected that the time designated by the video / audio receiving apparatus 20 in the video stream is the 154th frame. This frame cannot be played back alone, but requires playback of a frame (Iframe) that can be played back alone. The video / audio file analysis unit 6 detects that reproduction from the 151st frame is necessary. Therefore, the video / audio file analysis unit 6 returns information to acquire data of the 151st and subsequent frames. Similarly, as a result of the analysis, the specified time is the 256th frame as a result of the analysis, but since it cannot be reproduced alone, information is returned to acquire data from the preceding 251st frame. The video / audio data acquisition unit 4 returns the video / audio data acquired based on this information and the reproduction time thereof to the video / audio packet generation unit 2.

Next, in the state shown in FIG. 14, another video / audio file f2
Consider a case where a transmission request is received for.

This video / audio file f2 is shown in FIGS.
The video stream has a variable bit rate of 30 frames per second, and the audio has a fixed bit rate of 50 frames per second.

Here, as in the case where the video / audio file f1 is requested, the analysis is performed in step S52, and the analysis result is recorded in the video / audio file analysis result storage unit 7 as shown in FIG. 15 in step S57. You.

In this case, since the reproduction time of the frame of the audio stream is fixed, the frame number can be immediately obtained from the reproduction time. Further, since the frame size is fixed, when data is recorded in a continuous area, the frame offset can be immediately obtained from the start offset of the continuous area and the frame number.

In the following, when the analysis result of step S53 in FIG. 10 is stored in the video / audio file analysis result storage unit 7, in step S54, the main storage area for storing the analysis result in the video / audio file analysis result storage unit 7 The case where there is no description will be described.

In this case, in step S61 of FIG. 11, a frame from which information is to be deleted is selected from the video / audio file analysis result storage unit 7, and in step S62,
The information of the selected frame is deleted from the video / audio file analysis result storage unit 7. If the information on all the frames of the video / audio file is deleted in step S63, the information on the entire video / audio file is deleted in step S64. Later, step S in FIG.
At 57, the analysis result of step S53 is stored in the video / audio file analysis result storage unit 7.

Here, the case where the reproduction of the video / audio files f1 and f2 progresses and the main memory allocated to the video / audio file analysis result storage unit 7 for storing the analysis result of the frame information newly becomes insufficient. This will be explained in more detail for an example.

As described above, when the main memory is insufficient,
The recorded information must be deleted. As a policy for selecting information to be deleted, information on the most recently accessed frame can be selected. According to this policy, in the case of the specific examples shown so far, frame 1 of the video / audio file f1 is most preferable, and thereafter, frame 2 and frame 3 are arranged in this order. For example, one second of frame information can be selected, and information of frames 1 to 30 of the video stream and information of frames 1 to 50 of the audio stream can be deleted.

As another policy, when the offset of a frame is calculated and stored from the size of the frame and stored, a part of the frame for which the offset is obtained is preferentially left, and other frame information is deleted first. There are ways.

For example, when a frame that can be synchronized appears in the video stream every 0.5 seconds, the first frame at the beginning and the 16th frame at 0.5 second are left, and the other frames, that is, the 2nd through 15th frames are left. And delete the 17th to 30th frames. Similarly, for audio streams,
If the 1st and 16th frames left at the same time as the 1st and 16th frames left in the video stream can be synchronized, the information is retained if it can be synchronized, and if not, the information of other synchronizable frames is left. .

As a result, the analysis information of the frame relating to the video / audio file f1 is as shown in FIG.

As described above, the main storage area of the video / audio file analysis result storage unit 7 can be used efficiently.

When all the frame information is lost as a result of deleting the frame information of the video / audio file, the information of the entire video / audio file can be deleted. For example, transmission requests from all video and audio distribution devices for the video and audio file f1 are stopped, and the
It is assumed that as a result of continuing the transmission to No. 2, all the frame information relating to the video / audio file f1 has been deleted. In this case, the video / audio file f in the format shown in FIG.
1 can be deleted from the video / audio file analysis result storage unit 7.

When a video / audio file is updated, all analysis results of the video / audio file relating to the file may be deleted and analyzed again as needed (for example, when a transmission request is made for the first time thereafter). There is a method of performing an analysis and storing the result, or a method of immediately performing an analysis and storing the result). As a result, consistency of the contents of the video / audio file can be maintained between the video / audio file storage device 4, the video / audio file analysis result storage unit 7, and the video / audio packet generation unit 2.

As described above, according to the present embodiment, it is possible to shorten the response time from the transmission request of the video and audio to the start of the transmission, and to suppress the use of the main memory of the video and audio distribution device. It is.

As described above, each of the above functions is
It can be realized as software. In addition, the present embodiment can be implemented as a program for causing a computer to execute predetermined means (or for causing a computer to function as predetermined means, or for causing a computer to realize predetermined functions), The present invention can also be implemented as a computer-readable recording medium on which the program is recorded.

The configuration illustrated in the embodiment of the present invention is merely an example, and is not intended to exclude other configurations, and a part of the illustrated configuration is replaced with another one or illustrated. Another configuration obtained by omitting a part of the configuration, adding another function or element to the illustrated configuration, or combining them is also possible. Also, another configuration that is logically equivalent to the illustrated configuration,
Another configuration including a portion logically equivalent to the illustrated configuration, another configuration logically equivalent to a main part of the illustrated configuration, and the like are also possible. Further, another configuration that achieves the same or similar purpose as the illustrated configuration, another configuration that achieves the same or similar effect as the illustrated configuration, and the like are also possible. Various variations of the various components exemplified in the embodiments of the present invention can be implemented in appropriate combinations. Further, the embodiments of the present invention relate to an invention as an individual device, an invention about two or more related devices, an invention as a whole system, an invention about components inside an individual device, or a method corresponding thereto. It encompasses and includes inventions related to various aspects, stages, concepts or categories, such as inventions. Therefore, the present invention can be extracted from the contents disclosed in the embodiments of the present invention without being limited to the illustrated configuration.

The present invention is not limited to the above-described embodiments, but can be implemented with various modifications within the technical scope.

[0117]

According to the present invention, in response to a plurality of independent transmission requests for video and audio, the analysis of the video and audio files is performed as required, and the analysis results are shared. The response time to the request can be shortened, and the usage amount of the main memory of the video and audio distribution device can be suppressed.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration example of a video and audio distribution device according to an embodiment of the present invention.

FIG. 2 is an exemplary view showing an example of information recorded in a video / audio file analysis result storage unit according to the embodiment.

FIG. 3 is an exemplary view showing an example of information recorded in a video / audio file analysis result storage unit according to the embodiment;

FIG. 4 is an exemplary view showing an example of a structure of a video / audio file recorded in a video / audio file storage unit and an example of information recorded in a video / audio file analysis result storage unit according to the embodiment;

FIG. 5 is an exemplary view showing an example of a structure of a video / audio file recorded in a video / audio file storage unit and an example of information recorded in a video / audio file analysis result storage unit according to the embodiment;

FIG. 6 is an exemplary flowchart illustrating an example of processing of a video / audio transmission request input unit according to the embodiment.

FIG. 7 is an exemplary flowchart illustrating an example of processing of a video / audio transmission request input unit according to the embodiment.

FIG. 8 is a flowchart showing an example of a video / audio packet generation process according to the embodiment;

FIG. 9 is an exemplary flowchart illustrating an example of processing of generating a video / audio packet according to the embodiment.

FIG. 10 is an exemplary flowchart illustrating an example of processing of a video / audio file analysis unit according to the embodiment.

FIG. 11 is an exemplary flowchart illustrating an example of processing of a video / audio file analysis unit according to the embodiment.

FIG. 12 is a view showing an example of information recorded in a video / audio file analysis result storage unit according to the embodiment;

FIG. 13 is a view showing an example of information recorded in a video / audio file analysis result storage unit according to the embodiment.

FIG. 14 is an exemplary view showing an example of information recorded in a video / audio file analysis result storage unit according to the embodiment;

FIG. 15 is a view showing an example of information recorded in a video / audio file analysis result storage unit according to the embodiment;

FIG. 16 is an exemplary view showing an example of information recorded in a video / audio file analysis result storage unit according to the embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Video / audio transmission request input part 2, 2a, 2b ... Video / audio packet generation part 3: Video / audio packet output part 4 ... Video / audio file storage part 5 ... Video / audio data acquisition part 6 ... Video / audio file analysis part 7 ... Video Audio file analysis result storage unit 10 Video / audio distribution device 20 Video / audio receiving device

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Toru Imai 1 Toshiba-cho, Komukai Toshiba-cho, Saisaki-ku, Kawasaki-shi, Kanagawa F-term in the Toshiba R & D Center (reference) JA01 LA11 LA14 5C064 BA07 BC16 BC20 BC23 BC25 BD02 BD08 BD13

Claims (7)

[Claims] 1. A video / audio distribution apparatus which generates a packet including a relevant video / audio frame in response to a transmission request of a video / audio file received from an external request source, and transmits the packet to the request source. First storage means for storing a stream composed of video and audio divided into a plurality of frames and a video / audio file including meta information on each frame of the stream and the entire file; and, when the packet is generated, the transmission request Acquiring means for acquiring a video / audio frame corresponding to the above from the first storage means; and analyzing information relating to the frame of the video / audio file stored in the first storage means and analysis information relating to the entire video / audio file. Analysis means for analyzing the video / audio file as needed to respond to the acquisition means; A second storage unit for storing all or a part of the analysis information created by the analysis unit, wherein the analysis information created by the analysis unit is stored in the second storage unit.
In the case of saving in the storage means, the analysis information on the entire video / audio file is stored in its entirety, and the analysis information on the frames of the video / audio file is stored on at least a part of predetermined frames. The analysis means, when the analysis information is requested from the acquisition means, and the corresponding analysis information is stored in the second storage means, responds the stored information; If not, the video / audio distribution apparatus responds with analysis information created by analyzing the video / audio file stored in the first storage unit. 2. The method according to claim 1, wherein the analyzing unit selects a frame to be deleted and associates the frame with the frame if the recording area for storing the analysis information on the frame of the video / audio file cannot be secured in the second storing unit. 2. The analysis information to be deleted from the second storage means.
2. The video and audio distribution device according to claim 1. 3. The video / audio file stored in the second storage means when the content of the video / audio file stored in the first storage means is changed. 2. The video and audio distribution apparatus according to claim 1, wherein the analysis information related to the information is deleted and updated when a transmission request is received immediately or for the first time after the change. 4. When storing the analysis information relating to the frame of the video / audio file in the second storage means, among the frames of the video / audio file, a frame which can be reproduced without decoding other frames is used. The recording position of the frame in the video / audio file and the length in terms of the number of bits of the frame are stored, and the recording position of the frame in the video / audio file is not stored for a frame that requires decoding of another frame. The video / audio distribution apparatus according to claim 1, wherein a length according to the number of bits of the frame is stored. 5. A first storage means for storing a stream composed of video and audio divided for each frame and a video / audio file including meta information on each frame of the stream and the entire file, and an external request source In response to the transmission request for the video and audio file received from
Processing means for generating a packet including the video / audio frame and transmitting the packet to the request source; analysis information relating to the frame of the video / audio file stored in the first storage means; and analysis relating to the entire video / audio file Analyzing means for analyzing the video / audio file as necessary to respond information to the processing means; and a second means for storing all or a part of the analysis information created by the analyzing means. A video / audio file analysis method in a video / audio distribution device, comprising: a storage unit, wherein, when the analysis information is requested from the processing unit, the analysis information is stored in the second storage unit. If stored in the means, respond the stored information; if not, the video and audio stored in the first storage means Responding to the analysis information created by analyzing the file, if the analysis information is created because the corresponding analysis information is not stored in the second storage unit, Analysis information on the entire video / audio file should be saved in its entirety,
A video / audio file analysis method, wherein analysis information on frames of the video / audio file is stored for at least a part of predetermined frames. 6. A computer functioning as a video / audio distribution device for generating a packet including a relevant video / audio frame in response to a video / audio file transmission request received from an external request source and transmitting the packet to the request source. A computer-readable recording medium having recorded thereon a stream of video and audio divided for each frame, and a video and audio file including meta information about each frame of the stream and the entire file. 1 storage function; an acquisition function of obtaining a video / audio frame corresponding to the transmission request from the first storage function when generating the packet; and the video / audio stored in the first storage function. The analysis information on the frame of the file and the analysis information on the entire video / audio file are collected. An analysis function for analyzing the video / audio file as needed to respond to the function; a second storage function for storing all or a part of the analysis information created by the analysis function; And the analysis information created by the analysis function is stored in the second
In the case of storing in the storage function, the analysis information on the entire video / audio file stores all its contents, and the analysis information on the frames of the video / audio file is stored on at least a predetermined part of the frames. When the analysis function is requested by the acquisition function for the analysis information, if the corresponding analysis information is stored in the second storage function, the stored information is stored in the second storage function. Respond from the analysis function to the acquisition function, and if not stored, respond to the acquisition function from the analysis function with analysis information created by analyzing the video / audio file stored in the first storage function. Computer-readable recording medium on which a program for causing the computer to function is recorded. 7. A computer functioning as a video / audio distribution device for generating a packet including a relevant video / audio frame in response to a transmission request of a video / audio file received from an external request source and transmitting the packet to the request source. A first storage function for storing a stream composed of video and audio divided for each frame, and a video / audio file including meta information about each frame of the stream and the entire file; and An obtaining function for obtaining, from the first storage function, a video and audio frame corresponding to the transmission request when generating, analysis information regarding the frame of the video and audio file stored in the first storage function, and the video In order to respond to the acquisition function with the analysis information on the entire audio file, the An analysis function for analyzing an audio file; and a second storage function for storing all or a part of the analysis information created by the analysis function, and the analysis created by the analysis function. Information from the second
In the case of storing in the storage function, the analysis information on the entire video / audio file stores all its contents, and the analysis information on the frames of the video / audio file is stored on at least a predetermined part of the frames. When the analysis function is requested by the acquisition function for the analysis information, if the corresponding analysis information is stored in the second storage function, the stored information is stored in the second storage function. Respond from the analysis function to the acquisition function, and if not stored, respond to the acquisition function from the analysis function with analysis information created by analyzing the video / audio file stored in the first storage function. To make it work.