JP2014017741A - Content transmission device, content reproduction device, content distribution system, control method for content transmission device, control method for content reproduction device, control program, and recording medium - Google Patents

Abstract

An object of the present invention is to determine whether or not it is necessary to reproduce additional content according to the reproduction timing and realize appropriate content reproduction.
A server for transmitting original content of MPEG-DASH standard to a client transmits event information together with an event information adding unit for generating event information including additional content and a reproduction information identifier, and segment data. And a content transmission unit 22.
[Selection] Figure 1

Description

  The present invention relates to a content transmission apparatus that transmits content, a content reproduction apparatus that acquires and reproduces content, a content distribution system, a control method for a content transmission apparatus, a control method for a content reproduction apparatus, a control program, and a recording medium.

  With the spread of the Internet and high performance of computers, distribution of large-capacity contents such as moving images via the Internet is widely performed. For example, there is a service called VOD (Video On Demand) that provides content such as a moving image in response to a user request. In VOD, for example, data is transmitted and received between a server (content providing apparatus) and a client (content reproduction apparatus) using HTTP (HyperText Transfer Protocol).

  Here, various technologies have been developed for content distribution by HTTP. For example, the Motion Picture Experts Group (MPEG) is promoting international standardization using an adaptive streaming technique using HTTP as an MPEG-DASH (Dynamic Adaptive Streaming over HTTP) standard.

  In MPEG-DASH, content is time-divided into a plurality of segments and transmitted in segment units. Each segment is composed of one or a plurality of fragments. The content is composed of one or a plurality of periods, and one period includes one or a plurality of segments.

  In MPEG-DASH, a plurality of representations having different quality types (reproduction quality such as bit rate and image resolution and types such as data format) are prepared for one content. For example, a plurality of segment data encoded at different bit rates for each segment is prepared. Accordingly, a client that receives and reproduces content can execute adaptive streaming by changing the bit rate of the requested content (segment) in accordance with the reception status of the content.

  In MPEG-DASH, MPD (media presentation description) is associated with content, and the content is managed by MPD. MPD is content metadata and describes content management information being developed by DASH in XML format.

  In other words, MPD is information used by the client when acquiring / reproducing content. Specifically, the client acquires the MPD from the server before receiving the content distribution. The client sequentially acquires media segments based on the URL of each media segment described in the MPD, and plays back the content.

  A specific description example of MPD will be described with reference to FIG. FIG. 11 is a diagram illustrating a description example of MPD. As shown in FIG. 11, MPD 200 describes content type information 201 (attribute value of attribute “type”) indicating whether the content is live content or time-shifted content. In the illustrated example, the attribute value of the attribute “type” is “dynamic”, indicating that the content associated with the MPD 200 is live content. On the other hand, in the case of time shift content, “static” is described in the attribute value of the attribute “type”.

  Further, since the content associated with the MPD 200 is live content, the MPD 200 describes distribution start time information 202 (attribute value of the attribute “availabilityStartTime”) indicating the time at which the server starts streaming distribution of the content. The In the example shown in the figure, the attribute value of the attribute “availabilityStartTime” is “2012-05-31T19: 00Z”, which indicates that streaming distribution starts on May 31, 2012 at 19:00.

  Also, holding period information 203 (attribute value of attribute “timeShiftBufferDepth”) indicating the time from when the server starts distributing each media segment until it is discarded is described. In the example shown in the figure, the attribute value of the attribute “timeShiftBufferDepth” is “PT7D”, which indicates that it is discarded after 7 days from the start of distribution. In other words, this information 203 indicates a period during which the client can acquire content.

  In addition, acquisition destination information 204 indicating a content acquisition destination is described. In the illustrated example, the URL of the server is described as the acquisition destination information 204.

  In addition, period information 205 relating to each period dividing the content reproduction period is described. In the illustrated example, the period information 205 describes the period start time (the attribute value of the attribute “start”) based on the period ID and the content distribution start time.

  In addition, segment information 206 relating to each media segment belonging to the period is described. In the illustrated example, the segment information 206 describes a segment ID, a segment bit rate, and a URL from which the segment is acquired.

Japanese Patent Laying-Open No. 2005-348144 (released on December 15, 2005)

“ISO / IEC 23009-1”, [online], April 1, 2012, ISO / IEC, [search June 19, 2012], Internet <URL: http://standards.iso.org/ittf /PubliclyAvailableStandards/c057623_ISO_IEC_23009-1_2012.zip>

  In the conventional MPEG-DASH, the contents are changed by periodically updating the MPD. However, periodic updates could not cope with sudden content changes. For example, it is difficult to display an emergency alert or breaking news when distributing live content.

  However, although not officially defined, in MPEG-DASH, the server can add the additional content to the original segment and transmit it so that the additional content can be displayed superimposed on the original content on the client. .

  However, in the case of original content that assumes both real-time playback and time-shifted playback, suddenly added additional content such as emergency alerts is useful in real-time playback, but in time-shifted playback, The freshness of the information may be reduced and may become useless. Conventionally, since it has not been considered to determine whether or not additional content needs to be reproduced according to the reproduction timing, the additional content is reproduced while being superimposed on the original content even in time-shift reproduction.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to determine whether or not additional content needs to be reproduced according to the reproduction timing, and to realize appropriate content reproduction and content transmission apparatus and content A reproduction apparatus, a content distribution system, a content transmission apparatus control method, a content reproduction apparatus control method, a control program, and a recording medium.

  In order to solve the above problems, a content transmission apparatus according to the present invention is a content transmission apparatus that transmits original content composed of a plurality of segments to a content reproduction apparatus for each segment, As a response to a request for requesting transmission of a segment, a transmission unit that transmits a response including the segment to the content reproduction device, and an information generation unit that generates event information including additional content information and a reproduction information identifier, The additional content information is main data of additional content to be reproduced by being superimposed on the original content by the content reproduction device or information for acquiring the additional content, and the reproduction information identifier may be live reproduction. Even for time shift playback, An unrestricted playback instruction for causing the content playback device to play back additional content, or a limited playback instruction for causing the content playback device to play back additional content only in the case of live playback. The transmission means includes the segment and the information The event information generated by the generating means is transmitted.

  A content transmission apparatus control method according to the present invention is a content transmission apparatus control method for transmitting original content composed of a plurality of segments to a content reproduction apparatus for each segment in order to solve the above-described problem. As a response to the request for transmission of each segment from the content reproduction device, a transmission step for transmitting a response including the segment to the content reproduction device, and event information including additional content information and a reproduction information identifier are generated. The additional content information is main content data of the additional content to be reproduced by the content reproduction device superimposed on the original content or information for acquiring the additional content, and the reproduction information identifier Is live playback , An unlimited reproduction instruction for causing the content reproduction apparatus to reproduce additional content even in time-shifted reproduction, or a limited reproduction instruction for causing the content reproduction apparatus to reproduce additional content only in the case of live reproduction. In the step, the event information generated in the information generation step is transmitted together with the segment.

  According to the above configuration, the content transmission device transmits the main content data of the additional content to be reproduced by being superimposed on the original content or the information for acquiring the additional content, and the reproduction information identifier together with the segment of the original content. To do. The content reproduction apparatus can acquire the additional content based on the information for acquiring the additional content. Further, the content reproduction apparatus can determine whether or not the acquired additional content needs to be reproduced based on whether the reproduction of the original content is live reproduction or time shift reproduction based on the reproduction information identifier.

  Therefore, the content transmission apparatus can cause the content reproduction apparatus to determine whether or not the additional content needs to be reproduced according to the reproduction timing, and can effectively reproduce the content.

  In the content transmission apparatus according to the present invention, it is preferable that the information generation unit adds the event information to a header portion of the segment.

  In the content transmission apparatus according to the present invention, it is preferable that the information generation unit adds the event information to a header of the response.

  In the content transmission apparatus according to the present invention, the information generation means includes reproduction time information indicating a reproduction time of the additional content, and information for determining whether the reproduction is a live reproduction or a time shift reproduction. It is preferable to generate event information further including tolerance width information indicating an allowable range of a difference between the time indicated by the time information and the time when the content reproduction apparatus actually reproduces the additional content.

  According to the above configuration, in the content reproduction apparatus that has acquired the reproduction time information and the allowable width information, the difference between the time indicated by the reproduction time information and the time when the additional content is actually reproduced is indicated by the allowable width information. If it is within the allowable range, it is determined that the reproduction of the original content is live reproduction, and if the difference is outside the allowable range, it is determined that the reproduction is time-shifted reproduction. Therefore, the content transmission apparatus can cause the content reproduction apparatus to determine whether the reproduction of the original content is live reproduction or time shift reproduction.

  Also, in the content transmission apparatus according to the present invention, when the reproduction information identifier is an unlimited reproduction instruction, the information generation means generates event information further including expiration date information indicating an expiration date of the unlimited reproduction instruction. Is preferred.

  According to the above configuration, when the playback information identifier is an unlimited playback instruction, the content playback apparatus that has acquired the expiration date information is based on the additional content information if within the expiration date indicated by the expiration date information. The additional content is reproduced while being superimposed on the original content. On the other hand, if it is not within the expiration date indicated by the expiration date information, the event information is ignored and only the original content is reproduced. Therefore, the content transmission apparatus can cause the content reproduction apparatus to determine whether or not the additional content needs to be reproduced based on the expiration date.

  The content playback apparatus according to the present invention is a content playback apparatus that acquires and plays back original content composed of a plurality of segments from a content transmission apparatus for each segment in order to solve the above-described problem. Sends a request for transmitting each segment to the transmitting device, and obtains a response including the segment as a response to the request, and reproduces the original content based on the segment obtained by the obtaining unit Additional content which is main content data of the additional content to be reproduced with the segment being superimposed on the original content or information for acquiring the additional content. Event including information and playback information identifier In the case of an unlimited playback instruction indicating that additional content is to be played back regardless of whether the playback information identifier is live playback or time-shifted playback, the additional content is generated based on the additional content information. This is a limited playback instruction indicating that the additional content is played back only when the playback information identifier is live playback when the playback information identifier is live playback, and when the playback of the original content is live playback, based on the additional content information When the additional information is superimposed on the original content and the reproduction information identifier is the limited reproduction instruction and the reproduction of the original content is time-shift reproduction, the event information is ignored and only the original content is reproduced. It is characterized by doing.

  In addition, in order to solve the above-described problem, the content playback apparatus control method according to the present invention acquires the original content composed of a plurality of segments from the content transmission apparatus for each segment and plays back the content. In the acquisition step of transmitting a request requesting transmission of each segment to the content transmission device, and acquiring a response including the segment as a response to the request, and in the segment acquired in the acquisition step A reproduction step of reproducing the original content on the basis thereof, and in the reproduction step, in the acquisition step, body data of the additional content to be reproduced and superimposed on the original content or the additional content is acquired together with the segment. Information for When the event information including the additional content information and the playback information identifier is acquired, an unlimited playback instruction indicating that additional content is played back regardless of whether the playback information identifier is live playback or time shift playback Based on the additional content information, the additional content is reproduced while being superimposed on the original content, and is a limited reproduction instruction indicating that the additional content is reproduced only when the reproduction information identifier is live reproduction. In the case of playback, based on the additional content information, the additional content is played by being superimposed on the original content, the playback information identifier is the limited playback instruction, and the playback of the original content is time-shifted playback, It is characterized in that event information is ignored and only the original content is played back.

  According to the above configuration, the content playback apparatus refers to the event information, and when the playback information identifier is an unlimited playback instruction, plays back the additional content superimposed on the original content based on the additional content information. When the playback information identifier is a limited playback instruction and the playback of the original content is live playback, based on the additional content information, the additional content is superimposed on the original content for playback, and the playback information identifier is the limited playback instruction. When the reproduction of the original content is time shift reproduction, the event information is ignored and only the original content is reproduced. Therefore, the content reproduction apparatus can determine whether or not the additional content needs to be reproduced according to the reproduction timing, and has an effect of appropriately executing the content reproduction.

  In the content reproduction apparatus according to the present invention, it is preferable that the acquisition unit acquires the event information added to the header portion of the segment.

  In the content reproduction apparatus according to the present invention, it is preferable that the acquisition unit acquires the event information added to the response header.

  In the content playback apparatus according to the present invention, the playback means further includes, in addition to the event information acquired by the acquisition means, playback time information indicating a playback time of the additional content, and whether the playback is live playback or time shift playback. Information that is to be determined, and includes tolerance width information indicating an allowable range of a difference between the time indicated by the reproduction time information and the time when the content reproduction apparatus actually reproduces additional content; Based on whether or not the difference between the time when the additional content is actually played and the time indicated by the playback time information is within the allowable range indicated by the allowable width information, the time whether the playback of the original content is live playback or not It is preferable to determine whether it is shift reproduction.

  According to the above configuration, the content playback apparatus can determine whether the playback of the original content is live playback or time shift playback based on the playback time information and the tolerance range information.

  In the content reproduction apparatus according to the present invention, the reproduction unit may include the reproduction information when the event information acquired by the acquisition unit further includes expiration date information indicating an expiration date of the unlimited reproduction instruction. If the identifier is an unrestricted playback instruction, if the identifier is within the expiration date indicated by the expiration date information, the additional content is reproduced by being superimposed on the original content based on the additional content information, while the expiration date indicated by the expiration date information is indicated. If not, it is preferable to ignore the event information and reproduce only the original content.

  According to said structure, the content reproduction apparatus can determine the necessity of reproduction | regeneration of additional content based on the expiration date which the said expiration date information shows.

  The content distribution system according to the present invention preferably includes the content transmission device and the content reproduction device.

  According to said structure, a content delivery system has an effect similar to the said content transmission apparatus and the said content reproduction apparatus.

  The content transmission device and the content reproduction device may be realized by a computer. In this case, the content transmission device is operated by causing the computer to operate as each unit of the content transmission device and the content reproduction device. In addition, a control program for realizing the content reproduction apparatus on a computer and a computer-readable recording medium on which the control program is recorded also fall within the scope of the present invention.

  As described above, the content transmission apparatus according to the present invention includes, as a response to a request for requesting transmission of each segment from the content reproduction apparatus, a transmission unit that transmits a response including the segment to the content reproduction apparatus, Information generating means for generating event information including content information and a reproduction information identifier, wherein the additional content information is main content data of the additional content or the additional content to be reproduced by being superimposed on the original content by the content reproduction device The playback information identifier is only used in the case of live playback or time-shift playback, in the case of an unlimited playback instruction that causes the content playback device to play back additional content, or live playback. Additional content is added to the content playback device. The is intended to represent a restriction reproduction instruction for reproducing said transmitting means, together with the segment, and transmits the event information the information generating means has generated.

  Further, the content transmitting apparatus control method according to the present invention includes a transmission step of transmitting a response including the segment to the content reproducing apparatus as a response to the request for transmitting each segment from the content reproducing apparatus, Information generating step for generating event information including content information and a playback information identifier, wherein the additional content information is main content data of the additional content or the additional content to be played back by being superimposed on the original content by the content playback device The playback information identifier is only used in the case of live playback, time-shift playback, unlimited playback instructions that cause the content playback device to play additional content, or live playback. In the content playback device It is intended to represent a restriction reproduction instruction to reproduce the pressurized contents, in the transmission step, together with the segment, and transmits the event information generated in the information generating step.

  Therefore, the content transmission apparatus can cause the content reproduction apparatus to determine whether or not the additional content needs to be reproduced according to the reproduction timing, and can effectively reproduce the content.

  In addition, the content reproduction device according to the present invention transmits a request for requesting transmission of each segment to the content transmission device, and obtains a response including the segment as a response to the request, and the acquisition Reproduction means for reproducing the original content based on the segment acquired by the means, and the reproduction means includes body data of additional content to be reproduced by the acquisition means superimposed on the original content together with the segment, or When additional content information, which is information for acquiring the additional content, and event information including a playback information identifier are acquired, the additional content regardless of whether the playback information identifier is live playback or time shift playback In the case of an unlimited playback instruction indicating that the Based on the content information, the additional content is reproduced while being superimposed on the original content, and is a limited reproduction instruction indicating that the additional content is reproduced only when the reproduction information identifier is live reproduction. In some cases, based on the additional content information, the additional content is reproduced while being superimposed on the original content, and when the reproduction information identifier is the limited reproduction instruction and the reproduction of the original content is time-shift reproduction, the event information Is ignored and only the original content is played.

  The content playback apparatus control method according to the present invention includes: an acquisition step of transmitting a request for requesting transmission of each segment to the content transmission apparatus and acquiring a response including the segment as a response to the request; A reproduction step of reproducing the original content based on the segment acquired in the acquisition step. In the reproduction step, the additional content that is superimposed on the original content and reproduced together with the segment in the acquisition step If the main content data or additional content information that is information for acquiring the additional content and event information including the playback information identifier are acquired, even if the playback information identifier is live playback, time-shift playback is performed. Even with additional content In the case of an unrestricted playback instruction that indicates that the content is to be played, limited playback that indicates that the additional content is played back by being superimposed on the original content based on the additional content information, and that the additional content is played back only when the playback information identifier is live playback If the reproduction of the original content is live reproduction, the additional content is reproduced by being superimposed on the original content based on the additional content information, the reproduction information identifier is the limited reproduction instruction, When playback is time-shift playback, the event information is ignored and only the original content is played back.

  Therefore, the content reproduction apparatus can determine whether or not the additional content needs to be reproduced according to the reproduction timing, and has an effect of appropriately executing the content reproduction.

1, showing an embodiment of the present invention, is a block diagram illustrating an example of a configuration of a content distribution system. FIG. It is a figure which shows an example of the principal part structure of the server and client which are contained in the said content delivery system. It is a flowchart which shows an example of the content transmission process of the said server. It is a flowchart which shows an example of the content acquisition reproduction process of the said client. It is a figure which shows the data structure of the conventional segment data. It is a figure which shows the data structure of the segment data to which event information was added. It is a figure which shows the example of a description of the event information added to the segment. It is a figure which shows the reproduction timing of segment # s1- # s3 of the client A which performs live reproduction, and the client B which performs time shift reproduction. It is a figure which shows an example of the operation | movement sequence in the case of describing event information in the HTTP header of the HTTP response message which transmits a segment data main body, and reproducing additional content on an original content based on the event information. It is a figure which shows an example of the HTTP message in the case of describing event information in the HTTP header of the HTTP response message which transmits a segment data main body, and reproducing additional content on an original content based on the event information. It is a figure which shows the example of description of MPD.

  An embodiment of the present invention will be described below with reference to FIGS. First, an outline of the content distribution system of the present embodiment will be described based on FIG.

[Outline of content distribution system]
FIG. 2 is a diagram showing an outline of the content distribution system 6 according to the present embodiment. As shown in FIG. 2, the content distribution system 6 includes a server 1, a client 2, an MPD storage device 4, and a segment storage device 5.

  As shown in FIG. 2, the client 2 connects to the server 1. The server 1 is connected to the MPD storage device 4 and the segment storage device 5. Each device is connected by an arbitrary network of wired communication or wireless communication.

  The server 1 is a content transmission device that receives a content transmission request from the client 2 and transmits the content. The server 1 transmits MPD data to the client 2 in advance before transmitting content main data (segment data). The server 1 acquires the MPD data and the segment data from the MPD storage device 4 and the segment storage device 5 on the network 7, but is not limited to this. For example, each server 1 may hold MPD data and segment data locally.

  The client 2 is a content reproduction device that reproduces content acquired from another device such as the server 1 or content stored in the own device. The client 2 is, for example, a digital television, a recorder, an STB (Set Top Box), a PC, a mobile phone, a smartphone, a game machine, a PDA (Personal Digital Assistant), a digital camera, a digital video, or the like.

  The configuration of the content distribution system 6 is not limited to the example shown in FIG. For example, the content distribution system 6 may include a plurality of servers 1 or a plurality of clients 2. The content distribution system 6 may include one or more proxies 3 that relay data between the server 1 and the client 2.

  In the present embodiment, the transmission protocol on the network in the content distribution system 6 is HTTP, which is widely used as a hypertext transfer protocol. The content distributed by the server 1 is video content, and the content is segmented MP4 data. That is, in the present embodiment, the content distribution system 6 distributes content based on the above-described MPEG-DASH standard.

[Configuration of each device]
Next, the main configuration of the server 1 and the client 2 will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of a main configuration of the server 1 and the client 2.

(About the server)
As shown in FIG. 1, the server 1 is configured to include a server control unit 11, a server storage unit 12, and a server communication unit 13.

  The server communication unit 13 communicates with other devices such as the client 2, the MPD storage device 4, and the segment storage device 5 by wireless communication means or wired communication means, and exchanges data according to instructions from the server control unit 11. Is.

  The server control unit 11 performs various operations by executing a program read from the server storage unit 12 to a temporary storage unit (not shown), and comprehensively controls each unit included in the server 1. is there.

  In this embodiment, the server control unit 11 is configured to include a content acquisition unit 21, a content transmission unit (transmission unit) 22, and an event information addition unit (information generation unit) 23 as functional blocks. Each functional block (21 to 23) of the server control unit 11 includes a CPU (central processing unit) that stores a program stored in a storage device realized by a ROM (read only memory) or the like, such as a RAM (random access memory) or the like. This can be realized by reading out and executing the temporary storage unit realized in the above.

  The content acquisition unit 21 acquires MPD data from the MPD storage device 4 or segment data from the segment storage device 5 based on an instruction from the content transmission unit 22. When acquiring the MPD data, the content acquisition unit 21 outputs the acquired MPD data to the content transmission unit 22, and when acquiring the segment data, the content acquisition unit 21 outputs the acquired segment data to the event information addition unit 23.

  Note that the content acquisition unit 21 may acquire MPD data and / or segment data in advance regardless of whether there is an instruction from the content transmission unit 22. In this case, the content acquisition unit 21 stores the MPD data and segment data acquired in advance in the server storage unit 12, and MPD data and segment data from the server storage unit 12 based on an instruction from the content transmission unit 22. Is read.

  When the content transmission unit 22 receives a request from the client 2, the content transmission unit 22 transmits a response to the request to the client 2. Specifically, when a request for requesting transmission of content management information (MPD) is received from the client 2, the content acquisition unit 21 is instructed to acquire the MPD of the content, and the MPD data is acquired from the content acquisition unit 21. Then, a response including the acquired MPD data is transmitted to the client 2. Further, when a request for transmission of a segment constituting the content is received from the client 2, the content acquisition unit 21 is instructed to acquire the segment, and when the segment data is acquired from the event information addition unit 23, the segment data Is sent to the client 2.

  When an instruction to insert additional content is input from the user via an operation unit (not shown), the event information adding unit 23 performs time shift based on the instruction even if the main content data of the additional content and live playback are performed. A reproduction information identifier indicating an unrestricted reproduction instruction for instructing reproduction of additional content even for reproduction or a limited reproduction instruction for instructing reproduction of additional content only in the case of live reproduction is generated. The event information adding unit 23 adds event information including main content data and a reproduction information identifier of the additional content to the segment data acquired from the content acquisition unit 21 and outputs the segment data to the content transmission unit 22.

  Further, the event information adding unit 23 may generate additional content position information indicating the position where the main data of the additional content exists, instead of generating the main data of the additional content based on the insertion instruction. For example, the event information adding unit 23 generates the URL of the device that holds the additional content as the additional content position information.

  Further, the event information adding unit 23 generates a position where MPD data including additional content position information indicating a position where the main body data of the additional content exists instead of generating the main body data of the additional content based on the insertion instruction. The updated MPD data position information shown may be generated.

  Hereinafter, the additional content position information and the updated MPD data position information are collectively referred to as additional content acquisition destination information. The additional content acquisition destination information is information for acquiring the main data of the additional content. Furthermore, the main data of the additional content and the additional content acquisition destination information are collectively referred to as additional content information. That is, the event information includes additional content information and a reproduction information identifier.

  Further, the event information addition unit 23 may further generate event information by adding reproduction time information and allowable width information. The reproduction time information is information indicating the reproduction (start) time of the additional content instructed (planned) by the server 1. The time indicated by the reproduction time information may be, for example, an NTP time stamp. The allowable width information is information for determining whether live reproduction or time-shifted reproduction, and the allowable range of the difference between the time indicated by the reproduction time information and the time when the client 2 actually reproduces the additional content. It is information to show.

  Further, when the playback information identifier is an unlimited playback instruction, the event information adding unit 23 may generate event information by adding expiration date information indicating the expiration date of the unlimited playback instruction.

  Although details will be described later, when adding event information to segment data, the event information adding unit 23 may add it to the segment data body in a box format defined by ISO / IEC 14496-12 (ISOBFF) ( For example, event information may be added (inserted) to the header portion of the segment data), and event information may be added to the HTTP header of the HTTP response message that transmits the segment data body.

  Note that the additional content generated by the event information adding unit 23 may be any content such as text, image, sound, and moving image. Specifically, the additional content is an emergency warning such as a weather warning, an emergency earthquake warning, a tsunami warning, an incident / accident warning, an election bulletin, a news bulletin such as a sports bulletin, an advertisement, a bonus video, or the like.

  When there is no instruction to insert additional content, the event information addition unit 23 outputs the segment data acquired from the content acquisition unit 21 to the content transmission unit 22 as it is.

  The server storage unit 12 stores programs, data, and the like referred to by the server control unit 11, and may store MPD data, segment data, and the like acquired by the content acquisition unit 21, for example.

  In the present invention, “live playback” means that the client 2 plays the original content almost simultaneously with the time when the server 1 delivers the original content. Actually, processing delays by the server 1 and the client 2, delays on the network, and the like occur, so that it is impossible for the client 2 to reproduce strictly in real time. In the present invention, setting an allowable range for this delay and allowing the client 2 to perform substantially real-time playback within the allowable range is referred to as “live playback”. On the other hand, reproduction by the client 2 outside the allowable range is referred to as “time shift reproduction”.

(About the client)
As shown in FIG. 1, the client 2 includes a client control unit 41, a client storage unit 42, a client communication unit 43, a display unit 44, and an audio output unit 45. The client 2 may include members such as an operation unit and a voice input unit, but the members are not shown because they are not related to the feature points of the invention.

  The client communication unit 43 communicates with other devices such as the server 1 by wireless communication means or wired communication means, and exchanges data according to instructions from the client control unit 41.

  The display unit 44 displays an image according to an instruction from the client control unit 41. The display unit 44 only needs to display an image in accordance with an instruction from the client control unit 41. For example, an LCD (liquid crystal display), an organic EL display, a plasma display, or the like can be applied.

  The audio output unit 45 receives an electrical signal from the client control unit 41, converts the received electrical signal into sound, and outputs the sound to the outside of the client 2. The audio output unit 45 is a so-called speaker.

  The client control unit 41 performs various operations by executing a program read from the client storage unit 42 to a temporary storage unit (not shown), and comprehensively controls each unit included in the client 2. is there.

  In the present embodiment, the client control unit 41 includes a content acquisition unit (acquisition unit) 51, a content analysis unit 52, and a content reproduction unit (reproduction unit) 53 as functional blocks. In each functional block (51 to 53) of the client control unit 41, the CPU reads a program stored in a storage device realized by a ROM or the like to a temporary storage unit realized by a RAM or the like and executes the program. Can be realized.

  The content acquisition unit 51 transmits a request to the server 1 via the client communication unit 43 and acquires content (an MPD associated with the content and a segment constituting the content) from the server 1.

  Specifically, when a content acquisition (playback) instruction is input from the user via an operation unit (not shown), the content acquisition unit 51 issues a request for requesting transmission of management information (MPD) of the content. Send to server 1. Then, the content acquisition unit 51 receives the MPD data of the content as a response to the request. The content acquisition unit 51 refers to the received MPD data, and transmits a request to the server 1 for requesting transmission of a segment constituting the content. The content acquisition unit 51 acquires the segment data of the content as a response to the request. The content acquisition unit 51 outputs the acquired segment data to the content analysis unit 52.

  When the content analysis unit 52 acquires segment data from the content acquisition unit 51, the content analysis unit 52 analyzes the acquired segment data, and extracts event information added to the segment data when event information is added to the segment data. is there. The content analysis unit 52 outputs the segment data and the extracted event information to the content reproduction unit 53. When event information is not added, the content analysis unit 52 outputs the segment data acquired from the content acquisition unit 51 to the content reproduction unit 53 as it is.

  In addition, when the additional content information included in the event information is the additional content position information or the updated MPD data position information, the content analysis unit 52 adds the main body data of the additional content based on the additional content position information or the updated MPD data position information. To get.

  For example, when the update MPD data position information is included in the event information, the content analysis unit 52 instructs the content acquisition unit 51 to acquire the update MPD data from the position indicated by the update MPD data position information. . When the content acquisition unit 51 acquires the updated MPD data, the content analysis unit 52 acquires the main data of the additional content from the position indicated by the additional content position information described in the update MPD data. To instruct.

  That is, the additional content information included in the event information output from the content analysis unit 52 to the content reproduction unit 53 is main data of the additional content.

  The content analysis unit 52 may store segment data and event information in the client storage unit 42. In this case, the content analysis unit 52 reads the segment data and event information from the client storage unit 42 based on an instruction from the content reproduction unit 53 and outputs the segment data and event information to the content reproduction unit 53.

  When the content reproduction unit 53 acquires segment data from the content analysis unit 52, the content reproduction unit 53 refers to the MPD data and reproduces the content based on the acquired segment data.

  Further, when the content playback unit 53 acquires event information together with the segment data from the content acquisition unit 51, the content playback unit 53 checks whether the playback information identifier included in the event information indicates an unlimited playback instruction or a limited playback instruction. When the reproduction information identifier indicates an unlimited reproduction instruction, the content reproduction unit 53 reproduces the original content based on the segment data and superimposes the additional content on the original content based on the main data of the additional content included in the event information. And play it.

  On the other hand, when the playback information identifier indicates a limited playback instruction, the content playback unit 53 determines whether playback of the original content is live playback or time shift playback. In the case of live playback, the content playback unit 53 plays back the original content based on the segment data, and plays back the additional content superimposed on the original content based on the main data of the additional content included in the event information. On the other hand, in the case of time shift reproduction, the content reproduction unit 53 ignores the event information and reproduces the original content based only on the segment data.

  Here, when the reproduction time information and the allowable width information are included in the event information, the content reproduction unit 53 may determine whether to perform live reproduction or time shift reproduction based on the reproduction time information and the allowable width information. . Specifically, the content reproduction unit 53 determines whether or not the difference between the time indicated by the reproduction time information and the time when the additional content is actually reproduced is within the range indicated by the allowable width information. The content reproduction unit 53 determines that the reproduction is live reproduction when the difference is within the range indicated by the allowable width information, and determines that it is time shift reproduction when the difference is not within the range indicated by the allowable width information. .

  Further, when the client 2 is a live playback terminal, the content playback unit 53 may always determine that the playback of the original content is live playback. Further, when the client 2 is a time shift playback terminal, the content playback unit 53 may always determine that playback of the original content is time shift playback. That is, when the reproduction method of the client 2 is determined in advance, the content reproduction unit 53 may determine whether the reproduction is live reproduction or time shift reproduction based on the setting.

  Further, when the reproduction information identifier indicates an unlimited reproduction instruction and the expiration date information is included in the event information, the content reproduction unit 53 reproduces the additional content within the expiration date indicated by the expiration date information. Only in this case, the additional content may be reproduced by being superimposed on the original content. In this case, when the time for reproducing the additional content is not within the expiration date indicated by the expiration date information, the content reproduction unit 53 ignores the event information and reproduces the original content based only on the segment data.

  The client storage unit 42 stores programs, data, and the like referred to by the client control unit 41. For example, MPD data and segment data acquired by the content acquisition unit 51, and event information extracted by the content analysis unit 52 are stored. Etc. may be stored.

[Server processing]
Next, content transmission processing of the server 1 will be described with reference to FIG. FIG. 3 is a flowchart illustrating an example of content transmission processing of the server 1.

  As shown in FIG. 3, the server 1 waits for a request to be transmitted from the client 2 (S1, S4). Here, when the content transmission unit 22 receives a request for requesting transmission of content management information (MPD) from the client 2 (YES in S1), the content transmission unit 22 sends the received request to the content acquisition unit 21. The MPD associated with the content indicated by is instructed to be acquired.

  The content acquisition unit 21 that has received an instruction from the content transmission unit 22 acquires MPD data from the MPD storage device 4 via the server communication unit 13 (S2). The content acquisition unit 21 outputs the acquired MPD data to the content transmission unit 22. Then, the content transmission unit 22 transmits a response including the MPD data acquired from the content acquisition unit 21 to the client 2 (S3).

  Further, when the content transmission unit 22 receives a request for transmitting a segment from the client 2 (YES in S4), the content transmission unit 22 acquires the segment indicated by the received request from the content acquisition unit 21. To instruct.

  The content acquisition unit 21 receiving the instruction from the content transmission unit 22 acquires segment data from the segment storage device 5 via the server communication unit 13 (S5). The content acquisition unit 21 outputs the acquired segment data to the event information addition unit 23.

  When the event information adding unit 23 acquires the segment data from the content acquiring unit 21, the event information adding unit 23 determines whether or not there is an instruction to insert additional content from the user (S6). When there is an instruction to insert additional content (YES in S6), the event information adding unit 23 adds event information to the acquired segment data and outputs the event information to the content transmitting unit 22, and the content transmitting unit 22 The response including the segment data to which the event information is added, acquired from 23, is transmitted to the client 2 (S7). On the other hand, when there is no instruction to insert additional content (NO in S6), the event information adding unit 23 outputs the acquired segment data as it is to the content transmitting unit 22, and the content transmitting unit 22 acquires from the event information adding unit 23. A response including the segment data thus transmitted is transmitted to the client 2 (S8).

[Client processing]
Next, content acquisition / playback processing of the client 2 will be described with reference to FIG. FIG. 4 is a flowchart illustrating an example of content acquisition / playback processing of the client 2.

  As shown in FIG. 4, first, the content acquisition unit 51 transmits a request for requesting transmission of content management information (MPD) to the server 1 (S11). And the content acquisition part 51 receives the response with respect to this request, and acquires MPD data contained in a response (S12).

  Next, the content acquisition unit 51 refers to the received MPD data and transmits a request for requesting transmission of a segment constituting the content to the server 1 (S13). And the content acquisition part 51 receives the response with respect to this request, and acquires the segment data contained in a response (S14).

  The content analysis unit 52 acquires segment data from the content acquisition unit 51, and checks whether event information is added to the segment data (S15). When event information is not added (NO in S15), the content analysis unit 52 outputs the acquired segment data as it is to the content reproduction unit 53. The content reproduction unit 53 reproduces the original content based on the acquired segment data. On the other hand, when the event information is added to the segment data (YES in S15), the content analysis unit 52 outputs the segment data and the event information to the content reproduction unit 53.

  The content playback unit 53 checks whether the playback information identifier included in the event information is an unlimited playback instruction or a limited playback instruction (S17). If the reproduction information identifier is an unlimited reproduction instruction (YES in S17), the content reproduction unit 53 further determines whether or not the expiration date information is included in the event information (S18). When the event information includes the expiration date information (YES in S18), the content reproduction unit 53 includes the event information in the event information when the time for reproducing the additional content is within the expiration date indicated by the expiration date information (YES in S19). The additional content based on the main data to be reproduced is superimposed on the original content and reproduced (S21). On the other hand, when the time for reproducing the additional content is not within the expiration date indicated by the expiration date information (NO in S19), the content reproduction unit 53 ignores the event information and reproduces the original content based only on the segment data. (S20).

  In S18, when the expiration date information is not included in the event information (NO in S18), the content reproduction unit 53 reproduces the additional content based on the main data included in the event information by superimposing it on the original content ( S21).

  If the playback information identifier is a limited playback instruction in S17 (NO in S17), the content playback unit 53 further determines whether playback of the original content is live playback or time shift playback (S22). ). In the case of live reproduction (YES in S22), the content reproduction unit 53 reproduces the additional content based on the main data included in the event information by superimposing it on the original content (S23). On the other hand, in the case of time shift reproduction (NO in S22), the content reproduction unit 53 ignores the event information and reproduces the original content based only on the segment data (S24).

  If all the segment data constituting the original content has not been received after the processing of S16 and S20 to S24 (NO in S25), the content acquisition unit 51 sends a request for transmission of the next segment to the server 1 Transmit (S26) and execute S14 and subsequent steps. On the other hand, when all the segment data constituting the original content has been received (YES in S25), the client 2 ends the content acquisition / playback process.

[Example 1]
Next, a specific embodiment will be described in which event information is added to the segment data body in the box format defined by ISOBFF, and additional content is superimposed on the original content and reproduced based on the event information.

  First, a conventional basic data structure of segment data will be described with reference to FIG. FIG. 5 is a diagram showing a data structure of conventional segment data. As shown in FIG. 5, conventional segment data 80 includes one styp (Segment Type Box) 81, one sidx (Segment Index Box) 82, and moof (Movie Fragment Box) 83 and mdat (Media Data Box). It is composed of one or more sets of 84.

  The styp 81 is information indicating the segment type and / or version information. The sidx 82 is information related to random access points in the segment. The moof 83 and the mdat 84 are information regarding the fragments constituting the segment.

  Next, the data structure of segment data to which event information is added will be described with reference to FIG. FIG. 6 is a diagram illustrating a data structure of segment data to which event information is added. As shown in FIG. 6, the segment data 85 to which event information is added includes one emes (Event Message Box) 86 in addition to the conventional configuration.

  The emes 86 is event information including at least additional content information and a reproduction information identifier. In this embodiment, the reproduction information identifier is represented by “piid”, the unlimited reproduction instruction is represented by “permanent”, and the limited reproduction instruction “temporally”. That is, the attribute “piid” is described in the emes 86, and “permanent” or “temporally” is described as the attribute value.

  As shown in FIG. 7, it is assumed that emes_a to emes_c, which are emes 86, are added to the segments # s1 to # s3, respectively. Further, it is assumed that “permanent” is described as the attribute value of the attribute “piid” in emes_a and emes_b, and “temporally” is described as the attribute value of the attribute “piid” in emes_c.

  In this case, in the case of live playback, the content playback unit 53 plays back additional content included in emes_a to emes_c. On the other hand, in the case of time shift reproduction, the content reproduction unit 53 reproduces additional content included in emes_a and emes_b, but does not reproduce additional content included in emes_c.

  Next, an example of determining live playback or time shift playback based on playback time information and allowable width information will be described with reference to FIG. FIG. 8 is a diagram illustrating the reproduction timing of segments # s1 to # s3 of client A that performs live reproduction and client B that performs time-shift reproduction.

  Here, it is assumed that reproduction time information and allowable width information are described in emes_a to emes_c of segments # s1 to # s3 shown in FIG. Also, the reproduction time information described in emes_a to emes_c, respectively, indicates times T1 to T3. Further, the times when the client 2 actually reproduces the additional contents included in emes_a to emes_c are t1 to t3, respectively.

  As shown in FIG. 8, in the client A, the difference between the time T3 indicated by the reproduction time information described in emes_c and the time t3 when the client A actually reproduces the additional content is within the allowable range indicated by the allowable width information. Therefore, the client A reproduces the additional content included in emes_c by superimposing it on the original content. On the other hand, since the difference between the time T3 indicated by the reproduction time information described in emes_c and the time t3 when the client B actually reproduces the additional content is not within the allowable range indicated by the allowable width information, the client B , Ignores emes_c and plays only the original content.

  In this way, when event information is added to the segment data body, event information is not lost on the way, even if it passes through a proxy that ignores the HTTP header etc. You can be notified.

[Example 2]
Next, a specific embodiment in which event information is described in an HTTP header of an HTTP response message that transmits a segment data body, and additional content is superimposed on the original content and reproduced based on the event information is shown in FIGS. 10 will be described. FIG. 9 is a diagram illustrating an example of an operation sequence when event information is described in an HTTP header of an HTTP response message that transmits a segment data body, and additional content is superimposed on the original content and reproduced based on the event information. It is. FIG. 10 shows an example of an HTTP message when event information is described in an HTTP header of an HTTP response message that transmits a segment data body, and additional content is superimposed on the original content and reproduced based on the event information. FIG.

  As shown in FIGS. 9 and 10, the content acquisition unit 51 of the client 2 refers to the MPD data and transmits a request message 91 for requesting transmission of a certain segment to the server 1. Here, the event information adding unit 23 generates event information including main content data of the additional content and a reproduction information identifier indicating the limited reproduction instruction based on the insertion instruction of the additional content. The event information addition unit 23 instructs the content transmission unit 22 to describe the event information in the HTTP header of the HTTP response message for transmitting the segment data body acquired by the content acquisition unit 21 from the segment storage device 5. Then, the content transmission unit 22 adds event information as a MIME Type parameter to the Content-Type header of the HTTP response message that transmits the segment data body. Then, as a response to the request message 91, the content transmission unit 22 transmits a response message 92 including the segment main data and event information added to the header to the client 2.

  Upon receiving the response message 92, the client 2 reproduces the additional content by superimposing it on the original content in the case of live reproduction, and reproduces the original content ignoring the event information in the case of time shift reproduction.

  As described above, when adding event information to the HTTP header of the response message, it is not necessary to generate a segment to which event information is added for each representation even when the server prepares a plurality of representations. In other words, the additional processing accompanying event information addition does not depend on the number of representations.

[Supplement]
The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

  Finally, each block of the server 1 and the client 2, particularly the server control unit 11 and the client control unit 41, may be configured by hardware logic, or may be realized by software using a CPU as follows. .

  That is, the server 1 and the client 2 include a CPU (central processing unit) that executes instructions of a control program that realizes each function, a ROM (read only memory) that stores the program, and a RAM (random access memory) that expands the program. ), A storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is a recording medium in which program codes (execution format program, intermediate code program, source program) of control programs for the server 1 and the client 2 which are software for realizing the functions described above are recorded so as to be readable by a computer. Can also be achieved by reading the program code recorded on the recording medium and executing it by the computer (or CPU or MPU).

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM (registered trademark) / flash ROM.

  Further, the server 1 and the client 2 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Further, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention can be used for a content transmission device that transmits content of the MPEG-DASH standard and a content playback device that acquires and plays back the content.

1 server 2 client 4 MPD storage device 5 segment storage device 6 content distribution system 21 content acquisition unit 22 content transmission unit (transmission means)
23 Event information adding part (information generating means)
51 Content acquisition unit (acquisition means)
52 content analysis unit 53 content playback unit (playback means)

Claims (16)

A content transmission device that transmits original content composed of a plurality of segments to a content reproduction device for each segment,
As a response to a request for requesting transmission of each segment from the content reproduction device, a transmission unit that transmits a response including the segment to the content reproduction device;
Information generating means for generating event information including additional content information and a reproduction information identifier,
The additional content information is information for acquiring main content data of the additional content to be reproduced by being superimposed on the original content by the content reproduction device or the additional content.
The playback information identifier is an unlimited playback instruction that causes the content playback device to play back additional content, whether it is live playback or time-shift playback, or additional content is played back on the content playback device only in the case of live playback. Indicates a limited playback instruction to be performed,
The content transmission apparatus, wherein the transmission means transmits the event information generated by the information generation means together with the segment.   The content transmission apparatus according to claim 1, wherein the information generation unit adds the event information to a header portion of the segment.   The content transmission apparatus according to claim 1, wherein the information generation unit adds the event information to a header of the response.   The information generation means includes playback time information indicating a playback time of the additional content, and information for determining whether to perform live playback or time shift playback, the time indicated by the playback time information, and the content playback device 4. The content transmission device according to claim 1, further comprising: generating event information further including tolerance width information indicating a tolerance range of a difference from a time when the additional content is actually played back. 5.   5. The information generation unit according to claim 1, wherein when the reproduction information identifier is an unlimited reproduction instruction, the information generation unit generates event information further including expiration date information indicating an expiration date of the unlimited reproduction instruction. The content transmission device according to claim 1. A content playback apparatus that acquires and plays back original content composed of a plurality of segments from a content transmission apparatus for each segment,
An acquisition means for transmitting a request for transmission of each segment to the content transmission device, and acquiring a response including the segment as a response to the request;
Playback means for playing back the original content based on the segment acquired by the acquisition means,
The reproducing means is
The acquisition means acquires, together with the segment, main data of additional content to be reproduced superimposed on the original content or additional content information that is information for acquiring the additional content, and event information including a reproduction information identifier if you did this,
Even if the playback information identifier is live playback or time-shift playback, in the case of an unlimited playback instruction indicating that additional content is to be played, the additional content is played back by being superimposed on the original content based on the additional content information And
This is a limited playback instruction indicating that additional content is played back only when the playback information identifier is live playback. When playback of the original content is live playback, the additional content is superimposed on the original content based on the additional content information. Let it play,
A content playback apparatus, wherein when the playback information identifier is the limited playback instruction and the playback of the original content is time-shift playback, the event information is ignored and only the original content is played back.   The content reproducing apparatus according to claim 6, wherein the acquisition unit acquires the event information added to a header portion of the segment.   The content reproducing apparatus according to claim 6, wherein the acquisition unit acquires the event information added to a header of the response.   The playback means includes, in addition to the event information acquired by the acquisition means, playback time information indicating a playback time of the additional content, and information for determining whether to play live or time-shifted. When the allowable range information indicating the allowable range of the difference between the time indicated by the time information and the time when the content reproduction device actually reproduces the additional content is included, the time when the additional content is actually reproduced, It is determined whether the reproduction of the original content is live reproduction or time shift reproduction based on whether or not the difference from the time indicated by the reproduction time information is within an allowable range indicated by the allowable width information. The content reproduction apparatus according to any one of claims 6 to 8.   When the event information acquired by the acquisition unit further includes expiration date information indicating the expiration date of the unrestricted playback instruction, when the playback information identifier is an unrestricted playback instruction, the playback unit If it is within the expiration date indicated by the information, the additional content is reproduced and superimposed on the original content based on the additional content information. On the other hand, if it is not within the expiration date indicated by the expiration date information, the event information is ignored. The content playback apparatus according to claim 6, wherein only the original content is played back. The content transmission device according to any one of claims 1 to 5,
A content distribution system comprising: the content reproduction device according to claim 6. A content transmitting apparatus control method for transmitting original content composed of a plurality of segments to a content reproduction apparatus for each segment,
A transmission step of transmitting a response including the segment to the content reproduction device as a response to the request for transmission of each segment from the content reproduction device;
Generating an event information including additional content information and a reproduction information identifier, and
The additional content information is information for acquiring main content data of the additional content to be reproduced by being superimposed on the original content by the content reproduction device or the additional content.
The playback information identifier is an unlimited playback instruction for causing the content playback device to play back additional content, whether it is live playback or time shift playback, or the additional content is played back on the content playback device only in the case of live playback. Indicates a limited playback instruction to be performed,
In the transmission step, the event information generated in the information generation step is transmitted together with the segment. A method for controlling a content reproduction device that acquires and reproduces original content composed of a plurality of segments from a content transmission device for each segment,
An acquisition step of transmitting a request for requesting transmission of each segment to the content transmission device, and acquiring a response including the segment as a response to the request;
Replaying the original content based on the segment acquired in the acquisition step,
In the playback step above,
In the acquisition step, together with the segment, the main content data of the additional content to be reproduced superimposed on the original content or additional content information that is information for acquiring the additional content, and event information including a reproduction information identifier are acquired. if you did this,
Even if the playback information identifier is live playback or time-shift playback, in the case of an unlimited playback instruction indicating that additional content is to be played, the additional content is played back by being superimposed on the original content based on the additional content information And
This is a limited playback instruction indicating that additional content is played back only when the playback information identifier is live playback. When playback of the original content is live playback, the additional content is superimposed on the original content based on the additional content information. Let it play,
A method for controlling a content playback apparatus, wherein, when the playback information identifier is the limited playback instruction and playback of the original content is time shift playback, the event information is ignored and only the original content is played back.   A control program for operating the content transmission device according to any one of claims 1 to 5, wherein the control program causes a computer to function as each of the means.   A control program for operating the content reproduction apparatus according to any one of claims 6 to 10, wherein the control program causes a computer to function as each of the means.   A computer-readable recording medium in which the control program according to at least one of claims 14 and 15 is recorded.

Images