JP2008005288A - Information processing apparatus, information processing method, program, and program storage medium - Google Patents

Abstract

Information about sub-paths displayed in picture-in-picture is presented to a user.
A GUI display screen 391 displays, for example, a main timeline 401 corresponding to the entire primary video playback section and a slider 402 indicating the current primary video playback position on the main timeline 401. Corresponding to the timeline 401, it can be seen which secondary video can be played as the sub-screen 302 and the secondary audio is played back in correspondence with the primary video playback section. A sub timeline 403 corresponding to Secondary Video or Secondary Audio included in the stream reproduced by this PlayList file is displayed. Then, an icon 404 indicating whether the stream associated with the sub timeline 403 is Secondary Video or Secondary Audio is displayed. The present invention can be applied to a playback apparatus.
[Selection] Figure 34

Description

  The present invention relates to an information processing device, an information processing method, a program, and a program storage medium, and more particularly, an information processing device suitable for use when a plurality of video streams can be combined and displayed. The present invention relates to an information processing method, a program, and a program storage medium.

  In the DVD (Digital Versatile Disc) video standard, when AV (Audio Visual) content such as a movie recorded on a recording medium is played, an interactive operation such as audio switching or subtitle switching is provided to the user. (For example, refer nonpatent literature 1). Specifically, for the AV content displayed on the display device 1 in FIG. 1, the user operates the audio switching button 11 or the subtitle switching button 12 of the remote controller 2 to switch the audio or subtitle. Switch. For example, when the voice 1 is set as the initial state and the user operates the voice switching button 11 of the remote controller 2, the voice 1 is switched to the voice 2 as shown in FIG.

  AV content on a DVD video is recorded in MPEG (Moving Picture Experts Group) 2 program stream format. As shown in FIG. 3, the MPEG2 program stream includes a video stream (video in FIG. 3), a plurality of audio streams (audios 1, 2, and 3 in FIG. 3), and a plurality of sub-picture streams (in FIG. 3). The sub-pictures 1, 2, 3) are multiplexed so as to be reproduced in AV synchronization with the video stream. The sub-picture stream (sub-picture 1, 2, 3) is a stream in which a bitmap image is run-length encoded, and is mainly used for subtitles.

  In general, a plurality of audio streams are used for recording audio in different languages, and a plurality of sub-picture streams are used for recording subtitles in different languages. When the video is being played back, the user can interactively select audio or subtitles in a desired language using the remote controller 2.

DVD Specifications for Read-Only Disc Part 3; Version1.1

  In addition, there is a technique in which a Sub path is used to reproduce additionally recorded sound (for example, so-called after-recording sound) (for example, Patent Document 1).

JP 2000-158972 A

  Also, there is a technique for expressing a plurality of video streams on a timeline on the same time axis in order to edit a plurality of video data (for example, Patent Document 2).

JP 2000-348469 A

  In the technique described in Patent Document 1, the Sub path is used to reproduce the additionally recorded sound (for example, so-called after-recording sound). There is no disclosure of a stream, that is, a configuration for combining and reproducing two audio streams or two video streams.

  Further, in the technique described in Patent Document 2, a plurality of video streams can be expressed on a timeline on the same time axis for editing work. These are independent video streams picked up based on individual time codes, and are not synthesized and reproduced.

  Therefore, for example, when two video streams are combined and reproduced, there has been no technique for grasping at which position each stream is combined on the reproduction time axis.

  The present invention has been made in view of such a situation, and makes it possible to grasp the state of synthesis when a plurality of video streams are synthesized and reproduced.

  An information processing apparatus according to an aspect of the present invention analyzes first reproduction means for obtaining reproduction management information for controlling reproduction of a stream, and the reproduction management information obtained by the first acquisition means, From the analysis result, a second acquisition unit that acquires information on one or a plurality of sub reproduction paths, which is different from the main reproduction path of the stream, is acquired by the second acquisition unit. And information generation means for generating information on the main reproduction path and the stream reproduced by the sub reproduction path based on the information on the sub reproduction path.

  The second acquisition unit may acquire information regarding the reproduction position of the sub reproduction path.

  The information generation means reproduces all the sub reproduction paths managed for reproduction by the reproduction management information in association with the reproduction time axis of the first stream reproduced by the main reproduction path. Image information indicating the reproduction position of the second stream can be generated.

  The information generation unit may generate the image information including information indicating each reproduction range of the plurality of second streams reproduced by the sub reproduction path.

  The information generation means may generate the image information including information indicating a type of each of the plurality of second streams reproduced by the sub reproduction path.

  A third acquisition unit that acquires at least a part of the stream data can be further provided, and the information generation unit includes the stream data acquired by the third acquisition unit. On the basis of at least a part of the image information, the image information including at least a part of the image data constituting the plurality of second streams reproduced by the sub reproduction path can be generated.

  The second acquisition means can acquire a part of image data constituting a stream reproduced by the primary reproduction path and the secondary reproduction path, and the information generation means Is based on at least a part of the image data that constitutes the stream acquired by the second acquisition means, among the image data that constitutes the first stream that is reproduced by the main reproduction path. Image information including at least a part and at least a part of the image data constituting the second stream reproduced by the sub reproduction path can be generated.

  A display control means for controlling display of information on the stream generated by the information generation means can be further provided.

  A third acquisition unit that acquires the stream data, and a reproduction that controls reproduction of the stream data acquired by the third acquisition unit based on the reproduction management information acquired by the first acquisition unit Control means, and the display control means superimposes information on the stream generated by the information generation means on the stream data whose reproduction is controlled by the reproduction control means. The display of information regarding the stream can be controlled so that the information is displayed.

  Operation input acquisition means for acquiring an operation input by a user can be further provided, and the second acquisition means can acquire information related to the reproduction position of the secondary reproduction path. The information generating means reproduces the reproduction data by all the sub reproduction paths managed for reproduction by the reproduction management information in association with the reproduction time axis of the first stream reproduced by the main reproduction path. Image information indicating a reproduction position of the second stream can be generated, and the display control unit can control generation of the image information generated by the information generation unit. And the playback position of the second stream by the user with reference to the image information whose display is controlled by the display control means by the operation input acquisition means. When an operation input for selecting any one of them is acquired, the playback control unit controls playback of the first stream and the second stream from the selected playback position. be able to.

  An operation input acquisition means for acquiring an operation input by a user can be further provided. The second acquisition means includes information on the reproduction position of the secondary reproduction path and images constituting the stream. A part of the data can be acquired, and the information generation unit is configured to acquire the main reproduction based on at least a part of the stream data acquired by the second acquisition unit. An image including at least a part of the image data constituting the first stream reproduced by the path and at least a part of the image data constituting the second stream reproduced by the sub reproduction path Information may be generated, and the display control unit may control generation of the image information generated by the information generation unit. The operation input acquisition unit selects any of the image data constituting the second stream by the user with reference to the image information whose display is controlled by the display control unit. When the operation input is acquired, the reproduction control unit receives the first stream and the second stream from the first reproduction position of the second stream corresponding to the image data selected by the user. Playback can be controlled.

  An information processing method according to an aspect of the present invention is an information processing method for an information processing apparatus that generates information about a stream to be played back, and acquires and manages playback management information for controlling playback of the stream. Analyzing the reproduction management information, obtaining information relating to one or a plurality of secondary reproduction paths, which is different from the primary reproduction path of the stream, and obtaining information relating to the obtained secondary reproduction path And generating information related to the stream reproduced by the main reproduction path and the sub reproduction path.

  The program of one aspect of the present invention or the program stored in the program storage medium controls acquisition of reproduction management information for controlling reproduction of a stream, analyzes the reproduction management information for which acquisition is controlled, The playback path is different from the main playback path of the stream, controls acquisition of information on one or more secondary playback paths, and based on the information on the secondary playback path whose acquisition is controlled, Causing the computer to execute a process including a step of generating information relating to the stream reproduced by the main reproduction path and the sub reproduction path.

  In one aspect of the present invention, reproduction management information for controlling the reproduction of a stream is acquired, the reproduction management information is analyzed, and the reproduction path is different from the main reproduction path of the stream, and one or more Information on the secondary reproduction path is acquired, and information on the main reproduction path and the stream reproduced by the secondary reproduction path is generated based on the information on the secondary reproduction path.

  The network is a mechanism in which at least two devices are connected and information can be transmitted from one device to another device. The devices that communicate via the network may be independent devices, or may be internal blocks that constitute one device.

  The communication is not only wireless communication and wired communication, but also communication in which wireless communication and wired communication are mixed, that is, wireless communication is performed in a certain section and wired communication is performed in another section. May be. Further, communication from one device to another device may be performed by wired communication, and communication from another device to one device may be performed by wireless communication.

  The playback device may be an independent device or a block that performs playback processing of the recording / playback device.

  As described above, according to one aspect of the present invention, the reproduction management information is analyzed. In particular, based on the information on the secondary reproduction path, information on the main reproduction path and the stream reproduced by the secondary reproduction path is obtained. Thus, not only the stream reproduced by the main reproduction path but also the contents and reproduction position of the stream reproduced by the sub reproduction path can be presented to the user in an easily understandable manner.

  Embodiments of the present invention will be described below. Correspondences between constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

  An information processing apparatus according to an aspect of the present invention includes first acquisition means (for example, PlayList read control unit 351 in FIG. 32) that acquires reproduction management information (for example, PlayList) for controlling reproduction of a stream, Analyzing the reproduction management information acquired by the first acquisition means, and based on the analysis result, is a reproduction path different from the main reproduction path of the stream, and information on one or more sub reproduction paths (for example, , Image data, or IN_time and OUT_time) second acquisition means (for example, PlayList analysis unit 352 and IN_time / OUT_time acquisition unit 356 in FIG. 32) and the sub-acquisition acquired by the second acquisition means. Information on the stream reproduced by the main reproduction path and the sub reproduction path (for example, data on the GUI display screen in FIG. 33 to FIG. Information generation means for generating (information stored in 32 reproduction start position information storage control unit 358).

  The second acquisition unit can acquire information (for example, IN_time and OUT_time) regarding the playback position of the secondary playback path.

  The information generating means is associated with the reproduction time axis of the first stream reproduced by the main reproduction path (for example, the main timeline 401 in FIG. 34), and all of the reproduction managed by the reproduction management information. Image information (for example, data on the GUI display screen 391 of FIG. 33 or FIG. 34) indicating the playback position of the second stream played back by the sub playback path of the above can be generated.

  The information generation unit can generate the image information including information (for example, a sub timeline 403) indicating a reproduction range of each of the plurality of second streams reproduced by the sub reproduction path.

  The information generation unit can generate the image information including information (for example, an icon 404) indicating each stream type of the plurality of second streams reproduced by the sub reproduction path.

  A third acquisition unit (for example, the stream data read control unit 354 in FIG. 32) that acquires at least a part of the stream data can be further provided, and the information generation unit is Based on at least a part of the acquired stream data, includes at least a part (for example, a thumbnail) of image data constituting the plurality of second streams reproduced by the sub reproduction path. The image information can be generated.

  The second acquisition means may acquire a part of image data (for example, image data that is a basis of a thumbnail) that constitutes a stream that is reproduced by the primary reproduction path and the secondary reproduction path. The information generation means can configure a first stream to be reproduced by the main reproduction path based on at least a part of the image data constituting the stream acquired by the second acquisition means. At least a part of the image data (for example, the thumbnail image 441 in FIG. 35) and at least a part of the image data constituting the second stream reproduced by the sub reproduction path (for example, FIG. Image information including 35 thumbnail images 442) (for example, data of the GUI display screen 421 in FIG. 35) can be generated.

  The display control means (for example, GUI display control part 359 of FIG. 32) which controls the display of the information regarding the said stream produced | generated by the said information production | generation means can be further provided.

  Based on the third acquisition means for acquiring the stream data (for example, the stream data read control unit 354 in FIG. 32) and the reproduction management information acquired by the first acquisition means, the third acquisition means. Further comprising a reproduction control means (for example, a reproduction control unit 355 in FIG. 32) for controlling reproduction of the stream data acquired by the display control means, wherein the display control means controls the stream data whose reproduction is controlled by the reproduction control means. In addition, it is possible to control the display of the information related to the stream (control the image composition by the OSD 30) so that the information related to the stream generated by the information generating means is displayed superimposed.

  An operation input acquisition unit (for example, an operation input acquisition unit 353 in FIG. 32) that acquires an operation input by a user can be further provided, and the second acquisition unit includes information on the reproduction position of the sub reproduction path (for example, , IN_time and OUT_time), and the information generating means associates with the playback time axis (for example, timeline 401 in FIG. 34) of the first stream played back by the main playback path, Image information (for example, data on the GUI display screen 391 in FIG. 33 or FIG. 34) indicating the playback position of the second stream played back by all the secondary playback paths whose playback is managed by the playback management information. The display control means can control the generation of the image information generated by the information generation means, and the operation input acquisition means allows the display to be displayed. When an operation input for selecting any one of the playback positions of the second stream by the user referring to the image information whose display is controlled by the control means is acquired, the playback control means Playback of the first stream and the second stream from the playback position can be controlled.

  An operation input acquisition unit (for example, an operation input acquisition unit 353 in FIG. 32) that acquires an operation input by a user can be further provided, and the second acquisition unit includes information on the reproduction position of the sub reproduction path (for example, , IN_time and OUT_time) and part of the image data constituting the stream (for example, image data that is the basis of the thumbnail), and the information generation means is the second acquisition means 35, at least a part of the image data constituting the first stream reproduced by the main reproduction path (for example, the thumbnail image 441 in FIG. 35). ) And at least a part of the image data constituting the second stream reproduced by the sub reproduction path (for example, the thumbnail of FIG. Image information including the image 442) (for example, data of the GUI display screen 421 in FIG. 35) can be generated, and the display control unit controls generation of the image information generated by the information generation unit. An operation input for selecting any one of the image data constituting the second stream by the user referring to the image information whose display is controlled by the display control unit by the operation input acquisition unit is possible. When acquired, the reproduction control means controls reproduction of the first stream and the second stream from the first reproduction position of the second stream corresponding to the image data selected by the user. be able to.

  An information processing method according to an aspect of the present invention is an information processing method of an information processing device that generates information about a stream to be played back, and acquires playback management information (for example, PlayList) for controlling playback of the stream (For example, the process of step S41 in FIG. 37), the obtained reproduction management information is analyzed, and is a reproduction path different from the main reproduction path of the stream, and one or a plurality of sub reproduction paths. Information (for example, image data or IN_time and OUT_time) is acquired (for example, processing in steps S42 to S51 in FIG. 37), and the main reproduction is performed based on the acquired information regarding the sub reproduction path. Information on the stream reproduced by the path and the sub reproduction path (for example, data on the GUI display screen of FIGS. 33 to 35 or reproduction start position information of FIG. 32) Generates information) stored in the 憶 controller 358 (e.g., including processing) step in step S53 of FIG. 37.

A program according to one aspect of the present invention is a program for causing a computer to execute processing for generating information regarding a stream to be played back, and acquiring playback management information (for example, PlayList) for controlling playback of the stream (For example, the process of step S41 in FIG. 37), the playback management information whose acquisition is controlled is analyzed, and is a playback path different from the main playback path of the stream, and one or more Information on the secondary reproduction path (for example, image data or IN_time and OUT_time) is controlled (for example, processing in steps S42 to S51 in FIG. 37), and information on the secondary reproduction path whose acquisition is controlled. On the basis of the information related to the stream reproduced by the main reproduction path and the sub reproduction path (for example, GUI display in FIGS. 33 to 35) To produce information which is stored in the reproduction start position information storage control unit 358 of the surface of the data) or 32 (for example, step S53 in FIG. 37)
Causes a computer to execute processing including steps.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 4 shows an example of an application format on a local storage (for example, a hard disk or a memory) of a playback apparatus (described later with reference to FIG. 29) to which the present invention is applied or on a recording medium to be mounted. FIG. The recording medium may be a magnetic disk or a semiconductor memory in addition to the optical disk described later.

  The application format has two layers, PlayList and Clip, for managing AV (Audio Visual) streams. Here, a pair of an AV stream and clip information (Clip Information) that is information accompanying the AV stream is considered as one object, and these are collectively referred to as a Clip. Hereinafter, the AV stream is also referred to as an AV stream file. In addition to audio data and video data, the AV stream file may include various stream files that are reproduced together with these data. Clip information is also referred to as a Clip information file.

  In general, a file used in a computer or the like is handled as a byte string, but the content of an AV stream file is expanded on the time axis, and an access point of a clip is mainly designated by a PlayList with a time stamp. That is, PlayList and Clip are layers for AV stream management.

  When the access point in the Clip is indicated by the PlayList with a time stamp, the Clip Information file is used to find address information to start decoding in the AV stream file from the time stamp.

  The PlayList is a collection of AV stream playback sections. One playback section in a certain AV stream is called PlayItem, which is represented by a pair of an IN point (playback start point) and an OUT point (playback end point) of the playback section on the time axis. Therefore, the PlayList is composed of one or a plurality of PlayItems as shown in FIG.

  In FIG. 4, the first PlayList from the left is composed of two PlayItems, and the two PlayItems refer to the first half and the second half of the AV stream included in the left Clip, respectively. Also, the second PlayList from the left is composed of one PlayItem, whereby the entire AV stream included in the right Clip is referenced. Further, the third PlayList from the left is composed of two PlayItems, and the two PlayItems refer to a portion of the AV stream included in the left Clip and a portion of the AV stream included in the right Clip, respectively. ing.

  For example, when the disc navigation program of FIG. 4 specifies the left PlayItem included in the first PlayList from the left as the information indicating the playback position at that time, it is included in the left Clip referred to by the PlayItem. The first half of the AV stream is played back. Thus, the PlayList is used as playback management information for managing playback of AV stream files.

  The disc navigation program has a function of controlling the playback order of the PlayList and interactive playback of the PlayList. The disc navigation program also has a function of displaying a menu screen for the user to instruct execution of various reproductions. This disc navigation program is described in a programming language such as Java (registered trademark), and is prepared on a recording medium.

  In this embodiment, a playback path created by a sequence of one or more PlayItems (by successive PlayItems) in the PlayList is called a main path (Main Path), and in the PlayList, parallel to the Main Path (in parallel) ) And a playback path created by a sequence of one or more sub paths (which may be discontinuous or may be continuous) is referred to as a sub path. That is, the application format on the local storage in the playback apparatus (to be described later with reference to FIG. 29) or on the recording medium to be loaded is a sub path (Sub) that is played back in association with (in addition to) the main path. Path) can be included in the PlayList.

  FIG. 5 is a diagram for explaining the structure of the main path and the sub path. The PlayList can have one main path and one or more sub paths. One main path is created by a sequence of one or more PlayItems, and one sub-path is created by a sequence of one or more SubPlayItems.

  In the case of the example in FIG. 5, the PlayList has one main path made up of a sequence of three PlayItems and three sub paths. Each PlayItem constituting the main path is assigned ID (Identification) in order from the top. Specifically, the main path is made up of PlayItems with PlayItem_id = 0, PlayItem_id = 1, and PlayItem_id = 2. Subpaths are also assigned IDs in order from the top, Subpath_id = 0, Subpath_id = 1, and Subpath_id = 2. A subpath of Subpath_id = 0 includes one SubPlayItem, a subpath of Subpath_id = 1 includes two SubPlayItems, and a subpath of Subpath_id = 2 includes one SubPlayItem.

  The stream referred to by the SubPlayItem included in the subpath of Subpath_id = 0 is assumed to be, for example, a Japanese dubbing sound of a movie, and may be played in place of the audio stream of the AV stream file referenced by Mainpath. Also, the stream referred to by SubPlayItem included in the subpath of Subpath_id = 1 is assumed to be a director's cut of a movie, for example, and a comment such as a movie director is included only in a predetermined part of the AV stream file referred to by Main Path. The case is considered.

  A Clip AV stream file referred to by one PlayItem includes at least video stream data (main image data). In addition, the Clip AV stream file may or may include one or more audio streams that are played back at the same timing (synchronously) as the video stream (main image data) included in the Clip AV stream file. It does not have to be. Furthermore, the Clip AV stream file may or may not include one or more bitmap subtitle streams (Presentation Graphic streams) that are played back at the same timing as the video stream included in the Clip AV stream file. Good. The Clip AV stream file may or may not include one or more interactive graphics streams that are played back at the same timing as the video stream included in the Clip AV stream file. The video stream included in the Clip AV stream file and the audio stream, bitmap subtitle stream file, or interactive graphics stream reproduced at the same timing as the video stream are multiplexed. That is, a Clip AV stream file referred to by one PlayItem includes video stream data, zero or more audio streams to be played in accordance with the video stream, zero or more bitmap subtitle stream data, and zero or more. The interactive graphics stream data is multiplexed.

  That is, a Clip AV stream file referred to by one PlayItem includes a plurality of types of streams such as a video stream, an audio stream, a bitmap subtitle stream file, or an interactive graphics stream.

  Also, one SubPlayItem refers to a video stream, an audio stream, subtitle data, or the like of a stream (different stream) different from the Clip AV stream file referred to by the PlayItem.

  When playing a PlayList that has only the main path, the user's operation of switching audio or subtitles selects audio or subtitles from the audio stream and sub-picture stream multiplexed in the clip referenced by the main path. It is only possible. On the other hand, when playing a PlayList having a main path and a sub path, in addition to the audio stream and sub picture stream multiplexed in the Clip AV stream file referenced by the main path, the clip audio stream referenced by the SubPlayItem And a sub-picture stream can be selected and reproduced.

  In this way, since a plurality of SubPaths are included in one PlayList and each SubPath refers to each SubPlayItem, an AV stream with high expandability and high flexibility can be realized. In other words, in addition to the Clip AV stream referred to by MainPath, a SubPlayItem can be added later.

  FIG. 6 is a diagram illustrating an example of a main path and a sub path. In FIG. 6, an audio playback path that is played back at the same timing as the main path (synchronized with AV) is represented using a sub path.

  The PlayList in FIG. 6 includes one PlayItem with PlayItem_id = 0 as a main path and one SubPlayItem as a sub path. One PlayItem () with PlayItem_id = 0 in the main path refers to the main Clip AV stream. SubPlayItem () includes the following data. First, SubPlayItem () includes Clip_Information_file_name for designating a Clip referred to by a Sub Path in the PlayList. In the example of FIG. 6, an auxiliary audio stream (audio stream) with SubClip_entry_id = 0 is referred to by SubPlayItem. Also, SubPlayItem () includes SubPlayItem_IN_time and SubPlayItem_OUT_time for designating the playback section of the Sub Path in the Clip (Auxiliary audio stream here). Further, SubPlayItem () includes sync_PlayItem_id and sync_start_PTS_of_PlayItem for specifying the time at which the Sub Path starts to be reproduced on the time axis of the Main path. In the case of the example in FIG. 6, sync_PlayItem_id = 0 and sync_start_PTS_of_PlayItem = t1. As a result, it is possible to specify the time t1 at which the Sub Path starts playback on the time axis of PlayItem_id = 0 of the main path. That is, in the case of the example in FIG. 6, the reproduction start time t1 of the main path and the start time t1 of the sub path are the same time.

  Here, the Clip AV stream of the audio referred to by the Sub Path must not include STC discontinuities (system time base discontinuities).

  Also, information included in SubPlayItem () (information specifying a Clip referred to by Sub Path, information specifying a playback section of Sub Path, and time at which Sub Path starts playback on the time axis of Main path) Based on the information), the audio stream of the clip AV stream different from the clip AV stream (mainly reproduced main AV stream) referred to by the main path can be referred to and reproduced.

  As described above, PlayItem and SubPlayItem manage Clip AV stream files, respectively. Here, the Clip AV stream file (main AV stream) managed by PlayItem and the Clip AV stream file managed by SubPlayItem are the same file. Multiple files may be multiplexed, or different files may be used.

  Similar to the example of FIG. 6, a subtitle stream playback path that is played back at the same timing as the main path can be expressed using a sub path.

  FIG. 7 is a diagram illustrating another example of the main path and the sub path. In FIG. 7, an audio playback path that is played back at the same timing as the main path (synchronized with AV) is represented using a sub path. Here, the main Clip AV stream file referred to by PlayItem in the main path is the same as in FIG.

  For example, a clip AV stream referenced by the main path is set as one movie content (AV content), an auxiliary audio stream (Clip) referenced by the sub-path audio path is set as a director's comment for the movie, and is referenced by the main path. Such an arrangement is used in the case where an auxiliary audio stream referred to by the audio path of the sub path is mixed (reproduced) with the audio stream of the clip AV stream. That is, the configuration of the main path and the sub path shown in FIG. 7 is that when the user inputs an instruction to listen to the director's comment together with the movie while watching the movie, in other words, This is used when the audio of the Clip AV stream referenced by the path and the auxiliary audio stream referenced by the sub-path audio path are mixed and reproduced.

  In FIG. 7, three PlayItems are arranged in the main path as PlayItem_id = 0, 1, 2, and two SubPlayItems are arranged in the subpath (Subpath_id = 0). SubPlayItem (described later in FIG. 28) called by SubPath_id = 0 (described later in FIG. 26) is an Auxiliary audio stream (English Auxiliary audio stream clip of SubClip_entry_id = 0 and SubClip_entry_id = 1 in Japanese) SubPlayItem_IN_time and SubPlayItem_out_time for specifying the subpath playback section of (Auxiliary audio stream clip).

  7 is compared with FIG. 6, in FIG. 7, an auxiliary audio stream (English or Japanese audio stream) of SubClip_entry_id = 0, 1 can be referred to by SubPlayItem. In other words, a SubPlayItem is used to refer to a plurality of audio stream files, and when the SubPlayItem is reproduced, the audio stream file is selected from the plurality of audio stream files and reproduced. In the case of the example in FIG. 7, one audio stream file is selected and reproduced from the English audio stream file and the Japanese audio stream file. Specifically, one of subClip_entry_id = 0 and 1 is selected (for example, based on a user instruction), and an auxiliary audio stream referred to by the ID is reproduced. In addition to this, when playback in combination with an audio stream referenced by the main path is further selected, the 1st audio stream referenced by the main path and the 2nd audio stream file referenced by the audio path of the sub path are selected. Are mixed and played back. Note that two audio streams referenced in the two sub-paths may be mixed and played back, which will be described later.

  Next, a data structure (syntax) for specifically realizing the main path and sub path structures described with reference to FIGS. 5 to 7 will be described.

  FIG. 8 is a diagram illustrating an example of a file system of a data file that can be played back by the playback apparatus. FIG. 8 shows, as an example, a case where a data file that can be played back by a playback apparatus is supplied by a recording medium such as an optical disk, and this file system has a directory structure.

  In this file system, a directory with the name “BDMV” is prepared under “root”, and a file with the name “Index.bdmv” and a file “NavigationObject.bdmv” Contains a file with a name. Hereinafter, these files will be referred to as an Index file and a NavigationObject file, respectively, as appropriate. As appropriate, each file is referred to by adding “file” to “file name”, or each directory is referred to by adding “directory” to “directory name”.

  The Index file is a file in which an index table is described, and includes information related to a menu for reproducing a data file that can be reproduced by the reproducing apparatus. For example, the playback device displays a playback menu screen including items such as all content included in a data file that can be played back by the playback device, only a specific chapter is played, repeated playback, and an initial menu is displayed. Display on the display device based on the Index file. The navigation object to be executed when each item is selected can be set in the index file index table. When one item is selected from the playback menu screen by the user, the playback device is set in the index file index table. Execute the NavigationObject command.

  The NavigationObject file is a file containing NavigationObject. The NavigationObject includes a command for controlling playback of a PlayList included in a data file that can be played back by the playback device. For example, the playback device selects one of the NavigationObjects included in the file system. By executing, the content can be reproduced.

  The BDMV directory also includes a directory with the name “BACKUP” (BACKUP directory), a directory with the name “PLAYLIST” (PLAYLIST directory), and a directory with the name “CLIPINF” (CLIPINF directory). , A directory (STREAM directory) in which the name “STREAM” is set, and a directory (AUXDATA directory) in which the name “AUXDATA” is set.

  In the BACKUP directory, files and data for backing up files and data that can be played back by the playback device are recorded.

  A PlayList file is stored in the PLAYLIST directory. As shown in the figure, each PlayList file is given a name obtained by adding an extension “.mpls” to a file name consisting of a 5-digit number.

  A Clip Information file is stored in the CLIPINF directory. Each Clip Information file is named by adding the extension “.clpi” to a file name consisting of five digits as shown in the figure.

  In the STREAM directory, a Clip AV stream file and a substream file are stored. Each stream file is given a name obtained by adding an extension “.m2ts” to a file name consisting of five digits as shown in the figure.

  The AUXDATA directory is not included in the Clip AV stream file or substream file, but is referred to from the Clip AV stream file or substream file, or data used independently of the Clip AV stream file or substream file. And other files are stored. In the example of FIG. 8, the AUXDATA directory stores sound data such as subtitle font files with the name “11111.otf” and sound effects with the name “sound.bdmv”. ing.

  Hereinafter, the audio data that is mainly played back and output is referred to as a primary audio stream (Primary audio stream) .For example, clicks and sound effects generated in response to user operation inputs, such as sound.bdmv in the AUXDATA directory described above. The described audio data is referred to as sound data. Sound data can be synthesized into the primary audio stream. Also, audio data that is different from the primary audio stream and can be synthesized with the primary audio stream, such as sub audio, is referred to as a secondary audio stream. Note that there may be a plurality of secondary audio streams that are different from the primary audio stream and can be synthesized with the primary audio stream.In this case, for example, using different IDs, What is necessary is just to make it distinguishable.

  Also, when data files that can be played back on the playback device are distributed on an optical disc, for example, the content author, such as a production company or a movie distribution company, or the title author that is the supply source of this recording medium is identified. Therefore, the author_id that is the identifier assigned to each title author and the disc_id that is the identifier assigned to identify the type of optical disc produced by the title author indicated by the author_id can be rewritten by the user or the like. It cannot be recorded as secure electronic data or physically by pits.

  In addition, when a data file that can be played back by a playback device is recorded on a removable recording medium other than an optical disk or downloaded via a network, an id corresponding to author_id and author_id is assigned. Each can be distinguished, and preferably has a directory structure similar to that shown in FIG. In addition, even if an author_id and an id corresponding to author_id are not assigned to a data file that can be played back by the playback device, the name “Index.bdmv” is the same as in the case described with reference to FIG. The set file, the file with the name "NavigationObject.bdmv" is included, the file group with the name "BACKUP", the file group with the name "PLAYLIST", "CLIPINF" Necessary file groups are included among the file group set with the name of "," the file group set with the name of "STREAM", and the file group set with the name of "AUXDATA".

  FIG. 9 shows the syntax of the Index file (Index.bdmv).

  Type_indicator is a field in which the value of “INDX” encoded based on ISO646 is described.

  version_number is a 4-character string indicating the version number of Index.bdmv, and the value “0089” encoded based on ISO646 is described.

  Indexes_start_address is a field in which the start address of the Indexes () block is described.

  In AppInfoBDMV (), an individual identifier indicating the supply source of the data file including this Index file is described.

  In Indexes (), for example, a menu is read, a title is searched, a jump is made to a predetermined title, or a recording medium such as an optical disk in which a data file including this Index file is recorded is stored in the playback device. A link with an application (NavigationObject) that executes a process that automatically starts when attached is described. The syntax of Indexes () will be described later with reference to FIG.

  Padding_word is inserted based on the syntax of Index.bdmv, and is represented by 0 or a positive integer.

  An example of the syntax of Indexes () is shown in FIG.

  The length is a field indicating how many bytes of information are described in Indexes ().

  FirstPlayback is a data block that includes information related to a NavigationObject that is executed first when data of this file system is reproduced. Specifically, it is a data block that includes information related to a NavigationObject that is automatically executed first when a recording medium such as an optical disk on which data of this file system is recorded is loaded in a playback apparatus.

  FirstPlayback_mobj_id_ref is a field for designating the value of mobj_id of the NavigationObject that is executed first. mobj_id is an ID to which NavigationObject is uniquely given. Also, if the NavigationObject that is automatically executed at the start of playback is not set, that is, no application is executed at the start of playback, and any application is executed in response to an instruction from the user. In this case, “0Xffff” is described in this field.

  TopMenu () is a data block that includes information related to NavigationObject of TopMenu that is called when the user displays a menu screen.

  TopMenu_mobj_id_ref is a field for designating the value of mobj_id of NavigationObject of TopMenu. When TopMenu is not set, “0xFFFF” is described in this field.

  number_of_Titles is a field representing the number of titles described in Indexes ().

  Title [title_id] () is a block in which information about each title uniquely distinguished by title_id is described. title_id is assigned from 0.

  Title_playback_type [title_id] is information indicating the playback type of the title specified by the title_id, for example, a movie title that is a general content with a video to be played and audio, or a video that is played by user operation input And Interactive Title, which is a content that can be interactively exchanged, such as a change in voice or sound. For example, when the title is Movie Title, playback processing based on PlayList is executed.

  Title_access_type [title_id] is information indicating whether or not the title specified by the title_id is permitted to be reproduced using Title_Search.

  reserved_for_future_use is a 29-bit field whose data description content is not set in preparation for future expansion.

  Title_mobj_id_ref [title_id] is a field for specifying the value of mobj_id of the NavigationObject in which the title specified by the title_id is entered.

  FIG. 11 shows the data structure of the PlayList file. The PlayList file is a data file having an extension “.mpls” stored in the PLAYLIST directory in a recording medium attached to the playback device or a storage unit inside the playback device (for example, a local storage including a hard disk). It is.

  type_indicator describes information indicating the type of the file. That is, in this field, information indicating that this file type is PlayList (MoviePlayList) that is playback management information for managing video playback.

  version_number indicates four character characters indicating the version number of the xxxx.mpls (MoviePlayList).

  PlayList_start_address indicates the start address of PlayList () with the relative number of bytes from the start byte of the PlayList file as a unit.

  PlayListMark_start_address indicates the head address of PlayListMark () with the relative number of bytes from the head byte of the PlayList file as a unit.

  ExtensionData_start_address indicates the start address of ExtensionData () in units of the relative number of bytes from the start byte of the PlayList file.

  AppInfoPlayList () stores parameters related to playback control of the PlayList, such as playback restrictions. Details of AppInfoPlayList () will be described later with reference to FIG.

  PlayList () stores parameters related to the main path and sub path of the PlayList. Details of PlayList () will be described later with reference to FIG.

  PlayListMark () stores PlayList mark information, that is, information about a mark that is a jump destination (jump point) in a user operation or command that commands chapter jump or the like.

  Private data can be inserted into ExtensionData ().

  Next, the syntax of AppInfoPlayList () will be described with reference to FIG.

  The length indicates a syntax length equal to or less than length, that is, the number of bytes of AppInfoPlayList () from immediately after this length field to the end of reserved_for_future_use. After this length, 8-bit reserved_for_future_use is prepared.

  PlayList_playback_type is a field that describes information indicating whether the PlayList is to be played back by continuous playback, random access, or shuffle playback.

  Playback_count is a field that describes information indicating the number of PlayItems used for playback in the case of a PlayList that performs random access and shuffle.

  UO_mask_table () is information that determines restrictions on user operations related to special playback such as pause, chapter search, jump, fast forward, fast reverse (backward fast playback), or display.

  PlayList_random_access_flag is flag information for controlling jump playback from another PlayList. Here, when PlayList_random_access_flag = 1 is set, jump playback from another PlayList is prohibited.

  For example, in a playback device, for example, a user operation that commands a jump of a playback position from the playback position referenced by the current PlayList to the beginning or chapter position of a predetermined PlayItem referenced by another PlayList, etc. Device-specific and extended user operations can be prepared. PlayList_random_access_flag is, for example, when a jump from the playback position of a Clip AV Stream referenced by another PlayList to a Clip AV Stream referenced by this PlayList (movement of the playback position) is commanded by these user operations This is flag information for setting whether or not to restrict this user operation.

  Note that when a jump (movement of the playback position) is commanded by a navigation command rather than a jump (movement of the playback position) commanded by a user operation, the PlayList_random_access_flag is ignored (the command is executed and the command is generated) Accordingly, the playback position is moved).

  is_MixApp is a flag indicating whether or not the stream reproduced by the PlayList file synthesizes sound and sound effects. That is, this is_MixApp is defined as a flag indicating whether or not the data referred to by this PlayList includes sound data or secondary audio stream that can be combined with the primary audio stream. Only by confirming this one flag, it is possible to quickly recognize whether or not audio data is synthesized.

  Lossless_may_bypass_mixer_flag is a flag relating to the reproduction of lossless audio. After this, 13-bit reserved_for_future_use is prepared.

  Next, FIG. 13 is a diagram illustrating the syntax of PlayList ().

  length is a 32-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of PlayList (). That is, this field indicates the number of bytes from reserved_for_future_use to the end of the PlayList. After this length, 16-bit reserved_for_future_use is prepared. In reserved_for_future_use, the description content of the data is not set in preparation for future expansion. number_of_PlayItems is a 16-bit field indicating the number of PlayItems in the PlayList. For example, in the example of FIG. 5, the number of PlayItems is three. The value of PlayItem_id is assigned from 0 in the order in which PlayItem () appears in the PlayList. For example, as shown in FIG. 5, PlayItem_id = 0, 1, and 2 are allocated.

  number_of_SubPaths is a 16-bit field indicating the number of SubPaths (number of entries) in the PlayList. For example, in the example of FIG. 5, the number of Sub Paths is three. The value of SubPath_id is assigned from 0 in the order in which SubPath () appears in the PlayList. For example, as shown in FIG. 5, Subpath_id = 0, 1, and 2 are allocated. In the subsequent for statement, PlayItems are referenced by the number of PlayItems, and Sub Paths are referenced by the number of Sub Paths.

  Next, the syntax of PlayItem () will be described with reference to FIG.

  length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of PlayItem (). Clip_Information_file_name [0] is a field for designating a Clip referred to by PlayItem. In the example of FIG. 14, the main Clip AV stream file is referred to by Clip_Information_file_name [0]. In PlayItem (), Clip_codec_identifier [0] that specifies the codec format of Clip, reserved_for_future_use in which the data description content is not set in preparation for future expansion, whether multi-angle playback is supported Is_multi_angle indicating the connection state, connection_condition indicating the connection state, and ref_to_STC_id [0] indicating information regarding the STC discontinuity (system time base discontinuity). Further, PlayItem () includes IN_time and OUT_time for designating the playback section of the PlayItem in the Clip. In the example of FIG. 14, the playback range of the main ClipAV stream file is represented by IN_time and OUT_time.

  In addition, PlayItem () includes UO_mask_table () for restricting user input and PlayItem_random_access_flag for controlling jump playback from other PlayItems.

  In UO_mask_table () of PlayItem () shown in FIG. 14, whether or not to allow various user operations that are input when Clip AV Stream is played with reference to this PlayItem is set. Has been.

  Then, PlayItem_random_access_flag of PlayItem () shown in FIG. 14 is a jump (playback position) from a playback position of a Clip AV Stream referred to by another PlayItem to a Clip AV Stream referred to by this PlayItem. Flag information for setting whether or not to restrict this user operation. Here, when PlayItem_random_access_flag = 1 is set, jump playback from other PlayItems is prohibited. A detailed description of this flag will be described later.

  Note that if a jump (movement of the playback position) is commanded by a navigation command rather than a jump (movement of the playback position) commanded by a user operation, PlayItem_random_access_flag is ignored (the command is executed and the command is generated) Accordingly, the playback position is moved).

  Further, PlayItem () includes still_mode indicating whether or not to display the last displayed video as a still image. Note that the case where there are a plurality of is_multi_angles (is_multi_angle == 1) is not directly related to the present invention, and the description thereof will be omitted.

  When the target PlayItem and one or more SubPaths to be played back are prepared, STN_table () selects, for example, audio switching, subtitle switching, or display / non-display of the sub-screen. A mechanism that allows a stream to be played to be selected from a clip that is referenced by the PlayItem and a clip that is referenced by one or more of these SubPaths, depending on the operation input of the commanding user or the generation of a command by an application. It is information for providing.

  For example, when PlayItem () and one or more SubPaths to be played back are prepared, a video stream included in a Clip referred to by PlayItem configuring Main Path is represented as a primary video stream (Primary video stream). The video stream included in the Clip referred to by the SubPlayItem constituting the SubPath is used as a secondary video stream (secondary video stream), or two of the video streams included in the Clip (Main Clip) referenced by the PlayItem. A stream is provided as a primary video stream and a secondary video stream, respectively, and these are mixed to provide a mechanism that can be played back on the full screen (primary video stream) and the sub-screen (secondary video stream). Such a technique is called a picture-in-picture technique.

  Similarly, for example, when PlayItem () and one or more SubPaths to be played back are prepared, the audio stream included in the Clip referred to by the PlayItem that constitutes the Main Path is the primary audio. Stream (Primary audio stream) and the audio stream included in the Clip referenced by the SubPlayItem constituting the SubPath is used as the secondary audio stream (Secondary Audio Stream), or the audio stream included in the Clip (Main Clip) referenced by the PlayItem Two of these audio streams are provided as a primary audio stream and a secondary audio stream, respectively, and a mechanism that can mix and reproduce these is provided.

  Then, the following first to fourth types of streams are prepared as streams that can be reproduced simultaneously with the primary video stream.

  The first type of stream is an audio stream defined by Primary_audio_stream_id in STN_table, that is, the above-described primary audio stream.

  The second type of stream is a video stream defined by Secondary_video_stream_id in STN_table, that is, the above-described secondary video stream.

  The third type of stream is an audio stream defined by Secondary_audio_stream_id in STN_table, that is, the above-described secondary audio stream.

  The fourth type of stream is a text subtitle stream (txtST) referred to in a sub path, a stream (PG, Presentation Graphics stream) obtained by run-length encoding a bitmap subtitle such as a DVD sub-picture, and the like. In STN_table, it is defined by PG_txtST_stream_id and IG_stream_id.

  STN_table () is information defining a combination of these streams.

  Next, the syntax of STN_table () will be described using FIG. 15 and FIG.

  This STN_table () is used to define a combination of a secondary video stream, a primary audio stream, a secondary audio stream, and a subtitle stream that can be simultaneously played in combination with the primary video stream. , Combinations of secondary video streams and secondary audio streams, and combinations of secondary video streams and presentation graphics (PG) or text subtitles for picture-in-picture applications.

  length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of STN_table (). 16-bit reserved_for_future_use is prepared after the length.

  number_of_primary_video_stream_entries indicates the number of streams to which primary_video_stream_id entered (registered) in STN_table () is given. primary_video_stream_id is information for identifying a video stream, and primary_video_stream_number is a video stream number that is used for video switching and is visible to the user.

  number_of_primary_audio_stream_entries indicates the number of streams to which primary_audio_stream_id entered in STN_table () is given. primary_audio_stream_id is information for identifying an audio stream, and primary_audio_stream_number is an audio stream number that can be seen by the user used for audio switching.

  number_of_PG_txtST_stream_entries indicates the number of streams to which PG_txtST_stream_id entered in STN_table () is given. In this, a stream (PG, Presentation Graphics stream) obtained by run-length encoding bitmap subtitles such as a DVD sub-picture and a text subtitle file (txtST) are entered. PG_txtST_stream_id is information for identifying a subtitle stream, and PG_txtST_stream_number is a subtitle stream number (text subtitle stream number) that can be seen by the user used for subtitle switching.

  number_of_IG_stream_entries indicates the number of streams provided with IG_stream_id entered in STN_table (). In this, an interactive graphics stream is entered. IG_stream_id is information for identifying an interactive graphics stream, and IG_stream_number is a graphics stream number that is visible to the user used for graphics switching.

  number_of_secondary_audio_stream_entries indicates the number of streams to which secondary_audio_stream_id entered (registered) in STN_table () is given. secondary_audio_stream_id is information for identifying an audio stream, and secondary_audio_stream_number is an audio stream number that is used for audio switching and is visible to the user.

  number_of_secondary_video_stream_entries indicates the number of streams to which secondary_video_stream_id entered (registered) in STN_table () is given. secondary_video_stream_id is information for identifying a video stream, and secondary_video_stream_number is a video stream number that is used for video switching and is visible to the user.

  And number_of_PiP_PG_textST_stream_entries_plus indicates the number of picture-in-picture subtitle streams (PiP_PG_textST_stream) entered (registered) in STN_table ().

  In the for loop of the primary video stream ID (primary_video_stream_id), primary_video_stream_id is given from 0 to one video elementary stream that is identified for each stream_entry () in order. Note that the primary video stream number (primary_video_stream_number) may be used instead of the primary video stream ID (primary_video_stream_id). In this case, primary_video_stream_number is given from 1 instead of 0. That is, primary_video_stream_number is obtained by adding 1 to the value of primary_video_stream_id. The video stream number is defined from 1 since it is a video stream number that is used for video switching and is visible to the user.

  The stream_entry () in each for loop will be described later with reference to FIG.

  Also, stream_attribute () in each for loop gives stream attribute information of one elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one elementary stream specified for each stream_entry () is described. The stream_attribute () will be described later with reference to FIG.

  Similarly, in the for loop of the primary audio stream ID (primary_audio_stream_id), primary_audio_stream_id is given from 0 to one audio elementary stream specified for each stream_entry () in order. As in the case of the video stream, the primary audio stream number (primary_audio_stream_number) may be used instead of the primary audio stream ID (primary_audio_stream_id). In this case, primary_audio_stream_number is given from 1 instead of 0. That is, primary_audio_stream_number is obtained by adding 1 to the value of primary_audio_stream_id. The audio stream number is an audio stream number that can be seen by the user and is used for audio switching.

  Similarly, PG_txtST_stream_id is assigned from 0 to one bitmap subtitle elementary stream or text subtitle identified in order for each stream_entry () in the for loop of the subtitle stream ID (PG_txtST_stream_id). Similarly, consecutive IDs are assigned to PiP_PG_txtST_stream. As in the case of the video stream, a subtitle stream number (PG_txtST_stream_number) may be used instead of the subtitle stream ID (PG_txtST_stream_id). In this case, PG_txtST_stream_number is given from 1 instead of 0. That is, PG_txtST_stream_number is obtained by adding 1 to the value of PG_txtST_stream_id. The subtitle stream number is a subtitle stream number (text subtitle stream number) that can be seen by the user used for subtitle switching, and is defined from 1.

  Similarly, in the for loop of the graphics stream ID (IG_stream_id), IG_stream_id is assigned from 0 to one interactive graphics elementary stream that is identified for each stream_entry () in order. As in the case of the video stream, a graphics stream number (IG_stream_number) may be used instead of the graphics stream ID (IG_stream_id). In this case, IG_stream_number is given from 1 instead of 0. That is, IG_stream_number is obtained by adding 1 to the value of IG_stream_id. Since the graphics stream number is a graphics stream number that can be seen by the user used for switching graphics, it is defined from 1.

  Similarly, in the for loop of the secondary audio stream ID (secondary_audio_stream_id), secondary_audio_stream_id is assigned from 0 to one audio elementary stream that is specified for each stream_entry () in order. As in the case of the video stream, a secondary audio stream number (secondary_audio_stream_number) may be used instead of the secondary audio stream ID (secondary_audio_stream_id). In this case, secondary_audio_stream_number is given from 1 instead of 0. That is, secondary_audio_stream_number is obtained by adding 1 to the value of secondary_audio_stream_id. The secondary audio stream number is a secondary audio stream number that is used for audio switching and is visible to the user, and is defined from 1.

  A block of comb_info_Secondary_audio_Primary_audio () indicates a combination of secondary audio and primary audio.

  In the comb_info_Secondary_audio_Primary_audio () block, the 8-bit field of number_of_primary_audio_stream_ref_entries indicates the number of primary audio streams that can be combined with the secondary audio stream. The 8-bit field of primary_audio_stream_id_ref indicates the ID of the primary audio stream that can be combined with the secondary audio stream.

  Similarly, in the for loop of the secondary video stream ID (secondary_video_stream_id), one audio elementary stream specified for each stream_entry () in order is given secondary_video_stream_id from 0. As in the case of the video stream, a secondary video stream number (secondary_video_stream_number) may be used instead of the secondary video stream ID (secondary_video_stream_id). In this case, secondary_video_stream_number is given from 1 instead of 0. That is, secondary_video_stream_number is obtained by adding 1 to the value of secondary_video_stream_id. The secondary video stream number is a secondary video stream number that is used for audio switching and is visible to the user.

  Next, a block of comb_info_Secondary_video_Secondary_audio () indicates a combination of a secondary video stream and a secondary audio stream.

  In the comb_info_Secondary_video_Secondary_audio () block, an 8-bit field of number_of_Secondary_audio_stream_ref_entries indicates the number of secondary audio streams that can be combined with the secondary video stream. The 8-bit field of Secondary_audio_stream_id_ref indicates the ID of a secondary audio stream that can be combined with the secondary video stream.

  A block of comb_info_Secondary_video_PiP_PG_textST () indicates a combination of a secondary video stream and a PG / text subtitle stream for a picture-in-picture application.

  In the comb_info_Secondary_video_PiP_PG_textST () block, the 8-bit field of number_of_PiP_PG_textST_ref_entries indicates the number of PG / text subtitle streams for the PIP application that can be combined with the secondary video stream. Also, the 8-bit field of PiP_PG_textST_stream_id_ref indicates the ID of the PG / text subtitle stream for the picture-in-picture application that can be combined with the secondary video stream.

  By using such STN_table, in addition to a combination of a secondary video stream, a primary audio stream, a secondary audio stream, and a subtitle stream (PG TextST stream) that can be played back simultaneously with the primary video stream, for a picture-in-picture application It is possible to prevent a conflicting combination from being selected as a combination of the PG / text subtitle stream (PiP PG TextST stream).

  FIG. 17 is a diagram illustrating the syntax of Stream_entry ().

  In type = 1, a 16-bit packet ID (PID) is used to identify one elementary stream from among a plurality of elementary streams multiplexed in a Clip (Main Clip) referenced by PlayItem. Is specified. ref_to_stream_PID_of_mainClip indicates this PID. That is, with type = 1, the stream is determined simply by specifying the PID in the main ClipAV stream file.

  In type = 2, when SubPath refers to multiple Clips at a time and each Clip multiplexes multiple elementary streams, multiple elementary streams of one Clip (SubClip) referenced by SubPath SubPath_id, Clip id, and packet ID (PID) of the SubPath are specified in order to identify one elementary stream from the list. ref_to_SubPath_id indicates this SubPath_id, ref_to_SubClip_entry_id indicates this Clip id, and ref_to_stream_PID_of_SubClip indicates this PID. This is used when a plurality of Clips are referred to in SubPlayItem and a plurality of elementary streams are referred to this Clip.

  As described above, when a type (two types of type 1 and type 2) is used, when a PlayItem and one or more SubPaths to be played back are prepared, one Clip and a reference to this PlayItem are provided. One elementary stream can be identified from the Clip referred to by the above SubPath. Note that type = 1 indicates a clip (main clip) referred to by the main path, and type = 2 indicates a clip (sub clip) referred to by the sub path.

  Type = 3 is for identifying the elementary stream included in the MainClip and referenced by the SubPath in the corresponding PlayItem.

  That is, when type = 3, SubPath refers to the same Clip as MainPath, in other words, a plurality of elementary streams are multiplexed on MainClip, and both MainPath and SubPath are multiplexed on MainClip. When referring to any of the streams, the Ref_to_stream_PID_of_MainClip is used to specify one elementary stream referenced by the SubPath from among the multiple elementary streams of the MainClip. (PID) is specified. Also, SubPath_id is indicated by ref_to_SubPath_id, the SubPath defined in PlayList () in FIG. 9 is specified by this SubPath_id, and the corresponding SubPlayItem (FIG. 28) is called from this Subpath, so that the elementary stream playback time ( IN_time, OUT_time) etc. can be grasped.

  It should be noted that type = 3, that is, a plurality of elementary streams are multiplexed on the MainClip, and both the MainPath and SubPath refer to any of the elementary streams multiplexed on the MainClip. This is a case where SubPath_type, which will be described later with reference to No. 27, is SubPath_type = 7.

  As described above, when a type (three types of type 1 to type 3) is used and a PlayItem and one or more SubPaths to be played back are prepared, a clip referred to by the PlayItem is further Even if it is referenced from SubPath, one elementary stream can be specified from the Clip.

  Next, the syntax of stream_attribute () will be described with reference to FIG.

  The length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of stream_attribute ().

  The stream_coding_type indicates the encoding type of the elementary stream as shown in FIG. As encoding types of elementary streams, MPEG-2 video stream, HDMV LPCM audio, Dolby AC-3 audio, dts audio, Presentation graphics stream, Interactive graphics stream, and Text subtitle stream are described. Note that this stream_coding_type may be extended so that a video stream compressed by another codec such as MPEG-4 AVC or VC-1 can be handled.

  video_format indicates the video format of the video elementary stream as shown in FIG. 480i, 576i, 480p, 1080i, 720p, and 1080p are described as the video format of the video elementary stream.

  frame_rate indicates the frame rate of the video elementary stream, as shown in FIG. As frame rates of the video elementary stream, 24000/1001, 24, 25, 30000/1001, 50, and 60000/1001 are described.

  The aspect_ratio indicates the aspect ratio information of the video elementary stream as shown in FIG. As aspect ratio information of the video elementary stream, 4: 3 display aspect ratio and 16: 9 display aspect ratio are described.

  audio_presentation_type indicates presentation type information of an audio elementary stream as shown in FIG. As the presentation type information of the audio elementary stream, single mono channel, dual mono channel, stereo (2-channel), and multi-channel are described.

  Sampling_frequency indicates the sampling frequency of the audio elementary stream as shown in FIG. As sampling frequencies of audio elementary streams, 48 kHz and 96 kHz are described.

  audio_language_code indicates the language code of the audio elementary stream (Japanese, Korean, Chinese, etc.).

  PG_language_code indicates the language code (Japanese, Korean, Chinese, etc.) of the bitmap subtitle elementary stream.

  IG_language_code, which indicates the language code (Japanese, Korean, Chinese, etc.) of the interactive graphics elementary stream.

  textST_language_code indicates the language code (Japanese, Korean, Chinese, etc.) of the text subtitle elementary stream.

  Character_code indicates the character code of the text subtitle elementary stream as shown in FIG. The character code of the text subtitle elementary stream includes Unicode V1.1 (ISO 10646-1), Shift JIS (Japanese), KSC 5601-1987 including KSC 5653 for Roman character (Korean), GB 18030-2000 (Chinese), GB2312 (Chinese) and BIG5 (Chinese) are described.

  A specific example of the syntax of stream_attribute () in FIG. 18 will be described below with reference to FIGS. 18 and 19 to 25.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 18) is an MPEG-2 video stream (FIG. 19), the stream_attribute () includes the video format (FIG. 20) and frame rate (FIG. 21) of the elementary stream. ), And aspect ratio information (FIG. 22).

  When the encoding type of the elementary stream (stream_coding_type in FIG. 18) is HDMV LPCM audio, Dolby AC-3 audio, or dts audio (FIG. 19), the stream_attribute () includes the presentation type information of the audio elementary stream. (FIG. 23), sampling frequency (FIG. 24), and language code are included.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 18) is a Presentation graphics stream (FIG. 19), stream_attribute () includes the language code of the bitmap subtitle elementary stream.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 18) is Interactive graphics stream (FIG. 19), stream_attribute () includes the language code of the interactive graphics elementary stream.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 18) is a Text subtitle stream (FIG. 19), stream_attribute () includes the character code (FIG. 25) and language code of the text subtitle elementary stream. .

  Note that the attribute information is not limited to this.

  As described above, when a PlayItem and one or more SubPaths to be played back are prepared, a stream_entry () is used to select a Clip referred to by the PlayItem and a Clip referred to by one or more SubPaths. The attribute information of the identified elementary stream (one if type = 1 and type = 2, two if type = 3) can be known by stream_attribute ().

  By examining this attribute information (stream_attribute ()), the playback device can check whether or not it has the function of playing back the elementary stream. Further, the playback device can select the elementary stream corresponding to the initial information of the language setting of the playback device by examining the attribute information.

  For example, it is assumed that the playback apparatus has only a playback function for a bitmap subtitle elementary stream and does not have a playback function for a text subtitle elementary stream. When the user instructs the playback apparatus to switch the language, the playback apparatus sequentially selects and reproduces only the bitmap subtitle elementary stream from the for loop of the subtitle stream ID (PG_txtST_stream_id).

  Also, for example, assume that the initial information of the language setting of the playback device is Japanese. When the user instructs the playback device to switch to the secondary audio (ie, secondary audio) or to mix and play the main audio and the secondary audio, the playback device uses the audio stream ID ( Only audio elementary streams whose language code is Japanese are sequentially selected from the for loop of (Audio stream id) and played.

  Further, for example, when an AV stream (movie) composed of a video stream and an audio stream, which is referred to by the main path, is reproduced, the user instructs the playback apparatus to switch the audio, and the audio stream which is referred to by the sub path. When mixing playback with (comment by a director or a performer) is instructed, the playback device further mixes (superimposes) the 2nd audio stream referenced by the sub path on the 1st audio stream referenced by the main path. Play with the video stream.

  Also, for example, when the user commands switching of audio and mixing playback of two audio streams referenced by two subpaths is commanded to the playback device (the syntaxes of FIGS. 16 and 17 are used, FIG. When the reproduction of the audio stream corresponding to the audio stream ID in which the 17 type is both 2 is instructed), the reproducing device mixes (superimposes) the two audio streams respectively referred to by the two sub paths. Play with the video stream.

  In this way, by providing STN_table () in PlayItem (), when PlayItem and one or more SubPaths to be played back are prepared, operations such as audio switching and subtitle switching by the user Provides a mechanism for selecting a stream to be played back from the Clip referenced by this PlayItem and the Clip referenced by one or more SubPaths, so that a stream or data file different from the AV stream to be played back can be selected. Even interactive operations can be performed.

  FIG. 26 is a diagram illustrating the syntax of SubPath ().

  length is a 32-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of Sub Path (). That is, this field indicates the number of bytes from reserved_for_future_use to the end of SubPath (). After this length, 8-bit reserved_for_future_use is prepared. SubPath_type is an 8-bit field indicating the application type of SubPath. SubPath_type is used, for example, when the type indicates whether the Sub Path is audio, bitmap subtitle, text subtitle, or the like. An example of this SubPath_type will be described later with reference to FIG.

  15-bit reserved_for_future_use is prepared after SubPath_type. is_repeat_SubPath is a 1-bit field that designates the SubPath playback method, and indicates whether the SubPath playback is repeated during the main path playback or the SubPath playback is performed only once. For example, when the playback timing of the main AV stream and the stream included in the clip specified by the sub path are different (when the main path is a still image slide show and the sub path audio path is used as the BGM (background music) of the main path ).

  8-bit reserved_for_future_use is prepared after Is_repeat_SubPath. number_of_SubPlayItems is an 8-bit field indicating the number of SubPlayItems (number of entries) in one SubPath. For example, number_of_SubPlayItems has one SubPlayItem with SubPath_id = 0 in FIG. 6 and two SubPlayItems with SubPath_id = 1. In subsequent for statements, SubPlayItems are referenced by the number of SubPlayItems.

  FIG. 27 is a diagram illustrating an example of SubPath_type (subpath type).

  Note that the description “Out-of-mux” is a subpath type (kind) when a TS including an ES referred to in a subpath is different from a TS including an ES referred to in a main path. That is, the ES referred to in the sub path indicates that the sub path type is not multiplexed in the TS including the play item referred to in the main path. Hereinafter, this type is referred to as a main path TS non-multiplex type path.

  The description “In-mux” is a subpath type (kind) in the case where a TS including an ES referred to in a subpath and a TS including one or more ES referred to in a main path are the same. That is, it is indicated that the ES referred to in the subpath is the type (type) of the subpath multiplexed in the TS including the ES referred to in the main path. In other words, the main TS included in the main clip includes the main Clip AV stream and the sub Clip AV stream. Hereinafter, this type is referred to as a main path TS multiplexing type path.

  In addition, when the ES referred to in the main path and the ES referred to in the sub path are asynchronous, the type (kind) of the sub path is hereinafter referred to as an asynchronous path. On the other hand, when the ES referenced in the main path and the ES referenced in the sub path are synchronized, the type (kind) of the sub path is hereinafter referred to as a synchronous path.

  SubPath_type = 0 and 1 are reserved.

  SubPath_type = 2 is an Audio presentation path of the Browsable slideshow. For example, SubPath_type = 2 is a main path TS non-multiplexed type path, and in the playlist, the audio presentation path referenced by the sub path and the main path referenced by the play item are asynchronous (asynchronous type). Pass).

  SubPath_type = 3 is an interactive graphics presentation menu (interactive graphics presentation menu). For example, SubPath_type = 3 is a main path TS non-multiplex type path, and an interactive graphics menu referenced by a sub path and a main path referenced by a play item are asynchronous (asynchronous type) in a playlist. This is a path of

  SubPath_type = 4 is a Text subtitle presentation path (a text subtitle presentation path). For example, SubPath_type = 4 is a main path TS non-multiplex type path, and in the playlist, the text subtitle presentation path referred to by the sub path and the main path referred to by the play item are synchronized. (This is a synchronous path).

  SubPath_type = 5 is Out-of-mux and AV Synchronized type of one or more elementary streams path (Primary audio / PG / IG / Secondary audio path) .Out of mux and AV synchronized type of Picture-in-Picture presentation path which Contains one or more elementary streams paths. In other words, SubPath_type = 5 is a main path TS non-multiplexed and synchronous path, and includes one or more ES (Primary audio / PG / IG / Secondary audio) paths, picture-in-picture presentation paths, and so on. Has been.

  Here, the picture-in-picture presentation path refers to a primary audio stream, a secondary video stream, and a secondary that are played back corresponding to a predetermined primary video stream (a video stream referred to in the main path) in the picture-in-picture method. It means one or more paths (types of such sub paths) of the audio stream and the subtitle stream.

  SubPath_type = 6 is Out-of-mux and AV non-Synchronized type of Picture-in-Picture presentation path which contains one or more elementary streams paths. That is, SubPath_type = 6 is a main path TS non-multiplexed and asynchronous path, and is a picture-in-picture presentation path (one or more ES paths).

  SubPath_type = 7 is In-mux type and AV Synchronized type of Picture-in-Picture presentation path which contains one or more elementary streams paths.

  That is, SubPath_type = 7 is a main path TS multiplexing type and synchronous type path, which is a picture-in-picture presentation path (one or more ES paths).

  SubPath_type = 8 to 255 is reserved.

  FIG. 28 is a diagram illustrating the syntax of SubPlayItem (i).

  The length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of Sub playItem ().

  In FIG. 28, there are a case where SubPlayItem refers to one Clip and a case where it refers to a plurality of Clips.

  First, a case where SubPlayItem refers to one Clip will be described.

  SubPlayItem includes Clip_Information_file_name [0] for specifying Clip. Also, Clip_codec_identifier [0] that specifies the codec system of Clip, reserved_for_future_use, is_multi_Clip_entries that indicates the presence / absence of multi-clip registration, and ref_to_STC_id [0] that is information about STC discontinuity points (system time base discontinuity points) including. When the is_multi_Clip_entries flag is set, the syntax when SubPlayItem refers to a plurality of Clips is referenced. In addition, SubPlayItem_IN_time and SubPlayItem_OUT_time for specifying the playback section of the Sub Path in the Clip are included. Furthermore, sync_PlayItem_id and sync_start_PTS_of_PlayItem are included to specify the time when the Sub Path starts to be played on the time axis of the main path.

  As described above, the sync_PlayItem_id and sync_start_PTS_of_PlayItem are used in the case of FIGS. 6 and 7 (when the playback timing of the stream included in the file indicated by the main AV stream and the sub path is the same), and the main AV stream and the sub path are used. When the playback timings of the streams included in the file indicated by are different (for example, when the still image referenced by the main path and the audio referenced by the sub path are not synchronized, such as BGM of a slide show composed of still images) ) Is not used. Also, SubPlayItem_IN_time, SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem are used in common in the Clip referred to by SubPlayItem.

  Next, a case where SubPlayItem refers to a plurality of Clips (if (is_multi_Clip_entries == 1b), that is, multi-clip registration is performed) will be described. Specifically, as shown in FIG. 7, a case where SubPlayItem refers to a plurality of Clips is shown.

  num_of_Clip_entries indicates the number of Clips, and Clip_Information_file_name [SubClip_entry_id] specifies Clips excluding Clip_Information_file_name [0]. That is, Clips such as Clip_Information_file_name [1] and Clip_Information_file_name [2] are specified excluding Clip_Information_file_name [0]. Further, SubPlayItem includes Clip_codec_identifier [SubClip_entry_id] that specifies the codec system of Clip, ref_to_STC_id [SubClip_entry_id] that is information on STC discontinuity points (system time base discontinuity points), and reserved_for_future_use.

  Note that SubPlayItem_IN_time, SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem are used in common among a plurality of clips. In the example of FIG. 7, SubPlayItem_IN_time, SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem are used in common between SubClip_entry_id = 0 and SubClip_entry_id = 1, and the Text based subtitle for the selected SubClip_entry_id is timeSubPlayItem_id Playback is based on SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem.

  Here, the value of SubClip_entry_id is assigned from 1 in the order in which Clip_Information_file_name [SubClip_entry_id] in SubPlayItem appears. Also, SubClip_entry_id of Clip_Information_file_name [0] is 0.

  Next, FIG. 29 is a block diagram showing the configuration of the playback apparatus 1 that plays back the data having the data structure described with reference to FIGS. 8 to 28, that is, the PlayList having the main path and the sub path described above.

  The controller 21 executes a control program prepared in advance or controls the storage drive 22 so as to be recorded on the recording medium 11 such as an optical disk or recorded on the local storage 24. By reading out the navigation program, developing it in the memory 23 and executing it, the overall operation of the playback device 1 is controlled. For example, the controller 21 can display a predetermined menu screen (GUI display screen) on an external display device when the recording medium 11 is loaded or when a predetermined operation input is received.

  That is, the controller 21 displays the menu screen (GUI display screen) recorded together with the content (contents configured by PlayList and AV stream) reproduced on the recording medium 11 such as an optical disk or the local storage 24. In addition to the control, it is also possible to generate data corresponding to the GUI display screen based on the content data reproduced on the recording medium 11 or the local storage 24 and display it on an external display device. . The details of the process in which the controller 21 generates data corresponding to the GUI display screen, controls the OSD 30, and displays the generated GUI display screen in a superimposed manner will be described later.

  The storage drive 22 reads data from the recording medium 11 according to control by the controller 21, and outputs the read data to the controller 21, the memory 23, or the AV decoder unit 26. When the information read from the recording medium 11 is a navigation program or a PlayList, the information read by the storage drive 22 is output to the controller 21 or the memory 23. When the information read from the recording medium 11 is an AV stream or text data, the information read by the storage drive 22 is output to the AV decoder unit 26.

  The memory 23 appropriately stores data necessary for the controller 21 to execute various processes. Furthermore, various registers in which information necessary for the reproduction process of the reproduction apparatus 1 is set are provided in a part of the storage area of the memory 23. The local storage 24 is composed of, for example, an HDD (Hard Disk Drive).

  The Internet interface 25 is connected to the network 2 by wire or wirelessly, communicates with the server 3 via the network 2 according to control from the controller 21, and downloads data downloaded from the server 3 to the local storage. 24. From the server 3, for example, data for updating the data described with reference to FIG. 4 recorded in the recording medium 11 mounted on the playback device 1 at that time is downloaded as content. The local storage 24 can record content downloaded from the server 3 via the network 2.

  The AV decoder unit 26 decodes the AV stream or text data supplied from the storage drive 22 or the local storage 24 and supplies the obtained video signal to an OSD (On Screen Display) 30 and also an audio signal. Is output to an external display device.

  Based on the control of the controller 21, the OSD 30 outputs the supplied video signal to an external display device or superimposes data of a predetermined menu screen (GUI display screen) on the supplied video signal, The generated image signal is output to an external display device.

  In the display device, based on the signal decoded by the AV decoder unit 26, for example, the content recorded on the recording medium 11 (video display, audio output) is output.

  The operation input unit 29 includes, for example, an input device such as a button, a key, a touch panel, a jog dial, and a mouse, and a receiving unit that receives a signal such as an infrared ray transmitted from a predetermined remote commander, and acquires a user operation input. To the controller 21.

  In addition, a drive 27 is connected to the controller 21 as necessary. For example, a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disk-Read Only Memory), DVD) A removable medium 28 including a magneto-optical disk (including MD (registered trademark) (Mini-Disk)), a semiconductor memory, or the like.

  First, the controller 21 reads out the PlayList file from the recording medium 11 such as HDD, Blu-ray Disc (trademark), DVD or the like via the storage drive 22 or from the local storage 24, and based on the information of the PlayList file. If necessary, data corresponding to the GUI display screen is generated, and from the recording medium 11 such as an HDD, Blu-ray Disc (trademark), or DVD or the local storage 24 via the storage drive 22. From this, the AV stream and AV data are read out. The user can use the operation input unit 29 to issue a command for switching audio, subtitles, or the like, a command for displaying a GUI display screen, or the like to the controller 21. Further, the controller 21 reads various setting information registered in the register of the memory 23, such as initial information of language setting of the playback device 1, for example.

  The PlayList file includes the above-described STN_table () in addition to the information on Main Path and Sub Path. The controller 21 receives the main Clip AV stream file referenced by the PlayItem included in the PlayList file, the sub Clip AV stream file referenced by the SubPlayItem, and the text subtitle data referenced by the SubPlayItem from the recording medium 11 via the storage drive 22. Alternatively, it is read from the local storage 24. Here, the main Clip AV stream file referred to by PlayItem and the sub Clip AV stream file referred to by SubPlayItem may be recorded on different recording media. For example, the main Clip AV stream file is recorded on the recording medium 11, and the corresponding sub Clip AV stream file is supplied via the network 2 (not shown) and stored in the local storage 24 such as an HDD. Also good. Conversely, the main Clip AV stream file may be stored in the local storage 24 such as an HDD, and the sub Clip AV stream file may be recorded on the recording medium 11.

  In addition, the controller 21 refers to various setting information registered in the register of the memory 23, selects the elementary stream corresponding to the playback function of itself (playback apparatus 1), and controls to play back, The elementary stream corresponding to the initial information of the language setting of the playback device 1 is selected and controlled to be played back.

  FIG. 30 is a block diagram illustrating a configuration example of the AV decoder unit 26 of the playback device 1 of FIG.

  The AV decoder unit 26 includes a switch 32, buffers 51 to 54, PID filter 55, PID filter 56, switches 57 to 59, PID filter 60, background decoder 71, 1st video decoder 72-1 and 2nd video decoder 72-2. Presentation graphics decoder 73, Interactive graphics decoder 74, 1st audio decoder 75-1, 2nd audio decoder 75-2, Text-ST composition 76, switch 77, preload buffer 81, background plane generator 91, 1st video Plane generator 92-1 and 2nd video plane generator 92-2, presentation graphics plane generator 93, interactive graphics plane generator 94, buffer 95, video data Management unit 96, mixing processing unit 97 and the mixing processor 101, is provided.

  Both the 1st video decoder 72-1 and the 2nd video decoder 72-2 decode the video stream. Basically, in STN_table (), the 1st video decoder 72-1 is for decoding the video stream given by primary_video_stream_id, and the 2nd video decoder 72-is for decoding the video stream given by Secondary_video_stream_id. 2. That is, the playback device 1 has two video decoders (1st video decoder 72-1 and 2nd video decoder 72-2) in order to decode two video streams. Also, the 1st video decoder 72-1 may be a video decoder that is used with priority over the 2nd video decoder 72-2. That is, when there is only one video stream to be decoded, the first video decoder is used regardless of whether the video stream is a video stream included in the main Clip AV stream file or a video stream included in the sub Clip AV stream file. It may be decoded at 72-1. In the following description, the 1st video decoder 72-1 and the 2nd video decoder 72-2 will be referred to as the video decoder 72 when not distinguished from each other.

  Both the 1st audio decoder 75-1 and the 2nd audio decoder 75-2 also decode the audio stream. Basically, in STN_table (), the one for decoding the audio stream given by primary_audio_stream_id is the 1st audio decoder 75-1, and the one for decoding the audio stream given by Secondary_audio_stream_id is the 2nd audio decoder 75-. 2. That is, the playback device 1 has two audio decoders (1st audio decoder 75-1 and 2nd audio decoder 75-2) in order to decode two audio streams. The 1st audio decoder 75-1 may be an audio decoder that is used preferentially over the 2nd audio decoder 75-2. That is, when there is only one audio stream to be decoded, the first audio decoder is used regardless of whether the audio stream is an audio stream included in the main Clip AV stream file or an audio stream included in the sub Clip AV stream file. It may be decoded in 75-1. In the following, the 1st audio decoder 75-1 and the 2nd audio decoder 75-2 are referred to as audio decoders 75 when not distinguished from each other.

  The file data read by the controller 21 is demodulated by a demodulation and ECC decoding unit (not shown), and error correction is performed on the demodulated multiplexed stream. The switch 32 selects the demodulated and error-corrected data for each stream type based on the control from the controller 21 and supplies the selected data to the corresponding buffers 51 to 54. Specifically, the switch 32 supplies background image data to the buffer 51, supplies data of the main Clip AV stream file to the buffer 52, and sub-clip data to the buffer 53 based on control from the controller 21. The switch 32 is switched so that the Text-ST data is supplied to the buffer 54.

  The buffer 51 buffers the background image data, the buffer 52 buffers the data of the main Clip AV stream file, the buffer 53 buffers the data of the sub Clip AV stream file, and the buffer 54 -Buffer ST data.

  The main Clip AV stream file is a stream (for example, a transport stream) obtained by multiplexing one or more streams in addition to video among video, audio, bitmap subtitle (Presentation Graphics stream), and interactive graphics. The sub-Clip AV stream file is a stream obtained by multiplexing one or more streams of video, audio, bitmap subtitle (Presentation Graphics stream), and interactive graphics. Note that the data of the text subtitle data file (Text-ST) may or may not be in the form of a multiplexed stream such as a transport stream.

  Further, when the main clip AV stream file, the sub clip AV stream file, and the text subtitle data are read from the storage drive 22 (recording medium 11) or the local storage 24, the respective files may be alternately read in a time division manner. Alternatively, before the main clip is read, the sub clip AV stream file and the text subtitle data may be read in advance and preloaded to the buffer (buffer 53 or buffer 54) in advance.

  Then, the stream data read from the buffer 52 which is the main Clip AV stream file read buffer is output to the PID (packet ID) filter 55 at the subsequent stage at a predetermined timing. The PID filter 55 distributes the input main Clip AV stream file to the decoders of the elementary streams in the subsequent stage according to the PID (packet ID) and outputs the result. That is, the PID filter 55 supplies the video stream to the PID filter 60 that is a supply source to the 1st video decoder 72-1 and the 2nd video decoder 72-2, and supplies the presentation graphics stream to the presentation graphics decoder 73. Is supplied to the switch 57, the interactive graphics stream is supplied to the switch 58 serving as the supply source to the interactive graphics decoder 74, and the audio stream is supplied to the first audio decoder 75-1 and the second audio decoder 75-2. Is supplied to the switch 59.

  The presentation graphics stream is, for example, bitmap subtitle data as described above, and the text subtitle data is, for example, text subtitle data as described above.

  The stream data read from the buffer 53, which is the sub-Clip AV stream file read buffer, is output to the PID (packet ID) filter 56 at the subsequent stage at a predetermined timing. The PID filter 56 distributes the input sub Clip AV stream file to the decoders of the elementary streams in the subsequent stage according to the PID (packet ID) and outputs the result. That is, the PID filter 56 supplies the video stream to the PID filter 60 that is a supply source to the 1st video decoder 72-1 and the 2nd video decoder 72-2, and supplies the presentation graphics stream to the presentation graphics decoder 73. Is supplied to the switch 57, the interactive graphics stream is supplied to the switch 58 serving as the supply source to the interactive graphics decoder 74, and the audio stream is supplied to the first audio decoder 75-1 and the second audio decoder 75-2. Is supplied to the switch 59.

  Also, the main Clip AV Stream video stream or the sub Clip AV Stream video stream is input to the PID filter 60 via the PID filter 55 or the PID filter 56. Therefore, the PID filter 60 distributes the input video stream to the subsequent 1st video decoder 72-1 or 2nd video decoder 72-2 according to the PID (packet ID) and outputs the result. That is, when a primary video stream is input, the PID filter 60 supplies the primary video stream to the 1st video decoder 72-1. On the other hand, when a secondary video stream is input, the PID filter 60 supplies the secondary video stream to the 2nd video decoder 72-2.

  Here, only the main clip AV stream video stream is used as the primary video stream, and the sub clip AV stream video stream is not used. However, the sub-Clip AV Stream video stream supplied from the PID filter 56 can also be supplied to the 1st video decoder 72 in consideration that the sub-Clip AV Stream video stream may be used as the primary video stream in the future. The PID filter 60 is configured. That is, only the video stream included in the sub clip specified by Subpath is played, the main clip is not used for playing the video stream, and the audio stream is played using the sub clip or the main clip.

  Data read from the buffer 51 that buffers background image data is supplied to the background decoder 71 at a predetermined timing. The background decoder 71 decodes the background image data and supplies the decoded background image data to the background plane generation unit 91.

  The 1st video decoder 72-1 decodes the video stream and outputs the decoded video data to the 1st video plane generating unit 92-1. The 2nd video decoder 72-2 decodes the video stream and outputs the decoded video data to the 2nd video plane generating unit 92-2.

  The switch 57 is one of the presentation graphics stream included in the main Clip AV stream file supplied from the PID filter 55 and the presentation graphics stream included in the sub Clip AV stream file supplied from the PID filter 56. The selected presentation graphics stream is supplied to the presentation graphics decoder 73 at the subsequent stage. The presentation graphics decoder 73 decodes the presentation graphics stream, and supplies the decoded presentation graphics stream data to the switch 77 serving as a supply source to the presentation graphics plane generation unit 93.

  The switch 58 also selects one of the interactive graphics stream included in the main Clip AV stream file supplied from the PID filter 55 and the interactive graphics stream included in the sub Clip AV stream file supplied from the PID filter 56. Or the selected interactive graphics stream is supplied to the subsequent interactive graphics stream decoder 74. The interactive graphics decoder 74 decodes the interactive graphics stream and supplies the decoded interactive graphics stream data to the preload buffer 81.

  The preload buffer 81 temporarily stores the interactive graphics stream supplied from the interactive graphics decoder 74. For example, in a case where there is a restriction that the playback device 1 can read only two TS (Transport streams) at a time, the playback device 1 obtains an interactive graphics stream in advance (preloading). In other words, when the video stream or the audio stream is played back, the pre-loaded interactive graphics stream can be played back simultaneously. In order to realize such a function, that is, to store the preloaded interactive graphics stream until the timing when the video stream or the audio stream is reproduced, the preloading buffer 81 is provided.

  The decoded interactive graphics stream stored in the preload buffer 81 is supplied to the interactive graphics plane generating unit 94 at a predetermined timing.

  Further, the switch 59 converts the audio stream included in the main Clip AV stream file supplied from the PID filter 55 and the audio stream included in the sub Clip AV stream file supplied from the PID filter 56 into the first audio decoder 75 in the subsequent stage. -1 or 2nd audio decoder 75-2. Specifically, the switch 59 switches the selection depending on the value of the type of stream_entry ().

  The 1st audio decoder 75-1 decodes the audio stream and supplies the decoded audio stream data to the mixing processing unit 101. The 2nd audio decoder 75-2 decodes the audio stream and supplies the decoded audio stream data to the mixing processing unit 101.

  Here, when two audio streams are superimposed and reproduced, the audio stream decoded by the 1st audio decoder 75-1 and the audio stream decoded by the 2nd audio decoder 75-2 are mixed into the mixing processing unit 101. To be supplied.

  For example, when the 1st audio stream referred to in the main path and the 2nd audio stream referred to in the sub path are superimposed and reproduced, the audio stream of the main Clip AV stream file decoded by the 1st audio decoder 75-1 The audio stream of the sub Clip AV stream file decoded by the 2nd audio decoder 75-2 is supplied to the mixing processing unit 101.

  Further, for example, when two audio streams referenced in the sub path are superimposed and reproduced, the audio stream of the sub Clip AV stream file decoded by the 1st audio decoder 75-1 and the 2nd audio decoder 75-2. The audio stream of the sub-Clip AV stream file decoded by the above is supplied to the mixing processing unit 101.

  The mixing processing unit 101 mixes (superimposes) the audio data from the 1st audio decoder 75-1 and the audio data from the 2nd audio decoder 75-2, and outputs the mixed data to the subsequent mixing processing unit 97. In the present embodiment, mixing (superimposing) the audio data output from the 1st audio decoder 75-1 and the audio data output from the 2nd audio decoder 75-2 is also referred to as synthesis. In other words, synthesis means to mix two audio data.

  The sound data selected by the switch 32 is supplied to the buffer 95 and buffered. The buffer 95 supplies the sound data to the mixing processing unit 97 at a predetermined timing. The sound data is, for example, sound effect data by menu selection or the like. The mixing processing unit 97 is the audio data mixed by the mixing processing unit 101 (audio data obtained by mixing the audio data output from the 1st audio decoder 75-1 and the audio data output from the 2nd audio decoder 75-2). The sound data supplied from the buffer 95 is mixed (superimposed or synthesized) and output as an audio signal.

  The data read from the buffer 54 that is a text subtitle read buffer is output to the text subtitle composition (decoder) 76 at the subsequent stage at a predetermined timing. The text subtitle composition 76 decodes the Text-ST data and supplies it to the switch 77.

  The switch 77 selects either the presentation graphics stream decoded by the presentation graphics decoder 73 or Text-ST (text subtitle data), and supplies the selected data to the presentation graphics plane generating unit 93. . That is, the subtitle image simultaneously supplied to the presentation graphics plane generating unit 93 is an output of either the presentation graphics decoder 73 or the text subtitle (Text-ST) composition 76. The presentation graphics stream that is simultaneously input to the presentation graphics decoder 73 is a stream that is separated from either the main Clip AV stream file or the sub Clip AV stream file. Therefore, the subtitle image simultaneously output to the presentation graphics plane generation unit 93 is a presentation graphics stream from the main Clip AV stream file, a presentation graphics stream from the sub Clip AV stream file, or text subtitle data. Decode output.

  Based on the background image data supplied from the background decoder 71, the background plane generation unit 91, for example, when displaying a video image in a reduced size or when displaying a GUI display screen described later other than during stream playback. A background plane serving as a wallpaper image is generated and supplied to the video data processing unit 96.

  The 1st video plane generating unit 92-1 generates a video plane based on the video data supplied from the 1st video decoder 72-1, and supplies this to the video data processing unit 96. The 2nd video plane generation unit 92-2 generates a video plane based on the video data supplied from the 2nd video decoder 72-2, and supplies this to the video data processing unit 96.

  The presentation graphics plane generation unit 93 generates, for example, a presentation graphics plane which is a rendering image based on the data (presentation graphics stream or text subtitle data) selected by the switch 77 and supplied, and this is converted into a video. The data is supplied to the data processing unit 96.

  The interactive graphics plane generating unit 94 generates an interactive graphics plane based on the data of the interactive graphics stream supplied from the interactive graphics decoder 74, and supplies this to the video data processing unit 96.

  The video data processing unit 96 generates a background plane from the background plane generation unit 91, a video plane from the first video plane generation unit 92-1, and a video plane and presentation graphics plane from the 2nd video plane generation unit 92-2. The presentation graphics plane from the unit 93 and the interactive graphics plane from the interactive graphics plane generation unit 94 are combined and output as a video signal.

  Note that the video data processing unit 96 performs picture-in-picture display, that is, both the video plane from the 1st video plane generation unit 92-1 and the video plane from the 2nd video plane generation unit 92-2. When receiving an input, a picture-in-picture display screen (described later with reference to FIG. 31) having a sub-screen (child screen) is generated in the main screen (parent screen).

  The switches 57 to 59 and the switch 77 are switched based on a selection from the user via the operation input unit 29 or to a side to which a file including target data is supplied. For example, when the audio stream is included only in the sub-Clip AV stream file, the switch 59 switches the switch to the side that supplies the sub-Clip AV stream file (the output of the PID filter 56).

  Next, playback processing in the playback device 1 will be described. When reproduction of a predetermined AV stream is instructed by the user using the operation input unit 29, the controller 21 is a local medium configured from the recording medium 11 via the storage drive 22 or an HDD (Hard Disk Drive) or the like. Based on the PlayList (PlayItem and SubPlayItem included in the PlayList) included in the PlayList file, the main Clip AV stream, the sub Clip AV stream, and the text subtitle data (Text-ST data) are read from the storage 24, and the AV decoder unit 26.

  The controller 21 supplies the background image data to the buffer 51, supplies the main Clip AV stream data to the buffer 52, supplies the sub Clip AV stream data to the buffer 53, and supplies the Text-ST data to the buffer 54. Switch 32 is switched to supply.

  Then, the switch 32 is switched, the background image data is supplied to the buffer 51, the data of the main Clip AV stream is supplied to the buffer 52, the data of the sub Clip AV stream is supplied to the buffer 53, and the text subtitle data is The data is supplied to the buffer 54, and each of the buffers 51 to 54 buffers the supplied data.

  The buffer 51 outputs the background image data to the background decoder 71, the buffer 52 outputs the stream data of the main Clip AV stream to the PID filter 55, and the buffer 54 converts the text subtitle data to the text subtitle composition 76. The buffer 53 outputs the stream data of the sub Clip AV stream to the PID filter 56.

  The PID filter 55 supplies the video stream to the PID filter 60, supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73, and supplies the interactive graphics stream to the interactive graphics decoder 74. The original switch 58 is supplied, and the audio stream is supplied to the switch 59. That is, different PIDs are attached to the video stream, presentation graphics stream, interactive graphics stream, and audio stream, respectively.

  Then, the PID filter 56 supplies the video stream to the PID filter 60, supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73, and supplies the interactive graphics stream to the interactive graphics decoder 74. Is supplied to the switch 58 serving as the supply source, and the audio stream is supplied to the switch 59.

  When the primary video stream is input, the PID filter 60 supplies the primary video stream to the 1st video decoder 72-1. On the other hand, when a secondary video stream is input, the PID filter 60 supplies the secondary video stream to the 2nd video decoder 72-2.

  Then, the switch 57 and the switch 58 subsequent to the PID filter 55 and the PID filter 56 select either the main clip or the sub clip based on the control from the controller 21. That is, the switch 57 selects the presentation graphics stream of the main Clip AV stream or the sub Clip AV stream supplied from the PID filter 55 and supplies it to the presentation graphics decoder 73 at the subsequent stage. In addition, the switch 58 selects the interactive graphics stream of the main Clip AV stream or the sub Clip AV stream supplied from the PID filter 55 and supplies the selected interactive graphics stream to the subsequent interactive graphics stream decoder 74.

  Further, when audio synthesis is not performed, the switch 59 selects either the main Clip AV stream supplied from the PID filter 55 or the audio stream of the sub Clip AV stream supplied from the PID filter 56. Then, it is supplied to the 1st audio decoder 75-1 at the subsequent stage. When an instruction for switching audio or an instruction for synthesizing audio is issued by the user, the switch 59 supplies the audio stream of the main Clip AV stream to the 2nd audio decoder 75-2, The audio stream of the sub Clip AV stream is supplied to the 1st audio decoder 75-1 and the 2nd audio decoder 75-2.

  The background decoder 71 decodes the background image data and outputs it to the background plane generation unit 91. The 1st video decoder 72-1 and the 2nd video decoder 72-2 decode the video stream, Are output to the 1st video plane generation unit 92-1 and the 2nd video plane generation unit 92-2, respectively, and the presentation graphics decoder 73 decodes the presentation graphics stream selected and supplied by the switch 57. Is output to the downstream switch 77, and the interactive graphics decoder 74 decodes the interactive graphics stream selected and supplied by the switch 58 and outputs the decoded interactive graphics stream. Output to revertive graphics plane generator 94, Text-ST composition 76 decodes the text subtitle data, and outputs it to the subsequent stage of the switch 77.

  Then, the 1st audio decoder 75-1 decodes the supplied audio stream (primary audio stream) selected by the switch 59, and outputs this to the mixing processing unit 101 at the subsequent stage. For example, in the reproduction process in the state where the user has not instructed to switch the audio, the audio data is not output from the 2nd audio decoder 75-2. Therefore, the mixing processing unit 101 outputs from the 1st audio decoder 75-1. The audio data thus obtained is supplied as it is to the subsequent mixing processing unit 97. In addition, in a state in which voice synthesis is instructed by the user, the 2nd audio decoder 75-2 is selected by the switch 59, decodes the supplied audio stream, and outputs this to the subsequent mixing processing unit 101. . The mixing processing unit 101 synthesizes (mixes) the audio data output from the 1st audio decoder 75-1 and the audio data output from the 2nd audio decoder 75-2, and supplies the resultant to the mixing processing unit 97 at the subsequent stage.

  The switch 77 selects either the data from the presentation graphics decoder 73 or the Text-ST composition 76 and supplies the selected data to the presentation graphics plane generating unit 93.

  The background plane generating unit 91 generates a background plane based on the background image data supplied from the background decoder 71, and the video plane generating unit 92 is based on the video data supplied from the video decoder 72. The presentation graphics plane generating unit 93 generates a video plane based on the data from the presentation graphics decoder 73 or the text-ST composition 76 selected and supplied by the switch 77. The interactive graphics plane generating unit 94 generates an interactive graphics stream data supplied from the interactive graphics decoder 74. An interactive graphics plane is generated based on the data.

  Then, the video data processing unit 96 combines the data of each plane and outputs it as video data.

  The buffer 95 buffers the supplied sound data and supplies it to the mixing processing unit 97 at a predetermined timing. The mixing processing unit 97 mixes (synthesizes) audio data (audio data output from the mixing processing unit 101) and sound data and outputs the result.

  By such processing, the main Clip AV stream, the sub Clip AV stream, and the text subtitle data are referred to and reproduced by the main path and sub path included in the PlayList. Since the main path and the sub path are provided in the PlayList, and the sub path is a clip AV stream different from the clip AV stream file specified by the main path, the sub clip AV stream can be specified. Data included in a sub clip, which is a different clip from the indicated main clip, and data included in the main clip can be reproduced together (at the same timing).

  By the way, as described above, the application format of data recorded on the recording medium 11 mounted on the playback apparatus 1 of the present invention or the local storage 24 of the playback apparatus 1 has two layers, PlayList and Clip. is doing. A PlayList is composed of a main path (Main Path) created by a sequence of one or more PlayItems (by consecutive PlayItems) and a parallel (in parallel) to the Main Path (by a sequence of one or more SubPlayItems (non- One or more sub paths (sub paths) created by sub play items that may be continuous or may be continuous may be included.

  By reproducing the data having such a configuration on the playback apparatus 1 described with reference to FIGS. 29 and 30, as shown in FIG. 31, each of the full screen (parent screen) 301 and the subsidiary screen 302 is displayed. It is possible to apply a picture-in-picture technique such as simultaneous playback of different contents.

  For example, as shown in FIG. 31, an AV stream (video stream 311 described as “Primary Video” in FIG. 31 and “Primary Audio” in FIG. 31) is displayed on all screens 301 in one screen. The content such as the main movie corresponding to the described audio stream 312) is reproduced, and the AV stream (described as Secondary Video in FIG. 31) is referred to in the sub-screen 302 arranged on the full screen 301 by the sub path. Applying a method of reproducing content different from the AV stream referred to in the main path, such as a commentary of a movie director corresponding to the video stream 313 and the audio stream 314) described as Secondary Audio It becomes possible to do.

  Specifically, for example, when the movie main part (Primary Video, Primary Audio) is displayed on the full screen 301, the director's commentary (Secondary Video, Secondary Audio) is nested in the sub-screen 302, for example. This is an application that synthesizes the sound as needed and reproduces it so that it is simultaneously displayed and output.

  Note that the playback position of the subtitle stream (the display position of subtitles and the like) does not have to be in the child screen, and may be any position on the entire screen. Therefore, in the example of FIG. 31, the subtitle stream 315 is drawn so as to protrude from the child screen 302.

  The user operations that the user is permitted with respect to the picture-in-picture application include whether to display secondary video, in other words, a user operation for commanding on / off of the picture-in-picture application, and a plurality of operations in one section. This is a user operation for switching the currently displayed secondary video to another secondary video when the secondary video is prepared. In addition, when secondary video is displayed, user operations that command on / off of secondary audio and change of secondary audio to be reproduced, and on / off of subtitles for picture-in-picture and subtitles are also provided. It is forgiven.

  In picture-in-picture, a plurality of Secondary Videos that can be displayed as the sub-screen 302 can be defined in one PlayList. For example, when a large number of picture-in-picture applications are defined in the PlayList, such as 255 Secondary Videos that can be displayed as the child screen 302 can be included in one PlayList, the user It is difficult to grasp picture-in-picture applications that can be reproduced in content.

  Further, for example, for a Primary Video displayed on the full screen 301, a specific scene can be called and played using a chapter or the like as has been conventionally performed. However, in contrast to this, with respect to the primary video playback section displayed on the entire screen 301, the user grasps which section of the secondary video can be played as the sub-screen 302 in which section. It is difficult. Therefore, a method for calling a specific scene of the Secondary Video displayed on the sub-screen 302 together with the Primary Video displayed on the full screen 301 is not provided.

  Also, Secondary Video and Secondary Audio are not necessarily prepared for all playback sections of Primary Video, and there may be cases where multiple Secondary Video and Secondary Audio are prepared in the same playback section. is there. Further, there are some types of secondary video and secondary audio corresponding to the secondary video that are not reproduced at a predetermined timing but are reproduced at a timing desired by the user (asynchronous). For this reason, for example, it is difficult to arbitrarily call a specific scene of Secondary Video and the corresponding Secondary Audio by specifying the playback start time.

  Therefore, in the playback apparatus 1, the user grasps which Secondary Video can be played back as the sub-screen 302 in which section with respect to the Primary Video playback section displayed on the full screen 301. It is assumed that it has a function to generate and display a GUI display screen that can be used.

  By referring to the GUI display screen, the user can efficiently select a secondary video and a specific scene of the secondary audio corresponding to the secondary video.

  In addition, the playback apparatus 1 allows the user to understand which Secondary Video can be played as the sub-screen 302 in which section with respect to the Primary Video playback section displayed on the entire screen 301. In addition to the GUI display screen that can be used, for example, the user can select a specific scene of either Primary Video or Secondary Video without clearly distinguishing between Primary Video and Secondary Video. A simple GUI display screen may be generated and displayed.

  FIG. 32 is a functional block diagram for explaining a function of generating and displaying such a GUI display screen among the functions of the controller 21.

  The controller 21 has various functions in addition to, for example, a function of generating and displaying a GUI display screen and a process of reproducing data recorded in the attached recording medium 11 or the local storage 24. However, in FIG. 32, functions other than the generation and display control of the GUI display screen and the reproduction control of the stream data are omitted.

  The PlayList read control unit 351 acquires a PlayList file. For example, during normal playback, the PlayList read control unit 351 detects the Clip_Information_file_name of the corresponding Clip based on the PlayItem and SubPlayItem included in the acquired PlayList file, and supplies the detected Clip_Information_file_name to the stream data read control unit 354. Also, the PlayList read control unit 351 supplies the acquired PlayList file to the PlayList analysis unit 352, for example, in order to display the GUI display screen as described above.

  The PlayList analysis unit 352 reads STN_table () from the supplied PlayList file, and acquires ref_to_Subpath_id corresponding to Secondary Video and Secondary Audio included in the stream played by this PlayList file based on the stream_entry of STN_table () And supplied to the IN_time / OUT_time acquisition unit 356.

  The operation input acquisition unit 353 acquires a user operation input supplied from the operation input unit 29 and supplies the operation input to the stream data read control unit 354, the reproduction start position information storage control unit 358, or the GUI display control unit 359.

  The stream data read control unit 354 controls reading of the stream data included in the corresponding Clip and supply to the AV decoder unit 26 based on the Clip_Information_file_name supplied from the PlayList read control unit 351. In addition, the stream data read control unit 354 starts the reproduction whose storage is controlled by the reproduction start position information storage control unit 358 based on the user's operation input supplied from the operation input acquisition unit 353 with reference to the GUI display screen. With reference to the information on the position, the reading of the stream data included in the corresponding Clip and the supply to the AV decoder unit 26 are controlled.

  The reproduction control unit 355 controls decoding and reproduction output of data supplied to the AV decoder unit 26 based on the control of the stream data read control unit 354.

  The IN_time / OUT_time acquisition unit 356 extracts SubPlayItems included in the corresponding Subpath based on the ref_to_Subpath_id corresponding to the Secondary Video and Secondary Audio supplied from the PlayList analysis unit 352, and extracts the SubPlayItem_IN_time and the SubPlayItem_IN_time SubPlayItem_Out_time is acquired and supplied to the GUI generation unit 357 and the reproduction start position information storage control unit 358.

  The GUI generation unit 357 generates data corresponding to the GUI display screen based on the SubPlayItem_IN_time and SubPlayItem_Out_time corresponding to the Secondary Video and Secondary Audio supplied from the IN_time / OUT_time acquisition unit 356, and the GUI display control unit 359. To supply. In addition, the GUI generation unit 357 causes the stream data read control unit 354 to correspond to the image data (for example, the head of the secondary video stream, that is, SubPlayItem_IN_time), based on the thumbnail image corresponding to the secondary video, as necessary. It is also possible to acquire partial frame image data or any other frame image data) and include it in the generated GUI display screen data. Further, the GUI generation unit 357 supplies information related to the generated GUI display screen data to the reproduction start position information storage control unit 358.

  The playback start position information storage control unit 358 relates to the SubPlayItem_IN_time and SubPlayItem_Out_time corresponding to the Secondary Video and Secondary Audio supplied from the IN_time / OUT_time acquisition unit 356, and the GUI display screen data supplied from the GUI generation unit 357. Based on the information, Secondary Video and Secondary Audio expressed on the GUI display screen and their playback start positions (SubPlayItem_IN_time) are stored in association with each other. As a result, when information related to the operation input performed by the user referring to the generated GUI display screen is acquired from the operation input acquisition unit 353, at least one of the Secondary Video or Secondary Audio desired by the user is obtained as the Primary Path Primary Video. Also, reproduction start position information for reproduction together with Primary Audio can be extracted and supplied to the stream data read control unit 354.

  The GUI display control unit 359 controls display of the GUI display screen data generated by the GUI generation unit 357 and cancellation of the display based on the user operation input supplied from the operation input acquisition unit 353. Specifically, the GUI display control unit 359 controls the OSD 30 to control superimposition of the GUI display screen data on the video signal or image signal supplied from the AV decoder unit 26.

  The controller 21 having the function as shown in FIG. 32 is in a state in which the content recorded in the recording medium 11 or the local storage 24 can be reproduced by mounting the recording medium 11 in the reproducing apparatus 1 or the like. When generating GUI display screen data.

  That is, the PlayList file read by the PlayList read control unit 351 is analyzed by the PlayList analysis unit 352, and ref_to_Subpath_id corresponding to the Secondary Video and Secondary Audio included in the stream played by the PlayList file is acquired, and IN_time The SubPlayItem included in the corresponding Subpath is extracted by the / OUT_time acquisition unit 356, and the respective SubPlayItem_IN_time and SubPlayItem_Out_time are acquired from these SubPlayItems.

  The GUI generation unit 357 supplies the SubVideoItem_IN_time and SubPlayItem_Out_time corresponding to the Secondary Video and Secondary Audio supplied from the IN_time / OUT_time acquisition unit 356 to the Secondary Video acquired by the stream data read control unit 354 as necessary. GUI display based on the image data that is the basis of the corresponding thumbnail image (for example, the beginning of the secondary video stream, that is, the frame image data of the portion corresponding to SubPlayItem_IN_time, or any other frame image data, etc.) Data corresponding to the screen is generated and supplied to the GUI display control unit 359.

  For example, when the user instructs display of the GUI display screen from the operation input unit 19 during playback of the main video Primary Video and Primary Audio such as a movie main part, the operation input acquisition unit 353 instructs display of the GUI display screen. The user's operation input is acquired, and the GUI display control unit 359 controls the display of the GUI display screen.

  At this time, for example, as shown in FIG. 33, in the part of the screen 381 on which the main movie or the like is displayed, the Secondary Video is displayed in any section with respect to the Primary Video playback section. As a result, a GUI display screen 391 is displayed so that the user can grasp whether or not playback is possible.

  The GUI display screen 391 will be described with reference to FIG.

  On the GUI display screen 391, for example, MainMovie, that is, a main timeline 401 corresponding to the entire playback period of the Primary Video, and a slider 402 indicating the playback position of the current Primary Video are displayed on the main timeline 401. In association with the main timeline 401, it can be seen which secondary video can be played back as the sub-screen 302 in relation to the primary video playback section, and the secondary audio is played back correspondingly. In addition, a sub timeline 403 (in FIG. 34, sub timeline 403-1 to sub timeline 403-5) corresponding to Secondary Video or Secondary Audio included in the stream reproduced by the PlayList file is displayed.

  Then, an icon 404 (in FIG. 34, the icon 404) indicating whether the stream associated with the sub time line 403 is associated with the sub time line 403 is Secondary Video or Secondary Audio. -1 to 404-4) are displayed.

  In addition, for example, the stream data read control unit 354 uses the image data that is the basis of the thumbnail image corresponding to the Secondary Video (for example, the beginning of the Secondary Video stream, that is, the frame image data of the portion corresponding to SubPlayItem_IN_time, Any one of the frame image data or the like) is acquired, and the icon 404 corresponding to the video stream among the icons 404 may be a so-called thumbnail image.

  Further, it is possible for the user to recognize that the sub timeline 403 corresponding to Secondary Video or Secondary Audio that is played back (asynchronous) at the user's desired timing is played back (asynchronous) at the user's desired timing. Is preferably displayed. Such a sub timeline 403 may be displayed so as to correspond to the reproduction start position with respect to the main timeline 401, for example, or may be displayed at a different position not corresponding to the main timeline 401. Also good.

  By displaying such a GUI display screen 391, the user can recognize which part of the main part is currently being reproduced on the entire screen 301, and how many picture-in-picture applications are in which reproduction position. It is possible to easily grasp visually how many asynchronous-in picture-in-picture applications exist.

  Therefore, the user can easily execute a user operation as to whether or not to display the secondary video by referring to the GUI display screen 391. Further, since the user can easily recognize whether or not there are a plurality of picture-in-picture applications, the user operation for commanding on / off of the picture-in-picture application can be easily and efficiently executed. . Further, the GUI display screen 391 is considered to be useful for the user's Forward Play / Backward Play operation.

  Also, the stream data read control unit 354 uses the image data that is the basis of the thumbnail image corresponding to the Secondary Video (for example, the head of the Secondary Video stream, that is, the frame image data of the portion corresponding to the SubPlayItem_IN_time, or any of the other) When the thumbnail image is displayed together with the sub timeline 403, the user can easily recognize the contents of the picture-in-picture application.

  When the display of the GUI display screen 391 is instructed when the Primary Path Primary Video and Primary Audio are not being played back, the background plane output from the AV decoder unit 26, that is, the so-called background image, The OSD 30 displays the GUI display screen 391 in a superimposed manner.

  When the user selects the sub timeline 403 or the icon 404, the reproduction start position information whose storage is controlled by the reproduction start position information storage control unit 358 is referred to, and the secondary video or secondary desired by the user is referred to. At least one of the audio is played along with the primary video and primary audio of the main path.

  When the sub timeline 403 or the icon 404 corresponding to the asynchronous secondary video or secondary audio is selected, for example, at least the secondary video or secondary audio desired by the user, together with the primary video and primary audio from the main path head position, are selected. Either can be played.

  Further, the GUI generation unit 357 can acquire the image data that is the basis of the thumbnail image corresponding to the Secondary Video by the stream data read control unit 354, and can generate the GUI display screen based on this. For example, as shown in FIG. 35, a GUI display screen 421 that allows the user to select a picture-in-picture application (Secondary Video and Secondary Audio) and MainMovie (Primary Video and Primary Audio) without having to distinguish between them in detail. It may be generated and displayed.

  In the GUI display screen 421 in FIG. 35, for example, in addition to the thumbnail image 411 corresponding to MainMovie, thumbnail images 442 corresponding to each picture-in-picture (in FIG. 35, thumbnail images 442-1 to 442-6) Is displayed.

  Note that when the display of the GUI display screen 421 described with reference to FIG. 35 is instructed while the Primary Path Primary Video and Primary Audio are being played back, an image signal output from the AV decoder unit 26 (for example, Primary The GUI display screen 421 is superimposed and displayed by the OSD 30 on the video signal (the video signal in which captions and the like are combined with Video). When the display of the GUI display screen 421 is instructed when the Primary Path Primary Video and Primary Audio are not being reproduced, the OSD 30 applies to the background plane output from the AV decoder unit 26, that is, the so-called background image. A GUI display screen 421 is displayed superimposed.

  The user who refers to such a GUI display screen 421 determines whether or not the picture-in-picture application is synchronized with MainMovie, and if it is synchronized, what playback position is defined in the PlayList. It is possible to instruct the reproduction of a desired picture-in-picture application without knowing details such as.

  Also in this case, when the user selects a thumbnail image 442 corresponding to each picture-in-picture, the reproduction start position information whose storage is controlled by the reproduction start position information storage control unit 358 is referred to, and the user Is reproduced together with the Primary Video and Primary Audio of the MainPath.

  Similarly, when the thumbnail image 442 corresponding to each of the picture-in-picture corresponding to asynchronous Secondary Video or Secondary Audio is selected, for example, along with the Primary Video and Primary Audio from the MainPath head position, the user It is assumed that at least one of the desired Secondary Video or Secondary Audio is played back.

  That is, the reproduction start position information storage control unit 358 receives supply of information of SubPlayItem_IN_time and SubPlayItem_Out_time corresponding to Secondary Video and Secondary Audio from the IN_time / OUT_time acquisition unit 356, and also displays a GUI display screen from the GUI generation unit 357. Receive information on data. Then, based on these pieces of information, the reproduction start position information storage control unit 358 displays each Secondary Video and any of the Secondary Video and the thumbnail image 442 that are represented as any sub timeline 403, icon 404, or thumbnail image 442 on the GUI display screen. Since the secondary audio and the playback start position (SubPlayItem_IN_time) are controlled so as to be stored in association with each other, the stream data read control unit 354 performs a picture-in-picture desired by the user based on this information. It is possible to instruct the start of reproduction in display.

  Specifically, the GUI display screen 391 described with reference to FIGS. 33 and 34 or the GUI display screen 421 described with reference to FIG. 35 is superimposed on the image signal or background image output from the AV decoder unit 26. When the user receives an operation input for selecting any one of the sub timeline 403, the icon 404, or the thumbnail image 442, the operation input acquisition unit 353 displays the sub timeline 403, Information indicating which of the icon 404 or the thumbnail image 442 is selected is supplied to the reproduction start position information storage control unit 358.

  The reproduction start position information storage control unit 358 extracts reproduction start position information for reproducing at least one of Secondary Video or Secondary Audio desired by the user together with the Primary Video and Primary Audio of the MainPath, and a stream data read control unit 354. To supply.

  The stream data read control unit 354 is based on the user operation input with reference to the GUI display screen supplied from the operation input acquisition unit 353 and the reproduction start position information supplied from the reproduction start position information storage control unit 358. Controls reading of Primary Video and Primary Audio included in the corresponding Clip, Secondary Video, and, if necessary, Stream data of Secondary Audio, and supply of the read stream data to the AV decoder unit 26. .

  The reproduction control unit 355 controls the decoding, synthesis, and reproduction output of data supplied to the AV decoder unit 26 based on the control of the stream data read control unit 354. Then, in the picture-in-picture display, the content reproduction is started from the position designated by the user.

  Next, with reference to the flowchart of FIG. 36, the reproduction process executed in the reproduction apparatus 1 will be described.

  In step S1, the PlayList read control unit 351 determines whether or not the recording medium 11 is loaded. If it is not determined in step S1 that the recording medium 11 is loaded, the process of step S1 is repeated until it is determined that the recording medium 11 is loaded.

  If it is determined in step S1 that the recording medium 11 is loaded, a GUI display preparation process described later with reference to FIG. 37 is executed in step S2.

  In step S <b> 3, the operation input acquisition unit 353 determines whether an operation input for instructing reproduction of a stream has been acquired. In step S3, when it is determined that the operation input commanding the reproduction of the stream has not been acquired, the process proceeds to step S6 described later.

  If it is determined in step S3 that an operation input commanding playback of a stream has been acquired, in step S4, the PlayList read control unit 351 reads a PlayList file from the attached recording medium 11, and includes the acquired PlayList file. Based on the PlayItem and SubPlayItem, the Clip_Information_file_name of the corresponding Clip is detected and supplied to the stream data read control unit 354.

  In step S5, the stream data read control unit 354 controls reading of the stream data included in the corresponding Clip and supply to the AV decoder unit 26 based on the Clip_Information_file_name of the PlayList supplied from the PlayList read control unit 351. . The reproduction control unit 355 controls the decoding and reproduction output of data supplied to the AV decoder unit 26.

  In step S3, when it is determined that the operation input for instructing the reproduction of the stream has not been acquired, or after the processing in step S5 is completed, in step S6, the operation input acquisition unit 353 determines whether the GUI display has been instructed. Judge whether or not. If it is determined in step S6 that the GUI display is not instructed, the process proceeds to step S13 described later.

  If it is determined in step S6 that GUI display has been commanded, in step S7, the GUI display control unit 359 controls the display of the GUI generated in the GUI display preparation process in step S2.

  In step S8, the operation input acquisition unit 353 determines whether an operation input for selecting a thumbnail or the like (any sub timeline 403, icon 404, or thumbnail image 442) corresponding to any SubPlayItem has been received. To do. If it is determined in step S8 that an operation input for selecting one has not been received, the process proceeds to step S11 described later.

  If it is determined in step S8 that an operation input for selecting one has been received, in step S9, the operation input acquisition unit 353 selects one of the sub timeline 403, the icon 404, or the thumbnail image 442. The information indicating whether or not is supplied to the reproduction start position information storage control unit 358. The reproduction start position information storage control unit 358 extracts reproduction start position information for reproducing at least one of Secondary Video or Secondary Audio desired by the user together with the Primary Video and Primary Audio of the MainPath, and a stream data read control unit 354. To supply. The stream data read control unit 354 acquires information indicating the corresponding reproduction start position.

  In step S10, the stream data read control unit 354 reads the stream data included in the corresponding Clip based on the information indicating the corresponding reproduction start position acquired from the reproduction start position information storage control unit 358, and the AV decoder unit. 26 is controlled. The playback control unit 355 controls decoding and playback output of data supplied to the AV decoder unit 26, that is, playback of a picture-in-picture application from a predetermined playback start position.

  When Subpath_type = 6, the subpath is asynchronous with respect to the main path, but the subpath, that is, the secondary video and secondary audio corresponding to the child screen are streamed based on the values of SubPlayItem_IN_time and SubPlayItem_Out_time as they are. The primary video and primary audio corresponding to the entire screen are controlled to be read from the head position reproduced by the PlayList, and each is synthesized and reproduced.

  If it is determined in step S8 that an operation input for selecting one has not been received, in step S11, the operation input acquisition unit 353 determines whether or not the termination of the GUI display has been commanded. If it is determined in step S11 that the GUI display end is not instructed, the process returns to step S6, and the subsequent processes are repeated.

  If it is determined in step S11 that the end of GUI display has been commanded, in step S12, the GUI display control unit 359 ends the display of the GUI displayed in step S6.

  After the process of step S10 or step S12 ends, in step S13, the operation input acquisition unit 353 determines whether or not the process end is commanded. In step S13, when it is determined that the end of the process is not instructed, the process returns to step S6, and the subsequent processes are repeated. In step S13, if it is determined that the end of the process has been commanded, the process ends.

  As a result of such processing, the GUI display screen is displayed, so that the user can easily and clearly recognize the contents of a plurality of picture-in-picture applications, the playback position with respect to the main stream, and the like. Then, based on the user's operation input referring to the GUI display screen, it is possible to start playback of the picture-in-picture display desired by the user.

  Next, the GUI display preparation process executed in step S2 of FIG. 36 will be described with reference to the flowchart of FIG.

  In step S <b> 41, the PlayList read control unit 351 reads the PlayList file recorded on the attached recording medium 11 and supplies it to the PlayList analysis unit 352.

  In step S42, the PlayList analysis unit 352 confirms each PlayItem included in the supplied PlayList, and acquires STN_table.

  In step S43, the PlayList analysis unit 352 determines whether Secondary Audio or Secondary Video exists in a stream that can be played by the supplied PlayList. If it is determined in step S43 that Secondary Audio or Secondary Video does not exist, the process returns to step S2 in FIG. 36 and proceeds to step S3.

  If it is determined in step S43 that Secondary Audio or Secondary Video exists, in step S44, the PlayList analysis unit 352 acquires ref_to_SubPath_id corresponding to Secondary Audio and Secondary Video from the stream_entry of STN_table, and IN_time / OUT_time acquisition unit 356.

  In step S45, the IN_time / OUT_time acquisition unit 356 selects one of the unprocessed SubPaths corresponding to the ref_to_SubPath_id acquired in step S44.

  In step S46, the IN_time / OUT_time acquisition unit 356 determines whether the selected SubPath is SubPath_type = 5. If it is determined in step S46 that SubPath_type = 5, the process proceeds to step S50 described later.

  When it is determined in step S46 that SubPath_type = 5 is not satisfied, in step S47, the IN_time / OUT_time acquisition unit 356 determines whether or not the selected SubPath is SubPath_type = 6.

  If it is determined in step S47 that SubPath_type = 6, in step S48, the IN_time / OUT_time acquisition unit 356 associates the asynchronous subpath with the first position of the main stream, that is, FIG. 33 and FIG. In the GUI display screen 391 described above, the corresponding sub timeline 403 is displayed in association with the head of the main timeline 401, or the corresponding thumbnail of the GUI display screen 421 described using FIG. 35 is selected. In order to set the playback start position of the main movie to the head of the stream, SubplayItem_IN_time ′ = 0 and SubPlayItem_OUT_time ′ = SubPlayItem_OUT_time−SubplayItem_IN_time are supplied to the GUI generation unit 357 and the playback start position information storage control unit 358.

  In step S47, when it is determined that SubPath_type = 6 is not satisfied, that is, SubPath_type = 7, or after the process of step S48 is completed, in step S49, the GUI generation unit 357 supports Secondary Video and Secondary Audio. Based on the respective SubPlayItem_IN_time and SubPlayItem_Out_time, data such as the corresponding main timeline 401, subtimeline 403, icon 404, thumbnail 441 or thumbnail 442 is generated. After the process of step S49 ends, the process proceeds to step S52 described later.

  If it is determined in step S46 that SubPath_type = 5, in step S50, the IN_time / OUT_time acquisition unit 356 responds based on the ref_to_Subpath_id corresponding to the Secondary Video and Secondary Audio supplied from the PlayList analysis unit 352. Extract SubPlayItem included in Subpath to analyze the stream. Specifically, the IN_time / OUT_time acquisition unit 356 acquires the respective SubPlayItem_IN_time and SubPlayItem_Out_time from those SubPlayItems, and also determines whether those streams are secondary audio or secondary video based on the STN_table. The type of the stream is detected, and the information is supplied to the GUI generation unit 357 and the reproduction start position information storage control unit 358.

  In step S51, the GUI generation unit 357, based on the respective SubPlayItem_IN_time and SubPlayItem_Out_time corresponding to the Secondary Video and Secondary Audio, supplied from the IN_time / OUT_time acquisition unit 356, and other analysis data, the corresponding main timeline 401, the sub timeline 403, the icon 404, or the like, or the data such as the thumbnail 441 or the thumbnail 442 is generated.

  After the process of step S49 or step S51 is completed, in step S52, the IN_time / OUT_time acquisition unit 356 determines whether the process has been completed for all the SubPaths. If it is determined in step S52 that the process has not been completed for all the SubPaths, the process returns to step S45, and the subsequent processes are repeated.

  If it is determined in step S52 that the processing has been completed for all the SubPaths, in step S53, the GUI generation unit 357 generates data corresponding to the GUI display screen 391 or the GUI display screen 421, and the GUI display control unit 359. To the playback start position information storage control unit 358, the information about the generated GUI display screen data.

  In step S54, the reproduction start position information storage control unit 358 supplies the SubPlayItem_IN_time and SubPlayItem_Out_time corresponding to the Secondary Video and Secondary Audio supplied from the IN_time / OUT_time acquisition unit 356, and the GUI supplied from the GUI generation unit 357. Based on the information about the display screen data, the thumbnails of Secondary Video and Secondary Audio represented on the GUI display screen and their playback start positions (SubPlayItem_IN_time) are stored in association with each other. The process returns to step S2 and proceeds to step S3.

  By such processing, stream data reproduced by a plurality of paths reproduced by the reproduction apparatus 1 is analyzed, and reproduction of Secondary Video or Secondary Audio reproduced by a secondary reproduction path among the plurality of paths is performed. A GUI display screen showing the position and its contents is generated.

  Specifically, for example, in the full screen 301, it is possible to recognize where the main part is currently being played back, and how many picture-in-picture applications exist at which playback position, or asynchronous pictures. A GUI display screen (for example, the GUI display screen 391 described with reference to FIGS. 33 and 34) that can easily visually grasp how many in-picture applications exist, and a picture-in-picture application Instructs playback of the desired picture-in-picture application without knowing details such as whether or not it is synchronized with MainMovie, and if so, what playback position is defined in the PlayList. GUI display screen (for example, the GUI display described with reference to FIG. 35) Surface 421) is generated.

  By referring to such a GUI display screen, the user can visually recognize clearly a reproducible picture-in-picture application.

  Further, by referring to such a GUI display screen, the user can instruct playback of a desired picture-in-picture application.

  Next, with reference to FIGS. 38 and 39, a method of manufacturing the recording medium 11 on which data that can be reproduced by the reproducing apparatus 1 is recorded will be described by taking the case where the recording medium 11 is a disc-shaped recording medium as an example. .

That is, as shown in FIG. 38, a master disc made of glass or the like is prepared, and a recording material made of photoresist or the like is applied thereon. As a result, a recording master is produced.

  Then, as shown in FIG. 39, in the software production processing unit, video data in a format that can be played back by the playback device 1 and encoded by the encoding device (video encoder) is stored in a temporary buffer, and is recorded by the audio encoder. The encoded audio data is stored in the temporary buffer, and data other than the stream (for example, Indexes, PlayList, PlayItem, etc.) encoded by the data encoder is further stored in the temporary buffer. Video data, audio data, and data other than the stream stored in each buffer are multiplexed together with a synchronization signal by a multiplexer (MPX), and an error correction code is added by an error correction code circuit (ECC). Is done. Then, a predetermined modulation is applied by a modulation circuit (MOD), and software that is once recorded on a magnetic tape or the like according to a predetermined format and recorded on a recording medium 11 that can be reproduced by the reproducing apparatus 1 is manufactured.

  This software is edited (premastered) as necessary, and a signal in a format to be recorded on the optical disc is generated. Then, as shown in FIG. 38, the laser beam is modulated corresponding to this recording signal, and this laser beam is irradiated onto the photoresist on the master. Thereby, the photoresist on the master is exposed in accordance with the recording signal.

  Then, this master is developed and pits appear on the master. The master plate thus prepared is subjected to a process such as electroforming to produce a metal master plate in which pits on the glass master plate are transferred. A metal stamper is further produced from this metal master, and this is used as a molding die.

  A material such as PMMA (acrylic) or PC (polycarbonate) is injected into this molding die by, for example, injection and fixed. Alternatively, 2P (ultraviolet curable resin) or the like is applied on the metal stamper and then cured by irradiation with ultraviolet rays. Thereby, the pits on the metal stamper can be transferred onto the replica made of resin.

  A reflective film is formed on the replica thus generated by vapor deposition or sputtering. Alternatively, a reflective film is formed on the generated replica by spin coating.

  Thereafter, the inner and outer diameters of the disk are processed, and necessary measures such as bonding two disks are performed. Further, a label is attached or a hub is attached and inserted into the cartridge. In this way, the recording medium 11 on which data reproducible by the reproducing apparatus 1 is recorded is completed.

  The series of processes described above can be executed by hardware or can be executed by software. In this case, the processing described above is executed by a personal computer 500 as shown in FIG.

  40, a CPU (Central Processing Unit) 501 performs various processes according to a program stored in a ROM (Read Only Memory) 502 or a program loaded from a storage unit 508 to a RAM (Random Access Memory) 503. Execute. The RAM 503 also appropriately stores data necessary for the CPU 501 to execute various processes.

  The CPU 501, ROM 502, and RAM 503 are connected to each other via an internal bus 504. An input / output interface 505 is also connected to the internal bus 504.

  The input / output interface 505 includes an input unit 506 including a keyboard and a mouse, a display including CRT and LCD, an output unit 507 including a speaker, a storage unit 508 including a hard disk, a modem, a terminal adapter, and the like. A communicator 509 is connected. A communication unit 509 performs communication processing via various networks including a telephone line and CATV.

  A drive 510 is connected to the input / output interface 505 as necessary, and a removable medium 521 made up of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted, and a computer program read therefrom is It is installed in the storage unit 508 as necessary.

  When a series of processing is executed by software, a program constituting the software is installed from a network or a program storage medium.

  As shown in FIG. 40, the program storage medium is configured only by a package medium including a removable medium 521 on which a program is recorded, which is distributed to provide a program to the user, separately from the computer. Rather, it is provided with a ROM 502 in which a program is recorded and a hard disk including a storage unit 508 provided to the user in a state of being pre-installed in the apparatus main body.

  In the present specification, the step of describing a computer program includes not only processing performed in time series according to the described order but also processing executed in parallel or individually even if not necessarily processed in time series. Is also included.

  Further, the drive 510 can read data recorded on the attached removable medium 521 and can record data on the attached removable medium 521. The personal computer 500 has the same function as the software production processing unit described with reference to FIG. 39 (for example, the CPU 501 is used to execute a program for realizing the same function as the software production processing unit). It goes without saying that it is possible.

  That is, the personal computer 500 generates data similar to the data generated by the software production processing unit described with reference to FIG. 39 by the processing of the CPU 501 or uses FIG. 39 generated by an external device. Data similar to the data generated by the software production processing unit described above can be acquired via the communication unit 509 or the removable medium 521 attached to the drive 510. Then, the personal computer 500 serves as a recording device that records data similar to the data generated or acquired by the software production processing unit described with reference to FIG. 39 on the removable medium 521 attached to the drive 510. Function can be realized.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

It is a figure explaining the conventional voice switching. It is a figure explaining the conventional voice switching. It is a figure explaining the structure of an MPEG2 program stream. It is a figure which shows the example of the application format on the recording medium with which the reproducing | regenerating apparatus to which this invention is applied is mounted | worn. It is a figure explaining the structure of a main path and a sub path. It is a figure explaining the example of a main path and a sub path. It is a figure explaining another example of a main path and a sub path. It is a figure explaining the data structural example of the data which can be reproduced | regenerated with a reproducing | regenerating apparatus. It is a figure which shows the syntax of index.bdmv. It is a figure which shows indexes syntax. It is a figure for demonstrating the data structure of a PlayList file. It is a figure for demonstrating the syntax of AppInfoPlayList (). It is a figure which shows the syntax of PlayList (). It is a figure which shows the syntax of PlayItem (). It is a figure which shows the syntax of STN_table (). It is a figure which shows the syntax of STN_table (). It is a figure which shows the syntax of stream_entry (). It is a figure which shows the syntax of stream_attribute (). It is a figure explaining stream_cording_type. It is a figure explaining video_format. It is a figure explaining frame_rate. It is a figure explaining aspect_ratio. It is a figure explaining audio_presentation_type. It is a figure explaining sampling_frequency. It is a figure explaining Character code. It is a figure which shows the syntax of SubPath (). It is a figure explaining SubPath_type. It is a figure which shows the syntax of SubPlayItem (i). It is a block diagram which shows the structure of a reproducing | regenerating apparatus. FIG. 30 is a block diagram illustrating a configuration of an AV decoder unit of the playback device in FIG. 29. It is a figure for demonstrating a picture in picture display. It is a functional block diagram which shows the function which the controller of the reproducing | regenerating apparatus of FIG. 29 has. It is a figure for demonstrating a GUI display screen. It is a figure for demonstrating a GUI display screen. It is a figure for demonstrating a GUI display screen. It is a flowchart explaining a reproduction | regeneration process. It is a flowchart explaining a GUI display preparation process. It is a figure for demonstrating manufacture of the recording medium which recorded the data which can be reproduce | regenerated with a reproducing | regenerating apparatus. It is a figure for demonstrating manufacture of the recording medium which recorded the data which can be reproduce | regenerated with a reproducing | regenerating apparatus. It is a figure which shows the structure of a personal computer.

Explanation of symbols

  1 playback device, 11 recording medium, 21 controller, 26 AV decoder section, 30 OSD, 32 switch, 33 AV decoder section, 34 controller, 51 to 54 buffer, 55, 56 PID filter, 57 to 59 switch, 71 background decoder , 72 video decoder, 73 presentation graphics decoder, 74 interactive graphics decoder, 75 audio decoder, 76 Text-ST composition, 77 switch, 91 background plane generation unit, 92 video plane generation unit, 93 presentation graphics plane generation Section, 94 interactive graphics plane generation section, 95 buffer 96 video data processing section, 97 mixi Processing unit, 101 mixing processing unit, 351 PlayList read control unit, 352 PlayList analysis unit, 353 operation input acquisition unit 353, 354 stream data read control unit, 355 playback control unit, 356 IN_time / OUT_time acquisition unit, 357 GUI generation unit , 358 Playback start position information storage control unit, 359 GUI display control unit, 391 GUI display screen, 401 main timeline, 402 slider, 403 sub timeline, 404 icon, 421 GUI display screen, 441, 442 thumbnail image

Claims (14)

First acquisition means for acquiring reproduction management information for controlling reproduction of a stream;
Analyzing the reproduction management information acquired by the first acquisition means, and from the analysis result, information relating to one or a plurality of sub reproduction paths is a reproduction path different from the main reproduction path of the stream. A second acquisition means for acquiring;
Information generating means for generating information on the main reproduction path and the stream reproduced by the sub reproduction path based on the information on the sub reproduction path acquired by the second acquisition means. Processing equipment. The information processing apparatus according to claim 1, wherein the second acquisition unit acquires information related to a reproduction position of the sub reproduction path. The information generating means is associated with the reproduction time axis of the first stream reproduced by the main reproduction path, and is reproduced by all the sub reproduction paths managed for reproduction by the reproduction management information. The information processing apparatus according to claim 2, wherein image information indicating a reproduction position of the second stream is generated. The information processing apparatus according to claim 3, wherein the information generation unit generates the image information including information indicating a reproduction range of each of the plurality of second streams reproduced by the sub reproduction path. The information processing apparatus according to claim 3, wherein the information generation unit generates the image information including information indicating a type of each of the plurality of second streams reproduced by the sub reproduction path. Further comprising third acquisition means for acquiring at least part of the stream data;
The information generation means is a set of image data that constitutes the plurality of second streams to be reproduced by the secondary reproduction path based on at least a part of the stream data acquired by the third acquisition means. The information processing apparatus according to claim 3, wherein the image information including at least part of the image information is generated. The second acquisition means acquires a part of image data constituting a stream reproduced by the primary reproduction path and the secondary reproduction path,
The information generating means is an image constituting the first stream reproduced by the main reproduction path based on at least a part of the image data constituting the stream obtained by the second obtaining means. The information processing apparatus according to claim 1, wherein image information including at least a part of data and at least a part of image data constituting the second stream reproduced by the sub reproduction path is generated. . The information processing apparatus according to claim 1, further comprising display control means for controlling display of information relating to the stream generated by the information generation means. Third acquisition means for acquiring the stream data;
Replay control means for controlling the reproduction of the stream data obtained by the third obtaining means based on the reproduction management information obtained by the first obtaining means,
The display control means displays the information on the stream so that the information on the stream generated by the information generation means is superimposed on the stream data whose reproduction is controlled by the reproduction control means. The information processing apparatus according to claim 8 to be controlled. It further comprises operation input acquisition means for acquiring operation input by the user,
The second acquisition means acquires information relating to the playback position of the sub playback path,
The information generating means is associated with the reproduction time axis of the first stream reproduced by the main reproduction path, and is reproduced by all the sub reproduction paths managed for reproduction by the reproduction management information. Generating image information indicating the playback position of the second stream;
The display control unit controls generation of the image information generated by the information generation unit,
When the operation input for selecting any one of the reproduction positions of the second stream by the user referring to the image information whose display is controlled by the display control unit is acquired by the operation input acquisition unit, The information processing apparatus according to claim 8, wherein the reproduction control unit controls reproduction of the first stream and the second stream from the selected reproduction position. It further comprises operation input acquisition means for acquiring operation input by the user,
The second acquisition means acquires information on the playback position of the secondary playback path and part of the image data constituting the stream,
The information generating means is based on at least a part of the stream data acquired by the second acquisition means, and includes the image data constituting the first stream reproduced by the main reproduction path. Generating image information including at least a part and at least a part of the image data constituting the second stream reproduced by the sub reproduction path;
The display control unit controls generation of the image information generated by the information generation unit,
An operation input for selecting any of the image data constituting the second stream by the user referring to the image information whose display is controlled by the display control unit is acquired by the operation input acquiring unit. In this case, the reproduction control unit controls reproduction of the first stream and the second stream from the reproduction position at the head of the second stream corresponding to the image data selected by the user. The information processing apparatus described in 1. In an information processing method of an information processing device that generates information about a stream to be played back,
Obtaining reproduction management information for controlling reproduction of the stream;
Analyzing the acquired playback management information to obtain information on one or more secondary playback paths that are different from the main playback path of the stream,
An information processing method including the step of generating information on the main reproduction path and the stream reproduced by the sub reproduction path based on the acquired information on the sub reproduction path. A program for causing a computer to execute processing for generating information about a stream to be played back,
Controlling the acquisition of playback management information for controlling playback of the stream;
Analyzing the playback management information whose acquisition is controlled, and controlling acquisition of information related to one or a plurality of secondary playback paths that are different from the main playback path of the stream;
A program for causing a computer to execute processing including a step of generating information on the main reproduction path and the stream reproduced by the sub reproduction path based on information on the sub reproduction path whose acquisition is controlled.   A program storage medium in which the program according to claim 13 is recorded.

Images