CN1700329B - Reproducing device, reproducing method, reproducing program and recording medium - Google Patents

Abstract

The invention discloses a reproducing device for reproducing content data recorded on a disc-shaped recording medium, for reproducing content data recorded on a disc-shaped recording medium. Reproduced non-real-time stream and real-time stream; storage means for storing the program code input by the input means; image data storage means for storing the image data input by the input means; first combining means for combining the decoded Movie data and the subtitle data after decoding; And the second combining means, according to the program code stored in the code storage device, the image data after decoding stored in the image data storage device and the combination of the movie and subtitle data after decoding data are combined.

Description

Reproducing device, reproducing method, reproducing program and recording medium

technical field

The present invention relates to a reproducing apparatus, a reproducing method, a reproducing program and a recording medium capable of allowing a user to interactively operate a program recorded in a mass recording medium such as a Blu-ray disc.

Background technique

In recent years, a blu-ray disc (blu-ray Disc) standard has been proposed for a recordable disc type recording medium that can be detached from a recording and reproducing apparatus. The Blu-ray disc standard specifies an optical disc having a recording medium having a diameter of 12 cm and a surface layer having a thickness of 0.1 mm. The Blu-ray Disc standard uses a blue-violet laser with a wavelength of 405nm and an objective lens with a numerical aperture of 0.85. The maximum recording capacity achieved by the Blu-ray Disc standard is 27 GB (gigabytes). As a result, two hours or more of broadcasting satellite (BS) digital high-definition broadcast programs adopted in Japan can be recorded without image quality degradation.

As an audio/video (AV) signal source (source) recorded in a recordable optical disc, analog signals such as conventional analog TV broadcasting, and digital signals such as digitized TV broadcasting such as BS digital broadcasting can be used. The Blu-ray Disc standard has established a method for recording the above-mentioned broadcasted AV signal.

On the other hand, as a derivative standard of the current Blu-ray Disc standard, a reproduction-only recording medium in which a piece of movie, music, etc. can be pre-recorded has been developed. A digital video disc (DVD) has been widely used as a disc-type recording medium in which movies or music are pre-recorded. Reproduction-only discs that conform to the Blu-ray Disc standard are very different from conventional DVDs, and are superior to conventional DVDs in terms of large recording capacity and high-speed transfer speed, which allow two hours or more of high-definition images to be recorded in high quality.

On the other hand, what the current Blu-ray Disc standard specifies is neither a method for displaying a list of video content on a disc on screen, nor a user interface feature that allows the user to move a cursor on the list And selects a video content that he or she wants to reproduce from the list. These functions are realized by a recording and reproducing apparatus main unit which records video content to a Blu-ray Disc and reproduces video content from the Blu-ray Disc. Thus, even if a video content is reproduced from the same recording medium, the layout of the content list screen depends on the recording and reproduction apparatus used, as does the user interface. Therefore, the user cannot use the Blu-ray disc very conveniently. Therefore, it is necessary to allow reproduction-only optical discs to display a menu screen, and other similar screens that have been designed by the optical disc (content) manufacturer, the display of which does not depend on the reproducing apparatus.

Also, a multi-episode function, which displays a selection screen while a video content is being reproduced, is generally called an interactive function. To realize the above-mentioned interactive function, the disc manufacturer must create a scenario (scenario), wherein he or she specifies a reproduction command and a branch of the video content, describes the scenario using a program language, script language, etc., and records the described scenario in on the disc. The playback device side reads and executes the script. As a result, the reproducing apparatus reproduces one video content, and displays a selection screen that allows the user to select a branch of the video content that the producer has specified.

Thus, the current Blu-ray Disc standard (Blu-ray Disc Rewritable Format Version 1.0) specifies neither a method of composing menu screens and branch selection screens specified by content producers, nor a method of describing user input processes. Therefore, so far, it is difficult to reproduce video content from a Blu-ray Disc according to such a script if the manufacturer does not consider the compatibility of the manufacturer and model of the reproducing device when designing the script.

The function of displaying subtitles is necessary for a playback-only disc on which a movie has been recorded. However, the existing Blu-ray Disc standard does not specify the function of describing subtitles.

On the other hand, the above-mentioned interactive function has been implemented in standards such as the DVD (Digital Video Disc) standard. For example, in DVD-Video, while a movie is being reproduced, a remote control director is used to call up the menu screen. For example, by selecting a key displayed on a menu screen, the user can execute a program for changing the current scene being reproduced. The DVD standard also stipulates the function of displaying subtitles. The above features allow users to switch from Japanese subtitles to prepared English subtitles and vice versa.

Regarding DVD, the menu screen is composed of a fixed sub-picture. When the menu screen is called up, the sub-pictures are displayed as a combined movie. Japanese Patent Application Laid-Open No. Hei 10-308924 (hereinafter referred to as Patent Document 1) describes a structure for combining sprite data and movie data and recording the combined data in a recordable DVD.

Next, an example of a menu screen will be briefly described based on the related reference art. By the reproducing means, the menu screen is reproduced before the main part of the movie is reproduced from the DVD. Typically, several keys are provided on the menu screen. Each key is assigned a predetermined action. When the user selects a key and causes the operation of the selected key to be performed, the operation assigned to the key is performed. For example, when the user selects the button "movie main part" and causes the operation of the selected button to be performed, the operation specified by the button is executed. As a result, the main part of the movie is reproduced from the DVD.

The user operates keys (direction keys) assigned as up, down, left, and right directions using, for example, a remote control commander (hereinafter referred to as a remote controller) to select keys displayed on the menu screen. Thereafter, the user uses the OK key to perform the designated operation of the selected key. In addition, each button has three states: normal state (unselected state), selected state, and execution state. To allow users to easily distinguish them, they have different images and colors. Typically, only one key is either selected or executed.

For example, in DVD-Video, each key is displayed using two types of data called sub-picture and highlight. FIG. 1 shows an example of a DVD menu screen 300 according to the related reference art. The menu screen 300 is called a "title menu". The menu screen 300 has three buttons 301A, 301B, and 301C, which respectively represent "reproduce the main part of the movie", "beginner screen" and "sound effect setting". In the example shown in FIG. 1, the color of the outer frame of the "reproduce main part of movie" button 301A is changed from the original color. This shows that the button 301A of "replay the main part of the movie" is already in the selected state.

In the above state, when the user operates the direction key on the remote controller, as shown in FIG. 2A , FIG. 2B , and FIG. 2C , he or she can make another key in the selected state. Just like the situation shown in FIG. 1 , the color of the frame of the selected button is different from the color of the unselected button (unselected button). For example, in the situation shown in FIG. 1, when the user operates the OK button provided on the remote controller, as shown in FIG. 3, the color of the "replay main part of the movie" button 301A changes to a color representing the execution state. Thereafter, the menu screen 300 is cleared, and the movie main part is reproduced. The above are the basic operations of the keys of the DVD video.

The menu screen 300 shown in FIG. 1 is composed of three types of data, the above three types of data are background picture 310, sub-picture 311, and highlight 312, as shown in FIG. 4A, FIG. 4B, and FIG. 4C respectively. The background picture 310 is a static picture, a movie of the main part of the DVD pre-recorded content, or the like.

As shown in FIG. 5, the sprite 311 has a bitmap, four color information (A0, B0, C0, D0), and coordinates (X, Y). Bitmap images are represented by two bits of information per pixel. The coordinates (X, Y) indicate the display start position of the sub screen 311 . Each color information A0, B0, C0, and D0 is monochrome information data constituted by a set of R (red), G (green), and B (blue) data. Each color R, G and B has 8 bits of information. Each pixel of a bitmap image has two bits of information. Of these two bits, one bit is selected from the above-mentioned four-color information (A0, B0, C0, D0) for each pixel. Color information and transparency data. The sprite 311 may have an area in which the background picture 310 is transparent. The display position of the upper left corner of the sub screen 311 is represented by coordinates (X, Y) relative to the background screen 310 .

In addition, the sprite 311 may have information indicating the start time and end time of the display, as well as commands that cause visual effects such as fade-in and fade-out to be applied on the sprite 311 .

In DVD-Video, multiple bitmap images cannot be displayed simultaneously. Thus, a menu screen 300 having several buttons on it as shown in FIG. 1 is displayed by one large bitmap image having three button images as shown in FIG. 4B. In the bitmap image of the sub-picture 311 shown in FIG. 4B, when an area outside the designated keys 301A, 301B, and 301C is a transparent area, and the sub-picture 311 is combined with the background picture 310, the keys 301A, 301B, and 301C The background image 310 outside the display area of the , becomes transparent.

A highlight (highlight) 312 is information for changing the four colors used by the sprite 311 to other four colors. As shown in FIG. 5 , as color information, the highlight 312 has color information (A1, B1, C1, D1) in the selected state and color information (A2, B2, C2, D2) in the execution state. These color information are four-color information, and each information is represented by 8-bit RGB like the above-mentioned sprite 311 .

Highlight 312 has a set of coordinates for the area in which the color is changed. The range of color change is not limited to the entire sprite 311 but a part of the square area of the sprite 311 . The number of square areas in the sub-screen 311 corresponds to the number of keys that can be selected by the user, and the highlight 312 can change the color of the above areas. The display position of each square area is indicated by the coordinates (X, Y) of its upper left corner and lower right corner. For example, the location of highlight 312A corresponding to key 301A is represented by coordinates (X1, Y1) and (X1', Y1'). Applied highlights 312B and 312C correspond to keys 301B and 301C, respectively.

For example, in the highlight 312A, the color information (A0, B0, C0, D0) of an area represented by the coordinates (X1, Y1) and (X1', Y1') of the background image 310 is changed to designate the selected state Color information for the color (A1, B1, C1, D1). Here, the color information A0 of the background image 310 is changed to the color information A1 of the highlight 312A. Similarly, the color information B0 of the background image 310 is changed to color information B1. The color information C0 is changed to color information C1. The color information D0 is changed to color information D1.

Next, an example of the color change of the highlight 312 will be described according to the change of the state of the key 301A of the menu screen 300 shown in FIGS. 1 , 2A, 2B, 2C and 3 . Assume that when the key 301A is in the unselected state, the frame, front and font of the key 301A are displayed by color information B0, color information C0, and color information C0, respectively. When the key 301A is in the selected state, its frame color information B0 is changed to the color information B1 corresponding to the selected state of the highlight 312A. At this time, the front color C0 and font color D0 are not changed. Thereafter, when the key 301A is in the execution state, the color of its selected state, that is, the front color C0 is changed to the color information C1. At this time, the frame color B1 and font color D0 of the selected state color are not changed.

When DVD-Video frames are normally reproduced, a frame corresponding to the background frame 310 is displayed. On the other hand, when reproducing a film with subtitles, it is necessary to reproduce the background picture 310 of the main part of the film, and to display the subtitle sub-picture 311, to combine the background picture 310 and the sub-picture 311 and display them.

However, in DVD-Video, each of the sub-picture 311, the highlight 312 representing the selection state, and the highlight 312 representing the execution state can only use up to four colors. Therefore, it is a difficult problem in the prior art that sprites with multiple colors cannot be displayed.

Furthermore, the highlight 312 can only change the color of the sprite 311 . Thus, the font of the key cannot be changed in, for example, the selection state and the execution state. In addition, the effect of changing the shape of the key cannot be achieved. Thus, the prior art cannot realize a rich user interface.

In addition, subtitles and keys are usually displayed in the same mode using the sub-picture 311 . Thus, subtitles and keys cannot be controlled and displayed separately. Also, a combined method cannot be implemented to set and combine the transparency of subtitles and buttons, and display the combined picture.

Also, in DVD-Video, when a menu screen is called, reproduction of movie data in its background is stopped. Therefore, usually even if the above-mentioned interactive function is realized, the flexibility of the user interface for realizing the function is very low.

In DVD-Video, no scheme is specified in the standard for generating effect sound in synchronization with display and change of subtitles. Therefore, it is a difficult problem in the prior art that sound effects cannot be generated synchronously with, for example, subtitles.

In addition, the above-mentioned standard does not specify a scheme for generating an effect sound for a button, for example, when the user puts a button in the selected state, an effect sound is generated, or when a button is in the selected state, the user operates the OK button, and a click sound is generated. Therefore, it is difficult to realize a rich user interface, which is a difficult problem in the prior art.

The above-mentioned effect sound is not sound data reproduced synchronously with a movie or a still picture displayed on a movie plane (for example, sound recorded in pairs with a movie), but is reproduced by display control of subtitles and buttons, or Audio data reproduced in synchronization with the display control described above.

Contents of the invention

Accordingly, an object of the present invention is to provide a reproducing apparatus, a reproducing method, a reproducing program and a recording medium capable of realizing a user interface with high flexibility for a large-capacity reproduction-only optical disc.

Another object of the present invention is to provide a reproducing apparatus, a reproducing method, a reproducing program and a recording medium capable of realizing a rich user interface for a large-capacity reproduction-only optical disc.

A first aspect of the present invention is a reproduction apparatus for reproducing content data recorded in a disc-type recording medium, the apparatus comprising input means for inputting a non-real-time stream reproduced from the recording medium and a real-time stream, the non-real-time The stream includes storing image data forming an operation screen to prompt the user to perform an operation and several segments of sound data used to reproduce effect sounds as at least one set of program code files, and the real-time stream includes at least film data and subtitle data; the reproducing device also Including storage means for storing the program code input by the input means; and comprising image data storage means for storing the image data input by the input means; the device also includes a first combining means for combining the decoded film data and decoded subtitle data, the above-mentioned film data is decoded and input by the input device, and the above-mentioned subtitle data is decoded and input by the input device; The program code combines the decoded image data stored in the image data storage device, and the decoded combination data of movie and subtitle data.

The second part of the present invention is a reproducing method for reproducing content data stored in a disc-type recording medium, the method comprising the steps of inputting a non-real-time stream reproduced from the recording medium and a real-time stream, the non-real-time stream comprising a Operating the screen to prompt the user to perform an operation image data and several sections of sound data used to reproduce the effect sound are stored as at least one set of program code files, and the real-time stream includes at least movie data and subtitle data; the input is stored in the input step the step of program code; the step of storing the image data input in the input step; the step of combining decoded film data and decoded subtitle data, the above-mentioned film data is decoded and input in the input step, and the above-mentioned subtitle data is Decoded and input in the input step; combine the decoded image data stored in the second storing step, and the combined data of the decoded movie and subtitle data, the combined data of the decoded movie and subtitle data being in the A combining step is generated by combining the decoded film and subtitle data according to the program code stored in the first storing step.

The third step of the present invention is a reproducing program for causing a computer device to execute a reproducing method for reproducing content data stored in a disc-type recording medium, the method including the step of inputting non-real-time data reproduced from the recording medium stream and real-time stream, the non-real-time stream includes storing image data constituting an operation screen to prompt the user to perform an operation and several pieces of sound data used to reproduce sound effects as at least one set of program code files, and the real-time stream includes at least a movie data and subtitle data; the step of storing the program code input in the input step; the step of storing the image data input in the input step; the step of combining decoded movie data and decoded subtitle data, the above movie data being Decoded and input in the input step, the above subtitle data is decoded and input in the input step; combining the decoded image data stored in the second storage step, and combined data of the decoded film and subtitle data, the above The combined data of the decoded movie and subtitle data is generated by combining the decoded movie and subtitle data in the first combining step according to the program code stored in the first storing step.

The fourth method of the present invention includes a recording medium from which a computer device can read a reproducing program for causing a computer device to execute a reproducing method for reproducing data stored in a disk-type recording medium. For content data, the method includes the step of inputting a non-real-time stream reproduced from a recording medium and a real-time stream including image data to compose an operation screen to prompt a user to perform an operation and pieces of sound for reproducing an effect sound The data is stored as a file of at least one group of program codes, and the real-time stream includes at least movie data and subtitle data; the step of storing the program code input in the input step; the step of storing the image data input in the input step; after combining the decoding The steps of the movie data and the decoded subtitle data, the above-mentioned movie data are decoded and input in the input step, and the above-mentioned subtitle data are decoded and input in the input step; combining the decoded stored in the second storage step Image data, and combined data of the movie after decoding and subtitle data, the combined data of the movie after the above-mentioned decoding and subtitle data is according to the program code stored in the first storage step in the first combination step, the movie after combination decoding and subtitle data are generated.

The fifth aspect of the present invention is a disk-type recording medium, wherein content data is recorded, the non-real-time stream includes at least program code, image data forming an operation screen to prompt the user to perform an operation, and the real-time stream includes at least film data and subtitle data, A file stored as a set of multi-segment sound data for recording effect sound, stored decoded image data, and combined data of decoded movie and subtitle data, which are combined according to the program code.

A sixth aspect of the present invention is a recording apparatus for recording content data on a disc-type recording medium, the apparatus comprising: recording apparatus for recording non-real-time streams and real-time streams, the non-real-time streams comprising a The image data for operating the screen to prompt the user to perform an operation and several pieces of sound data for reproducing the effect sound are stored as at least one set of program code files, and the real-time stream includes at least movie data and subtitle data; wherein the program code needs to be executed after decoding A combined operation of image data and a combined data. The decoded image data is obtained by decoding the image data recorded by the recording device, and the combined data includes decoded video data of the real-time stream and decoded subtitle data of the subtitle data of the real-time stream.

As described above, according to the first to fourth aspects of the present invention, a non-real-time stream and a real-time stream reproduced from a recording medium are input. The non-real-time stream includes a file storing image data constituting an operation screen to prompt a user to perform an operation and pieces of sound data for reproducing effect sounds as at least one set of program codes. A real-time stream includes at least movie data and subtitle data. Program codes are stored in code storage means. Image data is stored in an image data storage device. The input video data is decoded to obtain decoded video data, and the input subtitle data is decoded to obtain decoded subtitle data, and the decoded video data and decoded subtitle data are combined to obtain combined data. Combining the above-mentioned decoded image data and combination data stored in the image data storage device according to the program code stored in the code storage device, the above-mentioned combination data is a combination of decoded movie and decoded subtitle data. Thus, when content data is reproduced, operation screens using the same image data can be easily displayed at different times. In addition, an effect sound prompting the user to operate on the operation screen can be reproduced.

According to a fifth aspect of the present invention, it is recorded: a non-real-time stream, including at least one set of program codes; image data forming an operation screen to prompt the user to perform an operation; the real-time stream includes at least movie data and subtitle data; a file, storing A series of multi-segment sound data for recording effect sounds. Combining the stored image data and combination data according to the program code, the above combination data is a combination of decoded movie and decoded subtitle data. Thus, when content data is reproduced, operation screens using the same image data can be easily displayed at different times. In addition, an effect sound prompting the user to operate on the operation screen can be reproduced.

According to the sixth aspect of the present invention, a non-real-time stream is recorded in the recording medium, including at least one set of program code files; image data forming an operation screen to prompt the user to perform an operation; real-time stream, including at least film data and subtitles Data; a file that stores a series of multi-segment sound data for recording effect sounds. Based on the program code stored in the recording medium, the stored decoded image data is combined with combination data which is a combination of decoded movie and decoded subtitle data. Thus, when content data is reproduced, operation screens using the same image data can be easily displayed at different times. In addition, an effect sound prompting the user to operate on the operation screen can be reproduced.

As described above, the pre-recorded large-capacity optical disc according to the present invention has three independent planes, which are: a film plane for movies, a subtitle plane for subtitles, and a graphics plane for interactive functions such as menu screens. Screen. These planes are combined and displayed with each other. Thus, as an effect of the present invention, a movie can be displayed on the movie plane while a background such as a menu screen accompanying the movie is displayed on the graphics plane.

Furthermore, according to the invention, since a buffer is arranged for storing image data to be displayed on the graphics plane, the same image data can be repeatedly displayed on the graphics plane. Therefore, as an effect of the present invention, it is possible to configure a menu screen or the like having higher flexibility than before.

Furthermore, according to the present invention, display control for graphics displayed on a graphics plane is described using a display control command. Thus, as an effect of the present invention, the screen displayed on the graphics plane realizes the interactive function.

In addition, the pre-recorded large-capacity optical disc according to the present invention has three independent planes, which are: a film plane for movies, a subtitle plane for subtitles, and a graphics plane for an interactive function such as a menu screen. Screen. These planes are combined and displayed with each other. A common graphics object, which is an object format, is displayed on the subtitle plane, and forms the graphics plane. Determine the decoding model, display control commands, and operation model. Therefore, as an effect of the present invention, subtitles and buttons can be displayed synchronously with the movie.

In addition, a simple animation is implemented, which scrolls and moves the subtitles and buttons, and the content of the image can change according to the user's input, thereby beautifying the buttons.

Furthermore, according to the present invention, a decoding model is determined in which sound data is included in the graphic object, and the sound data is reproduced when the button image included in the graphic object is displayed. Thus, sound data can be easily reproduced in synchronization with displayed graphic objects.

Furthermore, according to the present invention, a command causing sound data to be reproduced is determined with respect to a display control command for a graphic object. Furthermore, sound data may be assigned to image data included in a graphic object with respect to a display control command of the object. Thus, as an effect of the present invention, sound data as effect sounds can be reproduced at any time, and subtitles and keys also have effect sounds.

Furthermore, according to the present invention, there is provided a scheme for generating an effect when a user selects a key on a menu screen displayed by a graphical user interface (GUI), or when he or she operates the OK key after selecting a key Sound, such as a click sound. Thus, in a reproducing apparatus, a reproducing method, a reproducing program, and a recording medium, a user is allowed to interactively operate a program pre-recorded on a large-capacity recording medium such as an optical disc, and a rich user interface is realized. Refer to the appended The above-mentioned objects, features and advantages of the present invention will be more apparent from the detailed description of the preferred embodiments below.

Description of drawings

The present invention will be more fully understood from the following detailed description taken in conjunction with the accompanying drawings. The same reference numerals denote the same parts in the drawings. in:

FIG. 1 shows a schematic diagram of an example of a DVD menu screen according to the prior art;

Fig. 2A, Fig. 2B and Fig. 2C show the schematic diagram of the button that changes the selection state through the operation of the remote controller;

Fig. 3 shows a schematic diagram of changing the color of a key to another color, indicating the execution state of the OK key;

4A, 4B and 4C are diagrams showing an example of the structure of a menu screen according to the related art;

Fig. 5 shows a schematic diagram of an example of a data structure of a sub-picture according to the related art;

FIG. 6 shows a schematic diagram of a scheme for specifying the reproduction sequence of an AV stream file;

Fig. 7 shows the UML diagram of the relation of clip AV stream, clip information, clip, play item and play list;

Figure 8 is a schematic diagram describing a method for accessing the same segment from several playlists;

FIG. 9 is a schematic diagram describing a management structure of files recorded on a recording medium;

Fig. 10 shows a schematic diagram of a syntax describing an example of the structure of the file "info.badv";

FIG. 11 shows a schematic diagram of a syntax describing an example of the structure of a block UIAppInfoBDAV();

Fig. 12 shows a schematic diagram of a syntax describing an example of the structure of a block TableOfPlayLists();

Fig. 13 shows a schematic diagram of a syntax describing an example of the structure of files "#####.rpls" and "#####.vpls";

FIG. 14 shows a schematic diagram of a syntax describing an example of the structure of a block UIAppInfoPlayList();

Fig. 15 shows a schematic diagram of a syntax describing an example of the structure of a block PlayList();

Fig. 16 shows a schematic diagram of a syntax describing an example of the structure of a block PlayItem();

Figure 17 is a schematic diagram depicting a bridging segment;

Fig. 18 shows a schematic diagram of a syntax describing an example of the structure of a block PlayListMark ();

Fig. 19 shows a schematic diagram of a syntax describing an example of the structure of a file "%%%%%.clpi";

FIG. 20 shows a schematic diagram of an example of a plane used as a display system of a screen according to an embodiment of the present invention;

Fig. 21 shows a schematic diagram of examples of resolutions and displayable colors of a film plane, a subtitle plane and a graphics plane;

Fig. 22 shows a schematic diagram of an example of a structure which combines a movie plane, a subtitle plane and a graphics plane;

Fig. 23 is a schematic diagram of an example of input and output data of a palette;

Figure 24 shows a schematic diagram of an example of a palette table in a palette;

FIG. 25 is a schematic diagram showing an example of a menu screen displayed on a graphics plane;

Fig. 26 is a schematic diagram showing an example of the internal structure of a script;

Fig. 27 is a schematic diagram describing a BD virtual player mode;

FIG. 28 shows a schematic diagram of an example of an event specifically determined for a BD virtual player;

29A and 29B are schematic diagrams showing an example of a method determined for a BD virtual player according to an embodiment of the present invention;

Fig. 30A, Fig. 30B, Fig. 30C and Fig. 30D are schematic diagrams illustrating an example of a method determined for a BD virtual player according to an embodiment of the present invention;

Fig. 31A, Fig. 31B, Fig. 31C and Fig. 31D are schematic diagrams showing an example of a method determined for a BD virtual player according to an embodiment of the present invention;

Fig. 32 shows the schematic diagram of the example of the structure of a script recorded on the disc;

Fig. 33 shows the schematic diagram that is used to construct the necessary file list of a script;

Figure 34 shows a schematic diagram of an example of a directory structure when a script is recorded in the optical disc;

Fig. 35 shows a schematic diagram of a more practical example of description of a script file;

Fig. 36 shows a schematic diagram of a more practical example of description of a script file;

Fig. 37 is a schematic diagram of a more practical example of description of an HTML document;

Fig. 38 shows a schematic diagram of a more practical example of description of a script file;

Fig. 39A and Fig. 39B have shown the functional block diagram of an example of player decoder structure;

Fig. 40 shows a schematic diagram of an example of a state change of a displayed key;

Fig. 41 is a schematic diagram describing categories of object types;

Fig. 42A, Fig. 42B and Fig. 42C show the schematic diagram of the example of the data structure of the image object according to the embodiment of the present invention;

Fig. 43 shows a schematic diagram of a state in which a graphic object is divided and stored in a PES packet;

Fig. 44 shows the functional block diagram of the example of the structure of the graphic object decoder model of decoding graphic object;

Fig. 45A, Fig. 45B, Fig. 45C and Fig. 45D show the schematic diagram of the example of the stored data quantity that changes in graphic object input buffer, PNG decoder, object buffer and plane buffer;

Fig. 46 is a schematic diagram describing plane refresh rate;

Figure 47 shows a schematic diagram of a syntax describing an example of a graphics object structure;

Figure 48 shows a schematic diagram of a syntax describing an example of the structure of a block GlobalPaletteTable();

49A and 49B are diagrams showing an example of a display control command list of the command group DispCmds(i);

50A and 50B are diagrams describing the command set_display_box (x1, y1, x2, y2) and the command set_clipping_box (a1, b1, a2, b2);

Fig. 51 is a schematic diagram describing determination of coordinate axes;

Fig. 52 is a schematic diagram describing an example of a change of a command group DispCmds(i) and a displayed graphic object;

FIG. 53A, FIG. 53B, FIG. 53C and FIG. 53D show schematic diagrams of examples in which subtitles are gradually displayed and faded in;

FIG. 54A and FIG. 54B are schematic diagrams showing an example in which a PNG image as a subtitle moves on a plane;

55A and 55B are schematic diagrams showing examples of displayed subtitle scrolling;

56A and FIG. 56B show schematic diagrams in which a frame representing a part of a PNG image is designated, the frame moves on the PNG image, and its position moves on a plane;

Fig. 57 is a schematic diagram describing the command group IispCmds(i), a command play_sound(sound_id) is added to the above command group, which shows an example of changing the displayed graphic object;

Fig. 58 shows a schematic diagram of an example of a data structure of a graphic object in which sound data has been assigned to a button image;

Fig. 59A, Fig. 59B, Fig. 59C are schematic diagrams showing an example of a method for placing sound data in a graphic object;

Figure 60 shows a functional block diagram of an example of a graphics object decoding model to which a sound data processing system has been added;

Fig. 61 shows a schematic diagram of a syntax describing an example of a graphic object structure to which sound data has been added;

Fig. 62 is a diagram showing an example of the data management structure when sound data is not located in a graphic object;

Fig. 63 shows a more practical schematic diagram for describing the reproduction of sound data without sound data located in a graphic object;

FIG. 64 is a schematic diagram describing a management structure of files recorded on a recording medium;

Fig. 65 shows a schematic diagram of a syntax describing an example of the file "HdmvSound.bdmv";

Fig. 66 shows a schematic diagram of the structure of the file "HdmvSound.bdmv";

FIG. 67 is a diagram showing an example of an image of an application for reproducing sound effects according to a third storage method;

68A and 68B are schematic diagrams showing an example of the structure of an interactive graphics stream;

Fig. 69 shows a schematic diagram of the structure of interactive composition fragments;

Fig. 70 is a schematic diagram describing a field neibour_info of an interaction composition segment;

FIG. 71 shows a schematic diagram of another example of a planar structure used as a screen display system according to an embodiment of the present invention;

Fig. 72 shows a schematic diagram of another example of the resolution and displayable colors of the movie plane, the subtitle plane, and the graphics plane;

Fig. 73 shows a block diagram of a structure according to another example, which structure combines a film plane, a subtitle plane and a graphics plane;

Figure 74A and Figure 74B show the functional block diagram of the example of the structure of player decoder;

Fig. 75 A and Fig. 75 B have shown the functional block diagram of another example of player decoder structure;

Fig. 76 is a schematic diagram describing the positioning of sound data files to subplayitems;

Fig. 77 is a schematic diagram describing a management structure for files recorded on a recording medium;

Fig. 78 shows a schematic diagram of an example of the internal structure of a script according to the second mode of the present invention, the script uses the initially determined script description language;

Figure 79 is a schematic diagram of a directory describing a script structure;

Figure 80A, Figure 80B and Figure 80C are the schematic diagrams that have described the directory of script structure;

Fig. 81 is a schematic diagram describing headers and sections;

82A and 82B are flow charts showing an approximate operation of a BD virtual player corresponding to a command described in a script according to the second mode of the present invention;

83A and 83B are flowcharts describing a reproduction operation according to a PlayList;

84A and 84B are diagrams showing examples of hierarchical structures of scripts;

FIG. 85 is a schematic diagram describing a management structure of files recorded in a recording medium;

Fig. 86 shows a schematic diagram of a syntax describing an example of the structure of a script file (scenario.pbc) of a script;

Figure 87 shows a schematic diagram of a syntax describing an example of the structure of a block GlobalCommand();

Figure 88 shows a schematic diagram of a syntax describing an example of the data structure of a block TitleEntry();

Figure 89 shows a schematic diagram of a syntax describing an example of the structure of a block scenario();

Fig. 90 is a diagram showing an example of commands used in a script according to the second mode of the present invention.

Detailed ways

Next, the present invention will be described in the following order.

1. Outline in BDAV format

2. The first mode of the present invention

2-1. About plane

2-2. Menu screen

2-3. About the script

2-4. About the virtual player model

2-5. About the method

2-6. Execution of commands

2-7. Decoder model

2-8. User interface

2-9. About data structure

2-10. Decoder model for graphics objects

2-11. Regarding the transmission speed of graphics

2-12. Details of Graphics Object Structure

2-13. About the effect sound

2-14. About method for storing audio data on optical disc

2-14a. First storage method

2-14b. Second storage method

2-14c. The third storage method

2-15. Another example of plane

2-16. Another example of decoder model

2-17. Another example of decoder model

3. The second mode of the present invention

3-1. About the script

3-2. Script directory

3-3. About the virtual player model

3-4. About grammar

3-5. About commands

4. Other

1. Outline in BDAV format

First, in order to easily understand the present invention, a management structure specified in "Blu-ray Disc Rewritable Format Version 1.0 Part 3 Audio Visual Specification" for pre-recording on a Blu-ray Disc named AV (Audio/Video) will be described. the content of the data. In the following description, this management structure is referred to as BDAV format.

A bit stream that has been encoded according to a coding system such as MPEG (Moving Picture Experts Group) video or MPEG audio, and multiplexed according to the MPEG-2 system is called a clip AV stream (or simply AV stream). The clip AV stream is recorded in the disc as one file by the file system specified in "Blu-ray Disc Rewritable Format Version 1.0 Part 2" for Blu-ray Disc. This file is called a clip AV stream file (or simple AV stream).

A clip AV stream file is a management unit in the file system. Therefore, it cannot be said that the user can easily understand that the clip AV stream file is a management unit in the file system. From the viewpoint of user convenience, it is necessary to record a necessary scheme for combining the video content divided into several fragment AV stream files and reproduce the combined video content; and also record a necessary scheme for reproducing the fragment AV stream Only a part of the file; also record a necessary information to smoothly perform a specific reproduction and search reproduction, etc., as a database. "Blu-ray Disc Rewritable Format Version 1.0 Part 3" as a standard for Blu-ray Disc specifies such a database.

FIG. 6 shows a scheme for specifying a part of an AV stream file, processing only the necessary part thereof, and reproducing the processed part. In FIG. 6, a playlist (PlayList) allows a part or all of the AV stream file to be specified, and only a necessary part thereof is reproduced. When the user needs to reproduce a content, he or she selects the content in the unit of the play list. A playlist is a video/audio unit in which the user defaults to expect a piece of content to be played back continuously.

The simplest structure of a playlist is an AV stream file obtained from the beginning of recording a content to the end of the recording operation. Unless the AV stream file is edited, it becomes a playlist.

The play list is made up of information including information indicating reproduction of an AV stream file, and information for setting a reproduction start point and a reproduction stop point, which designate the reproduction start position and reproduction stop position of the AV stream file. An information pair consisting of reproduction start point information and reproduction stop point information is called a PlayItem (PlayItem). A playlist is composed of a set of play items. When one play item is played back, the part of the AV stream file related to the play item is played back.

As described above, the clip AV stream is a bit stream whose video data and audio data have been multiplexed in the format of MPEG2TS (Transport Stream). Information about a clip AV stream is recorded in one file as clip information.

A group of clip AV stream files and clip information files with their corresponding clip information are handled as one object and called a clip. A clip is an object composed of a clip AV stream and clip information.

Files are usually handled as sequences of bytes. The content of the clip AV stream file is developed on a time base. Entry points in a segment are regularly specified on a time base. If a time stamp of an access point to a predetermined section is given, a section information file can be used to find information of an address from which data is read from a section AV stream file.

All playlists and clips recorded on a disc are managed using volume information.

FIG. 7 shows a schematic diagram of Unified Modeling Language (uml), which represents the relationship between the above-mentioned clip AV stream, clip information (stream attributes), clips, playitems, and playlists. A playlist is associated with one or more playitems. A PlayItem is associated with a Clip. A segment can be associated with multiple play items whose start and/or end points are different. One clip AV stream file is referenced from one clip. Similarly, a clip information file is referenced from a clip. In addition, there is a one-to-one correspondence between one clip AV stream file and one clip information file. With the structure defined as above, a reproduction order can be specified non-destructively by reproducing only any part without changing the list.

Furthermore, as shown in FIG. 8, the same segment can be referenced from multiple playlists. In the example shown in FIG. 8 , Clip 1 is referenced from two Playlists 2 and 3 . In FIG. 8, the horizontal direction of segment 1 represents a time base. Playlist 2 refers to areas a to f of segment 1, which include commercial message areas b, c and one scene e. On the other hand, playlist 3 refers to areas d to g of section 1, which includes a scene e. When PlayList 2 is designated, areas a to f of Clip 1 can be reproduced. When PlayList 3 is designated, areas d to g of Clip 1 can be reproduced.

Next, referring to FIG. 9, a management system for files recorded in a recording medium specified in "Blu-ray Disc Rewritable Format Version 1.0 Part 3" will be described. Files are managed hierarchically in a directory structure. Create a directory (such as the root directory shown in FIG. 9 ) in the recording medium. Under this directory, files are managed by a recording and reproducing system.

Under the root directory, there is a directory BDAV. As shown in FIG. 9, multiple directories such as BDAV, BDAV1, BDAV2, . . . , BDAVn, etc. may be provided. In the following description, a plurality of directories BDAV, BDAV1, BDAV2, . . . BDAVn are represented by a directory BDAV. Only the representative directory BDAV will be described below.

Under the directory BDAV, there are the following six types of files.

(1) info.bdav

(2) menu.tidx, mark.tidx

(3) menu.tdt1, menu.tdt2, mark.tdt1, mark.tdt2

(4) #####.rpls, #####.vpls

(5)%%%%%.clpi

(6)*****.m2ts

In the files "#####.rpls" and "#####.vpls" classified as (4), "####" represents an arbitrary number. Similarly, in the file "%%%%%.clpi" classified as (5), "%%%%%" represents an arbitrary number. In the file "*****.m2ts" classified as (6), "*****" represents "**" corresponding to the file "%%%%%.clpi" file in a one-to-one relationship ***.m2ts" files. For example, the number "*****" can be the same as the number "%%%%%".

The file "info.bdav" classified as (1) has information of all directories BDAV. The files "menu.tidx" and "mark.tidx" classified as (2) have information of thumbnail screens. The files "menu.tdt1", "menu.tdt2", "mark.tdt1" and "mark.tdt2" classified as (3) have thumbnail screens. The extensions "tdt1" and "tdt2" of these files indicate whether the data of these file thumbnails is encrypted or not.

The files "#####.rpls" and "#####.vpls" classified as (4) are files having playlist information. The files "#####.rpls" and "#####.vpls" are located under the PLAYLIST directory, and the directory PLAYLIST is located under the directory BDAV.

A file "%%%%%.clpi" classified as (5) is a file having clip information. The file "%%%%.%.CLP" is located under the directory CLIPINF, and the directory CLIPINF is located under the directory BDAV. The file "*****.m2ts" classified as (6) is a clip AV stream file having a clip AV stream. This clip AV stream file is related to a clip information file "%%%%%.clpi" having a file name number "******". The file "*****.m2ts" is located under the directory STREAM, and the directory STREAM is located under the directory BDAV.

Next, each file will be described in detail. The file "info.bda" classified as (1) is the only file located under the directory BDAV. The syntax shown in Fig. 10 describes an example of the structure of the file "info.bdav". In this example, the syntax is expressed in the description method of C language, which is used as a program description language for computer devices. This also applies to diagrams in other representation languages.

In FIG. 10, the file "info.bdav" is divided into blocks corresponding to different functions. The field type-indicator describes the string "BDAV", which indicates that the file is "info.bdav". The field version_number describes the version of the file "version_number". Block UIAppInfoBDAV() describes information about files located under directory DBAV. The block TableofPlayList() describes information about the arrangement of the playlist. The block MakersPrivateData( ) describes the private information of the maker of the recording and reproducing apparatus.

An address indicating the start of a single block is described at the beginning of the file "info.bdav". For example, the field TableOfPlayList_Start_address represents the start position of the block "TableOfPlayList()" using the relative number of bytes in the file.

The syntax shown in FIG. 11 describes an example of the structure of the block UIAppInfoBDAV(). The field length indicates the length in bytes following the field length up to the end of the block UIAppInfoBDAV(). BDAV_character_set represents the character set of the character sequence described in the field BDAV_name of the block UIAppInfoBDAV(). As a character setting, ASCII, Unicode, etc. can be selected.

The flag BDAV_protect_flag indicates whether or not the user is allowed to watch the content located under the directory BDAV unconditionally. When the flag is set to "1" and the user enters the correct personal identification number (PIN), he or she is allowed to view the content located under the directory BDAV. On the contrary, when the flag BDAV_protect_flag is set to "0", even if the user does not input his or her PIN, he or she is allowed to watch the content located under the directory BDAV.

The personal identification number PIN is described in the domain PIN. The personal identification number PIN is composed of, for example, 4 Arabic numerals, and each Arabic numeral is from 0 to 9. The personal identification number PIN indicates a personal identification number required when authenticating reproduction control. The Arabic numerals of the PIN are coded according to, for example, the International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) 646 standard.

Using the information described above in the block UIAppInfoBDAV( ), the reproduction restriction of the directory BDAV is specified. As described below, the playback restriction of each playlist is the flag playback_control_flag determined in the block UIAppInfoBDAV() described in the files "#####.rpls" and "#####.vpls" to describe.

In this example, the restart function can be used in order to restart playback of the content under the directory BDAV. The resume function may designate a playlist of content to be reproduced preferentially. It is assumed that the restart function is used when the user wants to restart reproduction of a content from the last stopped position.

In FIG. 11, the flag resume_valid_flag describes whether the resume function is valid. When the value of the flag is set to "0", the restart function is invalid. This restart function is valid when the flag is set to "1". Here, the playlist specified by the field resume_PlayList_file_name is used as the playlist to be reproduced preferentially.

The field ref_to_menu_thumbnail_index describes the area of the thumbnail number that identifies the thumbnail screen with the feature of the directory BDAV. In the Blu-ray Disc standard, a still picture characteristic of a directory BDAV is called a menu thumbnail. The thumbnail screen is a menu thumbnail of the directory BDAV, which has an index thumbnail_index described in the field ref_to_menu_thumbnail_index.

The field BDAV_name_length describes the byte length of the name of the directory BDAV described in the field BDAV_name. The number of bytes described in the field BDAV_name_length is valid for the character string of the field BDAV_name, and the field BDAV_name_length describes the name of the directory BDAV. The rest of the byte sequence after the valid character string described in the field BDAV_name_length may have any value.

The syntax shown in Fig. 12 describes an example of the structure of the block TableOfPlayLists(). The field number_of_PlayLists describes the number of playlists located under the directory BDAV. The number of playlists is a variable loop. The field number_of_PlayLists follows the loop of the "for" statement. The arrangement of playlists described in the field PlayList_file_name in the "for" statement loop indicates the order of playlists displayed on the playlist table display screen or the like. A playlist is specified with a filename such as "#####.rpls" or "#####.vpls" in the field PlayList_file_name.

As mentioned above, the files "#####.rpls" and "#####.vpls" are under the directory PLAYLIST. There is a one-to-one correspondence between these files and individual playlists.

The syntax shown in FIG. 13 describes an example of the structure of the files "#####.rpls" and "#####.vpls". In FIG. 13, either of the files "#####.rpls" and "#####.vpls" has blocks corresponding to function information. The field type_indicator describes a character string representing the file. The field version_number describes the version of this file.

The block UIAppInfoPlayList() describes the attribute information of the playlist. The block PlayList( ) describes information about the play items that make up the play list. Block PlayListMark() describes information about marks added to a playlist. The block MakersPrivateData( ) describes the unique information of the device maker, which has been recorded in the playlist file. The fields PlayList_start_address, PlayListMark_start_address and MakersPrivateData_start_address are located at the beginning of each file "#####.rpls" and "#####.vpls". These fields describe the start address of the corresponding block as 32-bit address information.

Since the start address of each block is described at the beginning of each file "#####.rpls" and "#####.vpls", any length of data padding vocabulary can be placed in front of any block and / or behind. However, the start position of the block UIAppInfoPlayList() is fixed at the 320th byte from the start of each of the above files, and the block UIAppInfoPlayList() is each file "#####.rpls" and "#####.vpls " start block.

The syntax shown in FIG. 14 describes an example of the block UIAppInfoPlayList() structure. The block UIAppInfoPlayList() describes different types of property information about a playlist. This attribute information is not directly used to reproduce PlayList. The field PlayList_character_set describes the character set of character string information on PlayList.

The flag playback_control_flag describes whether display of information and reproduction of playlists are restricted according to a personal identification number PIN. When the value of the flag playback_control_flag is, for example, "1", unless the user inputs a correct personal identification code PIN, information such as a thumbnail screen of the playlist is not displayed, and the playlist cannot be reproduced. The flag write_protect_flag is an anti-erasing flag. It is necessary to construct the user interface so that the user cannot easily erase the playlist when the value of write_protect_flag is "1". The flag is_played_flag describes a playlist that has already been reproduced. The flag is_edited_flag describes an edited playlist.

The field time_zone describes the time zone in which the playlist is recorded. The field record_time_and_date describes the date and time at which the playlist was recorded. The field PlayList_duration describes the reproduction duration of the PlayList.

The fields maker_ID and maker_model_code describe information that can identify the model and maker of the recording device that last updated the playlist. The fields maker_ID and maker_model_code may eg be numbers or the like. The field channel_number describes the channel number of the recorded clip AV stream. The field channel_name describes the channel name. The field channel_name_length describes the length of the channel name described in the field channel_name. In the field channel_name, a character string having the length described in the field channel_name_length is valid. The field PlayList_name describes a PlayList name having an effective length of a value described in the field PlayList_name_length. The field PlayList_detail describes detailed information of a PlayList having an effective length of the value described in the field PlayList_detail_length.

The syntax shown in Fig. 15 shows an example of the structure of the block PlayList(). The field length describes the byte length immediately after the field length until the end of this block PlayList(). The field PL_CPI_type describes the type of feature point information (CPI) of PlayList. The field number_of_PlayItems describes the number of PlayItems constituting the PlayList. The field number_of_SubPlayItems describes the number of PlayItems which are post-recorded audio (SubPlayItems) for adding to PlayList. In short, a playlist can have child playitems when the playlist satisfies a predetermined condition.

The block PlayItem() describes the information of the play item. The block SubPlayItem( ) describes the information of the subplayitem.

The syntax shown in Fig. 16 describes an example of the structure of the block PlayItem(). The field Clip_Information_file_name describes a character string of a file name of a clip information file (the file having the extension clpi) which has a one-to-one correspondence with a clip related to a PlayItem. The clip information file has the extension "clpi".

The field Clip_codec_identifier describes the coding system of the clip to which the PlayItem refers. In the above example, the field Clip_codec_Identifier is fixed to the value "M2TS". The field connection_condition describes information on how a PlayItem is connected to the next PlayItem. In other words, the field connection_condition describes whether the PlayItem can be reproduced without interruption.

The field ref_to_STC_id describes a sequence STC_sequence of a segment, which relates to this PlayItem. The sequence STC_sequence is a unique structure of the Blu-ray Disc standard. This structure represents a range in which the program clock reference (PCR) is continuous on the time base, and PCR is the reference of MPEG2TS (Transport Stream). A unique number STC_id in a segment is assigned to a sequence STC_sequence. In the sequence STC_sequence, since a continuous time base can be determined, the start time and end time of the play item can be uniquely specified. The start point and end point of each PlayItem will exist in the same sequence STC_sequence. The field ref_to_STC_id describes the sequence STC_sequence with number_STC_id.

The fields IN_time and OUT_time respectively describe the time stamps pts (presentation_time_stamp) of the start point and end point of the PlayItem in the sequence STC_sequence.

The block BridgeSequenceInfo( ) describes information about a bridge clip (Bridge_Clip). As shown in FIG. 17, a bridge clip is a bit stream created when implemented for reproducing a PlayItem without interruption. By reproducing the bridge segment instead of reproducing the original bit stream at the boundary of the previous PlayItem and the current PlayItem, two PlayItems can be reproduced without interruption. Since the function of the bridge segment is not closely related to the present invention, the description of this function will be omitted.

The syntax shown in Fig. 18 describes an example of the structure of the block PlayListMark(). The block PlayListMark() has a data structure describing mark information. Flags are structures that describe the time of a playlist. Using a flag, a function of setting a search point to a playlist, a function of dividing a playlist into segments, and the like can be realized. Furthermore, using one flag, it is possible to indicate the display start and display end times on a graphics plane (to be described later).

The field length describes the length in bytes immediately after the field length to the end of the block PlayListmark(). The field number_of_PlayList_marks describes the number of marks in the PlayList. A "for" statement loops through the flags for information. The flag mark_invalid_flag describes whether the flag is valid. When the value of the flag mark_invalid_flag is "0", it indicates that the flag is valid. When the value of the mark_invalid_flag is "1", it indicates that although the mark exists in the database, the mark transmitted to the user is an invalid flag.

The field mark_type describes the type of mark. There is a mark representing the position of one screen of a thumbnail screen (representative screen) as a playlist; a restart mark representing a position where reproduction is resumed; a segment mark representing a search point; a mark representing an area to be skipped and reproduced A jump mark; a mark indicating the reading start time of the graphic image; a mark indicating the display start time of the graphic image; a mark indicating the display stop time of the graphic image, and the like.

The field mark_name_length describes the data length of the field mark_name (to be described below). The field maker_ID describes the maker of the recording device that created the logo. The field maker_ID is used as a unique identifier for identifying a maker. The field ref_to_PlayItem_id describes which PlayItem has the time specified by this flag. The field maker_time_stamp describes the time specified by this flag.

The field entry_ES_PID describes to which elementary stream this flag is added (that is, whether this flag is added to a stream in which picture data and/or sound data are encoded). The field ref_to_menu_thumbnail_index and the field ref_to_mark_thumbnail_index describe a thumbnail screen that visually expresses a mark. The thumbnail picture may be, for example, a still picture, which is extracted at a time designated by a mark.

The field duration is used when a marker has a length on a time base. When using the jump flag, the field duration describes for which duration the jump is performed.

The field maker_information is an area that describes the unique information of the user. The field make_name describes the name assigned to a flag. The size of the mark is described in the field mark_name_length.

Fig. 19 shows syntax describing an example of the structure of the file "%%%%%.clpi". As mentioned above, the file "%%%%%.clpi" is located under the directory CLIPINF. A file "%%%%%.clpi" is created for each AV stream file (file "*****.m2ts"). The file "%%%%%.clpi" has blocks corresponding to function information. The character string described by the field type_indicator indicates the file. The field version_number describes the version of the file.

The block ClipInfo() describes information about the clip. The block SequenceInfo( ) describes information about the discontinuity point of the PCR, which represents the time coordinate of the transport stream of the MPEG2 system. Block ProgramInfo() describes information about a program in the MPEG2 system. The block CPI() describes information about feature point information CPI, which indicates a feature position in the AV stream. Mark information described in the block ClipMark( ) represents a search index point added to a clip and business start and/or end points. The block MakersPrivateData( ) describes the unique information of the maker of the recording device.

Address information representing the start of each block in the file "%%%%%.clpi" is described as fields SequenceInfo_start_address, ProgramInfo_start_address, PCI_start_address, ClipMark_start_address, and MakersPrivateData_start_address. Since the function of the clip file "%%%%%.clpi" is not closely related to the present invention, the description of this file is omitted.

Since the BDAV format has the above-mentioned data structure, the PlayList is composed of PlayItems describing sets of start points and end points of parts to be reproduced in the Clip AV stream, and thus can be managed in reproduction units recognizable by the user. content on the disc.

2. The first mode of the present invention

Next, a first mode of the present invention will be described. According to the present invention, the above-mentioned BDAV format is extended. Movie data and subtitle data are corresponding to the real-time decoding system, and picture data are corresponding to the storage type decoding system, which are provided independently, and these data are combined and displayed. As a result, a user interface with higher flexibility than the prior art can be realized.

Furthermore, the BDAV format is extended so as to be compatible with the playlist specified in "Blu-ray Disc Rewritable Format Version 1.0 Part 3". As a result, expansion of interactive functions is realized. Hereinafter, this extended BDAV format is referred to as BDMV format. The BDMV format is suitable for reproduction-only optical discs (Blu-ray Disc Read Only Memory: BD-ROM).

2-1. About plane

According to the first mode of the present invention, the planar structure shown in Fig. 20 is used. Movie plane 10 is shown at the rearmost (bottom). The movie plane 10 handles a picture (mainly movie data) specified by a playlist. The subtitle plane 11 is displayed on the film plane 10 . The subtitle plane 11 processes subtitle data displayed when reproducing a movie. Graphics plane 12 is displayed on top. The graphics plane 12 relates to character data for menu screens, and relates to graphic data such as bitmaps for keys. The display screen is composed of the above three planes.

The difference between the present invention and conventional DVD-Video is that subpictures for subtitles, menu screens, buttons, etc. are divided into a subtitle plane 11 and a graphics plane 12 in order to independently control subtitles and buttons. As mentioned above, in conventional DVD-Video, graphics such as menu screens, buttons and subtitles are controlled by the same pattern. They are displayed on the same plane. The number of bitmap frames that can be displayed at one time is limited to one. Therefore, in DVD-Video, multiple bitmap pictures cannot be displayed simultaneously. In contrast, according to the present invention, since the subtitle plane 11 and the graphics plane 12 are independently arranged for subtitles and graphics, respectively, the above-mentioned problems with DVDs are solved.

It can be considered that the subtitle plane 11 and the graphics plane 12 are the extended parts of "Blu-ray Disc Rewritable Format Version 1.0 Part 3".

The movie plane 10, the subtitle plane 11 and the graphics plane 12 can be displayed independently. The movie plane 10, the subtitle plane 11, and the graphics plane 12 have the resolutions and display colors shown in FIG. The film plane 10 has a resolution of 1920 pixels×1080 lines, a data length of 16 bits per pixel, and a YCbCr (4:2:2) color system, where Y represents a luminance signal, and Cb and Cr represent color difference signals. YCbCr (4:2:2) has a luminance signal Y of 8 bits per pixel, and color difference signals Cb and Cr of 8 bits each. Two horizontal pixels of the color difference signals Cb and Cr constitute data of one color data.

Graphics Plane 12 has a resolution of 1920 pixels by 1080 lines, a sampling depth of 8 bits per pixel, and a color system selectable from YCbCr (4:4:4) and RGB (4:4:4), where R (red ): G (green): B (blue) = 4:4:4. The subtitle plane 11 has a resolution of 1920 pixels by 1080 lines, a sampling depth of 8 bits per pixel, and a color system with a mapping address of 8 color bits using a palette of 256 colors.

Graphics plane 12 and subtitle plane 11 can be alpha blended in 256 levels. When the graphics plane 12 and subtitle plane 11 are combined with another plane, opacity can be set in 256 levels. The opacity can be set per pixel. In the following description, the opacity α is expressed in the range (0≤α≤1), wherein the opacity α=0 indicates complete transparency; the opacity α=1 indicates complete opacity.

The subtitle plane 11 handles picture data in PNG (Portable Network Graphics File Format) format, for example. Similarly, the graphics plane 12 can handle picture data in PNG format. In the PNG format, the sampling depth of a pixel ranges from 1 bit to 16 bits. When the sampling depth is 8 bits or 16 bits, an alpha channel, that is, opacity information (called alpha data), can be set for each pixel component. When the sampling depth is 8 bits, 256 levels of opacity can be specified. Using the opacity information of the alpha channel, alpha blending can be performed. Paletted images with up to 256 colors can be used. The components (indexes) of the prepared palette can be represented using index numbers.

Furthermore, picture data handled by the subtitle plane 11 and the graphics plane 12 is not limited to the PNG format. Alternatively, picture data that is compression-coded according to a system such as JPEG; picture data that is run-length compressed; or bitmap data that is not compression-coded, can be used.

FIG. 22 shows an example of the structure of three planes combined with reference to FIGS. 20 and 21 . The movie data of the movie plane 10 is supplied to the 422/444 conversion circuit 20 . The 422/444 conversion circuit 20 converts the color system of film data from YCbCr (4:2:2) to YCbCr (4:4:4), and inputs the converted data to the multiplying device 21 .

The picture data of the subtitle plane 11 is input to a palette 22 . The palette 22 outputs RGB (4:4:4) screen data. When the opacity of alpha blending is specified for the picture data, the specified opacity α1 (0≦α≦1) is input from the palette 22 .

An example of input/output data of the palette 22 is shown in FIG. 23 . The palette 22 holds palette information as a table corresponding to, for example, PNG format files. The index number is referenced as an address of the picture data of the 8-bit input data from the palette 22 . Based on the index number, RGB (4:4:4) data each consisting of 8-bit data is output. In addition, the data α of the alpha channel is obtained from the palette 22, which represents opacity.

FIG. 24 shows an example of a palette table that the palette 22 has. The value of the 256-color index is from [0x00] to [0xFF] (where [0x] represents a hexadecimal symbol), the above-mentioned values specify three initial color values R, G and B, and opacity α, which are represented by 8 data bit representation. The palette 22 refers to a palette table from input PNG format picture data, and outputs color data R, G, and B (RGB data) and opacity α of 8 bits per pixel each according to an index value specified by the picture data.

RGB data output from the palette 22 is supplied to an RGB/YCbCr conversion circuit 29 . The RGB/YCbCr conversion circuit 29 converts RGB data into luminance signal Y, color difference signals Cb and Cr (hereinafter collectively referred to as YCbCr data) of 8 bits each. This is because flat data should be combined in a common data format. The data is unified into YCbCr data, which is the data format of movie data.

The YCbCr data and opacity data α1 output from the RGB/YCbCr conversion circuit 29 are input to the multiplying device 23 . The multiplying means 23 multiplies the input YCbCr data by the opacity data α1. The result of the multiplication is input to an input terminal of the adding means 24 . The multiplying means 23 multiplies each of the luminance signal Y, the color difference signals Cb and Cr of the YCbCr data by the opacity data α1. The complement (1-α1) of the opacity data α1 is supplied to the multiplying means 21 .

The multiplication means 21 multiplies the film data input from the 422/444 conversion circuit 20 by the complement number (1-α1) of the opacity α1. The result of the multiplication is input to the other input of the adding means 24 . The addition means 24 adds the multiplication results of the multiplication means 21 and the multiplication means 23 . As a result, the movie plane 10 and subtitle plane 11 are combined. The addition result of the addition means 24 is input to the multiplication means 25 .

Screen data of the graphics plane 12 is input to an RGB/YCbCr conversion circuit 27 . When the color system of the screen data of the graphics plane 12 is RGB (4:4:4), it is converted into YCbCr (4:4:4) and output from the RGB/YCbCr conversion circuit 26 . The YCbCr data output from the RGB/YCbCr conversion circuit 26 is input to a multiplication device 27 .

The picture data used by the graphics plane 12 is in PNG format, and opacity data α2 (0≤α2≤1) can be set for each pixel of the picture data. The opacity data α2 is supplied to the multiplying means 27 . The multiplication means 27 multiplies each of the luminance signal Y, color difference signals Cb and Cr of the YCbCr data input from the RGB/YCbCr conversion circuit 26 by the opacity data α2. The multiplication result of the multiplication means 27 is input to one input terminal of the addition means 28 . A complement (1-α2) of the opacity data α2 is supplied to the multiplying means 25 .

The multiplying means 25 multiplies the addition result of the adding means 24 by the complement of the opacity α2 (1-α2). The multiplication result of the multiplication means 25 is input to the other input terminal of the addition means 28 . The addition means 28 adds the multiplication results of the multiplication means 25 and the multiplication means 27 . As a result, the graphics plane 12 is combined as a result of combining the film plane 10 and the subtitle plane 11 .

When the opacity α of the non-picture area in the subtitle plane 11 and the graphics plane 12 is set to 0 (α=0), the planes below the plane 11 and the plane 12 become transparent. As a result, for example, movie data of the movie plane 10 can be displayed as the background of the subtitle plane 11 and the graphics plane 12 .

The result shown in Fig. 22 can be realized by either hardware or software.

2-2. Menu screen

A screen that guides the user to perform an operation, for example, a menu screen, may be displayed on the graphics plane 12 . FIG. 25 shows an example of the menu screen 60 displayed on the graphics plane 12. As shown in FIG. On the menu screen 60, features and images are displayed at specific locations. The menu screen 60 provides the user with characters and images a Graphical User Interface (GUI) that allows him or her to select a new operation.

"Link" describes a method of accessing a predetermined file having character string or image data. When a user designates character strings or image data on a screen with, for example, a pointing device, he or she can access a predetermined file according to an access method designated by the character string or image data. "Key" can have, for example, three types of image data representing a normal state, a selected state, and a pressed state for "link". When the user designates a key image, the image data is changed according to the state of his or her operation so that he or she can easily recognize the state of the key.

When the user designates a "link" or a "button", he or she moves the cursor on the screen using a tool such as a mouse, and clicks the mouse button on the image of the character string or "link" or on the image of the "button" ( press the mouse button several times). The same operation can also be done with another pointing device other than a mouse. Alternatively, using a remote control commander or keystrokes of a keyboard, the user may assign a "link" or "key." Here, the user selects his or her desired "link" or "key" using a predetermined key such as an arrow key, and designates the selected "link" or "key" using an OK key or the like.

For example, as shown in FIG. 25 , a title 61 as image data is displayed on the upper portion of a menu screen 60 which is displayed on the graphics plane 12 . Below the title 61 are selection items 62A, 62B, 62C and 62D as links. When the user selects and designates one of the selection items 62A, 62B, 62C, and 62D using, for example, a key operation of a remote control commander, a file linked to the designated operation item is accessed.

At the lower position of the menu screen 60, keys 64 and 65 are displayed. Using the keys 64 and 65, subtitles can be displayed, and a language for outputting sound can be selected from, for example, English and Japanese. When the keys 64 and 65 are operated in the above-described manner, a file to be used as a setting screen for display thereof is accessed, and a predetermined screen is displayed.

Also, in the lower left part of the menu screen 60, a character string 63 describing a method for selecting an item is displayed. A character string 63 is displayed on the graphics plane 12 .

To display a screen such as that shown in FIG. 25, an arbitrary description language is required for describing a screen display method, link information, and the like. According to the first mode of the present invention, Hypertext Markup Language (HTML), which has been widely spread on the World Wide Web (WWW), is used as a description language.

As we all know, HTML documents use markup to describe the document structure. A marker consists of a pair of symbols that indicate the start and end of a range. Embed tags into a text to specify an arbitrary range. For example, an opening tag indicating the start of a range is described by surrounding an element determined to be a character string with symbols "<" and ">". Similarly, an end tag indicating the end of a range is described by surrounding the same character string as in the start tag with symbols "</" and ">". where attributes of the element represented by the markup can be described. Closing tags can be ignored. Also, some elements do not need to describe their closing tags.

Documents described in HTML are located in a web server connected to the Internet, and are published to the Internet by the web server. This document can be browsed by browsing software installed in a computer device connected to the Internet such as a personal computer. According to the first mode of the present invention, the HTML standard is uniquely extended in order to display a menu screen for a Blu-ray disc.

For example, on WWW, with link information described in an HTML document, browsing software can link and read another HTML document specified by the link information. On the other hand, it is contemplated that in the menu screen 60 for Blu-ray Disc, a playlist table may be displayed with image data, character strings, keys, and the like. Furthermore, it is contemplated that by specifying a playlist, the specified playlist can be read and reproduced from the optical disc.

The extension of HTML corresponds to, for example, a new function defined in a programming language, or a new application programming interface (API) defined.

2-3. About the script

In the example shown in FIG. 25, a list of playlists is displayed on the menu screen 60. As shown in FIG. In fact, images and sounds of the menu screen 60 and those generated according to items selected on the menu screen 60 are combined into a plurality of playlists. The scenario of branching a story into several branches can be accomplished when multiple playlists making up one menu item are interrelated. When a story is divided into several branches, the multi-story function and the parent function can be realized. The multi-story function makes the content of the story change according to the user's choice, and the parent function makes the scene change according to the user's age.

Although these functions are particularly effective for re-encoded discs, they are not specified in the current Blu-ray Disc standard, whose main purpose is to record/reproduce television broadcasts.

Information displayed on the screen can be described not only in terms of HTML but also in terms of, for example, XHTML (Extended HTML), which is a recent version of HTML 4.0. In addition, the information on the screen can be described in another markup language.

In the following description, a structure of multiple playlists is arranged, which is called a script. FIG. 26 shows an example of the internal structure of the script 70 . The script 70 has a plurality of playlists 73A to 73M. In addition, the script 70 has two parts (screens 80A and 80B) on which the graphics plane 12 is used to display the branch selection screen. For example, screen 80A has graphic data 74A and playlist 73C, which shows a branch selection screen. Similarly, screen 80B has graphic data 74B and playlist 73J, which shows a branch selection screen.

The script specifies the arrangement of the playlist and the time at which the playlist is displayed on the graphics plane 12 . The display time of the playlist on the graphics plane 12 can be specified by a flag on the playlist. How to determine the event handlers 71 and 72 in the script 70 will be described below.

In the example shown in FIG. 26 , the selection item (eg, selection item 62A) on the menu screen 60 corresponds to the playlist 73A indicated by "Enter" in the script 70 . In other words, when the selection item 62A is designated on the menu screen 60, the playlist 73A of the scenario 70 is reproduced. After playback of playlist 73A is completed, playlist 73B is played next. After playback of playlist 73B is completed, playlist 73C is played back. Thereafter, the graphic data 74A is read. As a result, screen 80A is displayed prompting the user to select a story.

After displaying screen 80A, the story is divided into branches according to the user's selection. In the example shown in FIG. 26, when the first selection is made, screen 80A is displayed. Thereafter, the playlists 73D, 73E, 73F are displayed one by one. As a result, the display of the script 70 is completed. After the reproduction of the playlist 73F is completed, the main menu screen (for example, the menu screen 60) may be displayed again.

When the second selection is made on the screen 80A, after the screen 80A is displayed, the play list 73G is reproduced. For example, a flag is set at a predetermined time in the play list 73G. After the playlist 73B is reproduced, the playlist 73G can be branched at the position of the mark, or completely reproduced according to the setting of the reproduction apparatus, another script by the user, or selection on the branch selection screen. After playlist 73G is reproduced, playlists 73M and 73I are reproduced next. Thereafter, the playlist 73J is reproduced.

When the playlist 73G is branched at the position of the mark, the playlists 73K and 73L are played back in succession. After playback of the playlist 73L is completed, playback is restarted at the position of the flag set in the playlist 73I.

In the playlist 73J, the graphic data 74B is read. Screen 80B is displayed, which prompts the user to select a branch of the story. In the first selection of the screen 80B, the playlist 73F is reproduced. In the second selection of the screen 80B, the play list 73K is reproduced starting from the position of the mark set in the play list 73K.

Flags, an input from the user, and player manipulation changes are detected according to an event-driven model. In other words, when reproduction of a playlist is started, the reproduction of a playlist is completed, when reproduction of a playlist is started, or when the user inputs data by key operation of the remote control commander, a flag is detected, and an event occurs . When the program has an event handler that executes on the occurrence of the current event, the player performs the action expected by the event.

The script 70 shown in FIG. 26 has two event handlers 71 and 72 . Wherein, the event handler 71 is an overall event handler, which describes an event handler that is valid for the entire script 70 .

Even if any one of the playlists 73A to 73M of the scenario 70 is being reproduced, when the menu key of the remote control commander is pressed, the menu screen 60 of the scenario table is displayed. In other words, an operation of performing reproduction processing of the playlist of the menu screen 60 will be described. Among them, as an event handler corresponding to an event that occurs when a menu button of the remote control commander is pressed (menu button event), a command to reproduce the playlist of the menu screen 60 is described as the overall event handler 71.

The event handler 72 is a local event handler which is executed only when a predetermined playlist is reproduced or a predetermined user input screen is displayed. For example, when the user specifies a displayed link on the screen 80A as a branch selection screen, an operation for reproducing another playlist is realized by describing a command with respect to the event specified by the link as a local event handler.

In the first mode of the present invention, the above-mentioned event handlers are determined using the scripting language ECMA script. ECMA Script is a cross-platform scripting language based on JavaScript (registered trademark). ECMA scripts are specified by the European Computer Manufacturers Association (ECMA). ECMA scripts have a high degree of affinity with HTML documents and can identify unique objects. Thus, ECMA script is suitable for the first mode of the present invention.

2-4. About the virtual player model

Next, a model of a rendering device provided to the graphics plane 12 and operating in the BDAV format extended with HTML and ECMA script will be described. The model reproduction device is called a BD (Blu-ray Disc) virtual player. The definition of the structure of a BD virtual player is called a BD virtual model.

Next, a BD virtual player model will be described with reference to FIG. 27 . The BD virtual player 30 reproduces data from a Blu-ray disc (hereinafter referred to as an optical disc). The BD virtual player 30 is an object in a computer environment such as a personal computer. The computer environment is not limited to multipurpose personal computers. The computer environment also includes a software environment installed in dedicated reproducing devices and/or recording and reproducing devices that reproduce data from, for example, Blu-ray discs or similar discs.

The BD virtual player 30 roughly has two states A and B. In state A, the BD virtual player 30 reproduces a playlist and graphics. In state B, the BD virtual player 30 stops reproducing playlists and graphics. A command to a certain object of the BD virtual player 30 performs a state change from one state to another, and designates the next operation in a certain state.

State A has multiple operations. As operations in state A, there are fast reproduction, variable speed reproduction such as reverse reproduction, and special reproduction such as skip reproduction, which can be started from any time in the disc. When displaying data of the graphics plane 12, variable speed reproduction and special reproduction of the BD virtual reproduction 30 need to be limited.

The playback control (PBC) program 40 corresponds to a script recorded on the optical disc. As will be described below, the script describes a reproduction method for the playlist recorded on the disc, and a display method for the menu screen. The PBC program 40 and the BD virtual player 30 exchange methods through an application programming interface (API) 41 to reproduce a playlist recorded on an optical disc.

In fact, when the state of the BD virtual player 30 is changed, the PBC program 40 causes the necessary information to be transferred through the API 41 to the common parameter 32 , which is determined as the dedicated memory of the BD virtual player 30 . The common parameter 32 is set by the method of exchanging between the PBC 40 and the BD virtual player 30 through the API 41 by directly using the playback command 31 .

According to the first mode of the present invention, as described above, the BD virtual player 30 is controlled under the event-driven model. When operating the BD virtual player 30, various events occur. The hardware/OS (Operating System) 50 generates an event, for example, when a user operates a key input or operates a remote control commander or a time interruption occurs. The event is sent to the BD virtual player 30 . Optionally, an event may be generated when a marker is detected in a rendered playlist. Further, an event may be generated by the BD virtual player 30 itself, such as detection of a state in which reproduction of the player has changed. Events are handled using the principle of event bubbles.

The type of event that occurs is determined in the BD virtual player model. When an event occurs, the event handler is executed if it has been described. If no event handler is described, the default event handler is executed, and an operation prescribed by the standard for the player is performed.

When a method for player operation is executed, with respect to this method, a return value indicates the result of whether or not the reproduction operation has normally started. On the other hand, using only the fact that an event occurred, one gets the end time, at which time a command is completed. Therefore, when a method for starting reproduction of the end portion of the script is described in the script program, it is preferable to describe the method together with an event handler that receives an event handler that receives the end of the script and executes it. .

Event handlers can be different in each script for the same event.

Interruption events of the BD virtual player 30 are roughly classified into (1) events occurring in content being reproduced; (2) events occurring by user's interruption; (3) events occurring due to a change in the state of the player.

Event (1) occurs in content being reproduced, and is a scheduled interruption. Event (1) occurs simultaneously whenever content is reproduced. When the BD virtual player 30 reproduces in a playlist, a marker detection interrupt occurs in the BD virtual player 30 when the time specified by the marker described in the playlist has elapsed on the disc. When a script specifies a timer, the timer interrupt event occurs at that specified time, or 10 seconds after the script specified timer set time.

Event (2) is a user interrupt whose occurrence and timing are unpredictable. An interrupt event occurs when the user operates a key of the remote control commander. In this case, since it cannot be determined when the user performs the key operation, the time cannot be predicted in advance.

Event (3) occurs due to a state change of the BD virtual player 30 . Event (3) notifies a change in the audio stream or subtitles. Occurs when the state of the player transitions from the playing state to the stopped state, or vice versa, for a content. An event caused by a change in the player's state can occur either with event (1), which occurs within the content being rendered, or within event (2), which occurs due to User interrupt event. As an example of an event notifying a change of a sound stream or a subtitle, when an interruption event of a key operation of a user's remote control commander occurs, the sound stream or a subtitle is changed. As a result, since the state of the BD virtual player 30 has changed, an event notifying the state change occurs.

In the first mode of the present invention, display control using the graphics plane 12 is described in an HTML (Hypertext Markup Language) 4.01 document or an XHTML (Extended HTML) document. Use HTML4.0 built-in events, and events for display screens for Graphics Plane 12. If you need events other than HTML4.0 built-in events, you can use ECMA script to describe them.

When HTML format and ECMA script are used in combination, if an event occurs, a process is performed depending on whether an event handler exists in the document, which is specified with the element attribute of the event. When an event handler exists, it is executed. Determines whether an overall event handler exists in the document when no event handler exists. As a result, if an overall event handler exists in the document, that event handler is executed. When the document does not describe an event handler in the script language, the BD virtual player 30 executes a default event prepared for the event.

An event handler can be described as an attribute of an element of an HTML document, or as an event-catching method of an ECMA script.

Next, a method for describing an event handler using an HTML document will be described. Built-in events described in HTML4.0 may be used, for example, event onload, event onunload, event onclick, and event onkeypress. Each of these events is described as an attribute in the tagged element.

The onload event occurs when the user agent finishes a window or all frames identified by the paired <FRAMESET></FRAMESET> tags. The event onload occurs when the menu screen is shown.

A window is a unit in which a browsing application displays an HTML document according to an HTML description. Use frames to display multiple HTML files in divided areas of a window. HTML files within frames and frames themselves are called frames. The event onload attribute can be used with element BODY and element FRAMESET.

The event onunload occurs when the user agent moves the HTML document away from a window or a frame. The event onunload attribute can be used with element BODY and element FRAMESET.

The event onclick occurs when an element is positioned with a spotter or similar. For example, the event onclick occurs when a mouse button click operation is performed. The event onclick attribute can be used with any HTML 4.0 element.

The event onkeypress occurs when a key is pressed or released from an element. For example, the event onkeypress occurs when a predetermined key is pressed on the keyboard, or a key of the remote control commander is pressed in an area determined by a particular element on the screen, and the key is in the selected state. The event onkeypress attribute can be used with any HTML 4.0 element.

In the first mode of the present invention, since the operation of the BD virtual player 30 cannot be sufficiently controlled with the above-mentioned events of HTML, it is necessary to determine an initial event. FIG. 28 shows an example of an initial event determined in the BD virtual player 30. As shown in FIG. Control events with ECMA scripts in HTML documents. As an attribute name specifying an event handler, "on" is added to the beginning of the event name.

The event TimerFired occurs when the value of the countdown timer is "0" or when the value of the countdown timer reaches a predetermined value. When reproduction is stopped or paused, the event PlayStopped and the event PlayStilled occur. The event StillReleased occurs when the suspended state is released. The event PlayPaused and the event PauseReleased occur when the user temporarily stops reproduction and when the user reproduces the paused state of reproduction. The event PlayStarted occurs when playback starts. The event PlayRepeated occurs when a region to be reproduced repeatedly is detected.

The event SPDisplayStatusChanged occurs when the display/non-display status of the sub-picture (subtitle) stream is changed. When the audio stream and the video stream to be reproduced change, the event SelectedAudioChanged and the event VideoStopped occur respectively.

Event ScenarioStarted and event ScenarioEnded occur when the start and end of the script are detected, respectively. When the start and end of the playlist are detected, the event PlayListStarted and the event PlayListEnded occur, respectively. When the start and end of a PlayItem are detected, the event PlayItemStarted and the event PlayItemEnded occur, respectively.

The event MarkEncountered occurs when a mark is detected while a playlist is being reproduced. This event is used when image data is displayed on the graphics plane 12, for example. The number of types of flags detected is specified in common parameter 32 .

The event ButtonPressed occurs when a button located on the screen is pressed. For example, when a key on the graphics plane 12 is virtually pressed by a mouse key operation or click operation, an event ButtonPressed is generated.

The event ValidPeriodStarted occurs when a valid period is started. A link can be selected for a valid period, when the period is specified, the event ValidPeriodStarted can be used. When the valid period ends, the event ValidPeriodEnded occurs. This event is used when the link is enforced.

The event KeyPressed occurs when a key of the remote control commander is pressed. Use the event handler's "transition" state, or similar, to identify the type of key that was pressed.

2-5. About the method

Figure 29A, Figure 29B, Figure 30A, Figure 30B, Figure 30C, Figure 30D, Figure 31A, Figure 31B, Figure 31C and Figure 31D show an example of the method that the BD virtual player 30 has, which is in the first embodiment of the present invention. determined in a mode. Through these commands, the operation and state of the BD virtual player 30, and restoration and control information about video streams and audio streams, as well as subpictures (image data on the subtitle plane 11), operations of the common parameters 32, for Programs for timer and key input interrupts, and control of screen data handles on the graphics plane 12.

When these methods are used in the PBC program, the operation of the BD virtual player 30 can be controlled. These methods are structured in API41 shown in FIG. 27 . According to the PBC program 40, these methods are accessed through the API 41 . According to these methods, reproduction of an optical disc for the BD virtual player 30 is controlled. A real example of the PBC program 40 will be described below.

Next, methods for player operations will be described. The method playScenario(scenarioNumber, [scenarioTime]) reproduces the scenario specified by "scenarioNumber". "scenarioNumber" is a URI (Universal Resource Identifier) indicating the location of a file describing the structure of the scenario. The method playPlayList(playListNumber) causes the playlist specified by "playPlayList" to be reproduced. The method playChapterMark(playListNumber, chapterNumber) reproduces a playlist specified by "playListNumber" from the chapter specified by "chapterNumber". The method playPlayItem(playListNumber, playListNumber) specifies a playlist from the play item specified by "playItemNumber" by "playListNumber". "playListNumber" is "playItem_id". When the value of "playItem_id" is designated as "0", playitems belonging to one playlist are reproduced from the beginning.

The method play(position)(object) moves the current position to an adjacent playlist or playitem. The parameter "position" is any one of "prev", "next", "top", "goUp", and "tail". The parameter "object" describes a move method to move an object (playlist, playitem or chapter).

The method stop() stops the reproduction of the script. In this case, the values of the standard registers are not preserved. The method resume() resumes reproduction from the last stopped position. The method playSoundEffect() reproduces a selected effect sound.

Next, methods for player state will be described. In Figure 30A, Figure 30B, Figure 30C, Figure 30D, the method getMenuDescriptionLanguage() gets the language of the displayed menu. The method getScenarioNumber( ), method getPlayListNumber( ) and method getChapterNumber( ) respectively obtain a scenario number, playlist number and chapter number being reproduced.

Next, a method for video streaming will be described. The method setVideoStreamNumber() describes the video stream to be decoded. The method getVideoStreamNumber(), method getVideoStreamStatus, and method getVideoStreamAttr() respectively obtain the video stream number, status, and attributes of the video stream being reproduced. Attributes of the video stream are, for example, encoding system, resolution, aspect ratio, display mode in the case of an aspect ratio of 4:3, and display/hiding of closed captions. The method setAngleNumber() describes the angle number; the method setAngleNumber() gets a selected angle number. The method getMaxVideoStream() gets the maximum number of video streams.

Next, the audio stream will be described. The method getAudioStreamAvailability() gets a message describing whether to obtain the specified audio stream included. The method getAudioStreamLanguage() gets information about the language of the specified audio stream. The method setAudioStreamStatus() gets the status of the specified audio stream. The method getsetAudioStreamStatus() specifies the status of the specified audio stream. The status of an audio stream can be, for example, whether to reproduce or not. The method setAudioStreamAttribute() gets the attributes of the specified audio stream.

Next, a method for a sub-picture stream (subtitle data) will be described. The method getSPStreamAvailability() gets an information describing whether a specified sprite stream is included. The method getSPStreamLanguage() gets the language used in the specified sprite stream. The method getSPDisplayStatus() gets the display status of the sub-picture stream. The method setSPDisplayStatus() describes the display status of the sub-picture stream. The display state of the sub screen stream is, for example, whether to display the sub screen stream or the like. The method getSPStreamAttribute() is to specify the attribute of the sprite stream. The attribute of the sub-picture stream is, for example, whether the sub-picture stream is displayed in an aspect ratio of 4:3 or in a wide screen.

Next, a method for the common parameter 32 will be described. In Fig. 31A, Fig. 31B, Fig. 31C, Fig. 31D, the method for the common parameter 32 is shown as the method for register read/write. The method clearReg() initializes all registers of the BD virtual player 30 storage area. The method segReg() sets a value of the specified register. The method getReg() reads a value from the specified register.

Next, the method for the timer will be described. The method sleep() causes the process to stop at the specified time. The method setTimeout() executes a function or process after the specified time has elapsed. The method setInterval() executes a process at the specified time interval. Timer methods can be specified in milliseconds. The method clearTimer() stops a process with the specified record timer. The method pauseTimer() temporarily stops a timer with the specified record ID. The method resumeTimer() restarts a timer with the specified record ID from the paused state.

As a method for keyboard input, the method getPressedKey( ) gets the type of a key input (pressed).

Next, a method for graphics will be described. The method loadGraphics(htmlfile, ID) reads a file specified by "htmlfile" and expands the file to graphics plane 12 in the non-display state. "ID" is assigned to an extended graphic image, and is referenced by a method which will be described later. The method showGraphics(ID) displays an image extended to the graphics plane 12 by the above-mentioned method load graphics(htmlfile, ID). The method hideGraphics(ID) hides the image specified by "ID".

Next, other methods will be described. The method random(input number num) generates a random number from 1 to "num". Random numbers are generated by unique definitions. The method catchEvent(eventname, eventhandler) executes the function specified by "eventhandler" when the event specified by "eventname" occurs.

2-6. Execution of commands

Next, the PBC program 40 will be actually described. In the first mode of invention, a script file is created for a script. When displaying the menu screen 60 on the graphics plane 12, one HTML file is created for one screen. Script files and HTML files have extensions "js" and "html" respectively. These extensions distinguish the two types of files. The file of the script program has a fixed file name, such as "starup.js", and when a CD is loaded into the drive device, the file starts to be executed.

Next, as an example, an optical disk having a script structure shown in FIG. 32 will be considered. This disc has two scenarios Scenario000 and Scenario001. The script Scenario000 displays a menu screen 91 with a link button to the scenario Scenario001. When an optical disc is loaded in the reproducing apparatus, a menu screen 91 is displayed. When the button 92 on the menu screen 91 is clicked, the scenario Scenario001 is reproduced. After the scenario Scenario001 is reproduced, the menu screen 91 is displayed again.

Fig. 33 shows an example of a file list necessary for the structure shown in Fig. 32 . In this example, six files are required: file "startup.js", file "Scenario000.js", file "000.html", file "00000.rpls", file "Scenario001.js", file "00001.rpsi ".

Among these files, the file "Scenario000.js" is a script file describing the structural information of the scenario Scenario000. The file "Scenario000.js" describes the structural information of the menu screen 91, that is, the scenario list screen. The file "000.html" is an HTML file describing layout information of the menu screen 91 . The file “00000.rpls” is a playlist file displayed as the background of the menu screen 91 . The file "Scenario001.js" is a script file describing the structural information of the scenario Scenario001. The file "00001.rpls" is a playlist file that describes information of a playlist reproduced according to the scenario Scenario001.

In FIG. 33, the content files reproduced based on the playlist files 00000.rpls" and 00001.rpls" (the click information file "%%%%%.clip" and the AV stream file "*****.m2ts") are omitted. ").

Each file shown in FIG. 33 is recorded on the optical disc according to the directory structure shown in FIG. 34 .

35 to 38 show actual examples of script files and HTML file descriptions. Fig. 35 is an example of description of the file "startup.js". The file "startup.js" determines the number of scripts and their names recorded on the disc. The method playScenario("scenario000") accesses the file "scenario000.js" and reproduces the scenario Scenario000. When the scenario Scenario000 is reproduced, the menu screen 91 is displayed.

Fig. 36 is an example of description of the file "scenario000.js". When a function is determined with "on" (see FIG. 28) and an event describing the method catchEvent() occurs, the content determined in the function is executed. In the example shown in FIG. 36, the HTML file "000.html" describing the layout information of the menu screen 91 is specified. The reading timing and the display timing of the display of the menu screen 91 to the graphics plane 12 are controlled. Also, the method playPlayList("00000.rpls") reproduces the playlist "00000.rpls".

In other words, the file “scenario000.js” causes the movies of the playlist “00000.rpls” to be displayed on the movie plane 10 . Also, the file "scenario000.js" displays the menu screen 91 on the graphics plane 12 when a marker is detected while displaying the playlist "00000.rpls".

Fig. 37 is an example of description of the file "000.html". In a portion surrounded by tags <style type="text/css"> and </style>, layout information in the display screen 91 for images referred to by "menu" and "scenario000" is described. In the example shown in Fig. 37, layout information is described in absolute coordinates on the screen. The image data referenced with the image name "menu" is displayed as an image with a width of 200 pixels and a height of 50 pixels, and its position is 200 pixels from the upper part of the screen and 800 pixels from the left end of the screen. Similarly, the image data referenced by the image name "scenario" is displayed as an image of 400 pixels in width and 100 pixels in height positioned at 700 pixels from the top of the screen and 700 pixels from the left end of the screen.

In the part surrounded by tags <script type="text/javascript"> and </script>, each event handler onMoverhander(f), onMounthander(f), onMclickhander(f) for mouse operation is determined. In the example shown in FIG. 37, the image data "201.png", "200.png", and "202.png" which are button images are associated with event handlers onMoverhander(f), onMounthander(f), onMclickhander(f) couplet. Also, the event handler onMclickhander(f) reproduces the script file "scenario001.js".

In a portion surrounded by tags <body> and </body>, image data on the graphics plane 12 of the menu screen 91 is described. Describes the file name ("100.png", "200.png") of the image data corresponding to the described in the section surrounded by tags <style type="text/css"> and </style> image name. When the image data events onMover, onMount, and onclick referred to by the image name "scenario000" occur according to the operation of a pointing device such as a mouse, the event handlers onMoverhander(f), onMounthander(f), onMclickhander(f) are respectively executed .

The event onMouseover occurs when the cursor is in the specified area. The event onMouseout occurs when the cursor leaves the specified area. The event onclick occurs when a predetermined operation occurs and the cursor is located in a specified area, and the predetermined operation may be a click operation of a pointing device such as a mouse.

Fig. 38 is an example of description of the file "scenario001.js". The function UOPControl() determines the operation when the script file "scenario001.js" is being reproduced. If the remote control menu key is pressed, the scenario Scenario000 for displaying the menu screen 91 is reproduced. The function playListEnded() determines an operation when the playlist of the scenario file "scenario001.js" has been reproduced, and the scenario scenario000 for displaying the menu screen 91 is reproduced. The script file "scenario001.js" reproduces the playlist "00001.rpls".

Next, the operations shown in FIGS. 35 to 38 will be described. When the disc is loaded into the playback device, the file "startup.js" is read from the disc. Access the file "scenario000.js" from the file "startup.js". When the script "scenario000" described in the file "scenario000.js" is executed, the movies of the playlist "00000.rpls" are displayed on the movie plane 10 according to the script shown in FIG. 36 .

The file "000.html" is accessed according to the time corresponding to the mark described in the playlist "00000.rpls". According to the description of the file "000.html", a menu screen 91 displaying a script form is developed on the graphics plane 12 and displayed. The menu screen 91 also includes a scenario "scenario000".

On the menu screen 91, there are an image file "100.png" of a character string such as "Menu", and an image file "200.png" of a character string such as "Scenario001". The above-mentioned image file is located in graphics plane 12, and the above-mentioned character string is displayed. In the movie plane 10 displayed as the background of the graphics plane 12, movies of the playlist "00000.rpls" are displayed. The movie of the playlist "00000.rpls" on the movie plane 10 and the menu screen 91 of the file "000.html" on the graphics plane 12 are overlapped and displayed on the same screen. As a result, the menu screen 91 is displayed together with the movie background.

Here, a predetermined opacity is assigned to one screen of the graphics plane 12 (menu screen 91). The menu screen 91 may be transparently displayed on the movie of the movie plane 10 . In this example, flags are described at the beginning and end of the playlist "00000.rpls". When the playlist "00000.rpls" is reproduced, the menu screen 91 is displayed. After reproducing the playlist "00000.rpls", the menu screen 91 is cleared.

On the menu screen 91, a cursor is displayed, which can be moved by a key operation of the user's remote controller. When the cursor overlaps with the image file "200.png", the event Mouseover specified by the file "000.html" occurs. When the event Mouseover occurs, the event handler onMovehandler( ) corresponding to the event onMouseover is executed to indicate the state of focusing on the image file "200.png". When the handler onMovehandler() is executed, the image file "200.png" is replaced with the image file "201.png". The image file "201.png" is a button image or similar, and its color is different from the image file "200.png".

When the cursor is located at the image file "201.png", if the user performs a click operation with a predetermined key of the remote control commander, an event handler onMovehandler( ) corresponding to the event onclick is executed. As a result, the image file "201.png" is replaced by the image file "202.png", indicating the state that the image file "201.png" has been selected. The image file "202.png" is a key image that virtually represents a state that a key has been pressed.

When the event handlers corresponding to the events "focused" and "clicked" are described in the file "000.html", a menu screen with an interactive function that responds to the user's input is completed.

When the button image "Scenario0001" on the menu screen 91 is clicked, the reproduction program for the scenario "Scenario0001" is executed. When the file "scenario001.js" is executed, the scenario "scenario001" is reproduced. As shown in FIG. 38, the method playPlayList("0001.rpls") described in the file "Scenario001" is accessed. As a result, the playlist "00001.rpls" is reproduced.

After the playlist "00001.rpls" is reproduced, a playlist reproduction end event PlayListEnded() is generated. The event handler playScenario(scenario000.js) corresponding to this event reproduces the scenario "scenario000.js". In this example, after the scenario "Scenario001" is reproduced, the menu screen 91 is displayed again.

After the scenario "Scenario001" is reproduced, even if the key specified by "keyID" is operated, the scenario "scenario000.js" is reproduced, and the menu screen 91 is displayed.

The descriptions of HTML and ECMA script shown in Fig. 35 to Fig. 38 are just examples. In other words, the current invention is not limited to the examples described above. HTML and ECMA script have flexibility in their description. Thus, similar operations can be accomplished even if HTML and ECMA scripts are partially changed.

2-7 Decoder Model

FIG. 39 shows an example of the structure of the player decoder 100 according to the first embodiment of the present invention. The player decoder 100 interprets data copied from an optical disc loaded in a drive (not shown), outputs an AV stream, and allows a user to interactively manipulate the output AV stream.

A CPU (Central Processing Unit) (not shown) controls all operations of the operator decoder 100 . The CPU monitors and controls the stream and data flow of the individual parts of the player decoder 100 .

When the disc is loaded into a drive device (not shown), the file "startup.js" is reproduced as described above. Read a script file as described in the file "startup.js". According to the description of the script file, an HTML file, the movie data displayed on the movie plane 10, the image data displayed on the subtitle plane 11 and the graphics plane 12, another script file accessed by the aforementioned script file are read from the CD , and sound data.

In the above description, among the data read out from the optical disc, such as streams of movie data, sub-picture (subtitle data) and sound data are called real-time streams, which need to be processed continuously. In contrast, non-real-time data such as script files, HTML files, image data files, and playlist files are called storage objects, which need not be processed continuously. Storage objects are stored in memory or the like, and are expanded therein. Dispose of storage objects when needed.

The player decoder 100 has two systems whose input channels are channel(1) and channel(2). The storage object is input to input 101 of channel (1). A real-time stream is input to input channel 202 of input channel (2). Optionally, storage objects may be input to input 202 . According to a first mode of the invention, real-time streams and partially stored objects are input to the input 202, eg MPEG2TS.

The real-time stream input to the input channel 202 is not limited to MPEG2TS. A stream in another format may be input as long as the real-time stream can be transmitted in units of packets and multiplexed with video data, audio data, still picture data, and the like. Here, a PID filter 110 to be described later is used as a separation signal circuit that separates video data, audio data, still picture data, and the like.

When the rotational speed of the optical disc in the drive is increased, for example, by a factor of two, the read transfer speed of the optical disc is increased, on a time-division basis, from the optical disc to perform two-system read operations of the channel (1) and (2) drives.

First, the system of the input channel (1) will be described. A storage object of the input channel (1) is input to the switch circuit 102 . The switch circuit 102 selects the output terminal 102A when a program code such as an ECMA script, an HTML file, etc. to be stored is input. The input program code is stored in the code buffer 104 .

When image data to be stored is input, the switch circuit 102 selects the output terminal 102B. As a result, the input image data is input to the switch circuit 103 . When the real-time stream input to the input terminal 202 does not include image data displayed on the subtitle plane 11 or the graphics plane 12, the switch circuit 103 selects the input terminal 103A. Image data input from the switch circuit 102 is stored in the content buffer 105 .

Similarly, when image data displayed on the subtitle plane 11 or the graphics plane 12 is included in the real-time stream input to the terminal 202, the switch circuit 103 selects the input terminal 103B. As a result, image data is stored in the content buffer 105 . The storage objects stored in the code buffer 104 and the content buffer 105 are read out when necessary, and provided to the multimedia engine 106 .

Image data to be stored stored in the content buffer 105 is also supplied to the sub picture decoder 116 and the still picture decoder 117 through the switch circuits 107 and 108, respectively.

The multimedia engine 106 includes an XML parser (parser) 106A, a script translator 106B, and a graphics translator 106C. The multimedia engine 106 may consist of independent hardware. Alternatively, the multimedia engine 106 may be realized by processing of a predetermined program executed by a CPU (not shown).

The XML parser 106A has a function of parsing XML (Extensible Markup Language) documents. In addition, XML parser 106A can also parse HTML language. The HTML document parsed by the XML parser 106A is converted into a format that the player decoder 100 can execute. The script translator 106B analyzes an ECMA script and converts it into a format that the player decoder 100 can execute. Graphics translator 106C decodes the image data to obtain a format that can be expanded on subtitle plane 11 and graphics plane 12 .

The multimedia engine 106 executes the processing of the XML parser 106A, the script translator 106B, and the graphics translator 106C using the working memory of the buffer 109 . For example, the XML parser 106A and the script translator 106B use the code buffer 109 a of the buffer 109 . Graphics translator 106C uses graphics buffer 109D of buffer 109 . The buffer 109 further includes a font buffer 109B, which stores font data for displaying character strings, and a tree buffer 109C, which stores the parsed results of the HTML document by the XML parser 106A in a hierarchical tree structure.

The multimedia engine 106 reads, for example, an ECMA script from the code buffer 104 . When necessary, the multimedia engine 106 reads another ECMA script and HTML document from the code buffer 104 and image data from the content buffer 105 according to the description of the ECMA script. Data stored in the code buffer 104 and the content buffer 105 may be stored in the code buffer 104 and the content buffer 105 until the data is no longer needed. Thus, the data stored in the code buffer 104 and the content buffer 105 can be repeatedly read out when necessary.

Furthermore, the multimedia engine 106 performs separation processing for various types of input data, JAVAvm (Java (registered trademark) virtual machine) functions, and the like. Also, the multimedia engine 106 receives a user's input from a remote control commander, a pointing device, and the like, and executes a process according to the user's input. The user's input is provided to sub picture decoder 116, still picture decoder 117 and audio decoder 118, MPGE video decoder 120, system decoder 121, which will be described later.

The image data processed by graphics translator 106C are supplied to sub-picture plane 132 and still picture plane 134 through switching circuits 130 and 131, respectively. In addition, image data is supplied to the graphic display 133 and the M/S conversion plane 136 . In this example, it is assumed that the image data supplied to the sub picture plane 132 and the still picture plane 134 have PNG format. The multimedia engine 106 controls when image data is provided to the faces 132 , 133 , 134 and 135 . The M/S conversion plane 136 controls the switch circuit 137 . The switch circuit 137 selects still picture data output from the still picture plane 134 (to be described later), and movie data output from the movie plane 135, and supplies the selected data to a downstream stage.

The sub-picture plane 132 and the graphics plane 133 correspond to the aforementioned subtitle plane 11 and graphics plane 12, respectively. The still picture plane 134 and/or the movie plane 135 correspond to the movie plane 10 described above. Each sprite plane 132, graphics plane 133, still plane 134, movie plane 135 is composed of, for example, frame memories.

The multimedia engine 106 also provides control signals that cause the still plane 134, movie plane 135, sprite plane 132 and graphics plane 133 to be selected to the presentation processor 139, which will be described later. Similarly, the multimedia engine 106 provides a control signal that controls the output of the audio stream at the presentation processor 141, which will be described later.

Next, the system of the input channel (2) will be described. The real-time stream input to the input terminal 202 as MPEG2TS is supplied to the PID filter 110 . The PID filter 110 extracts a PID (Packet Identification) from the MPEG2TS transport stream, and detects the attribute of the stream included in the transport packets. The PID filter 110 divides the input real-time stream into the corresponding system for each transport packet according to the property of the stream.

When a transfer packet includes image data of a storage target, the transfer packet is temporarily stored in the buffer TBn 111A. The transport packet is read out at a predetermined timing, and input to the switch circuit 103 through the selected input terminal 103B. Thereafter, the transfer packet is stored in the content buffer 105 through the switch circuit 103 .

When the PID filter 110 has determined that the transport packet includes subpicture data, that is, subtitle data, the transport packet is temporarily stored in the buffer TBn 111B and the buffer Bn 112B according to the PID. The transport packet is read out at a predetermined time, and input to the switching circuit 107 through the selected input terminal 107B. The transport packet is supplied to the sub-picture decoder 116 through the switch circuit 107 .

The sub-picture decoder 116 eliminates header information from the delivered transport packet, decodes subtitle data included in the transport packet, and obtains image data for subtitles. This image data is input to the input terminal 130B of the switch circuit 130 and expanded to the sub-picture plane 132 through the switch circuit 130 .

When the PID filter 110 determines from the PID that the transport packet has picture data, the transport packet is temporarily stored in the buffer TBn 111C and the buffer Bn 112C. The transport packet is read out at a predetermined time, and input to the switch circuit 108 through the selected terminal 108B. The transfer packet is supplied to the still picture decoder 117 through the switch circuit 108 .

The still picture decoder 117 eliminates the header information from the delivered transport packet, decodes the still picture data included in the transport packet, and obtains the still picture data. This still picture data is input to the input terminal 131B of the switch circuit 131 at a predetermined timing, and is expanded to the still picture plane 134 through the switch circuit 131 .

When the PID filter 110 determines from the PID that the transport packet has audio data, the transport packet is temporarily stored in the buffer TBn 111D and the buffer Bn 112D. The transport packets are read out at a predetermined time, and supplied to the audio decoder 118 . Audio data included in transport packets is compression-coded according to, for example, an MPEG-based system.

The audio decoder 118 also has, for example, a linear PCM (Pulse Code Modulation) audio decoder 119 . The audio decoder 118 eliminates header information from an input transport stream, decodes compression-coded audio data included in transport packets, and obtains linear PCM audio data.

The linear PCM audio data is output from the audio decoder 118 and input to the presentation processor 141 for audio. In the presentation processor 141 , under the control of the multimedia engine 106 , a sound effect is added to the linear PCM audio data and the data is obtained from the output terminal 142 .

When the PID filter 110 determines that the transport packet includes film data according to the PID, the transport packet is temporarily stored in the buffer TBn 111E, the buffer MBn 113, and the buffer EBn 114, and the transport packet is read out at a predetermined time and provided to MPEG video decoding device 120. The movie data included in the transport package is compression-encoded according to the MPEG2 system.

The MPEG video decoder 120 removes the header information from the delivered transport packet, decodes the movie data compressed and coded according to the MPEG2 system, and obtains the baseband movie data.

The film data output from the MPEG video decoder 120 is input to the input terminal 124A of the switch circuit 124 . In addition, movie data is input to the input terminal 124B of the switch circuit 124 through the buffer 123 . In the switch circuit 124, the input terminals 124A and 124B are selected at a predetermined timing. The output movie data is developed on the movie plane 135 .

When the PID filter 110 determines from the PID that the transport packet includes system information, the transport packet is supplied to the system decoder 121 through the buffers TBn 111F and Bsys 115 . The system decoder 121 eliminates header information from the submitted transport packet, and extracts system information therefrom. This system information is provided to, for example, a CPU (not shown).

The image data in the sprite plane 132 is provided to a palette 150, which corresponds to the palette 22 described above. The palette has 256 colors. Palettes refer to an index. Output YCbCr data. In addition, opacity data α1 is extracted. The YCbCr data and the opacity data α1 are supplied to the presentation processor 139 .

Image data on the graphics plane 133 is supplied to the RGB/YCbCr switch circuit 151 , which corresponds to the RGB/YCbCr switch circuit 26 . The RGB/YCbCr conversion circuit 151 converts the color system from RGB (4:4:4) to YCbCr (4:4:4). The YCbCr data output from the RGB/YCbCr conversion circuit 151 is supplied to the presentation processor 139 . Furthermore, opacity data α2 is extracted from the image data on the graphics plane 133 . The extracted opacity data is provided to the presentation processor 139 .

Image data on the still picture plane 134 is input to the input terminal 137A. In addition, the movie data on the movie plane 135 is input to the input terminal 137B of the switch circuit 137 . The image data on the still picture plane 134 or the movie data on the movie plane 135 are selected by the switch circuit 137 on the M/S conversion plane 136 . In other words, the still plane 134 is treated like the movie plane 135 . The output of the switching circuit 137 is supplied to a presentation processor 139 through an up/down converter 138 .

The up/down converter 138 is a circuit that converts image resolution. For example, the up/down converter 138 converts a high definition (HD) image having a high resolution to a standard definition (SD) image having a standard resolution.

The presentation processor 139 performs an α blending process using the opacity α1 of the image data of the subtitle plane 11 (sub-picture plane 132) and the opacity α2 of the graphics plane 12 (graphics plane 133) shown in FIG.

In other words, the representation processor 139 combines the image data of the still picture plane 134 or movie plane 135 selected by the switch circuit 137 with the image data of the sub-picture plane 132 according to the opacity α1 of the image data set to the sub-picture plane 132. combination. In addition, the representation processor 139 combines an image with the image data of the graphics plane 133 according to the opacity α2 in the image data which has been set to the graphics plane 133, the above-mentioned first image data being generated by the still picture plane 134 or the movie plane 135. and sprite plane 132 combined. An image data obtained from the output terminal 140 is obtained by combining the image data of the graphics plane 133, the image data (subtitle data) of the sub-picture plane 132, the image data of the still picture plane 134 or the image data of the movie plane 135.

The presentation processor 139 may perform an effect processing on the image data on a real-time basis.

In the above description, each part of the player decoder 100 is composed of hardware. However, the present invention is not limited to this example. For example, the player decoder 100 can be realized by processing of software. In this case, the player decoder 100 can be realized on one computer device. Optionally, the decoder 100 may be implemented by a combination of hardware and software. For example, audio decoder 118 and MPEG video decoder 120 may consist of hardware. Other parts of the player decoder 100 may consist of software.

A program can make a computer device execute the player decoder 100. The player decoder 100 can be made up of only software, or a combination of hardware and software, and the program is recorded in, for example, a CD-ROM (CD-ROM) etc. on the recording medium and supplied with it. The CD-ROM is loaded into the CD-ROM drive of the computer device. The program recorded on the CD-ROM is loaded into the computer device. As a result, the above-described processing can be executed on a computer device. Since the structure of this computer device is well known, its description is omitted.

2-8 User Interface

Next, a user interface according to one embodiment of the present invention will be described. FIG. 40 shows an example of state change of keys displayed on the graphics plane 12. As shown in FIG. There are two key display states: a key display state in which a key is displayed on the screen, and a key non-display state in which no key is displayed on the screen. The key non-display state changes to the key display state. After the key is cleared, the key display state changes to the key non-display state. The button display state further includes three states: normal state, selection state and execution state. The key display state can be changed among these three states. The key display state can be changed in one of these three states.

Next, referring to FIG. 25 , the state change of the key display state will be described in detail. The menu screen 60 is displayed when a disc is loaded into the player or when the user presses the menu key of the remote controller. While the menu state 60 is displayed, the keys 62A, 62B, 62C, 63D, 64 and 65 change from the non-display state to the state. Typically, one of the keys 62A, 62B, 62C, 63D, 64 and 65 is already selected. Now, assume that the key 62A is already in the selected state, and the other keys are in the normal state.

When the user operates, for example, the arrow keys of the remote controller, one of the keys (eg, key 62A) changes from a normal state to a selected state. In addition, the key 62A changes from the selected state to the normal state. The cursor has moved based on user action. When the user operates the OK key of the remote controller, the key 62B changes from the selection state to the execution state. As a result, a player operation designated by the key 62B is performed.

As mentioned above, player operations are described in a programming language or scripting language such as ECMA script. Programs and scripts for player operations are recorded on the disc. Programs and scripts for player operations can be recorded on the disc as separate files. Alternatively, graphic objects, programs and scripts operated by the player may be multiplexed with clip AV stream files, as will be described later for graphic objects.

2-9. About data structure

Next, together with the image data, the data structure of image data of keys constituting such a menu screen and control information will be described. Now, subtitles and graphics (still pictures) displayed, which do not constitute the movie, which is the main content recorded on the optical disc, will be described. In the following description, elements such as subtitles and graphics displayed on the screen are regarded as objects. There are three types of objects, namely subtitles, synchronous graphics and asynchronous graphics, as shown in Figure 41.

Subtitles are displayed with the movie, just like subtitles for movies. The subtitle image element is not associated with user input through eg a remote controller. Graphics are image elements such as buttons on menu screens that accept user input. Furthermore, graphics are divided into two categories: synchronous graphics and asynchronous graphics. Sync graphics are graphic elements that are synchronized with the movie. The synchronous figure is, for example, a branch selection screen, which is displayed at a specific time when the main part of the content is reproduced. Asynchronous graphics are graphic elements that are displayed asynchronously with the main part of the content being rendered. Examples of asynchronous graphics are menu screens that are initially displayed when a disc is loaded into the player, but also screens that are displayed based on user input. The image element displayed by the Java application program and the image element displayed according to the description of the HTML file on the browser software are asynchronous graphics, where the Java application program operates on the JavaVM.

In the relationship between each image element displayed on the film plane 10 and the main picture, subtitles and synchronization graphics are displayed in synchronization with the main picture. Thus, subtitles and synchronous graphics are of the synchronous type. On the other hand, since asynchronous graphics are displayed asynchronously with the main picture, they are of the asynchronous type as their name implies.

Subtitles and graphics can be classified according to plane. Subtitles are displayed on the subtitle plane 11 . Synchronous and asynchronous graphics are displayed on the graphics plane 12 .

Since subtitles and simultaneous graphics are being displayed while the main movie is being rendered, they tend to have a common data structure. Hereinafter, subtitles and sync graphics having a common data structure are referred to as graphics objects. Since graphics objects are always displayed synchronously with the movie being rendered, they can be easily handled when they are multiplexed with the movie.

42A, 42B and 42C show an example of the data structure of the graphic object 200 according to the first mode of the present invention. As shown in FIG. 42A, a graphics object 200 is composed of three areas, which are a graphics object title 201, a display control command table 202, and a PNG data area 203.

In the following example, it is assumed that image data as the graphics object 200 has a PNG format, and that the image data is PNG image data. Alternatively, the graphic object 200 may be image data in another format, such as bitmap data having a JPEG format, or image data compressed according to a run-length compression method, or bitmap data without compression encoding. For simplicity, image data will be expressed as PNG image, PNG image data, and so on.

In FIG. 42A , FIG. 42B , and FIG. 42C , the graphic object title 201 describes information indicating the attribute of the graphic object 200 . The attributes of the graphics object 200 are, for example, the data size of the graphics object 200, the number of PNG images that the graphics object 200 has, palette data for PNG image data common to the graphics objects 200, and identification information for identifying the graphics object 200. . This identification information is, for example, a number assigned to each graphic object 200 . Graphic object title 201 may further describe another information.

The display control command table 202 describes information necessary to control the display of the PNG image, such as the position where the PNG image possessed by the graphic object 200 is displayed, and the display start time, and the display end time thereof.

The PNG data area 203 describes image data compression-encoded according to the PNG format (hereinafter, this image data is referred to as PNG data). The PNG data area 203 may have a plurality of PNG data 203A, 203B, . . . , and 203n. The number of PNG data described in the PNG data area 203 is described in the graphics object header 201 .

Assume that a plurality of PNG data 203A, 203B..., 203n described in the PNG data area 203 are closely related images, like a group of animations or images that constitute the three states of the displayed keys. Like the picture. When the above PNG data 203A, 203B, . . . , and 203n constitute one graphic object, PNG images can be easily handled.

The graphic object 200 has time information, which describes the time at which the graphic object 200 can be displayed. For example, in an example where a real-time stream is transmitted as MPGE2TS, pts (Presentation Time Stamp) specified in MPEG2 (Moving Picture Experts Group 2) is used as time information. The pts is an output time management information of reproduction. The pts is measured by time at 90 kHz as a value with a length of 33 bits. When the STC (system clock) of the relevant decoder of the MPEG system coincides with pts, the relevant access unit is reproduced and output. Graphical object 200 may be displayed after the time indicated by pts. After the time indicated by pts, the start and end of display of the graphics object 200 are controlled with the display control command. Since the display of the graphic object 200 is managed by the display control command, after the display of the graphic object 200 is finished, the same graphic object 200 can be displayed.

FIG. 42B shows an example of a graphic object 200 for subtitles. PNG data area 203 describes an image as PNG data (1) 203A-1. When the graphic object 200 is a subtitle, the graphic object 200 only needs PNG data (1) 203A-1. Special effects such as fade in/out do not change the content of an image such as a subtitle added by adding a display control command, which causes the opacity of the PNG data (1)A-1 to be changed to the display control command table 202 .

On the other hand, when a special effect such as an animation of image change is applied to subtitles, several PNG data (2)B-1, PNG data (3)C-1, PNG data (4)D corresponding to the respective actions of the animation -1 can be described in a graphic object 200, as shown by the dotted line in FIG. 42B. Furthermore, PNG data of subtitles in different languages such as Japanese and English can be described as PNG data (1)A-1, PNG data (2)B-1 . . . in one graphics object 200 .

FIG. 42C shows an example of graphic objects 200 constituting a key. As described above, a key has three types of states, the normal state, the selected state, and the executed state. A key can be displayed with different images corresponding to the above three types of states. When a key is displayed using different images corresponding to the above-mentioned three types of states, data of three key images must be prepared. The data of the three image buttons is processed as one graphic object 200 . In other words, the PNG data area 203 of the graphic object 200 describes PNG data 203A- 2 , 203B- 2 and 203C- 3 , which respectively display buttons in the normal state, selected state, and executed state.

When the graphics object 200 has only the PNG data 203A-1 for subtitles indicated by the solid line in FIG. 42B, the display control command for the PNG data 203A-1 is described in the control command table 202 of the graphics object 200. When the graphic object 200 has several PNG data 203A-2, 203B-2 and 203C-2 as shown in FIG. Table 202 describes the display control commands.

When the initial state of the button of the graphic object 200 shown in FIG. 42C is designated as the selected state, the button image displayed at first and positioned at the beginning of the PNG data area 203 should not be the PNG data 203A-2 for the normal state, but is the PNG data 203B-2 for the selection state. According to the first mode of the present invention, display control is performed outside the graphic object 200 .

Should be specified by the external script program of graphic object 200 the initial state of each button, display start and display end, the program executed in the execution state of each button, etc., the script program can be, for example, the above-mentioned ECMA script or Java script . When the user operates the arrow keys of the remote controller and moves the cursor, the displayed PNG data for the key is changed. In this case, the player changes the PNG data for each key according to the user's input.

According to the first mode of the present invention, the graphics object 100 is divided into packets specified in MPEG2, multiplexed with a clip AV stream, and recorded on an optical disc as a clip AV stream file. As shown in FIG. 43, the graphics object 200 is divided and included in PES (Packed Elementary Stream) packets 210, 210, . . . , which are specified in MPEG2. Here, the graphics object header 201, the display control command table 202, and the respective PNG data 203A, 203B...203n are included at the beginning of the payload of the respective PES packets 210, 210,.... As a result, when the graphic object 200 is reproduced, each data thereof can be easily searched.

The graphics object 200 that is divided and included in PES (packet element stream) packets 210, 210, ... is further divided into TS packets that have a fixed data size of 188 bytes (not shown), and are connected with, for example, clip AV streams Video data and audio data streams are multiplexed.

2-10 About the decoder model for graphics objects

FIG. 44 shows a functional block diagram of an example of a graphical object decoder model 240 that decodes a graphical object 200 . The graphics object decoder model 240 mainly includes the multimedia engine 106 and the sub-picture decoder 116 of the player decoder 100 shown in FIG. 39 . In FIG. 44, for the sake of simplicity, the same parts as those in FIG. 39 are given the same symbols, and their descriptions are omitted. Since FIG. 44 shows the graphics object decoder model 240 further functionally than FIG. 39, the description of the graphics object decoder model 240 will be compared with the structure shown in these figures, when necessary.

The clip AV stream is supplied from the terminal 202 to the PID filter 110 . The function of the PID filter 110 is as a demultiplexer of MPEG TS (transport stream), and extracts film data, audio data and graphic objects 200 from the MPEG TS according to the PID of the TS. Movie data is supplied to buffer TBn111E, which is a video buffer. Audio data is supplied to the buffer 111D as an audio buffer. The graphics object 200 is provided to the buffer TBn 111B. It is the input buffer for graphics objects (labeled "GOBJ" in Figure 44).

The graphics object 200 is read from the buffer TBn 111B and supplied to the GOBJ parser 224. This GOBJ parser 224 is, for example, one of the functions of the sub-picture decoder 116 shown in FIG. 39 . GOBJ parser 224 reads graphic object header 201 from submitted graphic object information 200, extracts palette data from graphic object header 201, and separates display control command form 202 and PNG data area 203 from graphic object header 201. Palette data and display control command table 202 are provided to command processor/graphics extractor 225 . The PNG data 203A, 203B, . . . of the PNG data area 203 are temporarily stored in the PNG decoder buffer 226 . The PNG decoder buffer 226 corresponds to the buffer Bn112B shown in FIG. 39 .

The PNG data 203 stored in the PNG decoder buffer 226 is decoded by the PNG decoder 227, which is one of the functions of the sub-decoder 116, and outputs the decoded data as bitmap data. The bitmap data is stored in object buffer 228 . The object buffer 228 corresponds to the graphics buffer 109D shown in FIG. 39 .

The command processor/graphics divider 225 reads out the bitmap data stored in the object buffer 228 according to the display control command described in the display control command table 202, and transmits the bitmap data to the plane buffer 229 at a designated time . The plane buffer 229 corresponds to, for example, the sprite plane 132 and the graphics plane 133 in FIG. 39 . Plane buffers 229A and 229B (not shown) may be used for subtitles and graphics objects, not just subtitles. Alternatively, the sprite plane 132 and the graphics plane 133 may be different areas from the plane buffer 229 .

In addition, the command processor/graph partitioner 225 submits the palette data from the GOBJ parser 224 to the common palette table 230, which corresponds to the palette 150 shown in FIG. The command processor/graphics divider 225 has a part of the functions of the multimedia engine 106 and a part of the functions of the sub-picture decoder 116 shown in FIG. 39 .

As described above, when the graphic object 200 includes one key, the graphic object 200 includes PNG data 203A, 203B, and 203C corresponding to three types of states of the key. PNG data 203A, 203B, and 203C are decoded by PNG decoder 227 and stored in object buffer 228 .

On the other hand, an operation such as from the user's remote controller is received by the command processor/graphics splitter 225 . The command processor/graphics partitioner 225 reads a bitmap from the object buffer 228 and transfers the bitmap to the plane buffer 229 according to user input. When the user's input causes the state of the key to change from the selected state to the executed state, the key image corresponding to the executed state is selectively read from the object buffer 228 and transferred to the plane buffer 229 .

Command processor/graphics slicer 225 may perform special effect processing, such as extraction processing for bitmap data, which is read from object buffer 228 in accordance with display control commands.

According to the first mode of the present invention, since the sampling depth of one data of PNG data is 8 bits, data of 8 bits per pixel is arranged in the plane buffer 229 . The data of the plane buffer 229 is read out at intervals of a scan cycle of a display system that executes display processing of, for example, a display device or the like. The bitmap data read out from the plane buffer 229 is supplied to the common palette table 230 corresponding to the palette 150 shown in FIG. 39, for example. The common palette table 230 converts the bitmap data into true RGB (4:4:4) color information according to the palette index value, and extracts opacity data α1 and α2 from the bitmap data. Color information of RGB (4:4:4) is converted into color information of YCbCr (4:4:4) by a conversion circuit (not shown). The color information of YCbCr (4:4:4) is supplied to the presentation processor 139 shown in FIG. 39 together with the opacity data α1 and α2.

Special effects that require changing the palette and opacity, such as fade in/out, are done by the command processor/graphics splitter 225, which changes the data in the common palette table 230 according to the display control commands. Alternatively, common palette tables 230A and 230B (not shown) may be arranged for subtitles and graphics objects 200 instead of subtitles.

45A, FIG. 45B, FIG. 45C and FIG. 45D show graphics object input buffer, PNG decoder 227, object buffer 228, plane buffer 229 (buffer TBn 111B, hereinafter referred to as GOBJ input buffer) An example of changing the number of stored data objects. The PNG decoder 227 shows the amount of data stored in the buffer used when it decodes PNG data.

45A, 45B, 45C, and 45D show changes in data amounts of three graphic objects GOBJ#1, GOBJ#2, and GOBJ#3 on a time base. The decoding start time of the graphics object is represented by dts (decoding time stamp) of the MPEG2 system. The start time of an object's valid period is expressed in pts. An object ends at the time presentation_end described in the graphic object title 201 . The display control command instructs the display start and end of one image in the active period.

In FIG. 45D, the PNG data of the graphics object GOBJ#1 is input to the GOBJ input buffer. At time dts of GOBJ#1, decoding of PNG data starts. In FIG. 45C, PNG data is transferred from the GOBJ input buffer to the PNG decoder 227. The PNG data is decoded and bitmap data is obtained. In effect, the PNG data is temporarily moved from the GOBJ input buffer to the PNG decoder buffer 226 . The PNG decoder 227 performs decoding processing on the data stored in the PNG decoder buffer 226 .

Since the PNG decoder 227 has an upper limit of decoding speed, data is submitted from the GOBJ input buffer to the PNG decoder buffer 226 so that the transfer speed of the data does not exceed the decoding speed of the PNG decoder 227. Accordingly, the PNG data is input to the PNG decoder buffer 226 at a data transfer speed corresponding to the slope of a vertical line indicating that the processing time of the PNG data in the PNG decoder 227 is 0.

Even if all the PNG data is not input to the PNG decoder 227, decoding of the PNG data can be started. In the examples shown in FIGS. 45A, 45B, 45C, and 45D, after the object GOBJ#1 stored in the GOBJ input buffer is completely transferred to the PNG decoder 227, the next object GOBJ#2 in the GOBJ buffer The input of the PNG data is started.

Similarly, the PNG data of the object GOBJ#2 and the PNG data of the object GOBJ#2 are input to the PNG decoder buffer 226 at respective transfer speeds corresponding to the special slopes B and C, respectively. Actually, the slope B is different in each area.

When the effective period of the object GOBJ#1 starts at the pts time of GOBJ#1, the bitmap data of the object GOBJ#1 which has been decoded and stored in the PNG decoder buffer is transferred to the object buffer 228 (FIG. 45B). The active period in which the object GOBJ#1 is transferred to the object buffer 228 lasts until the end indicated by GOBJ#1.

During the effective period of the object GOBJ#1, when the command Display ONCmd. of GOBJ#1 is issued, the bitmap data of the object GOBJ#1 stored in the object buffer 228 is transferred to the plane buffer 229, and displayed (FIG. 45A). As will be described, the upper limit of the transfer speed for transferring bit map data to the plane buffer 229 varies depending on the influence of the bus width and the like. Thus, bitmap data is written into the plane buffer 229 at a transfer speed corresponding to a specific slope D, for example.

Similarly, the bitmap data of the other objects GOBJ#2 and GOBJ#3 are written in the plane buffer 229 at transfer speeds corresponding to slopes E, F, and G.

The object GOBJ#1 is continuously displayed until the command Display OFF cmd of GOBJ#1 is issued, which clears the indication of the object GOBJ#1. When the command Display OFFcmd. of the object GOBJ#1 is issued, the bitmap data of the object GOBJ#1 stored in the plane buffer 229 is deleted, and the object GOBJ#1 is cleared on the screen.

Object GOBJ#2 and object GOBJ#3 are successively input into the GOBJ buffer. Similar to the object GOBJ#1, the decoding of the object GOBJ#2 and the object GOBJ## starts at the time dts of GOBJ#2 and the time dts of GOBJ#3. The PNG data is supplied to the PNG decoder 227 . The PNG decoder 227 decodes PNG data with a PNG decoder buffer, and outputs bitmap data. The effective period of the object GOBJ#2 is specified time pts of GOBJ#2. The command Display ON cmd. of GOBJ#2 (not shown in FIGS. 45A to 45D ) displays the object GOBJ#2. The object buffer 228 transfers the bitmap data to the plane buffer 229 . Display the object GOBJ#2 until the command Display OFF cmd. of GOBJ#2 is issued.

In the example shown in FIGS. 45A to 45D , after the object GOBJ#2 is cleared with the command Display OFF cmd. (not shown) of GOBJ#2, the object GOBJ#2 is displayed again with the command Display ON cmd. of GOBJ#2 . The bitmap data of the object GOBJ#2 is held in the object buffer 228 until the effective period end time of GOBJ#1 indicates an end indication to the object GOBJ#2. Therefore, the object GOBJ#2 can be repeatedly displayed by the command Display ON cmd. of GOBJ#2.

On the other hand, the valid period specified for the object GOBJ#3 overlaps with the valid period specified for the object GOBJ#2. In this case, the object memory 228 stores a plurality of bitmap data in different areas according to its free capacity. For example, when the data of the object GOBJ#2 is transferred from the object buffer 228 to the plane buffer 229 and displayed, when the bit map data of the object GOBJ#3 is transferred from a different area of the object buffer 228, the two bit map data can be displayed simultaneously.

2-11. Regarding the transmission speed of graphics

Next, a case where a graphics object decoder model 240 (hereinafter referred to as a decoder model 240 ) is provided in a player will be considered. To make reproduction of data from the same optical disc compatible with different players, it is necessary to impose predetermined restrictions on the decoder model 240 . For example, decoder model 240 has an upper bound on graphics processing capabilities. Thus, when graphics data exceeding the upper limit of the capability is input, it is impossible to decode the graphics data well. As a result, graphic data cannot be displayed normally.

The minimum capabilities of graphics processing on the player side should be specified in a standard. On the other hand, graphics that can be handled in the minimum capacity prescribed by the standard can be prepared on the content producer side. Compatible reproduction capabilities can be maintained by matching the capability of graphics processing on the player side with the capability of graphics processing prepared on the content producer side.

According to the first mode of the present invention, in FIG. 44, the data transfer rate R(1) from the GOBJ parser 224 to the PNG decoder buffer 226, and the data transfer from the command processor 225 to the plane buffer 229 are specified. Speed R(2).

The data transfer rate R(1) specifies the number of data transfers to the PNG decoder buffer 226 per unit time. In other words, the slopes A, B, and C shown in FIG. 45C correspond to the data transfer speed R(1). This specifies a decoding capability indicating the amount of compression-coded graphic data that the PNG decoder 227 arranged downstream of the PNG decoder buffer 226 can decode within a unit time. Thus, by limiting the data transfer rate R(1), it is possible to prevent incomplete decoding and incorrect display of the input compression-coded graphic data.

The data transfer rate R(2) specifies the update rate of the image. The plane buffer 229 corresponds to the actual screen on the display device. Thus, the update speed of the graphics seen by the user depends on the speed at which data is written to the plane buffer 229 . The data transfer rate R(2) defines the minimum update interval for all planes, ie all screens in [bytes/second] units. In other words, the slopes D, E, F, and G shown in FIG. 45A correspond to the data transfer speed R(2).

When a part of the plane is updated, since the amount of image data to be updated is small, the update cycle is a cycle much shorter than the minimum update time interval specified by the data transfer rate R(2). However, the update cycle is not always proportional to the data amount of updated image data. The update interval is largely influenced by the arrangement of the image data on the plane.

Next, referring to FIG. 46, the update speed of the plane will be described in detail. Assuming that the object buffer 228 stores two graphics objects 460 and 461, the graphics objects 460 and 461 are written into the plane buffer 229 and displayed.

Graphics objects 460 and 461 are read from object buffer 228 and provided to command processor/graphics partitioner 225 . An output of the command processor/graphics divider 225 is bounded at the above-mentioned update rate R(2) to limit the update rate (update interval) on the screen.

Here, the update speed of graphics objects 460 and 461 depends more on how they are placed on the plane than on their total data volume. This is due to a conventional graphics processor performing graphics processing updating the planes in each rectangular area.

For example, a plane is updated with a square area 262 that includes all graphics objects 460 and 461 located in the plane. In other words, the command processor/graphic divider 225 forms the image data in the square area 262 according to the arrangement information in the graphic objects 460 and 461 . Image data in the square area 262 is supplied to the plane buffer 229 via a bus. The plane buffer 229 replaces the data in the area corresponding to the square area 262 with the data in the square area 262 according to the designated display position.

Since the image data output from the command processor/graphics divider 225 is bitmap data, the image data has a data amount depending on the area of the image, not the content of the image. In the example shown in FIG. 46, the data amount of the square area 262 including the graphic objects 460 and 461 can be represented by, for example, (width×height) pixels, that is, (width×height bytes).

Since the speed at which data is transferred to the plane buffer 229 is determined to be speed R(2) [bytes/second], it is clear that graphics objects 460 and 461 can be updated in {speed R(2)/(width×height)} seconds . After {speed R(2)/(width×height)} elapses, the next graphic object can be extracted. Therefore, when the disc manufacturer side creates a program that allows two graphics objects to be extracted at intervals of at least the above-mentioned time period, the same graphics can be displayed with any player. Thus, any player can maintain the reproduction capability.

After estimating the data transfer rate R(2), the animation speed of the subtitles can be determined so that the compatibility of reproduction can be maintained, as will be described below.

2-12. Details of Graphics Object Structure

Next, the structure of the graphic object 200 will be described in detail. The syntax shown in FIG. 47 describes an example of the structure of the graphics object 200. FIG. 47 shows a schematic diagram of a syntax describing an example of a graphic object 200 structure. Graphics object header 201, display command control table 202, and PNG data area 203 shown in FIG. 42A correspond to block GraphicsObjectHeader(), block GOBJCommandTable(), and block PNGImageReaion(), respectively.

The block GraphicsObjectHeader() begins with a field. The field length has an 8-bit data length, an integer of 0 or greater. The field length describes the bit length immediately after the field length until the end of the block GraphicsObjectHeader(). The field presentation_end_time_stamp has a data length of 33 bits, an integer of 0 or more. The field presentation_end_time describes the effective period end time of the graphic object 200 . The valid period of this graphic object is from pts of the PES packet header until the valid period end time described in the field presentation_end_time_stamp. The field Number_of_PNG_image has a data length of 8 bits, an integer of 0 or more, and describes the number of PNG images described in the block PNGImageRegion(). The field Number_of_DispCmds has a data length of 8 bits, an integer of 0 or more, and describes the number of display control commands described in the block GOBJCommandTable().

The block GlobalPaletteTable() in the block GraphicsObjectHeader() describes the information of the palette table common in the graphics object 200 . The palette table information described in the block GlobalPaletteTable() is the content described as the common palette table 230 . The field start_address_of_PNG_image(i) has a data length of 32 bits, an integer of 0 or greater, and describes the position of the i-th PNG image's data PNG_image(i) starting at the relevant number of bytes from the beginning of the block GraphicsObject() .

The field PNG_file_name(i) describes the file name of the PNG data starting from the field start_address_of_PNG_image(i). The content of field PNG_image(i) in block PNGImageRegion() is the same as in a single PNG file. Chunks PNGImageRegion() are created by concatenating one or more PNG files. For example, in FIG. 42A, PNG data 203A, 203B, . . . , 203n are concatenated, and a chunk PNGImageRegion() is created. Here, a file name can be described in the file PNG_file_name(i) so that the file will not be lost. Instead, individual PNG files are obtained when PNGImageRegion() is decomposed. A single field PNG_file_name(i) is a single file having a file name described in the field PNG_file_name(i).

The block GOBJCommandTable() is composed of command groups DispCmds(i), which are concatenations of display control commands executed at the same time. The command group DispCmds(i) describes a display control command starting at command execution_time (time) which describes the execution time. In other words, the part after the command execution_time (time) until the next command execution_time (time) constitutes the command group DispCmds(i).

As described above, the block PNGImageRegion() describes the field PNG_image(i), which is the data of an image compressed according to the PNG system.

Any amount of padding_word can be described between block GraphicsObjectHeader() and block GOBJCommandTable(). Similarly, any amount of padding_word can be described between blocks GOBJCommandTable() and PNGImageRegion().

The syntax shown in Fig. 48 describes an example of the structure of the above-mentioned block GlobalPaletteTable(). The field number_of_palette_entries describes the number of palette data before it. The maximum number of fields number_of_palette_entries is 256 when an image is described with an index number having a data length of 8 bits. Thus, 256 colors can be used. When only a part of 256 colors is used, only necessary palette data is required. The field number_of_palette_entries describes the number of indexes used.

The field palette_index_number describes the index number of the color and opacity assigned to the field red_value, field green_value, field blue_value, and field alpha preceding the field palette_index_number, and the image data indicates the color and opacity with the index number.

In the for statement loop of the block GlobalPaletteTable(), the field palette_index_number with the same value will not be described more than twice. Each field red_value, field green_value, field blue_value has a data length of 8 bits, and an integer of 0 or more. Field red_value, field green_value, and field blue_value specify red, green, and blue, respectively. The field alpha has a data length of 8 bits. The field alpha represents the opacity alpha. When the value of field alpha is 0, it means completely transparent. When the field alpha has a value of 255, it is completely opaque.

Each PNG image has a block of palette information PLTE. According to the first mode of the invention, instead of using the palette information PLTE, palette information determined by the block GlobalPaletteTable() is used. When displaying multiple PNG images at the same time, it is difficult to display the PNG images in correct colors if the PNG images use colors from different palettes. In other words, a plurality of PNG images described in the field PNG_image(i) of GraphicsObject( ) indicate the common block GlobalPaletteTable( ) and use the common palette table described in the block GlobalPaletteTable( ).

Next, the command group DispCmds(i) will be described. The command group DispCmds(i) describes a display control command, which controls the display of the graphic object 200 . In the command group DispCmds(i), the command execution_time(start_time) causes the commands described before the next command execution_time(start_time) to be executed at the specified time start_time. The start point of the time start_time is pts of the graphics object 200 . The unit of time start_time is the same as pts.

The command group DispCmds(i) can describe a plurality of commands, which are executed at the time start_time described in the command execution_time(start_time). The commands executed by the command group DispCmds(i) are executed simultaneously at the time start_time described in the command execution_time(start_time). If the time start_time described in the command execution_time(start_time) of the next command group DispCmds(i+1) elapses before the command described in the command group DispCmds(i) is executed, the execution of the command group DispCmds(i) is canceled. Instead, the next command group DispCmds(i+1) is executed.

Display control commands other than the command execution_time(start_time) described in the command group DispCmds(i) are considered to be listed in FIGS. 49A and 49B. These display control commands are assigned numbers as shown in Figs. 49A and 49B.

(1) Command execution_time(start_time)

(2) Commands to display graphic objects

(3) Command to clear graphic objects

(4) Commands to change the color and opacity of the used palette table

(5) Commands to set the position and size of graphic objects displayed on the plane

(6) Commands to set the display range of graphic objects

(7) Command to reproduce an effect sound

(8) Command to assign an effect sound to image data (PNG data)

The above seven types of commands are just examples. In other words, the commands in the command group DispCmds(i) are not limited to the above-mentioned commands. Other display control commands can be identified and added to the command group DispCmds(i).

The display start command (2) and display end command (3) of the graphic object 200 are so-called fade in/out commands described as command fade_in (fade_in_time) and command fade_out (fade_out_time), respectively.

The command fade_in(fade_in_time) specifies the fade-in operation. The command fade_in(fade_in_time) causes a graphic object 200 to gradually display from the non-display state to the display state. The fade-in operation is achieved by gradually increasing the value of the alpha blend's opacity α corresponding to the time fade_in_time. When the command fade_in (fade_in_time) follows the command execution_time (start_time), the transparent graphic object 200 gradually becomes opaque after the time start_time specified by the command execution_time (start_time). After the time specified by the argument time fade_in_time has elapsed, the opacity α of all palette indices is set to a value specified by the common palette table.

When the time fade_in_time of the command fade_in(fade_in_time) is set to 0, the graphics object 200 is immediately displayed with the color and opacity α specified in the palette table.

Fade out is the inverse of fade in. Fade out is specified by the command fade_out(fade_out_time). The command fade_out(fade_out_time)) causes a graphics object 200 to be gradually cleared from the display state. The fade-out operation is achieved by gradually reducing the value of the opacity α of the alpha blend corresponding to the time fade_in_out. When the command fade_out (fade_out_time) follows the command execution_time (start_time), the non-transparent graphic object 200 gradually becomes transparent after the time start_time specified by the command execution_time (start_time). After the time specified by the argument time fade_out_time has elapsed, the value of opacity α for all palette indices is set to 0. As a result, the graphic object 200 becomes completely transparent and invisible.

When the time fade_out_time of the command fade_out(fade_out_time) is set to 0, the graphics object 200 is cleared immediately.

A more natural fade-in and fade-out effect is better achieved when the value of the opacity α gradually changes between fading in and out over time. Optionally, in fade-in, after the time specified by time fade_in_time has elapsed, the value of opacity α should match the value specified in the palette table. However, the resolution and level of opacity alpha is not specified by the command. In practice, the resolution and level of opacity alpha depends on the installed system.

In the above examples, the commands are expressed in text such as "fade_in()" and "fade_out()" for high recognition. Actually, however, the commands fade_in( ) and command fade_out( ) are converted into predetermined binary values together with their arguments, and are described by DispCmds(i). The same applies to the commands described below.

Palette table color and opacity alpha change command (4) changes the palette information. This command is described in the format of change_palette(index, newR, newG, newB, newAlpah). A PNG image displayed on both the subtitle plane 11 and the graphics plane 12 refers to the common palette table shown in FIG. 24 , which is determined by the syntax shown in FIG. 48 . Palette information determined as GlobalPaletteTable() is used as a common palette table. Use the command change_palette(index, newR, newG, newB, newAlpah) to change the public palette information.

In other words, the value index, newR, newG, newB, newAlpah described as an argument in the command change_palette(index, newR, newG, newB, newAlpah) makes three of the color index values indicated by the palette number index. The original color values R, G, and B are changed to the values newR, newG, and newB, and the value of opacity α is changed to the value newAlpah.

Command (5) is used in the format of set_display_box(x1, y1, x2, y2), which sets the display position and size of a graphic object on a plane. Command (5) causes the graphic object 200 to be located in the square area (x1, y1) (x2, y2) defined by the coordinates (x1, y1) and (x2, y2) on the plane. Command (6) is used in the format set_clipping_box(a1, b1, a2, b2), which sets the display range of a graphics object. Command (6) causes a square area (a1, b1) (a2, b2) to be displayed on the plane, the square area being determined by the (a1, b1) and (a2, b2) coordinates of the PNG image of the graphics object 200.

Next, the command set_display_box(x1, y1, x2, y2) and the command set_clipping_box(a1, b1, a2, b2) will be described in detail with reference to FIGS. 50A and 50B. As shown in FIG. 51, on the coordinates of FIG. 50A and FIG. 50B, the upper left corner of the display screen is determined as the origin, the horizontal rightward direction is marked by x, the vertical downward direction is marked by y, and the coordinates are marked by (x, y).

As shown in FIG. 50A , set_clipping_box( a1 , b1 , a2 , b2 ) causes actually displayed one square area ( a1 , b1 )( a2 , b2 ) to be set in the PNG image 250 of the graphic object 200 . In the example shown in FIG. 50A , it is assumed that the square area ( a1 , b1 ) ( a2 , b2 ) is set smaller than the PNG image 250 . The command set_display_box (x1, y1, x2, y2) makes the real display position of the square area (a1, b1) (a2, b2) be set on the plane of the square area (x1, y1) (x2, y2) (see Figure 50B) . In other words, only the square area (a1, b1) (a2, b2) of the PNG image 250 is displayed facing the square area (x1, y1) (x2, y2) of the screen.

When the square area (a1, b1) (a2, b2) is larger than the actually displayed square area (x1, y1) (x2, y2), only the square in the square area (a1, b1) (a2, b2) is displayed PNG image of region (x1,y1)(x2,y2). On the contrary, when the square area (a1, b1) (a2, b2) is smaller than the actually displayed square area (x1, y1) (x2, y2), the square area (x1, y1) (x2, y2) in the square area ( The outside of a1, b1)(a2, b2) is treated as a transparent area.

When the above-mentioned display control command is described together with several execution_time (start_time) commands, subtitles and synchronous graphics that change with the lapse of time can be displayed. For example, in the graphic object 200 of FIG. 47, several command groups DispCmds(i) are described in the block GOBJCommandTable(). Each command group DispCmds(i) describes the display control command execution_time(start_time), and its time start_time is different, so that the command group DispCmds(i) is executed at the start time specified by start_time.

FIG. 52 shows a description of the command group DispCmds(i), and an example of display change of the graphic object 200. In FIG. 52 , the horizontal axis represents the lapse of time, whereas the vertical axis represents the opacity of the graphic object 200 . In Figure 52, the opacity increases in the upward direction. The time identified by Pts is the origin.

In the first command group DispCmds(0), the command set_display_box(800, 800, 1300, 900) sets a display area on the plane. The command set_clipping_box(0, 0, 500, 100) sets the display area of the PNG image of the graphic object 200 . The command fade_in(2 seconds) produces a two-second fade-in process at time[0]. In the following command group DispCmds(1), the command change_palette(index, newR, newG, newB, newAlpah) describes the color index values [1], [2], [3] and [4]. The command group DispCmds(1) also causes the color and opacity α indicated by the index values [1], [2], [3] and [4] to change at time [800]. The next command group DispCmds(2) causes a displayed graphics object 200 to fade out for two seconds at time [2000].

As shown in FIG. 52, when the command groups DispCmds(0), DispCmds(1), and DispCmds(2) are sequentially described, subtitles that change with the lapse of time, for example, can be realized. In other words, when the command groups DispCmds(0), DispCmds(1) and DispCmds(2) are used correctly, subtitles and button images are displayed as animations.

53A, 53B, 53C, and 53D show examples of gradually displaying fade-in of subtitles. In FIGS. 53A to 53D, fade-in is controlled so that subtitles are gradually displayed. The simple fade-in described above can be accomplished with a command similar to the command group DispCmds(0) shown in FIG. 52 .

54A and 54B show an example in which a PNG image 260 as a subtitle moves between planes. The PNG image 260 described above is implemented with a plurality of commands set_display_box(x1, y1, x2, y2). For example, in the first command group DispCmds(0), the command execution_time(start_time) sets the start time. The command set_clipping_box(a1, b1, a2, b2) sets the display area of the PNG image 260, as shown in Fig. 54A. The command set_display_box(x1, y1, x2, y2) sets the initial display area of a PNG image on a plane.

In the next command group DispCmds(1), the command execution_time(start_time) sets a time as the start time after the execution of the command group DispCmds(1). The command set_display_box(x1', y1', x2', y2') sets the display area to be moved on the plane. Similarly, in the next command group DispCmds(2), the command execution_time(start_time) sets a time after execution of the command group DispCmds(1) as the start time. The command set_display_box(x1″, y1″, x2″, y2″) sets the display area to be moved on the plane.

Thus, as shown in FIG. 54B, the PNG image 260 as a subtitle can be moved to a square area (x1, y1) (x2, y2), a square area (x1', y1') (x2', y2') and a square area on the plane. Area(x1", y1")(x2", y2").

55A and 55B show an example of moving and scrolling the display area 262 of the PNG image 261 which is a subtitle. This can be achieved with multiple commands set_clipping_box(a1, b1, a2, b2). For example, in the first command group DispCmds(0), the command execution_time(start_time) sets the start time. The command set_clipping_box(a1, b1, a2, b2) sets a square area 262 initially displayed on the PNG image 260, as shown in FIG. 55A. The command set_display_box(x1, y1, x2, y2) sets a square area of the PNG area 260 displayed on a plane.

In the next command group DispCmds(1), the command execution_time(start_time) sets a predetermined time after execution of the command group DispCmds(1) as the start time. The command set_clipping_box(a1', b1', a2', b2') sets the display area to be moved in the PNG image 260 . Similarly, in the next command group DispCmds(2), the command execution_time(start_time) sets a predetermined time after execution of the command group DispCmds(1) as the start time. The command set_clipping_box(a1″, b1″, a2″, b2″) sets the square area to be moved in the PNG image 260 .

Thus, as shown in FIG. 55B, the square area that is a part of the PNG image 261 moves from the square area (a1, b1) (a2, b2) to the square area (x1, y1) (x2, y2) of the square area on the plane ( a1', b1')(a2', b2') and square area a1", b1")(a2", b2"). Thus, the subtitles are scrolled.

56A and 56B show an example of setting a frame that is a part of the PNG image 265, the frame is moved into the PNG image 265, and the display position of the frame is moved to the plane. This can be achieved by executing multiple commands set_display_box(x1, y1, x2, y2) and set_clipping_box(a1, b1, a2, b2) simultaneously. For example, in the first command group DispCmds(0), the command execution_time(start_time) sets the start time. The commands set_display_box(x1, y1, x2, y2) and the commands set_clipping_box(a1, b1, a2, b2) set frame 266A (see FIG. 56A).

For example, the command set_display_box(x1, y1, x2, y2) sets the square area (x1, y1)(x2, y2) displayed on a plane. The command set_clipping_box(a1, b1, a2, b2) sets the square area (a1, b1)(a2, b2) displayed in the PNG image 265. The square area (x1, y1) (x2, y2) and the square area (a1, b1) (a2, b2) constitute a frame 266A.

In the next command group DispCmds(1), the command execution_time(start_time) sets a predetermined time after the command group DispCmds(0) as the start time. The command set_display_box(x1', y1', x2', y2') sets a square area (x1', y1')(x2', y2') on the plane. The command set_clipping_box(a1', b1', a2', b2') sets a square area (a1', b1')(a2', b2') on the PNG picture 265 . The square area (x1', y1') (x2', y2') and the square area (a1', b1') (a2', b2') constitute a frame 266B, and the frame 266A moves to the frame 266B. Also, in another command group DispCmds(2), the command execution_time(start_time) sets a predetermined time after the execution of the command group DispCmds(1) as the start time. The command set_display_box(x1", y1", x2", y2") sets a square area (x1", y1")(x2", y2") on the plane. The command set_clipping_box(a1″, b1″, a2″, b2″) sets a square area (a1″, b1″)(a2″, b2″) on the PNG picture 265 . The square area (x1", y1") (x2", y2") and the square area (a1", b1") (a2", b2") constitute a frame 266C, and the frame 266B moves to the frame 266C.

Thus, as shown in FIG. 56B, when moving a square area of a portion of the PNG image 265 of subtitles, the square area may be moved from frame 266A to frame 266B to frame 266C.

Thus, according to the first mode of the present invention, since the display control of the graphic object 200 is performed by the command group DispCmds(i), each display control command in the command group DispCmds(i) is grouped by the command execution_time(start_time), Different displays can be easily realized on the subtitle plane 11 and the graphics plane 12 .

2-13 About the effect sound

According to the first mode of the present invention, a sound output can be synchronized with the display control of the graphic object 200 . The sound output is determined by the display control command (7) and the display control command (8), the display control command (7) reproduces an effect sound, and the display control command (8) assigns the effect sound to the display control command (2) to ( The image data in 8), except the command execution_time(start_time) of the above-mentioned command group DispCmds(i). The sound data specifies a unique identification sound_id.

Effect sound is the sound reproduced in accordance with the display control of subtitles or buttons, or in synchronization with it, rather than in synchronization with the film or still picture displayed on the film plane (for example, in a film, recorded in pairs with the film sound).

As described in the Related Art section, the user operates keys (direction keys) designated as up, down, left, and right using, for example, a remote control commander (hereinafter referred to as a remote controller) to select a key displayed on a menu screen. . Thereafter, the user performs an operation assigned to the selected key through the ok key. Each button has three states: normal state (non-selection state), selection state and execution state. The above three states of each key specify different graphic objects. Thus, the button image (shape and color) can change for each of the three states of the button. Users can easily distinguish the state of keys using different images.

In addition, the three states of each key can assign different effect sounds. For example, an effect sound generated when a user selects a key is assigned to the key image of the "selected state". In addition, an effect sound such as a click generated when the key is in the execution state is assigned to the key image of the "execution state".

The effect sound assigned to the key image of the key "selected state" is associated with the identifier sound_id of the sound data generating the effect sound. When the user selects this button, the sound data associated with the identifier sound_id is read out from the memory of the player, and reproduced as an effect sound. Similarly, the effect sound assigned to the key image of the key "execution state" refers to the identifier sound_id of the sound data generating the effect sound. When the user selects the key and operates the OK key, the key becomes the execution state. Here, the sound data of the identifier sound_id of the sound effect associated with the key image of the "execution state" is read from the memory of the player and reproduced.

The display control command (7) of the command group DispCmds(i) reproduces an effect sound described in the play_sound(sound_id) format. The command play_sound(sound_id) reproduces the sound data identified by the identifier sound_id. When the command play_sound(sound_id) is described in the command group DispCmds(i), the sound data identified by the identifier sound_id is reproduced at the time start_time specified by the command execution_time(start_time).

For example, when the command play_sound(sound_id) is used together with the command fade_in(fade_in_time) and the command fade_out(fade_out_time), sound data as effect sound is reproduced while reproducing subtitles and/or clearing subtitles. Fig. 57 shows an example of using the command play_sound(sound_id). In the example shown in FIG. 57, the first command group DispCmds(0), command fade_in(2 seconds) causes a graphics object to fade in for two seconds at start time [0]. The command play_sound(1) reproduces the sound data identified by the identifier sound_id[1]. Thereafter, in the command group DispCmds(1), the command execution_time(800) changes the display color at time [800]. The command play_sound(2) reproduces the sound data identified by the identifier sound_id[2]. In the command group DispCmds(2), the command execution_time(2000) and the command fade_out(1 second) cause the graphics object to fade out at time [2000] for 1 second. The command play_sound(1) reproduces the sound data identified by the identifier sound_id.

The command (8) described in the format of set_sound(PNG_image_id, sound_id) specifies an effect sound to PNG data. The command set_sound(PNG_image_id, sound_id) reproduces the sound data specified by the identifier sound_id for the PNG data identified by the identifier PNG_image_id. The command set_sound(PNG_image_id, sound_id) reproduces the sound data identified by the identifier PNG_image_id when the PNG data identified by the identifier PNG_image_id is displayed. The identifier PNG_image_id of the PNG data is the same as the value of the loop counter i of the PNG_image(i) of the block PNGImageRegion() shown in FIG. 47 .

Use the command set_sound(PNG_image_id, sound_id) as the PNG data of the key in the selection state and execution state. As a result, when the normal state of the key is changed to the execution state or vice versa, it is possible to generate sound data as effect sounds assigned to the PNG data, the sound data representing the respective states. Besides this example, the command set_sound(PNG_image_id, sound_id) can be used for other PNG data besides keys.

FIG. 58 shows an example of the data structure of a graphic object 200 in which sound data is assigned to key images. PNG data 203A, 203B, and 203C for keys in the normal state, selection state, and execution state are described in the PNG data area 203 . In the example shown in Fig. 58, the display control command specifies coordinates and sound data to PNG data. The display start time of the PNG data and the initial display state of the keys are controlled by an external script program. Thus, the display control command is described as execution time [0].

The graphic object 200 shown in FIG. 58 describes only the command group DispCmds(0), which is executed in the display control command table 202 at [0] time by the command execution_time(0). Since the identifier PNG_image_id starts from [0], the identifier PNG_image_id[0] indicates the PNG data 203A in a normal state. The identifier PNG_image_id[1] indicates the PNG data 203B in the selected state; the identifier PNG_image_id[2] indicates the PNG data 203C in the execution state.

When the PNG data 203B of the key in the selected state is displayed by the command set_sound(1, 10), its identifier PNG_image_id is [1], and the sound data identified by the identifier sound_id[10] is reproduced as an effect sound. Similarly, when the PNG data 203C of the key in the execution state is displayed by the command set_sound(2, 11), its identifier PNG_image_id is [2], and the sound data identified by the identifier sound_id[11] is reproduced as an effect sound.

Although not shown in FIG. 39, one or more types of sound data may be pre-stored in a memory such as an external memory of the player. For example, the preparatory sound data may be pre-stored in a memory such as an external non-volatile memory of the player before shipment.

Alternatively, sound data as effect sound may be pre-recorded on an optical disc in which the graphic object 200 and the content as movie data have been recorded. When reproducing content from an optical disc, sound data can be read out. As a method of recording sound data on an optical disc, a file for the sound data is prepared. When reproducing content from an optical disc, the files are read in advance and stored in the player's memory.

Optionally, like graphics object 200, a PES packet including sound data is created. PES packets are divided into TS packets. TS packets are multiplexed with clip AV streams.

Optionally, sound data may be located in the graphical object header 201 of the graphical object 200 . Optionally, the sound data corresponding to a PNG image included in the graphic object 200 may include an area immediately before the PNG image data area 203 (not shown in FIGS. 42A to 42C ).

In any method, since the sound data can be read out from the disc in advance and stored in the memory of the player, a sound effect can be generated when the state of a button created with a PNG image changes to the selected state or the executed state . The sound data is assigned a unique identifier sound_id, and the sound data can be uniquely identified.

2-14 Regarding the method for storing sound data on an optical disc

There are three methods for storing sound data to a disc. Next, these three methods will be described.

(1) The graphics object 200 is multiplexed with sound data.

(2) Sound data is created corresponding to the identifier sound_id, and the sound data is not multiplexed with the clip AV stream.

(3) A data file is created for a plurality of pieces of sound data, and the sound data is not multiplexed with the clip AV stream.

2-14a First storage method

Next, referring to FIGS. 59A, 59B, and 59C, (1) a method of multiplexing sound data and graphic objects 200 (hereinafter referred to as a first storage method) will be described in detail. 59A and 59B show an example in which sound data is added to the graphic object 200 and multiplexed with the clip AV stream.

FIG. 59A shows an example of the sound data area 204 arranged after the PNG data area 203 of the graphics object 200. The sound data area 204 includes a plurality of sound data 204A, 204B, . . . , 204n. When the sound data 204A, 204B, . . . , 204n respectively correspond to the PNG data 203A, 203B, .

The sound data 204A, 204B, . , AAC (Advanced Audio Coding) or ATRAC (Adaptive Transform Acoustic Coding) files. When compression-encoded sound data is included, the player side should have an audio decoder conforming to the compression-encoding system.

FIG. 59B shows that sound data corresponding to the key state is included in the graphic object 200 constituting one key. In this example, sound data 204A-2 and sound data 204B-2 are included in the sound data area 203. When the key is in the selected state, the sound data 204A-2 is reproduced, and when the key is in the execution state, the sound data 204B-2 is reproduced. 2. On the other hand, PNG data 203A-2, 203B-2, and 203C-2 are contained in the PNG data area 203, which are keys for the normal state, selection state, and execution state.

In this case, the sound data area 204 follows the PNG data area 203 for the button image. The sound data area 204 includes sound data 204A-2 and sound data 204B-2. When the key is in the selected state, the sound data 204A-2 is reproduced, and when the key is in the execution state, the sound data 204B-2 is reproduced. Thus, when the PNG data of the key image is displayed, sound data corresponding to the state of the key is reproduced. Effect sounds reproduced by the player are mainly used as button click sounds. Thus, in the above structure, the main object of the present invention can be fully achieved.

FIG. 60 shows an example of the structure of a graphics object decoder model 240' in which a processing system for sound data is added to the graphics object decoder model 240 described in FIG. For the sake of simplicity, the same parts in FIG. 60 as those in FIG. 44 are marked with the same numerals, and their descriptions are omitted.

The clip AV stream as MPEG TS is input from the terminal 202 and supplied to the PID filter 110. A clip AV stream has a graphics object 200 containing audio data. The PID filter 110 extracts movie data, sound data, and graphic objects 200 from the clip AV stream. The extracted graphics object 200 is supplied to the GOBJ parser 224 through the buffer TBn111B. The GOBJ parser 224 reads out the graphic object title 201 from the graphic object 200 . GOBJ parser 224 extracts palette data from graphic object 200 according to graphic object title 201, and divides graphic object 200 into display control command table 202, PNG data area 203 and sound data area 204 according to graphic object title 201.

Palette data and display control command table 202 are provided to command processor/graphics divider 225 . Furthermore, the sound data 204A, 204B, . . . , 204n of the sound data area 204 are supplied to the command processor/graphics divider 225 and stored in corresponding buffers (not shown).

The command processor/graphics divider 225 reads out the sound data from the buffer according to the display control command described in the display control command table 202 submitted from the GOBJ parser 224, and outputs the sound data. When the sound data 204A, 204B, . . . , 204n contained in the graphics object 200 has been compression-coded, the command processor/graphics divider 225 decodes it, and outputs the decoded sound data.

The sound data output from the command processor/graphics divider 225 is supplied to the audio mixer 231 and output to the presentation processor 141 . When another sound data has been input to the audio mixer 231, this sound data is mixed with these sound data at a predetermined ratio, and output.

The syntax shown in Fig. 61 describes an example of the structure of the graphic object 200 containing the sound data shown in Figs. 59A and 59B. In the structure shown in FIG. 47, the field number_of_PNG_data of the block GraphicsObjectHeader() is followed by the field number_of_sound_data. The number of sound data segments stored in the block SoundDataRegion() is represented by an 8-bit integer not less than 0. The field start_address_of_sound_data(i) has a data length of 32 bits, an integer not smaller than 0, and describes the position where i-th sound data sound_data(i) starts at the corresponding number of bytes from the block GraphicsObject(). The block PNGImageRegion() is followed by padding_word, followed by the block SoundDataRegion(). Real sound data as data Sound_data(i) is contained in the block SoundDataRegion().

2-14b. Second storage method

Next, the method (2) is used to create sound data corresponding to the identifier sound_id, the sound data not being multiplexed with the clip AV stream (hereinafter this method is referred to as a second storage method). For example, as shown in FIG. 62, a directory SOUND containing sound data is located under the directory BDAV. The directory SOUND contains PCM waveform data as sound data. For example, a sound data file "sound1.aiff" having an AIFF format is located under the directory SOUND. When the disc is initially loaded into the player, all sound data files under the directory SOUND are read out and stored in the external memory of the player.

Each piece of sound data is assigned a unique identifier sound_id. The script or the like accesses necessary sound data from the memory using the identifier sound_id.

In this case, a sound identifier area 205 is arranged in the graphic object 200 as shown in FIG. 59C. Sound identifier data 205A and 205B are included in sound identifier area 205 . In the example shown in FIG. 59C, PNG data 203A, 203B, and 203C are included in the PNG data area 203, and PNG data 205A, 205B, and 205C correspond to the normal state, selection state, and execution state of the buttons in the PNG data area 203, respectively. The sound identifier data 205A, 205B are identifiers sound_id corresponding to the PNG data 203B and 203C, respectively. When the PNG data 203B is displayed, sound data corresponding to the identifier sound_id represented by the sound identifier data 205A stored in the memory of the player is reproduced.

For example, PNG data and sound data can be associated according to the display control command table 202 as shown in FIG. 58 .

Unlike the display control commands shown in FIGS. 49A and 49B, since the sound data is identified with the identifier sound_id, the effect sound of the sound data can be generated synchronously with the displayed graphics at any time.

In this method, since the sound data is read from the memory using the identifier sound_id, the number of types of effect sounds is limited using the identifier sound_id. Also, the number of types of effect sounds that can be used is limited by the capacity of the player's external memory.

Next, this method will be described in detail with reference to FIG. 63 . When the disc 400 is loaded, the player starts accessing the disc. Sound data is read from the directory SOUND located under the directory BDAV. The read sound data (PCM data) is stored in the external memory 410 of the player. Here, a unique identifier sound_id is assigned to each piece of sound data. Alternatively, an identifier sound_id may be added to each piece of sound data recorded on the optical disc 400 .

In this example, 16 pieces of audio data are read from the optical disc 400 . Identifiers sound_id=1 to 16 are assigned to these segments of sound data. In addition, the data sizes of these segments of sound data are obtained. In the example shown in FIG. 63, it is assumed that the sound data assigned to identifiers sound_id=1 to 16 in the example shown in FIG. 63 have data sizes of d1 byte, d2 byte, . . . , and d16 byte, respectively .

For example, on menu screen 420 displaying buttons 420A, 420B, and 420C, when an operation is performed for button 420C, sound data corresponding to identifier sound_id assigned to button 420C is read out from memory 410 . In the example shown in FIG. 63, sound data corresponding to an identifier sound_id=1 is assigned to the button 420C. The audio data read out from the memory 410 is processed in a predetermined manner and temporarily stored in the buffer 450B. Thereafter, the sound data is supplied to the audio mixer 231 . The audio mixer 231 mixes sound data with other sound data associated with, for example, movie data as a main part of content, and outputs the mixed data as sound.

The buffer 450A temporarily stores sound data associated with, for example, movie data as a main part of the content. When adjusting to read out the stored sound data from the buffers 450A and 450B, an effect sound corresponding to the key 420C is output from the buffer 450B at an appropriate timing of the sound data stored in the buffer 450A. In this example, the no_sound data reproduction mode is designated with the identifier sound_id=0.

In the above mode, the total capacity of sound data that can be read from the optical disc 400 is limited by the capacity of the memory 410 . In addition, the capacity of each piece of sound data is limited according to the capacity of the buffer 450B. In other words, when the capacity of the memory 410 is represented by the capacity M (bytes), and the capacity of the buffer B is represented by the capacity Dmax (bytes), the following two conditions must be satisfied.

(1) The capacity d1 of each piece of sound data stored in the memory 410 should be smaller than the capacity Dmax of the buffer 450B.

(2) The total capacity (d1+d2+...+dn) of the sound data stored in the memory 410 should be smaller than the capacity M of the memory 410.

In other words, when the conditions (1) and (2) specify rules on the player side and the disc manufacturer side, reproduction compatibility of sound data such as effect sound can be maintained.

As described above, in the case where the sound data is not multiplexed with the clip AV stream (the second storage method), when the disc is first loaded into the player, all the sound data is read therefrom. However, the present invention is not limited to these examples. In other words, audio data can be read out from the optical disc in multiple times. For example, all sound data used as one of the script parts is read out and stored in memory. Here, the sound data of the previous part of the script in the memory is erased. As a result, even if the data volume of the sound data of the scenario exceeds the capacity of the memory, the sound data can be processed.

All sound data can be recorded in a predetermined area on the disc. Alternatively, sound data may be recorded separately in a plurality of areas on the optical disc. When the sound data is separately recorded in a plurality of areas on the optical disc, the sound data for a certain part of the script may be recorded on the optical disc at a position corresponding to the script part. Alternatively, the sound data can be downloaded from a server via a network.

In the method for multiplexing sound data with clip AV streams (first storage method), as shown in FIGS. 59A and 59B , the number of sound data types is not limited. As a result, different types of sound data can be assigned to each image. When necessary, sound data is submitted together with the clip AV stream. Thus, any time a clip AV stream is delivered, different types of sound data can be used. Furthermore, in the first storage method, since the audio data is read together with the image data from the clip AV stream, the reading model can be easily constructed. Furthermore, the number of sound data files and the file size are not limited except for the capacity of the disc.

However, in the first method, when different graphic objects use the same sound data, since the graphic objects of each of them have the same sound data, the sound data becomes redundant. Furthermore, since the sound data should be extracted from the graphic object, after the clip AV stream signal is separated, the sound data should be separated from the graphic object.

2-14c The third storage mode

Next, a method (3) for creating one data file of pieces of sound data that is not multiplexed with clip AV streams (hereinafter referred to as a third method) will be described. The pieces of sound data are contained in, for example, the file "HdmvSound.bdmv". As shown in Figure 64, the file "HdmvSound.bdmv" is located under the directory BDAV of the file management structure, and the above-mentioned files are recorded in the recording medium determined by "Blu-ray Disc Rewritable Format Version 1.0 Part III" shown in Figure 9 . When the disc is loaded into the player, the file "HdmvSound.bdmv" is initially accessed and read. The file "HdmvSound.bdmv" is preloaded into the memory of the player before reading out a segmented AV stream (for example, a movie) recorded as a main part on an optical disc.

In Fig. 64, under the directory STREAM is the clip AV stream which is the main content of the disc.

The syntax shown in Fig. 65 describes an example of the file "HdmvSound.bdmv". A field length has a data length of 32 bits. This field length describes the byte length to the end of the file "HdmvSound.bdmv" after this field length. The field data_block_start_address has a data length of 32 bits. The field data_block_start_address describes the start address of the start byte of the block data_block for sound data in the file "HdmvSound.bdmv". The field number_of_sound_entries has a byte length of 8 bits. The field number_of_sound_entries describes the number of pieces of sound data (that is, effect sound data) contained in the file "HdmvSound.bdmv".

The argument for loop automatically specifies an identifier sound_id indicating a piece of sound data. The for loop describes information of the sound data indicated to the identifier sound_id corresponding to the argument.

The block attributes() describes the attributes of the sound data. The field channel_assignment describes the channel assignment of sound data. The field channel_assignment describes an attribute such as monaural or binaural. The field sampling_frequency describes the sampling frequency of the sound data, eg 48kHz. The field bits_per_sample describes the number of quantizer bits of the sound data, for example 16 bits.

The field sound_data_start_address has a data length of 32 bits. The field sound_data_start_address describes the start address of the block data_block of the start bytes of the sound data referred to in the field sound_id in the for loop. The field sound_data_length has a data length of 32 bits. The field sound_data_length describes the byte length of sound data. After the for loop, the block data_block is processed by inserting the word. The block data_block is an area in which the sound data is actually located.

Fig. 66 shows the structure of the file "HdmvSound.bdmv" according to the above syntax. An area from the beginning of the file to the field number_of_sound_entries is called header section 600 . This header section 600 is followed by a for loop section 601 and then a sound data section 602 .

The sound data located in the sound data section 602 are indicated by identifier sound_id=1 to identifier sound_id=n. For example, the end of a certain sound data and the start of the next sound data are joined on a byte boundary. Similarly, the sound data indicated by the identifier sound_id=1 to the identifier sound_id=n are consecutively arranged. In the sound data section 602, the start address is specified with the field sound_data_start_address in the for loop section 601 whose argument corresponds to the identifier sound_id. An area from the specified address specified by the field sound_data_length of the for loop is called sound data corresponding to the identifier sound_id of the argument of the loop.

FIG. 67 shows an example of an application image for reproduction of effect sound according to the third storage method. For simplicity, in FIG. 67, parts similar to those in FIG. 63 are identified by the same reference numerals, and their descriptions are omitted. When the disc is loaded into the player, the disc 400 is initially accessed, and the file "HdmvSound.bdmv" located under the directory BDAV is read out (preloaded) from the disc 400 . The read file "HdmvSound.bdmv" is stored in the external memory 410 of the player.

As shown in FIG. 66, each data segment in the loop portion 601 of the file "HdmvSound.bdmv" is specified based on address information and data length information. Each data segment is identified with a corresponding identifier sound_id. In the example shown in FIG. 67, the file "HdmvSound.bdmv" contains m pieces of sound data. Identifiers sound_id=1, 2, . . . , and m are assigned to m pieces of sound data. The capacity of each piece of sound data is obtained using data length information corresponding to the identifier sound_id and the number of quantizer bits bit_per_sample.

In this example, it is assumed that the sound data has been decoded according to the PCM system. However, the present invention is not limited to the above examples. The sound data may have been compression-decoded according to, for example, the MPEG1 audio layer 3 (mp3) system, the AAC (Advanced Audio Coding) system, or the ATRAC (Adaptive Transform Acoustic Coding) system, and stored in the sound data section 602 . In this case, each piece of sound data is located in the sound data section 602 in such a manner that the end of each piece of sound data is connected to the beginning of the next piece of sound data.

For example, assume that menu screen 420 displays buttons 420A, 420B, 420C. On the menu screen 420, a key 420C is composed of a key image state setting 430 having key images 420C-1, 420C-2, and 420C-3 representing a normal state, a selection state, and an execution state, respectively. When the key image changes according to the above three states, the user can easily distinguish these states. In the example shown in FIG. 67, the sound data with the identifier sound_id=2 is assigned to the key image 420C-2 in the selected state. The sound data with the identifier sound_id=2 is assigned to the key image 420C- 3 in the execution state. When the value of the identifier sound_id is "0", sound data is not output.

When the user operates the key 420C with the remote controller to change the selection state to the execution state, the image of the key changes from the key image 420C-2 to the key image 420C-3. Also, the identifier sound_id=1 assigned to the key image 420C- 3 is read out from the external memory 410 and reproduced.

In other words, the button image 420C-3 is displayed. In addition, based on the address information and data length information for the identifier sound_id=1, the memory 410 is accessed, the identifier sound_id=1 is associated with the button image 420C-3, and the sound with the identifier sound_id=1 is read out from the memory 410 data. The sound data read out from the memory 410 is temporarily stored in the buffer 450B, and supplied to the audio mixer 231 . The audio mixer 231 mixes the sound data read out from the memory 410 with another sound data which is movie data of the main part of the content output from the buffer 450A, and outputs the mixed data.

Similar to the second storage method shown in FIG. 63, in the third storage method, according to the capacity of the memory 410 and the capacity of the buffer 450B, the total capacity of the sound data read from the optical disc 400 and the capacity of each segment of sound data are limited. .

Next, the structure of a graphics object according to the third storage method of the present invention will be described. In the third storage method, the key image composes a Graphical User Interface (GUI) using the graphics plane 12, and the key image is decoded as one stream. Hereafter, the stream with which the button image is combined is called an interactive graphics stream.

For example, the menu screen 420 shown in FIG. 67, the button images of the buttons 420A, 420B, and 420C are combined into an interactive graphics stream 700 (see FIG. 68) and decoded. When a key such as the key 420C is composed of the key image set 430, the key image set 430 has a plurality of key image data 420C-1, 420C-2, and 420C-3, which together with the key image data of the keys 420A and 420B , decoded as an interactive graphics stream 700 . This interactive graphics stream 700 is multiplexed with the transport stream and recorded on an optical disc.

68A and 68B illustrate examples of interactive graphics 700 . As shown in FIG. 68A, the interactive graphic flow 700 has an interactive component program segment 701 and one or more key image data segments 702, 702, . . . and so on. These key image data 702, 702, . . . etc. designate different identifiers object_id. These segments of the above-mentioned key image data 702 are called identifier object_id. The above-mentioned segments of key image data 702, 702, .

As shown in FIG. 68B, interactive graphics stream 700 resides in the payloads of multiple transfer packets assigned to the same PID. Transport packets are multiplexed as a transport stream.

FIG. 69 shows an example of the structure of the interactive component program segment 701. The interactive component program segment 701 is composed of program segment description attribute, display time attribute, button design and a command. The program segment description attribute and the display time attribute describe the display attribute of the button and the attribute information of the display time determined in the interaction position program segment 701 .

The key design and command describe one or more key information 703, 703, . . . etc., which constitute a menu screen like a GUI. In the button information 703, a field button_number describes an identifier button_number that identifies each button. Each button is identified and refers to the identifier button_number. The field selected_state_sound_id describes the identifier button_number which refers to the sound data associated with the button image of the "selected state". The field activated_state_sound_id describes the identifier sound_id which relates to the sound data related to the key image of the "executed state". As shown in FIG. 70, the field neibour_info describes the button numbers Upper_button_number, Lower_button_number, Left_button_number, and Right_button_number on the screen adjacent to this button.

The field Normal_state_object_id describes the identifier object_id of the key image involved when the key is in the "normal state (non-selection state)". The field selected_state_sound_id describes the identifier object_id of the button image involved when the button is in the "selected state". The field activated_state_sound_id describes the identifier object_id of the button image involved when the button is in the "activated state". Field commands describe the operation information of a key assigned to an execution state.

2-15 Another plane example

In the above example, as shown in FIG. 20 , FIG. 21 , and FIG. 22 , for the graphics plane 12, YCbCr (4:4:4) or RGB (4:4:4) can be selected as the color system. However, the present invention is not limited to these examples. In other words, as shown in FIG. 71, the graphics plane 12' can be determined mainly for bitmap data of keys. As shown in Figure 72, similar to the subtitle plane 11, the graphics plane 12' can be composed of 1920 pixels × 1080 lines, the sampling depth of each pixel is 8 bits, and the color system uses 256 colors and the color palette of 8-bit color bitmap addresses plate.

In this case, the film plane 10, the subtitle plane 11, and the graphics plane 12' are composed of the structure shown in FIG. For simplicity, in FIG. 73, parts similar to those in FIG. 22 are denoted by the same reference numerals, and their descriptions are omitted.

Image data of the graphics plane 12' is input to the palette 26A, and output as color data of RGB (4:4:4). When the opacity of alpha blending is designated for image data, the designated opacity α2 (0≦α2≦1) is output from the palette table 26A. The input data and output data of the palette table 26A and the palette data stored in the palette table 26A are the same as those shown in FIGS. 23 and 24 . Therefore, their description will be omitted.

The RGB data output from the palette table 26A is supplied to the RGB/YCbCr conversion circuit 26B, and converted into YCbCr data as unified data of film data. The YCbCr data is output from the RGB/YCbCr conversion circuit 26B, and input to the multiplication device 27 .

When the image data used in the graphics plane 12' is data according to the PNG format, opacity data α2 (0≤α2≤1) may be set for each pixel of the image data. The opacity data α2 is supplied to the multiplying means 27 . The multiplying means 27 multiplies the luminance signal Y color difference data Cb and Cr of the YCbCr data output from the RGB/YCbCr conversion circuit 26B by the opacity data α2. The multiplication result of the multiplication means 27 is input to one input terminal of the addition means 28 . The complement of the opacity data α2, that is, (1-α2) is supplied to the multiplying means 25 .

2-16 Another example on the decoder model

FIG. 74 shows an example of the structure of the player decoder 100', wherein a sound data process for the color system of the graphic object 200 and the graphic plane 12' is provided to the player decoder 100 shown in FIG. Plane 12' has a palette of 256 colors, using 8-bit color bitmap addresses. For simplicity, in FIG. 74, parts similar to those in FIG. 39 are identified by the same reference numerals, and their descriptions are omitted.

Sound data that is not multiplexed with the clip AV stream is input to the input terminal 101 as, for example, data of the input channel (1). Thereafter, the sound data is supplied to the content buffer 105 through the switch circuits 102 and 103 . On the other hand, a clip AV stream whose sound data has been multiplexed with a graphic object 200 is input to the input terminal 202 . The PID filter 110 separates the graphics object 200 from the clip AV stream, and temporarily stores the graphics object 200 in the buffer TBn111A. Thereafter, the graphic object 200 is supplied to the content buffer 105 through the switch circuit 103 .

The graphic object 200 not containing sound data is multiplexed with the clip AV stream and supplied to the input terminal 202 . The PID filter 110 separates transport packets constituting the graphic object 200 from the fragment AV stream. This transfer packet is temporarily stored in the buffer TBn111B, or the buffer TBn111C. The transport packets stored in the buffer TBn111B are supplied to the buffer TBn112B. The graphic object graphic object 200 is collected according to the PID title. The graphics object 200 is supplied to the graphics decoder A500 through the switch circuit 107 . From the transport packets stored in the buffer Bn111C, the graphics object 200 is collected through the buffer Bn112C. The graphics object 200 is supplied to the graphics decoder B501 through the switching circuit 108 .

The graphics decoders A500 and B501 eliminate header information, decoded information data located in the transport packet, and image data for subtitles or bitmap data of graphics from the delivered transport packet.

On the other hand, image data located in a graphics object 200 containing audio data is read from the content buffer 105 . Image data is supplied to graphic decoders A500 and B501 through switch circuits 107 and 108, respectively.

In the example shown in FIG. 74, the graphics decoder A500 decodes PNG format image data. In contrast, the graphics decoder B501 decodes JPEG format image data. Graphics decoders A500 and B501 can decode other format image data. Optionally, the graphics decoders A500 and B501 can decode image data in different formats.

An output of the image decoder A500 is supplied to an input terminal 130B of the switch circuit 130 and an input terminal 131B of the switch circuit 131, and then supplied to the sub-picture plane 502 and the graphics plane 503 through the switch circuits 130 and 130, respectively.

The multimedia engine 106 has a sound player 106D, the structure of which is shown in FIG. 39 . The buffer 109 has a sound buffer 109E whose structure is shown in FIG. 39 . The audio player 106D decodes the audio data read from the content buffer 105 via the audio buffer 109E, obtains the decoded audio data as, for example, linear PCM audio data, and outputs it. The sound data output from the sound player 106D is supplied to the presentation processor 141 . Indicates that the processor 141 mixes the sound data submitted from the sound player 106D with another sound data, and outputs the mixed sound data to the output terminal 142 . The above-mentioned another sound data is delivered from the sound player 106D together with the data delivered from the audio decoder 118 .

The sound data is reproduced by the sound player 106D as an effect sound such as a click sound when a button image is clicked. This audio data is stored in the audio buffer 109E and reproduced by the audio player 106D. The sound player 106D executes a reproduction program as, for example, software for sound data.

The multimedia engine 106 reads out an ECMA script stored in, for example, the encoding buffer 104, analyzes the ECMA script, reads another ECMA script and an HTML document from the encoding buffer 104, and reads image data and audio data. Sound data may be stored in the content buffer 105 like other data.

Furthermore, the multimedia engine 106 receives a user command from a remote controller, a pointing device, etc., and processes the command. The multimedia engine 106 generates control signals based on the processing results of user commands and each script. The control signal is supplied to graphics decoders A500 and B501, audio decoder 118, MPEG video decoder 120 and system decoder 121.

The image data processed by the graphics divider 106C are supplied to the sub-picture plane 502 and the graphics plane 503 through the switching circuits 130 and 131, respectively. The sprite plane 502 and the graphics plane 503 are composed of frame memories, for example. The sub-picture plane 502 and the graphics plane 503 correspond to the subtitle plane 11 and the graphics plane 12' shown in FIG. 71, respectively.

In this example, it is assumed that the image data supplied from the graphics divider 106C to the sub-picture plane 502 and the graphics plane 503 is bitmap data, wherein the image data is decoded by the graphics divider 106C such as PNG format or FPEG format.

In addition, multimedia engine 106 provides a control signal to presentation processor 139 that causes sprite plane 502, graphics plane 503, and cine plane 135 to switch to another plane. The multimedia engine 106 provides a control signal to the presentation processor 141, which controls the output of the audio stream.

Image data on the sub-picture plane 502 is supplied to a palette 504 corresponding to the palette 22 shown in Fig. 73. The 256-color palette is indicated by an index. As a result, RGB data and opacity data α1 are output. RGB data is supplied to a conversion circuit 506, which corresponds to the RGB/YCbCr conversion circuit 29 shown in FIG. As a result, the color system of the image data is converted from RGB (4:4:4) to YCbCr (4:4:4). The YCbCr data output from the RGB/YCbCr conversion circuit 506 is supplied to the presentation processor 139 .

Image data on the graphics plane 503 is supplied to the palette 505 corresponding to the palette table 26A shown in Fig. 73. The 256-color palette is indicated by an index. As a result, RGB data and opacity data α2 are output. RGB data is supplied to a conversion circuit 507, which corresponds to the RGB/YCbCr conversion circuit 26B shown in FIG. As a result, the color system of the image data is converted from RGB (4:4:4) to YCbCr (4:4:4). The YCbCr data output from the RGB/YCbCr conversion circuit 507 is supplied to the presentation processor 139 .

In addition, the movie data on the movie plane 135 is supplied to the presentation processor 139 through the up/down converter 138 .

As shown above, the presentation processor 139 performs an alpha blending process using the opacity α1 of the image data on the subtitle plane 11 (sub-picture plane 502) and the opacity α1 of the image data on the subtitle plane 12' (graphics plane 503). Transparency alpha 2. This process mixes the image data on the film plane 10, the image data on the subtitle plane 11, and the image data on the graphics plane 12'. The presentation processor 139 may perform an effect processing in real time for image data. The image data in which the planes are mixed is supplied to the output terminal 140 . Effect processing has been performed in the plane.

2-17. Another decoder model example

Fig. 75 has shown the example according to the structure of the player decoder 100 " of the 3rd storage method that is used for sound data. For the sake of simplicity, in Fig. 75, the part similar to Fig. 39 and Fig. 74 is denoted by the same figure. Identifies, and omits their description.As mentioned above, in the 3rd storage method that is used for sound data, multi-segment sound data and the effect sound corresponding to button state are included in a file " HdmvSound.bdmv ", and this file is in Under the directory BDAV. When the disc is first accessed, the file "HdmvSound.bdmv" is read and stored in the memory of the player.

As the internal memory of the player, the sound buffer 109E of the buffer 109 can be used. For example, when an optical disc is first accessed, read audio data is input from the terminal 101 . The audio data is supplied to the audio buffer 109E via the switch circuit 102 and the content buffer 105 . When the program code stored in the code buffer 104 is executed, necessary sound data is read from the sound buffer 109E, which is input from the terminal 710.

A real-time stream input as an MPEG ES slave 202 is supplied to the PID filter 110. When the PID of the transport packet indicates that it contains the interactive graphics stream 700, the transport stream is temporarily stored in the buffer 111A. The transport stream is read out at an appropriate timing, input to the switch circuit 103 that selects the input terminal 103B, and stored in the content buffer 105 through the switch circuit 103 .

The interactive component program segment 701 of the interactive graphics stream 700 is read from the content buffer 105 and provided to the multimedia engine 106 . In addition, key image data 702, 702, .

Sound data input from the terminal 710 is temporarily stored in the buffer 711 . The mixer 712 mixes the sound data submitted by the terminal 710 with another sound data which is obtained from the audio decoder 118 at a predetermined mixing ratio, and supplies the mixed sound data to the presentation processor 141. submitted. The mixing ratio of the mixer 712 can be set at the player decoder 100" end under the control of the sound player 106D in response to user input.

In FIG. 75, the buffer 711 and the mixer 712 are separated for convenience. Actually, however, the buffer 711 and the mixer 712 are built in the presentation processor 141 . Thus, considering the connection with the structure shown in FIG. 67 , the buffer 711 corresponds to the buffer 450B, and the mixer 712 corresponds to the audio mixer 231 . The buffer 450A shown in FIG. 67 is built in the presentation processor 141.

When sound data has been compression-encoded and recorded on an optical disc, the sound data read from the optical disc is decoded into linear PCM audio data and stored in the buffer 711 . For example, the sound player 106D decodes the compression-coded sound data read from the optical disk, and stores the decoded sound data in the sound buffer 109E. The sound data may be decoded by the audio decoder 118 . Optionally, the sound data can also be decoded by software.

In the above-mentioned example, in the decoder shown in FIG. 75, before the content reproduction of, for example, a film recorded on the optical disc, the sound data as the effect sound is preloaded from the optical disc and stored in the internal memory of the player decoder. . When using the directory and file structure shown in FIG. 67, only the file "HdmvSound.bdmv" for sound data as effect sound can exist in the directory BDMV. When the player initially accesses the disc, the player preloads sound data files for effect sounds into the player's internal memory.

Except for this example, as shown in FIG. 76, it can be considered that the play list is realized by data for reproducing effect sounds. In this case, through, for example, the sub play item shown in FIG. 15, it is possible to access the sound data to be reproduced as the effect sound assigned to the key. For example, a file name (file "HdmvSound.bdmv") for sound data is described in the SubPlayItem.

Alternatively, as a method for adding reference information to a file containing sound data as effect sound, a block UIAppInfoPlayList() shown in FIG. 14 may be used. In this case, a field for the filename "HdmvSound.bdmv" can be added to the syntax of the block UIAppInfoPlayList().

Before reproducing a playlist, the player preloads into the external memory a sound data as an effect sound indicated by reference information (for example, sub play item) of a file including a sound as an effect sound data.

Fig. 77 shows an example of the management structure of files corresponding to another decoder mode. In the example shown in FIG. 77, under the directory STEAM, there are files "HdmvSound01.bdmv", "HdmvSound02.bdmv" and the like for sound data as sound effects. The above-mentioned files "HdmvSound01.bdmv", "HdmvSound02.bdmv", etc. have the same structure as the files "HdmvSound.bdmv" in Fig. 65 and Fig. Thus, a plurality of files for sound data as effect sounds can be recorded on the optical disc. A file for sound data can be indicated with reference information of a sound data file as effect sound (for example, sub play item or block UIAppInfoPlayList() as playlist attribute information). In this example, the type of sound data as effect sound can be changed for each playlist.

3. The second mode of the present invention

Next, a second mode of the present invention will be described. According to the first mode of the present invention, a reproduction program for reproducing data on an optical disc is described using ECMA script and HTML document. On the contrary, according to the second mode, a reproduction program is described with original data structure and description language instead of ECMA script and HTML document. The script description language used in the second mode of the present invention determines a command group to initially display control commands on the basis of adjustment navigation commands for DVD-Video. Thus, according to the second mode of the present invention, it is possible to display a more suitable and flexible menu screen for the Blu-ray disc than the first mode.

3-1. About the script

According to the second mode of the present invention, since the initially determined script description language is used, it is not necessary to determine an event handler different from the ECMA script used in the first mode. For example, since the initially determined script description language may require predetermined events necessary to execute the script, there is no need to determine the events of the script in the program.

Since the initially determined script description language is used, the external structure of the script shown in FIG. 26 does not require the overall event handler definition 71 and the local event handler definition 72 . Fig. 78 shows a script 70' described according to the second mode of the present invention. Playlist 73A depicts the incoming playlist as the start of script 70', which is executed first after the disc is loaded into the player. When a predetermined command is issued on the screen, the play list 73B is reproduced. After playlist 73B is played back, playlist 73C is played back. In the example shown in FIG. 78, when the play list 73C is reproduced, the graphic data 74A is read out, and a menu screen 80A is displayed, which prompts the user to select a branch of the story.

Thereafter, as shown in Fig. 26 of the first mode, the scenario 70' is reproduced. When a flag is set in the playlist, a branch or combination of the playlist can be implemented at the flag.

Each playlist has a series of commands (a program) for operations that are performed when a flag, user's input, or player's reproduction change is detected. The player executes a program to realize the above operations.

Even if any one of the playlists 73A to 73M of the scenario 70' is being reproduced, when the menu key of the remote control commander is pressed, the menu screen 60 for the list of scenarios is displayed (see FIG. 25). Next, the process of reproducing a playlist for the menu screen 60 will be described. In this case, when the menu button of the remote control commander is pressed, an event (menu button "on" event) is executed, and a command is used as the event handler corresponding to the event, and the above command will be used for the displayed The playlist of the menu screen 60 is described as a scenario.

3-2. Script directory

Define a script in the directory BDVM. A scenario consists of one or more playlists. The contents of scripts will be described with reference to FIGS. 79 , 80A, 80B, and 80C. Based on the connection of playlists, the structure of scripts can be roughly divided into three categories: (1) single playlist, (2) continuous playlist, (3) multiple playlists as shown in FIG. 79 .

A single playlist classified as (1) is a script composed of one playlist shown in FIG. 80A. For a single playlist, a timeline can be determined. There is no interruption during the reproduction of the script. When the content of a single playlist is a movie, only the main part of the movie is reproduced after the disc is loaded.

A continuous list classified as (2) is a scenario composed of a plurality of playlists arranged linearly without branching as shown in FIG. 80B . The above playlists are arranged in such a way that the end of one playlist is connected to the beginning of the next playlist. In continuous playlists, a timeline can be defined for each playlist. When the content of the continuous playlist is a movie, the scenario is composed of the menu screen and the main part of the movie. After the disc is loaded, a playlist is executed, which displays a menu screen. When the reproduction of the main part of the movie is designated on the menu screen, the next playlist is executed, and the main content of the movie is reproduced.

A plurality of playlists classified as (3) is a scenario with a junction which is a branch or combination of playlists. In multiple playlists, one timeline cannot be determined through all playlists, but one timeline is determined in each playlist. With a plurality of playlists, an interactive function and a game function of changing reproduced content according to user's input can be realized. When the contents of multiple playlists are movies, a multi-angle function can be implemented, which allows the user to select a desired angle among different angles shot for the same scene.

As will be described below, a script is determined for the directory BDMV. However, it is necessary to allow users to recognize scripts in smaller units. However, the units of the playlist do not always coincide with the units recognizable by the user. When a playlist describes three titles, it is necessary to allow the seek point for each title to be seen. The search point (entry point) is independent of the structure of the playlist, which will be described later as title and/or chapter.

Next, referring to FIG. 81 , titles and chapters are described. A title represents any reproduction starting point in a script. In the example shown in FIG. 81, Title 1 is located at the beginning of PlayList 470A. Title 2 is located in the middle of playlist 470D. The area up to Title 2 after the start of PlayList 470A is Title 1 . Chapters are the units into which titles are subdivided. A chapter can also be considered as a playback start point. Title 1 is subdivided into chapters. In the example shown in FIG. 81, title 1 has chapters 1, 2 and 3. Thus, Title 1 is divided into three parts. As shown in FIG. 81, each title and chapter can be located in the middle of the playlist.

3-3. About the virtual player model

According to the second mode of the present invention, the script described in the script description language can be reproduced by the BD virtual player model according to the first mode of the present invention shown in FIG. 27 . After the disc is loaded into the BD virtual player 30, it regards the script as the PBC program 40, and operates according to the description of the script described by the initially determined script description language according to the second mode of the present invention. The BD virtual player 30 is controlled according to an event that occurs when the BD virtual operator 30 is operated.

The script has two areas for commands, including a program with commands, which operate the player. The above two areas are called the overall command area and the local command area.

The global command area has global commands, which are valid for the entire script. For example, the overall program describes a program that causes the player to initialize parameters when a disc is loaded into the player, and to jump to a playlist constituting the menu screen. The local command area describes the program of the playlist. In the second mode of the present invention, partial commands are divided into four categories, which are pre-commands, item reproduction commands, post-commands, and key commands (pre_commands, play item commands, postcommands, and button commands).

82A and 82B show operations of the BD virtual player 30 corresponding to commands described as scripts according to the second mode of the present invention. FIG. 82A shows an example of the disc loading operation of the BD virtual player 30. As shown in FIG. As above, create a script for the directory BDMV. When the disc is loaded into the player and an initial access is performed for the disc (at step S30), the register, that is, the common parameter 32 is initialized (at step S31). In the next step S32, the program is read from the optical disk and executed. First access means an operation for reproducing a disc for the first time when the disc is loaded in a player, such as access initialization.

A command group (program) that is initially read out and executed when a disc is loaded into a player is called an overall command. The overall command describes, for example, an advertisement screen (trailer) and a jump command which jumps to a playlist constituting the menu screen. The player reproduces the playlist according to the above command.

Fig. 82B shows the operation of the player 30 when the player is in a stopped state and the user presses, for example, a reproduction key. This operation corresponds to a state change from state B to state A in FIG. 28 . In the stopped state (in step S40 ), the user causes the player 30 to operate in the reproduction mode using, for example, a remote controller (UOP: User Operation). As a result, the register (that is, the common parameter 32) is initialized (at step S41). In the next step A42, the BD virtual player 30 enters the playlist reproduction phase.

Next, with reference to FIGS. 83A and 83B , reproduction of a playlist in the playlist stage will be described. Fig. 83A shows an example in which a PlayList is composed of a single PlayItem. The playlist has a pre-command area, a playitem command area and a post-command area, which describe the respective commands. In the playlist reproduction stage, the pre-command of the pre-command area is executed (at step S10). After executing the pre-command, the player enters the play item reproduction stage of the play items that make up the play list (at step S11). In the playitem reproduction stage, a stream whose start point and end point are specified by a playitem is reproduced (at step S110). When the above-mentioned reproduction reaches the end point, a playitem command is executed (at step S111). After the PlayItem command is executed, the post command of the post command area is executed (at step S12). As a result, reproduction of the playlist is completed.

The subsequent command is usually a skip command, which is described as a skip command to be reproduced by a playlist, including the playlist of the menu screen. When there is no jump command, the player enters a stop state (state B shown in FIG. 28).

Fig. 83B shows an example in which a PlayList describes a plurality of PlayItems. In this case, the playlist has a pre-command area, playlist command area, and post-command area, which describe each command. When the playlist describes a plurality of playitems, the playitem command area describes a playitem stream and a playitem command of the playitems arranged in time sequence.

When the PlayList describes a plurality of PlayItems, in the PlayList reproduction stage, a pre-command is executed (at step S10). In the next playitem reproduction stage, one stream is reproduced from the start point and end point of each playitem, and one playitem command is executed for each playitem. In the example shown in FIG. 83B, the first play item stream is reproduced (at step S110-1). Thereafter, the corresponding PlayItem command is executed (at step S111-1). Thereafter, a second play item stream (not shown) is reproduced (at step S110-2). The corresponding PlayItem command is executed (at step S111-2). Repeat these operations a number of times equal to the number of play items. After the last playitem stream is reproduced (at step S110-n) and the corresponding playitem command is executed (at step S111-n), the playitem reproduction phase is ended (at step S12). As a result, the playlist reproduction phase ends.

According to the second mode of the present invention, scenarios, playlists, and playitems which are executed on the BD virtual player 30 can be considered hierarchically. In other words, a script layout 601 is placed on the BD virtual player 600 as shown in FIG. 84A. A playlist pane 602 with one or more playlists is located above the script pane 601 . A play item (PI) pane 603 is located above the playlist pane 602 . Each playlist has one or more play items.

In the hierarchical structure, playlists and playitems are executed by the BD virtual player through the script layout 601 . Thus, when a control command for a playlist is described in the script, branching of the playlist and the like can be easily realized. The same applies to the PlayItem shown in Fig. 84B.

3-4. About grammar

Next, a method for storing commands and a database describing a script according to the second mode of the present invention will be described on an optical disc. In the second mode of the present invention, it is assumed that data necessary to realize the extended functions of the BDAV format is described in the scenario file "scenario.pbc". The script file "scenario.pbc" is located under the directory BDAV in the file management structure, and the above-mentioned file is recorded on the recording medium specified in "Blu-ray Disc Rewritable Format Version 1.0 Part 3" shown in FIG. 9 .

The syntax shown in Fig. 86 describes an example of the structure of the scenario file (scenario.pbc). The file "scenario.pbc" successively contains a file identifier symbol, a version number, and a block start address. The file "scenario.pbc" also includes blocks corresponding to functions. The file type_indicator has a data length of 32 bits, and describes a predetermined character string indicating that the file contains a script. A field version_number has a data length of 32 bits, and describes a version number. The field TitleEntry_start_address has a data length of 32 bits of an unsigned integer, and describes a value indicating the position at the start of the block TitleEntry( ) in the relevant number of bytes from the start of the file "scenario.pbc". Similarly, the field scenario_start_address has a data length of 32 bits of an unsigned integer, and describes the position at the start of the block scenario() with a value indicating the relative number of bytes from the start of the file "scenario.pbc".

The block GlobalCommand() starts at a fixed position, starting at byte 41 of the file. The block GlobalCommand( ) describes a program that is executed when the player initially accesses the disc (that is, when the player initially reproduces data from the disc after the disc is loaded in the player). The overall command is located in the block GlobalCommand(). The block GlobalCommand() is followed by any number of padding_words so that the blocks are separated from each other.

Block TitleEntry() describes a list of search points in the script. Create a script for directory BDAV. The script determines the reproduction order of a plurality of playlists located under the directory BDAV. The script that the user sees looks as if it is composed of multiple "titles" rather than a single image/sound unit.

When a disc contains three titles, the user sees the disc as if it contains three titles even if the disc has only one script which determines the reproduction order of the films. Alternatively, the user sees the disc as if divided into four titles, including a title menu screen with which the user can select one of three titles. Since the user can regard the menu screen as a picture/sound unit, according to the second mode of the present invention, the menu screen can be regarded as a title.

Therefore, since the script unit that determines the connection of the playlist is different from the unit that the user regards as an image/sound block, it is necessary to determine a search point in the script. Search points in scripts can be thought of as header entries.

The block Scenario() describes a "script". The block Scenario( ) describes information about the reproduction order of PlayLists, the local command area of each PlayList, and the like.

The syntax shown in Fig. 87 describes an example of the structure of block GlbalCommand(). The field length has a data length of 32 bits, an unsigned integer, and describes the byte length from the end of the field length to the end of the block GlobalCommand(). The field number_of_commands describes the number of the following field command. A field command has a data length of 32 bits, and describes a set of parameters for the player, a reproduction start command for specifying a playlist, a calculation command, and the like.

The syntax shown in Fig. 88 describes an example of the data structure of the block TitleEntry(). The field length has a data length of 32 bits, an unsigned integer, and describes the byte length from the end of the field length to the end of the block TitleEntry(). The field number_of_Titles describes the number of search points for the titles described by the for loop following this field. The field Entry_Playlist_file_name describes the file name of a playlist containing a title of a seek point. The field Title_name_character_set describes the character set representing the next field Title_name. The field Title_name describes the character string assigned to the name of the search point.

The field number_of_Chapters describes the number of chapters described later in the for loop. As mentioned above, chapters are sections of titles. Users can see chapters as headings. Chapters are used as search points for scripts. The field Title_number has a data length of 16 bits, an unsigned integer, and describes a title number in which a chapter corresponds to a current loop counter value of a for loop (hereinafter, a chapter is referred to as the chapter). The field chapter_entry_Playlist_file_name describes the file name of the playlist file of the playlist indicated by this chapter. The field chapter_ref_to_PlayItem_id describes the playlist item number indicated by this chapter. The field chapter_time_stamp describes the time of the PlayItem indicated by the chapter.

The block Title_Menu_Call_PL( ) describes the playlist that makes up the displayed menu when the user causes the player to reproduce the title of the optical disc. The user designates a title by pressing, for example, a title menu key on a remote commander that remotely operates the player. Each script has a title menu that the user can use to see the title's search point. When the user selects his or her desired title on the title menu using, for example, a remote commander, the player gets a playlist from the title entry list corresponding to the information described in the for loop after the field number_of_Title, and start reproducing the playlist.

In the block Title_Menu_Call_PL(), field tags describe attribute information about the title menu. The field TitleMenu_entry_PlayList_name describes a playlist constituting a title menu, or a playlist of an entry point of a playlist group. The field TitleMenu_ref_to_PlayItem_id describes a PlayItem number of a PlayItem which is the start of a PlayList. When reproducing a PlayList from the beginning, the value of the field TitleMenu_ref_to_PlayItem_id is [0].

The field TitleMenu_chapter_time_stamp describes the time when the item was played. The field TitleMenu_chapter_time_stamp describes the start time of a PlayItem when a PlayItem is reproduced from the beginning. The field UOP_mask_table() describes the information about which operations the user is restricted from. When the user performs an operation described in a field UOP_mask_table(), the player does not respond to the user's operation. When a user is prohibited from performing a fast forward operation, the prohibited user operation is described in the field UOP_mask_table().

The syntax shown in Fig. 89 describes an example of the structure of block scenario(). The block scenario() describes information about the script, that is, the link to the playlist. The field length describes the length in bytes from the end of the field to the end of the block scenario(). The field number_of_Playlist describes the number of playlists constituting one scenario. After the field number_of_Playlist is a for loop. In the for loop, describe information about the playlist (the playlist is called the playlist), which is a loop counter corresponding to the for loop.

After the for loop is the data of the playlist. The field flags describes the attribute information of the playlist. Since the data described by the field flags is not directly related to the second mode of the present invention, description thereof will be omitted. The field PL_UOP_mask_table( ) describes the user restricted operations in each playlist. When the above PlayList is reproduced, only user operations allowed in both the command UOP_mask_table() (to be described later) and the field PL_UOP_mask_table() can be performed. However, the final determination of whether the user's operation can be performed is based on the data described in the block PL_UOP_mask_table(), which will be described below together with the above commands and fields.

The field Parental_level describes necessary information to restrict viewers viewing the playlist. This information describes the age and age group of viewers who can view the playlist.

The field number_of_Pre_Commands describes the number of commands (pre-commands) constituting a program executed before reproducing the PlayList. The program is described in the field PL_Command(i). The field number_of_Post_Commands describes the number of commands constituting the program (pre-commands) executed before reproducing the PlayList. A program is described in the field PL_Command(i). The field Pre_Commands_Start_id describes the start number of the program executed in the command table before reproducing the PlayList. This number corresponds to parameter i of field PL_Command(i). The field Post_Commnads_Start_id describes the start number of a program which is executed in the command table after the playlist is reproduced. This number corresponds to parameter i of field PL_Command(i).

The field number_of_PlayItems describes the number of PlayItems constituting the PlayList. The field PL_UOP_mask_table( ) describes information about restrictions on user operations in each PlayItem. When reproducing a PlayItem, only user operations allowed in three types of tables, command UOP_mask_table() (to be described later), command PL_UOP_mask_table(), and field PL_UOP_mask_table(), can be performed.

The field PI_Commands_Start_id describes the start number of the command executed in the command table after PlayItem reproduction. This number represents parameter i in field PL_Command(i). The field number_of_PL_Commands describes the number of commands in the command table represented by the following for loop. The commands contained in the field PL_Command(i) are consecutively designated by the number i. The field PL_Command(i) describes a command. The number i is indicated by a field Pre_Commands_start_id, a field Post_Commands_start_id, a field PL_Commands_start_id, and so on.

3-5. Related commands

Fig. 90 shows an example of commands used in the script according to the second mode of the present invention. The above command is a method that the BD virtual player 300 has. However, it should be noted that the commands shown in FIG. 90 are commands used in a part of the script in the mode according to the second aspect of the present invention. Actually, more commands can be determined in order to realize various functions of the user interface according to the first mode of the present invention. For example, a command for reproducing sound data and a command for displaying keys may be further determined.

Next, a method for specifying a reproduction start position will be described. The method LinkPlayList(PlayListNumber) starts reproduction of a playlist specified by "PlayListNumber". The method LinkPlayItem(PlayListNumber, PlayItemNumber) starts reproduction of the play item specified by the play list. "PlayListNumber" is "PlayItem_id" whose value starts from "0". When the value of "PlayItemNumber" is specified as "0", the PlayList to which the PlayItem belongs is played back from the beginning.

The method Link(position)(object) jumps from the current position to the previous or subsequent playlist, playitem or chapter in the script. The parameter "position" describes one of "prev", "next", "top", "parent" and "tail". The parameter "object" describes the jump method for an object (playlist, playitem or chapter).

The method Exit stops the reproduction of the script. In this case, the value of the standard register is not maintained. The method RSM accesses the resume information stored in the memory of the player, sets the resume information to a register, and reproduces the scenario.

Next, a method for obtaining the status of the player will be described. The method getMenuDescriptionLanguage() gets the language of the displayed menu. The method getScenarioNumber( ) and method getChapterNumber( ) respectively obtain the scenario number, playlist number and chapter number being reproduced. The method getPlayerSupport() gets the version information of the player.

Next, a method for video streaming will be described. The method getVideoStreamAvailability() gets an information indicating whether a specified video stream is included. The method setVideoStreamNumber() gets the number of the selected video stream. The method getVideoStreamAttr() gets the attributes of the selected video stream. The attributes are, for example, the decoding system of the video stream, the resolution, the aspect ratio, the display mode when the aspect ratio is 4:3, and closed caption presence/absence. The method setAngleNumber() describes an angle number. The method getAngleNumber() gets a selected angle number. The method getMaxVideoStream() gets the maximum number of video streams.

To reproduce content data from an optical disc according to the script file "scenario.pbc", add the An engine that parses the script description language according to the second schema.

When the disc is loaded into the drive device (not shown) of the player decoder 100' shown in FIG. 74, first, the scenario file "scenario.pbc" is reproduced. The reproduced scenario file "scenario.pbc" is input from the input terminal 101 to the player decoder 101' as a storage object. The scenario file "scenario.pbc" is supplied to the code buffer 104 through the switch circuit 102 . The script file "scenario.pbc" analyzes the engine of the multimedia engine 106 . Read out the script data according to the analysis result. Movie data displayed on the movie screen 10, image data displayed on the subtitle plane 11 and graphics plane 12 (or graphics plane 12'), another script file, sound data, etc. read from the playlist file and a script file , are read from the disc.

4. Other

In the above example, the first, second and third storage methods of sound data are applied to the first mode of the present invention. However, the first, second and third storage methods of sound data are applied to the second mode of the present invention.

In the above example, the present invention is applied to Blu-ray discs. However, the present invention is applicable to large-capacity disc-type recording media of other systems.

Although the present invention has been shown and described with reference to preferred embodiments, it will be understood by those skilled in the art that the above and other various changes, omissions, and additions may be made in the form and details thereof without departing from the spirit and scope of the invention. .

Claims (13)

1. A reproducing device for reproducing content data recorded on a disc-type recording medium, the device comprising: Input means for inputting a non-real-time stream reproduced from a recording medium and a real-time stream, the non-real-time stream including stored program codes, image data constituting an operation screen to prompt the user to perform an operation and for reproducing to be assigned to the A file of several pieces of sound data that describe the effect sound of the image data, and the real-time stream includes at least movie data and subtitle data; a storage device for storing the program code input by the input device; an image data storage device for storing image data input by the input device; A first combining means for combining the decoded film data inputted and decoded by the input means and the decoded subtitle data inputted and decoded by the input means; and The second combination means combines the decoded image data stored in the image data storage means and the combination data of the decoded movie and subtitle data combined by the first combination means according to the program code stored in the code storage means. 2. A reproduction apparatus as claimed in claim 1, Wherein the position information of each piece of sound data among the multiple pieces of sound data stored in the file is associated with the identification information of the sound data. 3. A reproduction apparatus as claimed in claim 2, wherein sound data is associated with image data based on the identification information, and Wherein when the image data is displayed, the audio data associated with the image data is read out from the file according to the identification information. 4. The reproducing apparatus as claimed in claim 1, The file is input from the input device before the program code is executed. 5. The reproducing apparatus as claimed in claim 1, Wherein a plurality of said files are input from an input device. 6. A reproduction apparatus as claimed in claim 5, Where any one of the above-mentioned files is referred to from the program code at each reproduction unit of the program code, and the file is input from the input means every time it is accessed. 7. A reproducing method for reproducing content data recorded in a disc-type recording medium, the method comprising the steps of: Inputting a non-real-time stream and a real-time stream reproduced from a recording medium, the non-real-time stream including a file storing program codes, image data constituting an operation screen to prompt a user to perform an operation, and pieces of sound data for reproducing effect sounds, The real-time stream includes at least movie data and subtitle data; a step of storing the program code input in the input step; a step of storing the image data input in the input step; combining the decoded movie data inputted and decoded in the input step and the decoded subtitle data inputted and decoded in the input step; and The decoded image data stored in the second storing step and the combined data of the decoded film and subtitle data combined in the first combining step are combined based on the program code stored in the first storing step. 8. A recording method for recording content data, the method comprising: Record non-real-time streams and real-time streams, the non-real-time streams at least include program codes, image data forming an operation screen to prompt the user to perform an operation, the real-time streams include at least movie data and subtitle data; Record a file stored as a set of multi-segment sound data for recording effect sounds; According to the program code, the stored decoded image data and the combined data of the decoded film and subtitle data are combined. 9. The recording method as claimed in claim 8, Wherein the position information of each piece of sound data among the multiple pieces of sound data stored in the file is associated with the identification information of the sound data. 10. The recording method as claimed in claim 9, wherein the sound data is associated with the image data according to the identification information, and Wherein when the image data is displayed, the sound data associated with the image data is read from the file according to the identification information. 11. The recording method as claimed in claim 8, Wherein the file is read in advance before executing the program code. 12. The recording method as claimed in claim 8, Multiple files are recorded in it. 13. The recording method according to claim 11, Wherein the program code of the reproducing unit from the program code refers to any one of the above-mentioned files, and the file is input from the input device every time it is accessed.

Images