JP2001119302A - Encoding device, decoding device, information processing system, information processing method, and storage medium - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To provide an information processing system capable of sharing objects between a wide range of devices by enabling reproduction of objects beyond categories. SOLUTION: On a decoding side 130, 140, a notifying means 131, 141 acquires a state of its own device (a decoding function of its own device) and notifies the encoding side 110 of the state information. On the encoding side 110, encoding means 114,
117 and 119 encode the information in an encoding method that can be supported by the decoding device to which the encoded information is output, based on the status information from the decoding sides 130 and 140.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for processing coded data obtained by a coding system for integrally handling a natural language and a description language of computer graphics (hereinafter referred to as "CG"). The present invention relates to an encoding device, a decoding device, an information processing system, an information processing method, and a storage medium in which processing steps for executing the same are stored in a computer readable manner.

[0002]

2. Description of the Related Art Conventionally, for example, "Motion J" which is an intra-frame encoding system has been used as a moving image encoding system.
"H.26" using an encoding method such as "PEG" or "Digital Video" or inter-frame predictive encoding.
1 "," H.263 "," MPEG-1 "," MPEG
-2 "and the like are known.

[0003] The above-mentioned encoding method is based on ISO (In)
international Organization
for Standardization (International Organization for Standardization) and ITU (International Te1e)
communication Union (International Telecommunication Union).

[0004] International standardization work on the "MPEG-4" encoding system is underway as a general-purpose next-generation multimedia encoding standard that can be used in many fields such as computer, broadcasting, and communication.

[0005] "MPEG-4" has a feature that, unlike conventional encoding methods in units of frames or fields, encoding can be performed in units of objects, and techniques such as animation and CG can be integratedly handled. ing. Accordingly, encoding of animation of a human face or body (FBA encoding), encoding of a textured mesh (mesh encoding), or encoding of a still image is the same as encoding. It becomes possible in the scheme.

[0006] Also, "MPEG-4" is different from a conventional encoding method used in a system that emphasizes realizing synchronization between media and the like, and is capable of describing a scene as used in CG. It has become. This makes it possible to represent the display method and attributes of a plurality of objects.

The description of the scene described above is based on VRML (Virtual Realiy) widely used in the CG world.
BIFS (Binary Format for) based on ty Modeling Language
Scene). The details of this are described in the system part (I.
SO / IEC 14496-1). This allows for integrated visual and audio synchronization, C
It is possible to compose an image by the description of G.

[0008] As a system adopting the above-mentioned "MPEG-4" coding method, for example, there is a data processing system 800 as shown in FIG. The data processing system 800 includes an MPFG-4 encoder (hereinafter, simply referred to as “encoder”) 810 that performs an encoding process according to the “MPEG-4” encoding scheme, and an encoder 810.
And an MPEG-4 decoding device (hereinafter, simply referred to as a “decoding device”) 820 that decodes the data after the encoding process in FIG.

The encoding device 810 includes a scene setting unit 81
1, a BIFS encoding unit 812, a storage unit 813, a visual encoding unit 814, a storage unit 815, an audio encoding unit 816, and a multiplexer 817.

[0010] The decoding device 820 includes a demultiplexer 821, a BIF
It comprises an S decoding unit 822, a rendering unit 823, a visual decoding unit 824, a synthesizer 825, a display 826, an audio decoding unit 827, and a speaker 828.

In the above-described configuration, first, the storage unit 813 of the encoding device 810 stores a moving image of a natural image obtained by imaging (hereinafter, such a moving image is referred to as a “video”). Data, such as data representing the structure and movement of the face and body by CG, data representing texture by mesh, and the like are stored. The storage unit 815 stores audio data obtained by recording.

Therefore, for example, in the encoding device 810, the user selects image data and audio data necessary for a scene from the storage unit 813 or the storage unit 815 using the scene setting unit 811 and sets the display and output timings. decide. Also, a CG required for the scene description is configured. According to such an instruction from the user, the scene setting device 811
Generates scene description information such as a method of outputting a moving image and a sound of a natural image, and a command of an image generated by CG.

The BIFS encoding section 812 encodes the scene description information generated by the scene setting device 810.

The image data selected from the storage unit 813 is input to the visual encoding unit 814.

The visual encoding unit 814 includes a storage unit 81
3 is encoded by the encoding method described in the visual part of the MPEG-4 encoding method (ISO / IEC 14496-2). Specifically, the visual encoding unit 814 performs video encoding for encoding video, FBA encoding as a non-natural image encoding scheme, mesh encoding, still image encoding for encoding still images, and the like. Encoding can be performed in various encoding systems, and the data is encoded using an encoding system suitable for the image data to be encoded.

The audio data selected from the storage unit 815 is input to an audio encoder 816.

The audio encoding unit 816 includes a storage unit 81
5 from the audio part of the MPEG-4 encoding system (ISO / IEC 14496-).
Encode by the encoding method described in 3). Specifically, the audio encoder 816 is used for audio (for example, C
ELP encoding), audio (for example, AAC encoding),
Alternatively, it is possible to perform encoding using various encoding schemes such as description for synthesizing a character code (for example, TTS encoding), and the data is encoded in an encoding scheme suitable for the audio data to be encoded. Is performed.

The multiplexer 817 includes a BIFS encoding unit 81
2, the encoded data (BIFS encoded data) obtained in step 2, the encoded data (visual encoded data) obtained in visual encoding section 814, and the data obtained in audio encoding section 816. The encoded data (audio encoded data) is multiplexed and output according to the multiplexing method described in the system part of the MPEG-4 encoding method.

The multiplexed data output from the encoding device 810 is input to a decoding device 820, and is separated into respective encoded data by a separator 821.
That is, the demultiplexer 821 outputs the output of the multiplexer 817 to the BIFS coded data, the visual coded data, and the audio coded data by the operation reverse to the multiplexing operation of the multiplexer 817 of the coding device 810. Separate into data.

The BIFS decoding section 822 decodes the BIFS encoded data obtained by the separation in the separator 821,
A CG command or the like for describing a scene is reproduced.
The rendering unit 823 performs rendering in accordance with a CG command or the like reproduced by the BIFS decoding unit 822 to generate image data (such as a CG image).

The visual decoding unit 824 includes a separator 821
Decodes the visual coded data obtained by the separation in step (1), and reproduces or generates image data. At this time, the visual decoding unit 824 decodes the encoded data using a decoding method suitable for the encoded data to be decoded (a decoding method corresponding to the encoding method used for the encoded data).

The combiner 825 combines the image data reproduced or generated by the renderer 823 and the visual decoder 824. The display 826 displays the synthesizer 8
The composite image data obtained in step 25 is displayed.

The audio decoding unit 827 includes a separator 821
Then, the audio data is reproduced by decoding the encoded audio data obtained by the separation in step (1). At this time, the audio decoding unit 827 decodes the encoded data using a decoding method suitable for the audio encoded data to be decoded (a decoding method corresponding to the encoding method used for the encoded data).

The speaker 828 is connected to the audio decoding unit 82
The audio data reproduced in step 7 is output as sound.

As described above, the "MPEG-4" encoding method is very versatile and can handle various media in an integrated manner. However, a very large-scale configuration is indispensable to provide all of these functions. For this reason, they are classified into several categories using the concept of profiles and levels. As a result, a terminal that simply plays back video (a terminal device dedicated to video playback),
It has become possible to widely support various devices such as a device that can freely handle CG and the like.

[0026]

However, in the above-described conventional encoding method, the types of objects that can be reproduced by terminals of the same category are guaranteed, but objects of different categories are not reproduced. There was a problem. Thus, for example, C
Encoded data created by an encoding device that can freely handle G or the like cannot be reproduced by a terminal device dedicated to video reproduction.

Therefore, the present invention has been made to eliminate the above-mentioned drawbacks. By enabling reproduction of an object beyond a category, the object can be shared between a wide range of devices or systems. It is an object of the present invention to provide an encoding device, a decoding device, an information processing system, an information processing method, and a storage medium in which processing steps for executing the same are stored in a computer-readable manner.

[0028]

For such a purpose,
A first invention is an encoding device capable of encoding arbitrary information by a plurality of types of encoding schemes, and a decoding apparatus that acquires a decoding side state of encoded information as a result of the encoding. Side state acquiring means, and encoding means for encoding information based on the decoding state information obtained by the decoding state acquisition means, using an encoding method selected from the plurality of types of encoding methods. It is characterized by having.

According to a second aspect, in the first aspect,
The encoding unit includes a plurality of encoding units corresponding to the plurality of types of encoding schemes, and an encoding unit that selects information to be encoded from the plurality of encoding units based on the decoding-side state information. Means for supplying information to the means.

According to a third aspect, in the first aspect,
The plurality of types of encoding schemes include an encoding scheme of the MPEG-4 encoding scheme.

According to a fourth aspect, in the first aspect,
The encoding unit encodes description information for generating computer graphics image information, and a second encoding unit encodes at least the computer graphics image information using a natural image encoding method. Encoding means for encoding information by means selected from the first and second encoding means based on the decoding-side state information.

According to a fifth aspect, in the first aspect,
The encoding means comprises: first encoding means for encoding description information for generating synthesized speech information; and second encoding means for encoding at least the synthesized speech information by a natural acoustic encoding method. And information is encoded by means selected from the first and second encoding means based on the decoding-side state information.

According to a sixth aspect, in the first aspect, description information for encoding computer graphics image information is encoded.
Encoding means; encoding means for encoding the computer graphics image information by a natural image encoding method; encoding information obtained by the first encoding means; And output means for selecting and outputting any of the encoded information obtained by the encoding means.

According to a seventh aspect of the present invention, there is provided a first encoding means for encoding description information for generating synthesized speech information, and a second encoding means for encoding the synthesized speech information by a natural acoustic encoding method. Means, and output means for selecting and outputting one of the encoded information obtained by the first encoding means and the encoded information obtained by the second encoding means. It is characterized by the following.

According to an eighth aspect, in the sixth or seventh aspect, the decoding state acquisition means for acquiring the state of the decoding side, which is the output destination of the output means, and the decoding state acquisition means. Control means for controlling the selective output operation of the output means based on the decoding-side state information.

In a ninth aspect based on the fourth aspect or the sixth aspect, the first encoding means comprises a code of one of a BIFS encoding method and a non-natural image encoding method of an MPEG-4 encoding method. The description information for generating the computer graphics image information is encoded by an encoding method, and the second encoding unit is configured to encode at least the computer graphics image by a natural image encoding method of an MPEG-4 encoding method. The information is encoded.

In a tenth aspect based on the fifth aspect or the seventh aspect, the first encoding means comprises one of a BIFS encoding scheme of MPEG-4 encoding scheme and a non-natural acoustic encoding scheme. The description information for generating the synthesized speech information is encoded by a coding scheme, and the second encoding means encodes at least the synthesized speech information by a natural acoustic coding scheme of an MPEG-4 coding scheme. It is characterized by doing.

An eleventh invention is a decoding device for decoding input coded information, comprising: a notifying means for notifying a coding side, to which the coded information is input, of its own state. Decoding means for decoding the information encoded on the encoding side based on the notification by the notifying means.

In a twelfth aspect based on the eleventh aspect, the notifying means notifies information on a decoding function of the decoding means.

In a thirteenth aspect based on the twelfth aspect, the information of the decoding function has a function of decoding information in which description information for generating computer graphics image information is encoded. And at least one of the information as to whether or not the image information has a function of decoding the encoded information.

In a fourteenth aspect based on the twelfth aspect, the information of the decoding function is information as to whether or not the information has a function of decoding information in which description information for generating synthesized speech information is encoded. , And / or information on whether or not the audio information has a function of decoding the encoded information.

According to a fifteenth aspect, in the eleventh aspect, the encoding side has the function of the encoding device according to any one of the first to tenth aspects.

A sixteenth aspect of the present invention is a decoding device for decoding input coded information, comprising: a device status obtaining means for obtaining the status of the own device; and a device status information obtained by the device status obtaining device. And decoding means for decoding coded information selected from a plurality of types of the coded information, which is a result of performing coding by a plurality of types of coding schemes for each information based on the information.

In a seventeenth aspect based on the sixteenth aspect, the apparatus status acquisition means acquires information on a decoding function of the decoding means.

In an eighteenth aspect based on the seventeenth aspect, the information of the decoding function has a function of decoding information in which description information for generating computer graphics image information is encoded. And at least one of the information as to whether or not the image information has a function of decoding the encoded information.

In a nineteenth aspect based on the seventeenth aspect, the information of the decoding function is information as to whether or not the information has a function of decoding information in which description information for generating synthesized speech information is encoded. , And / or information on whether or not the audio information has a function of decoding the encoded information.

According to a twentieth aspect, in the eleventh or sixteenth aspect, the encoded information includes information in which information is encoded by an MPEG-4 encoding method.

A twenty-first invention is an information processing system in which a plurality of devices are communicably connected, wherein at least one of the plurality of devices is one of the first to tenth aspects. 3. Functions of an encoding device, and claims 11 and 2.
0. It has at least one of the functions of the decoding device according to any one of the above.

A twenty-second invention is an information processing method for encoding arbitrary information on the encoding side and decoding the encoded information on the decoding side, wherein the decoding side acquires the decoding side state information. It is characterized by including a state information acquiring step and an encoding step of encoding information by an encoding method based on the decoding side state information obtained in the decoding side state information acquiring step.

In a twenty-third aspect based on the twenty-second aspect, the decoding-side state information obtaining step includes a step of obtaining information on a decoding function of the decoding side as the decoding-side state information. Is characterized by including a step of encoding information by an encoding method which can be supported by the decoding side, based on the decoding side state information.

In a twenty-fourth aspect based on the twenty-second aspect, in the decoding-side state information obtaining step, the decoding side decodes information obtained by coding description information for generating computer graphics image information. Obtaining the information as to whether or not the decoding side information as the decoding side state information, the encoding step, the decoding side by the decoding side state information, the decoding side, the description information for generating computer graphics image information When the computer graphics image information is not provided with a function of decoding the encoded information, the method includes a step of encoding the computer graphics image information by a natural image encoding method.

In a twenty-fifth aspect based on the twenty-second aspect, in the decoding-side state information obtaining step, the decoding side has a function of decoding information obtained by coding description information for generating synthesized speech information. Obtaining the information as to whether or not the information as the decoding side state information, wherein the encoding step is the information in which the decoding side encodes the description information for generating the synthesized speech information by the decoding side state information. If there is no function of decoding the synthesized speech information, the method includes a step of encoding the synthesized speech information by a natural acoustic encoding method.

A twenty-sixth invention is an information processing method for encoding arbitrary information on the encoding side and decoding the encoded information on the decoding side, wherein a plurality of types of encoding schemes are provided for each piece of information. An encoding step of generating a plurality of types of encoded information encoded in the above, a decoding-side state information obtaining step of obtaining the decoding-side state information, and a decoding-side state information obtained by the decoding-side state information obtaining step A decoding step of decoding encoded information selected from the plurality of types of encoded information obtained in the encoding step based on the above.

In a twenty-seventh aspect based on the twenty-sixth aspect, the decoding side state information acquiring step is characterized in that the decoding side decodes information obtained by coding description information for generating computer graphics image information. Acquiring the information as to whether or not the information has the decoding side state information, wherein the encoding step includes a first encoding step of encoding descriptive information for generating the computer graphics image information; A second encoding step of encoding the computer graphics image information by a natural image encoding method, wherein the decoding step includes the following steps: If there is no function of decoding the information obtained by encoding the description information for generation, the second code Characterized in that it comprises a step of decoding the obtained encoded information in step.

In a twenty-eighth aspect based on the twenty-sixth aspect, the decoding-side state information acquiring step has a function that the decoding side decodes information obtained by coding description information for generating synthetic speech information. Acquiring the information as to whether or not the information is the decoding-side state information, wherein the encoding step includes a first encoding step of encoding descriptive information for generating the synthesized speech information; A second encoding step of encoding information by a natural acoustic encoding method, wherein the decoding step encodes descriptive information for the decoding side to generate the synthesized speech information by the decoding side state information. If the apparatus does not have a function of decoding the encoded information, the method includes a step of decoding the encoded information obtained in the second encoding step.

According to a twenty-ninth aspect, in the twenty-second or twenty-sixth aspect, the encoding step includes a step of encoding information according to an MPEG-4 encoding scheme.

According to a thirtieth invention, an encoding apparatus according to any one of claims 1 to 10, a decoding apparatus according to any one of claims 11 to 20, or an information processing system according to claim 21 A computer-readable storage medium stores a processing program for implementing the means.

According to a thirty-first aspect of the present invention, a computer readable storage medium stores the processing steps of the information processing method according to any one of claims 22 to 29.

[0059]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) The present invention provides, for example,
It is applied to a data processing system 100 as shown in FIG. This data processing system 100 is a system that employs the MPEG-4 encoding method,
And two decoding devices 130 and 140 are connected so that data can be exchanged with each other. Less than,
Such a data processing system 100 will be specifically described.

[Configuration of Encoding Device 110] Encoding device 1
Reference numeral 10 denotes a scene setting unit 111 that generates scene description information (scene description language) such as a method of outputting a moving image and a sound of a natural image and an image command generated by CG, and a scene description generated by the scene setting unit 111. B to encode information
An IFS encoder 114, a rendering unit 113 that renders scene description information generated by the scene setting unit 111 to generate image data, a storage unit 115 that stores various image data such as videos, and a storage unit 115
Data or rendering unit 113 stored in
, A storage unit 118 storing various audio data, and an audio coding unit 119 coding the audio data stored in the storage unit 118. And BI
The FS encoder 114, the visual encoder 117, and a multiplexer 121 that multiplexes and outputs each encoded data obtained by the audio encoder 119.

Here, in addition to the above configuration, the encoding apparatus 110 includes a decoder state determination unit 120 for acquiring state information of the decoding apparatus to which the multiplexer 121 outputs, and
A selector 112 for switching an output destination of the scene description information obtained by the scene setting unit 111 by the rendering unit 113 and the BIFS encoding unit 114 based on the information obtained by the decoder state determination unit 120;
0 based on the information obtained at 0
Image data to be input to the rendering unit 113
And a selector 116 for switching between the image data in the storage unit 115 and the image data in the storage unit 115.

The scene setting unit 111 generates scene description information such as a method of outputting a moving image of a natural image or a sound, a command of an image generated by CG, etc. in accordance with an instruction from a user.

The decoder state determiner 120 outputs the state information of the decoding device (here, the decoding device 130 or the decoding device 140) that outputs the coded data, for example, information about the coding method of the coded data that can be decoded. , And obtains a control signal based on the obtained information to the selector 112 and the selector 116, respectively.

The selector 112 is a decoder state determiner 1
The rendering device 113 or the BIFS encoder 11 outputs the scene description information relating to the CG generated by the scene setting device 111 based on the control signal from the
Select 4.

The rendering unit 113 includes a selector 112
And renders the scene description information supplied via the. And generates image data.

The BIFS encoder 114 is connected to the selector 11
2 is encoded by BIFS.

The selector 116 is a decoder state determiner 1
Renderer 113 based on the control signal from
Or the image data stored in the storage unit 115 is selected, and the selected image data is input to the visual encoding unit 117.

The visual encoding unit 117 has the selector 1
The image data supplied via the MPU 16 is encoded according to a visual encoding method of the MPEG-4 encoding method suitable for the image data. Specifically, the visual encoding unit 117 performs video encoding for encoding video, FBA encoding as a non-natural image encoding method, mesh encoding, still image encoding for encoding still images, and the like. Encoding can be performed in various encoding systems, and the data is encoded using an encoding system suitable for the image data to be encoded.

[0070] The audio encoding unit 119 is provided in the storage unit 11.
8 is encoded according to an audio encoding system of the MPEG-4 encoding system suitable for the audio data. Specifically, the audio encoder 119 is used for audio (for example, CELP encoding), audio (for example, AAC encoding), or description for synthesizing a character code (for example, TTS encoding). ), Description for sound synthesis (eg, MID
It is possible to perform encoding using various encoding methods such as I method), and encodes the data using an encoding method suitable for audio data to be encoded.

The multiplexer 121 is provided for the BIFS encoder 11
The BIFS coded data obtained in step 4, the visual coded data obtained in the visual coding section 117, and the audio coded data obtained in the audio coding section 119 are converted into MPEG-4 coding schemes. Multiplex according to the multiplexing method.

With the above configuration, the encoding device 11
0 is, in particular, for a CG image, BIFS-encoded scene description information for generating a CG, and a visual code for the CG, based on the state (decoding capability) of the decoding device at the output destination of the encoded data. And one of video-encoded (natural image-encoded) data, which is one of the functions.

[Configuration of Decoding Device 130]
Has a decoding function corresponding to various encoding methods used in the encoding device 110.
132 for separating each encoded data from the multiplexed data from
A BIFS decoding unit 133 that decodes the FS encoded data and reproduces the scene description information, and a rendering unit 134 that performs rendering according to the CG command of the scene description information reproduced by the BIFS decoding unit 133 and generates image data. A visual decoding unit 135 that decodes the visual coded data obtained by the separation in the separator 132 to reproduce or generate image data, and a rendering unit 1
Image data and visual decoding unit 1 generated in step 34
A synthesizer 136 for synthesizing the image data reproduced or generated at 35, a display 137 for displaying the synthesized image data obtained at the synthesizer 136, and audio encoded data obtained by the separation at the separator 132 An audio decoding unit 138 for decoding the audio data and reproducing the audio data;
And a speaker 139 for outputting audio data reproduced by the audio decoding unit 138.

Further, the decoding apparatus 130 grasps the state of the apparatus itself (here, the state of the type of the decoder of the apparatus, etc.), and notifies the encoding apparatus 110 of the state by transmitting a decoder state transmitter 131. It has. Accordingly, the decoder status transmitter 131 knows that the device itself can decode BIFS encoded data, visual encoded data, and audio encoded data.

[Configuration of Decoding Device 140] Decoding Device 140
Has only a decoding function corresponding to video encoding and audio encoding among various encoding methods used in the encoding device 110, and encodes each encoded data from the multiplexed data from the encoding device 110. A separator 142 for separating;
A video decoder 143 that decodes the visual coded data obtained by the separation in the separator 142 (video coded data obtained by video coding which is a function of the visual coding) to reproduce image data; Video decoder 1
A display 144 for displaying the image data reproduced at 43, and audio encoded data obtained by the separation at the separator 142 (audio encoded data obtained by audio encoding which is a function of audio encoding) Decoder 145 that decodes audio data to reproduce audio data, and speaker 1 that outputs audio data reproduced by audio decoder 145
46.

Further, decoding apparatus 140 grasps the state of its own apparatus (here, the state such as the type of decoder of its own apparatus), and decoder state transmitter 141 for notifying coding apparatus 110 of the state. It has. Therefore, the decoder status transmitter 141 recognizes that it can decode only the video coded data and the voice coded data.

[Encoding device 110 outputs to decoding device 1
Operation in Case of 30] Here, an operation in the case where the data encoded by the encoding device 110 is reproduced by the decoding device 130 will be described.

First, a moving image to be encoded by the encoding device 110 is, for example, an image as shown in FIG. In FIG. 2, "150" represents an image of a room created by CG. “151” represents a cow image cut out from a moving image (video) obtained by shooting. The data of the cow image 151 is stored in the storage unit 115.
The data of the cry of the cow is stored in the storage unit 118. “152” represents a rabbit image created by CG.

When the image shown in FIG. 2 is encoded, the scene setting device 111 generates a command or the like for generating the CG of the cow and the rabbit with respect to the room image 150 and the rabbit image 152. Is created as scene description information, and the scene description information is encoded by the BIFS encoding unit 114 in the BIFS encoding method.

On the other hand, the cow image 151 is read from the storage unit 115, and is encoded by the visual encoding unit 117 in the video encoding system of the MPEG-4 encoding system. Also, the audio data of the cry is read from the storage unit 118 and is read from the audio encoding unit 11.
9 is encoded by the audio encoding method of the MPEG-4 encoding method.

Therefore, prior to the above-described encoding, encoding apparatus 110 operates decoder state determiner 120 to communicate with decoder state transmitter 131 of decoding apparatus 130.

As described above, the decoding device 130 has a decoding function (BIFS decoding unit 133, visual decoding unit 1) corresponding to various encoding methods used in the encoding device 110.
35 and the audio decoding unit 138), BIFS code, various coded data of visual,
It recognizes that various encoded data of audio can be decoded, and notifies the encoding device 110 of this fact as the state of the decoding device 130 by the decoder status transmitter 131.

In the encoding device 110, when the decoder status judgment unit 120 receives the status information from the decoding device 130, it instructs each processing unit to execute the encoding based on the information. That is, the decoder state determiner 12
A value of 0 gives the selector 112 a control signal that sets the output destination to the BIFS encoding unit 114, and gives the selector 116 a control signal that sets the input destination to the storage unit 115.

Thus, the scene description information of the CG commands of the room and rabbit images 150 and 152 created by the scene setting device 111 is transmitted to the BI
Input to FS encoding section 114. The BIFS encoding unit 114 BIFS-encodes the scene description information (CG commands of the room and rabbit images 150 and 152) from the scene setting unit 111. BIFS encoder 114
Encoded data (BIFS encoded data) obtained in
Is supplied to the multiplexer 121.

The data of the cow image 151 stored in the storage unit 115 is read out to the visual encoding unit 117 via the selector 116. The visual encoding unit 117 communicates with the storage unit 1 via the selector 116.
The data of the cow image 151 read from 15 is encoded by an encoding method (video encoding method) suitable for the data. The encoded data (video encoded data) obtained by the visual encoding unit 117 is supplied to the multiplexer 121.

The audio data of the cow cry stored in the storage unit 118 is output from the audio encoding unit 11.
9 is read. Audio encoding section 119
Encodes the cow crying audio data read from the storage unit 118 using an encoding method (speech encoding method) suitable for the data. The encoded data (audio encoded data) obtained by the audio encoding unit 119 is supplied to the multiplexer 121.

As described above, regarding the CG commands of the room image 150 and the rabbit image 152, the decoding device 1
30 is a BIFS capable of decoding BIFS encoded data
Since it has the decoding unit 133, it is BIFS-encoded,
For the cow image 151, the decoding device 130 includes the visual decoding unit 135 capable of decoding the video encoded data. Therefore, for the audio data of the cow cry which is video-encoded, the decoding device 130 Since it has the audio decoding unit 138 that can decode the encoded data, the audio is encoded.

The multiplexer 121 multiplexes the BIFS coded data from the BIFS coder 114, the video coded data from the visual coder 117, and the audio coded data from the audio coder 119. Then, the multiplexed data is output to the decoding device 130.

FIG. 3A shows multiplexed data obtained by the multiplexer 121. The multiplexed data here includes encoded data units 161 to 164, as shown in FIG.
m BIFS code) converts the room image 150 into BI
This is a data section in which BIFS encoded data resulting from the FS encoding is stored, and is encoded data section 162 (Rabbi).
t BIFS code) converts the image 152 of the rabbit to B
This is a data section in which BIFS encoded data resulting from the IFS encoding is stored, and is encoded data section 163 (Catt).
The “le Video code” is a data portion in which video encoded data obtained by video encoding the cow image 151 is stored, and the encoded data portion 164 (Audio) is stored.
code) is a data section in which voice-encoded data as a result of voice-encoding a cow cry is stored. Such multiplexed data is input to the decoding device 130.

In the decoding device 130, the demultiplexer 132
Converts the multiplexed data as shown in FIG. 3A from the encoding device 110 into BIFS encoded data (161,
162), separates the video encoded data (163) and the audio encoded data (164), supplies the BIFS encoded data to the BIFS decoding unit 133, and supplies the video encoded data to the visual decoding unit 135; The audio encoded data is supplied to the audio decoding unit 138.

The BIFS decoding unit 133 decodes the BIFS encoded data (161, 162) from the separator 132, reproduces the CG commands of the room image 150 and the rabbit image 152, and renders the commands. 13
4

[0092] The rendering unit 134 interprets the command data from the BIFS decoding unit 133 and outputs the image 1 of the room.
The image 152 of the rabbit and the image 152 of the rabbit are respectively drawn, and each image data obtained as a result of the drawing is supplied to the synthesizer 136.

The visual decoding unit 135 includes a
, And reproduces the cow image 151, and supplies the image data to the synthesizer 136.

The synthesizer 136 combines the image data from the rendering unit 134 and the visual decoding unit 135, that is, the room image 150, the rabbit image 152, and the cow image 151, and displays the synthesized image data on the display 137. Supply. The display 137 includes the synthesizer 13
6 is displayed. Therefore, the image as shown in FIG. 2 is displayed on the display 137.

On the other hand, the audio decoding unit 138 decodes the encoded audio data (164) from the separator 132, reproduces the audio data of the cow voice, and outputs it through the speaker 139. Therefore, on the display screen of the display 137, the cry of the cow image 151 is heard by the speaker 1.
It is output from 39.

[Encoding device 110 outputs to decoding device 1
Operation in Case of 40] Here, the operation in the case where the data encoded by encoding apparatus 110 is reproduced by decoding apparatus 140 will be described. It should be noted that the moving image to be encoded by the encoding device 110 is an image as shown in FIG.

First, as in the case of the decoding apparatus 130 described above, the coding apparatus 110 operates the decoder state determination unit 120 and performs communication with the decoder state transmitter 141 of the decoding apparatus 140 prior to encoding.

The decoding device 140 is the same as the decoding device 13 described above.
Unlike 0, the video decoder 143 does not have a function of decoding BIFS coded data, and can decode only coded data (video coded data) obtained by video coding which is a function of visual coding. , And the audio decoder 145 that can decode only the audio encoded data, it is understood that only the video encoded data and the audio encoded data can be decoded.
The decoder status transmitter 141 notifies the encoding device 110 of the status of 40.

In the encoding device 110, when the decoder status judgment unit 120 receives the status information from the decoding device 140, it instructs each processing unit to execute the encoding based on the information. That is, the decoder state determiner 12
0 gives a control signal to the selector 112 to set the output destination to the rendering unit 113. In addition, the decoder state determination unit 120 sets the input destination to the rendering unit 113 when encoding the CG image of the room image 150 and the rabbit image 152 with respect to the selector 116, and outputs the cow image 151.
At the time of encoding, a control signal with the input destination being the storage unit 115 is provided.

Thus, the scene description information of the CG commands of the room and rabbit images 150 and 152 created by the scene setting device 111 is input to the rendering device 113 via the selector 112. The rendering unit 113 interprets the CG commands of the room and rabbit images 150 and 152 supplied via the selector 112 and renders the room and rabbit images 150 and 152.
Images 150 and 152 of the depicted room and rabbit
Is supplied to the visual encoding unit 11 via the selector 116.
7. The visual encoding unit 117 outputs the image 15 of the room and the rabbit supplied through the selector 116.
0, 152 is video encoded, which is a function of visual encoding. The encoded data (video encoded data) obtained by the visual encoding unit 117 is
Supplied to

The data of the cow image 151 stored in the storage unit 115 is read out to the visual encoding unit 117 via the selector 116. The visual encoding unit 117 communicates with the storage unit 1 via the selector 116.
The data of the cow image 151 read from 15 is video-encoded, which is a function of visual encoding. The encoded data (video encoded data) obtained by the visual encoding unit 117 is supplied to the multiplexer 121.

The audio data of the cow cry stored in the storage unit 118 is
9 is read. Audio encoding section 119
Encodes the audio data of the cow cry read from the storage unit 118 by voice encoding. Audio encoding unit 1
Encoded data (audio encoded data) obtained in 19
Is supplied to the multiplexer 121.

As described above, regarding the CG commands of the room image 150 and the rabbit image 152, the decoding device 1
40 does not have a BIFS decoding function and the decoding device 14
0 is video-encoded according to the video decoder 143 of the decoding device 14 for the cow image 151.
0 is provided with the video decoder 143, so that the audio data of the cow cry is video-encoded, and the decoding device 140 is provided with the audio decoder 145 capable of decoding the encoded audio data. Be transformed into

The multiplexer 121 multiplexes the video coded data from the visual coder 117 and the voice coded data from the audio coder 119, and sends the multiplexed data to the decoding device 130. Output.

FIG. 3B shows the multiplexed data obtained by the multiplexer 121. As shown in FIG. 3B, the multiplexed data here includes coded data units 171 to 174, and the coded data unit 171 (R
oom Video code) is the image 150 of the room.
Is a data portion in which video coded data as a result of video coding is stored, and is a coded data portion 172 (Rabb
it Video code) is a rabbit image 152
Is a data portion in which video encoded data as a result of video encoding is stored, and is encoded data portion 173 (Catt).
The “le Videocode” is a data section in which video encoded data obtained by video encoding the cow image 151 is stored, and an encoded data section 174 (Audio c) is stored.
mode) is a data section in which voice-encoded data obtained by voice-encoding a cow's cry is stored. Such multiplexed data is input to the decoding device 140.

In the decoding device 140, the separator 142
Converts the multiplexed data as shown in FIG. 3B from the encoding device 110 into video encoded data (171 to 1).
73), and audio encoded data (174).
The encoded video data is supplied to a video decoder 143, and the encoded audio data is supplied to an audio decoder 145.

The video decoder 143 decodes the video coded data (171 to 173) from the separator 142, and outputs a room image 150, a rabbit image 152, and a cow image 1
51 and reproduces the image data on the display 14.
4 The display 144 displays the video decoder 1
The image from 43 is displayed. Therefore, an image as shown in FIG. 2 is displayed on the display 144.

On the other hand, speech decoder 145
, And reproduces the voice data of the cow's voice, and outputs it via the speaker 146. Therefore, on the display screen of the display 144, the cry of the cow image 151 is output from the speaker 146.

As described above, in the present embodiment, the encoding configuration in encoding apparatus 110 is configured to perform encoding based on information (decoding function) of a decoding apparatus to which encoded data is output. Therefore, the encoded data corresponding to the decoding function of the output device of the encoding device 110 is provided to the decoding device. Therefore, there is no special function such as CG rendering.
Regardless of the decoding configuration of the decoding side, C
An image created on the encoding side that can freely handle G or the like can be suitably reproduced without causing information defects.

In the present embodiment, the encoding system employed in the data processing system 100 is an encoding system that handles CG and video in an integrated manner.
Although the PEG-4 encoding method has been described, the present invention is not limited to this.

[0111] Also, the storage unit 115,
The configuration of 118 and the data stored in each of them are not limited to the above-mentioned data, for example, moving image data stored in one portable medium,
Of course, it may be coded data.

Further, it is also possible to change the components shown in FIG. 1 described in FIG. 1 by software and to perform processing by an arithmetic unit or the like.

The multiplexing method in the data processing system 100 is not limited to the above-described multiplexing method. For example, a time-division multiplexing method for each VOP may be used. .

Further, for example, the decoder status transmitter 131 of the decoding device 130 recognizes that the demultiplexer 132 can separate each data constituting the data from the multiplexed data of the MPEG-4 encoding system. Then, the decoder status transmitter 141 of the decoding device 140 determines that the demultiplexer 142 has the multiplexing scheme H. If it is understood that each data constituting the data can be separated from the multiplexed data of H.223, the encoding apparatus 1
In 10, the multiplexing method of the multiplexer 121 may be changed as appropriate in accordance with the decoding device at the output destination.

Further, the communication for grasping the state of the decoding device at the output destination of the encoding device 110 is not limited to the communication described above. For example, by exchanging profiles, the decoding device It is also possible to realize grasping the state of.

(Second Embodiment) The present invention provides, for example,
It is applied to a data processing system 200 as shown in FIG. This data processing system 200 is a system that employs the MPEG-4 encoding method, and includes an encoding device 210
And two decoding devices 230 and 240 are connected so that data can be exchanged with each other. Less than,
Such a data processing system 200 will be specifically described.

[Configuration of Encoding Device 210] Encoding Device 2
Reference numeral 10 denotes an F for setting animation of a face, a body, and the like.
A BA setting unit 211, a rendering unit 213 that renders animation description information on animation of a face, a body, and the like based on the settings in the FBA setting unit 211 to generate image data, and a setting unit based on the settings in the FBA setting unit 211 An FBA encoder 214 for encoding animation description information related to animation of a face, a body, and the like, and a storage unit 215 for storing various image data such as videos.
A video encoder 217 for encoding image data stored in the storage unit 215 or image data obtained by the rendering unit 213; a terminal device 220 for inputting audio text data; A speech synthesizer 222 for generating speech data from the text data input at 220, a CELP encoder 223 for encoding the speech data obtained by the speech synthesizer 222 by a CELP encoding method, and a speech synthesizer TTS encoder 224 for encoding the audio data obtained in 222 by the TTS encoding method
And a multiplexer that multiplexes the encoded data obtained by the FBA encoder 214 or the video encoder 217 and the encoded data obtained by the CELP encoder 223 or the TTS encoder 224 and outputs the multiplexed data. 219.

Here, in addition to the above configuration, encoding apparatus 210 further includes a decoder state determination unit 226 for acquiring state information of the decoding apparatus to which the multiplexer 219 outputs, and
Based on the information obtained by the decoder state determination unit 226, the output destination of the animation description information obtained by the FBA setting unit 211 is determined by the rendering unit 213 and the FBA encoder 2.
14 and the video encoder 2 based on the information obtained by the decoder state determiner 226.
Image data to be input to the rendering device 21
Selector 216 for selecting from the image data from storage unit 3 or the image data from storage unit 215;
The coded data of the image input to the multiplexer 219 based on the information obtained in
A terminal device 2 based on information obtained by a selector 218 for selecting from the encoded data from the decoder or the encoded data from the video encoder 217 and the information obtained by the decoder state determiner 226.
Selector 2 for switching between the speech synthesizer 222 and the TTS encoder 224 to the output destination of the text data obtained at 20.
21 and the speech encoded data input to the multiplexer 219 based on the information obtained by the decoder state decision unit 226, the encoded data from the CELP encoder 223 or the encoded data from the TTS encoder 224. A selector 225 for selecting from encoded data is further added.

The decoder state determiner 226 outputs the state information of the decoding device (here, the decoding device 230 or the decoding device 240) that outputs the coded data, for example, information about the coding method of the coded data that can be decoded. , And obtains a control signal based on the obtained information by communicating with the selector 212, 216, 218, 221, 225.
Respectively.

The FBA setting unit 211 sets animation of a face, a body, etc., for example, in accordance with an instruction from the user.
Animation description information is generated based on the configuration of the face and body and the movement of each part thereof (the definition of the face and body and FBA parameters representing the movement).

The selector 212 is a decoder state decision unit 2
As an output destination of the animation description information generated by the FBA setting unit 211 based on the control signal from the
The rendering unit 213 or the FBA encoder 214 is selected.

The rendering unit 213 includes a selector 212
Renders the animation description information supplied through the. And generates image data.

The FBA encoder 214 has a selector 212
The animation description information supplied through the
FB described in the visual part of the G-4 encoding method
Encoding is performed by the A encoding method.

The selector 216 is connected to the decoder state determiner 2
Rendering device 113 based on the control signal from
Or the image data stored in the storage unit 215 is selected, and the selected image data is input to the video encoder 217.

The video encoder 217 includes a selector 216
Is encoded in accordance with the video encoding method of the MPEG-4 encoding method.

The selector 218 is connected to the decoder state determiner 2
26, the FBA encoder 214
And the encoded data (video encoded data) obtained by the video encoder 217 are selected, and the selected encoded data is transmitted to the multiplexer. 219.

[0127] The terminal device 220 generates, as text data, a sentence for generating a voice in accordance with the animation of the face, body, or the like in accordance with an instruction from the user.

The selector 221 is connected to the decoder state determiner 2
The speech synthesizer 2 outputs the text data generated by the terminal device 220 based on the control signal from the
22 or the TTS encoder 224 is selected.

The voice synthesizer 222 generates voice data by performing voice synthesis processing on the text data supplied via the selector 216.

[0130] The CELP encoder 223 is a speech synthesizer 2.
The audio data obtained in step 22 is encoded by the CELP encoding method, which is a function of the audio encoding method of the MPEG-4 encoding method.

The TTS encoder 224 includes a selector 216
The text data supplied via the TTS is converted to a TTS which is a function of an audio encoding system of the MPEG-4 encoding system.
Encode using the encoding method.

The selector 225 is connected to the decoder state determiner 2
26, the CELP encoder 22
3. The coded data (CELP coded data) obtained in step 3 or the coded data (TTS coded data) obtained in the TTS encoder 224 is selected, and the selected coded data is multiplexed. Input to the box 219.

The multiplexer 219 outputs the encoded data (video encoded data) supplied via the selector 218,
The coded data (speech coded data) supplied via the selector 225 is multiplexed by the multiplexing method of the MPEG-4 coding method and output.

[Configuration of Decoding Device 230] Decoding Device 230
Is a separator 232 for separating each encoded data from the multiplexed data from the encoding device 210, and an FBA decoding for decoding the FBA encoded data obtained by the separation by the separator 232 and reproducing animation description information. Vessel 233 and F
A rendering unit 234 that generates image data by performing rendering in accordance with the animation description information and the like reproduced by the BA decoder 233, and decodes video encoded data obtained by separation by the separator 232 to generate image data. And a renderer 234 for playing back
, A display 237 for displaying the synthesized image data obtained by the synthesizer 236, and a separator 232 for synthesizing the image data generated by the image data and the image data reproduced by the video decoder 235. A TTS decoder 238 that decodes the TTS encoded data obtained by the separation and reproduces audio data, and a speaker 239 that outputs the audio data reproduced by the TTS decoder 238 are provided.

[0135] Furthermore, decoding apparatus 230 grasps the state of its own apparatus (here, the state of the type of its own decoder, etc.), and decoder state transmitter 231 for notifying encoding apparatus 210 of it. It has. Therefore, the decoder status transmitter 231 knows that it can decode the FBA encoded data, the video encoded data, and the TTS encoded data.

[Configuration of Decoding Device 240] Decoding Device 240
Is a separator 242 for separating each encoded data from the multiplexed data from the encoding device 210, and a video decoder for decoding the video encoded data obtained by the separation by the separator 242 and reproducing the image data. 243 and video decoder 2
A display 244 for displaying the image data reproduced at 43, and a CEL obtained by the separation at the separator 242.
The CELP decoder 245 includes a CELP decoder 245 that decodes P-coded data to reproduce audio data, and a speaker 246 that outputs audio data reproduced by the CELP decoder 245.

Further, decoding apparatus 240 grasps the state of its own apparatus (here, the state such as the type of the decoder of its own apparatus), and decoder state transmitter 241 for notifying coding apparatus 210 of the state. It has. Therefore, the decoder status transmitter 241 knows that it can decode only the video coded data and the CELP coded data.

[Encoding device 210 outputs to decoding device 2
Operation in Case of 30] Here, an operation in the case where the data encoded by the encoding device 210 is reproduced by the decoding device 230 will be described.

First, a moving image to be encoded by the encoding device 210 is, for example, an image as shown in FIG.

In FIG. 5, "250" represents a background moving image obtained by photographing. The data of the background image 250 is stored in the storage unit 215, and is stored in the storage unit 215.
It is encoded by a video encoding system of four encoding systems.

On the other hand, "251" represents a character image created by CG. In the character image 251, features of the face and body and their movements are parameterized, and these parameters (FBA parameters) are set by the FBA setting unit 211. Therefore, the FBA setting unit 211 generates animation description information based on the set parameters, and the animation description information is encoded by the FBA encoding method. Also, the character image 251
Is input by the terminal device 220. Therefore, the terminal device 220 generates text data based on the input voice, and the text data is encoded by the TTS encoding method.

Therefore, prior to the above-described encoding, the encoding apparatus 210 operates the decoder state determination decoder state determination unit 226 to communicate with the decoder state transmitter 231 of the decoding apparatus 230.

As described above, the decoding device 230
Since it is provided with the A decoder 233, the video decoder 235, and the TTS decoder 238, it is understood that each coded data can be decoded by the FBA coding method, the video coding method, and the TTS coding method. Decoding device 2
The state of 30 is notified to the encoding device 210 by the decoder state transmitter 231.

In the encoding apparatus 210, upon receiving the state information from the decoding apparatus 230, the decoder state determination unit 226 instructs each processing unit to execute encoding based on the information. That is, the decoder state determiner 22
6 supplies a control signal to the selector 212 to set the output destination to the FBA encoder 214, a control signal to the selector 216 to set the input destination to the storage unit 215, and outputs a control signal to the selector 221. A control signal for the TTS encoder 224 is provided to the destination, a control signal for the TTS encoder 224 is provided to the selector 225, and the background image 250 and the character image 2 are provided to the selector 218.
A control signal for appropriately switching the input destination between the video encoder 217 and the FBA encoder 214 according to the encoding of 51 is given.

Thus, the animation description information (FBA parameter information indicating the definition and movement of the face and body of the character image 251) created by the FBA setting unit 211 is transmitted to the FBA encoder 214 via the selector 212.
Is entered for The FBA encoder 214 performs FBA encoding on the animation description information supplied via the selector 212. The encoded data (FBA encoded data) obtained by the FBA encoder 214 is input to the selector 218.

The data of the background image 250 stored in the storage unit 215 is read out to the video encoder 217 via the selector 216. The video encoder 217 encodes the data of the background image 250 read from the storage unit 215 via the selector 216 by the video encoding method. The encoded data (video encoded data) obtained by the video encoder 217 is
8 is input.

The selector 218 is connected to the decoder state determiner 2
26, the FBA coded data from the FBA coder 214 and the video coded data from the video coder 217 are appropriately switched according to the control signal from the multiplexor 21.
9.

The text data obtained by the terminal device 220 is transmitted to the TTS encoder 224 via the selector 221.
Supplied to The TTS encoder 224 selects the selector 22
1 is coded by the TTS coding method. The encoded data (TTS encoded data) obtained by the TTS encoding unit 224 is
5 to a multiplexer 219.

As described above, with respect to the character image 251, since the decoding device 230 has the FBA decoder 233 capable of decoding FBA encoded data,
For the encoded and background image 250, the decoding device 23
0 is provided with a video decoder 235 capable of decoding video encoded data.
Is a TTS decoder 23 capable of decoding TTS encoded data
8 is provided, so that TTS encoding is performed.

The multiplexer 219 is connected to the selector 21
8 and the TTS encoded data from the selector 219
The multiplexed data is multiplexed according to the multiplexing method of the PEG-4 encoding method, and the multiplexed data is output to the decoding device 230.

FIG. 6 shows video coded data and FBA coded data in the multiplexed data obtained by the multiplexer 219. In FIG. 6, "26"
Each code indicated by 1 "to" 269 "represents the following information.

261: VOSSC (visual_object_seaq) representing the head of the visual sequence
ence_start code) 262: PLI (prof) indicating profile and level
ile_and_level_indication)
Reference numeral 263: VOSC indicating the head of the visual object
(Visual_object_start_cod
e) Code 264: ISVO (is_vis) indicating presence / absence of extension information
ual_object_identifier) code 265: VOTY indicating type of visual object
PE (visual_object_type) code,
VOTYPE = 000 if video encoded
1 266: VOL (Video Object Layer)
r) VOLSC (video_object) representing the beginning of
(t_layer_start_code) code This is followed by the code for the video. Here, the background image 2
Fifty video coded data follows. 267: VOTYPE code, VOTYPE = 0101 if FBA encoded, 268: FBAOPS representing the beginning of FBA encoded data
C (face_object_start_code)
The code following this is followed by the code for FBA. 269: VOSE indicating end of visual sequence
ND (visual_object_seaquens
e_end_code) code

Here, since the decoding device 230 can decode the coded data by the FBA coding, the FBA coded data is supplied to the decoding device 230. Therefore, in FIG. 6, the codes 267 and 268 are included. This shows a state where FBA encoded data is multiplexed. The format of the video coded data and the FBA coded data in the multiplexed data shown in FIG.
Since the details are described in O / IEC14496-2, the description is omitted here.

Accordingly, in the multiplexed data supplied to the decoding device 230, the video coded data and the FBA coded data according to the format shown in FIG. 6 are followed by the multiplexed data shown in FIGS. Similarly to the format shown in b), TTS encoded data, which is encoded data of audio, is multiplexed.

In the decoding device 230, the demultiplexer 232
Separates the data obtained by multiplexing the video coded data and the FBA coded data as shown in FIG. 6 from the coding device 210 and the TTS coded data into respective coded data, and FBA decoder 23 encodes the encoded data
3 and supplies the video encoded data to a video decoder 235.
And supplies the TTS encoded data to the TTS decoder 238.

The FBA decoder 233 decodes and reproduces the FBA coded data from the separator 232, that is, the FBA coded data after the FBA OPSC code 268 representing the head of the FBA coded data, and reproduces the animation description information ( The definition of the face and body of the character image 251 and the information of the FBA parameters representing the movement thereof) are rendered by the rendering unit 234.
To supply.

[0157] The rendering unit 234 is the FBA decoder 2
It interprets the animation description information from 33 and draws an image of the face or body of the character image 251, and supplies image data obtained as a result of the drawing to the synthesizer 236.

The video decoder 235 decodes the video coded data from the demultiplexer 232, that is, the video coded data after the VOLSC code 266 representing the head of the video coded data, reproduces the background image 250, and The data is supplied to the synthesizer 236.

The synthesizing unit 236 synthesizes the image data from the rendering unit 234 and the video decoder 235, that is, the character image 251 and the background image 250, and supplies the synthesized image data to the display 237. The display 237 displays the synthesized image from the synthesizer 236. Therefore, the display 237 shown in FIG.
Is displayed.

On the other hand, the TTS decoder 238
2 is decoded, the voice data of the character in the character image 251 is reproduced, and the data is output via the speaker 239. Therefore, on the display screen of the display 237, the voice of the character is
Output from the speaker 239.

[Output destination of encoding apparatus 210 is decoding apparatus 2
Operation in Case of 40] Here, an operation in the case where the data encoded by the encoding device 210 is reproduced by the decoding device 240 will be described. Note that the moving image to be encoded by the encoding device 210 is an image as shown in FIG. 5, as in the case of the decoding device 230 described above.

First, as in the case of the decoding device 230 described above, the coding device 210 operates the decoder state determination unit 226 and performs communication with the decoder state transmitter 241 of the decoding device 240 prior to encoding.

The decoding device 240 is the same as the decoding device 23 described above.
Unlike 0, it does not have a function of decoding FBA coded data, and can decode only coded data (video coded data) obtained by video coding, and a video decoder 243 obtained by CELP coding. CELP
Since a CELP decoder 245 capable of decoding only encoded data is provided, video encoded data and CELP
It understands that only the encoded data can be decoded, and this is set as the state of the decoding device 240 and the decoder state transmitter 241.
To the encoding device 210.

In the encoding device 210, upon receiving the status information from the decoding device 240, the decoder status decision unit 226 instructs each processing unit to execute the encoding based on the information. That is, the decoder state determiner 22
6 supplies a control signal to the selector 212 so that the output destination is the rendering unit 213, and inputs the input destination to the selector 216 by the rendering unit 213 and the storage unit 215 according to the encoding of the background image 250 and the character image 251. A control signal for appropriately switching is provided, a control signal for inputting the video encoder 217 to the selector 218, and an output destination for the audio synthesizer 22 to the selector 221.
2 to the selector 225 and a control signal to the CELP encoder 223 as an input destination.

Thus, the animation description information (FBA parameter information indicating the definition and movement of the face and body of the character image 251) created by the FBA setting unit 211 is transmitted to the rendering unit 213 via the selector 212.
Is entered for

The rendering unit 213 includes a selector 212
Interpretation of the animation description information supplied through the. And renders an image of the character image 251 such as a face and a body. Data of the image drawn by the rendering device 213 is input to the selector 216.

The data of the background image 250 stored in the storage unit 215 is read out to the selector 216.

The selector 216 is connected to the decoder state determiner 2
In accordance with the control signal from, image data from the rendering unit 213 and image data from the storage unit 215 are appropriately switched and supplied to the video encoder 217.

The video encoder 217 includes a selector 216
(The image data supplied by appropriately switching the background image 250 and the character image 251) supplied via the video encoding system. The encoded data (video encoded data) obtained by the video encoder 217 is supplied to the multiplexer 219 via the selector 218.

[0170] The text data obtained by the terminal device 220 is supplied to the speech synthesizer 222 via the selector 221. The voice synthesizer 222 performs a voice synthesis process on the text data from the terminal device 220,
Generates audio data of the text data and applies it to CE
This is supplied to the LP encoder 223. CELP encoder 22
3 converts the speech data from the speech synthesizer 222 into CELP
Encode using the encoding method. The encoded data (CELP encoded data) obtained by the CELP encoder 223 is supplied to the multiplexer 219 via the selector 225.

As described above, since the decoding device 240 cannot decode the FBA coded data with respect to the character image 251, the decoding device 240 can decode the video coded data with respect to the background image 250. Since the video decoder 243 includes a CELP decoder 245 capable of decoding CELP encoded data with respect to the audio data of the voice of the character that is video-encoded,
Encoded.

The multiplexer 219 selects the selector 21
8 and the CELP coded data from the selector 219 are multiplexed according to the multiplexing method of the MPEG-4 coding method, and the multiplexed data is output to the decoding device 240.

FIG. 7 shows video coded data in the multiplexed data obtained by the multiplexer 219. As shown in FIG. 7, in the multiplexed data here, following the video encoded data of the background image 250 by the codes 261 to 266 as shown in FIG. 6, the character image 251 by the codes 271 to 275 is obtained. Video encoded data is multiplexed. That is, following the VOSSC code 271 representing the head of video encoded data obtained as a result of video encoding the character image 251, an ISVO code 272, a VOTYPE code 273, and a VOLSC code 2 relating to the video encoded data of the character image 251.
74 follows. The VOTYPE code 273 in this case is
"0001".

Therefore, in the multiplexed data supplied to the decoding device 240, following the video encoded data of the background image 250 and the character image 251 according to the format shown in FIG. ) And CELP coded data, which are coded data of audio, are multiplexed in the same manner as the formats shown in FIGS.

In the decoding device 240, the demultiplexer 242
Separates the data obtained by multiplexing the video coded data as shown in FIG. 7 from the coding device 210 and the CELP coded data into respective codings, and converts the video coded data into video data. Supply to the decoder 243 and CELP
The encoded data is supplied to the CELP decoder 245.

The video decoder 243 outputs the video coded data from the demultiplexer 242, that is, the video coded data of the VOLSC code 265 or later representing the head of the video coded data of the background image 250 and the video coded data of the character image 251. It decodes the coded video data after the VOLSC code 274 representing the head of the data, reproduces the background image 250 and the character image 251 in synchronization, and supplies the reproduced image data to the display 244. The display 244 displays an image from the video decoder 243. Therefore, an image as shown in FIG. 5 is displayed on the display 244.

On the other hand, CELP decoder 245
The CPU decodes the CELP coded data from the character image, reproduces the voice data of the character in the character image 251, and outputs it through the speaker 246. Therefore, on the display screen of display 244, the voice of the character is output from speaker 246.

As described above, in the present embodiment, the encoding configuration in encoding apparatus 110 is configured to perform encoding based on information (decoding function) of a decoding apparatus to which encoded data is output. Therefore, even if the decoding device at the output destination of the encoding device 110 has a configuration in which an image or audio at a visual level cannot be restored depending on the category, encoded data corresponding to the configuration is stored in the decoding device. Will be given. Therefore, regardless of the decoding configuration of the decoding side, such as not having a special function such as rendering of FBA parameters, the encoding side capable of freely handling data or the like obtained by encoding an image using parameters or the like. Can be suitably reproduced without causing information defects.

In the present embodiment, the encoding system adopted in the data processing system 200 is an encoding system for integrally handling data and video obtained by encoding an image using parameters and the like. Although the MPEG-4 encoding method has been described, the present invention is not limited to this.

Further, the configuration of the storage unit 215 of the encoding device 210, stored data, and data to be subjected to TTS encoding or CELP encoding inputted from the terminal device 220 are limited to the above-described data. For example, the moving image data may be moving image data stored in one portable medium or encoded data.

Further, it is also possible to change each component shown in FIG. 4 described above with software, for example, to perform processing by an arithmetic unit or the like.

The multiplexing method in the data processing system 200 is not limited to the multiplexing method described above. For example, time division multiplexing of FBA coded data and video coded data in units of VOP is performed. Of course, it does not matter.

Further, for example, the decoder status transmitter 231 of the decoding device 230 recognizes that the demultiplexer 232 can demultiplex each data constituting the data from the multiplexed data of the MPEG-4 encoding method. Then, the decoder status transmitter 241 of the decoding device 240 determines that the demultiplexer 242 has the multiplexing scheme H.264 recommended by the ITU. If it is understood that each data constituting the data can be separated from the multiplexed data of H.223, the encoding device 2
In 10, the multiplexing method in the multiplexer 219 may be changed as appropriate in accordance with the decoding device at the output destination.

Further, the communication for grasping the state of the decoding device at the output destination of the encoding device 210 is not limited to the communication described above. For example, by exchanging profiles, the decoding device It is also possible to realize grasping the state of.

The same applies to the case where another visual coding method is used instead of the FBA coding method. For example, when mesh encoding is used, video encoding is performed similarly when the decoding device does not have a function of decoding information according to the mesh encoding method. Further, the audio encoding is not limited to this, but a MIDI sound source instead of the terminal 220, a MIDI encoder instead of the TTS encoder 224, an acoustic synthesizer instead of the speech synthesizer 222, CELP
A configuration such as an AAC encoder instead of the encoder is also possible.

(Third Embodiment) The present invention relates to, for example,
It is applied to a data processing system 300 as shown in FIG. The data processing system 300 is a system that employs the MPEG-4 encoding method, and includes an encoding device 310
And two decoding devices 330 and 340 are connected so as to be able to exchange data with each other.

Here, the data processing system 300 has the same configuration as the data processing system 100 shown in FIG. 1, but differs from the data processing system 100 mainly in the following configuration.

First, the data processing system 300 is configured to further include a storage device 350. The storage device 350 includes an encoding device 310 and decoding devices 330 and 340.
Is accessed from each of the. That is,
The encoding device 310 stores the generated various types of encoded data in the storage device 350, and the decoding devices 330 and 340 respectively select their own devices from the storage information of the storage device 350 according to the status (decoding function) information of the own device. And selects the coded data suitable for the state, and decodes it.

Therefore, encoding apparatus 310 has a data state determiner for acquiring the apparatus state (decoding function) from decoding apparatuses 330 and 340, and outputs the output of scene setting unit 111 according to the output of the data state determiner. There is no selector for selection, and the output of the scene setting unit 111 is provided to both the rendering unit 113 and the BIFS encoding unit 114. Accordingly, the encoding device 310 converts the CG command into a BIFG-encoded B
Along with the IFS encoded data, video encoded data obtained by video encoding an image obtained by drawing according to the command of the CG are obtained, and these encoded data are stored in the storage device 350.

The audio encoding section 119 encodes the audio data stored in the storage section 118 according to a plurality of encoding methods, and generates a plurality of types of encoded data. That is, the recorded sound or voice is MP
Each function of the EG-4 coding method, for example, is coded by the AAC coding method or the CELP coding method, and the synthesized sound is coded by the MIDI coding method or the TTS coding method. Data is encoded by the AAC encoding method or the CELP encoding method. Therefore, the storage device 350 also stores a plurality of types of encoded data of audio data, each of which has a different encoding method.

In the decoding apparatus 330, the decoder state determination unit 331 for acquiring the state of the apparatus itself notifies the obtained apparatus state to the demultiplexer 332 instead of notifying the encoding apparatus 310. It has been made like that. Similarly, in the decoding apparatus 340, the decoder state determination unit 341 that acquires the state of the own apparatus does not notify the acquired apparatus state to the encoding apparatus 310 but notifies the separator 342. It has been done. As a result, the separators 332 and 342 select and acquire the encoded data suitable for the own device state from the storage device 350 according to the own device state from the decoder state determiners 331 and 341.

The data processing system 3 as described above
The operation of 00 will be specifically described. In the data processing system 300 shown in FIG. 8, the parts operating in the same manner as the data processing system 100 shown in FIG. 1 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[Operation of Encoding Device 310] Here, a moving image to be encoded by the encoding device 310 is, for example,
Assume that the image is shown in FIG.

First, the scene description information of the CG command of the room and rabbit images 150 and 152 created by the scene setting unit 111 is input to the BIFS encoding unit 114 and the rendering unit 113, respectively.

The BIFS encoding unit 114 converts the scene description information (room and rabbit image 1) from the scene setting unit 111.
50, 152 CG commands) are BIFS-encoded. The coded data (BIFS coded data) obtained by the BIFS coding unit 114 is supplied to the multiplexer 321.

The rendering device 113 is a scene setting device 1
11 is rendered to generate image data. The image data obtained by the rendering unit 113 is input to the selector 116.

The data of the cow image 151 stored in the storage unit 115 is input to the selector 116.

The selector 116 has a rendering unit 113
And the image data from the storage unit 115 are appropriately selected and supplied to the visual encoding unit 117.

The visual encoding unit 117 has the selector 1
Image data from room 16 (room and rabbit images 150,
152, and the cow image 151) are encoded by an encoding method (video encoding method) suitable for the data. The encoded data (video encoded data) obtained by the visual encoding unit 117 is supplied to the multiplexer 321.

The audio data of the cry of the cow stored in the storage unit 118 is stored in the audio encoding unit 11.
9 is read. Audio encoding section 119
Encodes the audio data of the cow cry read from the storage unit 118 by, for example, each of the ACC encoding method and the CELP encoding method, and generates ACC encoded data and CELP encoded data of the audio data. I do. The audio encoded data (ACC encoded data, CELP encoded data) obtained by the audio encoding unit 119 is supplied to the multiplexer 321.

The multiplexer 321 is provided for the BIFS encoder 11
4, BIFS encoded data, visual encoding unit 1
17 and the audio encoded data from the audio encoding unit 119 are multiplexed.

FIG. 9 shows multiplexed data obtained by the multiplexer 321. The multiplexed data in this case is stored in the system header (Sys) as shown in FIG.
(Tem Header) 371 and various encoded data sections 372 to 378.

As will be described in detail later, the system header section 371 records the state of a node specified by ISO / IEC14496-1.

The encoded data section 372 (Room-BIF)
S codes) stores BIFS encoded data as a result of BIFS encoding the room image 150, and encodes the encoded data section 373 (Rabbit-BIFS code).
In s), BIFS encoded data obtained by BIFS encoding the rabbit image 152 is stored.

Encoded data section 374 (ACC Audio)
o code) stores ACC coded data obtained by coding the cow's bark according to the ACC coding method, and the coded data section 375 (CELP Audio c)
odes) stores CELP encoded data obtained as a result of encoding a cow's cry by the CELP encoding method.

The encoded data section 376 (Cutter-V)
In the “video-codes”, video encoded data obtained by video-encoding the cow image 151 is stored, and an encoded data section 377 (Room-Video code) is stored.
s) stores video-encoded data obtained by video-encoding an image in which the room image 150 is rendered, and an encoded data unit 378 (Rabbit-Video).
codes) stores video encoded data obtained by video encoding an image in which the rabbit image 152 is rendered.

FIG. 10 schematically shows a relationship when each encoded data of the system header 371 shown in FIG. 9 is set as a node. The node 38 shown in FIG.
1 to 392 are nodes indicating the following information.

Node 381: Represents the parent of all sequences. Node 382: represents the cry of the cow (151). Node 383: Represents a room image 150. Node 384: Represents a rabbit image 152. Node 385: Represents an image 151 of a cow. The nodes 382 to 385 are connected to the parent node 381, and are in a relationship of being automatically reproduced when the parent node 381 is reproduced.

Node 386: The cry of cow (151) is A
Points to CC encoded data. Node 387: Points to CELP encoded data of the cry of cow (151). These nodes 386 and 387 are cry nodes 38
2 and selectively reproduced according to the state of the decoding device.

Node 388: BIF image of room 150
Points to S-encoded data. Node 389: Indicates data obtained by video-coding the room image 150. These nodes 388 and 389 are the room nodes 383
, And are selectively reproduced according to the state of the decoding device.

Node 390: BI image of rabbit 152
Indicates FS encoded data. Node 391 indicates data obtained by video-encoding the rabbit image 152. These nodes 390 and 391 are the rabbit nodes 38
4 and selectively reproduced according to the state of the decoding device.

Node 392: Indicates data obtained by video-coding the cow image 151. This node 392 is connected to the cow node 385.

[0213] Data obtained by multiplexing each of the encoded data as described above is stored in the storage device 350.

[Operation of Decoding Apparatus 330] Here, a case where the encoded data obtained by encoding apparatus 310 as described above is decoded by decoding apparatus 330 will be described.

First, the decoder state determiner 331 has its own decoding function corresponding to the various encoding systems used in the encoding device 310, that is, the BIFS decoding unit 133, the visual decoding unit 135, and the audio decoding unit. Since it has the unit 138, the BIFS encoding method
It is understood that each encoded data by the video encoding system, the ACC encoding system, and the CELP encoding system can be decoded. Therefore, the decoder state determiner 331 determines to decode each of the coded data according to the BIFS coding method, the video coding method, and the ACC coding method, for example, in order to perform reproduction with high quality. This is notified to the separator 332.

[0216] The demultiplexer 332 is a decoder
In accordance with the notification from No. 1, the system header section 371 of the multiplexed data stored in the storage device 350 is interpreted,
For room image 150, BIFS coded data at node 388 (372), for rabbit image 152, BIFS coded data at node 390 (373), and for cow image 151, video coded data at node 392 (376). , The cow's call
CC encoded data (374), respectively,
The selected coded data is separated from the multiplexed data.

Therefore, BIFS decoding section 133 includes:
BIFS encoded data (372, 373) of the room and rabbit images 150 and 152 are supplied, video encoded data (376) of the cow image 151 is supplied to the visual decoder 135, and the audio decoder 138 is supplied to the audio decoder 138. Is
The ACC coded data (374) of the cow call is supplied.

The BIFS decoding section 133 decodes the BIFS coded data (372, 373) from the separator 132, reproduces the CG commands of the room image 150 and the rabbit image 152, and renders the commands to the rendering device. 13
4

[0219] The rendering unit 134 interprets the command data from the BIFS decoding unit 133 and outputs the image 1 of the room.
The image 152 of the rabbit and the image 152 of the rabbit are respectively drawn, and each image data obtained as a result of the drawing is supplied to the synthesizer 136.

[0220] The visual decoding unit 135 includes the demultiplexer 132.
Decodes the encoded video data (376), reproduces the cow image 151, and supplies the image data to the synthesizer 136.

The synthesizer 136 combines the image data from the rendering unit 134 and the visual decoding unit 135, that is, the room image 150, the rabbit image 152, and the cow image 151, and displays the synthesized image data on the display 137. Supply. The display 137 includes the synthesizer 13
6 is displayed. Therefore, the image as shown in FIG. 2 is displayed on the display 137.

On the other hand, audio decoding section 138 decodes ACC encoded data (374) from separator 132,
The voice data of the cow voice is reproduced and output through the speaker 139. Therefore, on the display screen of the display 137, the cry of the cow image 151 is output from the speaker 139.

[Operation of Decoding Device 340] Here, the case where the encoded data obtained by encoding device 310 as described above is decoded by decoding device 340 will be described.

First, the decoder state determiner 341 includes a video decoder 143 that enables its own device to decode only coded data (video coded data) obtained by video coding, which is a function of visual coding, and a CELP. Since the audio decoder 145 that can decode only the encoded data is provided, it is understood that only the video encoded data and the CELP encoded data can be decoded. Therefore, the decoder state determiner 341 uses the video encoding method and the CEL
It decides to decode each encoded data by the P encoding method, and notifies the separator 342 of the fact.

The separator 342 is connected to the decoder state determiner 34
In accordance with the notification from No. 1, the system header section 371 of the multiplexed data stored in the storage device 350 is interpreted,
Video encoded data at node 389 (377) for room image 150, video encoded data at node 391 (378) for rabbit image 152, and video encoded data at node 392 (376) for cow image 151 , About the cow's call, the CE of node 387
LP encoded data (375), respectively,
The selected coded data is separated from the multiplexed data.

Therefore, the video decoder 143 is supplied with the video encoded data (376 to 378) of the images 150 to 152 of the room, the rabbit and the cow, and the audio decoder 1
45, CELP coded data of the cry of the cow (37
5) is supplied.

The video decoder 143 decodes the video coded data (376 to 378) from the separator 142 and outputs the room image 150, the rabbit image 152, and the cow image 1
51 and reproduces the image data on the display 14.
4 The display 144 displays the video decoder 1
The image from 43 is displayed. Therefore, an image as shown in FIG. 2 is displayed on the display 144.

On the other hand, speech decoder 145
And decodes the CELP encoded data (375), reproduces the voice data of cow voice, and outputs it through the speaker 146. Therefore, on the display screen of the display 144, the cry of the cow image 151
6 is output.

In the present embodiment as described above, one data to be encoded is encoded by a plurality of types of encoding methods.
Since it is configured such that encoded data suitable for the state (decoding function) of the decoding device can be selected from these encoded data and can be decoded, it is possible to avoid a state where the decoding side cannot decode an image or audio. Further, even with a decoding device having no special configuration such as CG rendering, it is possible to suitably decode and reproduce information without causing loss of information.

In the present embodiment, the encoding system adopted in the data processing system 300 is an encoding system that handles CG and video in an integrated manner.
Although the PEG-4 encoding method has been described, the present invention is not limited to this.

[0231] The storage unit 115,
The configuration of 118 and the data stored in each of them are not limited to the above-mentioned data, for example, moving image data stored in one portable medium,
Of course, it may be coded data.

Further, it is also possible to change each component shown in FIG. 8 described above by software, and to perform processing by an arithmetic unit or the like.

The multiplexing method in the data processing system 300 is not limited to the multiplexing method described above. For example, a time-division multiplexing method in VOP units may be used. .

Also, the multiplexing method is not limited to this.
The multiplexing method H. H.223 multiplexing method may be used.

Further, an object of the present invention is to supply a storage medium storing program codes of software for realizing the functions of the host and the terminal according to the first to third embodiments to a system or an apparatus. It is needless to say that the present invention is also achieved when a computer (or CPU or MPU) of the system or the apparatus reads out and executes the program code stored in the storage medium. in this case,
The program code itself read from the storage medium realizes the function of each embodiment, and the storage medium storing the program code constitutes the present invention. As storage media for supplying the program code, ROM, floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R,
A magnetic tape, a nonvolatile memory card, or the like can be used. The functions of each embodiment are implemented by executing the program code read by the computer, and the OS or the like running on the computer performs actual processing based on the instructions of the program code. It goes without saying that a part or all of the above is performed and the function of each embodiment is realized by the processing. Further, after the program code read from the storage medium is written to a memory provided in an extension function board inserted into the computer or a function extension unit connected to the computer, the function extension is performed based on the instruction of the program code. C provided on board and function expansion unit
It goes without saying that a PU or the like performs part or all of the actual processing, and the processing realizes the functions of the embodiments.

[0236]

As described above, according to the present invention, the configuration on the encoding side performs encoding based on state information on the decoding side to which the encoded information is output (information on the decoding function of the decoding side, etc.). With such a configuration, the decoding side of the encoded information is provided with the encoded information suitable for the state of the decoding side. In addition, since the configuration of the decoding side is configured to acquire the encoding information based on the state information of the decoding side (information of the decoding function of the decoding side, etc.), the decoding side changes the state of the decoding side. The appropriate encoded information will be decoded.

This makes it possible to freely use CG, audio, and the like, regardless of the decoding configuration of the decoding side, such as rendering of a CG image (computer graphic image) or no special audio function. An image created on the encoding side, which can be handled by a computer, can be suitably reproduced without causing information defects.

That is, even if the terminal that can reproduce only the video is the decoding side, the operation of reproducing the image created on the encoding side, which is a device of the category handling CG images, can be performed easily and reliably. Also, the encoding information to be handled is M
If information obtained by an encoding method conforming to standardization such as PEG-4 is used, information can be shared between a wide range of devices. Further, each decoding side can perform decoding suitable for decoding capability from the same encoded information.

Thus, according to the present invention, information can be suitably encoded, decoded, and reproduced between the encoding side and the decoding side having different categories such as profiles.

[Brief description of the drawings]

FIG. 1 is a diagram for describing a configuration of a data processing system to which the present invention is applied in a first embodiment.

FIG. 2 is a diagram for explaining an example of a moving image to be encoded by an encoding device of the data processing system.

FIG. 3 is a diagram for explaining a configuration of the encoded data.

FIG. 4 is a diagram illustrating a configuration of a data processing system to which the present invention is applied in a second embodiment.

FIG. 5 is a diagram for explaining an example of a moving image to be encoded by an encoding device of the data processing system.

FIG. 6 is a diagram illustrating a configuration of the encoded data (data including FBA encoded data).

FIG. 7 is a diagram illustrating a configuration of the encoded data (data not including FBA encoded data).

FIG. 8 is a diagram illustrating a configuration of a data processing system to which the present invention is applied in a third embodiment.

FIG. 9 is a diagram illustrating a configuration of encoded data obtained by an encoding device of the data processing system.

FIG. 10 is a diagram for explaining a relationship (node relationship) between various types of encoded data included in the encoded data.

FIG. 11 is a block diagram showing a configuration of a conventional data processing system.

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 Data processing system 110 Encoding device 111 Scene setting unit 112, 116 Selector 113 Rendering unit 114 BIFS encoding unit 115, 118 Storage unit 117 Visual encoding unit 119 Audio encoding unit 120 Decoder state determination unit 121 Multiplexer 130 Decoding Device 131 Decoder state transmitter 132 Separator 133 BIFS decoder 134 Renderer 135 Visual decoder 136 Combiner 137 Display 138 Audio decoder 139 Speaker 140 Decoder 141 Decoder state transmitter 142 Separator 143 Video decoder 144 Display 145 Audio Decoder 146 Speaker

Claims (31)

[Claims] An encoding apparatus capable of encoding arbitrary information by using a plurality of types of encoding methods, wherein the decoding side acquires a state of a decoding side of encoded information which is a result of the encoding. Status acquisition means, and encoding means for encoding information based on the decoding side state information obtained by the decoding side state acquisition means, using an encoding scheme selected from the plurality of types of encoding schemes. An encoding device characterized by the above-mentioned. 2. The encoding unit includes: a plurality of encoding units corresponding to the plurality of types of encoding schemes; and information to be encoded based on the decoding-side state information. 2. An encoding apparatus according to claim 1, further comprising: information supply means for supplying the information to an encoding means selected from the following. 3. The method according to claim 1, wherein the plurality of types of encoding methods are MPEG.
2. The encoding apparatus according to claim 1, wherein the encoding apparatus includes an encoding scheme of a -4 encoding scheme. 4. An encoding apparatus according to claim 1, wherein said encoding means encodes descriptive information for generating computer graphics image information, and encodes at least said computer graphics image information by a natural image encoding method. And a second encoding unit for encoding the information by means selected from the first and second encoding units based on the decoding-side state information. An encoding device according to claim 1. 5. The encoding device according to claim 1, wherein the encoding unit encodes description information for generating synthesized speech information;
A second encoding unit that encodes the synthesized speech information at least using a natural acoustic encoding method, and includes information selected by the first and second encoding units based on the decoding side state information. 2. The encoding apparatus according to claim 1, wherein encoding is performed. 6. A first encoding means for encoding descriptive information for generating computer graphics image information, and a second encoding means for encoding said computer graphics image information in a natural image encoding system. Means; and output means for selecting and outputting one of the encoded information obtained by the first encoding means and the encoded information obtained by the second encoding means. An encoding device characterized by the above-mentioned. 7. A first encoding unit that encodes description information for generating synthetic speech information, a second encoding unit that encodes the synthetic speech information by a natural acoustic encoding method, Output means for selecting and outputting one of the coded information obtained by the first coding means and the coded information obtained by the second coding means. Encoding device. 8. A decoding-side state obtaining means for obtaining a decoding-side state which is an output destination of the output means, and a decoding-side state information obtained by the decoding-side state obtaining means. 8. The encoding apparatus according to claim 6, further comprising control means for controlling the selection output operation. 9. The method according to claim 1, wherein the first encoding means comprises MPEG-4.
The description information for generating the computer graphics image information is encoded by any one of a BIFS encoding method and a non-natural image encoding method, and the second encoding unit includes: 7. The encoding apparatus according to claim 4, wherein at least the computer graphics image information is encoded by a natural image encoding method of an MPEG-4 encoding method. 10. The MPEG encoding apparatus according to claim 1, wherein
The description information for generating the synthetic speech information is encoded by any one of a BIFS encoding method and a non-natural acoustic encoding method of a four-encoding method, and the second encoding means includes an MPEG encoding method. The encoding apparatus according to claim 5, wherein at least the synthesized speech information is encoded by a natural sound encoding scheme of a −4 encoding scheme. 11. A decoding device for decoding input coded information, comprising: a notifying unit for notifying a coding side, to which the coded information is input, of a state of the coding device; And decoding means for decoding the information encoded on the encoding side based on the notification from the decoding apparatus. 12. The information processing apparatus according to claim 1, wherein the notifying unit notifies information on a decoding function of the decoding unit.
2. The decoding device according to 1. 13. The information on the decoding function includes information as to whether or not the information has a function of decoding information in which description information for generating computer graphics image information is encoded.
The decoding apparatus according to claim 12, further comprising at least one of information indicating whether or not the image information has a function of decoding encoded information. 14. The information of the decoding function includes information as to whether or not it has a function of decoding information in which description information for generating synthetic speech information is encoded, and information in which the speech information is encoded. 13. The decoding device according to claim 12, comprising at least one of information on whether or not the device has a decoding function. 15. The decoding device according to claim 11, wherein the coding side has the function of the coding device according to any one of claims 1 to 10. 16. A decoding device for decoding input coded information, comprising: a device status obtaining unit for obtaining a status of the own device; and information based on the device status information obtained by the device status obtaining unit. Decoding means for decoding coded information selected from a plurality of types of the coded information, each of which is a result of coding with a plurality of types of coding schemes. 17. The decoding apparatus according to claim 16, wherein said apparatus state obtaining means obtains information on a decoding function of said decoding means. 18. The information of the decoding function includes information as to whether or not the information has a function of decoding information in which description information for generating computer graphics image information is encoded.
18. The decoding device according to claim 17, further comprising at least one of information indicating whether or not the image information has a function of decoding encoded information. 19. The information of the decoding function includes information as to whether or not it has a function of decoding information in which descriptive information for generating synthetic speech information is encoded, and information in which the speech information is encoded. 18. The decoding apparatus according to claim 17, wherein the decoding apparatus includes at least one of information indicating whether or not the apparatus has a decoding function. 20. The decoding device according to claim 11, wherein the encoded information includes information obtained by encoding information according to an MPEG-4 encoding method. 21. An information processing system in which a plurality of devices are communicably connected, wherein at least one of the plurality of devices is an encoding device according to any one of claims 1 to 10. An information processing system having at least one of a function and a function of the decoding device according to any one of claims 11 to 20. 22. An information processing method for encoding arbitrary information on the encoding side and decoding the encoded information on the decoding side, wherein the decoding-side state information acquiring step for acquiring the decoding-side state information is performed. And an encoding step of encoding information by an encoding method based on the decoding-side state information obtained in the decoding-side state information obtaining step. 23. The decoding side state information obtaining step,
Acquiring the information of the decoding function of the decoding side as the decoding side state information. The information processing method according to claim 22, comprising a step. 24. The decoding-side state information obtaining step,
The decoding side includes a step of acquiring information as to whether or not the decoding side has the function of decoding the information obtained by encoding the description information for generating the computer graphics image information as the decoding side state information. According to the decoding side state information, if the decoding side does not have a function of decoding information obtained by coding description information for generating computer graphics image information, the computer graphics image information is converted into a natural image coding scheme. 23. The information processing method according to claim 22, further comprising a step of performing encoding according to: 25. The decoding-side state information obtaining step,
The decoding side includes a step of acquiring, as the decoding side state information, information as to whether or not the decoding side has a function of decoding information obtained by encoding the description information for generating the synthesized speech information. When the decoding side does not have a function of decoding the information obtained by coding the description information for generating the synthesized speech information according to the decoding side state information, the decoding of the synthesized speech information by the natural acoustic coding method is performed. 23. The information processing method according to claim 22, comprising: 26. An information processing method for encoding arbitrary information on the encoding side and decoding the encoded information on the decoding side, wherein each information is encoded by a plurality of types of encoding systems. An encoding step of generating a plurality of types of encoded information; a decoding-side state information obtaining step of obtaining the decoding-side state information; and a decoding-side state information obtained by the decoding-side state information obtaining step. An information processing method comprising a decoding step of decoding encoded information selected from a plurality of types of encoded information obtained in the encoding step. 27. The decoding-side state information acquiring step,
The decoding side includes a step of acquiring, as the decoding side state information, information as to whether or not the decoding side has a function of decoding the information obtained by encoding the description information for generating the computer graphics image information. Encoding the description information for generating the computer graphics image information;
And a second encoding step of encoding the computer graphics image information according to a natural image encoding method. The decoding step includes the steps of: When the apparatus does not have a function of decoding information obtained by coding description information for generating graphics image information, the method includes a step of decoding the coding information obtained in the second coding step. The information processing method according to claim 26. 28. The decoding-side state information acquiring step,
The decoding side includes a step of obtaining, as the decoding side state information, information as to whether or not the decoding side has a function of decoding information obtained by coding the description information for generating the synthesized speech information. A first encoding step of encoding descriptive information for generating synthesized speech information, and a second encoding step of encoding the synthesized speech information by a natural acoustic encoding method, wherein the decoding step includes: According to the decoding side state information, if the decoding side does not have a function of decoding information obtained by coding the description information for generating the synthesized speech information, the coding obtained by the second coding step The information processing method according to claim 26, further comprising a step of decoding information. 29. The encoding method according to claim 29, wherein the encoding step comprises:
27. The information processing method according to claim 22, further comprising the step of encoding information by an encoding method. 30. An encoding device according to any one of claims 1 to 10, a decoding device according to any one of claims 11 to 20, or a means of the information processing system according to claim 21. A computer-readable storage medium for storing a processing program for the same. 31. A storage medium, wherein the processing steps of the information processing method according to claim 22 are stored in a computer-readable manner.