CN1534503A - Method, system and storage medium for realizing real-time video-audio session in network game - Google Patents

Abstract

A method for real-time video-audio conversation in network game by video-audio synchronization technology. The system is used for establishing a network transmission channel which does not pass through a network game server between network game clients to carry out real-time video and audio conversation. First, real-time video data is captured and compressed and encoded into video and audio data frames. Then, the video and audio data frames are packaged as network transmission packets and time stamps are added for video and audio synchronization. The network transmission packet is transmitted to other game clients through a network transmission channel. Finally, the game client end receiving the network transmission packet decodes the network transmission packet and synchronously plays the decoded video data and audio data according to the time volume label.

Description

Method, system and storage medium for realizing real-time audio-visual conversation in online games

(1) Technical field

The present invention relates to a method for audio-visual conversation, in particular to a method for realizing real-time audio-visual conversation in online games by means of audio-video synchronization technology.

(2) Background technology

The system architecture of a general online game is based on an online game server and multiple online game clients. The online game server is used to process and save the data transmitted by the game client, and the game client is used to display the game screen. And the interaction between users and online games. In other words, all the data of the online game clients are sent to the online game server for processing and computation. The real-time audio-visual meeting is to use the camera and microphone on the game client computer to transmit real-time audio-visual data through the network, so as to conduct talks with other online game clients.

The current system architecture of online games cannot support the function of simultaneous real-time video and audio conversations in online games. There are three main technical difficulties. First, when an online game uses a three-dimensional animation to monopolize the screen, it cannot display a real-time video image. The current video data processing technology cannot simultaneously display a real-time video image when the three-dimensional animation monopolizes the screen. Second, if the encoding, decoding and transmission of video and audio data are all carried out by the online game server, its load and bandwidth will not be able to bear the load, which relatively affects the execution speed of the online game. Third, if the real-time video image and the real-time audio cannot be synchronized, the effect of the real-time video conference will be affected.

In terms of audio-visual synchronization technology, if the audio-visual data is captured by the transmitting online game client, it is directly transmitted to the receiving online game client through the network, and the receiving online game client directly plays the received audio-visual data. When the network bandwidth is good and the data processing capacity is sufficient, the video and audio synchronization requirements can be fulfilled. However, if the transmitting online game client, the transmission network or the receiving online game client are slightly delayed, the audio and video will be out of sync, and the out of sync will continue to accumulate over time. In actual use, the network bandwidth and data processing capabilities of current online games cannot afford the demand for video and audio synchronization. Therefore, current online games do not support the function of simultaneous real-time audio and video conversations in online games.

(3) Contents of the invention

In view of this, the object of the present invention is to compress, encode and package the real-time video and audio data at the online game client by establishing a network transmission channel between the online game client and the online game client, and transmit it to other online games. client. The online game client that receives real-time audio-visual data decodes the audio-visual data, and cooperates with the module of "supporting real-time video streaming and three-dimensional animation at the same time", so that when the online game uses three-dimensional animation to monopolize the screen, it can also be played. Real-time video images can be displayed at the same time, and audio data can be played through headphones and speakers with a sound card.

"Support real-time video streaming and three-dimensional animation at the same time" module is to use real-time video data as a texture map and mix it into the game screen of online games to achieve the purpose of simultaneous display.

In order to achieve the above object, the present invention proposes a method for realizing real-time audio-visual conversations in online games using audio-video synchronization technology. Firstly, a network transmission channel is established between the network game clients. The network game clients execute the network game and connect to the network game server. The network transmission channel does not go through the network game server. The network transmission channel is established by the network game client, specifying or directly specifying the network addresses of the network game clients who want to conduct real-time audio-visual conversations according to the address book. The online game client sends a connection request to the online game client that wants to conduct a real-time audio-visual conversation, and then the designated online game client establishes a network transmission channel according to the connection request.

Then, the real-time video data and real-time audio data are retrieved at the online game client. Compress and encode real-time video data into a video data frame, compress and encode real-time audio data into an audio data frame, encapsulate the video data frame and audio data frame into a network transmission packet, and add a time stamp to the network transmission packet. The time stamp represents real time The synchronization relationship between the audio data and the real-time video data is the time information for generating the real-time audio data and the real-time video data. Then, the network transmission packet is transmitted to other online game clients through the network transmission channel.

When the bandwidth of the network transmission channel cannot support simultaneous transmission of real-time video data and real-time audio data, the priority of the real-time audio data is higher than that of the real-time video data, that is, the network transmission packet will give priority to the real-time audio data. The advantage of giving priority to audio data is that audio data can be compressed to a very small size, so even in the case of insufficient network bandwidth, the real-time performance of audio data can still be maintained, meeting the basic requirements of real-time audio-visual conversations.

Finally, the online game client receiving the network transmission packet will decode the network transmission packet into video data and audio data, synchronize the real-time audio data and real-time video data according to the time tag, and display the video data and play the audio data, which is realized in the online game The purpose of conducting real-time audio-visual conferences at the same time. Displaying video data is to use real-time video data as a texture map and mix it into the game screen of online games to achieve the purpose of simultaneous display.

The key point of audio-visual synchronization is how to maintain correct time information on the online game client playing audio-visual data. There will be no problem, because the system time of retrieving audiovisual data is the same as the system time of playing audiovisual data. When the online game client that captures the audio-visual data is different from the online game client that plays the audio-visual data, the system time of the online game client that plays the audio-visual data needs to be added with the aforementioned time stamp as the real-time audio data for playback and real-time display. The time of the video data.

In addition, when decoding the network transmission packet into video data and audio data, because the network transmission packet is given priority to the audio data, it may happen that the video data frame transmitted by the network transmission packet is less than the original video data frame , and when the network transmission packet is decoded, the data frame in the network transmission packet may also be discarded (drop) due to the reference of video data compression (IPB reference encoding). Therefore, the audio-video synchronization needs to display the real-time video data at intervals based on the aforementioned time stamp and the quantity of the real-time video data, so as to obtain the effect of audio-video synchronization. This synchronous action can be performed before or after decoding the video and audio data.

Secondly, the present invention proposes a system for realizing real-time audio-visual conversations in online games using audio-video synchronization technology, including an online game server and multiple online game clients. The network game server executes the main program of the network game. The online game client includes the online game client for real-time audio-visual conversation and the network transmission channel.

The online game client of the audio-visual conference transmitter includes a real-time data collector, a data encoder, a network transmission encapsulator and a network transmitter. The real-time data capture device is used for capturing real-time video data and real-time audio data on the online game client. The data encoder is used for compressing and encoding the real-time video data to generate a video data frame, and compressing and encoding the real-time audio data to generate an audio data frame. The network transmission wrapper encapsulates the video data frame and the audio data frame into a network transmission packet and adds a time tag to the network transmission packet. The time tag indicates the synchronization relationship between real-time audio data and real-time video data, which is to generate real-time audio data and real-time The time information of the video data. The network transmitter transmits the network transmission packet to the receiving network game client through the network transmission channel.

The network transmission channel is established by the network game client, specifying or directly specifying the network addresses of the network game clients who want to conduct real-time audio-visual conversations according to the address book. The online game client sends a connection request to the online game client that wants to conduct a real-time audio-visual conversation, and then the designated online game client establishes a network transmission channel according to the connection request. When the bandwidth of the network transmission channel cannot support simultaneous transmission of real-time video data and real-time audio data, the priority of the real-time audio data is higher than that of the real-time video data, that is, the network transmission packet will give priority to the real-time audio data.

The online game client end of the receiver of the audio-visual conversation includes a data decoder and an audio-visual player. The data decoder is used for decoding the network transmission packet into real-time video data and real-time audio data. The audio-visual player decodes the network transmission packet into video data and audio data, synchronizes the real-time audio data and real-time video data according to the time stamp, and displays the video data and plays the audio data. Displaying video data is to use real-time video data as a texture map and mix it into the game screen of the online game to achieve the purpose of simultaneous display of the screen. The aforementioned synchronization uses the system time of the online game client plus a time tag as the time for playing real-time audio data and displaying real-time video data. Alternatively, the real-time video data is displayed at intervals based on the time stamp and the quantity of the real-time video data, so as to obtain the effect of a real-time audio-visual conversation.

In addition, the present invention proposes a method for realizing real-time audio-visual conversation in an online game using audio-video synchronization technology, which is applicable to an online game client. The online game client is connected to the online game server to execute the online game and display the game screen on the monitor. Firstly, before the online game is executed, a network transmission channel is established to the external online game client, and then the online game is executed and connected to the game server, and the network transmission channel does not pass through the game server.

Then, use the real-time data extractor to capture real-time video data and real-time audio data. Compress and encode real-time video data and real-time audio data to generate network transmission packets, and add time tags to the network transmission packets. time information.

Then, the network transmission packet is transmitted through the network transmission channel. The network transmission channel is established by the network game client specifying or directly specifying the network addresses of the external network game clients according to the address book, and the address book may include the network addresses of all external network game clients. The online game client sends a connection request to a specified network game client, and then the specified network game client establishes a network transmission channel according to the connection request. When the bandwidth of the network transmission channel cannot support simultaneous transmission of real-time video data and real-time audio data, the priority of the real-time audio data is higher than that of the real-time video data, that is, the network transmission packet will give priority to the real-time audio data.

Finally, the online game client receives the network transmission packet through the network transmission channel. And decompress/decode the network transmission packet to generate real-time video data and real-time audio data. Finally, during the execution of the online game, the real-time audio data and the real-time video data are synchronized according to the time tag, and the real-time audio data is played and the real-time video data is displayed on the game screen. Displaying real-time video data is to use real-time video data as a texture map and mix it into the game screen of the online game to achieve the purpose of simultaneous display. The aforementioned synchronization uses the system time of the online game client plus a time tag as the time for playing real-time audio data and displaying real-time video data. Alternatively, the real-time video data is displayed at intervals based on the time stamp and the quantity of the real-time video data, so as to obtain the effect of a real-time audio-visual conversation.

In addition, the present invention proposes a storage medium for storing a computer program, the computer program is used to load into a computer system and make the computer system execute the above-mentioned real-time audio-visual conference in online games using audio-visual synchronization technology Methods.

In addition, the present invention proposes an online game computer system, which executes an online game and has a storage medium for storing a computer program, and the computer program is used for loading into the above-mentioned online game computer system and making the above-mentioned network The game computer system executes the above-mentioned method for realizing real-time audio-visual conversations in online games using audio-video synchronization technology.

(4) Description of drawings

FIG. 1 is a flowchart showing the implementation of the method disclosed in the present invention.

FIG. 2 is a functional block diagram showing the system disclosed in the present invention.

FIG. 3 is a detailed functional block diagram showing the system disclosed in the present invention.

FIG. 4 (comprising FIG. 4a and FIG. 4b ) is a schematic diagram showing an embodiment disclosed in the present invention.

(5) specific implementation

Please refer to FIG. 1 . FIG. 1 is a flowchart showing the implementation of the method disclosed in the present invention. First, before the online game is played, a network transmission channel is established between the online game clients (step S100). The network transmission channel can be established with any standard network protocol, such as Transmission Control Protocol/Internet Protocol (TCP/IP). This network transmission channel does not go through the network game server. The network transmission channel is established by the network game client, specifying or directly specifying the network addresses of the network game clients who want to conduct real-time audio-visual conversations according to the address book. The online game client sends a connection request to the online game client that wants to conduct a real-time audio-visual conversation, and then the designated online game client establishes a network transmission channel according to the connection request. Then, the online game program to be played can be executed and connected to the game server.

Then, execute the online game program and connect to the online game server (step S101). Retrieve real-time video data and real-time audio data from the online game client (step S102). Real-time video data can be imported by using a video capture device such as a camera, while real-time audio data can be imported by using an audio capture device such as a microphone.

Next, compress and encode the real-time video data to generate a video data frame, and compress and encode the real-time audio data to generate an audio data frame (step S104 ). This compression coding can use standard compression formats, such as H.263, MPEG1/2/4, G723, etc. Its function is to reduce the originally huge amount of video and audio data into a format suitable for network transmission after compression coding.

Because when transmitting real-time audio-visual data through the network, audio-visual smoothness and audio-visual synchronization are the most important considerations, so the method proposed by the present invention takes the audio data frame as priority, and encapsulates the video-information data frame and the audio-visual data frame as network transmission packets (step S106 ) to achieve smooth audio and video-video synchronization. And add a time tag in the network transmission packet, the time tag represents the synchronization relationship between the real-time audio data and the real-time video data, that is, the time information for generating the real-time audio data and the real-time video data (step S107).

That is, when the bandwidth of the network transmission channel cannot support simultaneous transmission of real-time video data and real-time audio data, the priority of the real-time audio data is higher than that of the real-time video data. In addition, a time stamp will be added to the network transmission packet as the basis for synchronizing real-time video data and real-time audio data.

Then, transmit the network transmission packet to other online game clients through the network transmission channel (step S108). Finally, the online game client receiving the network transmission packet will decode the network transmission packet into real-time video data and real-time audio data (step S110), and synchronize the real-time audio data and real-time video data according to the time tag (step S111), and play Real-time audio data and display real-time video data on the game screen (step S112). Displaying video data is to use real-time video data as a texture map and mix it into the game screen of online games to achieve the purpose of simultaneous display.

The synchronization in step S111 is to add a time stamp to the system time of the online game client playing audio and video, as the time for playing real-time audio data and displaying real-time video data. Alternatively, the real-time video data is displayed at intervals based on the time stamp and the number of real-time video data, that is, less video data is displayed in the same time interval, and the display time of each video frame is delayed to achieve the purpose of video and audio synchronization.

The synchronization in step S111 can also be performed before step S110. The difference is that if synchronization is performed first and then decoding is performed, video data may be discarded due to different decoders. If the decoding is performed first and then the synchronization is performed, the discarding of the video data generated by the decoder has already occurred, so the final quantity of the video data can be grasped.

Please refer to FIG. 2 , which is a functional block diagram of the system disclosed in the present invention. The present invention proposes a system for conducting real-time audio-visual conversations in online games, including an online game server 20 and a plurality of online game clients 24 , 26 , 28 , and 30 . The online game server 20 executes the main program of the online game. The online game clients 24, 26, 28, 30 include the online game clients 24, 26 and the network transmission channel 32 for real-time audio-visual conversation, and the online game clients 24, 26, 28, 30 are connected with the online game server 20 to form The online game server network 22 is used to carry out online games. The online game server network 22 is used to transmit the data of the online game clients 24, 26, 28, and 30 to the online game server 20 for computing and storage. Online game clients 24 and 26 for real-time audio-visual talks.

Please refer to FIG. 3 , which is a detailed functional block diagram of the system disclosed in the present invention. The online game client 300 of the audio-visual conference sender includes a real-time data capture unit 302 , a data encoder 304 , a network transmission encapsulator 306 and a network transmission unit 308 . The real-time data capture device 302 is used for capturing real-time video data and real-time audio data from the online game client. The real-time data capture device 302 can be a separate device for capturing real-time video data and real-time audio data respectively. It can also be designed as the same device to capture real-time video data and real-time audio data at the same time, and then perform subsequent data processing respectively. The real-time data capture device 302 may be a data capture device such as a camera and a microphone.

The data encoder 304 is used for compressing and encoding the real-time video data to generate a video data frame, and compressing and encoding the real-time audio data to generate an audio data frame. The data encoder 304 can be designed as a separate device for compressing and encoding real-time video data and real-time audio data respectively, or can be designed in the same device for compressing and encoding real-time video data and real-time audio data respectively.

The network transmission encapsulator 306 takes the audio data frame as the priority, encapsulates the video data frame and the audio data frame as a network transmission packet, and adds a time stamp in the network transmission packet. The time stamp represents the synchronization of real-time audio data and real-time video data The relationship is time information for generating real-time audio data and real-time video data. The network transmitter 308 transmits the network transmission packet to the online game client 320 of the receiver of the audio-visual conversation through the network transmission channel 310 . For example, the network transport encapsulator 306 , the network transporter 308 and the network transport channel 310 can all be written as modules in a computer-executable programming language to realize the above-mentioned functions.

The online game client 320 of the receiver of the audio-visual conversation includes a data decoder 322 and an audio-visual player 324 . The data decoder 322 is used for decoding the network transmission packet into real-time video data and real-time audio data. The data decoder 322 can be designed as a separate device to decode real-time video data and real-time audio data respectively, or can be designed in the same device to decode real-time video data and real-time audio data respectively. The aforementioned data encoder 304 and data decoder 322 can be written as modules in a computer-executable programming language, and executed in a computer to complete required encoding and decoding functions.

The video player 324 will decode the video data and audio data of the network transmission packets, synchronize the real-time audio data and real-time video data according to the time stamp, and display the video data and play the audio data. Displaying real-time video data is to use real-time video data as a texture map, mix it into the game screen of online games, and use a sound card to play real-time audio data through headphones and speakers to achieve the purpose of simultaneous display. The online game client for real-time audio-visual conversation, because the audio-visual conversation is a two-way conversation, has the functions of sending and receiving at the same time, that is, it has the components and functions included in 300 and 320 at the same time.

For example, please refer to FIG. 4 , which is a schematic diagram showing an embodiment disclosed in the present invention. A network transmission channel 50 is established between the online game clients, and the online game clients are connected to the online game server and execute the online game. When the online game is in progress, when the online game client wants to conduct a real-time audio-visual conversation, the camera 40 captures real-time video data at the online game client, and the microphone 42 captures real-time audio data at the online game client.

The data encoders 44 and 46 compress and encode the real-time video data into video data frames 440, 442, 444, 446, assuming there are P in total, and compress and encode the real-time audio data into audio data frames 460, 462, 464, assuming there are L in total. In this embodiment, the network transmission encapsulator takes the audio data frame as priority, and encapsulates the video data frame and the audio data frame into the network transmission packet 48 . That is to say, within a period of time, the audio frame 460 will be preferentially included in the packet 48, and the remaining bandwidth or time will be redistributed to the video data frame 440 to the video data frame 442, and the audio frame 460 and the video data frame 440 to 442 Encapsulated into a network transmission packet 48 , for example, only N are included in the packet 48 , and (P-N) are not included in the packet 48 . A time tag is added to the network transmission packet. The time tag represents the synchronization relationship between the real-time audio data and the real-time video data, which is the time information for generating the real-time audio data and the real-time video data.

The network transmission packet 48 is transmitted to another network game client through a network transmission channel 50 , such as the Internet. The data decoders 52 and 54 decode the network transmission packet 48 into real-time video data 520 to 522 and real-time audio data 540 . Since this packet is transmitted through the network, not all video data frames may be received, for example, only M (M is less than or equal to N), that is, the number of video data frames 440 to 442 will be greater than Or equal to the number of real-time video data 520 to real-time video data 522, etc., plus the video decompression method such as IPB mode, so some real-time video data will be discarded, for example, only K (K is less than or equal to M) will be is decompressed, but the real-time audio data 540 will be decompressed completely, so the predetermined effect of smooth audio and video-audio synchronization can be achieved.

Audio-visual players 56 and 58 display real-time video data with the module "supporting real-time video streaming and three-dimensional animation at the same time". on the game screen of the game. The audio-visual players 56 and 58 add a time stamp to the system time as the time for playing real-time audio data and displaying real-time video data. Alternatively, the audio-visual players 56 and 58 will display the real-time video data at intervals according to the time stamp and the quantity of the real-time video data, so as to achieve the effect of audio-video synchronization. The audio-video synchronization can be performed before or after the data decoders 52 and 54 . Audio-visual players 56, 58 play real-time audio data through earphones and loudspeakers 58 with sound cards, so as to achieve the function of simultaneously carrying out audio-visual talks in online games.

In a word, the method proposed by the present invention can establish a network transmission channel between online game clients independently of the network game structure, so as to realize the purpose of real-time audio-visual conversation in the network game. The present invention also adopts an audio-visual synchronous encapsulation mechanism with priority given to audio data and a technique of synchronizing audio-visual data with time tags to obtain the most important effects of smooth audio and audio-visual synchronization in real-time audio-visual conversations. Moreover, the method proposed by the present invention can solve the unsolvable problems of the current technology without increasing the extra burden of the online game server, and only increases the real-time performance and sense of reality of the virtual game world to achieve the purpose of the present invention.

Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make various equivalent changes or replacements without departing from the spirit and scope of the present invention. Therefore The protection scope of the present invention shall prevail as defined by the appended claims.

Claims (33)

1. one kind is implemented in the method that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, comprises the following steps:

Set up a Network Transmission channel between one first network game client and one second network game client, described Network Transmission channel is not through a network game server;

Carry out an online game parallel machine to described network game server in described first network game client and described second network game client;

In described online game is carried out, capture a plurality of first real-time video signal data and a plurality of first real-time message data in described first network game client;

In described first network game client, the described first real-time video signal data of compressed/encoded produce a plurality of first video signal data frameworks, and the compressed/encoded described first real-time message data produce a plurality of first message data frameworks;

In described first network game client, encapsulate the described first video signal data framework and the described first message data framework is a Network Transmission package, and adding a time label in described Network Transmission package, described time tag is in order to represent the described first real-time message data and described first real-time video signal data synchronization relation;

By described Network Transmission channel, described Network Transmission package is sent to described second network game client;

In described second network game client, described Network Transmission package decoding is produced a plurality of second real-time video signal data and a plurality of second real-time message data; And

In described second network game client, according to synchronous described second real-time message data of described time tag and the described second real-time video signal data, and play the described second real-time message data and show that the described second real-time video signal data are on the game picture of described online game.

2. as claimed in claim 1ly be implemented in the method that real-time audio-video talks are carried out in online game, it is characterized in that the described step of setting up described Network Transmission channel still comprises the following steps: with audio-visual simultaneous techniques

Described first network game client specifies or directly specifies the network address of described second network game client according to an address book, described address book comprises the network address of described second network game client;

Described first network game client sends a redirect request to described second network game client; And

Described second network game client is set up described Network Transmission channel according to described redirect request.

3. as claimed in claim 1ly be implemented in the method that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, in the step of the described Network Transmission package of described encapsulation, when the frequency range of described Network Transmission channel can't support to transmit simultaneously the described first real-time message data and the described first real-time video signal data, the right of priority of the described first real-time message data was higher than the right of priority of the described first real-time video signal data.

4. as claimed in claim 1ly be implemented in the method that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, in the step of the described Network Transmission package of described generation, described time tag is for producing the temporal information of the described first real-time message data and the described first real-time video signal data.

5. as claimed in claim 1ly be implemented in the method that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, in the step of described described second real-time message data synchronously and the described second real-time video signal data, described is that system time with described second network game client adds the above time tag with as the broadcast described second real-time message data and time of showing the described second real-time video signal data synchronously.

6. as claimed in claim 1ly be implemented in the method that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, in the step of described described second real-time message data synchronously and the described second real-time video signal data, described is quantity with described time tag and the described second real-time video signal data synchronously, the described second real-time video signal data is carried out the interval show.

7. as claimed in claim 1ly be implemented in the method that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, in the step of the described second real-time video signal data of described demonstration, be with the described second real-time video signal data as a texturing, sneak on the game picture of described online game.

8. one kind is implemented in the system that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, comprising:

One network game server is in order to carry out an online game; And

A plurality of network game clients, described network game client comprises one first network game client, one second online game client and a Network Transmission channel, described first network game client and described second network game client are online to described network game server, in order to carry out described online game, described Network Transmission channel is coupled to described first network game client and described second network game client.

9. as claimed in claim 8ly be implemented in the system that real-time audio-video talks are carried out in online game, it is characterized in that described first network game client also comprises with audio-visual simultaneous techniques:

One real time data acquisition device is used to described first network game client and captures a plurality of first real-time video signal data and a plurality of first real-time message data;

One data encoder, it is coupled to described real time data acquisition device, in order to the described first real-time video signal data of compressed/encoded, produces a plurality of first video signal data frameworks and the compressed/encoded described first real-time message data, produces a plurality of first message data frameworks;

One Network Transmission wrapper, it is coupled to described video signal data scrambler and described message data encoder, in order to encapsulate the described first video signal data framework and the described first message data framework is a Network Transmission package, and adding a time label in described Network Transmission package, described time tag is in order to represent the described first real-time message data and described first real-time video signal data synchronization relation; And

One Network Transmission device, it is coupled to described Network Transmission wrapper, by described Network Transmission channel, described Network Transmission package is sent to described second network game client.

10. as claimed in claim 9ly be implemented in the system that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, described Network Transmission wrapper, when the frequency range of described Network Transmission channel can't support to transmit simultaneously the described first real-time message data and the described first real-time video signal data, the right of priority of the described first real-time message data was higher than the right of priority of the described first real-time video signal data.

11. as claimed in claim 9ly be implemented in the system that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, in the described Network Transmission wrapper, described time tag is for producing the temporal information of the described first real-time message data and the described first real-time video signal data.

12. as claimed in claim 8ly be implemented in the system that real-time audio-video talks are carried out in online game, it is characterized in that described second network game client also comprises with audio-visual simultaneous techniques:

One data decoder, it is coupled to described Network Transmission channel, with producing a plurality of second message data so that the decoding of described Network Transmission package produces a plurality of second video signal datas and described Network Transmission package is deciphered; And

One audio and video player, it is coupled to described video signal data code translator and described message data decoder, in order to according to described time tag, the described second real-time message data and the described second real-time video signal data synchronously, and play described second video signal data and the described second message data.

13. as claimed in claim 12ly be implemented in the system that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, in the described audio and video player, described is that system time with described second network game client adds the above time tag with as the time of playing the described second real-time message data and showing the described second real-time video signal data synchronously.

14. as claimed in claim 12ly be implemented in the system that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, in the described audio and video player, described is quantity with described time tag and the described second real-time video signal data synchronously, makes the described second real-time video signal data carry out the interval and shows.

15. as claimed in claim 12ly be implemented in the system that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, described audio and video player, be with the described second real-time video signal data as a texturing, sneak on the game picture of described online game.

16. as claimed in claim 8ly be implemented in the system that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, described Network Transmission channel is the network address of being specified or directly specifying described second network game client by described first network game client according to an address book, described address book comprises the network address of described second network game client, and send a connection request to described second network game client, set up described Network Transmission channel by described second network game client according to described connection request again.

17. one kind is implemented in online game with audio-visual simultaneous techniques and carries out the method that real-time audio-video is talked, it is applicable to a network game client, described network game client is to be online to a network game server, in order to carry out an online game and to show on game picture to a display, it is characterized in that, comprise the following steps:

Set up Network Transmission channel to an external network game client, described Network Transmission channel is without described network game server;

Utilize a real time data acquisition device, capture one first real-time video signal data and one first real-time message data;

The described first real-time video signal data of compressed/encoded and the described first real-time message data, produce one first Network Transmission package, and adding a time label in described Network Transmission package, described time tag is in order to represent the described first real-time message data and described first real-time video signal data synchronization relation;

By described Network Transmission channel, transmit the described first Network Transmission package;

By described Network Transmission channel, receive one second Network Transmission package;

The described second Network Transmission package of decompression/decoding produces one second real-time video signal data and one second real-time message data; And

In described online game is carried out, according to described time tag, synchronous described second real-time message data and the described second real-time video signal data, and play the described second real-time message data and show that the described second real-time video signal data are in described game picture.

18. as claimed in claim 17ly be implemented in the method that real-time audio-video talks are carried out in online game, it is characterized in that the described step of setting up described Network Transmission channel comprises the following steps: with audio-visual simultaneous techniques

Specify another network address by one of described network game client and described external network game client;

Utilize the described network address, send a redirect request to the other end by one of described network game client and described external network game client; And

According to described redirect request, the other end of described network game client and described external network game client is set up described Network Transmission channel.

19. as claimed in claim 17ly be implemented in the method that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, produce in the step of the described first Network Transmission package, when the frequency range of described Network Transmission channel can't support to transmit simultaneously the described first real-time message data and the described first real-time video signal data, the right of priority of the described first real-time message data was higher than the right of priority of the described second real-time video signal data.

20. as claimed in claim 17ly be implemented in the method that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, setting up the step of described Network Transmission channel, is to carry out according to an address book, and described address book comprises the network address of described external network game client.

21. as claimed in claim 17ly be implemented in the method that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, in the step of described described second real-time message data synchronously and the described second real-time video signal data, described is that system time with described network game client adds the above time tag with as the broadcast described second real-time message data and time of showing the described second real-time video signal data synchronously.

22. as claimed in claim 17ly be implemented in the method that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, in the step of described described second real-time message data synchronously and the described second real-time video signal data, described is quantity with described time tag and the described second real-time video signal data synchronously, makes the described second real-time video signal data carry out the interval and shows.

23. as claimed in claim 17ly be implemented in the method that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, show in the step of the described second real-time video signal data, be with the described second real-time video signal data as a texturing, sneak into described game picture and be shown on the described display.

24. Storage Media, in order to store a computer program, described computer program is in order to be loaded in the computer system and to make described computer system carry out as each described method of carrying out the real-time audio-video talks in online game in the claim 17 to 23.

25. online game computer system, it is carried out an online game and has a Storage Media, described Storage Media is in order to store a computer program, and described computer program is in order to be loaded in the described online game computer system and to make described online game computer system carry out as each described method of carrying out the real-time audio-video talks in online game in the claim 17 to 23.

26. one kind is implemented in online game with audio-visual simultaneous techniques and carries out the method that real-time audio-video is talked, be used between one first network game client and one second network game client, described first network game client and described second network game client are all carried out online game parallel machine to a network game server, it is characterized in that described method comprises the following steps: at least

Set up a Network Transmission channel between described first network game client and described second network game client, and described Network Transmission channel does not pass through described network game server;

Capture a plurality of first real-time video signal data and a plurality of first real-time message data in described first network game client;

Produce a plurality of first video signal data frameworks and a plurality of first message data framework;

Encapsulate the described first video signal data framework and the described first message data framework is a Network Transmission package, and adding a time label in described Network Transmission package, described time tag is in order to represent the described first real-time message data and described first real-time video signal data synchronization relation;

Transmit described Network Transmission package to described second network game client;

Described Network Transmission package decoding is produced a plurality of second real-time video signal data and a plurality of second real-time message data; And

In described second network game client, according to described time tag, synchronous described second real-time message data and the described second real-time video signal data, and play the described second real-time message data and show that the described second real-time video signal data are on the game picture of described online game.

27. as claimed in claim 26ly be implemented in the method that real-time audio-video talks are carried out in online game, it is characterized in that the described step of setting up described Network Transmission channel also comprises the following steps: with audio-visual simultaneous techniques

Described first network game client is specified the network address of described second network game client according to an address book, and described address book comprises the network address of described second network game client;

Described first network game client sends a redirect request to described second network game client; And

Described second network game client is set up described Network Transmission channel according to described redirect request.

28. as claimed in claim 26ly be implemented in the method that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, in the step of the described Network Transmission package of described encapsulation, be earlier the described first real-time message data to be included in the described Network Transmission package, according to the residue frequency range the described first real-time video signal data included in the described Network Transmission package again.

29. as claimed in claim 26ly be implemented in the method that real-time audio-video talks are carried out in online game, it is characterized in that the described first video signal data framework and a plurality of first message data framework are produced by compressed/encoded with audio-visual simultaneous techniques.

30. as claimed in claim 26ly be implemented in the method that real-time audio-video talks are carried out in online game, it is characterized in that described Network Transmission package is to be sent to described second network game client by a Network Transmission channel with audio-visual simultaneous techniques.

31. as claimed in claim 26ly be implemented in the method that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, in the step of described described second real-time message data synchronously and the described second real-time video signal data, described is that system time with described second network game client adds the above time tag with as the broadcast described second real-time message data and time of showing the described second real-time video signal data synchronously.

32. as claimed in claim 26ly be implemented in the method that real-time audio-video talks are carried out in online game with audio-visual simultaneous techniques, it is characterized in that, in the step of described described second real-time message data synchronously and the described second real-time video signal data, described is quantity with described time tag and the described second real-time video signal data synchronously, makes the described second real-time video signal data carry out the interval and shows.

33. one kind is implemented in online game with audio-visual simultaneous techniques and carries out the system that real-time audio-video is talked, be used between one first network game client, one second online game client and the network game server, described network game server is in order to carry out an online game, it is characterized in that described system comprises at least:

One Network Transmission channel, described first network game client and described second network game client are online to described network game server, to carry out described online game, described Network Transmission channel is coupled between described first network game client and described second network game client to carry out the real-time audio-video talks.

Images