CN1960408A - Multimedia responsion method in interactive mode in use for interactive multimedia responsion method - Google Patents

Abstract

An interactive multimedia response method for an interactive multimedia response system, comprising: a user terminal and an interactive multimedia response server respectively register with an H.323 gatekeeper, when the user terminal dials a special service number belonging to the interactive multimedia response server , the H.323 gatekeeper guides the call of the user terminal to the interactive multimedia response server; the user terminal encodes the user's key or the item selected by the user on the prompt screen into a certain value, and inserts the value into the real-time payload The protocol packet is transmitted to the interactive multimedia response server; the interactive multimedia response server analyzes the user's keys or the items selected by the user on the prompt screen from the received real-time payload stream, and provides further prompts or directly provides corresponding content. The invention can realize the interactive multimedia response in the H.323 video network.

Description

Interactive multimedia response method for interactive multimedia response system

technical field

The invention relates to the field of multimedia communication, in particular to an interactive multimedia response method used in an interactive multimedia response system. Through the method of the invention, interactive multimedia application value-added services can be provided to various communication terminals based on the H.323 protocol.

Background technique

The existing H.323 network and terminal systems currently only support basic point-to-point or point-to-multipoint video calls and conference control functions, that is, only one user terminal can be established between one user terminal and another. Video calls or establish communication between one user terminal and multiple user terminals, while there are no standards and specifications for automatic voice response center systems. The function of the interactive multimedia response center is to receive service requests from users, and provide feedback to users by using menu switching and text-to-speech conversion. That is to say, the Interactive Multimedia Response Center (IMR) cannot be used in the H.323 network and terminal system, and the functional requirements for automatic multimedia interactive response through the H.323 packet network and the H.323 terminal cannot be met.

In the field of voice calls, there is currently an interactive voice response center (IVR) technology, which uses a set of IVR systems to receive service requests from users by using dual-tone multi-frequency DTMF signal reception and voice recognition. Menu toggles, text-to-speech provide feedback to the user. The terminals that can be served are mainly telephone terminals based on circuit switching networks such as landline phones and mobile phones. Since the IVR system can only provide voice as the answering method, the media method is relatively simple, and the user terminal does not have recording and playback functions, so many users need to use paper and pen to record some useful information, and the user's service convenience can be further improved. improve. At the same time, the IVR system cannot serve terminals such as videophones based on the H.323 packet switching network. It can only be used in the voice circuit-based switching network, and cannot be used in the H.323 video network. Therefore, how to upgrade this IVR to an IMR that can serve in the H.323 video network has become an urgent problem to be solved in the prior art.

The main limitations of IVR technology are: (1) terminal limitations, which cannot provide call center services for users based on H.323 videophone; (2) media limitations, IVR downlink only provides simple voice services to users, and cannot Supports text, pictures, videos, and pages; (3) The limitation of user interaction capabilities. The interaction mode for users to upload to the IVR only supports voice interaction, and does not support screen navigation. By introducing mature auxiliary systems in the prior art on the interactive response server, including speech recognition system (4001), text-to-speech conversion system (4002), Vxml (Voice XML) system (4003), database system (4004), It can complete speech recognition, obtain speech semantics, convert text into speech, obtain and format Internet information into a specified format, and it is easy for IVR downstream to provide users with text, pictures, video and page responses other than speech. Therefore, it can receive user button and page interaction information from the H.323 terminal, and support button interaction and screen navigation, that is, the above (3) point is to upgrade the IVR to IMR to realize the interactive multimedia response in the H.323 video network The key to the system and method. In addition, in order to realize a good interactive multimedia response, it is also necessary to acquire the interaction capability of the user terminal and adaptively select an interaction mode according to the interaction capability.

Contents of the invention

Therefore, an object of the present invention is to realize upgrading IVR to IMR by being able to receive user button interaction information and user page interaction information from the H.323 terminal, thereby realizing the interactive multimedia response system and method in the H.323 video network .

A further object of the present invention is to provide a further improved interactive multimedia response system in the H.323 video network that can automatically obtain the interactive capability information of the H.323 terminal, and can adaptively select a suitable feedback mode or combination of feedback modes and methods.

The solution of the present invention is: an interactive multimedia response method for an interactive multimedia response system, the interactive multimedia response system comprising H.323 broadband packet network, user terminal, H.323 gatekeeper and interactive multimedia Response server, the interactive multimedia response method includes: the user terminal and the interactive multimedia response server register with the H.323 gatekeeper respectively, and when the user terminal dials the special service number belonging to the interactive multimedia response server, the H.323 network The guard guides the call of the user terminal to the interactive multimedia answering server; after the user terminal gets the prompt of voice, picture or video from the interactive multimedia answering server, it encodes the user's button or the item selected by the user on the prompting screen into a certain value, insert the value into the real-time payload protocol packet and transmit it to the interactive multimedia response server; the interactive multimedia response server parses out the user's button or the user's selection on the prompt screen from the received real-time payload stream , provide further hints or provide corresponding content directly.

Among them, the Capability field of the H.245 protocol message is extended to include terminal capabilities combined with text, pictures, pages, and streaming; after the call of the user terminal is directed to the interactive multimedia response server, the interactive multimedia The response server sends the H.245 protocol inquiry message with the Capability field extended to the user terminal for inquiry. After receiving the H.245 protocol message with the Capability field extended, the user terminal responds in the H.245 protocol reply message with the Capability field extended according to its own capabilities.

Preferably, the terminal capabilities combined with text, pictures, pages, and stream playback include text codec processing capabilities, picture codec capabilities, stream playback capabilities, and page support capabilities.

Wherein, after receiving the above-mentioned H.245 protocol response message with the extended Capability field, the interactive multimedia response server determines the response mode according to the terminal capability included in the response message.

Preferably, the process of determining the response method is as follows: when the user terminal capability is voice, determine the response method as voice; when the user terminal capability is voice+text, determine the response method as voice+message; when the user terminal capability is voice +text+picture+page, determine the answering method as voice+WEB; when the user terminal capability is voice+stream playback, determine the answering method as voice+video.

The present invention encodes the key of the user or the item selected by the user on the prompt screen into a certain value, inserts the value into the real-time payload protocol packet and transmits it to the interactive multimedia response server. On the other hand, the interactive multimedia response The server parses the user's key or the item selected by the user on the prompt screen from the received real-time payload stream, thereby realizing receiving user key interaction information and user page interaction information from the H.323 terminal, and further realizing The interactive multimedia response system and method in the H.323 video network solve the technical problem to be solved by the present invention.

In addition, because the present invention expands the Capability field of the H.245 protocol message to include terminal capabilities combined with text, pictures, pages, and stream playback, the interactive multimedia response server sends the H.245 extended Capability field to the user terminal. .245 protocol query message, and respond according to its own capabilities in the H.245 protocol response message with the Capability field extended. Through such a system for querying and responding to terminal capabilities, an improved self-adaptive An improved multimedia response system and method for selecting response and feedback modes according to user terminal capabilities.

Description of drawings

Fig. 1 is a structural diagram of the interactive multimedia response system based on the H.323 protocol of the present invention.

Fig. 2 is a component structure diagram of the IMR server of the present invention.

Fig. 3 is a format diagram of an RTP packet encapsulating a key event according to the present invention.

FIG. 4 is a format diagram of an RTP packet encapsulating items of a page selected by a user according to the present invention.

Detailed ways

The interactive multimedia response system based on the H.323 protocol is shown in Figure 1.

The interactive multimedia response system shown in Figure 1 includes:

H.323 broadband packet network (2001), based on the TCP/IP protocol, is a packet network capable of carrying H.323 multimedia services.

A user terminal (100), which may be a traditional user terminal, that is, a telephone terminal, or a user terminal (100) with video services, including video phone terminals (1001, 1002) based on H.323 and Broadband mobile phones (1003, 100N) that support H.323.

The interactive multimedia response server (3002) is responsible for automatic connection of multimedia calls, response logic processing, voice playback and interaction with back-end auxiliary systems. It is expanded to support picture, webpage and obtain simply by connecting speech recognition system (4001), text-to-speech system (4002), Vxml system (4003) and database system (4004) by the IVR server of prior art, certainly, for Support the invention point of the present invention as follows, that is, the user's button or the item selected by the user on the prompt screen is coded into a certain value, and the value is inserted into the RTP packet and sent. This pairing must be supported in the IMR server (3002). Contains the support of the RTP protocol packet of coded button and page entry value, and this can realize support by simple programming, and this is that those skilled in the art know easily, and this server also will support the present invention to the H.245 protocol message The expansion of the Capability field makes it include terminal capabilities combined with text, pictures, pages, and stream playback, which can also be obtained through programming by those skilled in the art.

The H.323 gatekeeper (3001) is responsible for the registration of the user terminal (100) and the interactive multimedia response server (3002), and is also responsible for bandwidth management, call permission and alias mapping addressing functions.

The speech recognition system (4001) has the function of automatic speech recognition (ASR) to complete the automatic speech recognition. If the user uses speech as the interaction mode in the process of interacting with the call processing server, the call processing server invokes the ASR function to obtain the user's speech in real time Semantics for further analysis and processing.

Text-to-speech conversion system (4002), with the function of text-to-speech conversion (TTS), completes the automatic text-to-speech conversion function, responds to the call of the call processing server, converts the specified text into a voice file, and plays it in real time for the call processing server to call to the user.

The Vxml system (4003) completes the Internet information induction and formatting functions, uses VXML language to query, obtain and format Internet information into a specified format according to user interaction requests, and then realizes voice playback to users through TTS.

The database system (4004) mainly stores user data and business data, such as text information, VXML files, video files, recording files, etc.

Wherein, the interactive multimedia response (IMR) server system logical block diagram 2 shows:

The functions of each component of the Interactive Multimedia Response (IMR) server are as follows:

System Management Component (30022):

It mainly completes the business management and configuration functions of the IMR system, such as system configuration, business authentication, business order, business billing and other functions.

Downstream media processing component (30025):

It mainly deals with the generation, playback and delivery of feedback media from the IMR system to the end user. Such as providing users with voice prompts, message prompts, screen prompts, video prompts or feedback in combination.

Uplink media processing component (30024):

It mainly completes the reception and interpretation functions of the media flow and control flow from the user terminal to the IMR system, such as receiving voice, video, and page clicks from the user terminal.

Call Control Component (30023):

It mainly completes the call processing and call control functions between the user and the IMR system, such as queuing, connection, and release.

Business management component (30021):

The main operation is interactive menu logic control, that is, to control the business process and feedback method according to the judgment result of user interaction.

The above-mentioned components are basically supplemented by the corresponding components of the ISR from voice services to supporting pictures, videos and web pages, and are implemented by those skilled in the art through programming. The support for the reception of the following RTP packet encapsulated content and the reception of the extended H.245 protocol message is realized through programming. All of these can be realized by programming, and the key point of the present invention is to add the terminal-related capability field in the following keystrokes and page information of the RTP packet content encapsulation code and the expansion of the H.245 protocol.

In order to support the automatic acquisition of terminal interaction capability information in the IMR server, and automatically select the corresponding feedback/response mode or combination function, videophone and broadband mobile phone terminals should support the extension of the H.245 protocol through programming, and establish a connection with the server At the same time, it can exchange information on its ability to process voice, text, web pages, video, and streaming media.

The specific workflow is described below:

1) Start the user terminal (100), and register the terminal address, terminal alias and number together with the H.323 gatekeeper (3001)

2) IMR server (3002) also registers system address, alias and series special service number to H.323 gatekeeper (3001);

3) The user terminal (100) dials a special service number (such as 16800008), the call request of the terminal is analyzed by the H.323 gatekeeper, and the H.323 gatekeeper can resolve that the special service number belongs to an IMR (Interactive Multimedia Response) server (3002), then the guide terminal calls to the IMR (Interactive Multimedia Response) server (3002);

4) The IMR server (3002) automatically performs queuing or connection upon receiving the call request from the terminal;

5) After the IMR server (3002) connects to the terminal, it automatically obtains the service capability of the terminal. This step is optional and is a perfection of the present invention. It is mainly realized by expanding the content of capability exchange in H.245. The existing H.245 mainly stipulates that the capabilities exchanged include:

(1) Protocol processing capability: supported series of protocols and versions in H.323, etc.

(2) Audio codec capability: such as G.711, G.728, G.729, AMR, etc.

(3) Video codec capability: such as H.261, H.263, H.263+, H.264, IS11172, etc.

(4) Data capability: such as T.120, T.38, V.42bis, etc.

(5) Encryption capability: such as antiSpamAlgorithm, etc.

(6) User interaction capabilities: such as basicString, iA5String, dtmf, etc.

(7) Conference control capabilities: such as chairman control, H.230MVC, etc.

(8) Network capability: such as the maximum supported bit rate, etc.

(9) Audio telephony event processing capability: such as dynamic RTP encapsulation of telephony events, etc.

The content of capability exchange that needs to be further expanded includes:

Text codec processing capabilities: such as GB2312, Unicode, UTF-8, etc.

Picture codec capability: such as GIF, BMP, JPG, etc.

Streaming capability: such as rtsp, MMS, MPEG-4, etc.

Page support capability: such as HTTP1.0, HTTP1.1, etc.

The IMR server sends a capability request message to the user terminal about the support for the extended four types of capabilities. The message name is TerminalCapabilitySet, and the message type is Request. This message is to expand the Capability field in the original H.245 message to include the above 4 capabilities. owned. The Capability field in the original H.245 message is as follows:

Capability::=CHOICE

{

nonStandard NonStandardParameter,

receiveVideoCapabilityVideoCapability,

transmitVideoCapability VideoCapability,

receiveAndTransmitVideoCapabilityVideoCapability,

receiveAudioCapability AudioCapability,

transmitAudioCapability AudioCapability,

receiveAndTransmitAudioCapability AudioCapability,

receiveDataApplicationCapabilityDataApplicationCapability,

transmitDataApplicationCapabilityDataApplicationCapability,

receiveAndTransmitDataApplicationCapabilityDataApplicationCapability,

h233EncryptionTransmitCapability BOOLEAN,

h233EncryptionReceiveCapability SEQUENCE

{

h233IVResponseTime INTEGER(0..255), --unitsmilliseconds

...

},

...,

conferenceCapability ConferenceCapability

}The expanded format is as follows:

Capability::=CHOICE

{

nonStandard NonStandardParameter,

receiveVideoCapabilityVideoCapability,

transmitVideoCapability VideoCapability,

receiveAndTransmitVideoCapabilityVideoCapability,

receiveAudioCapability AudioCapability,

transmitAudioCapability AudioCapability,

receiveAndTransmitAudioCapability AudioCapability,

receiveDataApplicationCapabilityDataApplicationCapability,

transmitDataApplicationCapabilityDataApplicationCapability,

receiveAndTransmitDataApplicationCapabilityDataApplicationCapability,

PictureCapability ImageCapability,

StreamingCapability StreamingCapability,

TextCapability TextCapability,

WebCapability WebCapability,

h233EncryptionTransmitCapability BOOLEAN,

h233EncryptionReceiveCapability SEQUENCE

{

h233IVResponseTime INTEGER(0..255), --unitsmilliseconds

...

},

...,

conferenceCapability ConferenceCapability

}

In the underlined part above,

ImageCapability::=CHOICE

{

nonStandard NonStandardParameter,

JpgCapabilty JpgCapabilty,

GifCapabilty GifCapabilty,

BmpCapabilty BmpCapabilty,

…

}

StreamingCapability::=CHOICE

{

nonStandard NonStandardParameter,

mmsCapabilty mmsCapabilty,

rtspCapabilty rtspCapabilty,

mpeg4Capabilty mpeg4Capabilty,

…

}

TextCapability::=CHOICE

{

nonStandard NonStandardParameter,

GB2312Capabilty GB2312Capabilty,

Unicode Capabilty Unicode Capabilty,

UTF8Capabilty UTF8Capabilty,

…

}

WebCapability::=CHOICE

{

nonStandard NonStandardParameter,

http10Capabilty http10Capabilty,

http11Capabilty http11Capabilty,

…

}

After receiving the H.245 protocol message with the Capability field extended, the user terminal (100) responds according to its own capabilities in the H.245 protocol response message with the Capability field extended. Supported capabilities are indicated in the response packet, and unsupported capabilities are not responded. The message name is TerminalCapabilitySetAck, and the message type is Response. This message is also carried out according to the above-mentioned extended H.245 protocol. The capability description field in the message selects the table items supported by the terminal according to the expanded fields.

6) After the interactive multimedia response server (3002) receives the above-mentioned H.245 protocol response message with the Capability field extended, it determines the response mode according to the terminal capability contained in the response message, and performs further user-specific response according to the response mode. Operating tips. Specifically, according to the call established by the user, the overall response mode that can be provided on the IMR server side is predefined, such as supporting voice, picture, voice+picture mode. After completing the above-mentioned capability acquisition process, the IMR system can Choose the appropriate response method according to the terminal capability, as shown in the table below: terminal capability Response method Remark voice voice voice+text voice+message voice, text, picture, page Voice+WEB Voice + streaming Voice+Video other other

Table 1 Adaptive feedback mode selection table

Since the above step 5) is optional, this step is also optional, which is a further improvement of the present invention.

7) After the user terminal obtains the system voice, screen or video prompt from the terminal downlink media processing part of the IMR server, it can interact with the terminal uplink media processing part of the IMR system through buttons, voice, and screen to express the user's further operation intention . The situation of voice interaction will not be repeated. For the button and page, the user's button or the item selected by the user on the prompt screen is encoded as a certain value, because a real-time payload (RTP) protocol has been established between the terminal and the IMR server to transmit audio and video signals two-way channel, just insert this value into the real-time payload (RTP) protocol packet and transmit to the interactive multimedia response server (3002), and the uplink media processing part of the interactive multimedia response server (3002) receives the key input and input from the terminal For operations such as clicking on the screen, the service management part of the IMR cooperates with the call control part to analyze the user's button or the item selected by the user on the prompt screen from the received real-time payload (RTP) stream, and provide further prompts or Provide the corresponding content directly.

This process is specifically for the button mode: when the terminal determines through the prompt from the IMR server that it wants to interact through the button mode, the user presses a certain key (such as "1-9, *, #" etc.) on the terminal; For each button event, the corresponding decimal number is encapsulated into an RTP packet (extending the separate Payload type of the button event), and inserted into the existing RTP media channel between the terminal and the IMR server in real time for transmission. The RTP packet format of the button event As shown in Figure 3.

in:

Event type: 8 bits, identifying the event type in this RTP packet, set to 11111110, indicating that this RTP packet is a key event packet;

E: Set to 0 to indicate that the event has not been completely transmitted, and set to 1 to indicate that this packet is the last packet of this event.

R: always set to 0, it can be expanded for other purposes in the future;

Value: Send the specific key value. After the user presses the key, the terminal will immediately correspond the user key to a decimal value. The corresponding table is as follows: keypress(event) decimal number Remark 0-9 000000-000101 * 001000 # 001001 other to be determined

Table 2 User key event coding table

Event ID: From 0000000000000001-111111111111111, generated in sequence for recycling.

The uplink media processing module of the IMR server parses out the user's key value from the RTP stream, and sends the key value to the service management component, which further prompts or provides services according to the user's key press.

This process is specifically for the page mode: when the user terminal determines that it wants to interact through the screen mode through a prompt from the IMR server, the user will select an item in the screen fed back by the IMR server (by pressing the up, down, left, and right keys and the confirmation key); After the user presses the confirm button, the terminal encodes the unknown ID of the item selected by the user (encoded according to the row and column number of the entry point) into the Value value of the RTP packet and transmits it to the IMR system. The packet format is shown in Figure 4.

in:

Event type: 8 bits, identifying the event type in this RTP packet, set to 11111101, indicating that this RTP packet is a page interaction event packet;

E: Set to 0 to indicate that the event has not been completely transmitted, and set to 1 to indicate that this packet is the last packet of this event.

R: always set to 0, it can be expanded for other purposes in the future;

Value: transmit the specific item ID. After the user presses the key, the terminal will immediately correspond the user key to a decimal value. The corresponding table is as follows:

The first 3 bytes are row numbers, and the last 3 bytes are column numbers.

Event ID: From 0000000000000001-111111111111111, generated in sequence for recycling.

The IMR server can parse out the encoded items selected by the user on the web page according to the payload type of "picture text" in the RTP stream. The uplink media processing module of the IMR server, when receiving an event of type 11111101 in the RTP packet, parses out the code of the service interaction entry selected by the user from the packet, and hands it to the service management module for service provision or further service prompts.

The IMR server can be implemented in the form of a software multipoint control unit (MCU), which refers to a set of software systems running on a general hardware platform (such as a server, Compact PCI chassis, ATCA server architecture), and can complete IMR with the cooperation of software and hardware. server function.

The present invention has the following advantages in application: extensive terminal support and protocol support, expanding the support for terminals such as H.323 protocol and videophone; extending the uplink interaction from the user to the system to support buttons, voice, and screen modes at the same time ; The downlink presentation/feedback from the system to the user can be extended to voice, picture, video and pairwise combination; the ability of the system to perceive the user terminal can be expanded, and an appropriate downlink presentation mode can be adaptively selected.

Claims (6)

1. interactive multimedia answer method that is used for the interactive multimedia answering system, H.323 described interactive multimedia answering system comprises broadband packet (2001), user terminal (100), H.323 gatekeeper (3001) and interactive multimedia answering server (3002), and described interactive multimedia answer method comprises:

User terminal (100), interactive multimedia answering server (3002) are registered to gatekeeper (3001) H.323 respectively, when user terminal (100) was dialed the special service number that belongs to interactive multimedia answering server (3002), H.323 gatekeeper (3001) directed into interactive multimedia answering server (3002) with the calling of user terminal (100);

After user terminal (100) obtains the prompting of voice, picture or video from interactive multimedia answering server (3002), the clauses and subclauses that user's button or user selected on hint image are encoded to certain numerical value, this numerical value is inserted in real-time payload (RTP) protocol package is transferred to interactive multimedia answering server (3002);

Interactive multimedia answering server (3002) parses described user's button or the clauses and subclauses that the user selects at hint image from real-time payload (RTP) stream that receives, further prompting is provided or corresponding contents directly is provided.

2. according to the method for claim 1, it is characterized in that also comprising:

H.245 the Capability field of protocol message is expanded, made it comprise the terminal capability that is combined with text, picture, the page, spreads and put;

Calling at user terminal (100) is guided interactive multimedia answering server (3002) afterwards, and interactive multimedia answering server (3002) sends the above-mentioned H.245 agreement apply for information of having expanded the Capability field to user terminal (100) and inquires;

After user terminal (100) is received the H.245 protocol message of having expanded the Capability field, in having expanded the H.245 agreement response message of Capability field, reply according to the ability of oneself.

3. according to the method for claim 2, it is characterized in that:

Describedly be combined with text, picture, the page, the terminal capability of putting that spreads comprises text encoding and decoding disposal ability, picture coding/decoding capability, the exoergic power of spreading, page tenability.

4. according to the method for claim 2, it is characterized in that, interactive multimedia answering server (3002) after receiving the above-mentioned H.245 agreement response message of having expanded the Capability field, according to the terminal capability that in this response message, comprises, the definite response mode.

5. according to the method for claim 4, it is characterized in that the process of described definite response mode is as follows: when ability of user terminal was voice, the definite response mode was voice; When ability of user terminal was voice+text, the definite response mode was voice+message; When ability of user terminal was the voice+text+picture+page, the definite response mode was voice+WEB; When ability of user terminal is voice+spread when putting, the definite response mode is voice+video.

6. according to the method for claim 4, it is characterized in that: interactive multimedia answering server (3002) carries out the user terminal prompting according to the response mode of determining.

Images