WO2016177262A1 - Collaboration method for intelligent conference and conference terminal - Google Patents

Abstract

Provided are a collaboration method for an intelligent conference and a conference terminal. The method comprises: a conference terminal receiving multimedia information about a conference sent by a remote device; according to the multimedia information about the conference and a pre-set multimedia content, the conference terminal determining whether the user needs to participate in the conference; the conference terminal determining a participating state of the user; and if the conference terminal determines that the user needs to participate in the conference and the participating state of the user is an absent state, the conference terminal sending participation reminding information to the user, the participation reminding information being used to prompt the user to participate in the conference. The method provided in the embodiments of the present invention can accurately control the participation time of a user, so that the user will not miss the content of a conference, thereby improving the participation efficiency of the user.

Description

Intelligent conference collaboration method and conference terminal

The present application claims priority to Chinese Patent Application No. 201510227117.8, entitled "Collaborative Method for Intelligent Conference and Conference Terminal", which is filed on May 6, 2015, the entire contents of which is hereby incorporated by reference. in.

Technical field

The embodiments of the present invention relate to communication technologies, and in particular, to a method and a conference terminal for an intelligent conference.

Background technique

For many people who are busy with work, there is often a need to attend multiple meetings at the same time. But one can only participate in a traditional face-to-face meeting at the same time. The emergence of telephones, video conferencing systems, etc. allows users to join multiple remote conferences at the same time, but it is difficult for one person to keep track of the progress of multiple conferences. When attending certain meetings, there is often a large part of the time that the content of the meeting has little to do with the participants, but the participants must also participate. Most of the participants are in a state of listening to other people's speeches or doing irrelevant things. The participants' real interest in the meeting content or the meeting requires participants to participate in a shorter and more random time, thus causing participants to waste. A lot of time.

Since the content of the participants involved in the conference often only occupies a small part of the time period of the conference, in order to save time, the participants participate in the participation in a time-sharing manner, but when the party joins in this way The time point is also difficult to control. It may be that the previous topic is postponed or advanced, and it is impossible to participate in the correct time period. The meeting content that the participants are concerned about is missed, so that the participation of the participants is not efficient.

Summary of the invention

The embodiments of the present invention provide a method for collaborating on a smart conference and a conference terminal, which are used to solve the joining time of the participants in the prior art, which causes the participants to miss the conference content of the conference and invite the participants to participate. Inefficient technical problems.

In a first aspect, an embodiment of the present invention provides a collaboration method for an intelligent conference, including:

The conference terminal receives the multimedia information of the conference sent by the remote device; the multimedia information of the conference includes at least one of voice information, image information, and text information; wherein the conference terminal is a terminal that has joined the conference;

Determining, by the conference terminal, whether the conference requires the user to participate according to the multimedia information of the conference and the preset multimedia content, where the preset multimedia content includes: the voice content that the user pays attention to, the At least one of image content of interest of the user, associated information of the image content of interest of the user, and text content of interest of the user;

Determining the participation status of the user by the conference terminal; the participation status of the user includes an absent state or a user participation Conflict state

If the conference terminal determines that the conference requires the user to participate, and the participant's participation status is absent, the conference terminal sends the conference reminder information to the user; the conference reminder information is used for The user is prompted to participate in the meeting.

With reference to the first aspect, in a first possible implementation manner of the first aspect, the conference terminal determines, according to the multimedia information of the conference and the preset multimedia content, whether the conference requires the user to participate, including:

Determining, by the conference terminal, whether preset conference information exists in the multimedia information of the conference according to the multimedia information of the conference and the preset multimedia content; the preset conference information is used for characterization Information about the user participating in the meeting;

If the preset conference information exists in the multimedia information of the conference, the conference terminal determines that the conference requires the user to participate.

With reference to the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, the conference terminal determines, according to the multimedia information of the conference and the preset multimedia content, Whether there is preset meeting information in the multimedia information of the meeting, including:

If the conference terminal determines that the preset conference information does not exist in the multimedia information of the conference, the conference terminal determines whether the first multimedia information that is of interest to the user exists in the multimedia information of the conference;

When the first multimedia information exists in the multimedia information of the conference, the conference terminal determines the user attention of the first multimedia information according to the first multimedia information and the first mapping relationship. The first mapping relationship is a correspondence between different content in the preset multimedia content and user attention degree;

The conference terminal determines whether the user attention of the first multimedia information is greater than a preset user attention threshold; if yes, the conference terminal determines that the conference requires the user to participate; if not, the conference The conference terminal determines that the conference does not require the user to participate.

With reference to the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, the conference terminal determines, according to the multimedia information of the conference and the preset multimedia content, Whether the first multimedia information that the user pays attention to exists in the multimedia information of the conference, including:

The conference terminal detects and identifies the multimedia information of the conference, obtains a recognition result, and performs feature matching on the recognition result and the preset multimedia content to obtain a feature matching result;

Determining, by the conference terminal, whether the feature matching result is greater than a preset matching threshold;

If yes, the conference terminal determines that the first multimedia information exists in the multimedia information of the conference;

If not, the conference terminal determines that the first multimedia information does not exist in the multimedia information of the conference.

In conjunction with the first aspect to any one of the third possible implementation manners of the first aspect, in a fourth possible implementation manner of the first aspect, the method further includes:

If the conference terminal determines that the conference requires the user to participate, and the participant's participation status is absent, the conference terminal determines the form of the conference reminder information; wherein the conference reminder information Interface form At least one of a form, an image form, a video form, an audio form, and an instant message form;

The conference terminal sends the conference reminder information to the user, including:

The conference terminal sends the participation reminding information to the user according to the form of the participation reminding information.

In combination with the first aspect to any one of the fourth possible implementation manners of the first aspect, in a fifth possible implementation manner of the first aspect, the method further includes:

Determining, by the conference terminal, a feedback policy according to whether the conference requires the user participation and the participation status of the user; the feedback policy is used to indicate that the conference terminal sends the type of feedback data to the remote device;

The conference terminal sends the feedback data to the remote device according to the feedback policy; the feedback data includes: video content that interacts with the remote device, and audio content that interacts with the remote device At least one of textual content that interacts with the remote device.

With reference to the fifth possible implementation manner of the first aspect, in a sixth possible implementation manner of the first aspect, the conference terminal, according to whether the conference requires the user to participate in the conference, and the participant's participation status Identify feedback strategies, including:

If the conference terminal determines that the conference requires the user to participate, and the participant's participation status is absent, the feedback policy determined by the conference terminal is to send the first feedback data to the remote device. The first feedback data is used to indicate to the remote device that the conference terminal is notifying the user to join the conference.

With reference to the fifth possible implementation manner of the first aspect, in a seventh possible implementation manner of the first aspect, the conference terminal, according to whether the conference needs the user participation and the participation status of the user Identify feedback strategies, including:

If the conference terminal determines that the conference requires the user to participate, and the participant's participation status is absent, the feedback policy determined by the conference terminal is to send second feedback data to the remote device. The second feedback data is used to show the remote device the conference content related to the conference preset by the user.

With reference to the fifth possible implementation manner of the first aspect, in an eighth possible implementation manner of the first aspect, the conference terminal, according to whether the conference requires the user to participate in the conference, and the participant's participation status Identify feedback strategies, including:

If the conference terminal determines that the conference does not require the user to participate, and the participant's participation status is absent, the feedback policy determined by the conference terminal is to send a third feedback to the remote device. Data; the third feedback data is used to indicate to the remote device that the user is participating.

With reference to the fifth possible implementation manner of the first aspect, in a ninth possible implementation manner of the first aspect, the conference terminal, according to whether the conference requires the user to participate in the conference, and the participant's participation status Identify feedback strategies, including:

If the conference terminal determines that the conference requires the user to participate, and the participant's participation status is a user participation conflict state, the feedback policy determined by the conference terminal is to send the first to the remote device. The fourth feedback data is used to record the current conference content of the conference; the fourth feedback data is used to indicate to the remote device that the participant's participation status is a user participation conflict state.

With reference to the second possible implementation of the first aspect to any one of the ninth possible implementation manners of the first aspect, in the tenth possible implementation manner of the first aspect, The first multimedia information includes the face information of the first participant in the conference, and the associated information of the image content that the user pays attention to in the preset multimedia content is Determining identity information of the first participant; the conference terminal detects and identifies the multimedia information of the conference, obtains a recognition result, and performs feature matching on the recognition result and the preset multimedia content to obtain a feature. Matching results, including:

The conference terminal detects multimedia information of the conference, and determines a face position and a face size of the participant in the multimedia information;

The conference terminal performs feature extraction on the face position and the face size of the participant in the multimedia information to obtain a face feature of the participant;

The conference terminal matches each participant's face feature with a preset face information database to determine a first matching degree;

Determining, by the conference terminal, that the identification result is identity information of the participant whose first matching degree is greater than a preset first threshold in the face information database;

The conference terminal sets the identity information of the participant whose first matching degree is greater than the preset first threshold in the face information database and the identity of the first participant in the preset multimedia content. The information is matched to obtain the feature matching result.

In combination with the second possible implementation of the first aspect to any one of the ninth possible implementation manner of the first aspect, in the eleventh possible implementation manner of the first aspect, The first multimedia information includes first text information, and the text content of the user's attention is the first text information, and the conference terminal detects and identifies the multimedia information of the conference, obtains the recognition result, and Performing feature matching on the identification result and the preset multimedia content to obtain a feature matching result, specifically including:

The conference terminal detects multimedia information of the conference, and determines a location and a size of a text block area in the multimedia information of the conference;

And obtaining, by the conference terminal, a text block in the multimedia information of the conference according to a location and a size of the text block area in the multimedia information of the conference;

The conference terminal matches a text block in the multimedia information of the conference with a preset text information library, and determines a second matching degree;

The conference terminal determines that the recognition result is text information that the second matching degree in the text information library is greater than a preset second threshold;

The conference terminal matches the text information in the text information library with the second matching degree greater than the preset second threshold value and the first text information in the preset multimedia content, to obtain the feature. Match the result.

With reference to the second possible implementation of the first aspect to any one of the ninth possible implementation manners of the first aspect, in the twelfth possible implementation manner of the first aspect, The first multimedia information includes first text information, and the text content of the user's attention is the first text information, and the conference terminal detects and identifies the multimedia information of the conference, obtains the recognition result, and Performing feature matching on the identification result and the preset multimedia content to obtain a feature matching result, specifically including:

The conference terminal detects the multimedia information of the conference, determines the location and size of the text block area in the multimedia information of the conference, and obtains a text block in the multimedia information of the conference;

Determining, by the conference terminal, the recognition result according to a geometric feature of the text block;

The conference terminal matches the identification result with the first text information in the preset multimedia content to obtain the feature matching result.

With reference to the eleventh possible implementation manner of the first aspect or the twelfth possible implementation manner of the first aspect, in the thirteenth possible implementation manner of the first aspect, The information includes the first data information related to the first text information except the text type and the participant face type in the multimedia information of the conference, and the conference terminal detects the multimedia information of the conference. Identifying, obtaining a recognition result, and performing feature matching on the recognition result and the preset multimedia content to obtain a feature matching result, further comprising:

The conference terminal detects the multimedia information of the conference, and determines other data information in the multimedia information of the conference, except the text type and the participant face type;

Determining, by the conference terminal, the degree of relevance of the other data information and the first text information according to the first text information and the other data information;

Determining, by the conference terminal, that the identification result is data information that the correlation is greater than a preset third threshold in the other data information;

And the conference terminal matches the data information of the other data information that is greater than a preset third threshold with the first data information to obtain the feature matching result.

In conjunction with any of the fifth possible implementation of the first aspect to the thirteenth possible implementation of the first aspect, in the fourteenth possible implementation of the first aspect, the feedback data The feedback content preset for the user, or the data generated according to the feedback policy and the feedback content preset by the user.

In a second aspect, an embodiment of the present invention provides a conference terminal, including:

a receiving module, configured to receive multimedia information of a conference sent by the remote device, where the multimedia information of the conference includes at least one of voice information, image information, and text information, where the conference terminal is added to the conference terminal;

a first determining module, configured to determine, according to the multimedia information of the conference and the preset multimedia content, whether the conference needs the user to participate; wherein the preset multimedia content includes: the voice content that the user pays attention to At least one of image content of the user's attention, association information of the image content of interest of the user, and text content of the user's attention;

a second determining module, configured to determine a participating state of the user; the participating state of the user includes an absent state or a user participating conflict state;

a sending module, configured to send a reminder to the user when the first determining module determines that the conference requires the user to participate, and the second determining module determines that the participating state of the user is an absent state Information; the participation reminding information is used to prompt the user to participate in the meeting.

With reference to the second aspect, in a first possible implementation manner of the second aspect, the first determining module includes:

a first determining unit, configured to determine, according to the multimedia information of the conference and the preset multimedia content, whether preset conference information exists in the multimedia information of the conference, if the multimedia information of the conference exists Preset meeting When the information is discussed, it is determined that the conference needs the user to participate; the preset conference information is information used to represent that the user needs to participate in the conference.

In conjunction with the first possible implementation of the second aspect, in the second possible implementation of the second aspect, the first determining module further includes:

a second determining unit, configured to determine, when the first determining unit determines that the preset meeting information does not exist in the multimedia information of the conference, whether the first information of the user is present in the multimedia information of the conference Determining, by the first multimedia information, the first multimedia information according to the first multimedia information and the first mapping relationship, when determining that the first multimedia information that is of interest to the user exists in the multimedia information of the conference User attention of the body information; the first mapping relationship is a correspondence between different content in the preset multimedia content and user attention;

a determining unit, configured to determine whether the user attention level of the first multimedia information is greater than a preset user attention degree threshold; if yes, determining that the meeting requires the user to participate; if not, determining that the meeting is not The user is required to participate.

With reference to the second possible implementation manner of the second aspect, in a third possible implementation manner of the second aspect, the second determining unit is configured to detect and identify the multimedia information of the conference, and obtain the identifier Assending, the matching result is matched with the preset multimedia content to obtain a feature matching result, and determining whether the feature matching result is greater than a preset matching threshold; if yes, determining the multimedia information of the meeting The first multimedia information exists in the first multimedia information; if not, it is determined that the first multimedia information does not exist in the multimedia information of the conference.

In conjunction with the second aspect, the third possible implementation manner of the second aspect, in the fourth possible implementation manner of the second aspect, the conference terminal further includes:

a third determining module, configured to: when the first determining module determines that the conference needs the user to participate, and the second determining module determines that the participating state of the user is an absent state, determining the participation reminder a form of the information; wherein the form of the participation reminding information includes at least one of an interface form, an image form, a video form, an audio form, and an instant message form;

The sending module is specifically configured to send the participation reminding information to the user according to the form of the participation reminding information.

In conjunction with the second aspect, the fourth possible implementation manner of the second aspect, in the fifth possible implementation manner of the second aspect, the conference terminal further includes: a fourth determining module;

The fourth determining module is configured to determine, according to whether the conference requires the user to participate in the conference and the participation status of the user, the feedback policy is used to indicate that the conference terminal sends the conference terminal to the remote device. The type of feedback data;

The sending module is further configured to send the feedback data to the remote device according to the feedback policy; the feedback data includes: video content that interacts with the remote device, and the remote device At least one of interactive audio content, textual content that interacts with the remote device.

With reference to the fifth possible implementation manner of the second aspect, in a sixth possible implementation manner of the second aspect, the fourth determining module is specifically configured to: if the conference requires the user to participate, If the status of the user is absent, the feedback policy is determined to send the first feedback data to the remote device, where the first feedback data is used to The end device indicates that the conference terminal is informing the user to join the conference.

With reference to the fifth possible implementation manner of the second aspect, in a seventh possible implementation manner of the second aspect, the fourth determining module is specifically configured to: if the conference requires the user to participate, Determining the feedback policy is to send the second feedback data to the remote device, where the second feedback data is used to show the remote device The content of the meeting related to the meeting.

With reference to the fifth possible implementation manner of the second aspect, in the eighth possible implementation manner of the second aspect, the fourth determining module is specifically configured to: if the conference does not require the user to participate, Determining that the user's participation status is an absent state, determining that the feedback policy is to send third feedback data to the remote device; the third feedback data is used to indicate to the remote device that the user is participating in the conference .

In conjunction with the fifth possible implementation of the second aspect, in a ninth possible implementation manner of the second aspect, the fourth determining module is specifically configured to: if the conference requires the user to participate, If the user participates in the conflicting state, the feedback policy is to send the fourth feedback data to the remote device, and record the current conference content of the conference; the fourth feedback data is used to The remote device indicates that the participation status of the user is a user participation conflict state.

With reference to the second possible implementation of the second aspect to any one of the ninth possible implementation manner of the second aspect, in the tenth possible implementation manner of the second aspect, The first multimedia information includes the face information of the first participant in the conference, and the associated information of the image content that the user pays attention to in the preset multimedia content is the identity information of the first participant. The second determining unit specifically includes:

a first detecting subunit, configured to detect multimedia information of the conference, and determine a face position and a face size of the participant in the multimedia information;

a feature extraction sub-unit, configured to perform feature extraction on a face position and a face size of the participant in the multimedia information, to obtain a face feature of the participant;

a first matching sub-unit, configured to match a face feature of each participant with a preset face information database to determine a first matching degree;

a first determining subunit, configured to determine that the identification result is identity information of the participant whose first matching degree is greater than a preset first threshold in the face information database;

a second matching subunit, configured to use the identity information of the participant whose first matching degree is greater than a preset first threshold in the face information database and the first parameter in the preset multimedia content The identity information of the participants is matched, and the feature matching result is obtained.

In combination with the second possible implementation of the second aspect to any one of the ninth possible implementation manner of the second aspect, in the eleventh possible implementation manner of the second aspect, The first multimedia information includes the first text information, and the text content of the user's attention is the first text information, and the second determining unit specifically includes:

a second detecting subunit, configured to detect multimedia information of the conference, and determine a location and a size of a text block area in the multimedia information of the conference;

Obtaining a subunit, configured to obtain a text block in the multimedia information of the conference according to a location and a size of the text block area in the multimedia information of the conference;

a third matching subunit, configured to match a text block in the multimedia information of the conference with a preset text information library, and determine a second matching degree;

a second determining subunit, configured to determine that the recognition result is text information that the second matching degree in the text information library is greater than a preset second threshold;

a fourth matching sub-unit, configured to match the text information in the text information library that is greater than a preset second threshold with the first text information in the preset multimedia content, The feature matching result is obtained.

With reference to the second possible implementation of the second aspect to any one of the ninth possible implementation manner of the second aspect, in the twelfth possible implementation manner of the second aspect, The first multimedia information includes the first text information, and the text content of the user's attention is the first text information, and the second determining unit specifically includes:

a third detecting subunit, configured to detect multimedia information of the conference, determine a location and a size of a text block area in the multimedia information of the conference, and obtain a text block in the multimedia information of the conference;

a third determining subunit, configured to determine the recognition result according to a geometric feature of the text block;

And a fifth matching subunit, configured to match the identification result with the first text information in the preset multimedia content to obtain the feature matching result.

With reference to the eleventh possible implementation manner of the second aspect or the twelfth possible implementation manner of the second aspect, in the thirteenth possible implementation manner of the second aspect, The information includes the first data information related to the first text information in addition to the text type and the participant face type in the multimedia information of the conference, and the second determining subunit further includes:

a fourth detecting subunit, configured to detect multimedia information of the conference, and determine other data information in the multimedia information of the conference, except the text type and the participant face type;

a fourth determining subunit, configured to determine, according to the first text information and the other data information, a degree of correlation between the other data information and the first text information;

a fifth determining subunit, configured to determine that the identification result is data information that the correlation is greater than a preset third threshold in the other data information;

a sixth matching subunit, configured to match the data information whose correlation degree is greater than a preset third threshold value to the first data information in the other data information, to obtain the feature matching result.

With reference to any one of the fifth possible implementation manner of the second aspect to the thirteenth possible implementation manner of the second aspect, in the fourteenth possible implementation manner of the second aspect, the feedback data The feedback content preset for the user, or the data generated according to the feedback policy and the feedback content preset by the user.

The method for the collaboration of the intelligent conference and the conference terminal provided by the embodiment of the present invention, the conference terminal determines, according to the received multimedia information of the conference and the preset multimedia content, whether the conference needs the user to participate, and at the same time, the conference terminal Determining a participant's participation status; when the conference terminal determines that the conference requires the user to participate, and the user's participation status When the state is an absent state, the conference terminal sends the participant reminding information for prompting the user to participate in the conference. The method provided by the embodiment of the present invention determines whether the current conference requires the user to participate by using the conference terminal, and determines the sending reminder information to the user according to the current participation status of the user, so that the time for the user to join the conference can be accurately controlled, so that the user does not Missing the content of the meeting, thereby improving the efficiency of the user's participation.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief description of the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any inventive labor.

1 is a schematic diagram of an implementation framework of an intelligent conference collaboration robot device according to an embodiment of the present invention;

2 is a schematic diagram of networking of an intelligent video conference collaboration system according to an embodiment of the present invention;

FIG. 3 is a schematic flowchart diagram of Embodiment 1 of a method for collating intelligent conferences according to an embodiment of the present disclosure;

FIG. 4 is a schematic flowchart of Embodiment 2 of a method for collaborating on a smart conference according to an embodiment of the present disclosure;

FIG. 5 is a schematic flowchart of Embodiment 3 of a method for collating an intelligent conference according to an embodiment of the present disclosure;

FIG. 6 is a schematic flowchart of Embodiment 4 of a method for collating an intelligent conference according to an embodiment of the present disclosure;

FIG. 7 is a schematic flowchart of Embodiment 5 of a method for collating an intelligent conference according to an embodiment of the present disclosure;

FIG. 8 is a schematic flowchart of Embodiment 6 of a method for collating an intelligent conference according to an embodiment of the present disclosure;

FIG. 9 is a schematic flowchart of Embodiment 7 of a method for collaborating on an intelligent conference according to an embodiment of the present disclosure;

FIG. 10 is a schematic flowchart of Embodiment 8 of a method for collating an intelligent conference according to an embodiment of the present disclosure;

FIG. 11 is a schematic structural diagram of Embodiment 1 of a conference terminal according to an embodiment of the present disclosure;

FIG. 12 is a schematic structural diagram of Embodiment 2 of a conference terminal according to an embodiment of the present disclosure;

FIG. 13 is a schematic structural diagram of Embodiment 3 of a conference terminal according to an embodiment of the present disclosure;

FIG. 14 is a schematic structural diagram of Embodiment 4 of a conference terminal according to an embodiment of the present disclosure;

FIG. 15 is a schematic structural diagram of Embodiment 5 of a conference terminal according to an embodiment of the present disclosure;

FIG. 16 is a schematic structural diagram of Embodiment 6 of a conference terminal according to an embodiment of the present disclosure;

FIG. 17 is a schematic structural diagram of Embodiment 7 of a conference terminal according to an embodiment of the present disclosure;

FIG. 18 is a schematic structural diagram of Embodiment 8 of a conference terminal according to an embodiment of the present disclosure;

FIG. 19 is a schematic structural diagram of Embodiment 9 of a conference terminal according to an embodiment of the present disclosure.

detailed description

The technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Is the invention Some embodiments, but not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

The method of the embodiment of the present invention is applicable to an intelligent video conference collaboration system, and the system can be integrated into a conference terminal. The conference terminal can include, but is not limited to, a mobile phone, a personal digital assistant (PDA), a tablet computer, and a portable device. A communication device such as a device (for example, a portable computer) or a general computer (PC) may also be a smart conference collaboration robot device based on an intelligent video conference collaboration system, and an implementation framework diagram of the smart conference collaboration robot device can be seen in FIG. . In Figure 1, the bottom layer is the hardware layer of the robot device; above the hardware layer is the related device driver and protocol stack, the protocol stack may include a network protocol stack, such as a wireless fidelity (WiFi) protocol stack or a third generation mobile communication Network (3G) protocol stack, sensor protocol stack, etc., device driver can be GPS driver; the device driver and protocol stack is the operating system layer; the upper layer of the operating system is the application layer, including some functional modules of the intelligent conference collaboration robot and Intelligent video conferencing collaboration system. The embodiment of the present invention does not limit the form of the terminal.

Because the traditional participation process requires the user to participate in the whole meeting, even if only a small part of the content is related to the user, the user must participate in the entire meeting, otherwise it is possible to miss the small part of the content related to the user, which makes the user The participation efficiency is very low. Therefore, the present invention can integrate the above-mentioned intelligent video conference collaboration system in the terminal. When the terminal integrated with the intelligent video conference collaboration system receives the connection request sent by the remote device, it can replace the user (ie, the local conference). The user joins the conference, which means that the user can arrange the terminal integrated with the intelligent video conference collaboration system to participate in one or multiple conferences at the same time as a substitute for the user in the conference. FIG. 2 is a networking diagram of an intelligent video conference collaboration system according to an embodiment of the present invention.

In the traditional conference mode, participants can directly conduct face-to-face meetings or use audio and video conference systems to participate in the conference. In the embodiment of the present invention, the terminal integrated with the intelligent video conference collaboration system is used as a smart agent to participate in the conference instead of the participant. The intelligent video conference collaboration system can be regarded as a logical entity, and the physical implementation can be software or hardware equipment, and can also be deployed as a function module in the conference client software of the terminal. The terminal integrated with the intelligent video conference collaboration system can interact with the participants replaced by a graphical interface, voice, video, and the like.

The above-mentioned intelligent video conference collaboration system can support simultaneous interaction with multiple voice/video conferences. These voice/video conferences can be created by one or more video conference systems based on hard terminals or soft terminals. A terminal based on an intelligent video conferencing collaboration system can simulate a participant and can receive video, audio and data information from multiple voice/video conferences simultaneously. For example, the intelligent video conference collaboration system itself can be implemented as a conference client software, and a terminal installed with the intelligent video conference collaboration system can access one or more video conferences through a network as a participant; for example, intelligent video conference collaboration The system can be deployed on the same computer as independent software and conference client software, and communicates with the conference client software through the inter-process communication mechanism to obtain video, audio and data information of the conference.

The method designed in the embodiment of the present invention is intended to solve the problem that the participants cannot control the joining time of the participants in the prior art, and the participants miss the technical content of the meeting and the inconvenience of the attendees.

The technical solutions of the present invention will be described in detail below with specific embodiments. The following specific embodiments can be Combinations of the same or similar concepts or processes may not be described in some embodiments.

FIG. 3 is a schematic flowchart diagram of Embodiment 1 of a method for collating intelligent conferences according to an embodiment of the present invention. The execution body of the method according to the embodiment of the present invention may be a terminal (hereinafter referred to as a conference terminal or a local device) integrated with the above-mentioned intelligent video conference cooperation system. As shown in FIG. 3, the method includes:

S101: The conference terminal receives the multimedia information of the conference sent by the remote device, and the multimedia information of the conference includes at least one of voice information, image information, and text information, where the conference terminal is a terminal that has joined the conference. .

Specifically, after the conference terminal receives the conference connection indication sent by the remote device, the conference terminal accesses the conference, so that the conference terminal can participate in the conference as a virtual participant. That is, the conference terminal may receive multimedia information of the conference sent by the remote device of the conference through the conference server, and the multimedia information of the conference may include at least one of voice information, image information, and text information. Optionally, the foregoing image information may be transmitted by the remote device to the conference terminal by using a video. The image information may be the face information of the participant in the conference, and may also be information about other images involved in the conference (for example, , the picture involved in the film shared in the meeting), the above text information may be the text content in the film shared in the meeting.

S102: The conference terminal determines, according to the multimedia information of the conference and the preset multimedia content, whether the conference needs the user to participate; wherein the preset multimedia content includes: the voice content that the user pays attention to, the At least one of image content of interest of the user, associated information of the image content of interest of the user, and text content of interest of the user.

Specifically, after the conference terminal receives the multimedia information of the conference, the conference terminal determines, according to the multimedia information of the conference and the preset multimedia content, whether the current conference requires the user to participate. The preset multimedia content includes at least one of the voice content of the user's attention, the image content of the user's attention, the associated information of the image content of the user's attention, and the text content of the user's attention. Optionally, the conference terminal determines, according to the multimedia information of the conference and the preset multimedia content, whether the current conference needs the user to participate, or the conference terminal determines the correlation between the multimedia information of the conference and the preset multimedia content, according to the correlation degree. The size is used to determine whether the current conference requires the user to participate; or the conference terminal determines the proportion of the preset multimedia content in the multimedia information of the conference, and determines whether the current conference requires the user to participate according to the size of the ratio; The conference terminal determines whether the multimedia information of the conference has any one of the preset multimedia contents, to determine whether the current conference requires the user to participate. In an embodiment of the present invention, the embodiment of the present invention is not limited as long as it can determine whether the conference requires the user to participate according to the multimedia information of the conference and the preset multimedia content.

Optionally, the association information of the image content that the user pays attention to may be information related to the image content that the user pays attention to. For example, the image content that the user pays attention to is a face image, and the user included in the preset multimedia content is concerned. The association information of the image content may be identity information corresponding to the face information.

Optionally, the preset multimedia content may be preset in a memory inside the conference terminal, or may be preset in an external device connected to the conference terminal. When the conference terminal obtains the multimedia information of the conference, the conference terminal may be from the external device. The preset multimedia content is called for judgment (this can slow down the buffering pressure of the internal memory of the conference terminal).

S103: The terminal determines the participation status of the user; the participation status of the user includes an absent state or a user participation conflict state.

Optionally, the conference terminal may determine whether the user participates in the conference according to whether the voice of the user is detected, and whether the user can participate in the conference or determine whether the user is in the conference environment by whether the camera can capture the image of the user. Optionally, the user's participation status may include that the user is currently in an absent state (that is, the user is currently in the conference environment but does not participate in the conference discussion or the user is not in the conference environment) or the user participates in the conflict state, and the user participates in the conference state. This can include the user currently participating in the conference discussion. It should be noted that, after the “user is in the conference environment” can access the conference instead of the user, the user does not pay attention to the content of the conference, but does other important things, as long as the camera can capture the image of the user. The user is in the conference environment; the user is absent state can access the conference instead of the user for the conference terminal. The user is currently in the conference environment but does not participate in the conference discussion, or the user is not currently in the conference environment; For "whether the user participates in the discussion", it may be determined whether the user participates in the discussion by detecting whether the user's voice is received through the microphone or the sound detection software; for the above-mentioned "user participation conflict status", the user may be required to participate in the current conference, but the user If the conference is participating in another conference, the conference terminal determines that the user's participation status is the user's participation conflict status. For example, for the audio conference, the conference terminal determines that the current conference requires the user to participate, but the conference terminal detects that the microphone has already been detected. Open, the user is already participating in another meeting On the, the meeting participants considered that the user terminal on the state of the user participating state of conflict.

It should be noted that there is no order between the above S102 and S103, and the two can also be performed simultaneously.

S104: If the conference terminal determines that the conference requires the user to participate, and the participant's participation status is absent, the conference terminal sends the conference reminder information to the user; the conference reminder information Used to prompt the user to participate in the conference.

Optionally, the form of the participation reminding information sent by the conference terminal to the user may be in any form, as long as the user can be notified to participate in the conference in time, and the form of the reminder information is not limited in the embodiment of the present invention. .

The collaboration method of the smart conference provided by the embodiment of the present invention, the conference terminal determines, according to the received multimedia information of the conference and the preset multimedia content, whether the conference needs the user to participate, and the conference terminal determines the user's a conference state; when the conference terminal determines that the conference requires the user to participate, and the participant's participation status is absent, the conference terminal sends a conference to the user to prompt the user to participate in the conference. Reminder information. The method provided by the embodiment of the present invention determines whether the current conference requires the user to participate by using the conference terminal, and determines the sending reminder information to the user according to the current participation status of the user, so that the time for the user to join the conference can be accurately controlled, so that the user does not Missing the content of the meeting, thereby improving the efficiency of the user's participation.

FIG. 4 is a schematic flowchart diagram of Embodiment 2 of a method for collating intelligent conferences according to an embodiment of the present invention. This embodiment relates to a specific process of determining, by the conference terminal, whether the conference requires the user to participate according to the multimedia information of the conference and the preset multimedia content. Based on the foregoing embodiment, the foregoing S102 specifically includes:

S201: The conference terminal determines, according to the multimedia information of the conference and the preset multimedia content, the conference Whether there is preset conference information in the multimedia information; the preset conference information is information used to represent that the user needs to participate in the conference. If yes, execute S202, and if no, execute S203.

Optionally, the preset multimedia content may include preset conference information, where the preset conference information may be information in an audio format, or may be information in a video format, or may be in the form of image information, or may be The form of the text information may also be a combination of two or more of the above forms. The preset conference information is information that requires the user to participate in the current conference, that is, as long as the conference terminal determines that the preset conference information in the preset multimedia content exists in the multimedia information of the conference, the current conference can be determined. Users are required to participate.

Optionally, the conference terminal may determine whether the preset conference information exists in the multimedia information of the conference by determining whether the multimedia information of the conference is completely the same as or substantially the same as the preset conference information in the preset multimedia content. For example, the conference terminal may determine the similarity between the multimedia information of the conference and the preset conference information. When the similarity is greater than a certain threshold, the preset conference information may be determined in the multimedia information of the conference. For another example, the preset conference information may be the user's name XXX. When the conference terminal detects that the multimedia information of the conference includes the voice “reporting the conference-related content by XXX”, the conference terminal determines that the conference requires the user to participate; For example, the preset conference information may be the topic A that the user presets to be described in the conference. When the conference terminal detects that the multimedia information of the conference includes A, the conference terminal determines that the conference requires the user to participate. In general, the embodiment of the present invention does not limit the representation form of the preset conference information, as long as it can represent the current conference and requires the user to participate.

S202: The conference terminal determines that the conference requires the user to participate.

S203: The conference terminal determines whether the first multimedia information that is of interest to the user exists in the multimedia information of the conference. If yes, execute S204, if no, execute S207.

Specifically, when the conference terminal determines that the preset conference information does not exist in the multimedia information of the conference, the conference terminal further determines whether the first multimedia information of the user is present in the multimedia information of the conference. Specifically, the conference terminal determines whether the first multimedia information of the user's attention exists in the multimedia information of the conference, and may refer to the third embodiment shown in FIG. 5, which specifically includes the following steps:

S301: The conference terminal detects and identifies the multimedia information of the conference, obtains a recognition result, and performs feature matching on the recognition result and the preset multimedia content to obtain a feature matching result.

Specifically, after receiving the multimedia information of the conference sent by the remote device, the conference terminal detects and identifies the multimedia information of the conference, and determines text information or a character string or a picture information that can be recognized by the conference terminal, as a recognition result. It should be noted that the multimedia information of different forms of conferences may be different in the detection and identification manner adopted by the conference terminal. For example, when the multimedia information of the conference includes only the image information, the conference terminal extracts the feature parameters of the image, so that the conference terminal can determine the content of the image according to the feature parameters of the image, thereby obtaining the recognition result. Optionally, the feature parameter of the image information may be brightness, saturation, color, geometric feature, resolution, pixel feature, and the like of the image; when the multimedia information of the conference includes audio information, the conference terminal determines according to the voice recognition technology. The actual meaning of the voice in the audio information is obtained.

After the conference terminal obtains the identification result of the multimedia information of the conference, the conference terminal performs feature matching on the recognition result and the preset multimedia content to obtain a feature matching result. The feature matching result may be the same or related or close to the image content and the text content in the preset multimedia content (ie, the matching degree or the correlation degree).

S302: The conference terminal determines whether the feature matching result is greater than a preset matching threshold; if yes, execute S303, if no, execute S304.

Specifically, the conference terminal determines that the feature matching result may be the same or related or close to the preset multimedia content (ie, the matching degree or the correlation degree), and determines whether the correlation degree or the matching degree is greater than a preset. a matching threshold (the more the matching degree or the correlation exceeds the matching threshold, the more the correlation result is related to the preset multimedia content), and determining whether the first multimedia information of the user is in the multimedia information of the conference according to the determination result. .

S303: The conference terminal determines that the first multimedia information exists in the multimedia information of the conference.

Specifically, if the feature matching result is greater than a preset matching threshold, the conference terminal determines that the first multimedia information exists in the multimedia information of the conference, and the first multimedia information may be image data or user attention of the user. Text information or audio information of interest to the user.

Optionally, after the conference terminal determines that the first multimedia information that is of interest to the user exists in the multimedia information of the conference, the conference terminal may use the first multimedia information that the user pays attention to according to a certain policy (for example, the user pays attention to The image information, the voice information, and the text information are processed by format conversion or compression, and the processed first multimedia information is saved, which is convenient for the user to view later. For example, after the conference terminal determines that the first multimedia information is the image information of the user's attention or the voice information that the user pays attention to, the first multimedia information may be recorded as a file for storage, and the user may retrieve and order the files. For example, after the conference terminal determines that the first multimedia information is the text information that the user pays attention to, the conference terminal can directly save the text information as a file; for example, when the conference terminal determines that the first multimedia information is the user After the audio information is concerned, the conference terminal can convert the audio data into a meeting minutes in text form for saving.

S304: The conference terminal determines that the first multimedia information does not exist in the multimedia information of the conference.

In summary, the above S301 to S304 specifically describe how the conference terminal determines whether there is a process of the first multimedia information that the user pays attention to in the multimedia information of the conference.

S204: The conference terminal determines the user attention degree of the first multimedia information according to the first multimedia information and the first mapping relationship; the first mapping relationship is different content in the preset multimedia content. Correspondence with user attention.

Specifically, after the foregoing S203, after the conference terminal determines that the first multimedia information that is of interest to the user exists in the multimedia information of the conference, the conference terminal determines the first multimedia according to the first multimedia information and the first mapping relationship. The user relationship of the volume information, the first mapping relationship may be a correspondence between the different content in the preset multimedia content and the user attention degree. Optionally, in the first mapping relationship, the preset multimedia content is The different content may correspond to different user attention levels, and may also correspond to the same user attention for some different content in the preset multimedia content.

Optionally, the first mapping relationship may be preset in a memory inside the conference terminal, or may be preset in an external device connected to the conference terminal, where the conference terminal determines that the first multimedia information exists in the multimedia information of the conference. When you can The first mapping relationship is invoked from the external device to determine the user attention of the first multimedia information (this can slow down the cache pressure of the internal memory of the conference terminal).

S205: The conference terminal determines whether the user attention level of the first multimedia information is greater than a preset user attention threshold; if yes, execute S206; if not, execute S207.

S206: The conference terminal determines that the conference requires the user to participate.

S207: The conference terminal determines that the conference does not require the user to participate.

Specifically, after the conference terminal determines the user attention degree of the first multimedia information, determining whether the user attention degree of the first multimedia information is greater than a preset user attention threshold, when the first multimedia information When the user attention degree is greater than the preset user attention threshold, the conference terminal determines that the current conference requires the user to participate. When the user attention degree of the first multimedia information is less than or equal to the preset user attention threshold, then The conference terminal determines that the current conference does not require the user to participate.

The method for the collaboration of the smart conference provided by the embodiment of the present invention receives the multimedia information of the conference sent by the remote device through the conference terminal, and determines whether the multimedia information of the conference exists in the multimedia information of the conference according to the multimedia information of the conference and the preset multimedia content. Setting the conference information, and determining that there is no preset conference information, further determining whether the first multimedia information of the user is present in the multimedia information of the conference, and if the first multimedia information exists, Determining the user attention level of the first multimedia information and the preset user attention threshold value to determine whether the current conference requires the user to participate. The method provided by the embodiment of the present invention determines whether the current conference requires the user to participate by using the conference terminal, and determines the sending reminder information to the user according to the current participation status of the user, so that the time for the user to join the conference can be accurately controlled, so that the user does not Missing the content of the conference, thereby improving the efficiency of the user's participation; in addition, the conference terminal can not only identify the audio content of the user in the conference, but also identify the image content and text content of the user in the conference. The media information, so that users can obtain more comprehensive meeting information when they need to view the meeting content of the meeting that they are absent, and further improve the efficiency of the user's participation.

On the basis of the foregoing embodiment, optionally, before the foregoing step S104, the method may further include: if the conference terminal determines that the conference requires the user to participate, and the participation status of the user is an absent state, The conference terminal determines the form of the participation reminding information; wherein the form of the participation reminding information includes at least one of an interface form, an image form, a video form, an audio form, and an instant message form; Specifically, the conference terminal sends the participation reminding information to the user according to the form of the participant reminding information.

Optionally, the conference terminal determines the form of the participation reminding information sent to the user, and may include at least one of an interface form, an image form, a video form, an audio form, and an instant message form. For example, when the user is in the conference environment but is not involved in the discussion (such as the user is in front of the computer's display screen, although in the conference environment, but the user is doing other things, the user's participation status is also absent. Therefore, the conference terminal can remind the user through a special interface, a special color or an interface flashing; or the conference terminal can remind the user by playing a video or audio, optionally, the played video or audio can be a conference The audio data or video data that the terminal recognizes is of interest to the user, and may also be audio data or video data generated internally by the conference terminal or called by the conference terminal from the external device; When not in the conference environment (such as the user is not within the range that the camera can shoot, or the user is not in front of the computer's display), the conference terminal can remind the user to participate in the discussion by SMS, email and other instant messages. .

FIG. 6 is a schematic flowchart diagram of Embodiment 4 of a method for collating intelligent conferences according to an embodiment of the present invention. This embodiment relates to a specific process of detecting that a conference requires a user to participate in a conference and sends feedback data to the peer device of the user when the terminal replaces the user. Optionally, the method in FIG. 6 may have no timing relationship with S104 in the foregoing embodiment, and the two may be performed simultaneously. On the basis of the foregoing embodiment, further, after the foregoing S103, the method further includes:

S401: The conference terminal determines a feedback policy according to whether the conference requires the user participation and the participation status of the user; the feedback policy is used to indicate that the conference terminal sends the type of feedback data to the remote device.

S402: The conference terminal sends the feedback data to the remote device according to the feedback policy. The feedback data includes: video content that interacts with the remote device, and audio content that interacts with the remote device. At least one of textual content that interacts with the remote device.

Specifically, the type of the feedback data sent by the conference terminal to the remote device, which is indicated by the feedback policy, may include any one of the following first feedback data, second feedback data, third feedback data, and fourth feedback data. Types of. Optionally, the conference terminal may construct a decision tree according to whether the conference needs the user participation and the participation status of the user, and perform a decision tree to obtain a feedback strategy. Optionally, the feedback data may be a preset content that is preset by the user, and may be data generated according to the feedback policy and the feedback content preset by the user, where the feedback data may include: a video that interacts with the remote device. At least one of content, audio content that interacts with the remote device, and text content that interacts with the remote device. Optionally, the above feedback policy may also be used to indicate the form of the feedback data. For example, if the feedback data preset by the user is image data, but the form of the feedback data indicated in the feedback policy should be a text format, the conference terminal converts the feedback data of the image format preset by the user into a text format, and then sends the data to the text format. Remote device.

The conference terminal may determine the feedback policy according to whether the conference requires the user to participate in the conference and the participation status of the user, and may include the following four methods:

The first type: if the conference terminal determines that the conference requires the user to participate, and the participant's participation status is absent, the feedback policy determined by the conference terminal is sent to the remote device. First feedback data, the first feedback data is used to indicate to the remote device that the conference terminal is notifying the user to join the conference.

Specifically, when the conference terminal determines that the participant's participation status is absent, and the conference terminal determines that the current conference requires the user to participate, the conference terminal determines that the feedback policy is to send the first feedback data to the device of the other site (ie, the remote device). The first feedback data is used to indicate to the device of the other site that the conference terminal is notifying the user to join the conference. Optionally, the first feedback data sent by the conference terminal to the device of the other site may be sent to other sites to notify the user that the user is currently absent and is notifying the user to participate in the conference in time. The prompt information may be text, voice, or Video clip. For example, the prompt text information is superimposed on the image displayed in another meeting place, or the prompt voice is mixed in the audio stream sent to other sites, or the prompt image is superimposed on the image sent to other sites.

The second type: if the conference terminal determines that the conference requires the user to participate, and the participant's participation status is absent, the feedback policy determined by the conference terminal is sent to the remote device. The second feedback data is used to display, to the remote device, the conference content preset by the user and related to the conference.

Specifically, the conference terminal may have second feedback data that is preset by the user, and the second feedback data is used to display, to the remote device, the conference content that is preset by the user and related to the conference. . In some scenarios, the user knows in advance which topics are discussed in the conference or which opinions are published, so the user can pre-record audio or video files or pre-store relevant text information (such as shared film). If the file is used as the second feedback data, the conference terminal determines that the user's participation status is absent, and determines that the conference requires the user to participate in the discussion. The conference terminal determines that the feedback policy is to feedback the user preset to the device of the other site. Two feedback data.

The third type: if the conference terminal determines that the conference does not require the user to participate, and the participant's participation status is absent, the feedback policy determined by the conference terminal is to the remote device. Sending third feedback data; the third feedback data is used to indicate to the remote device that the user is participating in the conference.

Specifically, when the conference terminal determines that the participant's participation status is absent (that is, the user is currently in the conference environment but does not participate in the discussion or the user is not in the conference environment), and the conference terminal determines that the current conference requires the user to participate, the conference terminal The feedback policy can be determined to send third feedback data to devices of other sites. The third feedback data is used to indicate to the remote device that the user is participating. Optionally, the third feedback data may be a static user image, or a video of the user in the conference room, so that the participants of the other site can think that the user is in the conference environment and is participating in the current conference.

The fourth type: if the conference terminal determines that the conference requires the user to participate, and the participation status of the user is a user participation conflict state, the feedback policy determined by the conference terminal is to the far The terminal device sends the fourth feedback data, and records the current conference content of the conference. The fourth feedback data is used to indicate to the remote device that the participant's participation status is a user participation conflict state.

Specifically, the conference terminal can detect whether the user participates in multiple conferences. If the conflict occurs, the conference terminal determines that the user participation status is the user participation conflict status, and when the conference terminal also determines that the user is participating in the other time. If the meeting fails to participate in the current meeting, the meeting terminal determines that the feedback policy is to send the fourth feedback data to the site of the current meeting to inform the current status of the current site user, or to prompt the participants of the current site to wait for a period of time. At the same time, the conference terminal can record the current conference content of these venues (for example, the content of other participants' speeches), and the content of the speeches of other participants can be understood first when the user participates in subsequent participation. So that you can quickly integrate into the meeting. Optionally, the fourth feedback data may be in the form of audio, video, or text, which is not limited by the embodiment of the present invention.

It should be noted that after the user joins the conference, the user will take control of the conference instead of the conference terminal. At this time, the conference terminal no longer simulates the user participation, but sends the user real-time audio and video data to other participants. square. When the user leaves the conference discussion, the conference terminal can again participate in the conference instead of the user. For example, the conference terminal replaces the user to participate in three conferences, and then one of the conferences requires the user to participate, and the conference terminal sends the conference reminder information to the user to notify the user. After the conference is received, the user takes over the conference after receiving the reminder information, that is, the user himself participates in the conference. Optionally, for the video conference, after the conference terminal sends the conference reminder information, the camera is detected. When the user's real video is detected or some actual operation of the user's membership is detected (for example, the user clicks on a certain control), the conference terminal sends the third feedback data sent to the remote device (for example, the recorded user is in the conference). The video in the room) becomes a live video taken by the camera.

The collaboration method of the intelligent conference provided by the embodiment of the present invention, the conference terminal obtains the participation status of the user by detecting whether the user participates in the conference, and determines to the remote device according to the participation status of the user and whether the current conference requires the user to participate. The other site) sends the corresponding type of feedback data and determines to send the participant reminder information to the user, so that the user can join the meeting in time to interact with the participants of other sites. Therefore, the method provided by the embodiment of the present invention can avoid the user missing. The conference content related to itself improves the efficiency of the user's participation; in addition, the conference terminal can not only identify the audio content of the user in the conference, but also identify the image content and text content of the user's attention in the conference. The media information, so that users can obtain more comprehensive meeting information when they need to view the meeting content of the meeting that they are absent, so the user's participation efficiency is further improved.

FIG. 7 is a schematic flowchart of Embodiment 5 of a method for collating intelligent conferences according to an embodiment of the present invention. The embodiment relates to that the first multimedia information that is of interest to the user includes the face information of the first participant in the conference, and the associated information of the image content that the user pays attention to in the preset multimedia content is the The identity information of the first participant, and the specific process of the conference terminal obtaining the feature matching result. Based on the foregoing embodiment, the foregoing S301 specifically includes:

S501: The conference terminal detects the multimedia information of the conference, and determines a face position and a face size of the participant in the multimedia information.

Specifically, the conference terminal can detect the multimedia information of the received conference. Generally, the multimedia information of the conference can be a video image, and the conference terminal can identify all the participants in the video image in the video image. Face position and face size. The face detection performed by the conference terminal on the multimedia information of the conference is based on the pattern features of the face included in the video image, such as histogram features, color features, template features, structural features, and Haar features, ie, The conference terminal can detect these pattern features by image processing technology, and use these features to determine the face position and the face size.

S502: The conference terminal performs feature extraction on the face position and the face size of the participant in the multimedia information to obtain a face feature of the participant.

Specifically, after the conference terminal determines the face position and the face size of all the participants from the multimedia information of the conference, the conference terminal performs feature extraction according to the determined face positions and face sizes of the participants. The process of feature extraction is actually a process of character modeling a face, for example, a method that can utilize geometric features. Since the face is composed of eyes, nose, mouth, chin, etc., the geometric description of these parts and the structural relationship between them can be used as an important feature for recognizing human faces. These features are called geometric features. According to the shape description of the face organ and the distance characteristics between the two, the feature data that contributes to face classification can be obtained. The feature components usually include the Euclidean distance and curvature between the feature points. And angles, etc. According to the geometric feature method, the conference terminal can determine the facial features of all participants.

S503: The conference terminal matches each participant's face feature with a preset face information database to determine a first matching degree.

Specifically, after the conference terminal obtains the facial features of all the participants, the conference terminal can match the facial features with the preset facial information database, and determine the facial features and presets of each participant. The first matching degree of the face information of the face information database. Optionally, the preset face information database may be used by the user to load the face and other information of the user with high participation probability involved in the processor of the conference terminal by using some loading software. The preset face information database may include a plurality of face information, and each face information corresponds to identity information of one participant. For example, suppose the conference terminal determines the facial features of the four participants from the multimedia information of the conference, and then the conference terminal sets the facial features and face information of the four participants (A, B, C, and D). Each face information in the library is matched to determine the respective first matching degree.

S504: The conference terminal determines that the identification result is the identity information of the participant whose first matching degree is greater than a preset first threshold in the face information database.

Specifically, the conference terminal determines that the first matching degree of the face feature of the participant A and the face information of the A' in the face information database is greater than the first threshold, and the participant B participates. After the C and the participant D match all the face information in the face information database, the first matching degree is less than the first threshold, and the conference terminal determines that the recognition result is the face information of the A' in the face information database. The identity information of the corresponding participant.

Optionally, the execution body of the foregoing S503 and S504 may also be a network device that can communicate with the conference terminal, such as a network server, that is, the terminal sends the facial features of each participant to the network device, and the network device The face feature of each participant is matched with the preset face information database on the network device to obtain the first matching degree, and the final recognition result is determined to be that the first matching degree in the face information database is greater than the preset number. The identity information of the participant of the threshold is sent to the conference terminal by the identity information of the participant whose first matching degree is greater than the preset first threshold in the face information database.

S505: The conference terminal compares the identity information of the participant whose first matching degree is greater than a preset first threshold in the face information database with the identity of the first participant in the preset multimedia content. The information is matched to obtain the feature matching result.

Specifically, the conference terminal continues to determine whether the identity information of the participant corresponding to the face information of the A′ matches the identity information of the first participant in the preset multimedia content, that is, the foregoing A′ is determined. Whether the identity information of the participant corresponding to the face information is the same as or similar to the identity information of the first participant, thereby obtaining the feature matching result, and determining whether the current meeting requires the user to participate according to the feature matching result. It should be noted that, the conference terminal determines whether the current conference needs the user to participate according to the feature matching result. For details, refer to the description in the foregoing Embodiment 2 and Embodiment 3, and details are not described herein again.

FIG. 8 is a schematic flowchart of Embodiment 6 of a method for collating intelligent conferences according to an embodiment of the present invention. On the basis of the foregoing embodiment, the embodiment relates to that when the first multimedia information that the user pays attention includes the first text information in the conference, and the text content that the user pays attention to is the first text information, the conference The specific process in which the terminal obtains the feature matching result. Based on the foregoing embodiment, the foregoing S301 specifically includes:

S601: The conference terminal detects the multimedia information of the conference, and determines a location and a size of the text block area in the multimedia information of the conference.

S602: The conference terminal obtains a text block in the multimedia information of the conference according to the location and size of the text block area in the multimedia information of the conference.

Specifically, the conference terminal may detect the multimedia information of the received conference. Generally, the multimedia information of the conference may be a video image, and the conference terminal may detect, in the video image, all the text block regions existing in the video image. The location and size, which in turn determines all the text blocks in the multimedia information of the conference. Optionally, the conference terminal may perform the detection of the location and size of the text block region by using a texture-based method, or an edge detection-based method or a connected domain-based method. For example, the texture-based method mainly obtains the position and size of the text block in the video image based on the difference between the background and the character texture characteristics, thereby obtaining all the text blocks in the multimedia information of the conference.

S603: The conference terminal matches the text block in the multimedia information of the conference with a preset text information library, and determines a second matching degree.

S604: The conference terminal determines that the recognition result is text information that the second matching degree in the text information library is greater than a preset second threshold.

Specifically, the conference terminal may perform matching between the texts in all the text blocks in the multimedia information of the determined conference and the preset text information library by using a template matching method, and calculate text and preset text information in the text block. A second matching degree between each piece of text information in the library, and the text information in the text information library with the second matching degree of the text in the text block being greater than the preset second threshold is used as the recognition result. It should be noted that the text information in the text information library is text information recognizable by the conference terminal (for example, a character string recognizable by the conference terminal), but the text in the text block is not recognized by the conference terminal, and therefore, the conference The terminal needs to find the text information having the same or similar meaning to the text of the text block (that is, the text information whose second matching degree is greater than the second threshold) from the text information library by using the template matching method, and use the same as the recognition result.

S605: The conference terminal matches the text information in the text information library that is greater than the preset second threshold with the first text information in the preset multimedia content, to obtain the feature. Match the result.

Specifically, the conference terminal matches the text information in the text information database with the second matching degree greater than the preset second threshold value and the first text information in the preset multimedia content, that is, determining the text information library. Whether the second matching degree is greater than the preset second threshold value and the first text information in the preset multimedia content is the same or similar, thereby determining the feature matching result, and determining whether the current meeting needs according to the feature matching result. User participation. It should be noted that, the conference terminal determines whether the current conference needs the user to participate according to the feature matching result. For details, refer to the detailed description of the foregoing embodiment 2 and the third embodiment, and details are not described herein again.

FIG. 9 is a schematic flowchart of Embodiment 7 of a method for collating intelligent conferences according to an embodiment of the present invention. On the basis of the foregoing embodiment, the embodiment relates to that when the first multimedia information that the user pays attention includes the first text information in the conference, and the text content that the user pays attention to is the first text information, the conference The terminal obtains another specific process of feature matching results. Based on the foregoing embodiment, the foregoing S301 specifically includes:

S701: The conference terminal detects the multimedia information of the conference, determines the location and size of the text block area in the multimedia information of the conference, and obtains a text block in the multimedia information of the conference.

For details, refer to the description process of S601 above, and details are not described herein again.

S702: The conference terminal determines the recognition result according to the geometric feature of the text block.

Specifically, the conference terminal may extract some geometric features of the text in the text block by using a geometric feature extraction method, such as end points of the text, bifurcation points, concave and convex portions, and line segments in various directions such as horizontal, vertical, and tilt, closed loop, and the like. According to the position and mutual relationship of these features, logical combination judgment is made to determine the text information recognizable by the conference terminal as the recognition result.

S703: The conference terminal matches the identification result with the first text information in the preset multimedia content, to obtain the feature matching result.

Specifically, after the conference terminal converts the text of the text block into the text information recognizable by the conference terminal by using the geometric feature extraction method, the conference terminal matches the text information with the first text information in the preset multimedia content, that is, Determining whether the text information is identical or similar to the first text information in the preset multimedia content, thereby determining a feature matching result, and determining, according to the feature matching result, whether the current meeting requires the user to participate. It should be noted that, the conference terminal determines whether the current conference needs the user to participate according to the feature matching result. For details, refer to the description in the foregoing Embodiment 2 and Embodiment 3, and details are not described herein again.

FIG. 10 is a schematic flowchart diagram of Embodiment 8 of a method for collating intelligent conferences according to an embodiment of the present invention. On the basis of the foregoing embodiment, the embodiment relates to: when the first multimedia information that is of interest to the user includes the multimedia information of the conference, except for the text type, the participant face type, and the first When the text information is related to the first data information, the conference terminal obtains another specific process of the feature matching result. Based on the foregoing embodiment, the foregoing S301 specifically includes:

S801: The conference terminal detects the multimedia information of the conference, and determines other data information in the multimedia information of the conference, except the text type and the participant face type.

Specifically, the method in this embodiment is that the first multimedia information that the user pays attention to is no longer the text information or the face information of the participant, but the multimedia information of the conference is other than the above implementation. The first data information related to the first text information in the seventh example. Optionally, when the first text information is description information of a certain image, the first data information is actually related content of the image described by the first text information, for example, when the first text information describes When the color of a picture is used, the first data information may be a color depth or brightness of the picture; when the first text information is a description of a GUI interface, the first data information may be the color of the GUI interface. Attribute information such as brightness or size.

S802: The conference terminal determines, according to the first text information and the other data information, a degree of relevance of the other data information to the first text information.

S803: The conference terminal determines that the recognition result is that the correlation degree in the other data information is greater than a preset third threshold. Value data information.

Specifically, when the conference terminal determines, from the multimedia information of the conference, other data information except the text type and the participant face type, determining whether the other data information is related to the first text information. And determining the relevance thereof, and using the data information of the other data information that the correlation is greater than the preset third threshold as the recognition result. For example, when the first data information is the brightness of the first text information (assumed to be a piece of art word) that the user pays attention to, if the conference terminal determines from the multimedia information of the conference, other than the text type and the participant face type. If the data information is the brightness or color of the word of art, if the degree of correlation between the two determined by the conference terminal is greater than the third threshold, the conference terminal uses the data information as the recognition result.

S804: The conference terminal matches the data information of the other data information that is greater than a preset third threshold with the first data information, to obtain the feature matching result.

The collaboration method of the intelligent conference provided by the embodiment of the present invention, the conference terminal obtains the participation status of the user by detecting whether the user participates in the conference, and determines to the remote device according to the participation status of the user and whether the current conference requires the user to participate. The other site) sends the corresponding type of feedback data and determines to send the participant reminder information to the user, so that the user can join the meeting in time to interact with the participants of other sites. Therefore, the method provided by the embodiment of the present invention can avoid the user missing. The conference content related to itself improves the efficiency of the user's participation; in addition, the conference terminal can not only identify the audio content of the user in the conference, but also identify the image content and text content of the user's attention in the conference. The media information, so that users can obtain more comprehensive meeting information when they need to view the meeting content of the meeting that they are absent, so the user's participation efficiency is further improved.

One of ordinary skill in the art will appreciate that all or part of the steps to implement the various method embodiments described above may be accomplished by hardware associated with the program instructions. The aforementioned program can be stored in a computer readable storage medium. The program, when executed, performs the steps including the foregoing method embodiments; and the foregoing storage medium includes various media that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

FIG. 11 is a schematic structural diagram of Embodiment 1 of a conference terminal according to an embodiment of the present invention. As shown in FIG. 11 , the conference terminal may include a receiving module 10, a first determining module 11, a second determining module 12, and a sending module 13.

The receiving module 10 is configured to receive multimedia information of the conference sent by the remote device, where the multimedia information of the conference includes at least one of voice information, image information, and text information. The terminal of the conference;

The first determining module 11 is configured to determine, according to the multimedia information of the conference and the preset multimedia content, whether the conference needs the user to participate; wherein the preset multimedia content includes: the voice that the user pays attention to At least one of content, image content of the user's attention, association information of the image content of interest of the user, and text content of the user's attention;

a second determining module 12, configured to determine a participation status of the user; the participation status of the user includes an absent state or a user participation conflict state;

The sending module 13 is configured to determine, at the first determining module 11, that the conference requires the user to participate, and the The determining module 12 determines that the participation status of the user is an absent state, and sends the meeting reminding information to the user; the participating reminding information is used to prompt the user to participate in the meeting.

The conference terminal provided by the embodiment of the present invention may perform the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.

FIG. 12 is a schematic structural diagram of Embodiment 2 of a conference terminal according to an embodiment of the present disclosure. On the basis of the foregoing embodiment shown in FIG. 11, further, as shown in FIG. 12, the first determining module 11 includes: a first determining unit 111, configured to use the multimedia information of the conference and the pre- Determining whether there is preset conference information in the multimedia information of the conference, and if the preset conference information exists in the multimedia information of the conference, determining that the conference requires the user to participate; The preset conference information is information for characterizing that the user is required to participate in the conference.

Further, the first determining module 11 further includes: a second determining unit 112 and a determining unit 113. The second determining unit 112 is configured to determine whether the multimedia information of the conference exists in the multimedia information of the conference when the first determining unit 111 determines that the preset conference information does not exist in the multimedia information of the conference. Determining the first multimedia information that is of interest to the user, and determining that the first multimedia information that is of interest to the user exists in the multimedia information of the conference, determining, according to the first multimedia information and the first mapping relationship a user attention degree of the first multimedia information; the first mapping relationship is a correspondence between different content in the preset multimedia content and a user attention degree; the determining unit 113 is configured to determine the Whether the user attention of the first multimedia information is greater than a preset user attention threshold; if yes, determining that the conference requires the user to participate; if not, determining that the conference does not require the user to participate.

Further, the second determining unit 112 is specifically configured to detect and identify the multimedia information of the conference, obtain a recognition result, and perform feature matching on the recognition result and the preset multimedia content to obtain a feature. Matching the result, and determining whether the feature matching result is greater than a preset matching threshold; if yes, determining that the first multimedia information exists in the multimedia information of the conference; if not, determining the multimedia information of the conference The first multimedia information does not exist in the middle.

The conference terminal provided by the embodiment of the present invention may perform the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.

FIG. 13 is a schematic structural diagram of Embodiment 3 of a conference terminal according to an embodiment of the present disclosure. On the basis of the foregoing embodiment shown in FIG. 11 or FIG. 12, further, as shown in FIG. 13, the conference terminal further includes: a third determining module 14 configured to determine, at the first determining module 11, The conference needs the user to participate, and the second determining module 12 determines the form of the participation reminding information when the participation status of the user is an absent state; wherein the form of the participation reminding information includes an interface. At least one of a form, an image form, a video form, an audio form, and an instant message form;

The sending module 13 is specifically configured to send the participation reminding information to the user according to the form of the participation reminding information.

It should be noted that the indication shown in FIG. 13 is based on the structure of the embodiment shown in FIG. 12. Of course, FIG. 13 can also be based on the embodiment shown in FIG. 11, and the present invention only shows one of them.

FIG. 14 is a schematic structural diagram of Embodiment 4 of a conference terminal according to an embodiment of the present disclosure. In the above Figure 11 or Figure 12 or Figure On the basis of the embodiment shown in FIG. 13, further, as shown in FIG. 14, the conference terminal further includes: a fourth determining module 15 configured to: according to whether the conference requires the user to participate in the conference and the user's participation a state determination feedback policy; the feedback policy is used to indicate that the conference terminal sends the type of feedback data to the remote device;

The sending module 13 is further configured to send the feedback data to the remote device according to the feedback policy; the feedback data includes: video content that interacts with the remote device, and the remote end At least one of audio content that the device interacts with, and text content that interacts with the remote device. The feedback data is feedback content preset by the user, or data generated according to the feedback policy and the feedback content preset by the user.

Optionally, the fourth determining module 15 is configured to determine that the feedback policy is to the remote device, if the user is required to participate in the conference, and the participant is in an absent state. Sending first feedback data, the first feedback data is used to indicate to the remote device that the conference terminal is notifying the user to join the conference.

Optionally, the fourth determining module 15 is configured to determine that the feedback policy is to the remote device, if the user is required to participate in the conference, and the participant is in an absent state. Sending second feedback data; the second feedback data is used to show, to the remote device, the conference content that is preset by the user and related to the conference.

Optionally, the fourth determining module 15 is configured to: if the conference does not require the user to participate, and the participant's participation status is absent, determine that the feedback policy is to the remote end. The device sends third feedback data, where the third feedback data is used to indicate to the remote device that the user is participating.

Optionally, the fourth determining module 15 is specifically configured to: if the user needs to participate in the conference, and the participation status of the user is a user participation conflict state, determine that the feedback policy is The remote device sends the fourth feedback data, and records the current conference content of the conference. The fourth feedback data is used to indicate to the remote device that the participant's participation status is a user participation conflict state.

It should be noted that the indication shown in FIG. 14 is based on the structure of the embodiment shown in FIG. 13. Of course, FIG. 14 can also be based on the embodiment shown in FIG. 11 or FIG. 12, and the present invention only shows one of them. .

The conference terminal provided by the embodiment of the present invention may perform the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.

FIG. 15 is a schematic structural diagram of Embodiment 5 of a conference terminal according to an embodiment of the present disclosure. On the basis of the foregoing embodiment shown in FIG. 12 or FIG. 13 or FIG. 14 , further, as shown in FIG. 15 , if the first multimedia information that the user pays attention to includes the first participant in the conference The second information determining unit 112 includes: the first detecting subunit, the second information determining unit 112, wherein the second determining unit 112 includes: 1121, a feature extraction sub-unit 1122, a first matching sub-unit 1123, a first determining sub-unit 1124, and a second matching sub-unit 1125.

The first detecting sub-unit 1121 is configured to detect multimedia information of the conference, and determine a face position and a face size of the participant in the multimedia information.

a feature extraction sub-unit 1122, configured to perform feature extraction on a face position and a face size of the participant in the multimedia information, to obtain a face feature of the participant;

a first matching sub-unit 1123, configured to match a face feature of each participant with a preset face information database to determine a first matching degree;

a first determining subunit 1124, configured to determine that the identification result is identity information of the participant whose first matching degree is greater than a preset first threshold in the face information database;

a second matching sub-unit 1125, configured to: use the identity information of the participant whose first matching degree is greater than a preset first threshold in the face information database, and the first one of the preset multimedia content The identity information of the participants is matched, and the feature matching result is obtained.

It should be noted that the indication shown in FIG. 15 is based on the structure of the embodiment shown in FIG. 14. Of course, FIG. 15 can also be based on the embodiment shown in FIG. 12 or FIG. 13, and the present invention only shows one of them. .

The conference terminal provided by the embodiment of the present invention may perform the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.

FIG. 16 is a schematic structural diagram of Embodiment 6 of a conference terminal according to an embodiment of the present disclosure. On the basis of the embodiment shown in FIG. 12 or FIG. 13 or FIG. 14 , further, as shown in FIG. 16 , if the first multimedia information that the user pays attention to includes the first text information, the user pays attention to The text content is the first text information, and the second determining unit 112 specifically includes: a second detecting subunit 1126, an obtaining subunit 1127, a third matching subunit 1128, a second determining subunit 1129, and a fourth Matching subunit 1130.

The second detecting sub-unit 1126 is configured to detect multimedia information of the conference, and determine a location and a size of a text block area in the multimedia information of the conference;

The obtaining subunit 1127 is configured to obtain a text block in the multimedia information of the conference according to the location and size of the text block area in the multimedia information of the conference;

a third matching sub-unit 1128, configured to match a text block in the multimedia information of the conference with a preset text information library, and determine a second matching degree;

a second determining subunit 1129, configured to determine that the recognition result is text information that the second matching degree in the text information library is greater than a preset second threshold;

The fourth matching sub-unit 1130 is configured to match the text information in the text information library with the second matching degree greater than a preset second threshold value to the first text information in the preset multimedia content. , obtaining the feature matching result.

It should be noted that the indication shown in FIG. 16 is based on the structure of the embodiment shown in FIG. 14. Of course, FIG. 16 can also be based on the embodiment shown in FIG. 12 or FIG. 13, and the present invention only shows one of them. .

The conference terminal provided by the embodiment of the present invention may perform the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.

FIG. 17 is a schematic structural diagram of Embodiment 7 of a conference terminal according to an embodiment of the present disclosure. On the basis of the embodiment shown in FIG. 12 or FIG. 13 or FIG. 14 , further, as shown in FIG. 17 , if the first multimedia information that the user pays attention to includes the first text information, the user pays attention to The text content is the first text information, and the second determining unit 112 specifically includes: a third detecting subunit 1131, a third determining subunit 1132, and a fifth matching subunit 1133.

The third detecting sub-unit 1131 is configured to detect multimedia information of the conference, determine a location and a size of a text block area in the multimedia information of the conference, and obtain a text block in the multimedia information of the conference;

a third determining subunit 1132, configured to determine the recognition result according to a geometric feature of the text block;

The fifth matching sub-unit 1133 is configured to match the identification result with the first text information in the preset multimedia content to obtain the feature matching result.

It should be noted that the indication shown in FIG. 17 is based on the structure of the embodiment shown in FIG. 14. Of course, FIG. 17 can also be based on the embodiment shown in FIG. 12 or FIG. 13, but the present invention only shows one of them. .

The conference terminal provided by the embodiment of the present invention may perform the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.

FIG. 18 is a schematic structural diagram of Embodiment 8 of a conference terminal according to an embodiment of the present disclosure. On the basis of the foregoing embodiments shown in FIG. 12 to FIG. 17, further, as shown in FIG. 18, if the first multimedia information includes multimedia information of the conference, except for the text type and the participant. The second determining subunit 1129 includes a fourth detecting subunit 1134, a fourth determining subunit 1135, and a fifth determining subroutine, in addition to the first data information related to the first text information other than the face type. Unit 1136 and sixth matching subunit 1137.

The fourth detecting sub-unit 1134 is configured to detect the multimedia information of the conference, and determine other data information in the multimedia information of the conference, except the text type and the participant face type;

a fourth determining subunit 1135, configured to determine, according to the first text information and the other data information, a degree of correlation between the other data information and the first text information;

a fifth determining subunit 1136, configured to determine that the identification result is data information that the correlation is greater than a preset third threshold in the other data information;

The sixth matching sub-unit 1137 is configured to match, in the other data information, the data information whose correlation is greater than a preset third threshold with the first data information, to obtain the feature matching result.

It should be noted that the indication shown in FIG. 18 is based on the structure of the embodiment shown in FIG. 14. Of course, FIG. 18 can also be based on the embodiment shown in any of FIG. 12 or FIG. 13 or FIG. 16 or FIG. Only one of them is shown.

The conference terminal provided by the embodiment of the present invention may perform the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.

FIG. 19 is a schematic structural diagram of Embodiment 9 of a conference terminal according to an embodiment of the present disclosure. As shown in FIG. 19, the conference terminal may include a processor 20, such as a CPU; a memory 21, at least one communication bus 22. Communication bus 22 is used to implement a communication connection between components. The memory 21 may include a high speed RAM memory, and may also include a non-volatile memory NVM, such as at least one disk memory, in which various programs may be stored for performing various processing functions and implementing the method steps of the present embodiment.

In the embodiment of the present invention, the conference terminal further includes: a receiver 23 and a transmitter 24.

The receiver 23 is configured to receive multimedia information of a conference sent by the remote device, where the multimedia information of the conference includes at least one of voice information, image information, and text information, where the conference terminal is Join the said Terminal of the conference;

The processor 20 is configured to determine, according to the multimedia information of the conference and the preset multimedia content, whether the conference needs the user to participate, and determine a participation status of the user; where the preset The multimedia content includes: at least one of the voice content of the user's attention, the image content of the user's attention, the associated information of the image content of the user's attention, and the text content of the user's attention, the user's participation The status includes an absent state or a user participation conflict state;

The transmitter 24 is configured to send the conference reminder information to the user when the processor 20 determines that the conference requires the user to participate, and the participant's participation status is an absent state; The reminder information is used to prompt the user to participate in the conference.

The conference terminal provided by the embodiment of the present invention may perform the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.

Further, the processor 20 is specifically configured to determine, according to the multimedia information of the conference and the preset multimedia content, whether preset conference information exists in the multimedia information of the conference; if the conference multimedia If the preset conference information exists in the information, it is determined that the conference needs the user to participate; and the preset conference information is information used to represent that the user needs to participate in the conference.

Further, the processor 20 is further configured to: when the preset conference information does not exist in the multimedia information of the conference, determine whether the first multimedia that is of interest to the user exists in the multimedia information of the conference. And determining the user attention degree of the first multimedia information according to the first multimedia information and the first mapping relationship, when the first multimedia information exists in the multimedia information of the conference, And determining whether the user attention of the first multimedia information is greater than a preset user attention threshold; if yes, determining that the conference requires the user to participate; if not, determining that the conference does not require the user Participating; wherein the first mapping relationship is a correspondence between different content in the preset multimedia content and user attention.

Further, the processor 20 is specifically configured to detect and identify the multimedia information of the conference, obtain a recognition result, and perform feature matching on the recognition result and the preset multimedia content to obtain a feature matching result. And determining whether the feature matching result is greater than a preset matching threshold; if yes, determining that the first multimedia information exists in the multimedia information of the conference; if not, determining that the multimedia information of the conference is not There is the first multimedia information.

The conference terminal provided by the embodiment of the present invention may perform the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.

Optionally, the processor 20 is further configured to determine a form of the participation reminding information when the conference requires the user to participate, and the participant's participation status is an absent state; The form of the participation reminder information includes at least one of an interface form, an image form, a video form, an audio form, and an instant message form; and the transmitter 24 is specifically configured to: according to the form of the participation reminding information The user sends the participation reminding information.

Optionally, the processor 20 is further configured to: determine, according to whether the conference requires the user to participate in the conference and the participant's participation status, the feedback policy is used to indicate that the conference terminal is located The number of feedback sent by the remote device The sender 24 is further configured to send the feedback data to the remote device according to the feedback policy; the feedback data includes: video content that interacts with the remote device, and The remote device performs at least one of audio content that interacts with text content that interacts with the remote device. The feedback data is feedback content preset by the user, or data generated according to the feedback policy and the feedback content preset by the user.

Optionally, the processor 20 is configured to: when the conference requires the user to participate, and the participant's participation status is absent, determine that the feedback policy is to send the first to the remote device. a feedback data, the first feedback data is used to indicate to the remote device that the conference terminal is notifying the user to join the conference.

Optionally, the processor 20 is configured to: when the conference requires the user to participate, and the participant's participation status is absent, determine that the feedback policy is to send the first to the remote device. The second feedback data is used to display, to the remote device, the conference content preset by the user and related to the conference.

Optionally, the processor 20 is configured to: when the conference does not require the user to participate, and the participant's participation status is absent, determine that the feedback policy is sent to the remote device. The third feedback data is used to indicate to the remote device that the user is participating in the conference.

Optionally, the processor 20 is configured to determine that the feedback policy is to the remote end when the user needs to participate in the conference, and the participant's participation status is a user participation conflict state. The device sends the fourth feedback data, and records the current conference content of the conference. The fourth feedback data is used to indicate to the remote device that the participant's participation status is a user participation conflict state.

The conference terminal provided by the embodiment of the present invention may perform the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.

As a possible implementation manner of the embodiment of the present invention, optionally, if the first multimedia information that is of interest to the user includes the face information of the first participant in the conference, the preset multimedia The associated information of the image content that is of interest to the user in the content is the identity information of the first participant; the processor 20 is specifically configured to detect the multimedia information of the conference, and determine the participation of the participant. The face position and the face size in the multimedia information, and the feature extraction of the face position and the face size of the participant in the multimedia information to obtain the face of the participant Feature, matching each participant's face feature with a preset face information database, determining a first matching degree, and determining the recognition result as the first match in the face information database The identity information of the participant whose degree is greater than the preset first threshold, and the identity information of the participant whose first matching degree is greater than the preset first threshold in the face information database and the pre-predetermined The first participant in the multimedia content Matching identity information, wherein said matching result is obtained.

As another possible implementation manner of the embodiment of the present invention, optionally, if the first multimedia information that is of interest to the user includes the first text information, the text content that the user pays attention to is the first text information. The processor 20 is configured to detect multimedia information of the conference, determine a location and a size of a text block area in the multimedia information of the conference, and perform multimedia in the conference according to the text block area. Position and size in the information, obtain a text block in the multimedia information of the conference, and match a text block in the multimedia information of the conference with a preset text information library to determine a second matching degree, thereby determining The recognition result is that the second matching degree in the text information library is greater than a pre- a text information of the second threshold, and the second matching degree in the text information library is greater than a preset second threshold text information and the first text information in the preset multimedia content Matching, obtaining the feature matching result.

As a third possible implementation manner of the embodiment of the present invention, optionally, if the first multimedia information that is of interest to the user includes the first text information, the text content that the user pays attention to is the first text information. The processor 20 is configured to detect multimedia information of the conference, determine a location and a size of a text block area in the multimedia information of the conference, and obtain a text block in the multimedia information of the conference. And determining the recognition result according to the geometric feature of the text block, and matching the recognition result with the first text information in the preset multimedia content to obtain the feature matching result.

As a fourth possible implementation manner of the embodiment of the present invention, further, based on the foregoing third possible implementation manner or the fourth possible implementation manner, if the first multimedia information includes the The processor 20 is configured to perform multimedia information on the conference, in the multimedia information of the conference, in addition to the text type and the participant face type, the first data information related to the first text information. Detecting, determining data information other than the text type and the participant face type in the multimedia information of the conference, and determining the other data according to the first text information and the other data information Correlation of the information with the first text information, and determining that the recognition result is data information of the other data information that the correlation is greater than a preset third threshold, and then the other data information is The data information whose correlation is greater than the preset third threshold matches the first data information, and the feature matching result is obtained.

The conference terminal provided by the embodiment of the present invention may perform the foregoing method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.

Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that The technical solutions described in the foregoing embodiments may be modified, or some or all of the technical features may be equivalently replaced; and the modifications or substitutions do not deviate from the technical solutions of the embodiments of the present invention. range.

Claims (30)

A collaborative method for intelligent meeting, characterized in that it comprises: The conference terminal receives the multimedia information of the conference sent by the remote device; the multimedia information of the conference includes at least one of voice information, image information, and text information; wherein the conference terminal is a terminal that has joined the conference; Determining, by the conference terminal, whether the conference requires the user to participate according to the multimedia information of the conference and the preset multimedia content, where the preset multimedia content includes: the voice content that the user pays attention to, the At least one of image content of interest of the user, associated information of the image content of interest of the user, and text content of interest of the user; Determining, by the conference terminal, a participation status of the user; the participation status of the user includes an absent state or a user participation conflict state; If the conference terminal determines that the conference requires the user to participate, and the participant's participation status is absent, the conference terminal sends the conference reminder information to the user; the conference reminder information is used for The user is prompted to participate in the meeting. The method according to claim 1, wherein the conference terminal determines whether the conference requires the user to participate according to the multimedia information of the conference and the preset multimedia content, including: Determining, by the conference terminal, whether preset conference information exists in the multimedia information of the conference according to the multimedia information of the conference and the preset multimedia content; the preset conference information is used for characterization Information about the user participating in the meeting; If the preset conference information exists in the multimedia information of the conference, the conference terminal determines that the conference requires the user to participate. The method according to claim 2, wherein the conference terminal determines, according to the multimedia information of the conference and the preset multimedia content, whether preset conference information exists in the multimedia information of the conference, and include: If the conference terminal determines that the preset conference information does not exist in the multimedia information of the conference, the conference terminal determines whether the first multimedia information that is of interest to the user exists in the multimedia information of the conference; When the first multimedia information exists in the multimedia information of the conference, the conference terminal determines the user attention of the first multimedia information according to the first multimedia information and the first mapping relationship. The first mapping relationship is a correspondence between different content in the preset multimedia content and user attention degree; The conference terminal determines whether the user attention of the first multimedia information is greater than a preset user attention threshold; if yes, the conference terminal determines that the conference requires the user to participate; if not, the conference The conference terminal determines that the conference does not require the user to participate. The method according to claim 3, wherein the conference terminal determines, according to the multimedia information of the conference and the preset multimedia content, whether the first information of the user is present in the multimedia information of the conference. Multimedia information, including: The conference terminal detects and identifies the multimedia information of the conference, obtains a recognition result, and performs feature matching on the recognition result and the preset multimedia content to obtain a feature matching result; Determining, by the conference terminal, whether the feature matching result is greater than a preset matching threshold; If yes, the conference terminal determines that the first multimedia information exists in the multimedia information of the conference; If not, the conference terminal determines that the first multimedia information does not exist in the multimedia information of the conference. The method according to any one of claims 1 to 4, wherein the method further comprises: If the conference terminal determines that the conference requires the user to participate, and the participant's participation status is absent, the conference terminal determines the form of the conference reminder information; wherein the conference reminder information The form includes at least one of an interface form, an image form, a video form, an audio form, and an instant message form; The conference terminal sends the conference reminder information to the user, including: The conference terminal sends the participation reminding information to the user according to the form of the participation reminding information. The method according to any one of claims 1 to 5, wherein the method further comprises: Determining, by the conference terminal, a feedback policy according to whether the conference requires the user participation and the participation status of the user; the feedback policy is used to indicate that the conference terminal sends the type of feedback data to the remote device; The conference terminal sends the feedback data to the remote device according to the feedback policy; the feedback data includes: video content that interacts with the remote device, and audio content that interacts with the remote device At least one of textual content that interacts with the remote device. The method according to claim 6, wherein the conference terminal determines a feedback policy according to whether the conference requires the user to participate in the conference and the participation status of the user, including: If the conference terminal determines that the conference requires the user to participate, and the participant's participation status is absent, the feedback policy determined by the conference terminal is to send the first feedback data to the remote device. The first feedback data is used to indicate to the remote device that the conference terminal is notifying the user to join the conference. The method according to claim 6, wherein the conference terminal determines a feedback policy according to whether the conference requires the user to participate in the conference and the participation status of the user, including: If the conference terminal determines that the conference requires the user to participate, and the participant's participation status is absent, the feedback policy determined by the conference terminal is to send second feedback data to the remote device. The second feedback data is used to show the remote device the conference content related to the conference preset by the user. The method according to claim 6, wherein the conference terminal determines a feedback policy according to whether the conference requires the user to participate in the conference and the participation status of the user, including: If the conference terminal determines that the conference does not require the user to participate, and the participant's participation status is absent, the feedback policy determined by the conference terminal is to send a third feedback to the remote device. Data; the third feedback data is used to indicate to the remote device that the user is participating. The method according to claim 6, wherein the conference terminal determines a feedback policy according to whether the conference requires the user to participate in the conference and the participation status of the user, including: If the conference terminal determines that the conference requires the user to participate, and the participant's participation status is a user participation conflict state, the feedback policy determined by the conference terminal is to send the first to the remote device. The fourth feedback data is used to record the current conference content of the conference; the fourth feedback data is used to indicate to the remote device that the participant's participation status is a user participation conflict state. The method according to any one of claims 3 to 10, wherein if the first multimedia information that is of interest to the user includes face information of the first participant in the conference, the preset The association information of the image content that the user pays attention to in the multimedia content is the identity information of the first participant; the conference terminal detects and identifies the multimedia information of the conference, obtains a recognition result, and identifies the identifier The result is matched with the preset multimedia content to obtain a feature matching result, which specifically includes: The conference terminal detects multimedia information of the conference, and determines a face position and a face size of the participant in the multimedia information; The conference terminal performs feature extraction on the face position and the face size of the participant in the multimedia information to obtain a face feature of the participant; The conference terminal matches each participant's face feature with a preset face information database to determine a first matching degree; Determining, by the conference terminal, that the identification result is identity information of the participant whose first matching degree is greater than a preset first threshold in the face information database; The conference terminal sets the identity information of the participant whose first matching degree is greater than the preset first threshold in the face information database and the identity of the first participant in the preset multimedia content. The information is matched to obtain the feature matching result. The method according to any one of claims 3 to 10, wherein if the first multimedia information that the user is interested in comprises first text information, the text content that the user pays attention to is the first text information. And the conference terminal detects and identifies the multimedia information of the conference, obtains a recognition result, and performs feature matching on the identifier and the preset multimedia content to obtain a feature matching result, which specifically includes: The conference terminal detects multimedia information of the conference, and determines a location and a size of a text block area in the multimedia information of the conference; And obtaining, by the conference terminal, a text block in the multimedia information of the conference according to a location and a size of the text block area in the multimedia information of the conference; The conference terminal matches a text block in the multimedia information of the conference with a preset text information library, and determines a second matching degree; The conference terminal determines that the recognition result is text information that the second matching degree in the text information library is greater than a preset second threshold; The conference terminal matches the text information in the text information library with the second matching degree greater than the preset second threshold value and the first text information in the preset multimedia content, to obtain the feature. Match the result. The method according to any one of claims 3 to 10, wherein if the first multimedia information that the user is interested in comprises first text information, the text content that the user pays attention to is the first text information. , the conference terminal pair The multimedia information of the conference is detected and identified, and the recognition result is obtained, and the matching result is matched with the preset multimedia content to obtain a feature matching result, which specifically includes: The conference terminal detects the multimedia information of the conference, determines the location and size of the text block area in the multimedia information of the conference, and obtains a text block in the multimedia information of the conference; Determining, by the conference terminal, the recognition result according to a geometric feature of the text block; The conference terminal matches the identification result with the first text information in the preset multimedia content to obtain the feature matching result. The method according to claim 12 or 13, wherein the first multimedia information comprises, in addition to the text type, the participant face type, and the first The first data information related to the text information, the conference terminal detects and identifies the multimedia information of the conference, obtains a recognition result, and performs feature matching on the recognition result and the preset multimedia content to obtain feature matching. The results also include: The conference terminal detects the multimedia information of the conference, and determines other data information in the multimedia information of the conference, except the text type and the participant face type; Determining, by the conference terminal, the degree of relevance of the other data information and the first text information according to the first text information and the other data information; Determining, by the conference terminal, that the identification result is data information that the correlation is greater than a preset third threshold in the other data information; And the conference terminal matches the data information of the other data information that is greater than a preset third threshold with the first data information to obtain the feature matching result. The method according to any one of claims 6 to 14, wherein the feedback data is feedback content preset by the user, or generated according to the feedback policy and the feedback content preset by the user. data. A conference terminal, comprising: a receiving module, configured to receive multimedia information of a conference sent by the remote device, where the multimedia information of the conference includes at least one of voice information, image information, and text information, where the conference terminal is added to the conference terminal; a first determining module, configured to determine, according to the multimedia information of the conference and the preset multimedia content, whether the conference needs the user to participate; wherein the preset multimedia content includes: the voice content that the user pays attention to At least one of image content of the user's attention, association information of the image content of interest of the user, and text content of the user's attention; a second determining module, configured to determine a participating state of the user; the participating state of the user includes an absent state or a user participating conflict state; a sending module, configured to send a reminder to the user when the first determining module determines that the conference requires the user to participate, and the second determining module determines that the participating state of the user is an absent state Information; the participation reminding information is used to prompt the user to participate in the meeting. The conference terminal according to claim 16, wherein the first determining module comprises: a first determining unit, configured to determine, according to the multimedia information of the conference and the preset multimedia content, whether preset conference information exists in the multimedia information of the conference, if the multimedia information of the conference exists When the preset conference information is determined, it is determined that the conference needs the user to participate; and the preset conference information is information used to represent that the user needs to participate in the conference. The conference terminal according to claim 17, wherein the first determining module further comprises: a second determining unit, configured to determine, when the first determining unit determines that the preset meeting information does not exist in the multimedia information of the conference, whether the first information of the user is present in the multimedia information of the conference Determining, by the first multimedia information, the first multimedia information according to the first multimedia information and the first mapping relationship, when determining that the first multimedia information that is of interest to the user exists in the multimedia information of the conference User attention of the body information; the first mapping relationship is a correspondence between different content in the preset multimedia content and user attention; a determining unit, configured to determine whether the user attention level of the first multimedia information is greater than a preset user attention degree threshold; if yes, determining that the meeting requires the user to participate; if not, determining that the meeting is not The user is required to participate. The conference terminal according to claim 18, wherein the second determining unit is configured to detect and identify the multimedia information of the conference, obtain a recognition result, and use the recognition result and the preset Performing feature matching on the multimedia content, obtaining a feature matching result, and determining whether the feature matching result is greater than a preset matching threshold; if yes, determining that the first multimedia information exists in the multimedia information of the meeting; And determining that the first multimedia information does not exist in the multimedia information of the conference. The conference terminal according to any one of claims 16 to 19, wherein the conference terminal further comprises: a third determining module, configured to: when the first determining module determines that the conference needs the user to participate, and the second determining module determines that the participating state of the user is an absent state, determining the participation reminder a form of the information; wherein the form of the participation reminding information includes at least one of an interface form, an image form, a video form, an audio form, and an instant message form; The sending module is specifically configured to send the participation reminding information to the user according to the form of the participation reminding information. The conference terminal according to any one of claims 16 to 20, wherein the conference terminal further comprises: a fourth determining module; The fourth determining module is configured to determine, according to whether the conference requires the user to participate in the conference and the participation status of the user, the feedback policy is used to indicate that the conference terminal sends the conference terminal to the remote device. The type of feedback data; The sending module is further configured to send the feedback data to the remote device according to the feedback policy; the feedback data includes: video content that interacts with the remote device, and the remote device At least one of interactive audio content, textual content that interacts with the remote device. The conference terminal according to claim 21, wherein the fourth determining module is specifically configured to: if the conference requires the user to participate, and the participation status of the user is an absent state, determine the The feedback policy is to send first feedback data to the remote device, where the first feedback data is used to indicate to the remote device that the conference terminal is positive Notifying the user to join the meeting. The conference terminal according to claim 21, wherein the fourth determining module is specifically configured to: if the conference requires the user to participate, and the participation status of the user is an absent state, determine the The feedback policy is to send second feedback data to the remote device; the second feedback data is used to show the remote device the conference content related to the conference preset by the user. The conference terminal according to claim 21, wherein the fourth determining module is specifically configured to: if the conference does not require the user to participate, and the participation status of the user is an absent state, determine the location The feedback policy is to send third feedback data to the remote device; the third feedback data is used to indicate to the remote device that the user is participating. The conference terminal according to claim 21, wherein the fourth determining module is specifically configured to: if the conference requires the user to participate, and the participation status of the user is a user participation conflict state, Determining the feedback policy is to send fourth feedback data to the remote device, and record the current conference content of the conference; the fourth feedback data is used to indicate the participation status of the user to the remote device Participate in conflict status for users. The conference terminal according to any one of claims 18 to 25, wherein if the first multimedia information that is of interest to the user includes face information of the first participant in the conference, the preset The associated information of the image content that is of interest to the user in the multimedia content is the identity information of the first participant; the second determining unit specifically includes: a first detecting subunit, configured to detect multimedia information of the conference, and determine a face position and a face size of the participant in the multimedia information; a feature extraction sub-unit, configured to perform feature extraction on a face position and a face size of the participant in the multimedia information, to obtain a face feature of the participant; a first matching sub-unit, configured to match a face feature of each participant with a preset face information database to determine a first matching degree; a first determining subunit, configured to determine that the identification result is identity information of the participant whose first matching degree is greater than a preset first threshold in the face information database; a second matching subunit, configured to use the identity information of the participant whose first matching degree is greater than a preset first threshold in the face information database and the first parameter in the preset multimedia content The identity information of the participants is matched, and the feature matching result is obtained. The conference terminal according to any one of claims 18 to 25, wherein if the first multimedia information that is of interest to the user includes the first text information, the text content that the user pays attention to is the first text. The information, the second determining unit specifically includes: a second detecting subunit, configured to detect multimedia information of the conference, and determine a location and a size of a text block area in the multimedia information of the conference; Obtaining a subunit, configured to obtain a text block in the multimedia information of the conference according to a location and a size of the text block area in the multimedia information of the conference; a third matching subunit, configured to match a text block in the multimedia information of the conference with a preset text information library, and determine a second matching degree; a second determining subunit, configured to determine that the recognition result is text information that the second matching degree in the text information library is greater than a preset second threshold; a fourth matching sub-unit, configured to match the text information in the text information library that is greater than a preset second threshold with the first text information in the preset multimedia content, The feature matching result is obtained. The conference terminal according to any one of claims 18 to 25, wherein if the first multimedia information that is of interest to the user includes the first text information, the text content that the user pays attention to is the first text. The information, the second determining unit specifically includes: a third detecting subunit, configured to detect multimedia information of the conference, determine a location and a size of a text block area in the multimedia information of the conference, and obtain a text block in the multimedia information of the conference; a third determining subunit, configured to determine the recognition result according to a geometric feature of the text block; And a fifth matching subunit, configured to match the identification result with the first text information in the preset multimedia content to obtain the feature matching result. The conference terminal according to claim 27 or 28, wherein if the first multimedia information includes multimedia information of the conference, except for a text type, a participant face type, and the The first data sub-unit related to the text information, the second determining sub-unit further includes: a fourth detecting subunit, configured to detect multimedia information of the conference, and determine other data information in the multimedia information of the conference, except the text type and the participant face type; a fourth determining subunit, configured to determine, according to the first text information and the other data information, a degree of correlation between the other data information and the first text information; a fifth determining subunit, configured to determine that the identification result is data information that the correlation is greater than a preset third threshold in the other data information; a sixth matching subunit, configured to match the data information whose correlation degree is greater than a preset third threshold value to the first data information in the other data information, to obtain the feature matching result. The conference terminal according to any one of claims 21 to 29, wherein the feedback data is feedback content preset by the user, or generated according to the feedback policy and the feedback content preset by the user. The data.

Images