CN116600076A

CN116600076A - Video conference realization method and device, electronic equipment and storage medium

Info

Publication number: CN116600076A
Application number: CN202310880022.0A
Authority: CN
Inventors: 王伟伟
Original assignee: China Mobile Communications Group Co Ltd; China Mobile Suzhou Software Technology Co Ltd
Current assignee: China Mobile Communications Group Co Ltd; China Mobile Suzhou Software Technology Co Ltd
Priority date: 2023-07-18
Filing date: 2023-07-18
Publication date: 2023-08-15
Anticipated expiration: 2043-07-18
Also published as: CN116600076B

Abstract

The invention provides a video conference realization method, device, electronic equipment and storage medium, belonging to the technical field of image communication. The implementation method of the video conference includes: establishing a conference connection with the conference server through the second conference terminal; receiving the received video stream of the first conference terminal forwarded from the second conference terminal; wherein, the reception of the first conference terminal The video stream is obtained by the second conference terminal according to the combined video stream sent by the conference server; the combined video stream is obtained by the conference server according to the video stream received by the first conference terminal and the video stream received by the second conference terminal. generated by the received video stream; displaying a conference picture according to the received video stream of the first conference terminal. The invention can improve the efficiency of the video conference.

Description

Video conference realization method and device, electronic equipment and storage medium

Technical Field

The embodiment of the invention relates to the technical field of image communication, in particular to a method and a device for realizing video conference, electronic equipment and a storage medium.

Background

The network video conference system is a multimedia conference platform taking a network as a medium, and a user can break through the limitation of time and region and realize the face-to-face communication effect through the Internet. In the case of inconvenient on-the-fly conferences, web video conferences can provide a convenient way for multiparty communication.

When a large-scale network video conference in which more members participate is held, if all the members join the conference through respective devices, network resources of an office network are occupied, and a large load is caused to a video conference server.

Disclosure of Invention

The embodiment of the invention provides a method, a device, electronic equipment and a storage medium for realizing video conferences, which can improve the conference efficiency and save conference resources.

In order to solve the technical problems, the invention is realized as follows:

in a first aspect, an embodiment of the present invention provides a method for implementing a video conference, which is applied to a first conference terminal, including:

establishing conference connection with a conference server through a second conference terminal;

receiving a received video stream of a first conference terminal forwarded from the second conference terminal; the received video stream of the first conference terminal is obtained by the second conference terminal according to the combined video stream sent by the conference server side; the combined video stream is generated by the conference server side according to the received video stream of the first conference terminal and the received video stream of the second conference terminal;

and displaying the conference picture according to the received video stream of the first conference terminal.

Optionally, the implementation method of the video conference further comprises the following steps:

and sending the sending video stream of the first conference terminal to the second conference terminal, so that the second conference terminal can forward the sending video stream of the first conference terminal to the conference server.

Optionally, the implementation method of the video conference further includes:

acquiring conference operation information generated by user operation;

and generating a sending video stream of the first conference terminal according to the conference operation information.

Optionally, the establishing conference connection with the conference server through the second conference terminal further includes:

acquiring meeting authentication information according to input operation of a user;

transmitting the conference joining authentication information to the second conference terminal;

and establishing the conference connection according to authentication passing information returned by the second conference terminal based on the conference joining authentication information.

Optionally, the displaying the conference picture according to the received video stream of the first conference terminal includes:

combining the member head portraits corresponding to the first conference terminal and the member head portraits corresponding to the second conference terminal to form a grouping image;

generating a conference member display area in a conference picture according to the grouping image;

And displaying the conference picture according to the conference member display area.

In a second aspect, an implementation method of a video conference in an embodiment of the present invention is applied to a second conference terminal, and includes:

receiving conference authentication information sent by a first conference terminal;

the conference authentication information is sent to a conference server side, so that the conference server side authenticates the first conference terminal according to the conference authentication information;

establishing conference connection with the first conference terminal according to authentication passing information returned by the conference server for verifying the conference entering authentication information;

receiving a combined video stream sent by a conference server through the conference connection; the combined video stream is generated by the conference server side according to the received video stream of the first conference terminal and the received video stream of the second conference terminal;

according to the combined video stream, a received video stream of the first conference terminal is obtained; the first conference terminal is one of at least one conference terminal connected with the second conference terminal;

and sending the received video stream of the first conference terminal to the first conference terminal, so that the first conference terminal can display a conference picture according to the received video stream of the first conference terminal.

Optionally, the implementation method of the video conference further includes:

obtaining a received video stream of the second conference terminal according to the combined video stream;

and displaying the conference picture according to the received video stream of the second conference terminal.

Optionally, displaying the conference picture according to the received video stream of the second conference terminal includes:

displaying at least one head portrait display frame of a participant group in a participant member display area of the conference picture; wherein the at least one participant-grouped group image comprises a participant-grouped group image of the second conference terminal;

the grouping image of the participant group of the second conference terminal is generated according to the head portrait information of the second conference terminal and the head portrait information of at least one conference terminal connected with the second conference terminal.

Optionally, the implementation method of the video conference further includes:

receiving a sending video stream of the first conference terminal;

and transmitting the video stream transmitted by the first conference terminal to the conference server side according to the conference authentication information.

Optionally, the sending, to the first conference terminal, the received video stream of the first conference terminal includes:

Transmitting a received video stream of the first conference terminal to the first conference terminal through the conference connection in the case that the state of the conference connection is on;

under the condition that the conference connection state is not connected, caching the received video stream of the first conference terminal; and transmitting the cached received video stream of the first conference terminal to the first conference terminal through the conference connection under the condition that the state of the conference connection is converted from not being connected to being connected.

In a third aspect, an embodiment of the present invention further provides a method for implementing a video conference, where the method is applied to a conference server, and includes:

receiving conference authentication information of a first conference terminal sent by a second conference terminal;

checking the first conference terminal according to the conference entering authentication information, and returning authentication passing information to the second conference terminal under the condition that the checking is passed, so that the second conference terminal can establish conference connection with the first conference terminal according to the authentication passing information;

receiving a video stream sent by a first conference terminal and forwarded by the second conference terminal;

generating a received video stream of the first conference terminal according to the transmitted video stream of the first conference terminal;

Generating a combined video stream according to the received video stream of the first conference terminal;

and sending the combined video stream to a second conference terminal which is in conference connection with the first conference terminal, so that the second conference terminal can send the received video stream of the first conference terminal to the first conference terminal according to the combined video stream.

Optionally, the generating a combined video stream according to the received video stream of the first conference terminal includes:

and generating the combined video stream according to the received video stream of the first conference terminal, the received video stream of the second conference terminal and the received video streams of other conference terminals connected with the second conference terminal.

In a fourth aspect, an embodiment of the present invention further provides an implementation apparatus for a video conference, which is applied to a first conference terminal, including:

the connection module is used for establishing conference connection with the conference server side through the second conference terminal;

a receiving module, configured to receive a received video stream of a first conference terminal forwarded from the second conference terminal; the received video stream of the first conference terminal is obtained by the second conference terminal according to the combined video stream sent by the conference server side; the combined video stream is generated by the conference server side according to the received video stream of the first conference terminal and the received video stream of the second conference terminal;

And the display module is used for displaying the conference picture according to the received video stream of the first conference terminal.

In a fifth aspect, an embodiment of the present invention further provides an implementation apparatus for a video conference, which is applied to a second conference terminal, including:

the certificate receiving module is used for receiving meeting authentication information sent by the first meeting terminal;

the first sending module is used for sending the conference authentication information to the conference server side so that the conference server side authenticates the first conference terminal according to the conference authentication information;

the conference connection module is used for establishing conference connection with the first conference terminal according to authentication passing information which is returned by the conference server and is used for verifying the conference entering authentication information;

the receiving module is used for receiving the combined video stream sent by the conference server side through the conference connection; the combined video stream is generated by the conference server side according to the received video stream of the first conference terminal and the received video stream of the second conference terminal;

the combined video stream processing module is used for obtaining a received video stream of the first conference terminal according to the combined video stream; the first conference terminal is one of at least one conference terminal connected with the second conference terminal;

And the second sending module is used for sending the received video stream of the first conference terminal to the first conference terminal, so that the first conference terminal can display a conference picture in a display area according to the received video stream of the first conference terminal.

In a sixth aspect, an embodiment of the present invention provides a device for implementing a video conference, which is applied to a conference server, and includes:

the first receiving module is used for receiving the conference joining authentication information of the first conference terminal sent by the second conference terminal;

the verification module is used for verifying the first conference terminal according to the conference entering authentication information, and returning authentication passing information to the second conference terminal under the condition that verification passes, so that the second conference terminal can establish conference connection with the first conference terminal according to the authentication passing information;

the second receiving module is used for receiving the video stream sent by the first conference terminal and forwarded by the second conference terminal;

the first generation module is used for generating a received video stream of the first conference terminal according to the transmitted video stream of the first conference terminal;

the second generation module is used for generating a combined video stream according to the received video stream of the first conference terminal;

And the sending module is used for sending the combined video stream to a second conference terminal which is in conference connection with the first conference terminal, so that the second conference terminal can send the received video stream of the first conference terminal to the first conference terminal according to the combined video stream.

In a seventh aspect, the embodiment of the present invention further provides a system for implementing a video conference, where the system includes an implementing device for a video conference applied to a first conference terminal, an implementing device for a video conference applied to a second conference terminal, and an implementing device for a video conference applied to a conference server; the implementation device of the video conference applied to the first conference terminal is used for executing the implementation method of the video conference applied to the first conference terminal; the implementation device of the video conference applied to the second conference terminal is used for executing the implementation method of the video conference applied to the second conference terminal; the implementation device of the video conference applied to the conference server side is used for executing the implementation method of the video conference applied to the conference server side.

In an eighth aspect, an embodiment of the present invention further provides an electronic device, including: a processor, a memory and a program stored on the memory and executable on the processor, which when executed by the processor implements the steps of the method as provided in the first, second or third aspects above.

In a ninth aspect, embodiments of the present invention also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of the first, second or third aspects described above.

In the embodiment of the invention, the first conference terminal can join the video conference through the second conference terminal, receive the received video stream of the first conference terminal for generating the conference picture, and display the conference picture at the first conference terminal according to the received video stream of the first conference terminal, so that the first conference terminal can indirectly obtain the conference picture in the conference process through the second conference terminal to participate in the video conference, reduce the number of conference terminals directly connected with the conference server, reduce the load of the conference server, and save the conference resource of the network video conference.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:

Fig. 1 is a flow chart of a method for implementing a video conference according to an embodiment of the present invention;

fig. 2 is a schematic connection diagram of a conference terminal and a conference server provided by an embodiment of the present invention;

fig. 3 is a schematic view of a conference screen according to an embodiment of the present invention;

fig. 4 is a schematic view of a conference architecture according to an embodiment of the present invention;

FIG. 5 is a schematic view of another conference screen according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of another conference architecture according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of a device for implementing a video conference according to an embodiment of the present invention;

fig. 8 is a schematic diagram of an exemplary implementation device connection for a video conference according to the present invention;

fig. 9 is a signaling interaction diagram of an implementation device of a video conference according to an example of the present invention;

fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Furthermore, the use of "and/or" in the specification and claims means at least one of the connected objects, e.g., a and/or B, meaning that it includes a single a, a single B, and that there are three cases of a and B.

In embodiments of the invention, words such as "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "e.g." in an embodiment should not be taken as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.

The techniques described herein are not limited to long term evolution (Long Time Evolution, LTE)/LTE evolution (LTE-Advanced, LTE-a) systems and may also be used for various wireless communication systems such as code division multiple access (Code Division Multiple Access, CDMA), time division multiple access (Time Division Multiple Access, TDMA), frequency division multiple access (Frequency Division Multiple Access, FDMA), orthogonal frequency division multiple access (Orthogonal Frequency Division Multiple Access, OFDMA), single-carrier frequency division multiple access (Single-carrier Frequency-Division Multiple Access, SC-FDMA), and other systems.

In the embodiment of the present invention, the relationship between the first conference terminal, the second conference terminal and the conference server may include: the first conference terminal and other conference terminals access the conference server side through the second conference terminal to join the video conference. The network video conference in the embodiment of the invention is a video conference realized through a network.

Fig. 1 is a schematic diagram illustrating an implementation method of a video conference according to an embodiment of the present invention, which is applied to a first conference terminal and includes steps S11-S13. The first conference terminal may be a mobile terminal such as a mobile phone, a palm top computer, a tablet computer, a notebook computer, a wearable smart device, etc., or may be a fixed user terminal such as a dedicated conference device, a desktop computer, etc. The second conference terminal may be a dedicated network video conference device or a user terminal capable of participating in a network video conference.

Step S11: and establishing conference connection with a conference server side through the second conference terminal.

In this embodiment, the first conference terminal joins the network video conference through the conference server. The first conference terminal and the second conference terminal may be the same type of terminal or different types of terminals, and the second conference terminal may join the network video conference directly by establishing conference connection with the conference server side, or may join the network video conference by establishing conference connection with other conference terminals with the conference server side.

Step S12: receiving a received video stream of a first conference terminal forwarded from a second conference terminal; the received video stream of the first conference terminal is obtained by the second conference terminal according to the combined video stream sent by the conference server side; the combined video stream is generated by the conference server side according to the received video stream of the first conference terminal and the received video stream of the second conference terminal.

In this embodiment, the received video stream of the first conference terminal may be a video stream used to generate a conference picture of the first conference terminal, and may be a separate video stream or a video stream included in an audio-video composite stream.

The received video stream forwarded by the second conference terminal may be a video stream transmitted from the conference server side and forwarded by the second conference terminal, or may be a video stream transmitted from another conference terminal of the other conference and forwarded by the second conference terminal. The conference server side can send the combined video stream to the second conference terminal through at least one conference terminal, and can also directly send the combined video stream to the second conference terminal.

The combined video stream received by the second conference terminal at least comprises the received video stream of the second conference terminal, namely the video stream for generating the conference picture of the second conference terminal, and the received video stream of the first conference terminal. In case that the conference terminal connected to the conference server side through the second conference terminal includes only the first conference terminal, the combined video stream received by the second conference terminal includes a video stream for generating a conference picture at the second conference terminal and a video stream for generating a conference picture at the first conference terminal.

In this embodiment, the first conference terminal establishes a conference connection with the conference server through the second conference terminal to join the video conference, that is, the first conference terminal establishes a connection with the conference server through connection with the second conference terminal, so as to join the network video conference. The manner in which the first conference terminal is connected to the second conference terminal may include a manner of connection through a local area network, connection through the internet, connection through bluetooth, connection through hardware, and the like.

In one implementation manner, the first conference terminal and the second conference terminal are connected through a local area network, bluetooth or hardware, so that the physical position relationship of different conference terminals can be known through the connection relationship between the conference terminals, for example, the first conference terminal and the second conference terminal may be located in the same conference room, so that the first conference terminal can be connected with the second conference terminal in a short-range communication manner, and then join in a conference through the second conference terminal.

In another implementation, as shown in fig. 2, there may also be a third conference terminal 23, which joins the web video conference indirectly through the first conference terminal 21, the second conference terminal 22. Wherein the first conference terminal 21 is connected to the second conference terminal 22, the second conference terminal 22 is connected to the conference server 24 to join the network video conference, and the first conference terminal 21 is indirectly connected to the conference server 24 through the second conference terminal 22 to join the network video conference.

In the case of a conference terminal connected to the conference server side through the second conference terminal, including a plurality of conference terminals including the first conference terminal, the combined video stream received by the second conference terminal may further include a received video stream of another conference terminal joining the network video conference through the second conference terminal in addition to the first conference terminal.

In one embodiment, the first conference terminal is a user terminal having a display, and the second conference terminal is any one of a dedicated conference device and a user terminal. In the case that there are other conference terminals indirectly connected to the conference server side through the first conference terminal to join the video conference, the first conference terminal may also be a dedicated conference device.

Step S13: and displaying the conference picture according to the received video stream of the first conference terminal.

In this embodiment, the first conference terminal runs an application program for participating in the web video conference, or runs a tool for participating in the web video conference through a web page. Thus, a window for displaying a conference screen is operated at the first conference terminal. After receiving the received video stream of the first terminal, the data of the video stream may be parsed, and the conference screen may be rendered at a window for displaying the conference screen.

According to the embodiment of the invention, the first conference terminal can receive the received video stream of the first conference terminal for generating the conference picture through the second conference terminal, and the conference picture is displayed on the first conference terminal according to the received video stream of the first conference terminal, so that the first conference terminal can indirectly obtain the conference picture in the conference process through the second conference terminal to participate in the video conference, the number of conference terminals directly connected with the conference server is reduced, the load of the conference server is reduced, and the conference resource of the network video conference is saved.

In one embodiment, the implementation method of the video conference further includes:

In a specific implementation, the video stream sent by the first conference terminal to the second conference terminal may include only audio information and/or default display information of the first conference terminal. For example, the first conference terminal does not start the camera, but cannot acquire video data of the first conference terminal, and in other conference terminals, the first conference terminal correspondingly displays a default head portrait picture. The transmitted video stream of the first conference terminal includes data corresponding to the default avatar picture.

In another specific implementation manner, the video stream sent by the first conference terminal to the second conference terminal may include all data that is generated by the first conference terminal during the process of participating in the video conference and needs to be processed and received by the conference server. For example, the user operates message information, check-in information, hand lifting information, etc. generated by the first conference terminal in response to the conference call.

In a specific implementation manner, the access method of the video conference further includes:

and sending meeting authentication information of the first meeting terminal to the second meeting terminal, so that the second meeting terminal can forward a sending video stream of the first meeting terminal according to the meeting authentication information.

When the first conference terminal is connected with the second conference terminal to join the conference, the first conference terminal sends a conference joining certificate to the second conference terminal, and the second conference terminal can store the conference joining certificate of the first conference terminal, so that the conference joining certificate of the first conference terminal can be attached when the video stream sent by the first conference terminal is forwarded subsequently, and the conference server can identify that the sender is the first conference terminal after receiving the video stream sent by the first conference terminal.

In the case that a plurality of conference terminals enter a conference through the second conference terminal, the second conference terminal may carry the conference entry authentication information of each corresponding conference terminal when forwarding the transmission video stream.

In one embodiment, the access method of the video conference further includes:

acquiring conference operation information generated by user operation;

In this embodiment, the conference operation information generated by the user operation may include information generated by operating tools such as a button and an input box on the conference interface. For example, the user clicks a hand-up button or a check-in button of the conference interface, and conference operation information indicating hand-up or check-in is generated. For another example, the user inputs a conference utterance through an input box, and conference operation information including the utterance content is generated.

In a specific embodiment, generating the sending video stream of the first conference terminal according to the conference operation information may include generating the sending video stream of the first conference terminal according to video picture information obtained by shooting the first conference terminal and an instruction generated by the first conference terminal.

In one embodiment, establishing a conference connection with a conference server side through a second conference terminal includes:

and establishing the conference connection according to authentication passing information returned by the second conference terminal based on the conference joining authentication information. The conference connection is used for transmitting a sending video stream of the first conference terminal and a receiving video stream of the first conference terminal.

In this embodiment, the input operation of the user may include an operation of the user to input the conference number of the conference, and in a case where the password is required, the input operation of the user may also include an operation of the user to input the conference password.

According to the input operation of the user, obtaining the meeting authentication information may include: and generating meeting authentication information of the user according to the meeting number input by the user and account information of the user logging in the meeting. Or may include: and generating meeting authentication information of the user according to the meeting number, the meeting password and account information of the user logging in the meeting, which are input by the user.

After the first conference terminal sends the conference entering authentication information to the second conference terminal, the second conference terminal can forward the conference entering authentication information of the first conference terminal to the conference server terminal, the conference server terminal authenticates the conference entering authentication information of the first conference terminal, and after the authentication is passed, the authentication passing information is returned to the second conference terminal, so that the first conference terminal and the second conference terminal establish conference connection, and under the condition that the second conference terminal and the conference server terminal establish conference connection, the first conference terminal establishes conference connection with the conference server terminal indirectly through the second conference terminal, and joins the network video conference.

After conference terminals joining in the network video conference through the second conference terminal send conference authentication information to the second conference terminal, the second conference terminal stores the conference authentication information of each conference terminal, and after receiving the sending video stream of each conference terminal, the corresponding conference authentication information and the sending video stream are forwarded to the conference server side, so that the conference server side can determine the sender of the sending video stream according to the conference authentication information.

In one embodiment, the displaying the conference picture according to the received video stream of the first conference terminal includes:

combining and displaying the member head portraits corresponding to the first conference terminal and the member head portraits corresponding to the second conference terminal to form a grouping image;

and displaying a conference picture according to the conference member display area.

In this embodiment, the second conference terminal joins the conference terminal of the network video conference, and can perform the same group display with the second conference terminal. Similarly, other conference terminals for forwarding the video stream sent by the conference terminal and receiving the video stream are also displayed in the same group in the conference picture together with the conference terminal responsible for forwarding.

As shown in fig. 3, in the conference screen, a member presentation area 32 and a conference content presentation area 31 are included. The conference content display area 31 may be used to display video pictures taken by conference terminals corresponding to the presenter (i.e., the host) of the conference, and the member display area 32 may be used to display head portrait information corresponding to each participant member or video pictures taken by conference terminals of each participant member. Referring to the embodiment shown in fig. 3, the participating users include user a-user K. The conference terminals of the users A and B directly join the network video conference, the conference terminal of the user D joins the network video conference through the conference terminal of the user C, the conference terminals of the users E, F and G join the network video conference through the conference equipment of the conference room A, and the users H, I, J and K join the network video conference through the conference equipment of the conference room B. Thus, user C, user D belongs to one group, user E, user F, user G belongs to another group, and user H, user I, user J, user K belongs to yet another group. In the member presentation area 32 of the conference screen, each conference terminal directly joining the network video conference corresponds to one head portrait display frame 33. The user C and the user D are simultaneously displayed on the avatar display frame 33 corresponding to the user C, the user E, the user F, and the user G are simultaneously displayed on the avatar display frame 33 corresponding to the conference device of the conference room a, and the user H, the user I, the user J, and the user K are simultaneously displayed on the avatar display frame 33 corresponding to the conference device of the conference room B.

In this embodiment, by displaying all the participants in groups, each participant can visually check the grouping situation of the participant from the conference picture, so as to indirectly learn the relationship between the physical locations of the participants.

The embodiment of the invention also provides a method for realizing the video conference, which is applied to the second conference terminal and comprises the following steps:

In this embodiment, the received video stream of the first conference terminal and the received video stream of the second conference terminal are both generated according to the transmitted video stream of the conference terminal received by the conference server at the previous time.

In this embodiment, the second conference terminal can forward the received video stream of the first conference terminal that joins the video conference indirectly through the second conference terminal, so that the first conference terminal can obtain the video stream in the conference process through the second conference terminal, and display the conference picture.

In this embodiment, the conference screen generating manner on the second conference terminal may be consistent with the first conference terminal.

In one embodiment, the displaying the conference screen according to the received video stream of the second conference terminal includes:

In the embodiment of the invention, the grouping image of the reference group comprises head portraits of all members included in the reference group, a thumbnail display diagram is generated according to the head portraits of all members, and the thumbnail display diagram is used as the grouping head portraits of the reference group and is displayed in a head portraits display frame. The thumbnail display may include a set number of member avatar thumbnail images, which are expanded upon receiving click operation information of the user on the thumbnail display, presenting an avatar list of all members included in the participant group.

the conference authentication information is sent to the conference server side, so that the conference server side authenticates the first conference terminal according to the conference authentication information;

According to the authentication passing information of the conference server side authentication information, establishing conference connection with the first conference terminal; the conference connection with the first conference terminal is for transmitting a received video stream of the first conference terminal.

In this embodiment, the conference authentication information may be generated according to account information corresponding to the first conference terminal and a conference number input by a user of the first conference terminal, and if a password is set, the conference authentication information is also generated according to the password input by the user of the first conference terminal. The second conference terminal stores the conference authentication information, or stores at least part of the conference authentication information, and after receiving the combined video stream, the second conference terminal matches the identity information in the combined video stream according to the conference authentication information, recognizes the received video stream belonging to the first conference terminal and forwards the received video stream to the first conference terminal. After receiving the sending video stream of the first conference terminal, combining the sending video stream of the first conference terminal with the conference authentication information of the first conference terminal, carrying out identity information identification on the sending video stream of the first conference terminal by the conference authentication information, and forwarding to the conference server side.

In a specific implementation manner, the second conference terminal may start a transfer information sending function, so that other terminals connected with the second conference terminal through bluetooth, wireless network, wired network, and the like can detect the second conference terminal and join the network video conference through the second conference terminal.

receiving a sending video stream of the first conference terminal;

and sending the sending video stream to the conference server side according to the conference authentication information.

In this embodiment, the video stream sent by the first session terminal is a video stream sent by the first session terminal. Meanwhile, the video streams sent by other conference terminals are video streams sent by corresponding other conference terminals.

In one embodiment, the sending the received video stream of the first session terminal to the first session terminal includes:

In the embodiment of the invention, under the condition of stably maintaining the conference connection, the received video stream of the first conference terminal is directly sent to the first conference terminal. Under the condition that the conference connection is unstable and disconnection occurs, the received video stream of the first conference terminal is cached, and under the condition that the conference connection is switched on again, the cached video stream is sent to the first conference terminal, so that information missing transmission is reduced.

The embodiment of the invention also provides a method for realizing the video conference, which is applied to the conference server and comprises the following steps:

receiving a video stream sent by a first conference terminal and forwarded by the second conference terminal; the first conference terminal establishes conference connection with the conference server side through the second conference terminal so as to join in a video conference;

generating a received video stream of the first conference terminal according to the transmitted video stream of the first conference terminal; generating a combined video stream according to the received video stream of the first conference terminal;

In the embodiment of the present invention, the second conference terminal receives the sending video stream of the first conference terminal, and forwards the sending video stream of the first conference terminal according to the conference authentication information or other identity information of the first conference terminal, which may specifically be that the conference authentication information or other identity information of the first conference terminal is attached to the sending video stream of the first conference terminal, and sends the video stream to the conference server.

The conference server side can perform information processing according to the sending video stream of the first conference terminal and the sending video streams of other conference terminals, and generate a receiving video stream of the first conference terminal for displaying the conference picture on the first conference terminal, so that the first conference terminal can present the state of each conference terminal according to the receiving video stream of the first conference terminal.

Generating a combined video stream from the received video stream of the first conference terminal may include: and generating a combined video stream sent to the second conference terminal according to the received video stream of the first conference terminal, the received video streams of other conference terminals joining the network video conference through the second conference terminal and the received video stream of the second conference terminal.

In the embodiment of the invention, after the combined video stream is sent to the second conference terminal which is in conference connection with the first conference terminal, the second conference terminal can obtain the received video stream of the first conference terminal according to the combined video stream and forward the received video stream of the first conference terminal to the first conference terminal.

receiving conference authentication information of the first conference terminal sent by the second conference terminal;

and checking the first conference terminal according to the conference entering authentication information, and returning authentication passing information to the second conference terminal under the condition that the checking is passed, so that the second conference terminal can establish conference connection with the first conference terminal according to the authentication passing information.

In the embodiment of the invention, under the condition that the conference server side checks and passes other conference terminals joining in the network video conference through the second conference terminal, the authentication passing information is sent to the second conference terminal, and then the second conference terminal establishes conference connection with the other conference terminals according to the authentication passing information.

In one embodiment, the generating a combined video stream according to the received video stream of the first conference terminal includes:

In the embodiment of the invention, the number of the data transmission paths can be saved by combining the video streams to send the received video streams of the plurality of conference terminals, and the data of the plurality of received video streams are transmitted through the conference connection between the second conference terminal and the conference server. After the second conference terminal receives the combined video stream, the received video streams of the different conference terminals are disassembled and separated from the combined video stream, conference pictures are displayed at the second conference terminal according to the received video streams of the second conference terminal, and the received video streams of other conference terminals except the second conference terminal are correspondingly sent to other conference terminals.

Through the embodiment of the invention, a plurality of terminals can be indirectly connected with the conference server through the conference terminal connected with the conference server, join in the network video conference, realize accessing the remote video conference through the same terminal, and simultaneously can record the information of all corresponding participants for sign-in, data statistics and the like.

If one client of the web video conference product allows only one user to join a conference, as shown in fig. 4, each user 41 is connected to the conference server 42 through a base station or a switch through a corresponding conference terminal, i.e., a plurality of conference terminals are directly connected to the conference server 42, i.e., the video conference server in fig. 4. The conference terminal for the conference room is needed to be used for participating in the conference by a plurality of conference users, the conference picture is shown in fig. 5, the conference users cannot identify the identity of each other conference user in the conference, especially the conference users participating in the network video conference in the conference room through the conference equipment, each user is independent, the conference picture shown in fig. 5 is simple, the actual conference condition cannot be intuitively represented, and the conference condition of each user in the conference room cannot be represented. If the identity of other participating users is required to be identified, technical support such as face recognition and the like can be utilized, and the accuracy is low and omission is easy.

In an exemplary scenario, when an enterprise or organization holds a large conference in which all members participate in the manner shown in fig. 4, for example, all members join the conference as conference members through their respective terminal devices, network resources of an office network will be occupied, and a large load will be imposed on a conference server, so that users participating in the conference room through dedicated conference devices cannot be counted directly, and external sign-in software is needed. Even though the proxy server is used, only the proxy of the network layer is made, and the load of the server and the use of network bandwidth are not reduced; conversely, proxy nodes are added, increasing network overhead and cost.

In one example of the present invention, as shown in fig. 6, the basic topology structure of the connection between the conference server and each conference terminal is that the user C and the user D with adjacent seats in the same office place of the user can connect to the same PC (Personal Computer ) client (or other mobile phone clients) through the mobile phone plug-in, while in the conference room, the user can connect to the special conference equipment of the conference room through the respective mobile phone terminals, and join the conference through the conference terminal of the conference room. The conference server can thus authenticate the user C, D, E, F, G and create a session, respectively, while the user C, D, E, F, G can perform basic videoconferencing operations, such as hand lifting, speaking, etc., via their respective handsets.

The conference terminals indirectly access the conference through other conference terminals can be grouped according to the conference conditions, and the grouping conditions are displayed on a conference picture, as shown in fig. 3, through the conference picture displayed on the conference terminal, a participant can intuitively check the conference conditions of all current users, and the participant mode and the participant position of the user can be known through grouping. Under the condition that the mobile phone terminal is used as a first conference terminal and the PC terminal is used as a second conference terminal, the mobile phone terminal can establish wireless connection with the PC terminal through Bluetooth, wifi and other technologies and is connected with a computer participating in a video conference. The computer terminal and the conference server terminal which are participating in the video conference authenticate the mobile phone terminal, and after the mobile phone terminal passes the authentication, the user of the mobile phone terminal can realize basic operations such as sign in, hand lifting, speaking and the like through the terminal.

After the first conference terminal joins the network video conference through the second conference terminal, the user can perform basic conference operations such as hand lifting, sign-in and the like through the first conference terminal, and the conference server side can also recognize the identity of the user through the first conference terminal. Similarly, the conference room terminal can also join the conference through the scheme described in the proposal, and all users who enter the conference can perform basic conference operation. By the method, compared with the mode that conference members respectively and independently enter the conference, office network resources and server resources for holding the conference can be greatly saved, and meanwhile, the functions related to the video conference can be used to the greatest extent by each conference member.

The embodiment of the invention also provides a device for realizing the video conference, the structure is shown in fig. 7, and the device is applied to a first conference terminal and comprises:

a connection module 71, configured to establish a conference connection with a conference server through a second conference terminal;

a receiving module 72, configured to receive a received video stream of a first conference terminal forwarded from the second conference terminal; the received video stream of the first conference terminal is obtained by the second conference terminal according to the combined video stream sent by the conference server side; the combined video stream is generated by the conference server side according to the received video stream of the first conference terminal and the received video stream of the second conference terminal;

And the display module 73 is configured to display a conference picture according to the received video stream of the first conference terminal.

In one embodiment, the implementation apparatus of a video conference further includes:

and the sending module is used for sending the sending video stream of the first conference terminal to the second conference terminal, so that the second conference terminal can forward the sending video stream of the first conference terminal to the conference server.

the operation information acquisition module is used for acquiring conference operation information generated by user operation;

and the sending video stream generating module is used for generating a sending video stream of the first conference terminal according to the conference operation information.

In one embodiment, the connection module includes:

an input operation analysis unit for obtaining the conference authentication information according to the input operation of the user;

a credential information transmitting unit configured to transmit the conference joining authentication information to the second conference terminal;

and the authentication unit is used for establishing the conference connection according to authentication passing information returned by the second conference terminal based on the conference joining authentication information.

In one embodiment, a display module includes:

The grouping head portrait unit is used for combining and displaying the member head portraits corresponding to the first conference terminal and the member head portraits corresponding to the second conference terminal to form a grouping image;

the conference member display area unit is used for generating a conference member display area in a conference picture according to the grouping images;

and the picture unit is used for displaying the conference picture according to the conference member display area.

The embodiment of the invention also provides a device for realizing the video conference, which is applied to the second conference terminal and comprises the following steps:

the first sending module is used for sending the conference authentication information to a conference server side so that the conference server side authenticates the first conference terminal according to the conference authentication information;

the received video stream obtaining module is used for obtaining the received video stream of the second conference terminal according to the combined video stream;

and the display module is used for displaying the conference picture according to the received video stream of the second conference terminal.

In one embodiment, a display module includes:

the head portrait display frame unit is used for displaying at least one head portrait display frame of the participant group in the participant member display area of the conference picture; wherein the at least one participant-grouped group image comprises a participant-grouped group image of the second conference terminal;

a first terminal sending video stream receiving module, configured to receive a sending video stream of the first session terminal;

the first terminal sends a video stream sending module, which is used for sending the sending video stream to the conference server side according to the conference authentication information.

In one embodiment, the first terminal transmitting video stream transmitting module includes:

a first sending unit, configured to send a received video stream of the first conference terminal to the first conference terminal through the conference connection when the state of the conference connection is on;

a second sending unit, configured to buffer a received video stream of the first conference terminal when the conference connection state is not on; and transmitting the cached received video stream of the first conference terminal to the first conference terminal through the conference connection under the condition that the state of the conference connection is converted from not being connected to being connected.

The embodiment of the invention also provides a device for realizing the video conference, which is applied to the conference server side and comprises the following steps:

the first receiving module is used for receiving the conference authentication information of the first conference terminal sent by the second conference terminal;

In one embodiment, the second generating module is further configured to:

In one example of the present invention, the first conference terminal is a mobile phone terminal, the second conference terminal is a PC terminal, and as shown in fig. 8, a mobile phone terminal plug-in 81 is configured at the mobile phone terminal, and the mobile phone terminal plug-in 81 implements a function of an implementation device applied to a video conference of the first conference terminal. The PC side is configured with a PC side plug-in 82, and the PC side plug-in 82 implements a function of an implementation device applied to the video conference of the second conference terminal. The conference server side is configured with a video conference server side dedicated interface 83 for implementing the functions of the implementing device applied to the video conference of the conference server side.

The handset-side plug-in 81 includes an authentication controller, a connection controller, a receiver, and a repeater.

The authentication controller is used for acquiring user login information and sending conference access authentication information to the special video conference interface to apply for credentials under the condition that the mobile phone client logs in the user conference account. If the user conference account is not logged in or the user conference account is offline, the user conference account and the password can be used to connect the PC side plug-in unit and then call the special video conference interface to send the conference entering authentication information so as to acquire the certificate. The functions implemented by the authentication controller include the functions of the credential information transmitting module in the foregoing embodiments.

The connection controller is used for creating connection between the mobile phone plug-in and the PC plug-in, and maintaining the connection state, and can be automatically reconnected if disconnection occurs. The functions implemented by the connection controller include the functions of the connection module and the connection module of the foregoing embodiments.

The receiver is used for receiving the conference control operation of the mobile phone video conference client and receiving callback information of the PC side plug-in. The functions implemented by the receiver include the functions of the operation information acquisition module of the foregoing embodiment.

The repeater forwards conference control operations (such as user operations of hand lifting, sign-in and the like) of the mobile phone video conference client to the PC side plug-in under the condition that the mobile phone side plug-in has acquired the certificate; and the callback information of the PC side plug-in can be forwarded to the mobile phone video conference client side. The functions realized by the repeater comprise a transmitting video stream generating module and a transmitting module.

The PC-side plug-in 82 for the handset-side connection includes a credential manager, a connection manager, a receiver, and a transponder.

The credential manager is used for managing corresponding management of credentials, session and connection acquired by the mobile phone plug-in, and distinguishing the identity of the mobile phone plug-in connected at present. The functions implemented by the credential manager include the functions of the first sending module and the conference connection module in the foregoing embodiments.

The connection manager is used for managing all the connections connected to the current PC end plug-in, and the corresponding connection is obtained through the component before the forwarding and the receiver process the information. The functions realized by the connection manager include the functions of the combined video stream processing module, the first terminal sending video stream receiving module and the first terminal sending video stream sending module in the foregoing embodiments.

The receiver is used for receiving operations such as login, conference control and the like of the mobile phone plug-in unit, and can also receive information returned by the special interface of the video conference. The functions implemented by the receiver include the functions of the receiving module and the functions of the credential receiving module in the foregoing embodiments.

The repeater is used for forwarding information from the plug-in unit of the mobile phone terminal and calling a special interface of the video conference server terminal to realize operations such as login, conference control and the like; and the information returned by the special interface of the video conference can be forwarded to the plug-in unit at the mobile phone end. The functions implemented by the repeater include the functions of the second transmitting module in the foregoing embodiment.

Through the PC terminal plug-in, one or more connected mobile phone terminals can be managed, and meanwhile, data streams of a plurality of users are combined, so that the video conference connection of the plurality of mobile phone terminals sharing one PC terminal is realized, and the information and connection state of the participating user of each mobile phone terminal are uploaded in real time; the mobile phone terminal plug-in can perform conference control through video conference connection shared by the PC terminal.

The videoconference server-specific interface 83 includes a credential generator, receiver, and repeater.

The credential generator is used to generate and manage login credential information for the user. Meanwhile, the participant information of each client node is saved and is used for generating the client grouping view. The functions implemented by the credential generator include the functions of the first receiving module and the checking module of the foregoing embodiment.

The receiver is used for receiving the request data of the PC side plug-in and receiving the operation result returned by the existing interface of the video conference. The receiver implemented functions include the functions of the second receiving module of the previous embodiment.

The repeater is used for repeating the control information of the PC terminal plug-in unit to the existing interface of the video conference, can also repeat the operation result returned by the existing interface of the video conference to the PC customer terminal. The functions implemented by the repeater include the functions of the transmission module of the previous embodiment.

Receiving mobile phone end user information and conference control information forwarded by a PC end plug-in through the special interface of the video conference service end; user information is issued to the client, and the technical effect of generating a grouping view at other user clients can be realized by utilizing the video conference client; and forwarding the conference control information to the video conference server side, so that the conference control function can be realized.

Meanwhile, the conference server side is further provided with a video conference existing interface 84 for authenticating a video conference server side dedicated service and establishing a secure connection.

The conference server side is also provided with a video conference view window module which is used for generating corresponding user grouping views by the conference terminals according to the condition that the users stored by the conference server side are connected with the client nodes. Through the adjusted video conference client plug-in, receiving the conference user information issued by the video conference server special interface and grouping according to the PC end sharing connection condition, and generating a grouping view at the client end, the technical effect of displaying all the user conference conditions of the PC end plug-in sharing connection in a grouping manner can be realized.

It should be noted that, in the above components, the receiver and the transponder are distinguished to perform asynchronous interaction in consideration of machine performance and user experience. If performance is not a concern, the two components may also be combined into one component.

Based on the structure diagram shown in fig. 8, the signaling interaction process between the mobile phone end, the PC end and the conference server end in one example of the present invention is shown in fig. 9.

Referring to fig. 9, the video conference mobile phone plug-in joins the conference through the video conference PC client/conference terminal plug-in, and identity authentication is required first. Before a plurality of video conference mobile phone end plug-ins (respectively using different user accounts) are connected to the PC client end/conference terminal through mobile equipment, the video conference server end is required to be directly requested to acquire user identity credentials through the mobile equipment, and the user identity credentials acquired by different users are different.

If the video conference mobile phone terminal plug-in unit successfully acquires the user identity credential, the plug-in unit needs to carry the credential when being connected with the PC client terminal/conference terminal plug-in unit, and the PC client terminal/conference terminal plug-in unit acquires the conference authentication information according to the credential and sends the conference authentication information (i.e. the credential shown in fig. 9) to the video conference server terminal for identity authentication.

After the verification of the special interface of the video conference server end is passed, the PC client end/conference terminal plug-in establishes session connection for the video conference mobile phone end plug-in, and caches the conference authentication information of the user logged in by the mobile device according to the session information or session information for establishing the session connection. If a plurality of mobile phone end plug-ins are connected with the PC client end/conference terminal plug-in, a plurality of session connection pieces are created to respectively cache the conference authentication information of different devices.

The video conference mobile phone terminal sends a conference control request to the PC client/conference terminal through the mobile phone terminal plug-in, the PC client/conference terminal firstly finds the corresponding evidence of the device, and then the PC client/conference terminal encapsulates the conference control instruction through the corresponding evidence of the device, namely the operation ID (Identification) in FIG. 9, and sends the encapsulated data to the server terminal.

After the server receives the data, the data can be unpacked, the real equipment information for sending the request and the corresponding conference control instruction information are obtained, and after the server verifies that the error is absent, the corresponding conference control instruction is executed.

After the server side executes the relevant conference control instruction, an execution result is sent to a plug-in unit connected with the PC client side/conference terminal through a special interface, the PC client side/conference terminal judges whether the equipment which previously sends the conference control instruction is connected effectively, if so, session information containing the conference control instruction is forwarded, and if so, the session information (namely, the session information) is cached until the equipment is reconnected and then sent.

The server side can also actively send a message to the PC client/conference terminal plug-in through a special interface, for example, the PC client/conference terminal plug-in receives the message and then forwards the message to each effectively connected mobile device; if the message is sent separately, the server side needs to attach the connection information of the corresponding mobile terminal equipment through a special interface, and the PC client/conference terminal plug-in receives the message and then forwards the message to the corresponding mobile equipment.

Referring to fig. 10, an embodiment of the present invention further provides an electronic device 100, including a processor 101, a memory 102, and a computer program stored in the memory 102 and capable of running on the processor 101, where the computer program when executed by the processor 101 implements each process of the embodiment of the implementation method of the video conference, and the process can achieve the same technical effect, so that repetition is avoided and no further description is given here.

The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the processes of the implementation method embodiment of the video conference, and can achieve the same technical effects, so that repetition is avoided, and no further description is given here. Wherein the computer readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.

The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are to be protected by the present invention.

Claims

1. The implementation method of the video conference is characterized by being applied to a first conference terminal and comprising the following steps:

2. The method according to claim 1, wherein the method further comprises:

3. The method according to claim 2, wherein the method further comprises:

acquiring conference operation information generated by user operation;

4. A method according to any of claims 1-3, wherein said establishing a conference connection with a conference server side via a second conference terminal comprises:

5. The method of claim 1, wherein displaying a conference screen based on the received video stream of the first conference terminal comprises:

6. The implementation method of the video conference is characterized by being applied to a second conference terminal and comprising the following steps:

receiving the combined video stream sent by the conference server through the conference connection; the combined video stream is generated by the conference server side according to the received video stream of the first conference terminal and the received video stream of the second conference terminal;

7. The method of claim 6, wherein the method further comprises:

8. The method according to claim 6, wherein displaying a conference screen based on the received video stream of the second conference terminal includes:

and grouping images of the participant groups of the second conference terminal are generated according to member head images corresponding to the second conference terminal and member head images corresponding to at least one conference terminal connected with the second conference terminal.

9. The method of claim 8, wherein the method further comprises:

receiving a sending video stream of the first conference terminal;

10. The method of claim 6, wherein the sending the received video stream of the first conferencing terminal to the first conferencing terminal comprises:

under the condition that the conference connection state is not connected, caching the received video stream of the first conference terminal;

And transmitting the cached received video stream of the first conference terminal to the first conference terminal through the conference connection under the condition that the state of the conference connection is converted from not being connected to being connected.

11. The implementation method of the video conference is characterized by being applied to a conference server side and comprising the following steps:

12. The method of claim 11, wherein the generating the combined video stream from the received video stream of the first conference terminal comprises:

13. An implementation apparatus for a video conference, which is applied to a first conference terminal, includes:

14. An implementation apparatus for a video conference, which is applied to a second conference terminal, includes:

15. The device for realizing the video conference is characterized by being applied to a conference server side and comprising the following components:

16. The system for realizing the video conference is characterized by comprising a video conference realizing device applied to a first conference terminal, a video conference realizing device applied to a second conference terminal and a video conference realizing device applied to a conference server side; wherein the implementation means of the video conference applied to the first conference terminal is adapted to perform the method of any of claims 1-5; the implementation means of a video conference applied to a second conference terminal for performing the method of any of claims 6-10; an implementation device for video conferencing applied to a conference server side is adapted to perform the method of claim 11 or 12.

17. An electronic device, comprising: a processor, a memory and a program stored on the memory and executable on the processor, the program implementing the steps of the method according to any one of claims 1 to 5 when executed by the processor, or the method according to any one of claims 6 to 10 when executed by the processor, or the method according to claim 11 or 12 when executed by the processor.

18. A computer readable storage medium, characterized in that it has stored thereon a computer program which, when executed by a processor, implements the steps of the method according to any of claims 1 to 5; alternatively, the computer program when executed by a processor performs the steps of the method according to any of claims 6-10; alternatively, the computer program realizes the steps of the method according to claim 11 or 12 when executed by a processor.