Disclosure of Invention
The invention provides conference terminal and video wall integrated equipment and a system, and aims to solve the problems that in the prior art, the cost is high and the installation and the use are complicated because the video wall and the conference terminal are required to be arranged at the same time.
The invention is realized in such a way that a conference terminal and a video wall integrated device comprises:
the main control system is used for acquiring remote audio and video data, local audio and video data and conference related data;
The conference processing system is in communication connection with the main control system and is used for receiving remote audio and video data, local audio and video data and conference related data sent by the main control system, correspondingly processing the remote audio and video data, and sending the processed local audio and video data and conference related data to the audio and video playing equipment according to a first playing rule so as to play conference pictures and conference audios;
And the television wall processing system is in communication connection with the main control system and is used for receiving the remote video data sent by the main control system, correspondingly processing the remote video data and then sending the remote video data to video playing equipment according to a second playing rule so as to play the television wall picture.
In an embodiment, the apparatus comprises:
the audio acquisition module is in communication connection with the main control system and is used for acquiring local audio data;
and the video acquisition module is in communication connection with the main control system and is used for acquiring local video data.
In an embodiment, the apparatus further comprises:
and the communication module is in communication connection with the main control system and is used for establishing network connection with remote conference terminals of various conference sites participating in the conference so as to acquire remote audio and video data and conference related data sent by the remote conference terminals.
In one embodiment, the conference processing system includes:
The first video processing module is used for carrying out corresponding processing on the remote video data and the local video data;
The audio processing module is used for carrying out corresponding processing on the remote audio data and the local audio data;
and the data processing module is used for processing the conference related data.
In one embodiment, the television wall processing system comprises:
And the second video processing module is used for carrying out corresponding processing on the remote video data.
In an embodiment, the audio/video playing device includes a plurality of conference video displays and a voice playing device, and the conference processing system is configured to perform communication connection with the conference video displays through a video output interface, and send different video data to different conference video displays for display according to user requirements and types of the video data;
the conference processing system is also used for being in communication connection with the voice playing device through the voice output interface and used for playing voice through the voice playing device.
In an embodiment, the video playing device includes a video wall display, and the video wall processing system is configured to determine a split screen mode according to a user requirement or the number of remote video data, and perform split screen display in a playing interface of the video wall display according to the split screen mode.
In an embodiment, the apparatus comprises:
the noise reduction module is in communication connection with the main control system and is used for carrying out noise reduction treatment on the local audio data and the environmental noise in the remote language data;
and the echo cancellation module is in communication connection with the main control system and is used for canceling the echoes in the local audio data and the remote language data.
In an embodiment, the apparatus comprises:
the subtitle conversion module is used for respectively converting the local audio data and the remote audio data into subtitles;
And the translation module is used for translating the local audio data and the remote audio data into different languages and converting the different languages into subtitles in the corresponding languages through the subtitle conversion module so as to display the subtitles.
Audio data
In a second aspect, a conference terminal and video wall integrated system is provided, including:
The conference terminal and video wall integrated device is arranged at a main conference site and is used for performing audio/video interaction and supervision with each sub conference site;
And the conference terminals are respectively arranged at each conference site and are used for collecting remote audio and video data of each conference site and sending the remote audio and video data to the conference terminal and video wall integrated equipment.
The embodiment of the application provides a conference terminal and video wall integrated device and a system, wherein the conference terminal and video wall integrated device comprises a main control system, a conference processing system and a video wall processing system, wherein the main control system is used for acquiring remote audio and video data, local audio and video data and conference related data, the conference processing system is in communication connection with the main control system and is used for receiving the remote audio and video data, the local audio and video data and the conference related data which are transmitted by the main control system, correspondingly processing the remote audio and video data, the local audio and video data and the conference related data and transmitting the corresponding processed remote audio and video data to audio and video playing equipment according to a first playing rule so as to play a video wall picture. In the embodiment of the application, the conference terminal and the television wall function are realized in one device, so that the cost of a user is effectively saved, the installation is simple, the user can select to simultaneously open the conference terminal and the television wall function according to the needs, and only one of the conference terminal or the television wall function can be opened, so that various requirements of the user can be met, and the user experience is improved.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, an embodiment of the present application provides a conference terminal and video wall integrated device, including:
the main control system is used for acquiring remote audio and video data, local audio and video data and conference related data;
The conference processing system is in communication connection with the main control system and is used for receiving remote audio and video data, local audio and video data and conference related data sent by the main control system, correspondingly processing the remote audio and video data, and sending the processed local audio and video data and conference related data to the audio and video playing equipment according to a first playing rule so as to play conference pictures and conference audios;
And the television wall processing system is in communication connection with the main control system and is used for receiving the remote video data sent by the main control system, correspondingly processing the remote video data and then sending the remote video data to video playing equipment according to a second playing rule so as to play the television wall picture.
In the embodiment of the application, the main control system can comprise controllers such as a singlechip, a main control chip, an MCU and the like.
In the embodiment of the application, the remote audio and video data can specifically comprise audio data and video data of each sub-conference site, namely the remote audio data and the remote video data, and the audio and video data of conference personnel of each conference site can be specifically collected through conference terminals arranged at each sub-conference site.
In the embodiment of the application, the local audio/video data may include audio data and video data of a main conference site, i.e., local audio data and local video data.
In an embodiment of the present application, the conference related data may include document data, screen flow data, and the like.
In the embodiment of the application, after the main control module acquires the remote audio/video data, the local audio/video data and the conference related data, the main control module can determine the data sent to the conference processing system and the video wall processing system according to the conference requirement of the type of the data after preprocessing the remote audio/video data, the local audio/video data and the conference related data so as to display conference pictures and video wall pictures simultaneously.
In the embodiment of the application, the audio acquisition module is in communication connection with the main control system and is used for acquiring local audio data, and the video acquisition module is in communication connection with the main control system and is used for acquiring local video data. Specifically, the local audio and video data can be collected through the local audio collection module and the local video collection module and sent to the main control system to generate an audio file and a video file.
Further, the device may further include an audio source identification module configured to identify a main audio source of the current environment according to the image of the surrounding environment acquired by the video acquisition module, and the audio acquisition module is further configured to acquire an audio signal corresponding to the audio source identified by the audio source identification module. Under the condition that the number of participants is large, the voice of speaking or discussion in the environment is too loud, and the audio data of the current speaker cannot be correctly collected, so that the current speaker can be determined through the image collected by the video collection module, and the audio collection can be carried out through the audio collection module corresponding to the current speaker in a mode of closing or reducing the audio collection modules corresponding to other participants.
The audio acquisition module can be a wireless microphone, a wired microphone, a gooseneck microphone and the like. The audio acquisition module can comprise a plurality of audio acquisition modules which are respectively arranged around different participants and are used for acquiring audio data of the different participants
In an embodiment of the present application, the apparatus includes:
and the noise reduction module is in communication connection with the main control system and is used for carrying out noise reduction processing on the local audio data and the environmental noise in the remote audio data and removing the interference of the environmental noise on the audio data.
In the embodiment of the application, the equipment can also comprise an echo cancellation module which is in communication connection with the main control system and is used for canceling the echo in the local audio data and the remote audio data. To improve the fidelity of the sound.
In the embodiment of the application, the video acquisition module may include one or more video acquisition modules, for example, when the video acquisition module is one, the video acquisition module can be rotated to shoot each position of the meeting participants and the meeting place through omnibearing rotation, and shooting positions can be identified according to audio sources, for example, the current position to be shot is determined according to the sounds emitted by different meeting participants, and the video acquisition device is rotated to obtain corresponding pictures.
The video acquisition module can be a camera.
In the embodiment of the application, the device further comprises a communication module which is in communication connection with the main control system and is used for establishing network connection with the remote conference terminals of the conference sites participating in the conference so as to acquire remote audio and video data and conference related data sent by the remote conference terminals. So as to realize real-time bidirectional audio-video interaction.
The communication module may be a 4G communication module, a 5G, WIFI communication module, or the like.
The conference processing system comprises a first video processing module, an audio processing module and a data processing module, wherein the first video processing module is used for correspondingly processing remote video data and local video data, the audio processing module is used for correspondingly processing the remote audio data and the local audio data, and the data processing module is used for processing the conference related data.
The audio processing module is used for decoding the data through the audio decoding submodule after receiving the audio data, then placing each path of decoded audio data into each data buffer zone, taking out the data packet with the earliest time from each buffer zone queue by the audio synthesis submodule, mixing the audio data packets, and transmitting the mixed data to the voice playing equipment for playing.
The video rendering sub-module takes out data packets from each buffer area queue according to the sequence, decodes the data by the video decoding sub-module, and then renders each path of decoded video data to the video display area of the corresponding conference video display according to the current preset split screen mode.
The data processing module is used for firstly placing the data packets into a data buffer area queue when the received related data format is a video stream format, taking out the data packets from the buffer area queue according to the sequence by the data stream rendering sub-module, decoding the data by the video decoding sub-module, and rendering the decoded data stream to a data display area. If the received related data format is a picture format, the data stream rendering sub-module can directly render the picture to the data display area of the corresponding conference video display.
In the embodiment of the application, the television wall processing system comprises a second video processing module which is used for correspondingly processing the remote video data. The video rendering sub-module takes out data packets from each buffer area queue according to the sequence, decodes the data by the video decoding sub-module, and then renders each path of decoded video data to the video display area of the corresponding television wall picture respectively according to the currently set split screen mode.
In the embodiment of the application, the audio/video playing device comprises a plurality of conference video displays and a voice playing device, and the conference processing system is used for carrying out communication connection with the conference video displays through a video output interface and sending different video data to the different conference video displays according to the first playing rule for displaying.
The conference processing system is also used for being in communication connection with the voice playing device through the voice output interface and used for playing voice through the voice playing device.
Specifically, the conference processing system can put different video output pictures into a plurality of displays for playing according to different user requirements, conference project requirements and video types and numbers, for example, one path of video output displays a document or a screen stream data picture, the other path of video output displays video pictures of each conference site, or both paths of video output displays video pictures of different conference sites. To present data frames + interactive video frames, or main meeting place frames + interactive video frames,
The interactive video pictures can also be presented according to different split-screen modes, such as single split-screen, four split-screens, 16 split-screens and the like.
The voice output interface may include SDI, HDMI, DVI, CVBS, YPbPr or other interface formats.
The video output interface may include a plurality of interface forms such as HDMI, DVI, VGA, RGB, S terminals.
In the embodiment of the application, the video playing device comprises a television wall display, and the television wall processing system is used for determining a split screen mode according to the user requirement or the number of the remote video data and carrying out split screen display in a playing interface of the television wall display according to the split screen mode.
Specifically, the television wall subsystem can put television wall pictures into a plurality of displays for playing according to different user requirements and different project requirements in a way of outputting one or more paths of videos, and the displays can also display the television wall video pictures according to different split-screen modes, such as 16 split-screens, 25 split-screens, 36 split-screens, 64 split-screens and the like.
Further, the number of the split screens of the television wall processing system is larger than that of the conference system.
In embodiments of the present application, the television wall display may include a plurality of television wall displays.
In the embodiment of the application, the split screen mode can be switched according to the number of the participating users in real time, for example, in some scenes, the users with partial conference sites may have network delay or other conditions, so that the users gradually join in the conference after the conference starts, at this time, when the split screen interface of the television wall cannot meet the requirement of presenting all the video of the conference sites, the split screen module can be replaced, and when the split screen mode is maximum, that is, the number of split screens reaches the limit, at this time, another television wall display can be started for displaying the television wall interface.
In the embodiment of the application, the remote video data can also comprise terminal identifiers, each split screen interface in the television wall can respectively establish a mapping relation with the identifiers of different participant terminals, and when the corresponding video data is acquired, the corresponding split screen interface can be used for displaying.
In the implementation of the application, the video data displayed on each split screen interface in the video wall can be determined according to the sequence of joining the conference by different joining user terminals, the video data of the first joining user terminal is displayed on the first split screen interface, the video data of the second joining user terminal is displayed on the second split screen interface, and the like until all the video data are displayed.
In the embodiment of the application, the conference terminal and video wall integrated equipment can further comprise a subtitle conversion module, wherein the subtitle conversion module is used for converting local audio data and remote audio data into subtitles and outputting the subtitles corresponding to corresponding video pictures, so that the current conference content can be known by watching the subtitles when the environment is noisy and the voice of a user cannot be heard.
Further, the caption may include multi-language captions, for example, chinese, english, etc., and the apparatus may further include a translation module for translating the local audio data and the remote audio data into different languages, that is, translating the voices output by each user participating in the conference into different voices, and forming captions corresponding to the different languages through the caption module, so as to synchronously display the captions in different video display interfaces, thereby realizing communication between multiple languages and improving interaction fluency.
In the embodiment of the application, the device further comprises a video recording module for recording and storing the remote audio and video data and the local audio and video data, and the device can be directly called when the meeting needs to be tidied and other led so as to trace back the meeting.
The embodiment of the application provides a conference terminal and video wall integrated device which comprises a main control system, a conference processing system and a video wall processing system, wherein the main control system is used for acquiring remote audio and video data, local audio and video data and conference related data, the conference processing system is in communication connection with the main control system and is used for receiving the remote audio and video data, the local audio and video data and the conference related data which are transmitted by the main control system, correspondingly processing the remote audio and video data, the local audio and video data and the conference related data and then transmitting the corresponding processed remote audio and video data, the local audio and video data and the conference related data to audio and video playing equipment according to a first playing rule so as to play conference pictures and conference audios, and the video wall processing system is in communication connection with the main control system and is used for receiving the remote video data transmitted by the main control system and correspondingly processing the remote video data and then transmitting the remote video data to the video playing equipment according to a second playing rule so as to play video wall pictures. In the embodiment of the application, the conference terminal and the television wall function are realized in one device, so that the cost of a user is effectively saved, the installation is simple, the user can select to simultaneously open the conference terminal and the television wall function according to the needs, and only one of the conference terminal or the television wall function can be opened, so that various requirements of the user can be met, and the user experience is improved.
In an embodiment of the application, a conference terminal and video wall integrated system is also provided, which comprises the conference terminal and video wall integrated device, a plurality of conference terminals respectively arranged at each conference site and the conference terminal and video wall integrated device, and a plurality of conference terminals respectively arranged at each conference site and the conference terminal and video wall integrated device, wherein the conference terminal and video wall integrated device is arranged at a main conference site and used for performing audio-video interaction and supervision with each conference site, and the conference terminals are used for acquiring remote audio-video data of each conference site and sending the remote audio-video data to the conference terminal and video wall integrated device for displaying. The video interaction between the main meeting place and each sub meeting place can be realized, and the interaction video picture can be displayed.
The conference terminal can comprise an audio acquisition unit and a video acquisition unit, and is used for acquiring audio and video data of different sub-conference sites, and the conference terminal and video wall integrated equipment can acquire audio and video data of a main conference site and can send the audio and video data to the conference terminal, so that bidirectional audio and video interaction is realized.
In the embodiment of the application, the system also comprises a control device for switching, scaling, deleting and the like the played picture, and the device can be started and closed. For example, when a person in a certain conference room is reporting, the video image of the conference room may be enlarged in the conference display interface or moved to a central position for display, so that the person may watch it with emphasis.
The control device can be a touch device, such as a mobile phone, a tablet computer and the like, and controls the conference terminal and the integrated device through a touch instruction input by a user.
The application can simultaneously realize the conference terminal and the television wall function in one system, effectively saves the cost of users, has simple installation, can select to simultaneously start the conference terminal and the television wall function according to the needs, can also only start one of the conference terminal or the television wall function, can meet various requirements of the users, and improves the user experience.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.