CN103237203B - Audio and video synchronization method and system based on mobile terminal - Google Patents

Abstract

The invention is suitable for the communication field, and provides an audio and video synchronization method based on a mobile terminal, which comprises the following steps: starting a camera and a microphone of the mobile terminal, and prompting a user to align the camera to a display screen of display terminal equipment communicating with the mobile terminal; simultaneously sending a preset frame picture video signal and a section of preset audio signal to display terminal equipment; capturing the time of a preset frame video signal appearing on a display screen of display terminal equipment through a camera, and recording the time as a first time point; capturing the time when a preset audio signal starts to be played on the display terminal equipment through a microphone, and recording the time as a second time point; and adjusting the time difference of the audio data and the video data sent to the display terminal equipment by the mobile terminal according to the difference value between the first time point and the second time point. The invention also provides an audio and video synchronization system based on the mobile terminal. The audio and video synchronization method and system based on the mobile terminal can realize the synchronous playing of audio and video.

Description

A mobile terminal-based audio and video synchronization method and system

technical field

The present invention relates to the communication field, in particular to an audio and video synchronization method and system based on a mobile terminal.

Background technique

At present, more and more mobile terminals, such as mobile phones and tablet computers, support technologies such as HDMI, MHL or WIFI Display, which can transmit audio data and video data on the mobile terminal to a TV for playback and display. When applying these technologies, due to the reasons of data transmission and the delay caused by the processing of the received audio data and video data by the TV, the final image displayed on the TV and the sound played will be out of sync (such as human voice does not match the shape of the mouth).

The existing common method is that the individual manually adjusts the delay time of the sound on the mobile terminal according to the effect actually seen and heard, so that the image and the sound on the TV end are synchronized.

However, the existing processing method is relatively cumbersome to operate, and the accuracy is not high.

Therefore, there is an urgent need to design a method with simple operation and high precision, so that the audio and video data sent by the mobile terminal to the TV can be synchronized when the TV plays them.

Contents of the invention

In view of this, the purpose of the embodiments of the present invention is to provide a method and system for synchronizing audio and video based on a mobile terminal, aiming at solving the problems of cumbersome operation and low accuracy of the method for synchronizing audio and video data in the prior art.

The embodiment of the present invention is achieved in this way, a method for synchronizing audio and video based on a mobile terminal, the method comprising:

Turn on the camera and microphone of the mobile terminal, and prompt the user to align the camera with the display screen of the display terminal device communicating with the mobile terminal;

Simultaneously send a video signal of a preset frame picture and a preset audio signal to the display terminal device;

Capture the time when the video signal of the preset frame picture appears on the display screen of the display terminal device through the camera, and record it as the first time point;

capturing the time at which the preset audio signal starts playing on the display terminal device through the microphone, and recording it as a second time point;

The time difference between the audio data and the video data sent by the mobile terminal to the display terminal device is adjusted according to the difference between the first time point and the second time point.

Preferably, before the step of turning on the camera and microphone of the mobile terminal and prompting the user to align the camera with the display screen of the display terminal device communicating with the mobile terminal, the step further includes:

Enable the audio and video synchronization configuration function of the mobile terminal.

Preferably, before the step of simultaneously sending a preset frame picture video signal and a preset audio signal to the display terminal device, the step further includes:

sending the video signal of the identified frame picture to the display terminal device for display;

judging whether the recognition frame picture displayed on the display terminal device is captured by the camera;

If the recognition frame picture displayed on the display terminal device is not captured, the user is prompted to adjust the position of the camera to recapture the recognition frame picture displayed on the display terminal device.

Preferably, after the step of judging whether the camera captures the identification frame picture displayed on the display terminal device, it further includes:

If the identification frame picture displayed on the display terminal device is captured, the user is prompted that the identification frame picture has been successfully captured and the next operation can be performed.

On the other hand, the present invention also provides an audio and video synchronization system based on a mobile terminal to interact with a display terminal device, wherein the system includes:

an alignment module, configured to turn on the camera and microphone of the mobile terminal, and prompt the user to align the camera with the display screen of the display terminal device communicating with the mobile terminal;

A first sending module, configured to simultaneously send a preset frame picture video signal and a preset audio signal to the display terminal device;

The first recording module is used to capture the time when the video signal of the preset frame picture appears on the display screen of the display terminal device through the camera, and record it as the first time point;

The second recording module is configured to use the microphone to capture the time when the preset audio signal starts playing on the display terminal device, and record it as a second time point;

A time difference adjusting module, configured to adjust the time difference between the audio data and the video data sent by the mobile terminal to the display terminal device through the difference between the first time point and the second time point.

Preferably, the system also includes:

The synchronous opening module is used to open the audio and video synchronous configuration function of the mobile terminal.

Preferably, the system also includes:

The second sending module is used to send the identification frame picture video signal to the display terminal device for display;

A judging module, configured to judge whether the recognition frame picture displayed on the display terminal device is captured by the camera;

The position adjustment module is configured to prompt the user to adjust the position of the camera to recapture the recognition frame picture displayed on the display terminal device if the recognition frame picture displayed on the display terminal device is not captured.

Preferably, the system also includes:

The confirming module is used for prompting the user that the identification frame picture has been successfully captured and the next operation can be performed if the identification frame picture displayed on the display terminal device is captured.

In the embodiment of the present invention, the technical solution provided by the present invention automatically adjusts the time difference between the audio data and the video data sent by the mobile terminal to the display terminal device by calculating the delay of audio and video data after transmission and processing by the display terminal device The value can make the audio and video data sent by the mobile terminal to the display terminal device achieve synchronization effect when the display terminal device plays, and does not require manual operation by the user, and also has the advantage of high accuracy, thereby improving the user experience.

Description of drawings

Fig. 1 is the flow chart of the audio-video synchronization method based on mobile terminal in one embodiment of the present invention; And

FIG. 2 is a schematic structural diagram of an audio-video synchronization system based on a mobile terminal in an embodiment of the present invention.

detailed description

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

The specific embodiment of the present invention provides a kind of audio-video synchronous method based on mobile terminal, as shown in Figure 1, mainly comprises the following steps:

S101. Turn on the camera and microphone of the mobile terminal, and prompt the user to align the camera with the display screen of the display terminal device communicating with the mobile terminal;

S102. Simultaneously send a preset frame video signal and a preset audio signal to the display terminal device;

S103. Use the camera to capture the time when the video signal of the preset frame picture appears on the display screen of the display terminal device, and record it as a first time point;

S104. Use the microphone to capture the time when the preset audio signal starts playing on the display terminal device, and record it as a second time point;

S105. Adjust the time difference between the audio data and the video data sent by the mobile terminal to the display terminal device according to the difference between the first time point and the second time point.

An audio-video synchronization method based on a mobile terminal provided by the present invention automatically adjusts the ratio between the audio data and the video data sent by the mobile terminal to the display terminal device by calculating the delay of the audio-video data after transmission and processing by the display terminal device. The time difference can make the audio and video data sent by the mobile terminal to the display terminal device achieve a synchronous effect when the display terminal device plays, and does not require manual operation by the user, and also has the advantage of high accuracy, thereby improving the user's experience.

A mobile terminal-based audio-video synchronization method provided by the present invention will be described in detail below.

Please refer to FIG. 1 , which is a flowchart of a method for synchronizing audio and video based on a mobile terminal in an embodiment of the present invention. In this embodiment, the method is implemented by a mobile terminal.

In step S101, the camera and the microphone of the mobile terminal are turned on, and the user is prompted to aim the camera at the display screen of the display terminal device communicating with the mobile terminal.

In this embodiment, the step of enabling the audio and video synchronization configuration function of the mobile terminal is also included before step S101. Realize, that is to say, when the user turns on the camera and microphone of the mobile terminal, the audio and video synchronization configuration function of the mobile terminal is also automatically turned on and activated, so that the user's operation steps can be simplified.

In this embodiment, the display terminal device and the mobile terminal maintain a communication connection all the time. After the camera and microphone of the mobile terminal are turned on by the user, the user may be prompted to point the camera at the display terminal device communicating with the mobile terminal by popping up a dialog box. The display screen, so that the camera can capture the screen displayed by the display terminal device.

In step S102, a preset frame picture video signal and a preset audio signal are sent to the display terminal device at the same time.

In this embodiment, the step of adjusting the position of the camera is also included before step S102, that is: sending the identification frame picture video signal to the display terminal device for display; judging whether the camera captures the identification frame picture displayed on the display terminal device; if If the recognition frame picture displayed on the display terminal device is not captured, the user is prompted to adjust the position of the camera to recapture the recognition frame picture displayed on the display terminal device. In this embodiment, the identification frame picture can be a frame picture of a single color, such as a frame of blue picture, and this frame of blue picture is sent to the display terminal device for display, and then the camera is used to capture the picture displayed on the display terminal device. If it is not captured, it means that the camera position of the mobile terminal is not directly facing the display screen of the display terminal device. At this time, the user should be prompted to adjust the position of the camera to recapture the display screen of the display terminal device. For the blue screen displayed, the specific prompting method can also prompt the user to make adjustments in the form of a pop-up dialog box. In this embodiment, after the step of judging whether the camera has captured the identification frame picture displayed on the display terminal device, it further includes: if the identification frame picture displayed on the display terminal device is captured, prompting the user that the identification frame picture has been successfully identified Captured can proceed to the next step. In this embodiment, the step of adjusting the position of the camera can help the user to position the mobile terminal in a timely and accurate manner.

In this embodiment, step S102 simultaneously sends a preset frame picture video signal and a preset audio signal to the display terminal device, wherein the preset frame picture can also be a single-color frame picture, but the preset frame picture and the identification frame The pictures should be in different colors, for example, the preset frame picture is a frame of red picture.

In step S103, the camera captures the time when the video signal of the preset frame picture appears on the display screen of the display terminal device, and records it as the first time point.

In this embodiment, if the video signal of the preset frame picture is captured by the camera and appears on the display screen of the display terminal device at 12:20:12, the first recorded time point is 12:20:12 Of course, in order to further improve the accuracy of the test, the minimum unit of time of the mobile terminal can also be adjusted to millisecond level, for example, the first time point recorded can be 12:20:12:20 milliseconds.

In step S104, the time at which the preset audio signal starts to be played by the display terminal device is captured by the microphone, and recorded as a second time point.

In this embodiment, if the time when the preset audio signal is captured by the microphone and played on the display terminal device is 12:20:15, the second recorded time point is 12:20:15. Similarly, In order to further improve the accuracy of the test, the minimum unit of time of the mobile terminal may also be adjusted to millisecond level, for example, the recorded second time point may be 12:20:15:50 milliseconds.

In step S105, the time difference between the audio data and the video data sent by the mobile terminal to the display terminal device is adjusted according to the difference between the first time point and the second time point.

In this embodiment, if the recorded first time point can be 12:20:12:20 milliseconds, and the recorded second time point can be 12:20:15:50 milliseconds, the mobile terminal automatically calculates the first The difference between the time point and the second time point is 3 seconds and 30 milliseconds, so that the delay of the sound played on the display terminal device relative to the picture is 3 seconds and 30 milliseconds, and then the mobile terminal will The value automatically adjusts the time difference between the audio data and video data sent to the display terminal device, that is to say, the video data is sent after 3 seconds and 30 milliseconds, so that it is synchronized with the corresponding audio data when it is played by the display terminal device.

In other implementations, in order to further improve the accuracy of synchronous playback of audio and video, the accuracy of calculating the difference between the first time point and the second time point can also be improved by obtaining the average value through multiple tests, thereby improving the user's For experience, the way of multiple tests is to repeat the test steps S102 to S105, which will not be repeated here.

An audio-video synchronization method based on a mobile terminal provided by the present invention automatically adjusts the ratio between the audio data and the video data sent by the mobile terminal to the display terminal device by calculating the delay of the audio-video data after transmission and processing by the display terminal device. The time difference can make the audio and video data sent by the mobile terminal to the display terminal device achieve a synchronous effect when the display terminal device plays, and does not require manual operation by the user, and also has the advantage of high accuracy, thereby improving the user's experience.

The specific embodiment of the present invention also provides an audio-video synchronization system 20 based on a mobile terminal, which interacts with a display terminal device 30, as shown in FIG. 2 , wherein the audio-video synchronization system based on a mobile terminal mainly includes:

An alignment module 201, configured to turn on the camera and microphone of the mobile terminal, and prompt the user to align the camera with the display screen of the display terminal device communicating with the mobile terminal;

The first sending module 202 is configured to simultaneously send a preset frame picture video signal and a preset audio signal to the display terminal device;

The first recording module 203 is configured to use the camera to capture the time when the video signal of the preset frame picture appears on the display screen of the display terminal device, and record it as a first time point;

The second recording module 204 is configured to use the microphone to capture the time when the preset audio signal starts playing on the display terminal device, and record it as a second time point;

The time difference adjusting module 205 is configured to adjust the time difference between the audio data and the video data sent by the mobile terminal to the display terminal device according to the difference between the first time point and the second time point.

The audio and video synchronization system 20 based on the mobile terminal provided by the present invention automatically adjusts the time of audio data and video data sent by the mobile terminal to the display terminal device by calculating the delay of the audio and video data after transmission and processing by the display terminal device The difference can make the audio and video data sent by the mobile terminal to the display terminal device achieve a synchronous effect when the display terminal device plays, and does not require manual operation by the user. It also has the advantage of high accuracy, thereby improving the user experience. .

A mobile terminal-based audio-video synchronization system 20 provided by the present invention will be described in detail below.

Please refer to FIG. 2 , which is a schematic structural diagram of an audio-video synchronization system 20 based on a mobile terminal in an embodiment of the present invention. In this embodiment, the mobile terminal-based audio and video synchronization system 20 interacts with the display terminal device 30, wherein the mobile terminal-based audio and video synchronization system 20 includes an alignment module 201, a first sending module 202, and a first recording module 203 , a second recording module 204 , a time difference adjusting module 205 , a synchronization enabling module 206 , a second sending module 207 , a judging module 208 , a position adjusting module 209 and a confirming module 210 . In this implementation manner, the mobile terminal may be a mobile phone, a tablet computer, etc., which is not limited here.

The alignment module 201 is configured to turn on the camera and microphone of the mobile terminal, and prompt the user to align the camera with the display screen of the display terminal device 30 communicating with the audio-video synchronization system 20 based on the mobile terminal.

In this embodiment, the synchronization opening module 206 is used to enable the audio and video synchronization configuration function of the mobile terminal. In other embodiments, enabling the audio and video synchronization configuration function of the mobile terminal may also be realized by turning on the camera and the microphone of the mobile terminal. That is to say, when the user turns on the camera and the microphone of the mobile terminal, the audio and video synchronization configuration function of the mobile terminal is also automatically turned on and activated, thus simplifying the user's operation steps.

In this embodiment, the display terminal device 30 maintains a communication connection with the audio-video synchronization system 20 based on the mobile terminal. After the camera and the microphone of the mobile terminal are turned on by the user, the user may be prompted to align the camera with a pop-up dialog box. The display screen of the display terminal device 30 communicates with the audio-video synchronization system 20 based on the mobile terminal, so that the camera can capture the picture displayed by the display terminal device 30 conveniently.

The second sending module 207 is configured to send the video signal of the identified frame picture to the display terminal device 30 for display.

The judging module 208 is configured to judge whether the identification frame picture displayed on the display terminal device 30 is captured by the camera.

The position adjustment module 209 is configured to prompt the user to adjust the position of the camera to recapture the recognition frame picture displayed on the display terminal device 30 if the recognition frame picture displayed on the display terminal device 30 is not captured.

In this embodiment, the identification frame picture can be a frame picture of a single color, such as a frame of blue picture, and the second sending module 207 sends this frame of blue picture to the display terminal device 30 for display, and then uses the camera to capture The blue picture displayed on the display terminal device 30, the judging module 208 judges whether the camera captures it, if not, it indicates that the camera position of the mobile terminal is not facing the display screen of the display terminal device 30, and the user will be prompted to adjust The position of the camera is to recapture the blue picture displayed on the display terminal device 30, and the specific prompting method can also prompt the user to adjust by means of a pop-up dialog box. In this embodiment, the position adjustment module 209 adjusts the position of the camera to help the user adjust the position of the mobile terminal in a timely and accurate manner.

The confirmation module 210 is configured to prompt the user that the identification frame picture has been successfully captured and proceed to the next operation if the identification frame picture displayed on the display terminal device 30 is captured.

The first sending module 202 is configured to send a preset frame picture video signal and a preset audio signal to the display terminal device 30 at the same time. In this embodiment, the preset frame picture can also be a frame picture of a single color, but the preset frame picture and the identification frame picture should be of different colors, for example, the default frame picture is a frame of red picture.

The first recording module 203 is configured to capture the time when the video signal of the preset frame picture appears on the display screen of the display terminal device 30 through the camera, and record it as the first time point.

In this embodiment, if the video signal of the preset frame picture is captured by the camera and appears on the display screen of the display terminal device 30 at 12:20:12, the first recorded time point is 12:20:12 Seconds, of course, in order to further improve the accuracy of the test, the smallest unit of time of the mobile terminal can also be adjusted to the millisecond level, for example, the first recorded time point can be 12:20:12:20 milliseconds.

The second recording module 204 is configured to capture the time when the preset audio signal starts to be played by the display terminal device 30 through a microphone, and record it as a second time point.

In this embodiment, if the time when the preset audio signal is captured by the microphone and played on the display terminal device 30 is 12:20:15, the second recorded time point is 12:20:15. Similarly , in order to further improve the accuracy of the test, the minimum unit of time of the mobile terminal may also be adjusted to millisecond level, for example, the recorded second time point may be 12:20:15:50 milliseconds.

The time difference adjusting module 205 is configured to adjust the time difference between the audio data and the video data sent by the mobile terminal to the display terminal device 30 through the difference between the first time point and the second time point.

In this embodiment, if the recorded first time point can be 12:20:12:20 milliseconds, and the recorded second time point can be 12:20:15:50 milliseconds, the mobile terminal automatically calculates the first The difference between the time point and the second time point is 3 seconds and 30 milliseconds, so that the delay of the sound played on the display terminal device 30 relative to the picture is 3 seconds and 30 milliseconds, and then the mobile terminal calculates the The difference automatically adjusts the time difference between the audio data and video data sent to the display terminal device 30 , that is to say, the video data is sent after 3 seconds and 30 milliseconds, so that it is synchronized with the corresponding audio data when it is played by the display terminal device 30 .

In other implementations, in order to further improve the accuracy of synchronous playback of audio and video, the accuracy of calculating the difference between the first time point and the second time point can also be improved by obtaining the average value through multiple tests, thereby improving the user's Experience, the way of multiple tests is to repeat the operation of the modules 202 to 205 multiple times, which will not be repeated here.

A mobile terminal-based audio-video synchronization system 20 provided by the present invention automatically adjusts the audio data and audio data sent by the mobile terminal to the display terminal device 30 by calculating the delay of the audio-video data after transmission and processing by the display terminal device 30. The time difference of the video data can make the audio and video data sent by the mobile terminal to the display terminal device 30 achieve a synchronous effect when the display terminal device 30 plays, and does not require manual operation by the user, and also has the advantage of high accuracy , thereby improving the user experience.

In the embodiment of the present invention, the technical solution provided by the present invention automatically adjusts the time difference between the audio data and the video data sent by the mobile terminal to the display terminal device by calculating the delay of audio and video data after transmission and processing by the display terminal device The value can make the audio and video data sent by the mobile terminal to the display terminal device achieve synchronization effect when the display terminal device plays, and does not require manual operation by the user, and also has the advantage of high accuracy, thereby improving the user experience.

It is worth noting that in the above embodiments, the units included are only divided according to the functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are also It is only for the convenience of distinguishing each other, and is not intended to limit the protection scope of the present invention.

In addition, those of ordinary skill in the art can understand that all or part of the steps in the methods of the above-mentioned embodiments can be completed by instructing related hardware through programs, and the corresponding programs can be stored in a computer-readable storage medium. Storage media, such as ROM/RAM, magnetic disk or optical disk, etc.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. within range.

Claims (8)

1. based on an audio and video synchronization method for mobile terminal, it is characterized in that, described method comprises:

Open camera and the microphone of described mobile terminal, and point out user that described camera is aimed at the display screen with the display terminal of described communication of mobile terminal;

Send simultaneously and preset frame picture video signal and one section of preset audio signal extremely described display terminal;

By the time that the described default frame picture video signal of described camera seizure occurs at the display screen of described display terminal, and be recorded as very first time point;

Catch by described microphone the time that described preset audio signal starts in described display terminal to play, and be recorded as the second time point;

When sending voice data and video data to described display terminal, adjusting described mobile terminal by described very first time point with the difference of described second time point and sending to the voice data of described display terminal and the time difference of video data.

2. as claimed in claim 1 based on the audio and video synchronization method of mobile terminal, it is characterized in that, the camera of the described mobile terminal of described unlatching and microphone, and point out user described camera is aimed at the step of the display screen of the display terminal of described communication of mobile terminal before also comprise:

Open the audio-visual synchronization configuration feature of described mobile terminal.

3., as claimed in claim 1 based on the audio and video synchronization method of mobile terminal, it is characterized in that, described send simultaneously preset frame picture video signal and one section of preset audio signal to described display terminal step before also comprise:

Send and identify that frame picture video signal to described display terminal shows;

Judge whether capture by described camera the identification frame picture shown in described display terminal;

If do not capture the identification frame picture shown in described display terminal, then the position of pointing out user to adjust described camera catches the identification frame picture shown in described display terminal again.

4., as claimed in claim 3 based on the audio and video synchronization method of mobile terminal, it is characterized in that, described judgement is also comprised whether captured the step of the identification frame picture shown in described display terminal by described camera after:

If capture the identification frame picture shown in described display terminal, then point out user identify frame picture successfully capture can carry out next step operation.

5., based on an audio-visual synchronization system for mobile terminal, mutual with display terminal, it is characterized in that, described system comprises:

Alignment modules, for opening camera and the microphone of mobile terminal, and points out user that described camera is aimed at the display screen with the described display terminal of described communication of mobile terminal;

First sending module, for sending default frame picture video signal and one section of preset audio signal extremely described display terminal simultaneously;

First logging modle, for being caught the time that described default frame picture video signal occurs at the display screen of described display terminal by described camera, and is recorded as very first time point;

Second logging modle, for being caught the time that described preset audio signal starts in described display terminal to play by described microphone, and is recorded as the second time point;

Time difference adjusting module, for when sending voice data and video data to described display terminal, adjusting described mobile terminal by described very first time point with the difference of described second time point and sending to the voice data of described display terminal and the time difference of video data.

6., as claimed in claim 5 based on the audio-visual synchronization system of mobile terminal, it is characterized in that, described system also comprises:

Synchronous opening module, for opening the audio-visual synchronization configuration feature of described mobile terminal.

7., as claimed in claim 5 based on the audio-visual synchronization system of mobile terminal, it is characterized in that, described system also comprises:

For sending, second sending module, identifies that frame picture video signal to described display terminal shows;

Judge module, for judging whether capture by described camera the identification frame picture shown in described display terminal;

Position adjusting type modules, if for not capturing the identification frame picture shown in described display terminal, then the position of pointing out user to adjust described camera catches the identification frame picture shown in described display terminal again.

8., as claimed in claim 7 based on the audio-visual synchronization system of mobile terminal, it is characterized in that, described system also comprises:

Confirm module, if for capturing the identification frame picture shown in described display terminal, then point out user identify frame picture successfully capture can carry out next step operation.