TWI770762B - Audio and visual system and control method thereof - Google Patents

Abstract

An audio and visual system includes a speaker device, a camera device. The speaker device includes an ultrasound emitter. The camera device includes an ultrasound receiver and an audio source tracking circuit. The ultrasound emitter is configured to output an ultrasound signal. The ultrasound receiver is configured to receive the ultrasound signal, and, according to the ultrasound signal, output a notification signal. The audio source tracking circuit is configured, according to the notification signal, to output a tracking signal, in which the camera device is configured, according to the tracking signal, not to track the speaker device or to turn to track a sound source different from the speaker device. A controlling method of the audio and visual system is also disclosed herein.

Description

Audio-visual system and its control method

The present disclosure relates to an audio-visual system, and more particularly, to an audio-visual system related to the way of tracking speakers.

Under the circumstance that the conventional fixed camera cannot cover all the monitoring range, because the rotating zoom camera has a larger shooting range, the rotating zoom camera has gradually replaced the fixed camera or cooperated with the fixed camera to obtain better tracking Effect. In a video conference, the camera is usually tracked and automatically aimed at the speaker by judging the sound source of the local end, so that the remote conference participants can clearly see each other. However, in actual use, because the speaker at the local end will play the sound from the far end, the local end determines that the speaker is the speaker, and an abnormal situation occurs that the camera is aimed at the speaker. Therefore, how to make the lens operate effectively and normally has become one of the research and development issues in the current related fields.

An embodiment of the present disclosure relates to an audio-visual system, which includes a speaker device and a camera device. The speaker device includes a Ultrasonic transmitter. The photographing device includes an ultrasonic receiver and an audio source tracking circuit. The ultrasonic transmitter is used for outputting an ultrasonic signal; the ultrasonic receiver is used for receiving the ultrasonic signal and outputting a notification signal according to the ultrasonic signal; and the audio source tracking circuit is used for outputting a tracking signal according to the notification signal, wherein the photographing device is used for according to the tracking signal Do not track the speaker unit or turn to track a source other than the speaker unit.

An embodiment of the present disclosure relates to a control method of an audio-visual system. The control method of the audio-visual system includes: outputting an ultrasonic signal through an ultrasonic transmitter disposed in a speaker device; The ultrasonic receiver receives the ultrasonic signal, and outputs a notification signal according to the ultrasonic signal; outputs a tracking signal through an audio source tracking circuit arranged in the photographing device according to the notification signal; and controls the photographing device not to track or turn the speaker device according to the tracking signal A sound source other than a speaker unit.

100: AV system

110: Internet

112: Computer

114: Network Receiver

116: Photographic installations

118: Audio source tracking circuit

120: Ultrasonic receiver

122: Speaker unit

124: Ultrasonic Transmitter

200: AV system

226: Resampler

228: Audio processing circuit

230: Computational Circuits

232: Audio Receiver

400: AV system

434: Locator

436: Masker

S302, S304, S306, S308, S310, S312, S314, S316, S318, S502, S504, S506, S508, S510, S512: Steps

US: Ultrasonic signal

NS: notification signal

TS: Tracking Signal

MS: Microphone signal

RS: reference signal

AS: audio signal

FAS: Far-end audio signal

APS: Audio Processing Signal

RNS: Regional Notification Signal

Aspects of the present disclosure are best understood from the following detailed description when read in conjunction with the accompanying drawings. It should be noted that in accordance with standard practice in the industry, the various features are not drawn to scale. In fact, the dimensions of the various features may be arbitrarily increased or decreased for clarity of discussion.

FIG. 1 is a block diagram of an audio-visual system according to some embodiments of the present disclosure.

FIG. 2 is a block diagram of an audio-visual system according to some embodiments of the present disclosure.

FIG. 3 is a flowchart illustrating a control method for an audio-visual system according to some embodiments of the present disclosure.

FIG. 4 is a block diagram of an audio-visual system according to some embodiments of the present disclosure.

FIG. 5 is a flowchart illustrating a control method for an audio-visual system according to some embodiments of the present disclosure.

Terms used in this specification generally have their ordinary meanings in the technical field and specific context in which each term is used. The use of examples in this specification, including examples of any items discussed herein, are illustrative only and in no way limit the scope and meaning of the disclosure or any exemplary term. Likewise, the present disclosure is not limited to the embodiments given in this specification.

As used herein, "comprises," "includes," "has," "includes," "involves," and similar words are to be construed as variable, ie, not limited to, but mean to include.

References throughout this specification to "one embodiment," "an embodiment," or "some embodiments" mean that a particular feature, structure, implementation, or characteristic associated with the embodiment is included in at least one embodiment of the present disclosure. Thus, usages of the phrases "in one embodiment," "in one embodiment," or "in some embodiments" in various places throughout this specification may not necessarily all refer to the same embodiment. Furthermore, the particular features, structures, implementations or characteristics may be combined in any suitable manner in one or more embodiments.

FIG. 1 is a block diagram of an audio-visual system 100 according to some embodiments of the present disclosure. As shown in FIG. 1 , in some embodiments, the audio-visual system 100 includes a network 110 , a computer 112 , a camera device 116 and a speaker device 122 . The computer 112 is used to receive the transmission from the network 110 The far-end audio signal FAS is transmitted to the speaker device 122 through the general serial bus channel (not shown). The speaker device 122 is used for playing sound according to the audio signal AS. The computer 112 is used for capturing images received by the camera 116 through a general serial bus channel (not shown). In some embodiments, the above-mentioned photographing device 116 may be implemented by, for example, but not limited to, a pan/Tilt/Zoom camera (PTZ camera).

As shown in FIG. 1 , in some embodiments, computer 112 includes a network receiver 114 . The network receiver 114 is used for receiving the far-end audio signal FAS transmitted from the network 110, and transmitting the audio signal AS to the speaker device 122 through the general serial bus channel (not shown).

In a common practice, the photographing device 116 is used to determine the source of the sound and can automatically aim at the person speaking, so that the person participating in the remote conference clearly knows the person who is currently speaking. However, when the local speaker device 122 is used to play the audio signal AS sent from the computer 112 , the local camera device 116 may determine that the speaker device 122 is the speaker, causing the camera device 116 to turn to the speaker device 122 unexpectedly.

Compared with the above method, in the embodiment of the present invention, the speaker device 122 is used to notify the camera device 116 through ultrasonic waves as a medium, so as to prevent the camera device 116 from tracking the speaker device 122 . In some embodiments, the speaker device 122 is used to determine whether it plays a sound, and notify the camera device 116 not to track through ultrasonic waves. In some embodiments, the speaker device 122 is used to continuously notify the camera device 116 through ultrasonic waves, so that the camera device 116 determines the position of the speaker device 122 and only Areas other than the speaker device 122 are allowed to be tracked in a manner that will be described with reference to the embodiments shown below.

In some embodiments, as shown in FIG. 1 , the camera device 116 includes an audio source tracking circuit 118 and an ultrasonic receiver 120 , and the speaker device 122 includes an ultrasonic transmitter 124 . The ultrasonic transmitter 124 is used for outputting the ultrasonic signal US to the ultrasonic receiver 120 , and the ultrasonic receiver 120 is used for generating the notification signal NS according to the ultrasonic signal US. The audio source tracking circuit 118 is used for outputting the tracking signal TS according to the notification signal NS. In some embodiments, the camera device 116 is adapted to track a sound source different from the speaker device 122 according to the tracking signal TS. In other embodiments, the camera device 116 is configured to not track the speaker device 122 according to the tracking signal TS.

FIG. 2 is a block diagram of an audio-visual system 200 according to some embodiments of the present disclosure. In some embodiments, the speaker device 122 shown in FIG. 2 can be implemented by, for example, but not limited to, one or more pieces of speakers, and the one or more pieces of speakers can be used to play the audio signal AS.

Regarding the embodiment of Fig. 1, for ease of understanding, like elements in Fig. 2 are designated with the same reference numerals. Compared with the embodiment shown in FIG. 1 , the speaker device 122 in the audio-visual system 200 in FIG. 2 further includes a resampler 226 , an audio processing circuit 228 , a computing circuit 230 and an audio receiver 232 .

In some embodiments, as shown in FIG. 2 , the audio receiver 232 is used to receive the sound produced by the speaker device 122 playing the audio signal AS to generate the microphone signal MS. In some embodiments, audio reception The device 232 may be used to receive sound from one or more speakers.

In some embodiments, the resampler 226 is used to resample the audio signal AS received by the speaker device 122 to generate the reference signal RS.

In some embodiments, the audio processing circuit 228 is configured to receive at least one of the microphone signal MS generated by the audio receiver 232 and the reference signal RS generated by the resampler 226, and perform echo processing on at least one of them, and The audio processing signal APS is output to the calculation circuit 230 . In other embodiments, the audio processing circuit 228 receives both the reference signal RS and the microphone signal MS, performs echo processing on the microphone signal MS and the reference signal RS, and outputs the audio processing signal APS to the computing circuit 230 . In some embodiments, examples of audio processing circuitry 228 include, but are not limited to, Acoustic Echo Cancellation (AEC), Acoustic Echo Suppression (AES), and the like.

In some embodiments, the computing circuit 230 is configured to receive the audio processing signal APS to determine whether the speaker device 122 emits sound. In some embodiments, when the computing circuit 230 determines that the speaker device 122 emits sound, the ultrasonic transmitter 124 in the speaker device 122 is used to output the ultrasonic signal US to the ultrasonic receiver 120 of the camera device 116, so that the camera device 116 does not respond to the speaker device 122 to track. In some embodiments, when the computing circuit 230 determines that the speaker device 122 does not emit sound, the ultrasonic transmitter 124 in the speaker device 122 will not output the ultrasonic signal US, and the camera device 116 is still tracking.

FIG. 3 is a flowchart illustrating a control method of an audio-visual system according to some embodiments of the present disclosure. It should be understood that, unless the sequence of the steps mentioned in this embodiment is specifically stated, the sequence of the steps may be adjusted according to actual needs, and may even be performed simultaneously or partially simultaneously. In some embodiments, the AV system control method shown in FIG. 3 can be applied to the AV system 200 shown in FIG. 2 , but is not limited thereto. For the sake of clarity, the following control method of the audio-visual system in FIG. 3 is described in conjunction with the audio-visual system 200 shown in FIG. 2 .

First, in step S302, the resampler 226 resamples the audio signal AS to generate the reference signal RS.

Next, in step S304, the audio receiver 232 receives the sound of the speaker device 122 to generate the microphone signal MS.

Next, in step S306 , the audio processing circuit 228 performs echo processing on at least one of the microphone signal MS and the reference signal RS to output the audio processing signal APS to the computing circuit 230 . As described above, in some embodiments, the audio processing circuit 228 performs echo processing on both the microphone signal MS and the reference signal RS to output the audio processing signal APS to the computing circuit 230 , thereby reducing the operating environment of the computing circuit 230 interference from sound.

In such a case, in step S308, the calculation circuit 230 determines whether the speaker device 122 emits sound. If the speaker device 122 emits sound, the flow goes to step S310 , and if the speaker device 122 does not emit sound, the flow goes to step S318 .

When the speaker device 122 emits sound, in step S310, The ultrasonic transmitter 124 in the speaker device 122 outputs the ultrasonic signal US to the imaging device 116 .

Furthermore, in step S312 , the ultrasonic receiver 120 receives the ultrasonic signal US, and outputs the notification signal NS to the audio source tracking circuit 118 .

Next, in step S314 , the audio source tracking circuit 118 outputs the tracking signal TS according to the notification signal NS and transmits it to the photographing device 116 .

Then, in step S316, the camera device 116 does not track the speaker device 122 according to the tracking signal TS.

On the other hand, when the speaker device 122 does not emit sound, the ultrasonic transmitter 124 does not output the ultrasonic signal US in step S318. Therefore, the photographing device 116 keeps tracking the target.

FIG. 4 is a block diagram of an audio-visual system 400 according to some embodiments of the present disclosure. As shown in FIG. 4 , the audio-visual system 400 also includes the ultrasonic transmitter 124 shown in FIG. 2 . In some embodiments, compared with the embodiment shown in FIG. 2 , the ultrasonic transmitter 124 in the audio-visual system 400 is used to continuously output the ultrasonic signal US to the ultrasonic receiver 120 . In other embodiments, if the speaker device 122 shown in FIG. 4 is turned off and the audio signal AS is not played, the ultrasonic transmitter 124 does not output the ultrasonic signal US, and the photographing device 116 can continue to track the target object.

Regarding the embodiment of Fig. 1, for ease of understanding, like elements in Fig. 4 are designated by the same reference numerals. Compared with the embodiment shown in FIG. 1 , the photographing device 116 in the audio-visual system 400 in FIG. 4 further includes a positioner 434 and a mask 436 . The locator 434 is used to receive the ultrasonic connection The notification signal NS output by the receiver 120 . The masker 436 is used to determine the area in which the speaker device 122 is not to be tracked.

In some embodiments, the locator 434 is used to determine the relative position of the speaker device 122 and the camera device 116 according to the notification signal NS, such as, but not limited to, a horizontal angle, a vertical angle and a distance.

In some embodiments, the masker 436 is configured to output a region notification signal RNS about the masked region of the speaker device 122 according to the relative position, wherein the masked region is an area where the camera device 116 does not track the speaker device 122, and the masked region The range of the region is, for example, but not limited to, plus or minus 10 degrees horizontally and plus or minus 10 degrees vertically, and the range of other mask regions falls within the scope of this disclosure.

In some embodiments, the audio source tracking circuit 118 is configured to receive the region notification signal RNS to output the tracking signal TS to the camera device 116 . In other embodiments, the audio source tracking circuit 118 is used to determine the relative position of the camera device 116 and the speaker device 122 and the range of the mask area according to the notification signal NS.

FIG. 5 is a flowchart illustrating a control method for an audio-visual system according to some embodiments of the present disclosure. It should be understood that, unless the sequence of the steps mentioned in this embodiment is specifically stated, the sequence of the steps may be adjusted according to actual needs, and may even be performed simultaneously or partially simultaneously. The AV system control method shown in FIG. 5 can be applied to the AV system 400 shown in FIG. 4 , but is not limited thereto. For the sake of clarity, the following control method of the audio-visual system shown in FIG. 5 is described in conjunction with the audio-visual system 400 shown in FIG. 4 .

First, in step S502 , the ultrasonic transmitter 124 outputs the ultrasonic signal US, and transmits the ultrasonic signal US to the ultrasonic receiver 120 .

Next, in step S504 , the ultrasonic receiver 120 receives the ultrasonic signal US, and outputs a notification signal NS to the localizer 434 .

Next, in step S506, the locator 434 receives the notification signal NS, and determines the relative position of the speaker device 122 and the photographing device 116 according to the notification signal NS.

Next, in step S508 , the masker 436 outputs a region notification signal RNS about the masked area of the speaker device 122 to the audio source tracking circuit 118 according to the relative position.

Then, in step S510 , the audio source tracking circuit 118 is used for receiving the region notification signal RNS to output the tracking signal TS to the photographing device 116 .

Then, in step S512, the camera device 116 turns to a sound source different from the speaker device 122 according to the tracking signal TS.

The foregoing outlines features of several embodiments so that those skilled in the art may better understand aspects of the present disclosure. Those skilled in the art should appreciate that the present disclosure may be readily utilized as a basis for designing or modifying other processes and structures for carrying out the same purposes and/or achieving the same advantages of the embodiments described herein. Those skilled in the art should also realize that such equivalent constructions do not depart from the spirit and scope of the present disclosure, and that various changes, substitutions and alterations herein can be made without departing from the spirit and scope of the present disclosure.

100: AV system

110: Internet

112: Computer

114: Network Receiver

116: Photographic installations

118: Audio source tracking circuit

120: Ultrasonic receiver

122: Speaker unit

124: Ultrasonic Transmitter

US: Ultrasonic signal

NS: notification signal

TS: Tracking Signal

AS: audio signal

FAS: Far-end audio signal

Claims (12)

An audio-visual system, comprising: a speaker device, comprising: an ultrasonic transmitter, wherein the ultrasonic transmitter is used to output an ultrasonic signal, wherein when the speaker device emits a sound according to an audio signal, the ultrasonic transmitter outputs the ultrasonic signal; A photographing device, comprising: an ultrasonic receiver for receiving the ultrasonic signal and outputting a notification signal according to the ultrasonic signal; and an audio source tracking circuit for outputting a tracking signal according to the notification signal, wherein the photographing device It is used for not tracking the speaker device that emits the sound or turning to track a sound source different from the speaker device according to the tracking signal. The audio-visual system as claimed in claim 1, wherein the speaker device further comprises: an audio receiver for receiving the sound emitted by the speaker device to generate a microphone signal; a calculation circuit for according to a reference signal and The microphone signal determines whether the speaker device emits sound; and when the computing circuit determines that the speaker device emits the sound, the ultrasonic transmitter in the speaker device outputs the ultrasonic signal to the photographing device. The audio-visual system of claim 2, wherein when the computing circuit determines that the speaker device does not emit the sound, the ultrasonic transmitter in the speaker device will not output the ultrasonic signal, and the camera device is still tracking. The audio-visual system of claim 2, wherein the speaker device further comprises: a resampler for re-sampling an audio signal received by the speaker device to generate the reference signal. The audio-visual system of claim 4, wherein the speaker device further comprises: an audio processing circuit for performing echo processing on at least one of the microphone signal and the reference signal to output an audio processing signal to the computing circuit . The audio-visual system according to claim 1, wherein the photographing device further comprises: a locator for receiving the notification signal output by the ultrasonic receiver, and determining a relationship between the speaker device and the photographing device according to the notification signal a relative position; a masker for outputting an area notification signal about a masked area of the speaker device according to the relative position, wherein the masked area is an area where the camera device does not track the speaker device; and The audio source tracking circuit is used for receiving the area notification signal to output the tracking signal. A control method of an audio-visual system, comprising: outputting an ultrasonic signal by an ultrasonic transmitter arranged in a speaker device, wherein when the speaker device emits a sound according to an audio signal, the ultrasonic transmitter outputs the ultrasonic signal; receiving the ultrasonic signal by an ultrasonic receiver provided in a photographing device, and outputting a notification signal according to the ultrasonic signal; outputting a tracking signal through an audio source tracking circuit provided in the photographing device according to the notification signal; and According to the tracking signal, the camera device is controlled not to track the speaker device that emits the sound or turn to a sound source different from the speaker device. The method of claim 7, further comprising: receiving the sound emitted by the speaker device through an audio receiver disposed in the speaker device to generate a microphone signal; using an audio receiver disposed in the speaker device to generate a microphone signal; The computing circuit determines whether the speaker device emits the sound according to a reference signal and the microphone signal; and when the computing circuit determines that the speaker device emits the sound, the ultrasonic transmitter in the speaker device outputs the ultrasonic signal to the photographing device . The method of claim 8, further comprising when the computing circuit determines that the speaker device does not emit the sound, the ultrasonic transmitter in the speaker device will not output the ultrasonic signal, and the camera device is still tracking. The method of claim 8, further comprising: re-sampling an audio signal received by the speaker device by a re-sampler disposed in the speaker device to generate the reference signal. The method of claim 10, further comprising: performing echo processing on at least one of the microphone signal and the reference signal by an audio processing circuit disposed in the speaker device to output an audio processing signal to the computing circuit . The method according to claim 7, further comprising: receiving the notification signal output by the ultrasonic receiver through a locator disposed in the photographing device, and determining a relationship between the speaker device and the photographing device according to the notification signal Relative position; by means of a mask provided on the photographing device, output an area notification signal about a masking area of the speaker device according to the relative position, wherein the masking area means that the photographing device does not perform any operation on the speaker device tracking area; and receiving the area notification signal through the audio source tracking circuit to output the tracking signal.

Images