TWI844231B - System and method for providing karaoke service - Google Patents

Abstract

A system and a method for providing a karaoke service are provided. The method includes: receiving a request command from a first user equipment (UE) by a remote server; transmitting an audio signal and a time synchronization signal corresponding to the audio signal to the first UE through a first low latency channel by the remote server based on the request command; and outputting the audio signal according to the time synchronization signal by the first UE.

Description

System and method for providing singing service

The present invention relates to a wireless communication technology, and in particular to a system and method for providing singing accompaniment service.

Most of the existing online singing services only support the function of a single user singing a song, but do not support the function of singing in chorus. For online singing services that support the chorus function, the first user records the singing voice, and then adds the recorded singing voice to the song to play it to other users. In other words, the existing online singing services cannot provide a remote synchronous chorus function for multiple users.

The present invention provides a system and method for providing singing accompaniment service, which can allow multiple users at different locations to sing synchronously.

The present invention discloses a system for providing singing accompaniment service, comprising a remote server and a first user device. The first user device is communicatively connected to the remote server, wherein the remote server receives a request instruction from the first user device, and based on the request instruction, transmits an audio signal and a time synchronization signal corresponding to the audio signal to the first user device through a first low-latency channel, wherein the first user device outputs the audio signal according to the time synchronization signal.

In an embodiment of the present invention, the time synchronization signal includes a plurality of time stamps, wherein the first user equipment outputs an audio signal according to the plurality of time stamps.

In one embodiment of the present invention, the system further includes a second user device. The second user device is communicatively connected to the remote server, wherein the first user device transmits a sound signal corresponding to the audio signal to the remote server via a first low-latency channel, wherein the remote server transmits the sound signal, the audio signal, and the time synchronization signal to the second user device via a second low-latency channel.

In one embodiment of the present invention, the remote server outputs an audio signal and a sound signal according to a time synchronization signal.

In one embodiment of the present invention, the remote server receives an operation instruction from the first user equipment, and transmits an audio signal and a time synchronization signal to the first user equipment according to the operation instruction.

In one embodiment of the present invention, the above-mentioned operation instruction corresponds to at least one of the following operations: song request, song switching, pause, resume playing, jump, pitch adjustment and speed adjustment.

In one embodiment of the present invention, the remote server transmits a state synchronization signal corresponding to the operation command to the first user equipment, wherein the first user equipment outputs an audio signal according to the time synchronization signal and the state synchronization signal.

In one embodiment of the present invention, the first user equipment obtains a lyrics file, wherein the first user equipment outputs the lyrics file and an audio signal according to a time synchronization signal.

The present invention provides a method for providing a singing accompaniment service, comprising: receiving a request instruction from a first user device by a remote server; transmitting an audio signal and a time synchronization signal corresponding to the audio signal to the first user device through a first low-latency channel by the remote server based on the request instruction; and outputting the audio signal by the first user device according to the time synchronization signal.

Based on the above, the system of the present invention establishes a low-latency channel between a remote server and multiple user devices to transmit audio signals and time synchronization signals. The user device can determine the playing progress of the audio signal according to the time stamp in the time synchronization signal, and then play the audio signal for the user to listen to according to the time stamp. After recording the sound signal emitted by the user, the remote server can transmit the audio signal, the time synchronization signal, and the sound signal to another user device, so that the user device can play the audio signal and the sound signal according to the time synchronization signal, thereby achieving the effect of remote synchronous chorus.

In order to make the content of the present invention more clearly understood, the following embodiments are specifically cited as examples by which the present invention can be truly implemented. In addition, wherever possible, elements/components/steps with the same reference numerals in the drawings and embodiments represent the same or similar components.

FIG. 1 is a schematic diagram of a system 10 for providing singing-along service according to an embodiment of the present invention. The system 10 may include a remote server 300 and N user devices, where N is an arbitrary positive integer. The N user devices may include a user device 100 and a user device 200. The user device 100, the user device 200 and the remote server 300 may be connected to each other through a network for communication.

FIG2 is a schematic diagram of a remote server 300 according to an embodiment of the present invention. The remote server 300 may include a processor 310, a storage medium 320, and a transceiver 330.

The processor 310 is, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose micro control unit (MCU), microprocessor, digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), graphics processing unit (GPU), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar components or combinations of the above components. The processor 310 may be coupled to the storage medium 320 and the transceiver 330 , and access and execute a plurality of modules and various applications stored in the storage medium 320 .

The storage medium 320 is, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD) or similar components or a combination of the above components, and is used to store multiple modules or various applications that can be executed by the processor 310. In this embodiment, the storage medium 320 can store multiple modules including an audio control module 321, a singing music module 322, and a synchronization control module 323, and its functions will be described later.

The transceiver 330 transmits and receives signals wirelessly or wiredly. The transceiver 130 may also perform operations such as low noise amplification, impedance matching, mixing, up or down frequency conversion, filtering, amplification, and the like.

FIG3 is a schematic diagram of a user equipment 100 according to an embodiment of the present invention. The user equipment 100 may include a processor 110, a storage medium 120, a transceiver 130, and an input/output device 140.

The processor 110 is, for example, a central processing unit, or other programmable general-purpose or special-purpose microcontroller unit, microprocessor, digital signal processor, programmable controller, special application integrated circuit, graphics processor, image signal processor, image processing unit, arithmetic logic unit, complex programmable logic device, field programmable logic gate array or other similar components or combinations of the above components. The processor 110 can be coupled to the storage medium 120, the transceiver 130 and the input and output device 140, and access and execute multiple modules and various applications stored in the storage medium 120.

The storage medium 120 is, for example, any type of fixed or removable random access memory, read-only memory, flash memory, hard disk, solid state drive or similar device or a combination of the above devices, and is used to store multiple modules or various applications that can be executed by the processor 110. In this embodiment, the storage medium 120 can store multiple modules including an audio module 121 and a synchronization module 122, and its functions will be described later.

The transceiver 130 transmits and receives signals wirelessly or wiredly. The transceiver 130 may also perform operations such as low noise amplification, impedance matching, frequency mixing, up or down frequency conversion, filtering, amplification, and the like.

The input-output device 140 may include but is not limited to a keyboard, a mouse, a display, a touch screen, a microphone, or a speaker. The user may operate the user equipment 100 through the input-output device 140. The user equipment 100 may output information for the user's reference through the input-output device 140.

FIG4 is a schematic diagram of a user equipment 200 according to an embodiment of the present invention. The user equipment 200 may include a processor 210, a storage medium 220, a transceiver 230, and an input/output device 240.

The processor 210 is, for example, a central processing unit, or other programmable general-purpose or special-purpose microcontroller unit, microprocessor, digital signal processor, programmable controller, special application integrated circuit, graphics processor, image signal processor, image processing unit, arithmetic logic unit, complex programmable logic device, field programmable logic gate array or other similar components or combinations of the above components. The processor 210 can be coupled to the storage medium 220, the transceiver 230 and the input and output device 240, and access and execute multiple modules and various applications stored in the storage medium 220.

The storage medium 220 is, for example, any type of fixed or removable random access memory, read-only memory, flash memory, hard disk, solid state drive or similar device or a combination of the above devices, and is used to store multiple modules or various applications that can be executed by the processor 210. In this embodiment, the storage medium 220 can store multiple modules including an audio module 221 and a synchronization module 222, and its functions will be described later.

The transceiver 230 transmits and receives signals wirelessly or wiredly. The transceiver 230 may also perform operations such as low noise amplification, impedance matching, frequency mixing, up or down frequency conversion, filtering, amplification, and the like.

The input-output device 240 may include but is not limited to a keyboard, a mouse, a display, a touch screen, a microphone, or a speaker. The user may operate the user equipment 200 through the input-output device 240. The user equipment 200 may output information for the user's reference through the input-output device 240.

When the user device 100 wants to initiate a remote synchronous chorus, the user device 100 can send a request instruction to the remote server 300 through the transceiver 130, wherein the request instruction can include the music specified by the user. After receiving the request instruction, the accompaniment music module 322 of the remote server 300 can load the audio signal of the specified music according to the request instruction (for example, obtain the audio signal corresponding to the specified music from the lyrics and music library stored in the storage medium 320). Then, the accompaniment music module 322 can instruct the audio control module 321 to transmit the audio signal to the user device 100, and can instruct the synchronization control module 323 to transmit the time synchronization signal corresponding to the audio signal to the user device 100.

Specifically, the transceiver 330 of the remote server 300 and the transceiver 130 of the user equipment 100 can be used to establish one or more low-latency channels between the remote server 300 and the user equipment 100. The audio control module 321 can transmit audio signals to the user equipment 100 through the low-latency channel. The synchronization control module 323 can transmit time synchronization to the user equipment 100 through the low-latency channel. The audio control module 321 can use a low-latency audio open source software and hardware kit on the market (for example: JackTrip kit) to perform uncompressed and distortion-free low-latency audio transmission. The time synchronization signal can include multiple time stamps, wherein the multiple time stamps are used to indicate the playback progress of the audio signal. In one embodiment, the audio control module 321 may transmit the audio signal to the user equipment 100 according to the progress indicated by the timestamp of the time synchronization signal.

In one embodiment, the user device 100 may transmit an operation instruction to the remote server 300 via the transceiver 130. The accompaniment music module 322 of the remote server 300 may instruct the audio control module 321 to transmit an audio signal to the user device 100 according to the operation instruction, and may instruct the synchronization control module 323 to transmit a time synchronization signal to the user device 100 according to the operation instruction, wherein the operation instruction may correspond to operations such as song request, song switching, pause, resume playback, jump, pitch adjustment or speed adjustment. The synchronization control module 323 may also transmit a state synchronization signal corresponding to the operation instruction to the user device 100.

For example, if the remote server 300 receives an operation instruction for increasing the pitch from the user device 100, the audio control module 321 may increase the pitch of the transmitted audio signal. The state synchronization signal transmitted to the user device 100 by the synchronization control module 323 may indicate to the user device 100 that the pitch of the audio signal has been increased. If the remote server 300 receives an operation instruction for jumping from the user device 100, the synchronization control module 323 may adjust the timestamp in the transmitted time synchronization signal to update the playback progress of the audio signal. The state synchronization signal transmitted to the user device 100 by the synchronization control module 323 may indicate to the user device 100 that the playback progress of the audio signal has been jumped.

In one embodiment, the remote server 300 only accepts and executes operation instructions issued by user devices with operation permissions. For example, the user device 100 that initiates remote synchronized chorus may have the permission to send operation instructions to the remote server 300. The user device 100 may further send a message to the remote server 300 to notify the remote server 300 to open permissions to the user device 200, so that the remote server 300 can accept and execute the operation instructions from the user device 200.

The audio module 121 of the user equipment 100 may receive an audio signal from the remote server 300 through a low-latency channel. The synchronization module 122 of the user equipment 100 may receive a time synchronization signal and/or a state synchronization signal from the remote server 300 through a low-latency channel. The synchronization module 122 may output an audio signal through the input/output device 140 (e.g., play the audio signal through a speaker) according to multiple time stamps of the time synchronization signal and/or the state synchronization signal to play the audio signal to the user of the user equipment 100 for listening.

In one embodiment, the synchronization module 122 of the user equipment 100 can obtain a lyrics file (e.g., obtain a lyrics file from a lyrics library stored in the storage medium 320). The synchronization module 122 can output the lyrics file through the input/output device 140 (e.g., display the lyrics through a display) according to the time synchronization signal for user reference.

While the user of the user device 100 sings along with the audio signal, the user device 100 can capture the sound signal emitted by the user through the input and output device 140 (for example, capturing the user's singing voice through a microphone). The audio module 121 can transmit the sound signal corresponding to the audio signal to the remote server 300 through a low-latency channel.

The audio control module 321 of the remote server 300 can receive the sound signal from the user device 100 through the low-latency channel. After receiving the sound signal, the accompaniment music module 322 can instruct the audio control module 321 to transmit or forward the audio signal and the sound signal to the user device 200, and instruct the synchronization control module 323 to transmit the time synchronization signal corresponding to the audio signal and the sound signal to the user device 200.

Specifically, the transceiver 330 of the remote server 300 and the transceiver 230 of the user equipment 200 can be used to establish one or more low-latency channels between the remote server 300 and the user equipment 200. The audio control module 321 can transmit audio signals and sound signals to the user equipment 200 through the low-latency channel, and the synchronization control module 323 can transmit a time synchronization signal to the user equipment 200 through the low-latency channel, wherein the time synchronization signal can include multiple timestamps. In one embodiment, the audio control module 321 can transmit an audio signal or a sound signal to the user equipment 200 according to the progress indicated by the timestamp of the time synchronization signal.

In one embodiment, if the remote server 300 receives an operation instruction from the user device 100 or the user device 200, the accompaniment music module 322 may instruct the audio control module 321 to transmit an audio signal or a sound signal to the user device 200 according to the operation instruction, and may instruct the synchronization control module 323 to transmit a time synchronization signal to the user device 200 according to the operation instruction. The synchronization control module 323 may also transmit a state synchronization signal corresponding to the operation instruction to the user device 200.

The audio module 221 of the user device 200 can receive the audio signal and the sound signal from the remote server 300 through the low-latency channel. The synchronization module 222 of the user device 200 can receive the time synchronization signal and/or the state synchronization signal from the remote server 300 through the low-latency channel. The synchronization module 222 can output the audio signal and the sound signal through the input and output device 240 according to the multiple time stamps of the time synchronization signal and/or the state synchronization signal to play for the user of the user device 200 to listen to. Accordingly, the user of the user device 200 can synchronously listen to the background music (i.e., the audio signal) and the singing voice (i.e., the sound signal) emitted by the user of the user device 100, thereby achieving the effect of remote synchronous chorus.

FIG5 is a flow chart of a method for providing a singing service according to an embodiment of the present invention, wherein the method can be implemented by the system 10 shown in FIG1. In step S501, a remote server receives a request instruction from a first user device. In step S502, the remote server transmits an audio signal and a time synchronization signal corresponding to the audio signal to the first user device through a first low-latency channel based on the request instruction. In step S503, the first user device outputs the audio signal according to the time synchronization signal.

In summary, the present invention has the following features and effects: The system of the present invention can provide the function of synchronous chorus for multiple people in multiple locations. The playback and operation of vocals, accompaniment music, and information such as lyrics can be synchronized in real time between user devices. Compared with the existing accompaniment service system, the synchronization method of the present invention is implemented in a low-latency environment, which can avoid the time delay or accumulation caused by the external environment. In addition, the system's support for complex operations such as music jump, acceleration or deceleration is better than that of traditional accompaniment service systems. On the other hand, compared with the traditional accompaniment service system that downloads music to the client, the present invention transmits music to the user device through a low-latency channel from a remote server for real-time playback, avoiding the risk of music being stolen by the client.

10: System

100, 200: User equipment

110, 210, 310: Processor

120, 220, 320: Storage media

121, 221: Audio module

122, 222: Synchronous module

130, 230, 330: transceiver

140, 240: Input and output devices

300: Remote Server

321: Audio Control Module

322: Music module

323: Synchronous control module

S501, S502, S503: Steps

FIG. 1 is a schematic diagram of a system for providing singing accompaniment service according to an embodiment of the present invention. FIG. 2 is a schematic diagram of a remote server according to an embodiment of the present invention. FIG. 3 is a schematic diagram of a user device according to an embodiment of the present invention. FIG. 4 is a schematic diagram of a user device according to an embodiment of the present invention. FIG. 5 is a flow chart of a method for providing singing accompaniment service according to an embodiment of the present invention.

S501, S502, S503: Steps

Claims (6)

A system for outputting an audio signal comprises: a remote server; a first user device, communicatively connected to the remote server; and a second user device, communicatively connected to the remote server, wherein the remote server receives a request instruction from the first user device, and based on the request instruction, transmits an audio signal and a time synchronization signal corresponding to the audio signal to the first user device through a first low-latency channel, wherein the first user device outputs the audio signal according to the time synchronization signal, wherein the The first user equipment transmits a sound signal corresponding to the audio signal to the remote server through the first low-latency channel, and the remote server transmits the sound signal and the audio signal to the second user equipment through the second low-latency channel according to the time synchronization signal, wherein the remote server receives an operation instruction from the first user equipment, and transmits the audio signal and the time synchronization signal to the first user equipment according to the operation instruction, wherein the operation instruction corresponds to pitch adjustment. A system as described in claim 1, wherein the time synchronization signal includes multiple time stamps, and wherein the first user equipment outputs the audio signal according to the multiple time stamps. A system as described in claim 1, wherein the operation instruction further corresponds to at least one of the following operations: ordering songs, switching songs, pausing, resuming playback, jumping, and speed adjustment. A system as described in claim 1, wherein the remote server transmits a state synchronization signal corresponding to the operation instruction to the first user device, wherein the first user device outputs the audio signal according to the time synchronization signal and the state synchronization signal. A system as described in claim 1, wherein the first user device obtains a lyrics file, wherein the first user device outputs the lyrics file and the audio signal according to the time synchronization signal. A method for outputting an audio signal, comprising: receiving a request instruction from a first user equipment by a remote server; transmitting an audio signal and a time synchronization signal corresponding to the audio signal to the first user equipment through a first low-latency channel based on the request instruction; outputting the audio signal by the first user equipment according to the time synchronization signal; transmitting the audio signal and a time synchronization signal corresponding to the audio signal through the first low-latency channel by the remote server; The remote server transmits the sound signal of the first user device to the remote server, and the remote server transmits the sound signal and the audio signal to the second user device through a second low-latency channel according to the time synchronization signal; and the remote server receives an operation instruction from the first user device, and transmits the audio signal and the time synchronization signal to the first user device according to the operation instruction, wherein the operation instruction corresponds to the pitch adjustment.

Images