KR102290901B1 - Song playing system - Google Patents

Abstract

The present invention relates to a karaoke system to enable a user to intuitively and easily recognize lyrics. According to the present invention, the karaoke system comprises: a user terminal configured to prepare MR data including melody information for a predetermined song and accompaniment data including text-to-speech (TTS) data matching the MR data when a request for singing for the predetermined is input from a user; and a central server configured to communicate with the user terminal through a network, and when a request for transmission of the accompaniment data of the predetermined song is received from the user terminal, generating the TTS data including TTS information to be output between at least one or more adjacent lyric lines on the basis of a lyrics format including time information for each lyric line and time information for each lyric syllable of the predetermined song and transmit the TTS data together with the MR data as the accompaniment data to the user terminal. The user terminal is configured to execute the accompaniment data on the basis of an operation mode when the accompaniment data provided from the central server is prepared.

Description

Song accompaniment system {SONG PLAYING SYSTEM}

The present invention relates to a song accompaniment system, and more specifically, by outputting lyrics corresponding to a melody in advance by voice, the user can recognize the lyrics corresponding to the melody of Aeum, and changing the song accompaniment mode according to the user's voice command. It is about a possible song accompaniment system.

In the conventional song accompaniment, an identification code is assigned to each song for a plurality of songs, and accompaniment information and image information corresponding to each identification code are stored. That is, in the conventional song accompaniment, when the user selects a specific identification code, the pre-stored song accompaniment is output together with the image information to support the user to easily sing.

Meanwhile, various information may be included in the image information output through the display together with the music accompaniment output through the speaker. For example, lyric information, pitch information, timing information, and the like are such. That is, by displaying the lyrics on the display, it is possible to support the user to easily sing a song with the help of the lyrics synchronized to the accompaniment.

However, in an environment in which the user cannot view the display, for example, if the user is driving, or if the song accompaniment itself is a microphone-type device that does not support the display, only providing the melody as accompaniment information effectively allows the user to sing the song. There is bound to be a limit to the support that can be called.

Accordingly, a prior patent (KR10-2050785) has been proposed. The preceding patent is a song lyrics output method, song accompaniment and song accompaniment system, in which the accompaniment receives a MIDI file from the server, analyzes the melody gap and tempo in the accompaniment period, and reads the lyrics that lead to an appropriate tempo during the blank period. are doing

However, the previous patent requires additional processing time for loading in that the client-side half cycle itself needs to analyze based on the MIDI file, and in this case, the quality of service use depends on the performance of the half cycle itself. have no choice but to be dominated In addition, when the user uses the service while driving, for example, when the vehicle passes through an area where communication is not smooth, there is a limitation in using the service when a large-capacity MIDI file is transmitted through communication with the server in half a cycle. there has to be

Accordingly, a different type of service proposal is required.

US 10-2050785 B1

The present invention is to solve the above problems, and in a song accompaniment system that does not provide a display, a user can easily and intuitively recognize song lyrics, reduce the performance problem of the accompaniment itself, and An object of the present invention is to provide a song accompaniment system that reduces the load on data processing.

As a song accompaniment system according to an embodiment for realizing the object of the present invention, when a request to sing a specific song is input from a user, MR data including melody information for the specific song and the MR data are matched A user terminal configured to prepare accompaniment data including Text to Speech (TTS) data, and communicate with the user terminal through a network, and when a request for transmission of accompaniment data for the specific song is received from the user terminal, the TTS data including TTS information to be output between at least one or more adjacent lyric lines is generated based on a lyric format including time information for each lyric line and time information for each lyric syllable of a specific song, and the TTS data is used to generate the TTS data. and a central server configured to transmit to the user terminal as the accompaniment data together with the MR data, wherein the user terminal executes the accompaniment data based on an operation mode when the accompaniment data provided from the central server is prepared. is composed

In one embodiment of the present invention, when a request for singing a specific song is input by the user, the user terminal determines whether accompaniment data for the specific song is stored, and if the accompaniment data is stored , the accompaniment data may be prepared and executed without a request for transmission of the accompaniment data to the central server, and when the accompaniment data is not stored, the accompaniment data may be requested to be transmitted to the central server.

In an embodiment of the present invention, the user terminal may be controlled according to the user's voice, and the operation mode of the accompaniment data may be determined based on a command extracted from the voice input from the user.

In one embodiment of the present invention, the operation mode may include a singing mode in which only the MR data is executed and a practice mode in which the TTS data is executed together with the MR data.

In one embodiment of the present invention, the practice mode includes a first practice mode that reads a syllable of a next lyric line by TTS between adjacent lyric lines, and a TTS before starting a melody of each lyric line. A second practice mode for reading may be included.

In one embodiment of the present invention, in the second practice mode, the progress of the melody of the corresponding lyrics line may be stopped until the syllable of the lyrics line is read by TTS.

In an embodiment of the present invention, when a request for transmission of accompaniment data for the specific song is received from the user terminal, the central server determines whether accompaniment data for the specific song is stored, and the accompaniment When data is stored, the accompaniment data is transmitted to the user terminal without generating the TTS data, and when the accompaniment data is not stored, the TTS data is generated.

In an embodiment of the present invention, the central server extracts time information for an interval between adjacent lyrics lines based on the time information for each lyrics line in the lyrics format, and the length of the interval is a predetermined criterion In a shorter case, TTS data may not be generated between corresponding lyrics lines.

In an embodiment of the present invention, the central server is configured to: based on the time information for each lyric line and the time information for each lyric syllable in the lyric format, the number of syllables of the lyrics included in the rear lyric line among the adjacent lyric lines; The TTS provision time of the corresponding lyric line is calculated by multiplying the predetermined TTS provision time per syllable, and the time point preceding the TTS provision time of the lyric line from the start time of the first lyric syllable of the following lyric line, the TTS of the lyric line It can be decided at the time of delivery.

In an embodiment of the present invention, when the calculated TTS provision time of the lyrics line is longer than the time interval of the adjacent lyrics lines, the central server shortens the TTS provision time, so that the TTS of the lyrics line The provision point may be located within a time interval of the adjacent lyrics lines.

In the song accompaniment system according to the present invention, when information about a song to be sung is input from a user, an MR file including only a melody and a TTS data file generated based on time information for each syllable of song lyrics matching the MR file are generated from the server. By providing a half-cycle, there is no need for separate data processing and information analysis as a half-cycle, and only MR files and TTS data files, which are relatively small compared to MIDI files, need to be received, so communication capacity can be reduced, and the half-cycle itself performance problems, and the load on data processing according to transmission and reception from the server can be reduced. Accordingly, it is possible to provide a song accompaniment system in which a user can easily and intuitively recognize song lyrics without being affected by accompaniment performance.

1 is a conceptual illustration for explaining a song accompaniment system.
2 is a block diagram illustrating a song accompaniment system according to an embodiment of the present invention.
FIG. 3 is a block diagram illustrating the user terminal of FIG. 2 .
FIG. 4 is a block diagram illustrating the control unit of FIG. 2 .
5 is a flowchart illustrating an operation mode control of the controller.
FIG. 6 is a block diagram illustrating the central server of FIG. 2 .
7 is an exemplary diagram for explaining the insertion timing of the TTS generated by the accompaniment data generator of FIG. 6 .
8 is a flowchart for explaining a process for a user to use a service through a song accompaniment system according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily carry out the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in adding reference numbers to the components of each drawing, the same reference numbers are given to the same components even though they are shown in different drawings. In addition, in the present specification, when a component is "connected" or "connected" to another component, it is not only when both components are "directly" connected, but also other components are interposed therebetween. It includes cases where they are connected to each other, and also includes cases where they are connected “physically” as well as cases where they are “functionally or communicatively”. In addition, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

1 is a conceptual illustration for explaining a song accompaniment system. 2 is a block diagram illustrating a song accompaniment system according to an embodiment of the present invention. FIG. 3 is a block diagram illustrating the user terminal of FIG. 2 .

1 to 3 , the song accompaniment system 1000 of the present invention, when a user requests to sing a specific song among a plurality of songs managed by a central server, an MR file for the specific song and and provide a TTS data file generated based on time information for each syllable of song lyrics matching the MR file to the user terminal. The song accompaniment system 1000 of the present invention provides the user with information about a specific song when the user is driving and cannot continuously see the information displayed on the display, or the device does not include a display by itself. configured to provide TTS. 1 shows an example in which the song accompaniment system 1000 of the present invention interworks with T map to provide a song accompaniment service among navigation services.

The song accompaniment system 1000 is configured to communicate with the user terminal 100 and the user terminal 100 through a network, and when there is a request for accompaniment data for a specific song from the user terminal 100, the specific song and a central server 300 configured to transmit accompaniment data to the user terminal.

The user terminal 100 sings, sings along, or learns at least one or more songs or songs (hereinafter referred to as 'songs') through the song accompaniment system 1000 (hereinafter referred to as 'users'). As a device or portable terminal owned by a user, a user joins or logs in to the central server 300 through the network 10 in the form of a web, an application, or a web app. is configured to enable the use of a song accompaniment service by receiving at least one or more data about a song that the user wants to sing. The song accompaniment service may be a service according to payment, and payment may be priced according to, for example, a usage period based on year, month, and day or the number of songs.

Here, the portable terminal is, for example, a wireless communication device that guarantees portability and mobility, and includes a smartphone, a Personal Communication System (PCS), a Global System for Mobile communication (GSM), a Personal Digital Cellular (PDC), and a PHS. (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) It may include all types of handheld-based wireless communication devices such as terminals.

In addition, the device includes, for example, a desktop, a laptop, a tablet PC, etc. equipped with a communicable web browser (WEB Browser), and a smart microphone capable of performing an accompaniment function. Alternatively, it may be an electronic device or a navigation device installed in a vehicle.

In addition, the user terminal 100 is capable of communicating with the central server 300, controllable through a user's voice signal, and when an application to sing a specific song is input from the user, the user sings the song first. It is configured to determine whether the necessary data (hereinafter referred to as "accompaniment data") is stored in the storage device of the user terminal, and if the corresponding accompaniment data is not stored, request transmission of the accompaniment data to the central server do. The accompaniment data includes an MR file including song accompaniment information and a TTS data file including text to speech (TTS) information. Thereafter, upon receiving the accompaniment data from the central server 300 , the user terminal 100 is configured to execute the MR file and the TTS data file matching the MR information. A description of the TTS data file will be provided later.

Specifically, the user terminal 100 includes a communication unit 110 , an input unit 120 , a voice recognition unit 130 , a control unit 140 , a TTS data processing unit 150 , an output unit 160 , and a storage unit 170 ). includes

The communication unit 110 is configured to transmit and receive data to and from the central server 300 through the network 10 . The communication unit 110 includes a communication chipset for a wired LAN, a communication chipset for a wireless LAN, a communication chipset for connecting to a mobile communication network, and the like and circuits to transmit and receive communication packets from the network 10 . The network 10 includes a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a Personal Area Network (PAN), and a mobile communication network. (mobile radio communication network), Wibro (Wireless Broadband Internet), Mobile WiMAX, HSDPA (High Speed Downlink Packet Access), or can be implemented in all types of wired/wireless networks, such as a satellite communication network.

The input unit 120 functions as a microphone capable of receiving a user's voice signal, that is, a microphone signal, and receives a microphone signal corresponding to the user's voice through the input unit. The input unit 120 may include a diaphragm, etc. to convert the vibration (signal) from the input voice into an electrical signal and output it as a microphone signal. For example, through the input unit, application information for a specific song that the user wants to sing is input, and also, the user's voice signal when the user sings a song is input.

The voice recognition unit 130 is configured to convert the user's voice signal input through the input unit 120 into voice data, and to extract feature values from the voice data. For example, the voice recognition unit 130 extracts a feature value based on a relative amount of change that changes over time from voice data based on a language model learned based on artificial intelligence, and based on the feature value, It may be configured to recognize voice data in phoneme units to determine a command based on a probability value. Since the technical description for this is beyond the scope of the present invention, a detailed description thereof will be omitted, and voice recognition technology currently used in smartphones, smart speakers, and the like may be used.

Hereinafter, the operation of the control unit will be described in detail with reference to FIGS. 4 and 5 .

4 is a block diagram illustrating the control unit of FIG. 2 . 5 is a flowchart illustrating an operation mode control of the controller.

4 and 5, the controller 140 prepares accompaniment data of a specific song that the user wants to sing based on the user's command recognized through the voice recognition unit 130, and executes the accompaniment data. It is configured to perform mode control. Specifically, the control unit 140 includes an accompaniment data preparation unit 141 and an operation mode control unit 143 .

The accompaniment data preparation unit 141 specifies the title or name of a specific song from the input of the user's voice, and when an execution command such as singing the corresponding song is recognized from the user, the accompaniment data for the corresponding song is provided. prepared to perform. For example, the user command may include [song title] play a song, play a song, start, play a song, sing a song, sing a song, run, open a song, etc.

When an execution command for a specific song is input, the accompaniment data preparation unit 141 checks whether there is accompaniment data of the song requested by the user in the storage unit 170 to be described later, and the accompaniment data of the corresponding song in the storage unit 141 . If there is no , it is configured to request the central server 300 to transmit accompaniment data for the specific song. For example, if the user has previously requested for the same song or if it is a song provided by default in the service, since accompaniment data for the song is stored in the storage unit 170, it is unnecessary to communicate with the central server. Accompaniment data can be executed directly without the need for communication. That is, the accompaniment data preparation unit 141 scans the storage unit 170 and, when there is no accompaniment data for the corresponding song, receives the accompaniment data from the central server 300 and receives the accompaniment data from the storage unit 170 . stored in , and ready to run.

The operation mode control unit 143 is configured to execute the accompaniment data in an operation mode determined based on a user's command when accompaniment data is prepared from the accompaniment data preparation unit 141 .

The operation mode is largely divided into a singing mode in which only melody is output and a practice mode in which melody and TTS are output together. That is, in the singing mode, the TTS data file is not executed, only the MR file is executed, and in the practice mode, the TTS data file is executed together with the MR file. For example, the practice mode may be composed of a first practice mode, which is an ear singing mode, and a second practice mode, which is a mode for learning lyrics.

In the first practice mode, the TTS is used to read the lyrics of the next bar between each bar of the accompaniment (melody). In the second practice mode, the lyrics are read in advance by TTS before each measure of the accompaniment starts. If the first practice mode is a mode in which singing proceeds without interruption of the melody in a state in which the familiarity of the lyrics is incomplete but singing along is possible, the second practice mode is a case where the familiarity of the lyrics is not at all, and the delivery of the lyrics rather than the progress of the melody is performed. It is a mode for the purpose of learning through. Accordingly, in the second practice mode, if there is not enough time to transmit the lyrics between the measures of the melody, the melody may be stopped, the lyrics may be delivered, and then the melody may be executed again. However, the present invention is not limited thereto.

For example, when the accompaniment data is prepared, the operation mode control unit 143 selects whether the user proceeds to the singing mode or the practice mode through a guide voice, or prioritizes the singing mode, but the user's command It can be set in advance so that the operation mode is switched by . Alternatively, the corresponding operation mode may be executed separately according to the execution command from the beginning without the guidance voice. For example, the command for executing the practice mode may include [song title] want to practice, practice a song, tell me song lyrics, and the like.

In addition, when the practice mode is selected, the operation mode control unit 143 may further be configured to guide the user whether to execute in the first practice mode or in the second practice mode through a voice guidance. For example, a command for executing the first practice mode may include [song title] I want to practice, and the command for executing the second practice mode may include [song title] tell me lyrics. Referring to FIG. 5 , the user can change the operation mode through a command in the singing mode, and when a command for changing the operation mode is received, the user selects the practice mode method, thereby proceeding to the corresponding practice mode. can be

The operation mode control unit 143 of the control unit 140 according to the present embodiment has been described as an example of proceeding to the first practice mode or the second practice mode among the practice modes based on a user's command, but is not limited thereto. For example, the song proceeding in the first practice mode or the song proceeding in the second practice mode may be determined according to a song characteristic, for example, BPM. For example, when the central server generates a TTS data file, a TTS data file for a corresponding practice mode may be generated according to the song characteristics. That is, when generating TTS data, the practice mode of the corresponding song may be determined.

When the practice mode is executed according to the control signal of the controller 140 , the TTS data processing unit 150 reads TTS data from the prepared accompaniment data and outputs the TTS so as to match the MR data outputting the melody. The TTS data is generated based on the lyrics file in the central server 300, and the TTS data may include information on each of the practice modes, for example, the first practice mode and the second practice mode, and the It may be configured to execute corresponding data according to a control signal of the controller 140 .

Alternatively, the TTS data provided from the central server 300 may be TTS data corresponding to any one of the first practice mode and the second practice mode, and the TTS data processing unit 150 processes the TTS data. , may be configured to generate TTS data corresponding to the remaining practice modes. The generation of the TTS data will be described in detail in the configuration of the central server 300 .

The output unit 160 includes an audio digital analog converter (DAC), etc. to receive MR data and TTS data, convert it into an audio signal corresponding to the received MR data and TTS data, and output the received MR data and TTS data. The output unit 160 may be configured to output an audio signal to a speaker, including a speaker, or to output an audio signal to an externally connected speaker or earphone.

The storage unit 170 is configured to store a program executed by the control unit 160 and to store a program for processing TTS data. In addition, the storage unit 170 is configured to store and manage accompaniment data for each song provided from the central server 300 .

That is, in the song accompaniment system according to the present embodiment, when information about a song to be sung is input by the user, MR data and TTS data are provided from the central server to the user terminal. As a user terminal, separate data processing and information analysis There is no need to do this, and only the MR file and TTS data file, which are relatively small compared to the MIDI file, need to be received, so the communication capacity can be reduced, reducing the performance problem of the user terminal itself, and The load on data processing can be reduced. Accordingly, it is possible to provide a song accompaniment system in which a user can easily and intuitively recognize song lyrics without being affected by accompaniment performance.

Hereinafter, with reference to FIG. 6 , the central server 300 will be described in detail.

FIG. 6 is a block diagram illustrating the central server of FIG. 2 .

The central server 300 is configured to communicate with the user terminal 100 through the network 10, and when a request for providing accompaniment data for a specific song is received from the user terminal 100, the corresponding accompaniment data It is configured to create and provide.

The central server 300 includes a communication unit 310 , a customer management unit 320 , a control unit 330 , an accompaniment data generation unit 340 , and an accompaniment data management unit 350 .

The communication unit 310 is configured to transmit and receive data to and from the user terminal 100 through the network 10 . The communication unit 310 includes a communication chipset for a wired LAN, a communication chipset for a wireless LAN, a communication chipset for connecting to a mobile communication network, and the like and circuits to transmit and receive communication packets from the network 10 . A request for providing accompaniment data may be received from the user terminal 100 through the communication unit 310 , and the corresponding accompaniment data may be transmitted to the user terminal 100 .

The customer management unit 320 registers and manages a user who wants to use the song accompaniment service in the song accompaniment system 1000 as a customer. The customer management unit 320 is configured to receive registration of basic information such as a user's name, ID, password, etc. input from the user terminal 100, store and manage it in a customer DB, and the user accompaniment in the central server 300 It is configured to accumulate and store usage information, payment information, and the like for a song requesting data.

When a request for providing accompaniment data for a specific song is received from the user terminal 100, the control unit 330 determines whether accompaniment data for the specific song is stored with reference to the accompaniment data management unit 350, which will be described later. judge When there is accompaniment data for the specific song, the corresponding data is transmitted to the user terminal 100 through the communication unit 310, and when there is no accompaniment data for the specific song, an accompaniment data generating unit to be described later ( 340) to generate accompaniment data for the specific song. Meanwhile, when the central server 300 does not have a sound source file as a basis for generating accompaniment data, the controller 330 may receive a sound source file for a specific song from an external server.

The accompaniment data generating unit 340 is configured to generate accompaniment data based on a sound source file for a specific song. The accompaniment data is composed of MR data composed of a melody and TTS data for matching with the MR data to provide a TTS in a predetermined section, and the TTS data is a lyric file, that is, a file created when a sound source is produced. Matching data may be generated based on a lyrics format file including time information for each syllable of the song lyrics. For example, the accompaniment data generating unit 340 may generate TTS data by importing a specific song lyric format file to be generated from a lyric DB storing lyric format files for each song.

For example, the chorus of singer Kim Bum-soo's 'I miss you', 'I loved you like crazy/ Memories/ Memories/ I'm looking for you', for example, the lyrics format may include the following information.

[Lyric Format]

… 0:01:25.05,… It was like {\K402}{\K94}{\K136}seven{\K3} {\K46}{\K0}{\K46}{\K46}{\K46}{\K41}

… ,0:01:28.84,… {\K492}{\K140}{\K23} billions{\K124}{\K39} {\K23}{\K164}{\K23} billions{\K104}

… ,0:01:33.71,… {\K660}{\K47}You found {\K39}{\K8} {\K70}{\K70}{\K0} {\K46}{\K92}{\K106}

That is, three pieces of information for each measure are displayed, and the first time information among the information for each measure indicates the time at which the lyrics line is displayed on the screen. For example, "0:01:25.05" of the first bar means that the lyrics line of the corresponding bar is displayed on the screen at 1:25.05 seconds, that is, 85.05 seconds.

After that, the information of {\K402} indicates the waiting time until the syllable after the corresponding lyric line starts. In other words, it means that the first syllable 'mi' starts after 4.02 seconds, and overall, the syllable 'mi' starts at 85.05 seconds + 4.02 seconds = 89.07 seconds. In addition, the next information, {\K94}, indicates the time at which the immediately following syllable is sung, for example, the time at which the corresponding syllable of the lyrics is colored on the screen.

That is, based on the above information, the starting point of each syllable for each lyrics line is 'mi' - 89.07 seconds, 'seven' - 90.01 seconds, 'like' - 91.40 seconds, 'sa' - 91.86 for the first lyrics line. Second, 'Rang' - 92.32 seconds, 'Had' - 92.78 seconds, 'Dawn' - 93.24 seconds, for the second lyric line, 'Ki' - 93.76 seconds, 'Billion' - 95.16 seconds, 'Lee' - 95.39 seconds , 'chu' - 97.02 seconds, 'billion' - 97.25 seconds, 's' - 98.89 seconds, 'this' - 99.12 seconds, for the third lyric line, 'you' - 100.31 seconds, 'b' - 100.78 seconds, 'Find' - 101.25 seconds, 'Go' - 101.95 seconds, 'There' - 102.65 seconds, 'G' - 103.11 seconds, 'only' - 104.03 seconds can be read out.

That is, the interval between the first and second lyrics lines is 93.76 seconds - 93.24 seconds = 0.52 seconds, and the interval between the second and third lyrics lines is 100.31 seconds - 99.12 seconds = 1.19 seconds.

For example, the accompaniment data generating unit 340 may consider the interval between lyrics for each song or lyrics lines when generating TTS data from the lyrics format. That is, the accompaniment data generating unit 340 may be configured to determine the insertion of the TTS only when the interval is equal to or greater than a predetermined criterion. For example, if the predetermined criterion is 1 second, since the interval between the first and second lyrics lines is 0.52 seconds, the insertion of the TTS including the lyrics information of the second lyrics line is not made, and the second and third lyrics Since the interval between the lines is 1.19 seconds, a TTS including the lyrics information of the third lyrics line can be inserted. However, the present invention is not limited thereto. TTSs corresponding to all intervals are inserted into the TTS data, and when the user terminal 100 processes the TTS data, the TTS may be selected and provided according to the predetermined criterion or operation mode.

In addition, the accompaniment data generating unit 340 is configured to determine the start point of the TTS when the insertion of the TTS is determined. For example, the accompaniment data generator 340 may designate a TTS provision time per letter (syllable) of the TTS, that is, a pronunciation time, and the time may be preset or determined according to an operation mode. That is, in the case of the first practice mode and the second practice mode, a TTS provision time per letter of a TTS of a different standard may be designated.

For example, if the TTS provision time per character (syllable) is 0.3 seconds, the total number of letters in the third lyric line is 7 characters (syllables). It takes time. Therefore, since the corresponding TTS must be provided at least before the start of the first syllable of the third lyrics line, the TTS at 98.21 seconds, that is, 1 minute 38 seconds 21 can be determined. That is, referring to FIG. 7 , the TTS of 'I'm looking for you' is output in the section from 1 minute 38 seconds 21 to 1 minute 40 seconds 31 in the entire lyrics line. However, the present invention is not limited thereto. In contrast, in the above case, since the timing at which the TTS information is output overlaps the second lyric line, the TTS provision time of 2.1 seconds is shortened, for example, by speeding up the time per character, and the second lyric line Third, the TTS may be positioned within the interval of the lyrics line. This may also be determined according to the operation mode.

That is, the accompaniment data generating unit 340 is configured to generate accompaniment data for the corresponding song by generating a TTS data file for all lyric lines in the same process as above, and grouping it with the MR data of the corresponding song. .

The accompaniment data management unit 350 is configured to match the accompaniment data generated by the accompaniment data generation unit 340 with song information to store and manage the accompaniment data DB. Accordingly, when a request for providing accompaniment data from a plurality of user terminals is repeated, accompaniment data of a specific song stored and managed in the DB can be provided without the need to additionally generate accompaniment data.

Hereinafter, with reference to FIG. 8, a process of using a service by a user through a song accompaniment system will be described.

8 is a flowchart illustrating a process for a user to use a service through a song accompaniment system according to an embodiment of the present invention.

1 to 8 , first, a user accesses the central server 300 through the user terminal 100 to use the song accompaniment service provided by the song accompaniment system 1000 and joins as a member while registering ID, secret Enter information such as a number (S100).

Thereafter, when a data request for a specific song to be sung is input from the user through the user terminal 100 (S200), the user terminal 100 is configured to prepare accompaniment data for the specific song (S300). The accompaniment data includes MR data including only a melody and a TTS data file generated based on time information for each syllable of song lyrics matching the MR file. Meanwhile, in this process, the user terminal 100 determines whether the corresponding accompaniment data is stored in the user terminal 100 (S400), and if the corresponding accompaniment data does not exist, the central server 300 is configured to request the provision of accompaniment data.

Upon receiving the request for providing the accompaniment data, the central server 300 determines whether the accompaniment data is stored in the DB (S500), and if the accompaniment data exists, it is transmitted to the user terminal 100 and (S500), if it does not exist, it is configured to generate accompaniment data (S600). In this case, the generation of TTS data among the accompaniment data refers to the lyrics file based on the lyrics format including time information for each syllable of the lyrics matching the MR of the corresponding song. At this time, the generated TTS data is grouped with the MR data, matched with song information in the accompaniment data management unit, and stored and managed (S700). The generated accompaniment data is transmitted to the user terminal 100 .

When the accompaniment data is prepared (S300), the user terminal 100 executes the accompaniment data according to an operation mode according to a command input from the user (S800).

The operation mode is largely divided into a singing mode in which only the melody corresponding to the MR data is output among the accompaniment data and a practice mode in which the melody and TTS are output together. It may be configured as a second practice mode. That is, according to the user's command, the user can use the song accompaniment system in various modes.

Matters that are not described with respect to the process of using the service by the user of FIG. 8 are the same as those described for the song accompaniment system through FIGS. 1 to 7 or can be easily inferred from the described content, so the following description will be omitted. let it do

The process of the operation of the song accompaniment system according to the embodiment described with reference to FIG. 8 may also be implemented in the form of a recording medium including instructions executable by a computer, such as an application or program module executed by a computer. . Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The process of the song accompaniment system according to an embodiment of the present invention described above may be executed by an application basically installed in the terminal (which may include a program included in the platform or operating system, etc. basically installed in the terminal), It may be executed by an application (ie, a program) directly installed in the master terminal by a user through an application providing server such as an application store server, an application, or a web server related to the corresponding service. In this sense, the process of the song accompaniment system according to an embodiment of the present invention described above is implemented as an application (that is, a program) installed by default in the terminal or directly installed by the user, and a computer-readable record such as a terminal may be recorded on the medium.

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

1000: song accompaniment system
100: user terminal
110, 310: communication unit
120: input unit
130: voice recognition unit
140, 330: control unit
150: TTS data processing unit
160: output unit
170: storage unit
300: central server
320: customer care department
340: accompaniment data generation unit
350: accompaniment data management unit

Claims (10)

As a song accompaniment system,
a user terminal configured to prepare MR data including melody information for the specific song and accompaniment data including TTS (Text to Speech) data matching the MR data when a request for singing for a specific song is input from the user; and
Communication with the user terminal through a network, and when a request for transmission of accompaniment data for the specific song is received from the user terminal, based on a lyrics format including time information for each lyric line and time information for each lyric syllable of the specific song a central server configured to generate TTS data including TTS information to be output between at least one or more adjacent lyrics lines, and transmit the TTS data together with the MR data as the accompaniment data to the user terminal, and ,
The user terminal is configured to execute the accompaniment data based on an operation mode when the accompaniment data provided from the central server is prepared,
the operation mode includes a singing mode in which only the MR data is executed and a practice mode in which the TTS data is executed together with the MR data;
The practice mode is a first practice mode that reads the syllable of the next lyric line as TTS between adjacent lyric lines while the melody is in progress, and a second practice mode that reads the syllable of the lyric line leading to the TTS before the start of the melody in which the melody does not proceed. Includes 2 practice modes,
and the central server generates TTS data including information corresponding to each of the first practice mode and the second practice mode. The method of claim 1,
The user terminal determines whether accompaniment data for the specific song is stored when a request for singing for a specific song is input from the user, and when the accompaniment data is stored, the accompaniment data is transmitted to the central server. A song accompaniment system, characterized in that the accompaniment data is prepared and executed without a transmission request, and when the accompaniment data is not stored, the accompaniment data is requested to be transmitted to the central server. The method of claim 1,
The user terminal can be controlled according to the user's voice, and the operation mode of the accompaniment data is determined based on a command extracted from the voice input from the user. delete delete The method of claim 1,
The second practice mode is a song accompaniment system, characterized in that until the syllable of the lyric line is read by TTS, the progress of the melody of the corresponding lyric line is stopped. The method of claim 1,
The central server,
When a request for transmission of accompaniment data for the specific song is received from the user terminal, it is determined whether accompaniment data for the specific song is stored, and if the accompaniment data is stored, the TTS data is not generated. and transmitting accompaniment data to the user terminal, and generating the TTS data when the accompaniment data is not stored. The method of claim 1,
The central server extracts time information for an interval between adjacent lyric lines based on time information for each lyric line in the lyric format, and when the length of the interval is shorter than a predetermined criterion, between the lyric lines A song accompaniment system, characterized in that it does not generate TTS data. The method of claim 1,
The central server calculates, based on the time information for each lyric line and the time information for each lyric syllable in the lyric format, the product of the number of syllables of lyrics included in the rear lyric line among adjacent lyric lines and a predetermined TTS provision time per syllable through calculating the TTS provision time of the corresponding lyric line, and determining the TTS provision time of the lyric line from the start time of the first lyric syllable of the rear lyric line as the TTS provision time of the lyric line Song accompaniment system. 10. The method of claim 9,
The central server shortens the TTS provision time when the calculated TTS provision time of the lyric line is longer than the time interval of the adjacent lyric lines, so that the TTS provision time of the lyric line is equal to that of the adjacent lyric lines. A song accompaniment system, characterized in that it is positioned within a time interval.

Images