TWI497484B - Performance evaluation device, karaoke device, server device, performance evaluation system, performance evaluation method and program - Google Patents

Description

Performance evaluation device, karaoke device, server device, performance evaluation system, performance evaluation method, and program

The present invention relates to a technique for evaluating the merits of a musical composition.

For example, there is a technique for karaoke apparatus for singing (hereinafter, simply referred to as "karaoke apparatus" unless otherwise specified) for a karaoke apparatus having a scoring function for scoring the singers' performances. Patent Document 1 discloses a document which discloses such a technique. The karaoke apparatus disclosed in the document calculates the difference between the pitch extracted from the user's singing voice and the pitch extracted from the data prepared in advance as a guide melody for each note of the song, and calculates the basic based on the difference. Score. Further, the karaoke apparatus calculates a bonus point corresponding to the number of times the singing is performed when a singer such as a vibrato or a hiccup is used. The karaoke apparatus presents the total score of the basic score and the bonus points as a final evaluation result to the user. According to this technique, it is possible to reflect the singing result using a skill such as vibrato or upper glide sound.

In addition, as a document which discloses a technique of singing using a technique such as vibrato or upper-slide, the waveform of the vocal sound is detected, for example, Patent Documents 2 to 6.

[Previous Technical Literature] [Patent Literature]

[Patent Document 1] Japanese Patent Laid-Open Publication No. 2005-107334

[Patent Document 2] Japanese Patent Laid-Open Publication No. 2005-107330

[Patent Document 3] Japanese Patent Laid-Open Publication No. 2005-107087

[Patent Document 4] Japanese Patent Laid-Open Publication No. 2008-268370

[Patent Document 5] Japanese Patent Laid-Open Publication No. 2005-107336

[Patent Document 6] Japanese Patent Laid-Open Publication No. 2008-225115

However, in the case of the technique of Patent Document 1, even when the singing is performed on a singing part which is not intended to use a technique such as vibrato or upper glide, a bonus point is added. Therefore, there is a problem that the score that is suggested as the evaluation result deviates from the feeling of the person.

The present invention has been made in view of such a problem, and an object of the present invention is to provide an evaluation result closer to a person's feelings in the evaluation of a musical performance such as karaoke singing.

In order to solve the above problems, the present invention provides a performance evaluation apparatus including: an expression playing reference data acquiring unit that acquires a sounding start or a note group included in a music piece as a reference, and indicates that the music piece should be played in the music performance The expression playing reference and the expression playing the reference performance in the above-mentioned music; the pitch volume data generating mechanism generates a tone data indicating the pitch and volume of the performance sound from the performance sound of the music played by the player And a performance evaluation mechanism, wherein the characteristic of at least one of the pitch and the volume indicated by the pitch volume data generated by the pitch volume data generating means indicates a specific time indicated by the expression performance reference material in the music piece In the case where the characteristics of the expression performance performed by the above-described expression playing reference material are within the range, the evaluation of the performance of the above-described music performed by the above-mentioned performer is improved.

Moreover, the present invention provides a karaoke apparatus comprising: the performance evaluation apparatus; an accompaniment data acquisition unit that acquires accompaniment data indicating an accompaniment of a music piece; and a sound signal output mechanism that outputs an accompaniment according to the indication of the accompaniment material The sound signal of the music; the above-mentioned pitch volume data generating mechanism generates a representation corresponding to the sound from the sound The sound signal output from the sound signal output mechanism and the accompaniment played from the speaker is performed by the player to perform the pitch sound of the music and the volume sound volume data of the volume.

Moreover, the present invention provides a server device comprising: an expression playing appearance data acquiring unit that acquires a note indicating the music included in the music piece for each of a number of performance sounds of the music piece performed by an arbitrary player The beginning of the pronunciation of the note group is an expression playing performance appearing at one time of the reference; the expression playing reference data generating mechanism is based on any number of expressions played by the above-mentioned expression playing data obtaining means, for Each of the notes or note groups included in the music piece specifies whether any of the expressions at any one of the timings based on the pronunciation start time of the note or the note group appears at any frequency, according to the specific information. And generating an expression playing reference material that is to be performed in the performance of the music piece and the time when the expression is performed in the music piece, based on the pronunciation start time of the note or the note group included in the music piece; and a sending mechanism that will be generated by the above-described expression playing reference material generating mechanism Performance Evaluation of performance transmitted to the reference device.

Moreover, the present invention provides a singing evaluation system including: an expression playing reference data acquiring unit that acquires an expression playing in the performance of the music piece based on the pronunciation start time of the note or the note group included in the music piece. a first expression performance reference material that plays the time sequence that should be performed in the music piece; the tone volume data generation mechanism generates a tone volume data indicating the pitch and volume of the performance sound from the performance sound of the music piece played by the player a performance evaluation mechanism that is characterized by at least one of a pitch and a volume indicated by the pitch volume data generated by the pitch volume data generating means, and is represented by a specific one of the first expression performance reference data in the music In the case of the expression performance characteristics of the first expression playing reference material in the time range, the evaluation of the performance of the music performed by the above-mentioned performer is improved; the expression playing appearance data acquisition mechanism is performed for any number of arbitrary performances. For each of the performances of the music performed by the person, obtain the representation Above any of the above-mentioned music player is playing the notes contained in the note or The pronunciation start time of the group is an expression playing performance appearing at one time of the reference; and the expression playing reference data generating mechanism is based on any number of expression playing data acquired by the above-mentioned expression playing data obtaining means, Whether each of the notes or the note group included in the music played by any of the above-mentioned players specifies whether the expression of any of the expressions appears at any frequency based on the timing of the start of the pronunciation of the note or the note group, According to the specific information, the expression start time of the note or the note group included in the music played by any of the above-mentioned players is used as a reference, and the expression performance to be performed in the performance of the music performed by any of the above players is indicated. The expression plays a second expression playing reference in the sequence that should be performed by the music played by any of the above players.

Further, the present invention provides a performance evaluation method for obtaining an expression performance to be performed in the performance of the music piece and the expression performance in the music piece based on the pronunciation start time of the note or the note group included in the music piece. The timing expression playing reference material, the playing sound of the music piece played by the performer produces the pitch sound data indicating the pitch and volume of the playing sound, and the characteristics of at least one of the pitch and the volume indicated by the pitch volume data. When the expression performance characteristic to be performed by the above-described expression performance reference material in the specific time range indicated by the above-described expression performance reference material in the above-described music piece is expressed, the evaluation of the performance of the music piece by the player is improved.

Moreover, the present invention provides a program executable by a computer, and causing the computer to perform the following processing: an expression playing reference material acquisition process, which acquires the above-mentioned pronunciation start time of the note or the note group included in the music piece, indicating that the above The expression of the expression to be played in the performance of the music and the expression of the expression in the above-mentioned music, and the tone volume data generation processing, which is generated by the performance of the above-mentioned music played by the player, indicating the performance And a tone evaluation volume data; and a performance evaluation process, wherein the characteristics of at least one of the pitch and the volume indicated by the pitch volume data generated by the pitch volume data generating means are represented by the music The specific time range indicated by the above-mentioned expression performance reference material shall be performed by the above-mentioned expression playing reference material. In the case of the expression playing characteristic, the evaluation of the performance of the above-mentioned music performed by the above-mentioned performer is improved.

According to the present invention, it is possible to realize a performance evaluation apparatus that gives a higher evaluation to a player if the desired expression is performed at a desired timing in the performance of each piece of music. As a result, when the performer performs the expression of the expression, the evaluation is less than the feeling of the person.

1‧‧‧Singing Evaluation System

10‧‧‧ Karaoke device

10-m‧‧‧ Karaoke device

11‧‧‧ source

12‧‧‧ Speaker

13‧‧‧ microphone

14‧‧‧Display Department

15‧‧‧Communication interface

16‧‧‧Sound Adapter

17‧‧‧CPU

18‧‧‧RAM

19‧‧‧ROM

20‧‧‧ Hard disk

21‧‧‧Voice storage

30‧‧‧Server device

35‧‧‧Communication interface

37‧‧‧CPU

38‧‧‧RAM

39‧‧‧ROM

40‧‧‧ Hard disk

90‧‧‧Network

101‧‧‧Face expression reference material acquisition agency

102‧‧‧ tone volume data generating mechanism

103‧‧‧ Performance Evaluation Agency

APG‧‧‧Singing Analysis Program

DBS‧‧‧ singing sample database

DBRK‧‧‧ Reference Library

DBRS‧‧‧ Reference Library

HD‧‧‧ heading

MD-n‧‧‧ Track Information

S _A ‧‧‧Sound signal

S _I ‧‧‧ image signal

S _L ‧‧‧Output signal

S _M ‧‧‧ radio signal

S _P ‧‧‧Output signal

TR _AC ‧ ‧ accompaniment track

TR _LY ‧ ‧ lyrics track

TR _NR ‧‧‧Fan Ben singing reference track

VPG‧‧‧ singing evaluation program

Fig. 1 is a view showing the configuration of a singing evaluation system according to an embodiment of the present invention.

Fig. 2 is a diagram showing the waveform of the singer of the sustain.

Fig. 3 is a diagram showing the waveform of the vocal sound of the vibrato.

Fig. 4 is a diagram showing the waveform of the singing sound of the decorative sound.

Fig. 5 is a view showing the waveform of the singing sound of the upper sliding sound.

Fig. 6 is a diagram showing the waveform of the singing sound of the glide sound.

Fig. 7 is a flow chart showing the operation of the singing evaluation system according to an embodiment of the present invention.

Figure 8 is an example of statistics generated for sustain.

Figure 9 is an example of statistics for vibrato generation.

Figure 10 is an example of statistics for decorative sound generation.

Figure 11 is an example of statistics for the generation of upper glide.

Figure 12 is an example of statistics for glide generation.

Figure 13 is a block diagram showing the performance evaluation apparatus of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Fig. 1 is a view showing the configuration of a singing evaluation system 1 according to an embodiment of the present invention. The singing evaluation system 1 has a karaoke apparatus 10-m (m=1, 2...M, M series karaoke equipment The total number is set) with the server device 30. The karaoke apparatus 10-m is provided with one or a plurality of sets in each karaoke shop. The server device 30 is located in the system operation center. The karaoke apparatus 10-m and the server apparatus 30 are connected to the network 90, and can transmit and receive various materials to each other.

The karaoke apparatus 10-m is a device for evaluating the performance of the singing performance by the user's singing accompaniment and the lyric display completion, and the user's singing performance. Here, the karaoke apparatus 10-m evaluates whether the pitch of the user's singing voice and the volume are good or not, and the evaluation of the five types of expressions as shown below. No is the evaluation of the evaluation object. The scores of the evaluation results of the two evaluations are presented to the user together with the comment message.

A1. sustain

It is a deliberate delay in the expression of the song of the specific sound in the song. As shown in FIG. 2, in the case of performing the singing, the sound of the sound is changed from the sound before the singing sound to the time when the singing sound changes, and the two musical notes (notes) corresponding to the two sounds in the musical score (the singer singing). The transfer time is only delayed for a short time.

B1. vibrato

It sings in the face of a particular tune in the song while maintaining a subtle tone. As shown in FIG. 3, in the case of performing the singing, the pitch of the singing voice periodically changes across the height of the musical note corresponding to the note.

C1. decorative sound

It makes the sound of the specific sound in the song sing in the way of screaming on the way of pronunciation. As shown in FIG. 4, in the case of the singing, the pitch of the singing voice rises in the middle of the musical score corresponding to the note of the sound.

D1.

It is a singing method that is pronounced at a height close to the original sound after the specific sound in the song is sounded at a lower level than the original height. As shown in FIG. 5, in the case of performing the singing, the pitch of the beginning of the pronunciation of the singing voice is lower than the height of the musical note corresponding to the note in the musical score. and, The pitch of the singing voice slowly rises after the beginning of the pronunciation and reaches a height substantially equal to the height of the note.

E1. down sound (fall)

It is a singing method that pronounces a specific sound in a song with a height higher than the original height and approaches the original height. As shown in FIG. 6, in the case of performing the singing, the pitch of the beginning of the pronunciation of the singing voice is higher than the height of the musical note corresponding to the note in the musical score. Moreover, the pitch of the singing voice slowly descends after the start of the pronunciation, reaching a height substantially equal to the height of the note.

Returning to Fig. 1, the description of the overall singing evaluation system 1 is continued. The karaoke apparatus 10-m has a sound source 11, a speaker 12, a microphone 13, a display unit 14, a communication interface 15, a sound adapter 16, a CPU (Central Processing Unit) 17, and a RAM (Random Access Memory). Memory 18, ROM (Read Only Memory) 19, hard disk 20, and sound memory 21 (sequencer). The sound source 11 outputs a sound signal S _{A according to} various messages of a MIDI (Musical Instrument Digital Interface). The speaker 12 plays the signal given as a sound. The microphone 13 receives the sound and outputs a sound pickup signal S _M . The display section 14 displays an image corresponding to the image signal S _I . The communication interface 15 transmits and receives data between devices connected to the network 90.

The sound adapter 16 measures the pitch and volume of the sound signal S _M and produces tone volume data indicating the time variation of the sound signal. Specifically, the sound adapter 16 (corresponding to the tone volume data acquisition means) detects the pitch of the sound signal S _M given from the microphone 13 every time T _S (for example, T _S = 30 milliseconds), and uses the detection result as the signal _SP Output. Further, the sound adapter 16 detects the volume of the sound signal S _M supplied from the microphone 13 every time T _S , and outputs the detection result as the signal S _L .

The CPU 17 uses the RAM 18 as a work area and executes a program stored in the ROM 19 or the hard disk 20. Details of the operation of the CPU 17 will be described below. An IPL (Initial Program Loader) or the like is stored in the ROM 19. Memory in the hard disk 20 The track data MD-n (n=1~N) of various songs (the total number of N-type song types), the reference database DBRK, and the singing evaluation program VPG. The track data of each song MD-n records the accompaniment content of the song, the lyrics of the song, and the sung content of the song in the form of SMF (Standard MIDI File).

Specifically, as shown in the frame of FIG. 1, the track data MD-n includes a header HD, an accompaniment track TR _AC , a lyric track TR _LY , and a template vocal reference track TR _NR . In the header HD, information such as the track number, the name of the song, the type, the playing time, and the time base (the number of ticks corresponding to the time of one note in the quarter note) are described.

In the accompaniment track TR _AC , each note NT(i) indicating the accompaniment portion of the musical score of the song is recorded in chronological order (i indicates the order in which the note NT(1) from the head of the musical score is calculated). The pronounced event EV(i) _{ON is} the difference time DT between the event EV(i) _OFF indicating the silence, and the execution time difference (the number of ticks) indicating the before and after events.

In the lyric track TR _LY , the data D _LY indicating the lyrics of the song and the display time indicating the lyrics are displayed in chronological order (more specifically, the display time of each lyric and the display time of the lyrics before each of the lyrics) The time difference between the time difference (the number of clicks)) is DT.

In the singer-song reference track TR _NR , an event EV(i) _ON indicating the sound of the note of the note NT(i) of the singing part of the score of the song and an event indicating the muffling thereof EV(i) _OFF are described in chronological order. And the difference time DT indicating the execution time difference (the number of ticks) of the events before and after.

The reference database DBRK (corresponding to the expression playing reference material generating mechanism) memorizes five kinds of expression singing reference materials DD _a1 , DD _a2 , DD _a3 , DD _a4 , DD _a5 . The expression singing reference material DD _a1 indicates the evaluation of the timing of the sounding of the note NT(i) contained in the song as the reference time t _BS on each time t on the time axis and the sustaining singing at the time t. The data of each pair of VSR(t). The expression vocal reference material DD _a2 indicates that each of the time t on the time axis of the reference point t _{BS and} the tremolo singing at the time t are the pronunciation start time of the note NT(i) included in the song. Information on each pair of VSR(t). The expression singing reference material DD _a3 indicates the evaluation of the time when the pronunciation start time of the note NT(i) included in the song is the time t on the time axis of the reference point t _{BS and} the decorative sound singing at the time t. The data of each pair of VSR(t). The expression singing reference material DD _a4 indicates the evaluation of the situation in which the pronunciation start time of the note NT(i) included in the song is the time t on the time axis of the reference point t _BS and the case where the upper slide sound is performed at the time t. The data of each pair of VSR(t). The expression vocal reference material DD _a5 indicates the evaluation of the situation in which the pronunciation start time of the note NT(i) included in the song is the time t on the time axis of the reference point t _{BS and} the singer singing at the time t. The data of each pair of VSR(t). Hereinafter, the case where the five kinds of expression singing reference materials DD _a1 , DD _a2 , DD _a3 , DD _a4 , and DD _{a5 are} not distinguished is recorded as the expression singing reference material DD.

The singing evaluation program VPG has the following three functions.

A2. Standard evaluation function

Each event EV sound adapter by which the output signal line S _L 16 and the pitch of the S _P and volume as shown by the reference template singing track TR _NR (corresponding to a reference model acquired playing means) within (i) _ON and EV (i) The sample pitch PCH _REF and the template volume LV _REF of each note NT(i) determined by _OFF are compared, and the function of singing is evaluated based on the result of the comparison.

B2. Expression singing evaluation function

It is a function of: when the characteristic waveform of the expression singer appears in the pitch waveform indicated by the output signal S _{P of the} sound adapter 16, the pronunciation start time of the note NT(i) which becomes the object of the expression sing is used as the reference point. The appearance time of the characteristic waveform of the expression singing on the time axis of the t _BS , and the evaluation score VSR corresponding to the appearance time is selected from the evaluation scores VSR(t) of the equivalent expression singing reference data DD in the reference database DBRK ( t), based on the evaluation score VSR (t) evaluation of singing pros and cons.

C2. Evaluation result prompt function

It is calculated from the evaluation result of the evaluation performed by a2 and the evaluation result of the evaluation by b2 The score is such that the score is displayed on the display unit 14 together with the comment message.

The sound storage 21 is a case where the track data MD-n of the equivalent track is transferred from the hard disk 20 to the RAM 18 based on the singing start operation of the song by the remote controller (not shown), the track material MD- The events EV(i) _ON , EV(i) _OFF and data D _{LY are} supplied to each part of the device. Specifically, if the track data MD-n is stored in the RAM 18, the sound memory 21 is based on the time base of the header HD described in the track data MD-n and the beat specified by the remote controller (not shown). The length of time for specifying one click is long, and the following three processes are performed based on the count of the ticking after the length of time.

In the first processing, the sound storage 21 reads the subsequent event EV(i) _ON (or EV(i) _OFF ) whenever the calculated number of clicks coincides with the difference time DT in the accompaniment track TR _AC . And supplied to the sound source 11. When the sound source 11 supplies the event EV(i) _{ON to the} sound storage 21, the sound signal S _A designated by the event EV(i) _{ON is} supplied to the speaker 12, and if the event EV is supplied from the sound storage 21 ( i) _OFF , the supply of the sound signal S _A to the speaker 12 is stopped.

In the second processing, the sound storage 21 (corresponding to the lyric data acquisition means) reads the subsequent data D _LY and supplies it to the time when the calculation number of the ticks coincides with the difference time DT in the lyric track TR _LY The display unit 14 (corresponding to an image signal output mechanism). When the material D _LY is supplied from the sound storage 21, the display unit 14 converts the material D _LY into an image of the lyrics subtitle and displays the image on a display (not shown).

The first and second processes are performed by the sound storage 21, and the accompaniment sound is played from the speaker 12 and the lyrics are displayed on the display. The user listens to the accompaniment sound played from the speaker 12 and sings the lyrics displayed on the display toward the microphone 13. While the user is singing to the microphone 13, the microphone 13 outputs the sound signal S _{M of the} user's singing voice, and the sound adapter 16 outputs signals S _P and S _L indicating the pitch and volume of the signal S _M .

In the third processing, the sound storage 21 reads the subsequent event EV(i) _ON (or EV(i) every time the calculated number of clicks coincides with the difference time DT in the template singing reference track TR _NR . _OFF ) and supplied to the CPU 17. The CPU 17 evaluates the merits of the user's singing using the events EV(i) _ON and EV(i) _OFF supplied from the sound storage 21 and the output signals S _P and S _{L of the} sound adapter 16. Details are described below.

The server device 30 is a device that functions to support the provision of a server of a karaoke store. The server device 30 has a communication interface 35, a CPU 37, a RAM 38, a ROM 39, and a hard disk 40. The communication interface 35 transmits and receives data between devices connected to the network 90. The CPU 37 uses the RAM 38 as a work area, and executes various programs stored in the ROM 39 or the hard disk 40. Details of the operation of the CPU 37 are described below. An IPL or the like is stored in the ROM 39.

The singing sample database DBS, the reference database DBRS, and the singing analysis program APG are stored in the hard disk 40. The singing sample data DS group corresponding to one song is individually recorded in the singing sample database DBS. Singing sample data DS is a record of the pitch waveform and volume waveform of a singing voice when a singer with a certain level or higher is singing. The latest expression singing reference material DD stored in the reference database DBRK of each karaoke apparatus 10-m is stored in the reference database DBRS.

The singing analysis program APG has the following three functions.

A3. Storage function

It acquires the singing sample data DS of each song of one track from the karaoke apparatus 10-m one by one, and stores the obtained singing sample data DS in the function of the singing sample database DBS.

B3. Overwrite function

The function is as follows: for each of the singing sample data DS stored in the singing sample database DBS, searching for the characteristic waveform of the expression singing from the waveform indicated by the singing sample data DS, and generating a representation by the search result to become an expression The pronunciation start time of the object of the singing object NT(i) is the statistical data of the relationship between the time t on the time axis of the reference point t _BS and the appearance number Num of the expression singing at the time t, based on the contents of the statistical data. The evaluation score VSR(t) corresponding to each time t in the expression singing reference material DD in the reference database DBR is overwritten.

C3. Send function

It is a function of transmitting the expression singing reference material DD overwritten by the overwriting function to the karaoke apparatus 10-m in accordance with the request from the karaoke apparatus 10-m.

Next, the operation of this embodiment will be described. Fig. 7 is a flow chart showing the operation of this embodiment. 7, karaoke OK device 10-m of the CPU17 when to be the case of singing songs of the start operation of (S100: Yes (Yes)), the supply of the sound in the reservoir 21 control signals S ₀ so that sound in the reservoir 21 begins processing (The first to third processes described above) (S120). When the processing of the sound storage 21 is started, the CPU 17 performs the two processes of the standard singing evaluation processing (S130) and the expression singing evaluation processing (S140). Details of the two processes are as follows.

A4. Standard singing evaluation processing (S130)

In this process, the CPU 17 is supplied to the next sound in the event EV from reservoir 21 is supplied event EV (i) _{the ON} (i) until the time of the _OFF to NT corresponding to the i-th note (i) the time of audio pronunciation T _NT (i). CPU17 obtained pitch conversion event EV (i) the difference between the note number of _ON PCH _REF of the pitch template shown PCH _DEF between the pronunciation of audio time T _NT (i) the output signal S 16 of the adapter _P, and between the determining when the volume of the converter shown in FIG event EV after the signal S _P (i) _oN template of the beat volume LV _REF difference LV _DEF, and to the difference PCH _DEF LV _DEF control difference to the case of a specific range of notes NT (i ) singing is qualified. The CPU 17 performs the note determination from the start to the end of the singing, and multiplies the value obtained by dividing the number of the musical notes NT(i) judged as qualified by the number of all notes TN(i) at the end time of the singing by 100. The value is taken as the base score SR _BASE .

Further, in this process, CPU 17 determines whether there is illustrated the damper of audio tones within the adapter 16 as the output signal S _P wave, tremolo, wherein any of ornaments, the glide, the decline of the sound waveform of a singing expression. Here, for details of the method of determining the characteristic waveform of the sustaining sound, refer to Patent Document 2, and the details of the method for determining the characteristic waveform of the vibrato refer to Patent Document 3, and the details of the method for determining the characteristic waveform of the decorative sound refer to Patent Document 4, Upper Slider For details of the method of determining the characteristic waveform, refer to Patent Document 5, and the details of the method for determining the characteristic waveform of the glide are referred to Patent Document 6. The CPU 17 performs the feature waveform determination from the time when the user starts singing to the end of the singing, and multiplies the number of appearances of the expression singing at the end of the singing time by the specific coefficient as the addition score SR _ADD . Moreover, in this processing, the total of the base score SR _BASE and the addition score SR _ADD is taken as the standard score SR _NOR .

B4. Expression singing evaluation processing (S140)

In this process, the CPU 17 sets the time from the output of the sound source event EV(i) _{ON to} the output of the next event EV(i) _OFF as the sounding time T _NT corresponding to the sound of the ith note NT(i) ( i). Further, when the case wherein the expression singing tone waveforms shown occur within the time of the sound between the pronunciation T _NT (i) of the adapter output signal S _P wave 16, CPU 17 obtained within the expression pronunciation time T _NT (i) Singing The moment of appearance and the type of expression that appears. The CPU 17 generates an expression sing presenting information indicating the kind of the expression singer and the appearance time.

Then, the CPU 17 selects the evaluation score VSR(t) corresponding to the expression singing shown in the generated expression singing appearance data and its appearance time from the series evaluation score VSR(t) shown in the expression singing reference material DD. The CPU selects the evaluation score VSR(t) while the user starts singing to finish singing, and uses the average value of the evaluation score VSR(t) of the singing end time as the expression score SR _EX .

When the user's song singing ends, the CPU 17 performs an evaluation result presenting process (S150). In the evaluation result presentation processing, the CPU 17 selects the score of the higher of the standard score SR _NOR scored by the standard singing evaluation processing and the expression score SR _EX scored by the expression singing evaluation processing. Then, when the CPU 17 selects the standard score SR _NOR , the score SR _NOR is displayed on the display unit 14 with a comment message corresponding to the score SR _NOR such as "cool and delicate song". Furthermore, if the situation in the CPU17 selection expression score of SR _EX, SR _EX and the scores such as "full of emotion it", etc. correspond to the expression of SR _EX score comment message on the display unit 14.

Next, the CPU 17 performs sample transmission processing (S160). In the sample transmission processing, the CPU 17 uses the signals S _P and S _L output from the sound adapter 16 during the start to the end of the singing of the song as the singing sample data DS of the song, and includes the singing sample data DS and the step S130. The message MS1 of the basic score SR _BASE (singing evaluation data) is sent to the server device 30.

If the message MS1 is obtained from the karaoke device 10-m (S200: Yes), the CPU 37 of the server device 30 takes out the singing sample data DS and the basic score SR _BASE from the message MS1, and divides the basic score SR _BASE with the superior class and The non-superior's benchmark score SR _TH (for example, 80 points) is compared (S220). When the basic score SR _{BASE is} higher than the reference score SR _TH (S220: Yes), the CPU 37 stores the singing sample data DS taken out from the message MS1 in the singing sample database DBS (S230).

Next, the CPU 37 (corresponding to the expression playing appearance data generating means) performs overwriting processing (S240). In the overwrite processing, the CPU 37 performs the following five processing. In the first processing, the CPU 37 searches for the characteristic waveform of the sustaining sound from the pitch waveforms stored in the singing sample data DS stored in the singing sample database DBS, and generates an expression vocal appearance data indicating the search result (indicating that the sustaining sound is present). The pronunciation start time of the note NT(i) is the data of each time t on the time axis of the reference point t _BS ). Next, the CPU 37 sings the appearance data based on the expression regarding the sustain, and generates the time t at the time axis of the reference point t _{BS with} the pronunciation start time of the note NT(i) and the expression "song" at the time t The statistical data of the relationship of the occurrence number Num is overwritten with the evaluation score VSR(t) corresponding to each time t in the expression singing reference material DD _a1 based on the content of the statistical data.

Figure 8 is a diagram showing an example of statistics for sustain. This embodiment of the statistics, the t _BS advance and delay time t1 _a1 t _BS than the reference point in time than the reference point of time T1 _a1 distribution between time t4 _a1 T4 _a1 has the number of emotion of singing Num. Further, in the statistical data of this example, the maximum peak of the number Num appears at the time t2 _a1 immediately after the reference point t _BS , and the second peak of the number Num appears at a time t3 _a1 later than the time t2 _a1 . Whereby, after use of the embodiment of statistics override the singing expression references DD _a1, the evaluation time points t2 _a1 VSR (t2 _a1) Evaluation of the highest points of time t3 _a1 VSR (t3 _a1) second high.

In the second processing, the CPU 37 searches for the characteristic waveform of the vibrato from the pitch waveforms stored in the respective singing sample data DS of the singing sample database DBS, and generates an expression singing material indicating the search result (indicating the note NT appearing in vibrato) The pronunciation start time of (i) is the data of each time t on the time axis of the reference point t _BS ). Next, the CPU 37 sings the appearance data based on the expression regarding the vibrato, and generates the number of occurrences of the expression sings on the time axis representing the start time of the pronunciation of the note NT(i) as the reference point t _BS and the time t at the time t. The statistical data of the relationship of Num, based on the content of the statistical data, overwrites the evaluation score VSR(t) corresponding to each time t of the expression singing reference material DD _a2 .

Figure 9 is a diagram showing an example of statistics for vibrato. In the statistical data of this example, the number Num of expression sings is distributed between the reference point t _BS and the time t2 _a2 which is later than the reference point t _{BS by the} time T2 _a2 . Further, in the statistical data of this example, the maximum peak value of the number Num appears at the time t1 _a2 which is delayed by the time T1 _{a2 from} the reference point t _BS . Therefore, in the expression vocal reference material DD _a2 after the overwriting of the statistical data of this example, the evaluation score VSR(t1 _a2 ) at the time t1 _{a2 is} the highest.

In the third processing, the CPU 37 searches for the characteristic waveform of the decorative sound from the pitch waveforms stored in the singing sample data DS stored in the singing sample database DBS, and generates the expression singing appearance data indicating the search result (indicating that the decorative sound appears) The pronunciation start time of the note NT(i) is the data of each time t on the time axis of the reference point t _BS ). Next, the CPU 37 sings the appearance data based on the expression related to the decorative sound, and generates each time t on the time axis indicating the start time of the pronunciation of the note NT(i) as the reference point t _{BS and} the number of expression sings at the time t The statistical data of the relationship of Num is overwritten with the evaluation score VSR(t) corresponding to each time t in the expression singing reference material DD _a3 based on the content of the statistical data.

Fig. 10 is a view showing an example of statistics for decorative sounds. In the statistical data of this example, the number Num of expression sings is distributed between the reference point t _BS and the time t2 _a3 which is later than the reference point t _{BS by the} time T2 _a3 . Further, in the statistical data of this example, the maximum peak of the number Num appears at the time t1 _a3 which is delayed by the time T1 _{a3 from} the reference point t _BS . Therefore, in the expression singing reference material DD _a3 after the overwriting of the statistical data of this example, the evaluation score VSR(t1 _a3 ) at the time t1 _{a3 is} the highest.

In the fourth processing, the CPU 37 searches for the characteristic waveform of the upper sliding sound from the pitch waveforms stored in the singing sample data DS stored in the singing sample database DBS, and generates an expression vocal appearance data indicating the search result (indicating the note in which the above sliding sound appears) The pronunciation start time of NT(i) is the data of each time t on the time axis of the reference point t _BS ). Next, the CPU 37 generates an expression vocal appearance data based on the upper gliding sound, and generates an expression singer indicating the time t on the time axis of the reference point t _BS of the note NT(i) and the time t at the time t. The statistical data of the relationship of the number Num overwrites the evaluation score VSR(t) corresponding to each time t in the expression singing reference material DD _a4 based on the content of the statistical data.

Figure 11 is a diagram showing an example of statistics for the upper glide. In the statistical data of this example, the number Num of expression sings is distributed between the reference point t _BS and the time t2 _a4 which is later than the reference point t _BS delay time T2 _a4 . Further, the statistics of the embodiment, the reference point t _BS Num exhibits the largest number of peaks appear in the delay time t _BS than the reference point of time t1 _a 4 T1 _a4 manifestations number Num of the second peak appears. Whereby, after use of the embodiment of statistics override the reference DD _a4 singing expression, the evaluation time t divided _BS the VSR (t _BS) the highest evaluation point of time t1 _a4 VSR (t1 _a4) a second high.

In the fifth processing, the CPU 37 searches for the characteristic waveform of the glide sound from the pitch waveforms stored in the respective singing sample data DS of the singing sample database DBS, and generates an expression vocal appearance data indicating the search result (indicating the note of the following glide) The pronunciation start time of NT(i) is the data of each time t on the time axis of the reference point t _BS ). Next, the CPU 37 sings the appearance data based on the expression relating to the glide sound, and generates the time t of the expression on the time axis indicating the start time of the pronunciation of the note NT(i) as the reference point t _{BS and} the number of expressions of the singer at the time t The statistical data of the relationship of Num, based on the content of the statistical data, overwrites the evaluation score VSR(t) corresponding to each moment in the expression singing reference material DD _a5 .

Figure 12 is a diagram showing an example of statistics for glide sounds. Distribution between t2 _a5 embodiment of the statistics, t _BS t _BS delay in delay time than the reference time T1 _a5 point of time from the time t1 _a5 and the time T2 _a5 has the number of emotion singing Num. Further, in the statistical data of this example, the maximum peak of the number of occurrences Num is expressed at time t2 _a5 . Therefore, in the expression vocal reference material DD _a5 after the overwriting of the statistical data of this example, the evaluation score VSR(t2 _a5 ) at the time t2 _{a5 is} the highest.

In Fig. 7, the CPU 17 of the karaoke apparatus 10-m performs an inquiry process every time a predetermined inquiry time comes (S110: Yes) (S170). In the inquiry processing, the CPU 17 transmits the message MS2 for which the latest data is transmitted to the server device 30 (S170). If the message MS2 is received from the karaoke apparatus 10-m (S210: Yes), the CPU 37 of the server apparatus 30 will overwrite the content expression between the reception time of the last message MS2 and the reception time of the current message MS2. The singing reference material DD is sent to the karaoke apparatus 10-m of the transmitting end of the message M2 (S250). Upon receiving the expression singing reference material DD from the server device 30, the CPU 17 of the karaoke apparatus 10-m overwrites the expression singing reference material DD in the reference database DBRK to update its content (S180).

The above is the details of the configuration of this embodiment. According to this embodiment, the following effects can be obtained.

First, in the vocal evaluation process with an expression in the present embodiment, whenever the characteristic waveform of the expression singer appears in the waveform of the output signal of the sound adapter 16, the note NT(i) which becomes the object of the expression sing is sought. The start time of the pronunciation is the appearance time of the characteristic waveform of the expression singing on the time axis of the reference point, and the evaluation score VSR(t) corresponding to the appearance time is selected from each evaluation score VSR(t) in the singing reference material DD, based on The evaluation of this selection is based on VSR(t) to evaluate the pros and cons of singing. Therefore, according to the present embodiment, even if the user performs expression singing, if the timing is not appropriate, a good evaluation cannot be obtained. Therefore, according to the present embodiment, it is possible to present an evaluation result closer to the feeling of the person.

Secondly, in the present embodiment, for each of the expression singing reference materials DD stored in the singing sample database DBS, the characteristic waveform of the expression singing is searched from the waveform indicated by the data DD, and the search result is generated by the search result. The utterance of the note of the expression of the singer NT(i) is the statistical data of the relationship between the time on the time axis of the reference point and the number of appearances of the expression sings at the time, and is overwritten based on the contents of the statistical data. The evaluation corresponding to each moment in the singing reference material DD is divided into VSR(t). As a result, according to the present embodiment, it is possible to reflect the change in the tendency of the singing method of the perfect singing songs to the evaluation result.

Although an embodiment of the present invention has been described above, the present invention may have other embodiments. For example, as described below.

(1) In the above embodiment, CPU 17 outputs sound from the adapter detection signal S _P of the damper 16, vibrato, embellishments, glide on, down the five kinds of expressions singing sound. However, it is also possible to detect the expressions of the five other types of singing. For example, you can also detect the singing of the swaying.

(2) In the above embodiment, CPU 17 using both the sound adapter _L 16 output signal S _P and S of the standard evaluation processing singing, only the sound of the adapter output signals S 16 and S _{L P} represents the pitch of the signal S _P for Expression singing evaluation processing. However, the CPU 17 can also perform standard singing evaluation processing using only one of the signals S _P and S _L . Further, the CPU 17 can perform the expression singing evaluation processing using both of the signals S _P and S _L .

(3) In the expression singing evaluation processing of the above embodiment, the merits of the singing are evaluated based on the appearance timing of the characteristic waveform of the expression singing. However, it is also possible to evaluate the elements other than the appearance time of the characteristic waveform to which the expression sing is added (for example, the length or depth of each of the sustain, vibrato, decorative sound, upper-slide, and glide).

(4) In the expression singer evaluation processing of the above-described embodiment, the composition of the expression sing which appears in the vocal sound corresponding to each note included in the song is detected, but it is also possible to detect that a plurality of ones corresponding to the song are included The composition of the expression of singing in the singing voice of a note (note group). For example, if it is getting stronger. Fading expression singing in a series of multiple tones The expression of the singer in the singer is sung, so the detection and evaluation of the singer is ideal for the group of notes. Therefore, the expression vocal reference DD for such expressions is also ideally composed of musical note group units.

(5) In the above embodiment, the karaoke apparatus 10 transmits the singing sample data including the signals S _P and S _L outputted from the sound adapter 16 during the period from the start to the end of the singing of the song from the karaoke apparatus 10 to the server apparatus 30. The DS (tone volume data) is used in the server device 30 to perform the detection of each expression singing and the specific processing of the appearance timing by the singing sample data DS. Alternatively, a configuration is adopted in which the karaoke apparatus 10 transmits a sound signal S _M (a sound waveform data indicating a singing sound) indicating the sound collected by the microphone 13 to the server device 30, and performs the processing in the server device 30. The processing of the signal S _P and the signal S _L from the sound signal S _M (the processing performed by the sound adapter 16 in the above embodiment). Further, a configuration may be employed in which the karaoke apparatus 10 transmits to the server device 30 the type of the expression singer and the appearance timing thereof when the expression singer evaluation processing (S140) performed in accordance with the singing evaluation program VPG (expression) The vocal appearance data is not performed in the server device 30, and the expression vocal appearance data transmitted from the karaoke device 10 is used to update the expression vocal reference material DD.

(6) In the above embodiment, the server device 30 performs the generation of the statistical data and the overwriting of the expression singing reference material DD based thereon. However, each karaoke apparatus 10-m may also be a sound signal SM representing a vocal sound previously generated by the self-machine or obtained from other karaoke apparatus 10-m or via the server apparatus 30, or by the sound signal S _M The generated signal S _P and the signal S _L or the data indicating the type of the expression singing and the timing of the appearance of the signal (the expression singing appearance data) are pre-recorded on the hard disk 20, and the CPU 17 reads and uses the servo. The device device 30 performs the same processing as that performed by S240, that is, generation of statistical data and overwriting of the expression singing reference material DD based thereon.

(7) The evaluation method of the singing in the above embodiment and the manner in which the singer presents the evaluation result can be variously changed. For example, in the above-described embodiment, the standard score SR _NOR is calculated by adding the addition score SR _ADD calculated based on the number of appearances of the expression singing in the standard singing evaluation process (S130) to the basic score SR _BASE , but may be used. The composition of the basic score SR _BASE is calculated only without considering the appearance of the expression singing in the standard singing evaluation process. Further, in the above-described embodiment, the score of the higher of the standard score SR _NOR scored by the standard singing evaluation process and the expression score SR _EX scored by the expression singing evaluation process is displayed to the singer, but both of them may be displayed. And display the total score of the class, etc., and the other results to the singer's evaluation results.

(8) In the above embodiment, the singer who has the basic score SR _BASE higher than the reference score SR _TH is used as the superior in the update of the expression singing reference material DD, and only the singing sample data DS about the superior is used. Update the expression vocal reference DD. The selection method of the singing sample data DS used for the update of the expression singing reference material DD is not limited to this. For example, instead of the basic score SR _BASE , the standard score SR _{NOR of} the base score SR _BASE total addition score SR _{ADD may} be used as a speculative reference for the superior. In addition, it is also possible to adopt a configuration in which the upper threshold is set in addition to the lower threshold (reference score SR _TH ) in order to set the basic score SR _{BASE to} be higher than the upper score because the expression is not performed at all, and the upper threshold is set. The singing sample data DS of the singer whose basic score SR _BASE (or other score) of the upper threshold is not used for the update of the expression singing reference DD. In addition, instead of dividing the singer into the superior and the other two parts as described above, for example, the singer sample data DS of the singer whose base score SR _{BASE is} higher is added with a larger weight for the expression sing reference material. DD update.

(9) In the above-described embodiment, as an example of the performance evaluation apparatus for evaluating a musical performance, a performance evaluation apparatus provided for the singing karaoke apparatus and evaluating the singing performance is displayed. However, the performance evaluation apparatus of the present invention is not limited to the singing performance. The evaluation can also be applied to the evaluation of the performance of the music using various instruments. That is, the language "singing" used in the above embodiment can be replaced by the language of "normal performance". Further, in the performance evaluation device for evaluating the musical performance, for example, evaluation of the expression performance corresponding to each musical instrument such as choking of the guitar is performed. Further, in the case where the music piece is not a song but a piece of music for a musical instrument, the karaoke apparatus for musical instrument performance is configured such that the track material MD replaces the lyrics track TR _LY and includes, for example, information indicating the musical score and each section (for example, 2) representing the musical score. The difference between the display time of the bar or the 4 bar module, etc.) The music track of the data described in time series, the sound storage 21 (corresponding to the music data acquisition mechanism) and the display unit 14 are arranged according to the music track The image signal corresponding to the musical score of the accompaniment portion is output to the display along with the progress of the music. Further, in the karaoke apparatus for singing and the karaoke apparatus for musical instrument performance, the output processing of the image signals by the sound storage 21 and the display unit 14 may not be performed when the display of the singing or the music is not required.

(10) As understood from the above description, the performance evaluation apparatus of the preferred embodiment of the present invention is generally shown as the following apparatus, and the presence or absence of other elements or the specific aspect of other elements is arbitrary. As illustrated in FIG. 13, the expression playing reference data acquiring unit 101 acquires the expression starting time of the note or the note group included in the music, and indicates that the expression performance and the expression should be performed in the music performance. The expression playing reference data of the time sequence performed in the music piece; the pitch volume data generating unit 102 generates a tone volume data indicating the pitch and volume of the performance sound from the performance sound of the music piece played by the player; and the performance evaluation mechanism 103 And the characteristic of at least one of the pitch and the volume indicated by the pitch volume data generated by the pitch volume data generating unit 102 indicates that the expression should be played by the expression in a specific time range indicated by the expression performance reference material of the music piece. Improve the above-mentioned performance of the above performers when referring to the expression performance characteristics of the reference material Evaluation of the performance.

(11) In the above embodiment, the example of the performance evaluation apparatus of the present invention is provided in a karaoke apparatus as a so-called dedicated machine. However, the performance evaluation apparatus of the present invention is not limited to a dedicated machine. For example, a configuration of the performance evaluation apparatus of the present invention can be realized by performing processing in accordance with a program by various devices such as a personal computer or a portable information terminal (for example, a portable telephone or a smart phone) or a game machine device. Also, the program can also be stored and distributed In a recording medium such as a CD-ROM, or by using an electric communication cable such as the Internet.

The present application is based on Japanese Patent Application No. 2012-094853, filed on Jan.

[Industrial availability]

According to the present invention, when the expression is performed by the performer, an evaluation that is less deviating from the human feeling can be made.

1‧‧‧Singing Evaluation System

10-m‧‧‧ Karaoke device

11‧‧‧ source

12‧‧‧ Speaker

13‧‧‧ microphone

14‧‧‧Display Department

15‧‧‧Communication interface

16‧‧‧Sound Adapter

17‧‧‧CPU

18‧‧‧RAM

19‧‧‧ROM

20‧‧‧ Hard disk

21‧‧‧Voice storage

30‧‧‧Server device

35‧‧‧Communication interface

37‧‧‧CPU

38‧‧‧RAM

39‧‧‧ROM

40‧‧‧ Hard disk

90‧‧‧Network

APG‧‧‧Singing Analysis Program

DBRK‧‧‧ Reference Library

DBRS‧‧‧ Reference Library

DBS‧‧‧ singing sample database

HD‧‧‧ heading

MD-n‧‧‧ Track Information

S _A ‧‧‧Sound signal

S _I ‧‧‧ image signal

S _o ‧‧‧ control signal

S _L ‧‧‧Output signal

S _M ‧‧‧ radio signal

S _P ‧‧‧Output signal

TR _AC ‧ ‧ accompaniment track

TR _LY ‧ ‧ lyrics track

TR _NR ‧‧‧Fan Ben singing reference track

VPG‧‧‧ singing evaluation program

Claims (13)

A performance evaluation device comprising: an expression performance reference material acquisition mechanism that acquires a sequence of notes to be played in the performance of the music and a sequence in which the expression should be played in the music, and the note or the note group included in the music An expression playing reference material represented by a pronunciation start time; a pitch volume data generating mechanism that generates a tone volume data indicating a pitch and volume of the performance sound from a performance sound of the music played by the player; and a performance evaluation mechanism, The characteristic of at least one of the pitch and the volume indicated by the pitch volume data generated by the pitch volume data generating means is expressed by the expression in a specific time range indicated by the expression performance reference material in the music piece. When the characteristics of the expression performance performed by the reference material are played, the evaluation of the performance of the above-described music performed by the above-mentioned performer is improved. The performance evaluation apparatus of claim 1, further comprising: a tone volume data acquisition mechanism that acquires a tone volume indicating a pitch and volume of the performance sound for each of a plurality of performance sounds of the music played by any player An expression playing performance generating means for indicating at least one of a pitch and a volume indicated by the volume pitch data acquired by the pitch volume data acquiring means: one or more predetermined ones at an arbitrary timing of the music piece In the case of one of the characteristics of the expression of the expression, an expression performance occurrence data indicating the expression of the expression and the timing of the note start time based on the note or the note group included in the music piece is generated; and the expression performance reference a data generating mechanism based on any number of expressions played by the above-mentioned expression playing data generating means, Each of the musical notes or the note group included in the music is specified to be played at any frequency at any timing based on the pronunciation start time of the note or the note group, and is generated according to the specific information. Expression playing reference material. The performance evaluation device of claim 2, further comprising: an expression performance reference data storage mechanism for memorizing the expression performance reference material, based on the expression expression reference material generated by the expression expression reference data generation mechanism, and memorizing the expression of the expression performance reference data The above-mentioned facial expression reference material of the institution is overwritten. The performance evaluation apparatus according to any one of claims 1 to 3, further comprising: a template performance reference acquisition means for acquiring a model performance reference material indicating a pitch of the musical piece of the music piece, wherein the performance evaluation mechanism is based on the tone volume The result of the comparison between the pitch indicated by the pitch volume data generated by the data generating means and the pitch indicated by the model playing reference material is performed on the performance of the music performed by the player. The performance evaluation apparatus of claim 2, further comprising: a template performance reference acquisition means for acquiring a template performance reference material indicating a pitch of the musical composition of the music piece, wherein the performance evaluation mechanism is based on a tone generated by the pitch volume data generating means The pitch of the volume data and the pitch of the tone shown by the model playing reference material are used to evaluate the performance of the music performed by the player, and the pitch volume data obtained by the pitch volume data acquiring mechanism is The performance evaluation data indicates the evaluation result by the performance evaluation mechanism using the template performance reference material, or the other mechanism having the same mechanism as the performance evaluation mechanism is used in the same manner as the template performance reference material. As a result of the evaluation of the data, the expression playing reference data generating means uses the pitch volume data of the performance evaluation data that satisfies the specific condition among the pitch volume data acquired by the pitch volume data acquiring means, based on the appearance of the expression generating means by the expression The generated expression plays the appearance data, and the above expression expression reference material is generated. The performance evaluation apparatus of claim 1 or 2, further comprising: a template performance reference material acquisition unit that acquires a model performance reference material indicating a volume or a pitch that is a template of the music piece; the performance evaluation mechanism is based on The volume or pitch indicated by the pitch volume data generated by the pitch volume data generating means is compared with the volume or pitch indicated by the template playing reference material, and is determined to be related to the performance of the music piece performed by the player. a first score (SR _NOR ), and based on a comparison between the volume or pitch indicated by the pitch volume data generated by the pitch volume data generating means and the volume or pitch indicated by the expression playing reference material The second score (SR _NOR ) related to the performance of the music piece by the player is performed, and the performance evaluation of the performance of the music piece is performed based on the first score and the second score. A karaoke apparatus comprising: the performance evaluation apparatus according to any one of claims 1 to 5; an accompaniment data acquisition mechanism that acquires accompaniment data indicating the accompaniment of the music; and a sound signal output mechanism accompanying the accompaniment data Instructing to output a sound signal indicating the music of the accompaniment; and the pitch volume data generating mechanism generates tone volume data, the tone volume data indicating a sound signal corresponding to the output from the sound signal outputting mechanism The pitch and volume of the performance sound of the above-mentioned music performed by the above-mentioned player from the accompaniment of the speaker playback. The karaoke apparatus of claim 7, wherein the music piece is a song, and the karaoke apparatus further comprises: a lyrics data acquiring unit that acquires lyrics data indicating the lyrics of the song; and an image signal output mechanism whose output representation is The lyrics shown in the above lyrics data should be image signals of lyrics sung together with the accompaniment indicated by the sound signal currently output by the sound signal output mechanism. The karaoke apparatus of claim 7, wherein the music piece is a piece of music played by an instrument, and the karaoke apparatus further comprises: a music score data acquiring mechanism that acquires musical score data indicating a musical score of the music piece; and an image signal output mechanism, The output indicates the musical score indicated by the musical score data, and indicates an image signal of the musical score to be played in conjunction with the accompaniment indicated by the sound signal currently output by the sound signal outputting means. A server device comprising: an expression playing appearance data acquiring means for acquiring, for each of a plurality of performance sounds of a piece of music played by an arbitrary player, starting with a pronunciation of a note or a note group included in the music piece The expression of the expression playing performance appears at a timing of the reference; the expression playing reference data generating mechanism is based on any number of expression performance appearance data acquired by the expression playing appearance data acquiring mechanism, and is included in the music composition. Each of the note or the note group specifies whether any of the expressions at any timing based on the pronunciation start time of the note or the note group appears at any frequency, and the music is generated according to the specific information. Should play in the performance The facial expression playing and the facial expression should be performed in the above-mentioned musical composition, the expression playing reference material indicated by the pronunciation start time of the note or the note group contained in the music piece; and the transmitting mechanism, which will be played by the above expression The expression performance reference material generated by the reference generation mechanism is sent to the performance evaluation device. A performance evaluation system comprising: an expression performance reference material acquisition mechanism that acquires a sequence of notes and notes to be played in the performance of the music and a sequence in which the expression should be performed in the music a first expression performance reference material indicated by the pronunciation start time; a pitch volume data generation mechanism that generates a tone volume data indicating the pitch and volume of the performance sound from the performance sound of the music piece played by the player; the performance evaluation mechanism And the characteristic of at least one of the pitch and the volume indicated by the pitch volume data generated by the pitch volume data generating means is expressed in a specific time range indicated by the first expression performance reference data in the music piece In the case of the characteristics of the expression performance performed by the first expression performance reference material, the evaluation of the performance of the music performed by the player is improved; the expression performance appearance data acquisition mechanism is played by any player for any number of players. Each of the performances of the music is obtained by the above-mentioned The beginning of the pronunciation of the note or the note group contained in the above-mentioned music played by the player is an expression of the expression playing performance at a timing of the reference; and the expression playing reference material generating mechanism is based on the expression played by the above expression Any number of expression performance appearance data acquired by the data acquisition mechanism, and for each of the notes or note groups included in the music played by any of the above-mentioned players, the timing of the pronunciation start of the note or the note group is specified. Whether any of the expressions in a sequence appears at any frequency, according to the specific information, The expression performance to be performed in the performance of any of the above-mentioned players, and the timing at which the expression should be performed in the music played by any of the above-mentioned performers, the timing of the pronunciation of the notes or notes included in the music played by any of the above players. A second expression playing reference material represented as a reference. A performance evaluation method for obtaining a time period in which the expression performance to be performed in the performance of the music piece and the expression performance should be performed in the music piece, based on the pronunciation start time of the note or the note group included in the music piece as a reference The expression playing reference material, wherein the playing sound of the music piece played by the player produces a tone volume data indicating the pitch and volume of the playing sound, and is represented by at least one of the pitch and the volume indicated by the pitch volume data. When the characteristics of the expression performance performed by the above-described expression performance reference material in the specific time range indicated by the above-described expression performance reference material in the above-mentioned music piece are evaluated, the evaluation of the performance of the music piece performed by the above-mentioned performer is improved. A program executable by a computer, and causing the computer to perform the following processing: an expression playing reference material acquisition process, which acquires a timing at which the expression to be performed in the performance of the music piece and the expression should be performed in the music piece to the music piece The expression playing reference material represented by the pronunciation start time included in the note or the note group; the pitch volume data generation processing, which produces a tone indicating the pitch and volume of the performance sound from the performance sound of the music played by the player a volume data; and a performance evaluation process, wherein the characteristics of at least one of the pitch and the volume indicated by the pitch volume data generated by the pitch volume data generating means are expressed in the expression playing reference material in the music In the case where the characteristics of the expression performance to be performed by the above-described expression playing reference material in the specific time range are shown, the evaluation of the performance of the above-described music performed by the above-mentioned performer is improved.