JP2016031395A - Reference display device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a reference display device and program for performing intuitive and easy-to-understand display.SOLUTION: A CPU 11 displays a linear image from a subjective viewpoint whose depth direction corresponds to time and plane direction (vertical direction in this example) corresponds to musical scale. In a screen, a linear image corresponding to the reference data is displayed. A character image corresponding to each linear image is displayed. A score corresponding to a grade result is displayed in an upper part of the screen. The CPU 11 scrolls the linear image and background so that the character image moves in the depth direction along the linear image (namely, along the musical interval of a singer).SELECTED DRAWING: Figure 4

Description

  The present invention relates to a display device, and more particularly to a device for displaying a model (reference).

  2. Description of the Related Art Conventionally, karaoke devices display lyrics and model pitches on a display unit (see, for example, Patent Document 1). The pitch is displayed as a so-called piano roll. A piano roll is a linear image on the screen where the vertical axis represents the scale (the piano keyboard is vertical) and the horizontal axis corresponds to the time. Is displayed. Thereby, the singer can grasp | ascertain visually the timing and pitch which sing.

JP 2004-205817 A

  However, conventional piano rolls are not always intuitive and easy to understand. For example, when performing a duet singing by a plurality of people, even if an attempt is made to display each singer's piano roll on the same screen, the linear images overlap when the pitches overlap, making it difficult to see.

  Accordingly, an object of the present invention is to provide a reference display device and a program that perform a more intuitive and easy-to-understand display.

  The reference display device of the present invention includes a display unit, and an image processing unit that displays a linear image indicating a sound generation timing, a pitch, and a sound generation length on the display unit based on the reference data.

  The image processing unit displays the linear image from a subjective viewpoint in which the depth direction corresponds to the time axis and the planar direction corresponds to the pitch. The subjective viewpoint is a display mode that imitates the user's own visual field. However, a mode in which an image (character image or the like) corresponding to the user himself / herself is displayed and the character image or the like is displayed from behind is also equivalent to a subjective viewpoint.

  As a result, a linear image serving as a model is displayed in the subjective space of the user (singer, speaker, player, etc.). The user can more intuitively understand the sound generation timing and pitch of each sound, and can sing while having fun.

  The image processing unit displays an image corresponding to the user (a photograph taken by the user, a character image, etc.) at a position corresponding to the current pronunciation timing, and the image corresponding to the user is along the linear image. The linear image may be scrolled so as to move. In this case, the user can feel as if the character or the like is moved in accordance with his / her pronunciation, and can enjoy singing, language learning, or playing.

  Further, for example, when performing a duet singing, the reference display device displays reference data for each singer as a plurality of linear images side by side from the subjective viewpoint. Since the present invention displays the linear images side by side from the subjective viewpoint, the linear images do not overlap even on the same screen, and high visibility can be obtained.

  It is also possible to display a character corresponding to itself and a character corresponding to another singer in parallel. In this case, since the other singers are displayed in the subjective space of the user, the user can more feel the atmosphere of singing with the other singers.

  The reference display device includes input means for inputting a user's pronunciation result, and scoring means for scoring by comparing the user's pronunciation result with the reference data, and the image processing unit includes the scoring It is preferable to display the scoring results of the means together with the linear image.

  In this case, since the display image changes according to the result of the user's own singing or the like, the user can perform singing or the like while having more fun.

  The reference display device of the present invention can perform a more intuitive and easy-to-understand display.

It is the block diagram which showed the structure of the karaoke system. It is the block diagram which showed the structure of the karaoke apparatus. It is the block diagram which showed the structure of the center. It is a figure which shows the structure of the various data containing reference data. It is a figure which shows the example of a display of a reference. It is a figure which shows the example of a display of a reference. It is a flowchart which shows operation | movement of a karaoke apparatus. It is a display mode of a reference concerning another example. It is a display mode of a reference concerning another example. It is the block diagram which showed the minimum structure of the reference display apparatus.

  FIG. 1 is a diagram showing a configuration of a karaoke system provided with a reference display device of the present invention. The karaoke system includes a center (server) 1 connected via a network 2 such as the Internet and a plurality of karaoke stores 3.

  Each karaoke store 3 is provided with a relay device 5 such as a router connected to the network 2 and a plurality of karaoke devices 7 connected to the network 2 via the relay device 5. The repeater 5 is installed in a management room of a karaoke store. A plurality of karaoke apparatuses 7 are installed in each private room (karaoke box). Each karaoke device 7 is provided with a remote controller 9.

  The karaoke device 7 can communicate with other karaoke devices 7 via the relay 5 and the network 2. A karaoke system can communicate between karaoke apparatuses 7 installed in different places, and can perform a duet among a plurality of singers.

  FIG. 2 is a block diagram showing the configuration of the karaoke apparatus. FIG. 3 is a block diagram showing the configuration of the center 1.

  The karaoke device 7 corresponds to a reference display device of the present invention. The karaoke device 7 includes a CPU 11, a RAM 12, an HDD 13, a network interface (I / F) 14, an LCD (touch panel) 15, a microphone 16, an A / D converter 17, a sound source 18, a mixer (effector) 19, and a sound system (SS) 20. , A speaker 21, an MPEG decoder 22, a display processing unit 23, a monitor 24, an operation unit 25, and a transmission / reception unit 26.

  The CPU 11 that controls the operation of the entire apparatus includes a RAM 12, an HDD 13, a network interface (I / F) 14, an LCD (touch panel) 15, an A / D converter 17, a sound source 18, a mixer (effector) 19, and a decoder 22 such as an MPEG. The display processing unit 23, the operation unit 25, and the transmission / reception unit 26 are connected.

  The HDD 13 stores an operation program for the CPU 11. In the RAM 12 as a work memory, an area to be read for executing the operation program of the CPU 11, an area to read music data to play karaoke music, an area to read reference data such as a guide melody, a reservation list, a scoring result, etc. An area for temporarily storing the data is set.

  The HDD 13 stores music data for playing karaoke music. Further, the HDD 13 also stores video data for displaying a background video on the monitor 24. The video data stores both moving images and still images. The video data includes a character image to be described later. The music data and the video data are regularly distributed from the center 1 and updated.

  As shown in FIG. 3, the center 1 includes a CPU 31, a RAM 32, an HDD 33, and a network interface (I / F) 34. The HDD 33 stores an operation program for the CPU 31. In the RAM 32 which is a work memory, an area to be read for executing the operation program of the CPU 31 is set. The CPU 31 is a control unit that comprehensively controls the center 1. CPU31 performs the connection process which connects each karaoke apparatus 7, when performing a duet via a network. Moreover, the distribution process of the song data memorize | stored in HDD33 is performed.

  The HDD 33 stores music data and video data for distribution. Further, the HDD 33 stores information on each singer such as a name. For example, when each singer operates a karaoke apparatus and inputs the name of a specific singer, the information on the singer is transmitted to the center 1. CPU31 searches the name which concerns on the said singer from the singer information memorize | stored in HDD33. CPU31 connects the karaoke apparatus which input the name of the singer, and the karaoke apparatus which the searched singer uses. Thereby, duet singing via a network is realized.

  The HDD 33 further stores singing data and scoring data. The singing data is, for example, recorded singing sound when each singer has sung in the past. The scoring data indicates the scoring results at the time of singing. These singing data and scoring data are distributed when each singer makes a request. Thereby, each singer can perform a duet with the singing sound when he sang in the past, for example. The singing data and the scoring data may be sung by a singer or a voice actor, for example. Thereby, each singer can perform a duet with a favorite singer or a voice actor.

  Returning to FIG. 2, the karaoke apparatus 7 will be described. The CPU 11 is a control unit that comprehensively controls the karaoke apparatus, and functionally incorporates a sequencer to perform karaoke performance. The CPU 11 performs audio signal generation processing, video signal generation processing, scoring processing, and linear image display processing. Thereby, the CPU 11 functions as an image processing unit in the present invention.

  The touch panel 15 and the operation unit 25 are provided on the front surface of the karaoke apparatus. The CPU 11 displays an image corresponding to the operation information on the touch panel 15 based on the operation information input from the touch panel 15 to realize a GUI. The remote controller 9 also realizes the same GUI. The CPU 11 performs various operations based on operation information input from the remote controller 9 via the touch panel 15, the operation unit 25, or the transmission / reception unit 26.

  Next, a configuration for performing karaoke performance will be described. As described above, the CPU 11 functionally includes a sequencer. The CPU 11 reads music data corresponding to the music number of the reserved music registered in the reserved list in the RAM 12 from the HDD 13, and performs a karaoke performance with the sequencer.

  For example, as shown in FIG. 4, the music data includes a header in which a music number is written, a musical sound track in which performance MIDI data is written, a guide melody track in which MIDI data for guide melody is written, and lyrics It consists of a lyrics track in which MIDI data is written, a chorus track in which back chorus playback timing and audio data to be played are written, a breath position track indicating the timing of breathing, a technique position track indicating the timing of the singing technique, and the like. ing. The guide melody track, breath position track, and technique position track correspond to the reference data of the present invention. Reference data is model data for a singer to use as a reference for singing, and includes information indicating the sounding timing, pitch, and sounding length of each sound. Note that the format of the music data is not limited to this example. For example, existing music data without a breath position track and a technique position track may be prepared, and the breath position track and the technique position track may be prepared as separate data. In this case, it is not necessary to prepare new music data including the breath position track and the technique position track. However, song identification information such as a song number is described in the data of the breath position track and the technique position track. When reading the music data, the CPU 11 reads the corresponding breath position track and technique position track, and performs a sequence operation. Also, the format of the reference data is not limited to the MIDI format as described above. For example, the reference data indicating the breath position may be text data indicating the timing of the breath position (elapsed time from the beginning of the music). In addition, when the reference data is voice data (for example, recorded singing sound), the pitch is extracted from the voice data to extract the pitch, and the sound is generated from the timing and length at which the pitch is extracted. It is also possible to extract timing and pronunciation length. It is also possible to detect the silent section by detecting the volume (power), and when there is a silent section between each sound, the timing at which the silent section is extracted can be extracted as the timing of the breath position It is. In addition, when the pitch fluctuates regularly within a predetermined period, the timing (technical position) at which the singing technique is performed is extracted by determining that “vibrato” is being performed for the institution. It is also possible.

  The musical sound track stores information indicating the type, timing, pitch (key), strength, length, localization (pan), sound effect (effect), etc. of the musical instrument that generates the musical sound. In the guide melody track, information such as the sounding start timing of each sound corresponding to the model song and the length of the sounding are recorded.

  The sequencer controls the sound source 18 based on the data of the musical sound track and generates the musical sound of the karaoke song.

  The sequencer also reproduces the back chorus audio data (compressed audio data such as MP3 attached to the music data) at the timing designated by the chorus track. Further, the sequencer synthesizes the character pattern of the lyrics in synchronism with the progress of the song based on the lyrics track, converts the character pattern into a video signal, and inputs it to the display processing unit 23.

  The sound source 18 forms a musical sound signal (digital audio signal) according to data (note event data) input from the CPU 11 by processing of the sequencer. The formed tone signal is input to the mixer 19.

  The mixer 19 generates an acoustic effect such as an echo on the musical sound signal generated by the sound source 18, the chorus sound, and the singing voice signal of the singer input from the microphone (singing voice input means) 16 via the A / D converter 17. And mixing these signals.

  Moreover, when communicating between karaoke apparatuses 7 installed in different places and performing a duet, a singing voice signal is transmitted from another karaoke apparatus. The singing voice signal received from the other karaoke apparatus is also input to the mixer 19 and mixed with the singing voice signal input from the microphone 16 of the own apparatus.

  Each mixed digital audio signal is input to the sound system 20. The sound system 20 includes a D / A converter and a power amplifier, converts an input digital signal into an analog signal, amplifies it, and emits the sound from a speaker (musical sound generating means) 21. The effect that the mixer 19 gives to each audio signal and the balance of mixing are controlled by the CPU 11.

  The CPU 11 reads the video data stored in the HDD 13 and reproduces the background video and the like in synchronism with the generation of musical sounds by the sequencer and the generation of the lyrics telop. The video data of the moving image is encoded in the MPEG format.

  The CPU 11 can also download video data such as a photograph representing a singer or video data from the center 1 and input it to the display processing unit 23. The photograph representing the singer can be taken on the spot with a camera (not shown) provided on the karaoke device or the remote control 9, or with a camera provided on a mobile terminal owned by the user. is there.

  The CPU 11 inputs the read video data of the background video to the decoder 22. The decoder 22 converts the input data such as MPEG into a video signal and inputs it to the display processing unit 23. In addition to the video signal of the background video, the display processing unit 23 also receives the video signal of the linear image and the video signal of the character image based on the guide melody track, together with the character pattern of the lyrics telop.

  In the conventional piano roll, the vertical axis is the scale (the piano keyboard is vertical), and the horizontal axis is the time corresponding to the sound start timing and the length of the sound. The image was displayed. However, in this embodiment, as shown in FIG. 5A, a linear image is displayed from a subjective viewpoint in which the depth direction corresponds to the time axis and the planar direction (in this example, the vertical direction) corresponds to the scale. It is. The subjective viewpoint is a display mode that imitates the user's own visual field. As shown in FIG. 5A, an aspect in which an image (character image or the like) corresponding to the user himself / herself is displayed and the character image or the like is displayed from behind is also equivalent to a subjective viewpoint.

  A large spherical image 502 displayed so as to overlap the linear image in the figure is the sound generation timing of each sound in the guide melody track. As a result, the user can grasp that the sound is generated at the timing when the spherical image 502 is displayed. The small spherical image 503 corresponds to the timing of scoring, and a score is given if the pitches of the singing sound and the guide melody match. When a score is given, a star-shaped image 505 is displayed so as to flow, and the score is added to the score column 504 displayed at the upper right in the figure. In addition, when the pitches are continuously matched, for example, “25 combo” is displayed as shown in the upper part of the figure, thereby grasping the number of pitches that can be matched continuously. be able to.

  Note that the linear image referred to in the present invention is not limited to the long and narrow line as shown in FIG. 5B, and as shown in the example of FIG. An image having a width of 1 is extended in one direction (in this example, the depth direction).

  As shown in FIG. 5B, in this embodiment, the linear images corresponding to the reference data (in FIG. 5A, the linear image 501A, the linear image 501B, and the linear image 501C) have a depth. It is displayed to extend in the direction. In addition, the corresponding character image 101A, character image 101B, and character image 101C are displayed in each of the linear image 501A, the linear image 501B, and the linear image 501C, respectively. The score corresponding to the scoring result is displayed at the top of the screen.

  First, the CPU 11 generates a linear image based on information on the sound generation start timing and the sound generation length of each sound included in the guide melody track. Then, the CPU 11 smoothly connects the linear images of the sounds. The inclination of the connection part of each sound is displayed as an image having the same inclination uniformly, for example, by connecting with a time length corresponding to a sixteenth note. However, there are actually different ways of singing for each song, and the manner of changing the pitch is not uniform. Therefore, it is preferable to display with a different inclination for each connection portion of each sound. In this case, the reference data may include information that specifies an inclination according to a change in the pitch of the connected portion of each sound. Note that the linear image may be read and displayed as image data (corresponding to the linear image) prepared for each song in advance.

  Thereafter, the CPU 11 makes the linear image discrete at the breathing timing indicated by the breath position track. For example, from a conventional objective viewpoint, as shown in FIG. 5B, there is a breathing timing between the pronunciation of “Akai” and the beginning of “Hanaga”. The line image of “Akai” and the line image of “Hanaga” are separated.

  Then, the CPU 11 displays the linear image from a subjective viewpoint as shown in FIG. Thereby, the singer can grasp the pronunciation timing and pitch of each sound more intuitively. Note that it is not essential in the present invention to connect linear images smoothly. However, by connecting the linear images smoothly, the singer can visually grasp how to connect each sound and the timing of breathing. In addition, since the CPU 11 makes the line image discrete at the breathing timing indicated by the breath position track (because the reference data includes information indicating the breathing timing), it is clearly distinguished from the silent section and the breathing section. This allows the user to know the exact breathing position.

  Further, the CPU 11 displays a character at the current singing position. In this example, the character image 101A is displayed at the lower center of the screen, and the linear image and the character image 101A move in the depth direction along the linear image (that is, along the pitch of the singer). Scroll the background. If the singer sings the pitch by mistake, the character image 101A will be displayed at a position away from the linear image, and the singer intuitively grasps how far the pitch is off. Can do.

  Moreover, CPU11 displays the score according to the scoring result. Scoring is performed by comparing the singer's singing voice with the guide melody track. Scoring is performed by comparing the pitch of the singing voice and the guide melody for each note of the guide melody track. That is, a high score is given when the pitch of the singing voice matches the pitch of the guide melody track for a predetermined time or longer (within the allowable range). The timing of pitch change is also taken into account. Further, points are added based on the presence or absence of singing techniques such as timing of pitch change, vibrato, inflection, and squealing (moving gently from a low pitch).

  In the scoring process, whether or not the singer performs breath breathing at the breath breathing timing included in the breath position track may also be added. Whether or not breathing has been performed depends on whether no sound is input from the microphone 16 within a predetermined time including the breathing timing (the input level is less than a predetermined threshold) or when a breathing sound is collected. When the voice is input from the microphone 16 (the input level is equal to or higher than the predetermined threshold value), it is determined that breathing is not performed. Whether or not the breathing sound has been collected is determined by comparing with the waveform of the breathing sound, for example, by pattern matching or the like. Note that the CPU 11 may perform display such that the character image falls from the ground (a portion where the linear image is interrupted) when the breathing is not detected.

  In the scoring process of this embodiment, it is preferable to give a higher score when the same technique is detected at the timing of each technique included in the technique position track.

  The scoring process may be performed in each karaoke apparatus, but may be performed in the center 1 (or another server). Moreover, when performing duet with another karaoke apparatus via a network, you may perform a scoring process with one karaoke apparatus which processes typically.

  As described above, the singer can enjoy the karaoke like a game because the character image 101A moves according to his / her singing and the scoring result of his / her singing can be confirmed by scoring. Can do.

  In the example of FIG. 5A, when performing a duet singing, a character image 101B corresponding to the other singer and a character image 101C corresponding to the other singers are displayed side by side. The music data for duet music includes a plurality of reference data (guide melody tracks) for a plurality of singers. Therefore, the CPU 11 displays the line images corresponding to each guide melody track side by side, and displays the character image corresponding to each line image. Thereby, the user can feel more the atmosphere of singing with other singers. Moreover, if a score is added according to the scoring result of each singer and it is set as an aspect, it can feel that each singer is earning a score while cooperating, and a sense of unity can be obtained more.

  Note that a sensor that can detect the attitude of the microphone 16 such as a gyro sensor or an acceleration sensor may be provided, and the subjective viewpoint image may be changed according to the attitude of the microphone 16. For example, in the example of FIG. 5A, the character image 101A is displayed near the center of the left and right, but when the singer turns the microphone 16 to the right, the character image and the linear image on the screen are displayed. The display moves so that the character 101C approaches the left and right centers. As a result, when the singer turns to the right, the subjective viewpoint screen is also displayed to the right, resulting in a more realistic and intuitive display mode.

  When the singer selects music data that is not for a duet song (when the same melody is sung by different singers), the CPU 11 displays, for example, the same linear images side by side, and corresponds to each linear image. To display the character image.

  In the example of FIG. 5A, the vertical direction is displayed from the subjective viewpoint corresponding to the musical scale, but for example, as shown in FIG. 6, the horizontal direction is displayed from the subjective viewpoint corresponding to the musical scale. Good. In this case, the scale is lower in the left direction and higher in the right direction.

  In the present embodiment, an example in which a linear image is displayed from a subjective viewpoint has been described. However, for example, as shown in FIG. 8, it may be displayed from an objective viewpoint. In the example of FIG. 8, the left-right direction corresponds to the time axis, and the up-down direction corresponds to the musical scale. In this example, the linear image corresponds to the ground image, and the ground image is interrupted at the breath connection timing. Then, the linear image and the background are scrolled so that the character image 101A moves along the linear image (ground image). At the breath connection timing, since the ground is interrupted, the character image 101A falls from the ground when the breath connection (state in which no sound is input from the microphone 16) is not detected. Also in this example, the singing result is displayed on the screen. Therefore, also in this example, the singer can enjoy karaoke like a game.

  Next, the operation of the karaoke system will be described with reference to a flowchart. FIG. 7 is a flowchart showing the operation of the karaoke system.

  First, the CPU 11 receives a music request from a singer (s11). Further, the CPU 11 determines whether or not it is a request for a duet song via the network (s12). If the request is duet duet via the network, a connection request is made to the center 1 (s13). At this time, the CPU 11 displays an image prompting whether or not to perform a duet with a singer of another karaoke apparatus connected via the network on the monitor 24, and accepts a duet song via the network. For example, when a singer inputs the name of a specific user using the touch panel 15, the operation unit 25, or the remote control 9, information related to the name is transmitted to the center 1.

  The center 1 receives the connection request (s51), reads the singer information stored in the HDD 33, and performs connection processing with the karaoke apparatus of the corresponding singer (s52). Thereby, each karaoke apparatus is connected with another karaoke apparatus (s14), and a duet song is attained by transmitting / receiving singing data of each singer.

  Next, the CPU 11 of the karaoke apparatus 7 reads the requested music data (s15) and performs a drawing process (s16). That is, the CPU 11 reads the video data stored in the HDD 13, generates a background video, and synthesizes a character image with the background video. Further, the CPU 11 generates a linear image based on information on the sound generation start timing and the sound generation length of each sound included in the guide melody track. The CPU 11 reads the breath timing information from the breath position track, smoothly connects the linear images of each sound, and displays the line images in a discrete manner at the breath timing indicated by the breath position track. The linear image is synthesized with a background video in correspondence with each character image. In the case of a duet song, a plurality of linear images and character images are displayed side by side.

  The CPU 11 preferably displays the linear image related to the phoneme of the prompt sound and the linear image after the phoneme of the prompt sound in a discrete manner. The prompt sound is represented by “t” or “t” in Japanese kana notation, and is silent between the following sound. The CPU 11 extracts the prompt sound from the lyrics track, and disperses the linear image at the timing of emitting the extracted prompt sound. For example, in the case of the lyrics “if you use”, there is a prompt sound after “ka”, so the linear image of “ka” and the subsequent linear image of “ta” are displayed separately. In addition, an image indicating that there is a phoneme related to the prompting sound (in this example, a square image with “tsu”) may be further displayed.

  Moreover, it is preferable that CPU11 reads a technique position track | truck and displays a singing technique on a linear image. For example, in the example of FIG. 5A, the beginning is a portion where a technique of scrambling (singing while lifting from a pitch lower than the reference pitch) is performed. Therefore, an image 506 is displayed that is reminiscent of a rising pitch like “no”. Further, as shown in FIG. 6, in addition to the wavy line-shaped image, a wavy line image 507 like “m” is separately displayed for the vibrato section. Thereby, the singer can grasp | ascertain easily what kind of singing technique is performed at which timing.

  The vibrato section may be displayed by changing the linear image into a wavy line. Further, when the “position” information is included in the technique position track, the head position of the sound corresponding to the “time” information (in this example, the “ka” sound) is delayed. This makes it easier for the singer to intuitively understand the “tame” that intentionally delays the singing of the specific sound (“ka” sound in this example). When the technique position track includes “Kobushi” information, the linear image is temporarily raised at a position corresponding to the timing of the “Kobushi”. Thereby, it is also possible to obtain a linear image corresponding to “Kobushi”, which is a singing technique for changing the voice color of a specific sound so as to beat in the middle of pronunciation. When the technique position track includes “fall” information, the line image is changed from the position corresponding to the “fall” timing to a lower pitch, so that the “fall” singing is performed. A linear image corresponding to the technique may be used.

  Further, the CPU 11 may display an image (for example, an image such as “V”) for prompting breathing together with a linear image based on information indicating breathing timing indicated by the breath position track.

  Further, the CPU 11 may read information indicating the volume of each sound included in the guide melody track, and perform a process of changing to a linear image corresponding to the volume of each sound. For example, the thickness of the line is changed according to the volume, or the color of the line is changed according to the volume.

  When different lyrics are continuously generated with the same pitch, if the linear images are connected, the user may have difficulty grasping at which timing the next lyrics are to be generated. Therefore, as shown in FIG. 5A, the CPU 11 displays, for example, a spherical image superimposed on a linear image at the sound generation timing of each sound in the guide melody track. Thereby, the user can grasp | ascertain that sounding is performed at the timing when the said spherical image is displayed. In the example of FIG. 5A, a small spherical image is displayed at the beat timing and a large spherical image is displayed at the sounding timing.

  In the drawing process, the CPU 11 reads out the lyrics track and displays the lyrics image in association with each linear image.

  Next, the CPU 11 performs a scoring process as the karaoke performance progresses (s17). The scoring process is performed by comparison with the reference data as described above.

  The CPU 11 transmits the scoring result in the scoring process and the singing data related to the singing sound input from the microphone 16 to the center 1 (s18). The center 1 receives the scoring results and singing data from each karaoke device 7 and transmits the singing data to another corresponding karaoke device 7 (s53). Thereby, the singing voice in each karaoke device 7 is transmitted to the other karaoke devices 7. In each karaoke apparatus 7, its own singing voice and the singing voice of the duet partner are mixed and emitted from the speaker 21. Further, the center 1 aggregates the scoring results received from each karaoke device 7 and then transmits it to each karaoke device 7. Thereby, in each karaoke apparatus 7, the scoring result which totaled its own scoring result and the duet partner's scoring result can be obtained, for example, the score according to the scoring result after summation is displayed.

  The above processing is performed until the end of the music performance. The CPU 11 of the karaoke device 7 determines whether or not the performance of the music has ended (s19), and scrolls the linear image and moves the character image according to the progress of the music until the performance of the music is completed. Moreover, the score according to the scoring result is displayed. Similarly, the CPU 31 of the center 1 determines whether or not the performance of the music has ended (s54), and transmits and receives song data and scoring results from each karaoke device 7 until the performance of the music is completed.

  In the present embodiment, the karaoke device 7 is used to perform a karaoke performance and display a linear image from a subjective viewpoint. However, for example, an information processing device (microphone, etc.) such as a PC, a smartphone, or a game machine owned by the user. The reference display device of the present invention can be realized also by using a speaker and a display and a display unit. Note that the music data and the reference data need not be stored in the display device, and may be downloaded from the server each time and used.

  Moreover, in this embodiment, although the example which displays the guide melody in karaoke as a linear image was shown, the mode change of the example of a wind instrument performance is displayed as a linear image, for example, and the linear image interrupts at the breathing timing The same effect can be obtained. Further, for example, in language learning, a similar effect can be obtained by displaying a linear image indicating the model's pronunciation timing and pronunciation length and interrupting the linear image at the breath timing and the prompt sound.

  Although FIG. 5 shows a mode in which a character image corresponding to the user himself and a character image corresponding to another user are respectively displayed, displaying a plurality of linear images and a plurality of character images It is not essential in the invention. For example, as shown in FIG. 9 (A), only a linear image may be displayed from a subjective viewpoint in which the character image is not displayed, the depth direction corresponds to the time axis, and the plane direction corresponds to the pitch. Although the lyrics are also displayed in the example of FIG. 9A, the display of the lyrics is not essential in the present invention. Also in this case, as shown in FIG. 9B, the linear image extending in the depth direction is displayed so as to approach the front as time elapses. You can intuitively grasp the pitch and enjoy singing and playing musical instruments.

  As shown in FIG. 10, the reference display device of the present invention includes a monitor 24 that is a display unit, and a CPU 11 that functions as an image processing unit that performs a linear image display process, and the CPU 11 stores the data in the HDD 13. What is necessary is just to make it the aspect which displays the linear image based on the music data currently performed (it is an example of the reference data of this invention) with a subjective viewpoint. Other hardware configurations are not essential elements in the present invention.

  Further, as described above, the reference data does not need to be stored in the HDD 13, and may be downloaded and used from the outside (for example, a server) each time. In addition, the decoder 22, the display processing unit 23, and the RAM 12 may be incorporated in the CPU 11 as a part of the function of the CPU 11.

DESCRIPTION OF SYMBOLS 1 ... Center 2 ... Network 3 ... Karaoke store 5 ... Relay machine 7 ... Karaoke apparatus 9 ... Remote control 11 ... CPU
12 ... RAM
13 ... HDD
DESCRIPTION OF SYMBOLS 15 ... Touch panel 16 ... Microphone 17 ... A / D converter 18 ... Sound source 19 ... Mixer 20 ... Sound system 21 ... Speaker 22 ... Decoder 23 ... Display processing part 24 ... Monitor 25 ... Operation part 26 ... Transmission / reception part 31 ... CPU
32 ... RAM
33 ... HDD
101 ... Character images 101A, 101B, 101C ... Character images 501A, 501B, 501C ... Linear images

Claims (5)

A display unit;
An image processing unit that displays on the display unit a linear image indicating a sounding timing, a pitch, and a sounding length based on the reference data;
A reference display device comprising:
The reference display device, wherein the image processing unit displays the linear image from a subjective viewpoint in which a depth direction corresponds to a time axis and a planar direction corresponds to a pitch.   The reference display device according to claim 1, wherein the image processing unit displays different reference data side by side as the plurality of linear images from the subjective viewpoint.   The image processing unit displays an image corresponding to the user at a position corresponding to the current pronunciation timing, and scrolls the linear image so that the image corresponding to the user moves along the linear image. The reference display device according to claim 1. An input means for inputting a user's pronunciation result;
Scoring means for scoring by comparing the user's pronunciation results with the reference data,
The reference display device according to claim 1, wherein the image processing unit displays a scoring result of the scoring unit together with the linear image. In an information processing device equipped with a display unit,
A program for executing an image processing step of displaying a linear image indicating a sound generation timing, a pitch, and a sound generation length on the display unit based on reference data,
The image processing step displays the linear image from a subjective viewpoint in which a depth direction corresponds to a time axis and a planar direction corresponds to a pitch.

Images