WO2019176029A1 - Sound detection device - Google Patents

Abstract

The present invention provides a sound detection device including: a sound data obtaining unit that obtains first sound data; a detection unit that sequentially detects, from the obtained first sound data, and in a prescribed period, the intensity of a fundamental of the first sound data and the intensities of one or more harmonics of the fundamental; and a data generation unit that sequentially generates display data for displaying a first image indicating the detected intensity of the fundamental of the first sound data and the detected intensities of the one or more harmonics.

Description

Sound detection device

The present invention relates to a sound detection device that generates sound data for visually recognizing an input sound.

Technology for visualizing the performance sound when playing musical instruments has been developed. For example, Patent Literature 1 discloses a technique for displaying twelve semitones constituting one octave by labeling them on a circle.

Special table 2010-504539

The technique of Patent Document 1 can visualize the pitch, but cannot visualize the sound quality. The sound quality is related to, for example, the ratio between the intensity of the fundamental tone included in the sound and the intensity of the harmonic.

One of the objects of the present invention is to provide a sound detection device that can visualize the intensity of fundamental and overtone included in a sound.

A sound detection device according to an embodiment of the present invention includes a sound data acquisition unit that acquires first sound data, the intensity of the fundamental sound of the first sound data in the predetermined period of the acquired first sound data, and the fundamental sound And a display for displaying a first image indicating the intensity of the fundamental tone and the intensity of the one or more overtones of the detected first sound data. A data generation unit that sequentially generates data.

In the first image, the intensity of the fundamental tone and the intensity of the one or more overtones may be displayed in a magnitude based on each intensity.

In the first image, the fundamental tone and the one or more harmonics may be displayed in different colors.

The detection unit may include a pitch determination unit that determines a pitch of the fundamental tone, and an intensity detection unit that detects the intensity of the fundamental tone and the intensity of the one or more harmonics.

In the first image, at least one of the intensity of the fundamental sound and the intensity of the one or more harmonics may be displayed in a color corresponding to the pitch of the fundamental sound.

In the first image, the pitch of the fundamental tone may be displayed.

In the first image, the intensity of the one or more harmonics may be displayed normalized to the intensity of the fundamental tone.

In the first image, the intensity of the one or more harmonics may be displayed at an arbitrary position from a position where the intensity of the fundamental tone is displayed.

In the first image, the intensity of the one or more harmonics is displayed in a radial direction from a position where the intensity of the fundamental tone is displayed, and the intensity of two or more harmonics is displayed in a predetermined radial direction. The other harmonics displayed in the predetermined radial direction may have an octave relationship with respect to the harmonics displayed at the position closest to the position where the intensity of the fundamental tone is displayed.

In the first image, the intensity of the one or more harmonics may be displayed at a position farther from a position where the intensity of the fundamental is displayed as the order of the harmonic is higher.

The first image may include a time axis and an axis indicating the order of harmonics.

The display data is data for further displaying a second image indicating the intensity of the fundamental sound of the second sound data different from the first sound data and the intensity of one or more harmonics of the fundamental sound of the second sound data. There may be.

The data generation unit is configured to store the intensity of the fundamental tone of the second sound data from the storage unit that stores the intensity of the fundamental tone of the second sound data and the intensity of one or more harmonics of the fundamental tone of the second sound data. You may acquire the intensity | strength of the 1 or more overtone of the fundamental tone of 2 sound data.

The second sound data is acquired by the sound data acquisition unit together with the first sound data, and the intensity of the fundamental sound of the sound data of the predetermined period of the second sound data and the intensity of one or more harmonics of the fundamental sound, May be sequentially detected by the detection unit.

The first image and the second image may be displayed separately from each other.

The first image and the second image may be displayed adjacent to each other.

The first image and the second image may be displayed in a superimposed manner.

A program according to an embodiment of the present invention acquires sound data from a computer, and sequentially sets the intensity of a fundamental tone of sound data of a predetermined period and the intensity of one or more harmonics of the fundamental tone among the obtained sound data. A program for executing detection and sequentially generating display data for displaying an image indicating the intensity of the detected fundamental tone and the intensity of the one or more harmonics.

According to the present invention, it is possible to provide a sound detection device that can visualize the intensity of a fundamental tone and the intensity of a harmonic contained in a sound.

It is the schematic which shows an example of the hardware constitutions of the sound display apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of the sound detection apparatus which concerns on one Embodiment of this invention. It is the figure which showed an example of the harmonic component table | surface of a predetermined period. It is the flowchart which showed the flow which produces | generates display data. It is an example of the overtone shown on the pole seat table. It is an example which shows the intensity | strength of the fundamental tone in the sound data of a predetermined period, and the intensity | strength of one or more harmonics of the fundamental tone on a polar coordinate. It is an example which shows the intensity | strength of the fundamental tone in the sound data of a predetermined period, the intensity | strength of one or more harmonics of this fundamental tone, and the pitch of this fundamental tone on a polar coordinate. It is the schematic which shows an example of the hardware constitutions of the sound display apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of the sound detection apparatus which concerns on one Embodiment of this invention. Polar coordinates of the intensity of the fundamental tone based on the first display data and the intensity of one or more harmonics of the fundamental tone, and the intensity of the fundamental tone of the reference sound and the intensity of the one or more harmonics of the fundamental tone based on the second display data It is an example of the example of a display shown simultaneously above. Polar coordinates of the intensity of the fundamental tone based on the first display data and the intensity of one or more harmonics of the fundamental tone, and the intensity of the fundamental tone of the reference sound and the intensity of the one or more harmonics of the fundamental tone based on the second display data It is an example of the example of a display shown simultaneously above. Polar coordinates of the intensity of the fundamental tone based on the first display data and the intensity of one or more harmonics of the fundamental tone, and the intensity of the fundamental tone of the reference sound and the intensity of the one or more harmonics of the fundamental tone based on the second display data It is an example of the example of a display shown simultaneously above. It is an example of the example of a display which displays the intensity | strength of the fundamental tone of a sound, and the intensity | strength of one or more harmonics of the fundamental tone. It is an example of the example of a display which displays the intensity | strength of the fundamental tone of a sound, and the intensity | strength of one or more harmonics of the fundamental tone.

Hereinafter, a sound detection apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings. The following embodiments are examples of embodiments of the present invention, and the present invention should not be construed as being limited to these embodiments. Note that in the drawings referred to in the present embodiment, the same portion or a portion having a similar function is denoted by the same reference symbol or a similar reference symbol (a reference symbol simply including A, B, etc. after a number) and repeated. The description of may be omitted.

(Embodiment 1)
[Configuration of sound display device]
FIG. 1 is a schematic diagram illustrating an example of a hardware configuration of a sound display device 10 including a sound detection device according to the present embodiment. The sound display device 10 includes a control unit 11, a storage unit 13, a display unit 15, and a signal processing unit 17. A sound input unit (for example, a microphone) 19 is connected to the signal processing unit 17. The sound display device 10 may optionally include an operation unit 21 and a communication unit 23. These components are connected to each other via a bus 25.

The control unit 11 includes an arithmetic processing circuit such as a CPU. The control unit 11 causes the CPU to execute the control program 13 a stored in the storage unit 13 to realize various functions in the sound display device 10. The realized functions include a function of generating display data for displaying on the display unit 15 the intensity of the fundamental tone and the intensity of the harmonics included in the input sound.

The storage unit 13 is a storage device such as a nonvolatile memory or a hard disk. The memory | storage part 13 memorize | stores the control program 13a for implement | achieving the sound detection apparatus 100 mentioned later. The control program 13a may be provided in a state stored in a computer-readable recording medium such as a magnetic recording medium, an optical recording medium, a magneto-optical recording medium, or a semiconductor memory. In this case, the sound display device 10 only needs to include a device that reads the recording medium. The control program 13a may be downloaded via a network such as the Internet.

The display unit 15 is a display device such as a liquid crystal display or an organic EL display, and displays a screen representing the intensity of the fundamental tone and the overtone included in the input sound based on the control by the control unit 11.

The signal processing unit 17 includes an A / D converter and the like. The sound is converted into an electrical signal by the sound input unit 19 and input to the signal processing unit 17, and A / D converted by the signal processing unit 17 and output to the control unit 11.

The operation unit 21 is a device such as an operation button, a keyboard, or a mouse provided on the operation panel and a remote controller, and outputs a signal corresponding to the input operation to the control unit 11. Based on the control of the control unit 11, the communication unit 23 is connected to a communication line such as the Internet or a LAN, and transmits / receives information to / from an external device such as a server. Note that the function of the storage unit 13 may be realized by an external device capable of communicating in the communication unit 23.

[Configuration of sound detection device]
The sound detection device included in the sound display device 10 generates display data for displaying the intensity of the fundamental tone and the harmonic overtone included in the input sound on the display unit 15. The configuration of the sound detection device described below is realized by executing the control program 13a included in the control unit 11 of the sound display device 10 and stored in the storage unit 13.

FIG. 2 is a block diagram showing the configuration of the sound detection apparatus according to the first embodiment of the present invention. The sound detection device 100 includes a sound data acquisition unit 101, a detection unit 103, and a data generation unit 109. Hereinafter, each component of the sound detection apparatus 100 will be described in detail.

The sound data acquisition unit 101 sequentially acquires sound data. The sound data acquisition unit 101 acquires sound data from the sound input unit 19 such as a microphone that converts the input sound into sound data and outputs the sound data. Here, the sound input to the sound input unit 19 may be a sound of an instrument played by the performer. The sound data acquired by the sound data acquisition unit 101 is digitized by the signal processing unit 17. The sound data acquisition unit 101 sequentially outputs the acquired sound data to the detection unit 103.

The detection unit 103 acquires sound data from the sound data acquisition unit 101. The detection unit 103 sequentially acquires sound data for a predetermined period from the acquired sound data, and sequentially detects the intensity of the sound data for the predetermined period. The sound data for a predetermined period includes a fundamental tone and harmonics of the fundamental tone. In other words, the detection unit 103 sequentially detects the intensity of the fundamental tone for a predetermined period and the intensity of the harmonic overtone of the fundamental tone. The detection unit 103 analyzes a spectrum obtained by performing FFT (Fast Fourier Transform) on sound data of a predetermined period, and detects the intensity of the fundamental tone and the intensity of one or more harmonics included in the sound data of the predetermined period. Is detected. The detecting unit 103 sequentially outputs the detected fundamental tone intensity and one or more overtone intensities to the data generating unit 109.

The detection unit 103 can also determine the pitch of the fundamental tone by analyzing a spectrum obtained by performing FFT on sound data for a predetermined period. In this case, the detection unit 103 associates the detected fundamental tone intensity and the intensity of one or more harmonics with the pitch data indicating the pitch of the fundamental tone, and outputs it to the data generation unit 109.

When analyzing the spectrum obtained by performing FFT on sound data of a predetermined period, it may be possible to efficiently process the fundamental tone pitch, the fundamental tone intensity, and the overtone intensity separately. When separately detecting the pitch of the fundamental tone, and the intensity of the fundamental tone and the intensity of the harmonic overtone, the detection unit 103 may include a pitch determination unit 105 and an intensity detection unit 107, as shown in FIG. The sound data is transmitted from the sound data acquisition unit 101 to the pitch determination unit 105 and the intensity detection unit 107.

The pitch determination unit 105 detects the pitch of the fundamental tone from the acquired sound data. The pitch determination unit 105 detects a zero cross when the waveform of sound data in a predetermined period changes from negative to positive, and specifies the pitch (frequency) of the fundamental sound by measuring the time interval of the zero cross. At this time, from this sound data, a high-frequency component that becomes a noise component may be cut by a low-pass filter, or a DC component may be cut by a high-pass filter. In addition, the pitch determination unit 105 may determine the pitch of the fundamental tone by analyzing a spectrum obtained by performing FFT on sound data of a predetermined period, or by analyzing an envelope of sound data of a predetermined period. When analyzing the spectrum obtained by FFT, the time window length may be changed according to the pitch of the fundamental tone. For example, when the pitch of the fundamental tone is high, the pitch of the fundamental tone can be accurately determined by increasing the time window length. Alternatively, the pitch determination unit 105 may determine the pitch of a fundamental tone for a predetermined period using the YIN algorithm (Hino Kawahara, “YIN, a fundamental frequency estimator for speech and music”, April 2002, The Journal of the Acoustical Society of America, Volume 111, Issue 4, pp.1917-1930 [searched on February 20, 2018], Internet <HYPERLINK "http://audition.ens.fr/adc/pdf/2002# JASA # YIN.pdf "http://audition.ens.fr/adc/pdf/2002#JASA#YIN.pdf 〉).

The intensity detection unit 107 analyzes the spectrum obtained by performing FFT on the sound data for each predetermined period, and detects the intensity of the fundamental sound and one or more harmonics included in the sound data for each predetermined period. The detection unit 103 associates the pitch data indicating the pitch of the fundamental tone, the intensity of the fundamental tone, and the intensity of one or more overtones of the fundamental tone, and outputs them to the data generation unit 109.

The data generation unit 109 sequentially acquires the intensity of the fundamental tone and the intensity of one or more harmonics of the fundamental tone from the detection unit 103. The data generation unit 109 sequentially generates display data for displaying on the screen of the display unit 15 a first image indicating the intensity of the acquired fundamental tone and the intensity of one or more overtones.

For example, the data generation unit 109 associates the fundamental tone, each harmonic, and their intensities based on the intensity of the fundamental tone and the intensity of one or more harmonics of the fundamental tone detected by the detection unit 103. A harmonic overtone component table may be generated. FIG. 3 is a diagram illustrating an example of a harmonic component table for a predetermined period generated by the data generation unit 109. In FIG. 3, “overtone 1” is the fundamental tone, and indicates the intensity from the second overtone to the thirty-second overtone. The unit of intensity may be dB or linear. The data generation unit 109 sequentially generates a harmonic component table as shown in FIG. 3 as display data based on the intensity of the fundamental tone and the intensity of one or more harmonics of the fundamental tone detected for each predetermined period. The display data generated by the data generation unit 109 is not limited to the harmonic component table shown in FIG. 3, and data in which the fundamental tone and the intensity of the fundamental tone are associated with each other, and one or more harmonics. It is only necessary to include data associated with the intensity corresponding to each overtone.

Display data generated in the data generation unit 109 is sequentially output to the display unit 15 via the bus 25 of the sound display device 10. In the display unit 15, based on the display data output from the data generation unit 109, a first image indicating the intensity of the fundamental tone and the intensity of one or more overtones in a predetermined period is sequentially displayed.

FIG. 4 displays on the display unit 15 an image indicating the intensity of the fundamental tone and the intensity of one or more harmonics included in the sound data for a predetermined period, which is executed by the sound detection apparatus 100 shown in FIG. It is the flowchart which showed the flow which produces | generates the display data for doing.

First, the detection unit 103 acquires sound data for a predetermined period from the sound data acquired via the sound data acquisition unit 101 (S1). Next, the detection unit 103 detects the intensity of the fundamental tone in the sound data for a predetermined period and the intensity of one or more harmonics of the fundamental tone (S2). Although not shown here, the pitch of the fundamental tone may be determined. Next, display data is generated by the data generation unit 109 based on the intensity of the detected fundamental tone and the intensity of one or more harmonics of the fundamental tone (S3). This display data is data for displaying an image indicating the intensity of the fundamental tone and the intensity of one or more harmonics of the fundamental tone on the display unit 15 of the sound display device 10. The above flow of S1 to S3 is executed for the sound data of all the predetermined periods in the acquired sound data.

FIG. 5 is an example of harmonics shown on the pole table. The numbers on the polar coordinates indicate the harmonic order. In FIG. 5, the intensity of the fundamental tone that is the first harmonic is displayed at the center on the polar coordinates, and the intensity of one or more harmonics relative to the fundamental is displayed in the radial direction from the position where the intensity of the fundamental is displayed. . Here, the intensity of the harmonic overtone is displayed further away from the position where the intensity of the fundamental tone is displayed as the order of the harmonic overtone is higher. In FIG. 5, the intensities of the overtones that are in an octave relationship with the fundamental tone (second overtone, fourth overtone, eighth overtone, sixteenth overtone, and thirty-second overtone) are based on the position where the intensity of the fundamental tone is indicated. It is displayed on an axis 501 extending upward in the figure. In FIG. 5, the harmonic overtone (third overtone) having a relationship of 5 degrees with respect to the fundamental tone is displayed on an axis 503 that extends downward in the figure from the position where the intensity of the fundamental tone is displayed. . Overtones that have an octave relationship with the third overtone (sixth overtone, twelfth overtone, and twenty-fourth overtone) are displayed on axis 503. In FIG. 5, the harmonic overtone (fifth overtone) having a relationship of 3 degrees with respect to the fundamental tone is displayed and displayed on an axis 505 extending in the right direction in the drawing with the position of the fundamental tone as a base point. Overtones that are in an octave relationship with the fifth overtone (the tenth overtone and the twentieth overtone) are shown on the axis 505. In FIG. 5, a harmonic (seventh harmonic) having a relationship of 7 degrees with respect to the fundamental tone is displayed on an axis 507 that extends in the left direction in the drawing with the position where the intensity of the fundamental tone is displayed as a base point. Overtones that are in an octave relationship with the seventh overtone (14th overtone and 28th overtone) are shown on axis 507. That is, the nth harmonic is displayed on the inner peripheral side closest to the position (center on the polar coordinates) where the intensity of the fundamental tone is displayed, and the n × m (m is an integer of 2 or more) harmonic is displayed in the radial direction.

In the example shown in FIG. 5, the position of the fundamental tone is set at the center of the pole table, and the arrangement position of the harmonics is set according to the frequency of the harmonic over the fundamental frequency. In other words, the position of each overtone relative to the position of the fundamental tone is fixed and displayed, and the intensity of the fundamental tone and the intensity of the harmonic overtone are displayed in a state where the position of each harmonic overtone relative to the position of the fundamental tone is fixed. The arrangement of overtones shown on the pole table shown in FIG. 5 is an example. The arrangement of overtones shown on the pole chart may vary depending on the instrument being played.

As shown in FIG. 5, the data generation unit 109 arranges the fundamental tone at the center on the polar coordinates, and arranges each overtone around the location of the fundamental tone according to the frequency of the harmonic over the fundamental frequency. Display data may be generated so as to be displayed on the display unit 15.

FIG. 6 is an example of a first image displayed on the display unit 15 based on the display data generated by the data generation unit 109. In the first image 600 in FIG. 6, the intensity of the fundamental tone and the intensity of one or more harmonics of the fundamental tone in the sound data for a predetermined period are shown on polar coordinates. In FIG. 6, as in FIG. 5, the fundamental tone is arranged at the center of the polar coordinates, and each harmonic overtone is arranged around it in accordance with the frequency of the harmonic over the fundamental frequency. The first image 600, the intensity of the fundamental tone and the intensity of the harmonic overtone are represented by the size of a circle displayed on polar coordinates. That is, the greater the circle size, the greater the strength, and the smaller the circle size, the smaller the strength. For example, in the first image 600, a circle 601 indicating the fundamental tone that is the first overtone shown at the center of the polar coordinates is displayed the largest, a circle 603 indicating the fourteenth overtone is displayed second largest, and the fifteenth overtone is displayed. A circle 605 shown is displayed third largest. This indicates that in the sound data of a predetermined period, the intensity of the sound is fundamental tone> 14th overtone> 15th overtone. When the intensity of the harmonic overtone is equal to or lower than a predetermined intensity, the size of the graphic indicating the intensity of the harmonic overtone having the intensity below the predetermined intensity may be constant.

The first image 600 shows the intensity of the fundamental tone and the intensity of one or more overtones in the sound data for a predetermined period. The display unit 15 sequentially displays images indicating the intensity of the fundamental tone and the intensity of one or more overtones in the sound data for a predetermined period. In the image 600, the fundamental tone and one or more harmonics of the fundamental tone may be displayed in different colors. That is, in the first image 600, the circle 601 indicating the fundamental tone and the circles indicating the overtones around it (including the circles 603 and 605) shown at the center of the polar coordinates may be displayed in different colors. In FIG. 6, the polar coordinate grid is indicated by a broken line and a dotted line, but the grid may be omitted from the image 600. In the image 600, the intensity of the fundamental tone and the intensity of the harmonic overtone are indicated by circles, but the present invention is not limited to this.

When the detection unit 103 of the sound detection device 100 determines the pitch of the fundamental tone, the display unit 15 may display the intensity of the fundamental tone and the intensity of the harmonic overtone and simultaneously display the pitch of the fundamental tone. FIG. 7 is an example of a first image displayed on the display unit 15 based on the display data generated by the data generation unit 109. In the first image 700 in FIG. 7, the intensity of the fundamental tone, the intensity of the harmonic overtone of the fundamental tone, and the pitch of the fundamental tone in the sound data for a predetermined period are shown on polar coordinates. As shown in FIG. 7, the pitch of the fundamental tone may be displayed at the same time inside a circle indicating the strength of the fundamental tone. The first image 700 indicates that the pitch of the fundamental tone is “C3”. Note that the display mode of the pitch of the fundamental tone is not limited to this. For example, at least one of the intensity of the fundamental tone and one or more harmonics of the fundamental tone may be displayed in a color corresponding to the pitch of the fundamental tone.

As described above, in the present embodiment, the intensity of the fundamental tone and the intensity of the overtone included in the performance sound of the input musical instrument are sequentially displayed on the display unit 15 of the sound display device 10 so that the performer can perform the performance sound. The features of can be directly recognized.

(Second Embodiment)
In the first embodiment, the intensity of a fundamental tone and one or more intensities of the fundamental tone are displayed on the display unit 15 for the sound input to the sound input unit 19 of the sound display device 10. In the second embodiment, for two different sounds, a sound that can simultaneously display the intensity of the fundamental tone and the harmonic overtone included in one sound and the intensity of the fundamental tone and the harmonic overtone contained in the other sound. The display device will be described.

[Configuration of sound display device]
FIG. 8 is a schematic diagram illustrating an example of a hardware configuration of a sound display device 10A including the sound detection device according to the present embodiment. The sound display device 10A includes a control unit 11, a storage unit 13A, a display unit 15, a signal processing unit 17, an operation unit 21, and a communication unit 23. A sound input unit 19 is connected to the signal processing unit 17. These components are connected to each other via a bus 25. Regarding the sound display device 10A, a configuration different from the sound display device 10 described in the first embodiment will be mainly described.

In the sound display device 10A, the storage unit 13A stores a control program 13a, reference sound display data 13b, reference sound data 13c, and accompaniment sound data 13d for realizing a sound detection device 100A described later. The reference sound display data 13b includes information indicating the intensity of the fundamental tone and the overtones included in the reference sound for each predetermined period. For example, the reference sound display data 13b may include a harmonic component table in which each harmonic and its intensity are associated with each other as shown in FIG. The reference sound data 13c is data indicating the melody of the song. The accompaniment sound data 13d is data indicating the accompaniment of a song. The reference sound data 13c and the accompaniment sound data 13d may be data expressed in a MIDI format such as SMF. The reference sound display data 13b, the reference sound data 13c, and the accompaniment sound data 13d may be downloaded via the communication unit 23 via a network such as the Internet. Note that either one of the reference sound data 13c and the accompaniment sound data 13d may be omitted.

In the sound display device 10A, the intensity of the fundamental tone and the harmonic overtone of the performance sound of the musical instrument played by the performer and the intensity of the reference tone and overtone of the reference tone of the music played by the performer are displayed on the display unit 15 at the same time. Display data for displaying the intensity of the fundamental and overtones of the reference sound is stored in the storage unit 13A as reference sound display data 13b. The sound detection device included in the sound display device 10A executes the control program 13a stored in the storage unit 13 by the control unit 11 to thereby show the first and second harmonics included in the input sound. Display data for simultaneously displaying the image and the second image indicating the intensity of the fundamental tone and overtone of the reference sound on the display unit 15 is generated. The user of the sound display device 10 </ b> A can specify a reference sound or accompaniment sound corresponding to the music to be played by the operation unit 21. The performer plays the musical piece to be played with a musical instrument in accordance with the designated reference sound or accompaniment sound. At this time, the designated reference sound or accompaniment sound is read from the storage unit 13 </ b> A and output to a sound output device such as a speaker or headphones via the communication unit 23. A performer plays a musical instrument in accordance with a reference sound or accompaniment sound output from a speaker or headphones.

[Configuration of sound detection device]
The sound detection device included in the sound display device 10A includes a first image indicating the intensity of the fundamental tone and the overtone intensity of the performance sound of the musical instrument by the performer, and the fundamental tone and the overtone of the reference sound corresponding to the song played by the performer. Display data for simultaneously displaying the second image indicating the intensity of the image is generated. The configuration of the sound detection device described below is realized by executing the control program 13a included in the control unit 11 of the sound display device 10A and stored in the storage unit 13A.

FIG. 9 is a block diagram showing a configuration of a sound detection apparatus according to the second embodiment of the present invention. In FIG. 9, the same reference numerals are assigned to the same or similar components as those of the sound detection device 100 described in the first embodiment, and detailed description thereof is omitted. As shown in FIG. 9, the sound detection device 100A includes a sound data acquisition unit 101, a detection unit 103, and a data generation unit 909.

The performer designates a song to be played in advance via the operation unit 21. The control unit 11 reads the reference sound display data 13b corresponding to the music designated by the performer via the operation unit 21 and the reference sound data 13c and / or accompaniment sound data 13d from the storage unit 13A. Alternatively, the control unit 11 downloads the reference sound display data 13b, the reference sound data 13c, and / or the accompaniment sound data 13d via the communication unit 23 via the network. The control unit 11 outputs the acquired reference sound data 13c and / or accompaniment sound data 13d to a sound output device such as a speaker or headphones via the communication unit 23, and the performer can output the reference sound output from the sound output device. And / or play the instrument to the accompaniment sound.

The sound data acquisition unit 101 sequentially acquires sound data. This sound data is based on the sound obtained when the performer plays the instrument. The detection unit 103 acquires sound data from the sound data acquisition unit 101, and sequentially detects the intensity of the fundamental tone and the intensity of the harmonic overtone of the fundamental tone. The detection unit 103 may determine the pitch of the fundamental tone of the sound data for a predetermined period.

The data generation unit 909 sequentially acquires the intensity of the fundamental tone and the intensity of one or more harmonics of the fundamental tone from the detection unit 103. Further, the data generation unit 909 acquires the reference sound display data 13b acquired via the storage unit 13A or the communication unit 23, and acquires the reference sound display data corresponding to the predetermined period. The data generation unit 909 obtains the first image indicating the intensity of the fundamental sound of the sound data for a predetermined period and the intensity of one or more harmonics obtained from the detection unit 103, and the fundamental sound of the reference sound corresponding to the predetermined period. Display data for simultaneously displaying the second image showing the intensity of the second tone and the intensity of one or more overtones of the fundamental tone, and sequentially outputting them to the display unit 15 via the bus 25.

The display unit 15 sequentially displays the first image and the second image on the screen based on the display data acquired from the data generation unit 909. The display unit 15 displays the first image and the second image simultaneously on the screen.

FIG. 10 is an example of the first image and the second image displayed on the display unit 15 based on the display data generated by the data generation unit 109. FIG. 10 shows an example in which the first image and the second image are respectively displayed on polar coordinates, and the first image and the second image are displayed adjacently and simultaneously.

10, the first image 1001 shows the intensity of the fundamental tone and the intensity of one or more harmonics of the fundamental tone in the sound data for a predetermined period. The second image 1003 shows the intensity of the fundamental tone of the reference sound and the intensity of one or more harmonics of the fundamental tone for the predetermined period. The first image 1001 and the second image 1003 are displayed on polar coordinates, and are displayed adjacent to each other. As shown in FIG. 10, the color of the first image 1001, that is, the color of the circle indicating the intensity of the fundamental tone and the harmonic overtone in the first image 1001, is the color of the second image 1003, that is, the second image 1003. It may be displayed differently from the color of the circle indicating the intensity of the fundamental and overtones of the reference sound. Thus, on the display unit 15 of the sound display device 10 </ b> A, the first image 1001 and the second image 1003 are displayed so as to be distinguished by an observer who views the display unit 15. The display position of the fundamental tone intensity and the intensity of one or more harmonics in the first image 1001 and the second image 1003 is the same as the harmonics display position described with reference to FIG.

In order to distinguish between the first image 1001 and the second image 1003, FIG. 10 shows an example in which the color of the first image 1001 and the color of the second image 1003 are displayed differently. It is not limited to. For example, as shown in FIG. 11, in order to distinguish between the first image 1101 and the second image 1103, in the first image 1101, a circle is displayed on the polar coordinates to indicate the fundamental tone and overtone strength of the performance sound. In the second image 1103, a rectangle may be displayed on polar coordinates for indicating the intensity of the fundamental tone and the harmonic overtone of the reference tone.

10 and 11, the first images 1001 and 1101 and the second images 1003 and 1103 are displayed on the display unit 15 so as to be adjacent to each other. However, the display positions of the first image and the second image are not limited to this. For example, as shown in FIG. 12, the first image 1201 and the second image 1203 may be displayed in duplicate on the same polar coordinate on the display unit 15. The first image 1201 and the second image 1203 are displayed so as to be distinguished by an observer who visually recognizes the display unit 15. In FIG. 12, in the first image 1201, a circle is displayed on the polar coordinates to indicate the intensity of the fundamental tone and harmonics of the performance sound, and in the second image 1203, the polar coordinates are indicated to indicate the intensity of the fundamental tone and harmonics of the reference sound. A rectangle is displayed above. The method for distinguishing between the first image 1201 and the second image 1203 is not limited to this, and the first image 1201 and the second image 1203 may be displayed so that the color of one image is different from the color of the other image. Good.

The display format of the first image and the second image on the display unit 15 and the color and shape of the first image and the second image may be set by the user of the sound display device 10 </ b> A by the operation unit 21.

In the present embodiment, the intensity of the fundamental tone and harmonics included in the performance sound of the input musical instrument and the intensity of the fundamental tone and harmonics included in the reference tone are simultaneously displayed on the display unit 15 of the sound display device 10A. can do. Therefore, the performer can visually recognize and compare the characteristics of the performance sound and the reference sound directly with the performer.

In the above description of the second embodiment, a case is described in which the reference sound is a music piece, but the reference sound is not limited to a music piece and may be a predetermined phrase. By performing the same phrase as the reference sound, the performer determines the intensity of the fundamental sound and one or more harmonics of the performance sound of the phrase, and the intensity of the fundamental sound of the reference sound and the intensity of the one or more harmonics. Can be compared.

(Modification)
Below, the modification of 1st Embodiment and / or 2nd Embodiment which were described above is demonstrated.

[Modification 1]
In the first embodiment and the second embodiment, the case where the intensity of the fundamental tone of a sound and the intensity of one or more harmonics of the fundamental tone are sequentially displayed on polar coordinates has been described as an example. However, the display form of the intensity of the fundamental sound and the intensity of one or more harmonics of the fundamental sound is not limited to this. For example, as shown in FIG. 13, the intensity of the fundamental tone of a sound and the intensity of one or more harmonics of the fundamental tone may be displayed on a graph in which time is plotted on the horizontal axis and harmonic order is plotted on the vertical axis. Good. By displaying in this way, it is possible to visually recognize changes in the intensity of the fundamental tone and the intensity of one or more harmonics of the fundamental tone. Also in the first modification, the position of each overtone relative to the position of the fundamental tone is fixed and displayed, and the intensity of the fundamental tone and the intensity of the harmonic overtone are displayed in a state where the position of each overtone relative to the position of the fundamental tone is fixed. Modification 1 described above can be applied to the first embodiment and the second embodiment.

[Modification 2]
Furthermore, as shown in FIG. 14, the intensity of the fundamental tone of the sound and the intensity of one or more harmonics of the fundamental tone are displayed on a graph in which the horizontal axis indicates the order of the harmonics and the vertical axis indicates the intensity of the harmonics. May be. Displaying in this way may make it easier for an observer to visually recognize the ratio of the intensity of each harmonic to the intensity of the fundamental tone. Modification 2 described above can be applied to the first embodiment and the second embodiment.

[Modification 3]
In the first embodiment and the second embodiment, in the data generation unit of the sound detection devices 100 and 100A, the data generation unit 109 includes the intensity of the fundamental tone detected by the detection unit 103 and the intensity of one or more harmonics of the fundamental tone. Display data for displaying the intensity of the fundamental tone and the intensity of one or more overtones is generated. However, when the intensity of the harmonic overtone is smaller than a predetermined value, there is a possibility that the intensity of the harmonic overtone cannot be displayed on the display unit 15.

Therefore, the data generation unit 109 acquires the intensity of the fundamental tone detected by the detection unit 103 and the intensity of one or more harmonics of the fundamental tone, and one or more harmonics of the fundamental tone with respect to the acquired fundamental tone intensity. The display data for displaying the image indicating the intensity of the fundamental tone and the intensity of the one or more overtones normalized may be sequentially generated. Modification 3 described above can be applied to the first embodiment and the second embodiment. In particular, when the third modification is applied to the second embodiment, when the harmonics of the reference sound are normalized not only to the harmonics of the performance sound by the performer but also to the fundamental tone of the reference sound, The difference between the intensity of the overtone of the performance sound and the intensity of the overtone of the reference sound can be visually recognized more clearly.

[Modification 4]
In the first embodiment and the second embodiment, the sound detection devices 100 and 100A detect the intensity of the fundamental tone and the overtone intensity of the performance sound of the musical instrument by a single player. However, the sound detecting device detects the intensity of the fundamental tone and the intensity of the harmonic overtones of the performance sounds of each instrument by two or more players, that is, two or more different sound data, and displays them on the display unit 15. Display data may be generated.

When visualizing the characteristics of the fundamental and overtones in the performance sound of an instrument played by two performers, the sound data acquisition unit 101 simultaneously acquires two different sound data. These sound data are data obtained by converting a performance sound acquired by the sound input unit 19 such as a microphone into sound data and digitized by the signal processing unit 17. The detection unit 103 sequentially detects the intensity of the fundamental tone of the sound data for a predetermined period and the intensity of one or more harmonics of the fundamental tone for each of the first sound data and the second sound data, and the data generation units 109 and 909 First display data for displaying an image indicating the intensity of the fundamental sound of the detected first sound data and the intensity of the one or more harmonics, and the intensity of the fundamental sound and the one or more harmonics of the second sound data Second display data for displaying an image indicating the intensity is sequentially generated.

On the basis of the generated first display data and second display data, two images indicating the intensity of the fundamental tone and the intensity of the harmonics in the performance sound of the musical instrument by the two performers are displayed on the screen of the display unit 15. . The two images may be displayed so as to be distinguished from each other on the screen of the display unit 15. For example, as described in the second embodiment, the color or shape of one image may be displayed different from the color or shape of the other image. The two images may be displayed side by side on the screen of the display unit 15 or may be displayed in a superimposed manner.

In the fourth modification described above, it is possible to visualize the characteristics of the fundamental and overtones in the performance sound of the musical instrument by the two performers. For example, it is possible to visually recognize the characteristics of the performance sound of the musical instrument by the teacher and the characteristics of the performance sound of the musical instrument by the student. Modification 4 described above can be applied to the first embodiment and the second embodiment. When displaying two images indicating the intensity of the fundamental tone and the intensity of the harmonic overtones in the performance sound of the musical instrument by the two performers, one or more harmonics of the fundamental tone is displayed for the intensity of the fundamental tone of each performance sound. May be displayed, and an image showing the intensity of the fundamental tone and the intensity of one or more overtones normalized may be displayed. In this case, the observer can more clearly recognize the difference between the intensity of the overtone of the musical instrument performance by one player and the intensity of the harmonic of the musical instrument performance by the other player.

[Modification 5]
In the second embodiment, the sound detection device 100A includes the intensity of the fundamental tone in the performance sound of the performer and the intensity of one or more harmonics of the fundamental tone, the intensity of the fundamental tone of the reference tone, and one or more harmonics of the fundamental tone. The display data for displaying side by side or superimposed on the intensity is generated. However, the sound detection apparatus calculates the difference between the intensity of the fundamental tone and the intensity of one or more harmonics of the fundamental tone, and the intensity of the fundamental tone of the reference sound and the intensity of the one or more harmonics of the fundamental tone. Display data for display may be generated. For example, as shown in FIG. 5, when a harmonic is displayed on the pole table, the size of the graphic indicating the strength of the harmonic is set to one or more harmonics in the performance sound of the performer with respect to the strength of one or more harmonics of the reference sound. You may show based on the difference of intensity | strength. In this case, the color of the graphic indicating the intensity of the harmonics may be displayed differently depending on whether the intensity of the one or more harmonics in the performance sound is small or large with respect to the intensity of the harmonics of the reference sound. Instead of the reference sound, a performance sound by another player who serves as a model may be used.

[Modification 6]
The sound display devices 10 and 10A of the first and second embodiments perform scale setting according to the type of musical instrument played by the performer, and display an image corresponding to the scale set in the display unit 15. May be. The performer may input the type of musical instrument to be used before playing the musical instrument into the sound display devices 10 and 10A via the operation unit 21 in advance. Alternatively, the sound display devices 10 and 10A may estimate the type of musical instrument based on the range and tone color of the musical instrument used for performance. In this case, the sound display devices 10 and 10A may request the performer to output the sound of the musical instrument to be used before acquiring the sound data.

In the sixth modification described above, the sound characteristics can be displayed in accordance with the type of musical instrument used by setting the scale. For example, a reference sound according to the type of instrument used can be automatically presented to the performer. In addition, when displaying an image indicating the intensity of the fundamental tone and the harmonic overtone of the sound data performed by the performer, the intensity of the specific harmonic can be weighted and displayed according to the type of the instrument.

DESCRIPTION OF SYMBOLS 10,10A ... Sound display apparatus, 11 ... Control part, 13 ... Memory | storage part, 15 ... Display part, 17 ... Signal processing part, 19 ... Sound input part, 21 ... Operation part, 23 ... Communication part, 100, 100A ... Sound Detection device, 101 ... sound data acquisition unit, 103 ... detection unit, 105 ... pitch determination unit, 107 ... intensity detection unit, 109, 909 ... data generation unit

Claims (18)

A sound data acquisition unit for acquiring first sound data;
A detection unit that sequentially detects the intensity of the fundamental sound of the first sound data and the intensity of one or more harmonics of the fundamental sound in a predetermined period of the acquired first sound data;
A data generation unit for sequentially generating display data for displaying a first image indicating the intensity of the fundamental sound and the intensity of the one or more harmonics of the detected first sound data;
Including a sound detection device. The sound detection device according to claim 1, wherein in the first image, the intensity of the fundamental sound and the intensity of the one or more overtones are displayed in a magnitude based on each intensity. The sound detection device according to claim 1, wherein in the first image, the fundamental tone and the one or more harmonics are displayed in different colors. The detector is
A pitch determination unit for determining the pitch of the fundamental tone;
The sound detection device according to claim 1, further comprising: an intensity detection unit configured to detect an intensity of the fundamental sound and an intensity of the one or more overtones. The sound detection device according to claim 4, wherein in the first image, at least one of the intensity of the fundamental tone and the intensity of the one or more harmonics is displayed in a color corresponding to a pitch of the fundamental tone. The sound detection device according to claim 4, wherein a pitch of the fundamental tone is displayed in the first image. 3. The sound detection device according to claim 2, wherein in the first image, the intensity of the one or more harmonics is normalized and displayed with respect to the intensity of the fundamental sound. In the first image,
The sound detection device according to claim 1, wherein the intensity of the one or more overtones is displayed at an arbitrary position from a position where the intensity of the fundamental sound is displayed. In the first image,
The intensity of the one or more overtones is displayed in a radial direction from a position where the intensity of the fundamental tone is displayed,
When the intensity of two or more harmonic overtones is displayed in a predetermined radial direction, the harmonics displayed in the position closest to the position where the intensity of the fundamental tone is displayed are displayed in the predetermined radial direction. The sound detection apparatus according to claim 8, wherein the other overtones have an octave relationship. 10. The sound detection device according to claim 9, wherein in the first image, the intensity of the one or more harmonics is displayed at a position farther from a position where the intensity of the fundamental is displayed as the order of the harmonic is higher. The sound detection device according to claim 1, wherein the first image includes a time axis and an axis indicating the order of harmonics. The display data is data for further displaying a second image indicating the intensity of the fundamental sound of the second sound data different from the first sound data and the intensity of one or more harmonics of the fundamental sound of the second sound data. The sound detection device according to claim 1. The data generator is
From the storage unit for storing the intensity of the fundamental sound of the second sound data and the intensity of one or more harmonics of the fundamental sound of the second sound data, the intensity of the fundamental sound of the second sound data and 1 of the fundamental sound of the second sound data are stored. The sound detection device according to claim 12, wherein the intensity of one or more overtones is acquired. The second sound data is acquired by the sound data acquisition unit together with the first sound data,
The sound detection device according to claim 12, wherein the intensity of a fundamental tone of sound data of a predetermined period of the second sound data and an intensity of one or more harmonics of the fundamental tone are sequentially detected by the detection unit. The sound detection device according to any one of claims 12 to 14, wherein the first image and the second image are displayed separately from each other. The sound detection device according to claim 15, wherein the first image and the second image are displayed adjacent to each other. The sound detection device according to claim 15, wherein the first image and the second image are displayed in a superimposed manner. On the computer,
Get sound data,
Sequentially detecting the intensity of the fundamental sound of the sound data of a predetermined period and the intensity of one or more harmonics of the fundamental sound among the acquired sound data;
Sequentially generating display data for displaying an image indicating the intensity of the detected fundamental tone and the intensity of the one or more harmonics;
A program for running

Images