JP2006030609A - Voice synthesis data generating device, voice synthesizing device, voice synthesis data generating program, and voice synthesizing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily generate voice synthesis data for generating a natural synthesized voice. <P>SOLUTION: An input voice waveform 1 is analyzed by a segmentation part 3, a pitch detection part 4, and a sound volume detection part 5 respectively, and segment information showing continuance lengths of a phoneme symbol and a phoneme, pitch information showing variation in pitch, and sound volume information showing variation in sound volume are outputted. Further, a timbre data detection part 7 of a timbre information extraction part 6 applies a timbre analysis of the same kind with an object voice synthesis part 12 to the input voice waveform 1 and a timbre difference extraction part 10 outputs the difference value between the analysis result and dictionary data of a corresponding phoneme as timbre data. A voice synthesis data generation part 11 generates voice synthesis data from the segment information, pitch information, sound volume information, and timbre data. The voice synthesis part 12 to which the voice synthesis data are inputted varies frame data read out of a dictionary by the pitch information, sound volume information, and timbre data, synthesizes a voice based upon the varied frame data, and outputs the synthesized voice. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention is generated by a speech synthesis data generation device, a speech synthesis data generation program, and a speech synthesis data generation device or a speech synthesis data generation program that analyze input speech and generate speech synthesis data to be given to the speech synthesizer. The present invention relates to a speech synthesizer and a speech synthesis program for performing speech synthesis based on the synthesized speech data.

FIG. 5 is a diagram showing a configuration of a general speech synthesis system.
In FIG. 5, 51 is a character string input unit, 52 is a speech synthesis data generation unit that generates speech synthesis data corresponding to the input character string, 53 is a speech synthesis unit that synthesizes speech based on the speech synthesis data, and 54 is a synthesis. An audio output unit.
Here, the “speech synthesis data” input to the speech synthesizer 53 includes phoneme symbols, phoneme length (time length), prosodic information (pitch, volume change information), voice quality information (synthesized speech dictionary). A voice quality parameter corresponding to the selection or synthesis method; for example, the frequency of the first formant) is common. These can be arbitrarily input, and natural speech synthesis can be performed by creating and inputting appropriate speech synthesis data.
However, in order to create this speech synthesis data as pre-processing, generally, a character string (a character string mixed with Kana-Kanji in Japanese. For other languages, for example, an English character string in English, In Korean, you can input Korean character strings, etc. (or character strings with special symbols such as “increase pitch”) (“input string”), and use the rules (syntax Through the analysis and symbol analysis, the speech synthesis data is generated according to the phoneme length and the rules for generating the prosodic data.

Further, in Patent Document 1, reference data including a character string and height, strength, and length data is created from a musical score, and actual singing voice segmentation is performed using the reference data. Singing voice synthesis data generating device is described.
JP-A-3-7995

  However, in the method for generating input speech synthesis data to the speech synthesizer according to the conventional method described above, in order to generate data according to the “rules”, utterances that deviate from the rules (mainly prosodic changes and phoneme lengths). ”) (For example, the same“ Ohayo ”has various utterance methods that express humanity, naturalness, and personality). It was difficult to generate (to some extent by the user's input that adds prosodic symbols = pitch increase, etc.) or impossible (those that cannot input prosodic symbols). Of course, it is of course difficult to directly create “speech synthesis data”.

FIG. 6 and FIG. 7 are shown as examples showing various expressions of actual speech. Both figures show (A) speech waveform, (B) pitch, and (C) volume when another speaker utters without specifying how to speak “Ohayo”.
It is required to reproduce such differences in expression and changes in voice quality by speech synthesis.

  In view of such circumstances, the present invention provides a speech synthesis data generation apparatus, a speech synthesis data generation program, and the speech synthesis that can easily create speech synthesis data that can utter natural synthesized speech. It is an object of the present invention to provide a speech synthesizer and a speech synthesis program for synthesizing speech based on speech synthesis data generated by a data generation device or a speech synthesis data generation program.

In order to achieve the above object, a speech synthesis data generation apparatus and a speech synthesis data generation program according to the present invention analyze an actual speech waveform, segment information indicating a phoneme symbol and a phoneme duration, pitch information, and volume. Extract the information, perform the same analysis as the synthesized speech dictionary of the target speech synthesizer, find the difference value with the same phoneme data in the synthesized speech dictionary, output it as timbre data, these The main feature is that speech synthesis data is generated from segment information, pitch information, volume information, and timbre data.
Also, the speech synthesizer and speech synthesis program of the present invention inputs the speech synthesis data, and converts the frame data corresponding to the phoneme uttered read from the synthesized speech dictionary into the pitch information, the volume information, and the timbre data. It is characterized by changing the output accordingly.

According to the speech synthesis data generation apparatus or the speech synthesis data generation program of the present invention, because of natural changes in phoneme length, prosodic information, and voice quality for natural speech synthesis with a small amount of data. It is possible to easily create speech synthesis data including the above information.
According to the speech synthesizer or speech synthesis program of the present invention, natural speech can be synthesized using the speech synthesis data generated by the speech synthesis data generation device or the speech synthesis data generation program.

FIG. 1 is a block diagram showing a configuration of an embodiment of a speech synthesis data generation device according to the present invention and a speech synthesis device that performs speech synthesis based on speech synthesis data generated by the speech synthesis data generation device. These devices can also be realized by program processing in a computer.
In FIG. 1, reference numeral 1 denotes an input phoneme symbol string input as a segmentation input. This is a symbol string corresponding to the content of the speech waveform 2 to be analyzed. For example, in Japanese, a hiragana such as “Ohayo” may be used, or it is actually used inside the speech synthesizer. It may be a phoneme symbol string (for example, using SAMPA (Speech Assessment Methods Phonetic Alphabet). In this case, “ohajo”, etc.). However, in the case where the segmentation unit 3 is a pure speech recognition apparatus that does not require auxiliary information for such segmentation, this is not necessarily required.

Reference numeral 2 denotes an input speech waveform to be analyzed. This may be a recorded audio waveform file or input from a microphone.
Reference numeral 3 denotes a segmentation unit that performs segmentation of input speech based on information of the input phoneme symbol string 1 and the input speech waveform 2. For example, as shown in FIG. 6 and FIG. 7, the segment information output from here is from “seconds” to “seconds” and “ha” is from “seconds” to “seconds”. It is information indicating where a certain phoneme section is from where in time. When the phoneme symbol string is SAMPA, “o” is “o”, “ha” is “ha”, and “yo” is “jo”. Here, if the segmentation unit 3 is a “speech recognition unit”, similar information can be obtained without input of the input phoneme symbol string 1 by segmentation by speech recognition processing of the input speech waveform 2.

Reference numeral 4 denotes a pitch detection unit that detects a pitch from the input speech waveform 2 by an appropriate method. The pitch detection method may be any method such as measurement of a zero cross section.
As a result, a pitch trajectory along the time axis is obtained as information, as shown in FIG. As for the time resolution here, for example, it is sufficient if there is a frame period interval at the time of speech synthesis. If linear interpolation is performed on the side, less information is required.

Reference numeral 5 denotes a volume detector that detects the volume from the input voice waveform 2 by an appropriate method. As a sound volume detection method, for example, a method of obtaining an average energy within a certain frame section, for example, a 10 msec section is used.
As a result, as shown in FIG. 6C and FIG. 7C, the volume trajectory along the time axis is obtained as information. The time resolution here is, for example, sufficient if there is a frame period interval at the time of speech synthesis, and if the volume does not change much, it has data only for the part where the volume changes greatly, and a speech synthesizer in the meantime If linear interpolation is performed on the side, less information is required.

  A timbre information extraction unit 6 is shown within a dotted line in FIG. The timbre information extraction unit 6 includes a timbre data detection unit 7, an average timbre data extraction unit 9, and a timbre difference extraction unit 10 as shown in the figure. By providing the information extracted by the timbre information extraction unit 6 as a timbre parameter to the speech synthesizer 12, a change in timbre can be reproduced.

The timbre data detection unit 7 performs timbre analysis by the same method as that used by the target speech synthesis unit 12.
FIG. 2 is a diagram illustrating an example of timbre analysis in the timbre data detection unit 7.
(A) of FIG. 2 is an input speech waveform, and the timbre analysis is performed by cutting out the input speech waveform at each time interval (analysis frame period) indicated at the lower point.
FIG. 2B shows an example of the timbre analysis result at a certain time.
In the analysis method shown in this figure, four sets of frequency and intensity (frequency and intensity) are obtained for each frame. If this set is associated with the formant frequency and its intensity, it can be easily imagined that this information represents the information of “tone” and “phoneme”. This analysis is performed for all frames. Needless to say, these pieces of information include information including temporal changes in timbre. The four sets are indexed in order and the following processing is performed.

  Reference numeral 8 denotes a synthesized speech dictionary of the speech synthesizer. This stores information obtained by the same method as the analysis method of the timbre data detection unit 7 for each unit of speech synthesis (for example, “o”, “ha”, “yo”, respectively).

9 is a consonant part frame (for example, “ha”) in the synthesized speech dictionary 8 (for example, “ha” dictionary) corresponding to the current segment information (analyzed by the segmentation unit 3). The average of the “h” part of the data and the average of the data in the vowel part frame (for example, the “a” part of “ha”), and the average of which is the average timbre data A timbre data extraction unit. For example, the average of frequency and intensity of the first (frequency, intensity) group in each frame, the average of frequency and the intensity of the second (frequency, intensity) group, and so on. Obtain average tone data.
As a modification, the average tone color data may not be obtained here, and the data in the synthesized speech dictionary 8 (frame data actually used at the time of synthesis) may be extracted as they are.

  Reference numeral 10 denotes a timbre difference extraction unit that obtains a difference between the average timbre data obtained by the average timbre data extraction unit 9 and the timbre data obtained by the timbre data detection unit 7 every moment. The difference is the difference between the timbre data and the average timbre data of the phonemes to which the timbre data corresponds (for example, the average timbre data of “h” when the segment information result is “h”). In the modified example of the average tone color data extraction unit 9 (when the dictionary data is extracted as it is), the difference from the frame data used at the time of synthesis is obtained.

An example of difference information (tone color data) extracted by the timbre difference extraction unit 10 is shown in FIG.
FIG. 3A illustrates the difference between the strengths in each (frequency, strength) pair for each index number, and FIG. 3B similarly relates to the frequency. The figure which calculated | required the difference is illustrated.
When this timbre data is added to a set of (frequency, intensity) of the synthesized frame at each time at the time of voice synthesis, a change in timbre of the input voice can be reproduced. The new (frequency, intensity) pair reproduces the temporal change of the timbre of the input voice.
Here, as a modification, instead of giving this timbre difference data to the speech synthesizer as timbre data from moment to moment, the obtained timbre difference data is all averaged within the time of the input waveform. If it is given to the voice synthesizer as `` tone data '' once, the timbre change from moment to moment cannot be given, but instead, the overall timbre is close to the timbre of the input voice, not the synthesis dictionary as it is Can be.

11 is a speech synthesis data generation unit that converts the “segment information”, “pitch information”, “volume information”, and “timbre data” obtained so far so as to match the format corresponding to the target speech synthesizer. It is.
FIG. 4 is a diagram illustrating an example of speech synthesis data output from the speech synthesis data generation unit 11. 4A shows data for utterance (voice utterance data: utterance start time, utterance end time, utterance phoneme), and FIG. 4B shows a pitch trajectory (pitch data: pitch change time and its pitch). , (C) shows the volume trajectory (volume data: volume change time and its volume), and (D) shows timbre data (tone change time, change (difference) intensity and frequency for each index number). These are output in a format that can be input to the target speech synthesizer.

The speech synthesis data generating device of the present invention is configured by the above constituent elements. The segmentation unit 3, the pitch detection unit 4, the volume detection unit 5, the timbre information extraction unit 6 (the timbre data detection unit 7, the average timbre data extraction unit 9 and the timbre difference extraction unit 10), and the speech synthesis data generation unit 11 It can be realized by computer program processing (speech synthesis data generation program).
By supplying the speech synthesis data generated by the speech synthesis data generation apparatus or the speech synthesis data generation program to the speech synthesizer, it is possible to synthesize high quality speech with a small amount of data.

A speech synthesizer 12 synthesizes speech according to the input speech synthesis data. The synthesized speech dictionary 8 identical to the synthesized speech dictionary 8 connected to the average tone color data extracting unit 9 is also connected to the speech synthesizer 12. That is, the same synthesized speech dictionary 8 is provided in each of the speech synthesis data generation device and the speech synthesis device.
The voice synthesizer 12 changes the frame data read from the synthesized voice dictionary 8 based on the input voice synthesized data, performs voice synthesis based on the changed frame data, and outputs the synthesized voice. Specifically, frame data corresponding to the phoneme to be uttered is read from the synthesized speech dictionary 8 according to the voice utterance data, and the frame data is changed according to the pitch data at this time and the volume according to the volume data. Furthermore, it has means for adding the timbre (difference) data to the frame data. As a result, even the timbre change is reproduced.
The addition is performed in the case of the synthesis method described in this embodiment, and it goes without saying that multiplication, subtraction, etc. are performed in accordance with the target speech synthesis method. The voice synthesizer 12 can also be realized by program processing by a computer.
Thereby, a high-quality synthesized speech waveform 13 is obtained.

  As described above, according to the speech synthesis data generation device or the speech synthesis data generation program of the present invention, it is possible to directly generate “speech synthesis data” from an input speech waveform, which is accompanied by an appropriate prosodic symbol. If converted to an “input character string”, it can be directly applied to a synthesis system that accepts only input of a character string as a conventional input. For example, the converted input character string is the input character string 51 in FIG. However, this is limited to a speech synthesis system that can input prosodic symbols.

Such speech synthesis data generated by the speech synthesis data generation apparatus or the speech synthesis data generation program of the present invention can be applied to the following cases.
(Application example 1)
For a terminal such as a mobile phone having the above-described speech synthesizer of the present invention, the user creates desired speech synthesis data using the speech synthesis data generation device or the speech synthesis data generation program according to the present invention, and transmits it. To do. When another user who receives this data reproduces this data, it is possible to hear a natural synthesized voice with emotions as a voice from the terminal instead of a so-called uniform synthetic voice synthesized voice. Here, what is important is that the amount of data of the speech synthesis data is much smaller than that of raw speech data such as PCM.

(Application example 2)
It is very difficult to create data such as chatter and rap in a system that synthesizes singing by inputting note data, phoneme data, pitch bend, control change for volume change, and the like. The reason is that it is very difficult to create natural data because the length and intonation (change in pitch and volume) of the phoneme changes drastically compared to general singing. Therefore, in such a system, if the speech synthesis data generation device or the speech synthesis data generation program of the present invention is used, data can be extracted from natural speech input, so that such speech synthesis data can be easily created. Is possible.

In addition, not only in a speech synthesis system but also in a musical sound MIDI synthesizer system, for example, when synthesizing a musical sound rich in inflections and timbres such as saxophone, the pitch, volume, etc. , Change the timbre from the input performance, and its sequence data (program number: timbre number underlying the timbre, timbre parameters such as exclusive data, note number, volume control control change value, pitch bend value, etc.) If you give to the synthesizer, you can get a realistic synthesized sound. In this case, the phoneme segment part is not necessary, but it is better to detect the break of the note.
Furthermore, when synthesizing not only speech but also expressive musical sounds that contain a large amount of information, creation of input data to the synthesis system (for example, formant sound source) becomes difficult manually. If the speech synthesis data generation apparatus of the present invention is used to assist the data creation, such data creation is facilitated. However, it is necessary to have a standard performance. This can be said to be equivalent to motion capture in CG.

1 is a block diagram illustrating a configuration of an embodiment of a speech synthesis data generation device of the present invention and a speech synthesis device that performs speech synthesis based on speech synthesis data generated by the speech synthesis data generation device. FIG. It is a figure which shows the example of the timbre analysis in the timbre data detection part. It is a figure which shows the example of the difference information extracted by the said timbre difference extraction part 10, (A) is a figure which shows the difference regarding an intensity | strength, (B) is a figure which shows the example of the difference regarding a frequency. It is a figure which shows an example of the speech synthesis data output from the speech synthesis data production | generation part. It is a figure which shows the structure of a general speech synthesis system. It is a figure which shows the example of (A) audio | voice waveform, (B) pitch, and (C) volume when sounding "Ohayoyo". FIG. 7 is a diagram illustrating an example of (A) speech waveform, (B) pitch, and (C) volume when a speaker different from FIG. 6 pronounces “Ohayo!”.

Explanation of symbols

  1: input phoneme symbol string, 2: input speech waveform, 3: segmentation unit, 4: pitch detection unit, 5: volume detection unit, 6: tone color information extraction unit, 7: tone color data detection unit, 8: synthesized speech dictionary, 9: average timbre data extraction unit, 10: timbre difference extraction unit, 11: speech synthesis data generation unit, 12: speech synthesis unit, 13: synthesized speech waveform

Claims (8)

A speech synthesis data generation device that analyzes an input speech waveform and generates speech synthesis data to be supplied to the speech synthesizer,
Means for analyzing the input speech waveform and outputting segment information indicating the phoneme symbol and the phoneme duration, pitch information and volume information;
Means for performing the same kind of timbre analysis as in the speech synthesizer on the input speech waveform, and outputting a difference value between the analysis result and dictionary data of the corresponding phoneme as timbre data;
A speech synthesis data generation apparatus comprising: means for generating speech synthesis data from the segment information, the pitch information, the volume information, and the timbre data.   2. The speech synthesis data generation device according to claim 1, wherein the difference value is a difference value between the analysis result and an average value of frame data in dictionary data of the corresponding phoneme.   The speech synthesis data generation device according to claim 1, wherein the difference value is a difference value between the analysis result and data of temporally matching frames in the dictionary data of the corresponding phoneme.   4. The voice synthesis data according to claim 1, wherein the means for outputting the difference value as timbre data outputs the difference value calculated for each frame as the timbre data. Generator.   4. The speech synthesis according to claim 1, wherein the means for outputting the difference value as timbre data outputs the difference value averaged over a plurality of frames as the timbre data. Data generator. A speech synthesizer that synthesizes speech based on speech synthesis data including a phoneme symbol of speech to be synthesized and segment information indicating the duration of the phoneme, pitch information, volume information, and timbre data,
A speech synthesizer comprising means for changing and outputting frame data corresponding to a phoneme to be uttered read from a synthesized speech dictionary according to the pitch information, the volume information, and the timbre data. On the computer,
Analyzing the input speech waveform and outputting segment information indicating a phoneme symbol and a phoneme duration, pitch information and volume information;
Performing the same kind of timbre analysis as in the target speech synthesizer on the input speech waveform, and outputting a difference value between the analysis result and dictionary data of the corresponding phoneme as timbre data;
Generating a speech synthesis data from the segment information, the pitch information, the volume information, and the timbre data. A speech synthesis program for causing a computer to execute a process of synthesizing speech based on speech synthesis data including segment information, pitch information, volume information, and timbre data of speech to be synthesized,
A speech synthesis program comprising the step of changing and outputting frame data corresponding to a phoneme to be uttered read from a synthesis speech dictionary according to the pitch information, the volume information, and the timbre data.

Images