WO2012115212A1 - Speech-synthesis system, speech-synthesis method, and speech-synthesis program - Google Patents

Abstract

Provided is a technology for generating rules that allow highly-natural speech synthesis without unnecessarily collecting large amount of learning data. This speech-synthesis system includes: a learning database that stores learning data, said learning data being a set of feature quantities extracted from speech waveform data; a feature-quantity-space partitioning means that partitions a feature-quantity space, which is a space related to the learning data, into subspaces; a density-detection means that detects the density of each of the subspaces into which the feature-quantity space has been partitioned and generates and outputs density information indicating said densities; and a rule-generation means that, on the basis of the outputted density information, generates speech-synthesis rules for generating pronunciation information used in speech synthesis.

Description

Speech synthesis system, speech synthesis method, and speech synthesis program

The present invention relates to a speech synthesis system, a speech synthesis method, and a speech synthesis program, and more particularly, to a technique for realizing speech synthesis with high naturalness.

In recent years, with the advance of text-to-speech (TTS), many services and products using human-synthesized synthesized speech have been seen. In general, the TTS first analyzes the language structure or the like of text input by morphological analysis or the like (language analysis processing), and generates phoneme information to which accents or the like are given based on the result. Further, the TTS estimates the fundamental frequency (F0) pattern and phoneme duration based on the pronunciation information, and generates prosodic information (prosodic generation processing). Finally, the TTS generates a waveform based on the generated prosodic information and phonological information (waveform generation process).
As a method of prosody generation processing described above, as shown in Non-Patent Document 1, a method of modeling F0 patterns so that they can be expressed by simple rules and generating prosody using the rules is known. Yes. The method using rules is widely used because the F0 pattern can be generated with a simple model. However, there is a problem that the synthesized speech becomes mechanical because the prosody is unnatural.
On the other hand, in recent years, a speech synthesis method using a statistical method has attracted attention. A typical technique is described in Non-Patent Document 2. Non-Patent Document 2 discloses HMM speech synthesis using a hidden Markov model (HMM) as a statistical method. The technology of HMM speech synthesis generates speech using a prosodic model and a speech synthesis unit (parameter) model modeled using a large amount of learning data. Since the technology of HMM speech synthesis uses speech uttered by an actual human as learning data, a prosody that is more human can be generated compared to the F0 generation model described above.

Hiroya Fujisaki and Hiroshi Sudo, “Basic frequency pattern of Japanese word accent and model of its generation mechanism”, Journal of Acoustical Society of Japan, Vol.27, No.9, pp. 445-453, 1971. Keiichi Tokuda, “Application of Hidden Markov Model to Speech Synthesis”, Technical Report of IEICE, SP99-61, pp. 47-54, 1999.

However, in a speech synthesis method using a statistical method as described in the above-mentioned non-patent document, a correct F0 pattern may not be generated, resulting in an unnatural speech. The reason for this is that in the speech synthesis method using the statistical method, the learning data space is divided into subspaces (clustering) mainly based on the information amount of the learning data, resulting in a dense and dense state of information amount in the space. This is because there is a sparse space with little learning data.
As one method for solving this problem, a model learning with a larger amount of data is conceivable. However, it is difficult to collect a large amount of learning data, and it is unrealistic because it is unclear how much data amount should be collected.
As described above, an object of the present invention is to provide a technique for generating a rule that enables speech synthesis with high naturalness without collecting an unnecessarily large amount of learning data.

In order to achieve the above object, a speech synthesis system according to the present invention includes a learning database that stores learning data that is a set of feature amounts extracted from speech waveform data, and a space related to the learning data that is stored in the learning database. A feature amount space dividing unit that divides a certain feature amount space into partial spaces, and a sparse state with respect to each partial space that is a feature amount space divided by the feature amount space dividing unit, and information indicating the sparse state A sparse / dense state detection unit that generates and outputs certain sparse / dense information, and generates a speech synthesis rule that is a rule for generating pronunciation information used for speech synthesis based on the sparse / dense information output from the sparse / dense state detection unit And rule generation means.
To achieve the above object, the speech synthesis method of the present invention stores learning data, which is a set of feature amounts extracted from speech waveform data, and sets a feature amount space, which is a space related to the stored learning data, as a partial space. , And detects the sparse / dense state for each partial space that is the divided feature amount space, generates and outputs the sparse / dense information that is information indicating the sparse / dense state, and based on the output sparse / dense information, A speech synthesis rule, which is a rule for generating pronunciation information used for speech synthesis, is generated.
In order to achieve the above object, a program stored in the recording medium of the present invention stores learning data that is a set of feature values extracted from speech waveform data, and a feature amount space that is a space related to the stored learning data. Divide into subspaces, detect a sparse / dense state for each subspace that is the divided feature amount space, generate and output sparse / dense information that is information indicating the sparse / dense state, and output the sparse / dense information Based on this, the computer is caused to execute processing for generating a speech synthesis rule that is a rule for generating pronunciation information used for speech synthesis.

According to the speech synthesis system, speech synthesis method, and speech synthesis program of the present invention, it is possible to generate a rule capable of realizing highly natural speech synthesis without collecting an unnecessarily large amount of learning data.

It is a block diagram which shows the structural example of the speech synthesis system 1000 which concerns on 1st Embodiment of this invention. It is a flowchart which shows an example of operation | movement of the speech synthesis system 1000 which concerns on 1st Embodiment of this invention. It is a block diagram which shows the structural example of the speech synthesis system 2000 which concerns on 2nd Embodiment of this invention. FIG. 4 is a schematic diagram of a decision tree structure created by binary tree structure clustering as a result of learning in the feature amount space dividing unit 1; FIG. 3 is a conceptual schematic diagram of a feature amount space that represents a clustering result of learning data by a feature amount space dividing unit 1. 12 is a flowchart illustrating an example of an operation of generating a speech synthesis rule in a preparation stage in the speech synthesis system 2000. 10 is a flowchart illustrating an example of an operation of creating a prosody generation model in the preparation stage in the speech synthesis system 2000. 6 is a flowchart illustrating an example of an operation in a speech synthesis stage in which speech synthesis processing is actually performed in the speech synthesis system 2000. It is a block diagram which shows the structural example of the speech synthesis system 3000 which concerns on 3rd Embodiment of this invention. 10 is a flowchart illustrating an example of an operation of creating a waveform generation model in the preparation stage in the speech synthesis system 3000. 10 is a flowchart illustrating an example of an operation in a speech synthesis stage in which speech synthesis processing is actually performed in the speech synthesis system 3000. It is a block diagram which shows an example of the hardware constitutions which implement | achieve the speech synthesis system 2000 which concerns on 2nd Embodiment.

First, in order to facilitate understanding of the embodiments of the present invention, the background of the present invention will be described.
In a technique using a statistical method as described in Non-Patent Document 2, a correct F0 pattern may not be generated, resulting in an unnatural voice.
More specifically, for example, there is a sufficient number of learning data of about several mora such as “person” (2 mora), “word” (3 mora), and “voice” (4 mora). Here, mora is a phrase unit of sound having a certain length of time, and is generally called a beat in Japanese. Therefore, a technique using a statistical method can generate a correct F0 pattern for a sound of several mora. However, there is a possibility that learning data such as “Albert Einstein Medical University” (18 mora) is extremely small or does not exist. For this reason, when a text including such a word is input, the F0 pattern is disturbed, causing a problem such as a shift of the accent position.
According to the embodiment of the present invention described below, a language analysis result belonging to a partial space with a small amount of learning data is not generated or is hardly generated. Therefore, according to the embodiment of the present invention, it is possible to avoid instability of speech synthesis due to lack of learning data, and it is possible to generate synthesized speech with high naturalness.
Embodiments of the present invention will be described below with reference to the drawings. In addition, about each embodiment, the same code | symbol is attached | subjected to the same component and description is abbreviate | omitted suitably. In the following embodiments, the case of Japanese is described as an example, but the application of the present invention is not limited to the case of Japanese.
<First Embodiment>
FIG. 1 is a block diagram showing a configuration example of a speech synthesis system 1000 according to the first embodiment of the present invention. Referring to FIG. 1, a speech synthesis system 1000 according to the present embodiment includes a feature amount space division unit 1, a sparse / dense state detection unit 2, a rule generation unit 3, and a learning database 4.
The learning database 4 stores a set of feature amounts extracted from speech waveform data as learning data. The learning database 4 stores pronunciation information that is a character string corresponding to the speech waveform data. The learning database 4 may store time length information, pitch information, and the like.
Here, the feature amount that is the learning data includes at least an F0 pattern that is time change information of F0 in the speech waveform. Furthermore, the feature amount that is learning data may include spectrum information obtained by fast Fourier transform (FFT) of a speech waveform, segmentation information that is time length information of each phoneme, and the like.
The feature amount space dividing unit 1 divides a space related to learning data stored in the learning database 4 (hereinafter referred to as “feature amount space”) into partial spaces. Here, the feature amount space is an N-dimensional space with N predetermined feature amounts as axes. The number N of dimensions is arbitrary. For example, when two feature amounts of spectrum information and segmentation information are used as axes, the feature amount space is a two-dimensional space.
The feature amount space dividing unit 1 may divide the feature amount space into partial spaces by binary tree structure clustering based on the information amount. The feature amount space dividing unit 1 outputs the learning data divided into partial spaces to the sparse / dense state detecting unit 2.
The sparse / dense state detecting unit 2 detects a sparse / dense state for each partial space generated by the feature amount space dividing unit 1 and generates sparse / dense information which is information indicating the sparse / dense state. The sparse / dense state detection unit 2 outputs the generated sparse / dense information to the rule generation unit 3.
Here, the density information is information indicating a density state of the amount of information of learning data. The density information may be an average value and a variance value of feature quantity vectors of learning data groups belonging to the partial space.
The rule generation unit 3 generates a speech synthesis rule based on the density information output from the density state detection unit 2.
Here, the speech synthesis rules are rules for generating pronunciation information, which is information necessary for synthesizing speech. The speech synthesis rules include at least language analysis information. Here, the language analysis information is information related to data and rules necessary for text language analysis processing. The language analysis information is information on data and rules for morphological analysis, for example.
The speech synthesis rules include information indicating a method for adding additional information for speech synthesis, which is information such as accent positions and accent phrase boundary positions, in addition to language analysis information.
The speech synthesis rule makes the score in the dictionary extremely low so that it is not output as a language analysis result for a language that is expressed by the F0 pattern belonging to a (sparse) subspace with little learning data, or 0 A rule such as
Note that the pronunciation information is information necessary for synthesizing speech, and may include information such as phonemes, syllable strings, and accent positions that express the utterance content. Specifically, the pronunciation information is subjected to language analysis processing such as morphological analysis on the text, and additional information for speech synthesis such as accent position and accent phrase boundary position is added to the result of the language analysis processing or changed. It is generated by performing processing such as adding.
For example, consider a case where a text including the word “Albert Einstein Medical University” is input. In this case, the pronunciation information related to the word is, for example, a character string “a ru ba-to ai N syu ta i N i kada @ i gaku” in Japanese. “@” Indicates an accent position. The rule that defines how the pronunciation information is generated is the above-described speech synthesis rule.
FIG. 2 is a flowchart showing an example of the operation of the speech synthesis system 1000 according to the first embodiment of the present invention.
As shown in FIG. 2, the feature amount space dividing unit 1 first divides a feature amount space that is a space related to learning data stored in the learning database 4 (step S1).
Next, the sparse / dense state detecting unit 2 detects the sparse / dense state of the information amount of the learning data in each partial space that is a part of the feature amount space divided by the feature amount space dividing unit 1, and indicates the sparse / dense state. The density information is generated (step S2). The sparse / dense state detection unit 2 outputs the generated sparse / dense information to the rule generation unit 3.
Next, the rule generation unit 3 generates a speech synthesis rule based on the density information output from the density state detection unit 2 (step S3).
As described above, according to the speech synthesis system 1000 according to the present embodiment, it is possible to avoid the instability of speech synthesis due to lack of learning data and to generate highly natural synthesized speech. Become. The reason is that the speech synthesis system 1000 generates a rule that does not generate or is difficult to generate pronunciation information belonging to a partial space with less learning data.
Second Embodiment
Subsequently, a second embodiment of the present invention will be described.
FIG. 3 is a block diagram showing a configuration example of the speech synthesis system 2000 according to the second embodiment of the present invention. Referring to FIG. 3, the speech synthesis system 2000 according to the present embodiment includes a learning database 4, a speech synthesis learning device 20, a prosody generation model storage unit 6, a language analysis dictionary 7, and a modified language analysis dictionary. 8 and the speech synthesizer 40.
The speech synthesis learning device 20 includes a feature amount space dividing unit 1, a sparse / dense state detecting unit 2, a rule generating unit 3, and a prosody learning unit 5. The feature amount space dividing unit 1 and the sparse / dense state detecting unit 2 have the same configuration as in the first embodiment.
In the present embodiment, HMM is used as a statistical method, and binary tree clustering is used as a feature space dividing method. When an HMM is used as a statistical method, clustering and learning are generally performed alternately. For this reason, in the present embodiment, the feature space division unit 1 and the prosody learning unit 5 are combined into the HMM learning unit 30 and do not take an explicitly divided configuration. However, this embodiment is merely an example of an embodiment of the invention, and the configuration of the invention is not limited to this when a statistical method other than the HMM is used.
Referring to FIG. 3, the speech synthesizer 40 includes a language analysis unit 9, a prosody generation unit 10, and a waveform generation unit 11.
In the present embodiment, it is assumed that sufficient learning data is stored in the learning database 4 in advance. That is, the learning database 4 stores feature amounts extracted from a large amount of speech waveform data. It is assumed that the learning database 4 stores F0 patterns, segmentation information, and spectrum information as feature values of speech waveform data. A set of these feature amounts is used as learning data. Further, it is assumed that the learning data is collected from the voice of one speaker.
First, learning by a statistical method using the learning database 4 is performed in the HMM learning unit 41 (the feature amount space dividing unit 1 and the prosody learning unit 5).
In the HMM learning unit 30, the feature amount space dividing unit 1 divides the feature amount space stored in the learning database 4 into partial spaces, as in the first embodiment. Specifically, the feature amount space dividing unit 1 divides the feature amount space stored in the learning database 4 into partial spaces by binary tree structure clustering. Hereinafter, the partial space generated by the feature amount space dividing unit 1 is also referred to as a cluster.
FIG. 4 is a schematic diagram of a decision tree structure created by binary tree structure clustering as a result of learning in the feature amount space dividing unit 1. As shown in FIG. 4, the binary tree clustering is a process of dividing the learning data into two nodes according to the questions arranged in the nodes P1 to P6, and finally the information amount of each divided cluster This is a method of clustering so that is uniform.
For example, in FIG. 4, the feature amount space division unit 1 determines whether “YES” or “NO” corresponds to the question arranged at the current node, and divides the learning data. In the example of FIG. 4, the feature amount space dividing unit 1 divides the learning data based on whether or not “the phoneme is a voiced sound”, which is a question initially placed at the node P1. Next, for example, the feature amount space dividing unit 1 divides the learning data divided by being determined as “YES” based on whether or not “the preceding phoneme is an unvoiced sound” which is a question arranged in the node P2. The feature amount space dividing unit 1 repeats such division and divides the learning data into one cluster at a stage where the division is made into a predetermined number of learning data.
FIG. 5 is a conceptual schematic diagram of the feature amount space showing the clustering result of the learning data by the feature amount space dividing unit 1. The vertical and horizontal axes in FIG. 5 indicate predetermined feature amounts.
FIG. 5 shows a case where the number of learning data belonging to each cluster is four. FIG. 5 shows how the number of mora of the learning data corresponding to each cluster and the type of the accent kernel are as a result of the division by the feature amount space dividing unit 1 until the number of learning data becomes four. It is shown. Here, the type of the accent kernel is a type indicating the position immediately before the pitch is greatly lowered in one accent phrase.
Note that FIG. 5 is a schematic diagram illustrating the concept to the last, and the number of axes is not limited to two. The feature amount space may be a 10-dimensional space with ten feature amounts as axes, for example.
As shown in FIG. 5, the feature amount space dividing unit 1 generates a large cluster in a space where the number of learning data is sparse, such as a cluster of 10 mora or more and an 8-type or more cluster. Such a cluster is a sparse cluster with a very small number of learning data.
The feature amount space dividing unit 1 outputs the learning data divided into the partial spaces to the sparse / dense state detecting unit 2 and the prosody learning unit 5.
The HMM learning unit 30 creates a prosody generation model together with the division of the feature amount space.
In the HMM learning unit 30, the prosody learning unit 5 learns the prosody model in the feature amount space divided by the feature amount space dividing unit 1, and creates a prosody generation model. That is, the prosody learning unit 5 creates a prosody generation model using the clustering result of the learning data (for example, the result of the binary tree clustering shown in FIG. 4) in the feature amount space dividing unit 1.
The prosody generation model storage unit 6 stores the prosody generation model created by the prosody learning unit 5.
Specifically, the prosody learning unit 5 statistically learns what prosody should be generated for the pronunciation information corresponding to the speech waveform data stored in the learning database 4 for each cluster. The prosody learning unit 5 uses the learning result as a model (prosody generation model) and stores it in the prosody generation model storage unit 6 in association with each cluster.
The learning database 4 may be configured not to store time length information and pitch information, and the prosody learning unit 5 may be configured to learn time length information and pitch information corresponding to pronunciation information from input speech waveform data. .
Next, the sparse / dense state detecting unit 2 extracts the sparse / dense information of each cluster in the learning data input from the feature amount space dividing unit 1. The density information may be, for example, a variance value regarding the number of mora of the accent phrase and the relative position of the accent kernel. At this time, for example, in the 3 mora 1 type cluster shown in FIG. 5, all data is 3 mora 1 type. Therefore, the variance value is 0.
The sparse / dense state detection unit 2 outputs the extracted sparse / dense information of each cluster to the rule generation unit 3.
Next, the rule generation unit 3 generates a speech synthesis rule based on the density information of each cluster. Here, it is assumed that the rule generation unit 3 generates a speech synthesis rule by correcting the existing language analysis dictionary 7. Here, the language analysis dictionary 7 is a dictionary that stores the above-described language analysis information that is data and rules necessary for text language analysis processing.
In the present embodiment, the rule generation unit 3 corrects the language analysis dictionary 7 with a policy of “no generation of accent phrase pronunciation information belonging to a sparse cluster as a language analysis result”.
Specifically, the rule generation unit 3 sets the threshold value of the variance value corresponding to the density information, and the rule generation unit 3 does not generate the pronunciation information of the accent phrase belonging to the cluster whose variance value is equal to or greater than the threshold value. Delete data. For example, assuming that the variance value of a 6-8 mora type 3 cluster is σA, and the variance value of a 10 mora or more type 8 cluster is σB, the rule generation unit 3 sets the threshold value σT of the variance value satisfying σA <σT <σB Set.
In this case, since the variance value of the 3 mora type 1 cluster is 0, the rule generation unit 3 does not correct the dictionary for the 3 mora type 1 accent phrase such as “I am” or “pillow”. Similarly, the rule generation unit 3 does not modify the dictionary for accent phrases belonging to 6-8 mora type 3 clusters such as “nuclear development (6 mora)”.
On the other hand, for an accent phrase belonging to a cluster of 10 mora or more and 8 types or more, such as “Albert Einstein Medical University (18 mora type 15)”, the rule generation unit 3 deletes the corresponding data from the dictionary and obtains the language analysis result Prevent output.
Or, when the language analysis dictionary 7 stores a score for language analysis, and the score calculation is used for language analysis, the rule generation unit 3 extremely reduces the score of the corresponding data so that the corresponding data is not selected. The language analysis dictionary 7 may be modified by replacing it with a value. Further, the rule generation unit 3 does not modify the language analysis dictionary 7 but can generate a speech synthesis rule by changing the language analysis unit 9 in the speech synthesis engine or an algorithm in the vicinity thereof. good.
The rule generation unit 3 outputs the speech synthesis rules that are the contents of the corrected language analysis dictionary 7 to the corrected language analysis dictionary 8.
The corrected language analysis dictionary 8 stores a speech synthesis rule that is the content of the language analysis dictionary 7 corrected by the rule generation unit 3 based on the above rules.
Next, a speech synthesis operation performed by inputting text will be described.
When a text to be subjected to speech synthesis is input, the language analysis unit 9 performs a language analysis process by morphological analysis or the like using the corrected language analysis dictionary 8 for the input text. The language analysis unit 9 generates pronunciation information from the result of the language analysis process, and outputs the pronunciation information to the prosody generation unit 10.
Next, the prosody generation unit 10 generates prosody information for the pronunciation information input from the language analysis unit 9 using the prosody generation model stored in the prosody generation model storage unit 6. The prosody generation unit 10 outputs the pronunciation information and the generated prosody information to the waveform generation unit 11.
The waveform generation unit 11 generates a speech waveform based on the pronunciation information and the prosody information generated by the prosody generation unit 10. The waveform generation unit 11 outputs the generated speech waveform as synthesized speech. The waveform may be generated based on a related technique, and the waveform may be generated by any method. The waveform generation unit 11 outputs the generated speech waveform as synthesized speech.
Next, referring to FIG. 6 and FIG. 7, the operation flow of the speech synthesis system 2000 is divided into two stages: a preparation stage for generating speech synthesis rules and prosody generation models, and a speech synthesis stage for actually performing speech synthesis processing. These will be described in order.
FIG. 6 is a flowchart illustrating an example of an operation of generating a speech synthesis rule in the preparation stage in the speech synthesis system 2000.
As shown in FIG. 6, the processing in steps S1 to S3 is the same as the processing in FIG.
After the process of S3, the rule generation unit 3 stores the speech synthesis rules, which are the contents of the corrected language analysis dictionary 7, in the corrected language analysis dictionary 8 (step S4).
FIG. 7 is a flowchart illustrating an example of an operation for creating a prosody generation model in the preparation stage in the speech synthesis system 2000.
The processing in step S1 is the same as the processing in FIGS.
After step S1, the prosody learning unit 5 learns the prosody model in the feature amount space divided by the feature amount space dividing unit 1, and creates a prosody generation model (step S2A).
Next, the prosody generation model storage unit 6 stores the prosody generation model created by the prosody learning unit 5 (step S3A).
Note that the processing in the preparation stage described in FIGS. 6 and 7 may be performed in the reverse order or in parallel.
FIG. 8 is a flowchart illustrating an example of the operation of the speech synthesis stage in which speech synthesis processing is actually performed in the speech synthesis system 2000.
As shown in FIG. 8, first, the language analysis unit 9 receives text to be speech synthesized (step S1B).
Next, the language analysis unit 9 performs language analysis processing on the input text according to the speech synthesis rules stored in the corrected language analysis dictionary 8 to generate pronunciation information (step S2B). The language analysis unit 9 outputs the generated pronunciation information to the prosody generation unit 10.
Next, the prosody generation unit 10 generates prosody information for the pronunciation information input from the language analysis unit 9 using the prosody generation model stored in the prosody generation model storage unit 6 (step S3B). The prosody generation unit 10 outputs the pronunciation information and the prosody information to the waveform generation unit 11.
Next, the waveform generation unit 11 generates a speech waveform based on the pronunciation information and the prosody information input from the prosody generation unit 10 (step S4B), and outputs the speech waveform as synthesized speech.
As described above, according to the speech synthesis system 2000 according to the present embodiment, disturbance of the F0 pattern due to insufficient learning data can be avoided, and speech synthesis with high naturalness can be performed. The reason is that prosody learning and extraction of sparse / dense information are performed based on the same clustering result, and the rule generation unit 3 generates a speech synthesis rule based on the sparse / dense information, so that the pronunciation data has sufficient pronunciation information. Is generated.
In this embodiment, the learning database is assumed to have collected the voice of one speaker. However, the learning database may be a collection of voices of a plurality of speakers. In the case of a database for learning a single speaker, there is an effect that it is possible to create a speech synthesis rule that can reproduce speaker characteristics such as a speaker's habit. In the case of a multi-speaker learning database, a general-purpose speech synthesis rule can be created.
<Third Embodiment>
Subsequently, a third embodiment of the present invention will be described.
FIG. 9 is a block diagram illustrating a configuration example of a speech synthesis system 3000 according to the third embodiment of the present invention.
Referring to FIG. 9, a speech synthesis system 3000 according to the third embodiment includes a speech synthesis learning device 21 and a speech synthesis device 41 in place of the speech synthesis learning device 20 and the speech synthesis device 40 according to the second embodiment. Further, a waveform generation model storage unit 12 is included. The speech synthesis system 3000 includes a speech synthesis dictionary 14 and a modified speech synthesis dictionary 15 in place of the language analysis dictionary 7 and the modified language analysis dictionary 8.
The speech synthesis learning device 21 includes an HMM learning unit 31 that generates a prosody generation model and a waveform generation model using the learning database 4 instead of the HMM learning unit 30. The HMM learning unit 31 further includes a waveform learning unit 12 in addition to the same configuration as the HMM learning unit 30.
The speech synthesizer 41 includes a waveform generation unit 17 that generates a waveform using the waveform generation model storage unit 16 instead of the waveform generation unit 10.
The waveform learning unit 12 learns the waveform model in the feature amount space divided by the feature amount space dividing unit 1 and creates a waveform generation model.
The waveform generation model is obtained by modeling the spectral feature amount of the waveform in the learning database. Specifically, the feature amount may be a cepstrum or the like. In the present embodiment, a model generated by the HMM is used as data for waveform generation. However, the speech synthesis method applied to the present invention is not limited to this, and another speech synthesis method, for example, a waveform connection method may be used. In this case, only the prosody generation model is learned by the HMM learning unit 31.
The waveform generation model storage unit 16 stores the waveform generation model created by the waveform learning unit 12.
The rule generation unit 13 generates a speech synthesis rule based on the density information of each cluster. Here, it is assumed that the rule generation unit 3 generates a speech synthesis rule by modifying the existing speech synthesis dictionary 14. Here, the speech synthesis dictionary 14 is used to add or change additional information for speech synthesis to the result of language analysis processing in addition to data and rules necessary for text language analysis processing. It is a dictionary that stores the rules.
The rule generation unit 13 corrects rules other than those relating to accent positions and accent phrase boundaries. Hereinafter, as a specific example, an operation in which the rule generation unit 13 corrects rules relating to “pause insertion / deletion” and “phrase change” will be described.
The rules relating to “insertion / deletion of poses” are rules such as “insert a pose at a natural position” and “delete a pose at an unnatural position” so that the sound becomes human-like. Specific rules include “one expiratory paragraph is N mora or less”, “pause after conjunction”, and the like.
Further, the rule relating to “change of wording” is a rule for changing a language analysis result generated from a standard text as a language to a wording unique to a speaker. For example, the word “broadcast” is usually read “hosoo”. However, some speakers may clearly read this as “Housou”. The rule representing this is a rule of “reading a long sound as a vowel”.
The correction of the speech synthesis dictionary 14 is performed according to the same policy as the correction of the language analysis dictionary 7 in the second embodiment. Specifically, a threshold value of the dispersion value is set. Then, the rule generation unit 13 deletes or adds a rule corresponding to the contents of the speech synthesis dictionary 14 so that an expression belonging to a cluster whose variance value is equal to or greater than a threshold value is not generated.
As a specific example, a case where a text “and broadcasting has started” is input will be described.
In the database 4 for learning, the voice of a speaker having the characteristics of “speak without pause” or “pronounce the word“ broadcast ”as“ hoso ”instead of“ hoso ”” ” Assume that waveform data is stored. In this case, when the feature space that is the learning data is divided, the cluster “pause after“ and ”and the cluster“ continuous vowels ”are very sparse or do not exist as clusters. Is done.
In this case, for example, as a modification of the rule relating to “pause insertion / deletion”, the rule generation unit 13 deletes the rule “insert a pose after a conjunction” from the rules stored in the speech synthesis dictionary 14. Alternatively, the rule generation unit 13 adds a rule “no pause after“ and ”” to the rule stored in the speech synthesis dictionary 14.
Further, as a modification of the rule regarding “change of wording”, the rule generation unit 13 is configured so that the text “broadcast” normally pronounced “hoso” is pronounced “hoso”. The generation unit 13 adds a rule “change a long sound to a vowel”.
The modified speech synthesis dictionary 15 stores the speech synthesis rules generated by the rule generation unit 13. Here, the speech synthesis rule generated by the rule generation unit 13 is a rule after the rule generation unit 13 corrects the rule stored in the existing speech synthesis dictionary 14 as described above.
Next, referring to the figure, the operation flow of the speech synthesis system 3000 is divided into two stages: a preparation stage for creating a speech synthesis rule, a prosody generation model and a waveform generation model, and a speech synthesis stage for actually performing speech synthesis processing These will be described in order.
First, in the preparation stage, the operation for creating the speech synthesis rules and the prosody generation model is the same as the operations shown in FIGS. 6 and 7 in the second embodiment, except that the speech synthesis rules to be generated are different. It is the same.
FIG. 10 is a flowchart illustrating an example of an operation of creating a waveform generation model in the preparation stage in the speech synthesis system 3000.
The processing in step S1 is the same as the processing in FIGS.
After step S1, the waveform learning unit 12 learns the waveform model in the feature amount space divided by the feature amount space dividing unit 1, and creates a waveform generation model (step S2C).
Next, the waveform generation model storage unit 16 stores the waveform generation model created by the waveform learning unit 12 (step S3C).
Note that the processing for creating the speech synthesis rules, the prosody generation model, and the waveform generation model in the preparation stage may be performed in any order or may be performed in parallel.
FIG. 11 is a flowchart illustrating an example of an operation in a speech synthesis stage in which speech synthesis processing is actually performed in the speech synthesis system 3000.
As shown in FIG. 11, the process in step S1B is the same as the process in FIG.
After step S1B, the language analysis unit 9 performs language analysis processing on the input text in accordance with the speech synthesis rules stored in the modified speech synthesis dictionary 15 to generate pronunciation information. When generating the pronunciation information, the language analysis unit 9 gives additional information such as “changes the long sound to a vowel” according to the rules stored in the corrected speech synthesis dictionary 15 in the pronunciation information (step S2D). The language analysis unit 9 outputs the pronunciation information to which the additional information is given to the prosody generation unit 10.
The process of step S3B is the same as the process in FIG.
Next, the waveform generation unit 17 generates a speech waveform using the waveform generation model stored in the waveform generation model storage unit 16 based on the pronunciation information and prosodic information input from the prosody generation unit 10 (step S4D). ). The waveform generation unit 17 outputs the speech waveform as synthesized speech.
As described above, according to the speech synthesis system 3000 according to the present embodiment, the modified speech synthesis dictionary 15 adds the corrected additional information to the pronunciation information, and thus faithfully reproduces a feature such as a song for each speaker. it can. Further, according to the present embodiment, when a waveform is generated with a waveform generation model belonging to a sparse cluster by using the same clustering result for waveform learning and extraction of sparse and dense information used for correcting pronunciation information, The problem that the sound quality deteriorates only in that part can be avoided.
Note that, even in a waveform connection method that does not use an HMM for waveform generation, data belonging to a cluster in which learning data is sparse lacks the data amount of the corresponding unit waveform. Therefore, according to this embodiment, even when the waveform connection method or the like is used, it is possible to avoid deterioration in sound quality in that data belonging to a sparse cluster is not used.
As mentioned above, although this invention was demonstrated with reference to each embodiment, this invention is not limited to the above embodiment. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. For example, the speech synthesis system according to each embodiment may store the extracted density information in a database (not shown) and use it appropriately with reference to a correspondence table or the like.
FIG. 12 is a block diagram illustrating an example of a hardware configuration that implements the speech synthesis system 2000 according to the second embodiment. Although the second embodiment will be described as an example here, a speech synthesis system according to another embodiment may be realized by a similar hardware configuration.
As shown in FIG. 12, each unit constituting the speech synthesis system 2000 includes a CPU (Central Processing Unit) 100, a communication IF (interface) 200 for network connection, a memory 300, and a storage such as a hard disk for storing a program. It is realized by a computer device including a device 400, an input device 500, and an output device 600. However, the configuration of the speech synthesis system 2000 is not limited to the computer apparatus shown in FIG.
The CPU 100 controls the entire speech synthesis system 2000 by operating an operating system. Further, the CPU 100 reads programs and data from a recording medium mounted on, for example, a drive device to the memory 300, and executes various processes according to the programs and data.
The recording device 400 is, for example, an optical disk, a flexible disk, a magnetic optical disk, an external hard disk, a semiconductor memory, etc., and records a computer program so that it can be read by a computer. The storage device 400 may be, for example, the learning database 4 or the prosody generation model storage unit 6. The computer program may be downloaded from an external computer (not shown) connected to the communication network.
The input device 500 receives input text from the user in the speech learning device 40, for example. The output device 600 outputs the synthesized speech that is finally generated.
In addition, the block diagram utilized in each embodiment described so far has shown the block of a functional unit instead of the structure of a hardware unit. Further, the means for realizing the constituent parts of the speech synthesis system 2000 is not particularly limited. That is, the speech synthesis system 2000 may be realized by one physically coupled device, or may be realized by connecting two or more physically separated devices in a wired or wireless manner and by a plurality of these devices. Also good. In that case, the two physically separated devices may be used as the speech synthesis learning device 20 and the speech synthesis device 40, respectively.
The program of the present invention may be a program that causes a computer to execute the operations described in the above embodiments.
In each of the above embodiments, characteristic configurations of a speech synthesis system, a speech synthesis method, and a speech synthesis program as described below are shown.
(Appendix 1)
A learning database that stores learning data that is a set of feature values extracted from speech waveform data;
A feature amount space dividing means for dividing a feature amount space, which is a space related to learning data stored in the learning database, into subspaces;
A sparse / dense state detecting unit that detects a sparse / dense state for each partial space that is a feature amount space divided by the feature amount space dividing unit, and generates and outputs sparse / dense information that is information indicating the sparse / dense state;
Rule generating means for generating a speech synthesis rule, which is a rule for generating pronunciation information used for speech synthesis, based on the density information output from the density state detecting means;
Speech synthesis system including
(Appendix 2)
Prosody learning means for learning a prosodic model and creating a prosody generation model in a subspace which is a feature amount space divided by the feature amount space dividing means;
Prosody generation model storage means for storing the prosody generation model created by the prosody learning means;
Prosody generation means for generating prosody information using the prosody generation model stored in the prosody generation model storage means for the pronunciation information generated according to the speech synthesis rules generated by the rule generation means;
The speech synthesis system according to claim 1, further comprising:
(Appendix 3)
It further includes a dictionary for storing rules necessary for text language analysis processing,
The rule generation means generates a rule for speech synthesis by correcting a rule stored in the dictionary.
The speech synthesis system according to appendix 1 or 2.
(Appendix 4)
A modified dictionary that stores the modified rules generated by the rule generating means as rules for speech synthesis;
Language analysis means for receiving input of text, generating pronunciation information based on the speech synthesis rules stored in the correction dictionary from the text, and outputting the pronunciation information to the prosody generation means;
The speech synthesis system according to supplementary note 3, further comprising:
(Appendix 5)
The rule generating means corrects the speech synthesis rule by deleting the data of the accent phrase determined to belong to a sparse partial space based on the sparse / dense information;
The speech synthesis system according to attachment 4.
(Appendix 6)
The rule generation means corrects a speech synthesis rule related to a pause insertion position or an input text phrase,
The speech synthesis system according to any one of appendices 3 to 5.
(Appendix 7)
The feature amount space dividing means divides the feature amount space into partial spaces by binary tree structure clustering based on the information amount.
The speech synthesis system according to any one of appendices 1 to 6.
(Appendix 8)
The prosody learning means performs learning of the prosody model by HMM learning.
The speech synthesis system according to any one of appendices 2 to 7.
(Appendix 9)
Waveform learning means for learning a waveform model and creating a waveform generation model in a partial space that is a feature amount space divided by the feature amount space dividing means;
Waveform generation model storage means for storing the waveform generation model created by the waveform learning means;
From the prosody information generated by the prosody generation unit, a waveform generation unit that generates a speech waveform using a waveform generation model stored in the waveform generation model storage unit and outputs the generated speech waveform as synthesized speech;
The speech synthesis system according to any one of appendices 1 to 8, further including:
(Appendix 10)
Stores learning data that is a set of features extracted from speech waveform data,
Dividing the feature amount space, which is a space related to the learning data to be stored, into subspaces;
Detects a sparse / dense state for each partial space that is the divided feature amount space, generates and outputs sparse / dense information that is information indicating the sparse / dense state,
Generate a speech synthesis rule that is a rule for generating pronunciation information used for speech synthesis based on the output density information.
Speech synthesis method.
(Appendix 11)
Stores learning data that is a set of features extracted from speech waveform data,
Dividing the feature amount space, which is a space related to the learning data to be stored, into subspaces;
Detects a sparse / dense state for each partial space that is the divided feature amount space, generates and outputs sparse / dense information that is information indicating the sparse / dense state,
Generate a speech synthesis rule that is a rule for generating pronunciation information used for speech synthesis based on the output density information.
A recording medium for storing a program that causes a computer to execute processing.
Although the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.
This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2011-035543 for which it applied on February 22, 2011, and takes in those the indications of all here.

As described above, the present invention can be suitably applied when constructing a speech synthesis system using learning data with a limited amount of information. For example, the present invention is suitably applied to a reading system for general text such as news articles and automatic response sentences.

DESCRIPTION OF SYMBOLS 1 Feature-value space division part 2 Density information extraction part 3, 13 Rule generation part 4 Learning database 5 Prosody learning part 6 Prosody generation model storage part 7 Language analysis dictionary 8 Modified language analysis dictionary 9 Language analysis part 10 Prosody generation part DESCRIPTION OF SYMBOLS 11, 17 Waveform production | generation part 12 Waveform learning part 14 Dictionary for speech synthesis 15 Dictionary for correction speech synthesis 16 Waveform generation model storage part 17 Waveform generation part 20, 21 Speech synthesis learning apparatus 30, 31 HMM learning part 40, 41 Speech synthesis apparatus 100 CPU
200 Communication IF
300 memory 400 storage device 500 input device 600 output device 1000, 2000, 3000 speech synthesis system

Claims (10)

A learning database that stores learning data that is a set of feature values extracted from speech waveform data;
A feature amount space dividing means for dividing a feature amount space, which is a space related to learning data stored in the learning database, into subspaces;
A sparse / dense state detecting unit that detects a sparse / dense state for each partial space that is a feature amount space divided by the feature amount space dividing unit, and generates and outputs sparse / dense information that is information indicating the sparse / dense state;
Rule generating means for generating a speech synthesis rule, which is a rule for generating pronunciation information used for speech synthesis, based on the density information output from the density state detecting means;
Speech synthesis system including Prosody learning means for learning a prosodic model and creating a prosody generation model in a subspace which is a feature amount space divided by the feature amount space dividing means;
Prosody generation model storage means for storing the prosody generation model created by the prosody learning means;
Prosody generation means for generating prosody information using the prosody generation model stored in the prosody generation model storage means for the pronunciation information generated according to the speech synthesis rules generated by the rule generation means;
The speech synthesis system according to claim 1, further comprising: It further includes a dictionary for storing rules necessary for text language analysis processing,
The rule generation means generates a rule for speech synthesis by correcting a rule stored in the dictionary.
The speech synthesis system according to claim 1 or 2. A modified dictionary that stores the modified rules generated by the rule generating means as rules for speech synthesis;
Language analysis means for receiving input of text, generating pronunciation information based on the speech synthesis rules stored in the correction dictionary from the text, and outputting the pronunciation information to the prosody generation means;
The speech synthesis system according to claim 3, further comprising: The rule generating means corrects the speech synthesis rule by deleting the data of the accent phrase determined to belong to a sparse partial space based on the sparse / dense information;
The speech synthesis system according to claim 4. The rule generation means corrects a speech synthesis rule related to a pause insertion position or an input text phrase,
The speech synthesis system according to any one of claims 3 to 5. The feature amount space dividing means divides the feature amount space into partial spaces by binary tree structure clustering based on the information amount.
The speech synthesis system according to any one of claims 1 to 6. The prosody learning means performs learning of the prosody model by HMM learning.
The speech synthesis system according to any one of claims 2 to 7. Stores learning data that is a set of features extracted from speech waveform data,
Dividing the feature amount space, which is a space related to the learning data to be stored, into subspaces;
Detects a sparse / dense state for each partial space that is the divided feature amount space, generates and outputs sparse / dense information that is information indicating the sparse / dense state,
Generate a speech synthesis rule that is a rule for generating pronunciation information used for speech synthesis based on the output density information.
Speech synthesis method. Stores learning data that is a set of features extracted from speech waveform data,
Dividing the feature amount space, which is a space related to the learning data to be stored, into subspaces;
Detects a sparse / dense state for each partial space that is the divided feature amount space, generates and outputs sparse / dense information that is information indicating the sparse / dense state,
Generate a speech synthesis rule that is a rule for generating pronunciation information used for speech synthesis based on the output density information.
A recording medium for storing a program that causes a computer to execute processing.

Images