JP2008158029A - Distributed data base system for voice synthesis - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a distributed data base system for voice synthesis capable of integrating and dealing with the data base, when the data base required for voice synthesis is distributed. <P>SOLUTION: The system includes: one or more voice servers in which a speaker and data required for synthesizing voice characterized by speaking tone, are stored; and a management server 10 for holding a management index 12 which each voice server stores. The management server 10 holds a voice file characteristic table 12b which expresses the speaker and the speaking tone of a voice file which is a base for generating data specified by the management index 12, by a keyword for showing their characteristics, in the management index 12. When request for designating the data required for performing voice synthesis by the keyword is received, the management index 12 is searched and information for showing corresponding data location is returned. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a speech synthesis distributed database system that stores data necessary for speech synthesis in a distributed manner.

Conventionally, there is a corpus-based speech synthesis method as one method for performing speech synthesis (Non-Patent Document 1).
In addition, “in a text-to-speech synthesis technology that synthesizes speech from text, optimum unit selection-type speech synthesis can be performed by a terminal device with relatively small calculation power. As a technology for the purpose, "In text-to-speech synthesis that synthesizes speech from text, the result of segment selection processing is output as secondary content for content generation and output. And the voice waveform synthesis processing with a light load. As a result, the segment selection process is performed on the server side, and the used segment information is transmitted to the terminal to be combined data. (Patent Document 1).
JP 2006-18133 A (summary) IEICE, IEICE Technical Report, SP2005-18, pp. 37-41 (2005. 5)

In a speech synthesizer as described above, data necessary for performing speech synthesis is usually stored in a device that performs synthesis processing or a storage device that is directly connected to the speech synthesizer. It is.
However, due to various circumstances such as various restrictions and administrative circumstances, the data necessary for performing speech synthesis is not necessarily integrated into one place, and a plurality of storage devices or servers May be distributed and stored. Furthermore, the data stored in a distributed manner may increase the amount of data as it is in each storage location and grow as an independent database.
The prior art described in Patent Document 1 described above functionally distributes the unit selection process and the waveform synthesis process, but does not support data distribution. Further, it goes without saying that a conventional speech synthesizer or the like does not consider such distributed data.
For this reason, there is a demand for a distributed database system for speech synthesis that can handle a database that is necessary for performing speech synthesis in a distributed manner.

A distributed database system for speech synthesis according to the present invention includes:
One or more voice servers that store the data necessary to synthesize speech characterized by speaker and tone;
A management server that holds an index of data stored in each voice server;
Have
The management server
A list that further expresses the speaker and tone of the voice file that is the basis for generating the data specified by the index, using keywords that express the characteristics;
When a request is received that specifies the data necessary for speech synthesis using the keyword,
The index and the list are searched, and information indicating the location of the corresponding data is returned.

  According to the distributed database system for speech synthesis according to the present invention, the location of each voice data and the like stored in the voice server is managed by the management server. It becomes possible to generate new audio data or the like by freely searching or combining them.

Embodiment 1 FIG.
FIG. 1 shows a schematic configuration of a distributed database system for speech synthesis according to Embodiment 1 of the present invention.
The distributed database system for speech synthesis according to the first embodiment stores speech servers 20, 21,... (Not shown below) that store data necessary to perform speech synthesis, and stores these speech servers. The management server 10 holds an index of the data being stored.
The management server 10 and the voice servers 20 and 21 are connected via the network 30.
Further, the management server 10 and the client device 1 are connected via the network 40. The client device 1 issues a search request and a generation request to the management server 10. Details of each request will be described with reference to FIGS.

The voice servers 20, 21 store data necessary for voice synthesis using a corpus-based voice synthesis method. The data here includes prosodic model data, acoustic model data, and audio files. Each content is described below.
(1) Prosodic model data This is data obtained by statistically modeling prosodic features such as pitch and sound length related to a speaker's wording in units of predetermined labels.
(2) Acoustic model data Data obtained by statistically modeling acoustic characteristics of a speaker's voice caused by the shape of the speaker's vocal tract and the like in predetermined label units.
(3) Audio file A file that stores audio actually uttered by a speaker, and constitutes one audio file in units of predetermined labels. In addition, an audio file obtained by reading a sentence aloud may be used instead of a label unit.
(4) Parameters Due to the implementation of the voice server, various calculated parameters related to the voice file may be stored together with the voice file. This parameter is used when (3) voice synthesis is performed using the voice file.
(5) Segment selection data Data required when selecting a segment waveform in the corpus-based speech synthesis method. Stores various calculated parameters associated with the audio file.

  “Statistical modeling” as described above refers to creating a statistical model necessary for speech synthesis processing by a technique such as a hidden Markov model, but the modeling technique is not limited to this and is unified. It is sufficient that it is statistically modeled by the above method.

For these files, one voice server may store all types of data (1) to (5), or may store only some types of files. In addition, some are configured as a single data file, and some are configured as a database with a plurality of data.
In the first embodiment, the following description will be given on the assumption that each is databased. Further, when simply referred to as “voice database”, the above (1) to (5) are generically indicated.
In the first embodiment, the “prosodic model database” and the “acoustic model database” refer to the above (1) to (2), and the “voice file database” includes the above (3) and (5). Point to. For the sake of simplicity, (3) the audio file and (4) parameters are not particularly distinguished, and “audio file” refers to both.

The management server 10 includes a speaker search unit 11, a management index 12, and a DB generation unit 14.
The management index 12 holds an index of data stored in the voice servers 20 and 21 on a table described later with reference to FIGS. By referring to this index, the location of data stored in each voice server can be known.
The management index 12 can be configured by a data file or the like stored in a storage device such as an HDD (Hard Disk Drive).
The speaker search unit 11 receives a request for searching for the location of data necessary for performing speech synthesis via the network 40, searches a table to be described later stored in the management index 12, and obtains the search result. Reply to the request issuer. Details will be described later with reference to FIGS.
The DB generation unit 14 accepts a request to generate prosodic model data, acoustic model data, or segment selection data using the audio file stored in the audio server 20, 21 or the like via the network 40. Execute the generation process. After the generation is completed, an index indicating the location of the generated data is stored in a table held by the management index 12.

The client device 1 includes a speaker selection unit 2 and a DB generation request unit 3.
The speaker selection unit 2 issues a request for searching for the location of data necessary for speech synthesis to the speaker search unit 11 via the network 40.
The DB generation request unit 3 sends the prosody model data, the acoustic model data, or the unit selection data to the DB generation unit 14 via the network 40 using the audio file stored in the audio server 20, 21 or the like. Issue a request to generate.

  Next, each table stored in the management index 12 will be described.

FIG. 2 shows a configuration of the database location table 12a stored in the management index 12 and an example of data.
The database location table 12a includes a “database number” column, a “server” column, a “DB type” column, and a “database name” column.
The “database number” column stores a unique number for identifying the voice database stored in the voice server 20, 21 or the like. The values in this column are uniquely numbered in the distributed database system for speech synthesis.
The “server” column stores the address of the server that stores the voice database specified by the value in the “database number” column. The value in this column can be in any format as long as it is information that can identify the location of the server, such as the server IP address and server name.
In the “DB type” column, the type of the speech database specified by the value of the “database number” column is any of the above-mentioned (1) prosodic model data to (3) speech file or (5) segment selection data. Information indicating whether or not is stored.
The “database name” column stores the name of the speech database specified by the value of the “database number” column. The values in this column need not be unique.

Next, the meaning represented by the data example in FIG. 2 will be described.
The data in the first to fourth lines represent data stored in the voice server 20 of FIG. For example, the data in the first row indicates that the server “SV20” stores a “prosodic model” database named “20A” and the database number is “1001”.
The data on the 5th to 7th lines represent the data stored in the voice server 21 of FIG.
The data on the 8th to 9th lines are not shown in FIG. 1, but represent the data stored in the server “SV22”. This shows that the server “SV22” stores “prosodic model 22A” and “voice file 22C”.
The data on the 10th line is not shown in FIG. 1, but represents data stored in the server “SV23”. According to this, it can be seen that the server “SV23” stores the “voice file 23C”.

FIG. 3 shows the configuration and data example of the audio file feature table 12b stored in the management index 12.
The voice file feature table 12b includes a “voice file number” column, a “speaker / tone” column, and a “feature keyword” column.
In the “voice file number” column, the value of the “database type” column of the “database type” in the database location table 12a that is “voice file” is stored.
The “speaker / tone” column stores information related to the speaker / tone of the audio file specified by the value of the “voice file number” column. In the data example of FIG. 3, it can be seen that the audio file specified by “audio file number = 1003” is an audio file uttered by “speaker A / tone A”.
In the “feature keyword” column, a keyword representing the feature of the audio file specified by the value of the “audio file number” column is stored. In the data example of FIG. 3, it can be seen that the audio file specified by “audio file number = 1003” is an audio file uttered with contents characterized by “male, grandfather”. Whether the elderly man actually spoke is another matter.

FIG. 4 shows a configuration and data example of the database generation source table 12c stored in the management index 12.
The database generation source table 12c has a “database number” column and an “audio file number” column.
In the “database number” column, a value corresponding to a value other than “sound file” in the “DB type” column in the “database number” of the database location table 12a is stored.
In the “audio file number” column, a value corresponding to the value of the “audio file number” column of the audio file feature table 12b is stored.

Next, the meaning represented by the data example in FIG. 4 will be described.
The database generation source table 12c means that the audio database specified by the “database number” column is generated using the audio file specified by the “audio file number” column.
For example, the data in the first row is an audio database (prosodic model 20A on SV20) specified by “database number = 1001”, an audio file specified by “audio file number = 1003” (audio file 20C of SV20 article) ).
As described above, the information about the creation source of the voice database is necessary because the quality of the voice database differs depending on which voice file is used to create the voice database.

When performing speech synthesis, a general user may not be interested in “from which speech file the prosody model 20A was created”, whereas a user who is familiar with speech synthesis has a higher For the purpose of obtaining quality synthesized speech, it may be desired to know in advance how much quality each speech database has. Also, for those who are engaged in research and development of speech synthesis, it is useful for business to keep track of the origin of each speech database.
In order to respond to such needs, in the present invention, information representing the location of each voice database is not managed by an index, but the origin is also managed, and any voice database is used for voice synthesis when performing voice synthesis. It was made possible to select whether to perform.

Next, the operation of the distributed database system for speech synthesis according to the first embodiment will be described.
By using the data held in each table described with reference to FIGS. 2 to 4, various searches and generation of a new voice database are possible. Hereinafter, specific operations will be described by taking several processes as examples.

FIG. 5 is a sequence diagram when a keyword representing the characteristics of an audio file is transmitted from the client terminal 1 to search for the corresponding audio file. The data held in each table is as shown in FIGS.
Hereinafter, each step will be described. In FIG. 5, step numbers are not shown for convenience of description, but it is assumed that a time series flows from top to bottom. The same applies to FIGS. 6 and 7.

(1) The user operates the client terminal 1 to input the keyword “female” and execute a search request.
(2) The speaker selection unit 2 issues a search request including the keyword “female” to the management server 10. The search request reaches the management server 10 via the network 40.
(3) The search request that reaches the management server 10 is processed by the speaker search unit 11. The speaker search unit 11 searches the audio file feature table 12b using “female” as a key.
(4) As a result of the search, data including “female” in the value of the “characteristic keyword” column is hit. In the data example of FIG. 3, the data in the third and fourth rows correspond.
(5) The speaker search unit 11 acquires the values of the “voice file number” column and “speaker / tone” column of the data in the third and fourth rows as the search results.
(6) The speaker search unit 11 returns the acquired values of the “voice file number” column and the “speaker / tone” column to the speaker selection unit 2.

  FIG. 6 is a sequence diagram when the search result of FIG. 5 is obtained and a speaker / tone is specified based on the search result and a voice database generation request is made. Hereinafter, each step will be described.

(1) The user operates the client terminal 1 and designates “speaker B / tone B” among the search results obtained in step (6) of FIG. Generation request of “acoustic model database”.
Here, the reason for letting the user select the “speaker / tone” pair is simply that the user wants the set of “speaker / tone”, as described above with reference to FIG. This is because the quality of the generated acoustic model database differs depending on which audio file is used.
(2) The DB generation request unit 3 issues a DB generation request to the management server 10 using “speaker B / tone B”, “acoustic model”, and “voice file number = 1008” as keys. The DB generation request reaches the management server 10 via the network 40.
(3) The search request that reaches the management server 10 is processed by the DB generation unit 14. The DB generation unit 14 searches the database generation source table 12c using “voice file number = 1008” as a key, and confirms the presence or absence of a generated database.
(4) As a result of the search, data having a value of “1008” in the “voice file number” column is hit. In the data example of FIG. 4, the data in the sixth row corresponds.
(5) The DB generation unit 14 acquires the value “1007” in the “database number” column of the data in the sixth row as a search result.
(6) The DB generation unit 14 searches the database location table 12a using “database number = 1007” as a key, and acquires the type of the voice database.
(7) As a result of the search, it is understood that the speech database specified by “database number = 1007” is a “prosodic model” database. That is, it can be seen that there is no acoustic model database generated using the audio file specified by “audio file number = 1008”.
(8) As described above, since the acoustic model database generated using the audio file specified by “audio file number = 1008” does not exist, the DB generation unit 14 indicates that the generation process is to be performed. Reply to 3. Since the creation of the acoustic model database is a time-consuming process, the client terminal 1 ends the process without waiting for a further reply.
(9) The DB generation unit 14 starts the generation process of the acoustic model database using the audio file specified by “audio file number = 1008”. Here, it is assumed that the file is generated on the server “SV22” where the audio file “1008” exists.
(10) When the generation of the acoustic model database is completed, the DB generation unit 14 registers information indicating the location in the next steps (11) and (12).
(11) The DB generation unit 14 registers a new entry representing the generated acoustic model database in the database location table 12a. In this case, the value of each column is “database number = 1010”, “server = SV22”, “DB type = acoustic model”, “database name = 22B”.
(12) The DB generation unit 14 registers a new entry representing the generation source of the generated acoustic model database in the database generation table 12c. In this case, the value of each column is “database number = 1010” and “sound file number = 1008”.

  If the target voice database for which the generation request has been issued has already been created, this is returned to the DB generation request unit 3 in step (8), and the subsequent steps are omitted. The same applies to the next FIG.

  FIG. 7 is a sequence diagram when the requests of FIGS. 5 and 6 are collectively executed. Hereinafter, each step will be described.

(1) The user operates the client terminal 1, inputs the keyword “female, joy”, and executes the acoustic model database generation request.
(2) The DB generation request unit 3 issues a DB generation request to the management server 10 using “female, joy” and “prosodic model” as keys. The DB generation request reaches the management server 10 via the network 40.
(3) The search request that reaches the management server 10 is processed by the DB generation unit 14. The DB generation unit 14 searches the audio file feature table 12b using “female, joy” as a key.
(4) As a result of the search, data including “female, joy” in the value of the “characteristic keyword” column is hit. In the data example of FIG. 3, the data in the fourth row corresponds.
(5) The DB generation unit 14 acquires values of the “voice file number” column and the “speaker / tone” column of the data in the fourth row as a search result.
(6) The DB generation unit 14 searches the database generation source table 12c using “voice file number = 11009” as a key, and confirms the presence or absence of the generated database.
(7) As a result of the search, data having a value of “1009” in the “voice file number” column is hit. In the data example of FIG. 4, there is no corresponding data.
(8) The DB generation unit 14 acquires a “null” value indicating that the corresponding data does not exist as a search result.
(9) Since the following processing is the same as steps (8) to (12) in FIG. 6, an outline sequence of the processing is shown in FIG. 7, and description thereof is omitted.

  In the above description, an example in which a prosodic model or an acoustic model is generated from an audio file has been described, but a new audio file can also be generated from an audio file. The processing in this case is the same as described above.

In the first embodiment, the table configuration in which speakers / tones of voice files and feature keywords are associated with each other has been described with reference to FIG. 3, but this is for explaining the generation of prosodic models and acoustic models from voice files. For convenience.
Similarly, for the prosody model and the acoustic model, a table that holds information representing features may be configured to facilitate the search.

  The table configurations described with reference to FIGS. 2 to 4 have been normalized to some extent, but the column configuration is not limited to this, and the contents of requests issued to the distributed database system for speech synthesis. Depending on the search efficiency, the column structure may be changed as appropriate, or normalization may be broken.

  Further, since the present invention relates to a database system that stores data necessary for performing speech synthesis in a distributed manner, the execution of speech synthesis is not mentioned, but the function of performing speech synthesis is the speech server 20, 21. Etc., or the management server 10 or the client terminal 1 may be provided.

In FIG. 2, the location of the voice server is indicated by the value of the “server” column, but the file path of the voice database in the voice server is not shown at all.
About the file path etc. in each audio | voice server, each audio | voice server itself is good also as managing on its own responsibility, and the management server 10 may provide a more detailed index. Which one to select may be appropriately designed.
In general, it seems that it is desirable to manage the file path and the like in the voice server by the voice server itself and hide it from outside the voice server.

As described above, according to the first embodiment, since the management is performed using the management index 12 indicating the location of each voice database stored in the voice servers 20, 21, etc., the voice database is configured in a distributed manner. Even in such a case, it is possible to freely search or combine them to generate a new voice database.
In addition, since it manages which voice file each voice database is generated, the quality of these voice databases can be estimated to some extent, and the user can select which voice database to use for voice synthesis It was possible.

Embodiment 2. FIG.
In the first embodiment, the processing at the time of requesting generation of a voice database mainly from the client terminal 1 has been described.
In the second embodiment of the present invention, an example will be described in which the client terminal 1 includes the speech synthesizer 4 and performs speech synthesis using a speech database acquired from the speech servers 20, 21 and the like.

FIG. 8 shows a schematic configuration of the distributed database system for speech synthesis according to the second embodiment.
Although the configuration of the distributed database system for speech synthesis itself is not changed from that in FIG. 1 described in the first embodiment, the client terminal 1 is newly provided with a speech synthesizer 4.
The client terminal 1 inquires the management server 10 about the location of the speech database, issues a generation request if it does not exist, and performs speech synthesis by the speech synthesizer 4 using the acquired speech database.

FIG. 9 is a sequence diagram when transmitting a keyword representing the characteristics of an audio file from the client terminal 1 and acquiring a prosodic model database generated using the corresponding audio file. The data held in each table is as shown in FIGS.
Hereinafter, each step will be described.

(1) The user operates the client terminal 1, inputs the keyword “female”, and executes a prosodic model database acquisition request.
(2) The speaker selection unit 2 issues a prosodic model database acquisition request including the keyword “female” to the management server 10. The acquisition request reaches the management server 10 via the network 40.
(3) The search request that reaches the management server 10 is processed by the speaker search unit 11. The speaker search unit 11 searches the audio file feature table 12b using “female” as a key.
(4) As a result of the search, data including “female” in the value of the “characteristic keyword” column is hit. In the data example of FIG. 3, the data in the third and fourth rows correspond.
(5) The speaker search unit 11 searches the database generation source table 12c using “voice file number = 1008 or 1009” as a key, and confirms the presence or absence of a generated database.
(6) As a result of the search, data having a value of “1008” or “1009” in the “voice file number” column is hit. In the data example of FIG. 4, the data in the sixth row corresponds.
(7) The speaker search unit 11 acquires the value “1007” in the “database number” column of the data in the sixth row as the search result.
(8) The speaker search unit 11 searches the database location table 12a using “database number = 1007” as a key, and acquires the type of the voice database.
(9) As a result of the search, it is understood that the speech database specified by “database number = 1007” is a “prosodic model” database.
(10) The speaker search unit 11 selects the value “SV22” in the “server” column of the voice database specified by “database number = 1007” and the voice file number “1008” that is the source of the selection. Reply to part 2.
(11) The speech synthesis unit 4 accesses the server “SV22” and acquires the prosody model 22A.

If a prosodic model corresponding to the keyword “female” is not found, a new prosody model may be generated by the same processing procedure as that described in FIG.
Further, it is found in step (10) that the prosodic model 22A has been generated using the voice file “1008”, but when it should be generated using another voice file, the search is performed using another keyword. Just re-execute.
When actually performing speech synthesis, it is necessary to acquire all of the prosodic model, acoustic model, speech file, and segment selection data.

  Although not described in the first and second embodiments, there may be a request for simply browsing the contents of each table. For example, a request such as “obtaining a list of voice databases stored in the server SV22” is also possible.

As described above, according to the second embodiment, since the client terminal 1 includes the speech synthesizer 4, the client terminal 1 can acquire each type of speech database and perform speech synthesis.
This is an effective implementation when it is desired to distribute speech synthesis processing to the client side.

  In the first and second embodiments, the initial data of each table may be set as appropriate by an administrator or the like. Subsequent data sets are automatically performed by the DB generation unit 14, so that subsequent management is unnecessary.

1 shows a schematic configuration of a distributed database system for speech synthesis according to Embodiment 1. The structure and data example of the database location table 12a stored in the management index 12 are shown. The structure and data example of the audio file feature table 12b stored in the management index 12 are shown. The structure and data example of the database generation source table 12c stored in the management index 12 are shown. It is a sequence diagram at the time of transmitting the keyword showing the characteristic of an audio | voice file from the client terminal 1, and searching an applicable audio | voice file. FIG. 6 is a sequence diagram when a voice database generation request is made by designating a speaker / tone according to the search result after obtaining the search result of FIG. 5. FIG. 7 is a sequence diagram when executing the requests of FIGS. 5 and 6 in a batch. Hereinafter, each step will be described. 4 shows a schematic configuration of a distributed database system for speech synthesis according to Embodiment 2. It is a sequence diagram at the time of transmitting the keyword showing the characteristic of an audio | voice file, and acquiring the prosodic model database produced | generated using the applicable audio | voice file.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Client apparatus, 2 Speaker selection part, 3 DB production | generation request part, 4 Speech synthesis part, 10 Management server, 11 Speaker search part, 12 Management index, 12a Database location table, 12b Voice file characteristic table, 12c Database generation source Table, 14 DB generation unit, 20 voice server, 21 voice server, 30 network, 40 network.

Claims (4)

One or more voice servers that store the data necessary to synthesize speech characterized by speaker and tone;
A management server that holds an index of data stored in each voice server;
Have
The management server
A list that further expresses the speaker and tone of the voice file that is the basis for generating the data specified by the index, using keywords that express the characteristics;
When receiving a request specifying the data necessary for speech synthesis by the keyword,
The distributed database system for speech synthesis, wherein the index and the list are searched and information indicating the location of the corresponding data is returned. Each of the voice servers
One or more of a prosody model database, an acoustic model database, or a speech file database is stored as the data necessary for speech synthesis,
The management server
Information representing whether each of the sound servers stores the prosodic model database, the acoustic model database, or the sound file database is held in the index,
When returning information indicating the location of the database stored in the voice server,
The distributed database system for speech synthesis according to claim 1, wherein information indicating which of the databases is stored in the speech server is also returned. The management server
Information indicating whether the prosodic model database, the acoustic model database, and the voice file database are generated based on the voice file on which voice server is held in the index,
When returning information indicating the location of the database stored in the voice server,
The distributed database for speech synthesis according to claim 2, wherein the database stored in the speech server returns a reply together with information indicating which speech server has generated the speech file. system. The management server
When receiving a request to specify the database necessary for speech synthesis by the keyword,
If the corresponding database does not exist even after searching the index and the list,
4. The distributed database for speech synthesis according to claim 3, wherein a speech file necessary for generating the database is searched from the list, and the requested database is generated using the corresponding speech file. system.

Images