WO2024247848A1 - Information processing device, information processing method, program, and information processing system - Google Patents

Abstract

This information processing device is provided with: a speaker search unit that performs a search based on the topic of text data, to select speaker data from among a plurality of sets of speaker data; and a speech synthesizing unit which generates synthesized speech data relating to the text data on the basis of the speaker data selected by the speaker search unit.

Description

Information processing device, information processing method, program, and information processing system

This technology relates to information processing devices, information processing methods, programs, and information processing systems, and relates to voice synthesis processing used for, for example, narration and dialogue in content.

In recent years, the number of contents using text-to-speech (TTS) narration has been increasing in distribution services. The speech synthesis software and speech synthesis services used provide multiple TTS speakers, allowing users to select the voice quality that best suits their content.
The following Patent Document 1 discloses a technique for selecting a male or female voice by tallying up the frequency of occurrence of each word used in a sentence to be read aloud in male and female sentences.

Japanese Patent Application Publication No. 11-296193

When the number of TTS speakers offered is small, users can listen to sample voices from all TTS speakers to choose a voice that suits their content, but this means that there are fewer options. On the other hand, if more TTS speakers are offered, it will take more time for users to listen to sample voices from all the speakers. Also, when there are voices with similar qualities, users may be unsure which voice suits their content.

To make it easier for users to select a voice, there are methods for distinguishing between voices based on the high and low frequency of the TTS speaker's voice, methods for distinguishing between voices based on the gender or age of the TTS speaker, and methods for distinguishing between voices based on the image of the TTS speaker's voice (for example, "mature voice" or "energetic voice").
However, even if a speaker is selected as "a male with a medium voice frequency," the desired voice quality is likely to differ depending on whether the content is a documentary program or a sports program.
In addition, the technology of Patent Document 1 limits the text to be read aloud to "spoken language." Therefore, it is still insufficient to suggest a voice according to the content.

This technology makes it possible to more accurately provide users with the audio that is most suited to the content from among the available audio options.

The information processing device according to the present technology includes a speaker search unit that performs a search based on a topic of text data to select speaker data from among a plurality of speaker data, and a voice synthesis unit that generates synthetic voice data of the text data using the speaker data selected by the speaker search unit.
Speaker data with a voice quality that corresponds to the topic (topic or genre) of the text data is selected, and the reading voice of the text data is synthesized using the speaker data.

FIG. 1 is an explanatory diagram of a system configuration according to an embodiment of the present technology; FIG. 2 is an explanatory diagram of a functional configuration of an information terminal according to an embodiment. FIG. 2 is an explanatory diagram of a functional configuration of a voice synthesis server according to an embodiment; FIG. 2 is an explanatory diagram of a functional configuration of a content analysis server according to an embodiment. FIG. 2 is an explanatory diagram of a functional configuration of a database according to an embodiment. FIG. 2 is an explanatory diagram of the processing of the TTS speaker suggestion system according to the embodiment; FIG. 2 is an explanatory diagram of an example of a topic used in the embodiment. FIG. 4 is an explanatory diagram of a learning process of voice features according to an embodiment. 1 is a flowchart of a topic model generation process according to an embodiment. 11 is a flowchart of a TTS reference data generation process according to an embodiment. 13 is a flowchart of a process of a voice synthesis server according to an embodiment. 11 is a flowchart of database processing according to an embodiment. FIG. 11 is an explanatory diagram of a display example on an information terminal according to an embodiment. 11 is a flowchart of a process for setting a time stamp according to a change in a feature amount of a voice according to an embodiment. 11 is a flowchart of a TTS reference data generation process with reference to a time stamp according to an embodiment. 11 is a flowchart of a process for selecting multiple TTS reference data from a database according to an embodiment. FIG. 11 is an explanatory diagram of the number of reference data items according to distribution according to the embodiment. FIG. 11 is an explanatory diagram of a display example on an information terminal according to an embodiment. FIG. 1 is a block diagram of an information processing apparatus according to an embodiment.

The embodiments will be described below in the following order.
1. System configuration
2. Operation of the TTS speaker suggestion system
<3. Processing of Content Analysis Server>
4. Processing of the voice synthesis server
5. Database Processing
<6. Display on information terminal>
7. How to handle cases where there are multiple speakers in the reference content
8. Proposing multiple speaker candidates
9. Configuration of information processing device
10. Summary and Modifications

1. System configuration
1 shows an example of the configuration of a TTS speaker suggestion system 1 according to an embodiment. The TTS speaker suggestion system 1 is a system that suggests to a user 10 a TTS speaker appropriate for the text of a content CT1 being produced on an information terminal 100, for example.

In the TTS speaker suggestion system 1, the information terminal 100, the voice synthesis server 200, the content analysis server 300, the database 400, and the content provider 500 are connected via a network 600.

User 10 operates information terminal 100 to create content CT1. In this disclosure, "content CT1" refers to a work that mainly uses audio, regardless of whether it contains video (moving images or still images). Information terminal 100 has the functions shown in Figure 2 for content creation.

The content production application 110 is a function for producing and editing the content CT1 in response to operations by the user 10.
The display 120 has a function of displaying images to the user 10 .
The character input unit 130 is an input function for the user 10 to input text data to the content CT1, for example.
The speaker 140 has a function of outputting audio to the user 10 .
The network communication unit 150 is a function for communicating with other devices via the network 600 .

This information terminal 100 is connected to a network 600 such as the Internet through a network communication unit 150, and can obtain synthetic voice data AD by transmitting text data TD to a voice synthesis server 200.
The text data TD to be transmitted is input by the user 10 during the production process by the content production application 110. For example, the user 10 uses the character input unit 130 to input the text data TD.

The voice synthesis server 200 has a role of converting the text data TD transmitted from the information terminal 100 into synthetic voice data AD.
For this reason, the voice synthesis server 200 comprises a text-to-phoneme symbol conversion unit 210, a reference data acquisition unit 220, a speaker search unit 230, a retained speaker data unit 240, a voice synthesis unit 250, and a network communication unit 260, as shown in FIG.

This voice synthesis server 200 is connected to the network 600 via the network communication unit 260. The voice synthesis server 200 receives text data TD transmitted from the information terminal 100.

The text-to-phoneme symbol conversion unit 210 in the speech synthesis server 200 is a function that converts the text data TD into phoneme data. A phoneme is a symbol that represents each individual voice sound. With this function, the speech synthesis server 200 generates phoneme data according to the text data TD transmitted from the information terminal 100.

The reference data acquisition unit 220 is a function that acquires TTS reference data RD from the database 400. The TTS reference data RD is data used to generate synthetic voice data AD that matches the content CT1. The reference data acquisition unit 220 transmits the text data TD sent from the information terminal 100 to the database 400, and requests the database 400 for TTS reference data RD that matches the content CT1.

The speaker search unit 230 is a function that uses the speech features RDX in the TTS reference data RD transmitted from the database 400 to search for speaker data held by the speech synthesis server 200 that has a voice similar to the speech features RDX (similar speech features).

The retained speaker data section 240 is a function for storing speaker data searched by the speaker search section 230. The speaker data includes a speaker ID for identifying individual speaker data, and voice quality data associated with the speaker ID.
The speech synthesis server 200 selects one of the multiple speaker data stored in the retained speaker data section 240 and proposes it to the user 10 .

The voice synthesis unit 250 has a function of generating synthetic voice data AD from text data TD. The voice synthesis unit 250 generates synthetic voice data AD corresponding to the text data TD based on the speaker data selected by the speaker search unit 230 and the phoneme data obtained by the text-to-phoneme symbol conversion unit 210.
The text-to-phoneme converter 210 is also called a grapheme-to-phoneme converter (G2P). The speech synthesizer 250 generally comprises components called a "synthesizer" and a "vocoder" (not shown).

The voice synthesis server 200 sends the generated synthetic voice data AD to the network 600 using the network communication unit 260. The synthetic voice data AD is then passed through the network communication unit 150 of the information terminal 100 and is used by the content production application 110.

The content analysis server 300 shown in Fig. 1 analyzes the content CT2 provided by the content provider 500. The content CT2 is a content that is generally distributed or broadcast, and is referred to as "reference content CT2" to distinguish it from the content CT1. This reference content CT2 also refers to a work with recorded audio, with or without video.
The content analysis server 300 analyzes the reference content CT2 to generate TTS reference data RD, which is then used by the voice synthesis server 200 during voice synthesis.

In this embodiment, the purpose is to provide a voice quality that is suitable for the text data TD input by the user 10, that is, a synthetic voice that is suitable for the content CT1. To achieve this, the role of the content analysis server 300 is to prepare information about a voice that is suitable for reading the text data TD.

As shown in FIG. 4, the content analysis server 300 has a content acquisition unit 310, a voice extraction unit 320, a voice recognition unit 330, a storage unit 340, a topic analysis unit 350, a voice feature acquisition unit 360, and a network communication unit 370.

The content analysis server 300 can communicate with the content provider 500, database 400, etc. via the network communication unit 370.

The content acquisition unit 310 has a function for acquiring reference content CT2 to be analyzed. The content acquisition unit 310 transmits an acquisition request RQ to the content provider 500 via the network communication unit 370, and acquires various reference content CT2. As an example of the reference content CT2, it is conceivable to download public domain internet radio. Alternatively, this can be achieved by individually requesting the content provider 500 to provide a content acquisition API (Application Programming Interface).

The audio extraction unit 320 extracts the audio of the reference content CT2 obtained from the content provider 500. When the reference content CT2 is a video, audio extraction is possible using existing software.

The voice recognition unit 330 converts the voice of the reference content CT2 extracted by the voice extraction unit 320 into text data, that is, an utterance TR.
The storage unit 340 stores the spoken sentence TR.

The topic analysis unit 350 determines the topic vector RDT for the text data transcribed by the speech recognition unit 330. The topic vector RDT is information that expresses what topic the content (text data) is about. Therefore, it can be said to be information that serves as an index for topic classification.
A topic is a term that indicates the type of content of the text data, such as the subject matter or genre. For example, in the case of movies, a topic can be a topic about a specific movie, a topic about the movie industry as a whole, or a genre such as "action movies" or "comedy movies." Topics can be a general term for topics or genres of topics.

The audio feature acquisition unit 360 converts the audio data of the reference content CT2 extracted by the audio extraction unit 320 into an audio feature RDX called an "x-vector."

With the above functions, the content analysis server 300 determines the following for the reference content CT2:
Topic vector RDT
・Speech feature RDX
- Content URL (RDU)
These pieces of information are compiled into one TTS reference data RD, which is then transmitted via the network 600 and stored in the database 400.

The database 400 receives a query from the speech synthesis server 200 and searches for speech features with a voice quality suitable for reading the text data TD. For the search, the TTS reference data RD analyzed by the content analysis server 300 is used.

As shown in FIG. 5, the database 400 includes a storage unit 410, a topic analysis unit 420, a topic similarity analysis unit 430, and a network communication unit 440.
The database 400 is capable of communicating with the content analysis server 300, the voice synthesis server 200, and the like via a network communication unit 440.

The storage unit 410 stores TTS reference data RD for various reference contents CT2. In the figure, N pieces of TTS reference data RD are stored as TTS reference data RD-1 to TTS reference data RD-N.
One TTS reference data RD includes a topic vector RDT, speech features RDX, and a URL (RDU) of reference content CT2.

The topic analysis unit 420 has a function of converting the text data TD transmitted from the speech synthesis server 200 into a topic vector TV.
The topic similarity analysis unit 430 is a function for searching for the TTS reference data RD having the highest similarity among the topic vector TV and the topic vector RDT of the TTS reference data RD stored in the storage unit 410 .

The content provider 500 shown in Fig. 1 is a provider that provides the reference content CT2. For example, a video posting site corresponds to the content provider 500. The content provider 500 is not limited to one, and may be composed of a plurality of companies, services, etc. Furthermore, the reference content CT2 provided is not limited to a video, and may be audio-only content provided by an Internet radio station.

2. Operation of the TTS speaker suggestion system
The operation of the TTS speaker suggestion system 1 configured as above will now be described.
First, the purpose of the operation of the TTS speaker suggestion system 1 according to the embodiment will be described.

The TTS speaker suggestion system 1 operates to provide a user 10 who is creating content CT1 with a voice having a voice quality suited to the content CT1.
For the narration and dialogue of the content CT1, it is necessary to adopt a voice that is appropriate for the content of the content CT1. If a variety of voice qualities are provided for the text of the content CT1, the user 10 will have more options, but the selection itself will be time-consuming. Furthermore, even if the user is able to select based on the high and low frequency of the voice, the gender and age of the speaker, and the image of the voice, the voice may not necessarily match the content of the content CT1.

Therefore, in the present embodiment, the TTS speaker suggestion system 1 generates synthetic voice with a voice quality close to that of the narrator or actor appearing in the reference content CT2 provided by the content provider 500, thereby suggesting to the user 10 a TTS speaker suitable for the content of the content CT1 being produced.
By reading the narration and lines of the content CT1 in a voice similar to that of the speaker appearing in the reference content CT2 that is widely listened to, it is expected that the audience will not feel uncomfortable with the atmosphere of the synthetic voice.

The flow of operations of each device in the TTS speaker suggestion system 1 will be described with reference to FIG.
First, as a prerequisite process for suggesting TTS speakers to the user 10, the content analysis server 300 generates a topic model TM and TTS reference data RD and stores them in the database 400 in a process ST1.

The speech synthesis server 200 requires TTS reference data RD to generate synthetic speech data AD by a speaker suitable for the genre and topic of the content CT1. Therefore, the content analysis server 300 analyzes the reference content CT2 and prepares the TTS reference data RD in the database 400. This process ST1 only needs to be performed sequentially and continuously for various reference contents CT2. In other words, it does not need to be synchronized with the process ST2 for suggesting speakers to the user 10 (information terminal 100).

In process ST1, the content analysis server 300 sends an acquisition request RQ to the content provider 500. In response, the content provider 500 sends reference content CT2 to the content analysis server 300.

After acquiring the reference content CT2, the content analysis server 300 analyzes the acquired reference content CT2 and generates a topic model TM and TTS reference data RD. These are then transmitted to the database 400 for storage.

The content analysis server 300 monitors the distribution of the reference content CT2 provided by the content provider 500, and can perform the above process ST1 as appropriate to generate TTS reference data RD. By focusing on analyzing popular reference content CT2, it is expected that TTS reference data RD can be obtained to suggest to the user 10 a voice quality that will be accepted by a wider range of people.

Next, process ST2 will be described. Process ST2 is executed in response to the user 10 transmitting text data TD from the information terminal 100.

The information terminal 100 transmits text data TD to the voice synthesis server 200. The text data TD is, for example, data of sentences such as narration or dialogue of the content CT1 that the user 10 is creating.

The voice synthesis server 200 sends the received text data TD to the database 400.

The database 400 performs "topic analysis" and "speech feature search" on the received text data TD, and selects one or more TTS reference data RD suitable for the text data TD. The database 400 then transmits the TTS reference data RD obtained by these processes to the speech synthesis server 200.

The speech synthesis server 200 uses the speech features RDX in the TTS reference data RD received from the database 400 to compare it with the speech features of the retained speakers stored in the retained speaker data section 240 in the speech synthesis server 200, and selects the speaker data with the most similar voice quality.

Then, the voice synthesis server 200 performs voice synthesis processing of the text data TD using the voice synthesis model of the speaker data, and generates synthetic voice data AD. The voice synthesis server 200 transmits the synthetic voice data AD to the information terminal 100.

In the information terminal 100, the user 10 uses the content production application 110 to incorporate the received synthetic voice data AD into the content CT1 being produced.

As described above, the TTS speaker suggestion system 1 suggests to the user 10 synthetic voice data AD with a voice quality suitable for the narration, lines, etc. of the content CT1.

Here, topic analysis and speech features will be explained.
First, the topic analysis will be described. Fig. 7 shows information (defined as a topic model) generated based on some reference contents CT2.

In the example of FIG. 7, for ease of explanation, the genre is divided into two topics #1 and #2. The "Latent Dirichlet Allocation (LDA)" was used for the genre division. Topic #1 has a high probability of occurrence of words related to "movies." It can be seen that words related to "smartphones" appear in topic #2. In this example, the words with the highest occurrence probability are displayed in groups of 10 in order of occurrence probability.
This topic model is used to find out which topic an utterance belongs to.

For example, suppose the utterance was "I went to the movies last weekend."
If we split this utterance into words, we get the following word sequence: If we leave only the nouns in this word sequence, we get a noun word sequence.

Sentence sequence of words = ['last week', 'last week', 'was', 'movie', 'to', 'watched', 'came', 'made', 'made', 'made']
Noun word string = ['last week', 'movie']

When topic analysis is performed on this string of noun words using the LDA method, the following topic vector is obtained. Here, a topic vector is defined as a probability distribution of which topic a certain spoken text will be classified into.

Topic #1: 0.8, Topic #2: 0.2
Topic vector = [0.8, 0.2]
This means that the above utterance has a high probability of appearing in relation to topic #1, and the topic of the utterance can be determined to be in line with topic #1.
Such a topic model is generated by the content analysis server 300 and used in processing in the database 400 .

Next, the speech feature will be described.
The voice quality of the speaker appearing in the reference content CT2 is compared with the voice quality of the speaker model stored in the speech synthesis server 200 using a feature called an "x-vector." An "x-vector" is also called "deep speaker embedding," and is a type of "speaker identification technology" that uses deep learning technology. Typically, an x-vector is a 512-dimensional vector.

We will explain how to use x-vector.
Figure 8 shows the learning process of x-vector. The voices of multiple speakers, including A, B, and C, are input to the neural network. For each voice, voice features are extracted for each unit time interval from t=1 to t=T.

These features are then max pooled. Max pooling is a process in which a fixed-size window is slid across the input data and the maximum value is selected within it, making it possible to reduce the input data.

The max-pooled features are subjected to speaker class determination in a classification unit, and speaker training is performed.
A part of the classification part obtained by this learning can be used as an x-vector. For example, the x-vector obtained when inputting the voice of person A can be thought of as a kind of voiceprint of person A.

This neural network can also be applied to speakers other than those used in the training. Therefore, by inputting the voices of the narrator and performers in the reference content CT2 into this neural network, the x-vectors of the speakers of each content can be obtained. Similarly, by inputting the voices of the speakers held by the voice synthesis server 200 into this neural network, the x-vectors of each speaker can be obtained.
By comparing the x-vectors of two speakers, it is possible to determine whether their voices are similar. For example, the cosine similarity between vectors or matching using PLDA (Probabilistic Linear Discriminant Analysis) can be used to measure the similarity of voices.

3. Analysis process of the content analysis server
The process of the content analysis server 300 in step ST1 of FIG. 6 will be described.
The processing of the content analysis server 300 has two model creation phases.
First, there is a phase in which a topic model is created, followed by a phase in which it is determined which topic each piece of reference content CT2 corresponds to, and TTS reference data RD is created.

Explain the topic model creation phase.
The content analysis server 300 collects a plurality of reference contents CT2 from the content provider 500, and creates a topic model from the spoken text in the reference contents CT2 by the LDA method. The purpose of this is to classify the reference contents CT2 provided by the content provider 500 into genres.

This will be described with reference to FIG. 4 and FIG.
9 is a flowchart of the processing performed by the processor as the content analysis server 300, in which the processing to be executed is indicated by a solid-line box, and for ease of understanding, data input or output to the processing is indicated by letters or symbols in parentheses. The storage unit 340 and database 400 where the data is stored are also shown. Step numbers are assigned to the processing performed by the processor as the content analysis server 300.
This flowchart description format is also used in the later-described FIGS. 10, 11, 12, 14, 15, and 16 in the same manner.

The content analysis server 300 acquires the reference content CT 2 from the content provider 500 by the content acquisition unit 310 .
In step S101, the audio extraction unit 320 performs an extraction process of audio data on the acquired reference content CT2.

In step S102, the extracted voice data is subjected to voice recognition processing by the voice recognition unit 330 and converted into a spoken sentence TR, which is text data.
The spoken sentence TR is stored in the storage unit 340 in step S103.

The processes of steps S101, S102, and S103 enclosed by dashed lines are repeatedly performed for a plurality of pieces of reference content CT2.
Then, the processes of steps S101, S102, and S103 are performed on a predetermined number (M pieces) of reference content CT2, and when the number of utterance sentences TR stored in the storage unit 340 reaches M pieces, topic analysis is performed by the topic analysis unit 350 using the stored M pieces of utterance sentences TR in step S110. As a result, a topic model TM is obtained.

The generated topic model TM is sent to the database 400 via the network communication unit 370 in step S111.

Next, the TTS reference data RD creation phase will be described.
The purpose of this phase is to create TTS reference data RD that indicates what topic vector a particular piece of reference content CT2 has, what characteristics the voice used in the reference content CT2 has, and in which URL (Uniform Resource Locator) it exists.

This will be described with reference to FIG. 4 and FIG.
In the content analysis server 300, the audio extraction unit 320 extracts audio data from the reference content CT2 acquired by the content acquisition unit 310 in step S121.

In step S123, the extracted voice data is subjected to voice recognition processing by the voice recognition unit 330, and an utterance sentence TR is obtained.
The utterance sentence TR is topic-analyzed by the topic analysis unit 350 in step S124. This results in a topic vector RDT. The topic vector RDT is a vector representing the probability that the utterance sentence TR belongs to a particular topic.
In this topic analysis process, the topic model TM obtained in the topic model creation phase of FIG. 9 is used.

The voice data extracted in step S121 is also used in step S125, and the voice feature acquisition unit 360 determines the voice feature RDX.

In step S126, the content analysis server 300 combines the topic vector RDT obtained as described above, the speech feature RDX, and the URL (RDU) of the reference content CT2 into one TTS reference data RD, and transmits the combined data to the database 400. As a result, the TTS reference data RD is additionally stored in the database 400.
In the description here, the utterance sentence TR is extracted again from the reference content CT2 in step S123, but the utterance sentence TR generated in step S102 of the topic model creation phase in FIG. 9 may be cached and used.

4. Processing of the voice synthesis server
Next, the processing of the voice synthesis server 200, particularly how the TTS reference data RD is used during voice synthesis, will be described with reference to FIG. 3 and FIG.

The user 10 inputs the text data TD to be converted into synthetic speech using a content production application 110 used in the information terminal 100. The text data TD is so-called natural text, and does not need to be special text prepared for the TTS speaker suggestion system 1. Therefore, the user 10 does not need to learn a special writing method in order to use the TTS speaker suggestion system 1.

The voice synthesis server 200 performs the process of FIG. 11 upon receiving the text data TD.
In step S201, the voice synthesis server 200 receives the text data TD, and the text-to-phoneme symbol converter 210 converts the natural text into phoneme data for voice synthesis.

In step S202, the voice synthesis server 200 performs a process of acquiring the TTS reference data RD by the reference data acquisition unit 220. Specifically, the voice synthesis server 200 transmits the text data TD to the database 400 and requests a search for the TTS reference data RD. In response to this, the database 400 receives the text data TD as an input, selects the TTS reference data RD whose topic vector RDT is closest to the topic vector of the text data TD from among the TTS reference data RD stored in the database 400, and transmits the selected TTS reference data RD to the voice synthesis server 200. The process of the database 400 will be described later.
The speech synthesis server 200 receives the TTS reference data RD thus selected in the database 400, that is, the corresponding reference data for the text data TD.

The speech synthesis server 200 receives the TTS reference data RD as the relevant reference data from the database 400, and in step S203, the speaker search unit 230 searches for a speaker.
In this case, the speech synthesis server 200 uses the speech feature amount RDX contained in the TTS reference data RD to calculate speakers with similar voice qualities from among the speaker models held in the held speaker data unit 240 .
Specifically, it is possible to obtain speakers with similar voice qualities by calculating the cosine similarity between the speech feature RDX sent from the database 400 and the speech feature of a speaker model held by the server 200. This process makes it possible to derive the speaker ID of the speaker data that is most suitable for the topic among the speaker models held by the speech synthesis server 200.

The voice synthesis server 200 inputs the phoneme data obtained in step S201 and the speaker ID obtained in step S203 to the voice synthesis unit 250, thereby obtaining synthetic voice data AD.
Then, in step S205, the voice synthesis server 200 transmits the synthetic voice data AD, the speaker ID, and the reference URL to the information terminal 100 via the network communication unit 260. The reference URL is the URL (RDU) of the reference content CT2 included in the TTS reference data RD acquired in step S202.

5. Database Processing
The processing of the database 400 will be described with reference to Fig. 5 and Fig. 12. This is processing in response to a search request from the speech synthesis server 200 in step S202 of Fig. 11 described above.

When the database 400 receives the text data TD from the speech synthesis server 200, the topic analysis unit 420 performs topic analysis in step S211 of FIG.
In this topic analysis, a topic analysis of the text data TD is performed using a topic model TM stored in the storage unit 410, and a topic vector TV is generated.

Next, in step S212, the database 400 performs a topic search using the topic similarity analysis unit 430. This is a process of searching for a topic vector RDT that is similar to the topic vector TV. Specifically, this process searches for TTS reference data RD (RD-1...RD-N) generated from various reference contents CT2 stored in the storage unit 410, whose topic vector RDT is similar to the topic vector TV of the text data TD.

Similar topic vectors correspond to content that is in the same genre or has a similar topic.
In other words, searching for TTS reference data RD with similar topic vectors can be said to be searching for TTS reference data RD generated based on the content CT1 produced by the user 10 and reference content CT2 that is of a similar genre or topic.

Cosine similarity can be used to search for a topic vector RDT similar to the topic vector TV. As a result, among multiple TTS reference data RD, the TTS reference data RD having the topic vector RDT with the highest similarity to the topic vector TV is selected as the optimal topic.
Then, the database 400 transmits the TTS reference data RD obtained as the optimal topic to the speech synthesis server 200 as the corresponding reference data for the current text data TD.

It is also possible to provide a threshold value for the degree of similarity, and if the degree of similarity does not exceed the threshold value, multiple pieces of TTS reference data RD may be transmitted to the speech synthesis server 200 .

<6. Display on information terminal>
An example of a display shown to the user 10 on the information terminal 100 as a result of the above processing will now be described.

The data that the information terminal 100 receives from the voice synthesis server 200 is the synthetic voice data AD and the URL of the reference content CT2.
The URL of the reference content CT2 is provided to show the user 10 information about what content's speaker's voice was used as reference for generating the synthetic speech.

FIG. 13 shows an example of a display on the display 120 of the information terminal 100. A text box 31, a speaker ID 32, a synthesis start button 33, a play button 34, and a reference URL 35 are displayed on the display 120.

The text box 31 is a box for inputting text data TD.
The speaker ID 32 is the speaker ID selected by the speech synthesis server 200 in step S203 of FIG.
The synthesis start button 33 is an operator for instructing the start of voice synthesis processing.
The playback button 34 is an operator for playing back the synthesized voice.
The reference URL 35 is the URL of the reference content CT2 that refers to the voice, and is displayed, for example, in the form of a link to the reference content CT2.

On this screen, the user 10 can operate the playback button 34 to listen to the voice of the speaker with the speaker ID 32 suggested by the speech synthesis server 200 .
Furthermore, by operating the reference URL 35, the reference content CT2 that the voice synthesis server 200 referred to when selecting the speaker ID 32 can be reproduced, and the voice of the narration or the like can be heard.
Therefore, the user 10 can not only hear the text being read by the speaker ID proposed by the voice synthesis server 200, but also hear the voice in the reference content CT2 that is similar in genre, etc. to the content CT1 being produced in order to select the speaker ID.

7. How to handle cases where there are multiple speakers in the reference content
Up to this point, we have assumed that each reference content CT2 will feature only one speaker. However, in actual reference content CT2, multiple speakers usually appear. For example, in television programs, different announcers are often in charge of different genres, such as field reports, weather forecasts, traffic information, and sports information.

We have explained x-vectors, which are features that represent voices, as audio features. By using x-vectors, it is possible to detect when the speaker changes within the reference content CT2. This is a technique called "speaker diarization," and by using this technique, it is possible for this technology to be able to handle cases where there are multiple speakers in a single piece of content.

This will be explained using FIG.
In step S131, the content analysis server 300 performs audio extraction on the reference content CT2 using the audio extraction unit 320, and acquires audio data.
Next, in step S132, the speech feature acquisition unit 360 extracts speech features RDX every unit time, for example, every 30 seconds. In step S133, feature change detection processing is performed on the speech features RDX for each unit time, and changes equal to or greater than a threshold value are monitored.

Cosine similarity can be used to detect feature changes. If there is a change equal to or greater than a threshold, this means that the speaker has changed, so a timestamp is recorded and stored in the timestamp database 341.
The time stamp database 341 is prepared by using, for example, a portion of the area of the storage unit 340 .

For example, in this manner, the content analysis server 300 monitors changes in the audio feature value RDX of the reference content CT2 and stores the points of change.
This makes it possible to handle reference content CT2 featuring multiple speakers with a configuration equivalent to that of the system described in the TTS reference data creation phase of FIG.

For example, FIG. 15 shows the process of the TTS reference data creation phase similar to that shown in FIG. 10. Note that the same processes as those in FIG. 10 are given the same step numbers and will not be described.

In the case of FIG. 15, in step S121A, the time stamp database 341 is used to extract audio from the reference content CT2 for each section determined by the time stamp.
Thereafter, the extracted voice data is subjected to the same process as in FIG. 10 to generate TTS reference data RD.

8. Proposing multiple speaker candidates
In the above explanation, it has been explained that one speaker best suited to the content CT1 is proposed based on the words (text data TD) spoken in the content CT1.
On the other hand, it is considered that some users 10 may want to try other speakers as options. Assuming such a user 10, an example of a process for providing multiple speakers will be described with reference to Fig. 16. Note that in Fig. 16, the same processes as those in Fig. 12 are assigned the same step numbers to avoid duplication of explanation.

In FIG. 12, the TTS reference data RD searched by the topic similarity analysis unit 430 in step S212 is only the one with the most similar topic vector.
16, in order to increase the number of voices proposed, the topic similarity analysis unit 430 selects a number of TTS reference data RDs to be searched in descending order of cosine similarity. In the figure, ten pieces of TTS reference data RDs (RD#1 to RD#10) are selected.

The TTS reference data RD#1 with the most similar topic vector RDT is designated as the "optimal speaker" data, and the data in descending order of similarity are designated as reference data #2, ..., #10.

Here, the cosine similarity will be explained again.
In the previous explanation of speech features, it was mentioned that x-vectors can be used as the voiceprint of a speaker. Cosine similarity is used to check for similar voices. In Fig. 11, it was mentioned that the speech synthesis server 200 obtains TTS reference data RD based on text data TD transmitted from the information terminal 100. The TTS reference data RD includes speech features RDX, which are x-vectors.
Since this speech feature RDX is a vector, it is possible to calculate the similarity of voice quality by calculating the cosine similarity with other speech features.

For example, the cosine similarity between two vectors a and b can be expressed as (Equation 1) and ranges from "-1" to "1".

When the cosine similarity is "1", the angle is 0 degrees and the vectors are in the same direction. In other words, the voice qualities are completely similar.
When the cosine similarity is "0", the angle between the two vectors is 90 degrees, meaning that the vectors are orthogonal to each other. In other words, it is irrelevant whether the voice qualities are similar or not.
When the cosine similarity is "-1", the angle is 180 degrees and the vector is in the opposite direction. This is a relationship of completely dissimilar voice qualities.

In the process of FIG. 16, the database 400 uses multiple (e.g., 10) TTS reference data RD (RD#1 to RD#10) to perform the similarity evaluation process of step S220.

This similarity evaluation process uses the speech feature RDX of the optimal speaker's TTS reference data RD#1 as the "reference feature" and calculates the cosine similarity between this and each speech feature RDX contained in TTS reference data RD#2 to TTS reference data RD#10.

In this case, TTS reference data RD having speech features RDX whose cosine similarity with the reference features is close to "0" is information on a speaker whose features are not dissimilar to the optimal speaker. Such TTS reference data RD#x is called an "orthogonal speaker."

TTS reference data RD having speech features RDX whose cosine similarity with the reference features is close to "-1" is information on a speaker whose features are not similar to those of the optimal speaker. Such TTS reference data RD#y is called the "reverse speaker."

The optimal speaker's TTS reference data RD#1, the orthogonal speaker's TTS reference data RD#x, and the opposite speaker's TTS reference data RD#y are combined into one data group and sent to the speech synthesis server 200.

In the speech synthesis server 200, in step S203 of FIG. 11, the speaker data stored in the stored speaker data section 240 is searched for speaker data similar to the TTS reference data RD. In this case, similar speaker data is searched for for each of the TTS reference data RD#1, RD#x, and RD#y.
Therefore, a speaker ID similar to the optimal speaker, a speaker ID similar to the orthogonal speaker, and a speaker ID similar to the backward speaker are obtained.
In this way, three voices with different voice qualities are proposed to the user 10.

In the above example, two speech features RDX with cosine similarities close to "0" and "-1" are obtained, but it is also possible to obtain multiple speech features with cosine similarities ranging from "-1" to "1".
In this way, proposing to user 10 reference data with orthogonal or opposite cosine similarities from among reference data with top topic vector similarities means searching for reference content CT2 that has a similar topic but different speaker voice qualities, and proposing to user 10 voice qualities similar to those reference content CT2.

It is also possible to change the number of speakers suggested to the user 10 depending on the variance of the speech features.
The following (Equation 2) is a matrix of the cosine similarity between the reference feature (the speech feature RDX of the optimal speaker) and the speech feature RDX of other TTS reference data RD.

　　

　　

Here, s1,i represents the cosine similarity between the reference feature and the i-th feature.
The variance σ ² of this matrix is shown in Equation 3, where μ is the average value.

By calculating the variance, we can get an indication of whether the proposed speakers converge to a single speech feature or are spread across multiple speakers.
Here, n is the number of TTS reference data RD to be evaluated for similarity. In the case of (Equation 2), n = 9. When 10 TTS reference data RD (RD#1 to RD#10) are initially selected, the similarity of TTS reference data RD#2 to RD#10 is evaluated with respect to TTS reference data RD#1, so n = 9.

For example, if the variance is close to zero, the voice qualities of the speakers of the topic are roughly similar. In that case, it is easier to obtain the desired number of speakers by increasing the number of reference data for evaluating the similarity of the topic vectors, rather than the top 10. For example, the top 20.
Conversely, if the variance is sufficiently large, the number of TTS reference data RD to be evaluated does not need to be very large.
This is expressed in the formula (4), where variance is used as the denominator. y is the number of TTS reference data RD to be evaluated.

Figure 17 is a graph of (Equation 4). It can be seen that the number of reference data to be evaluated changes depending on the variance value (V). When the variance is close to zero, 30 TTS reference data RD are used to examine the variation in voice quality. On the other hand, when the variance is large, a wide variety of voice qualities can be obtained from within 10 TTS reference data RD.

By doing the above, multiple speakers with variations in the voice quality of the speakers can be suggested to the user 10 while covering similar topics. In this case, it is conceivable that the display 120 of the information terminal 100 will display, for example, the image shown in FIG. 18.

"Soccer" is shown as an example of a topic. As in FIG. 13, a text box 31, a speaker ID 32, a synthesis start button 33, a play button 34, and a reference URL 35 are displayed on the display 120. However, the play button 34, the speaker ID 32, and the reference URL 35 are shown as a synthetic speech list 36. In other words, a list of multiple speakers who are candidates for use in the content CT1 is displayed.

The speakers are displayed in the synthetic speech list 36 in descending order of cosine similarity of the topic vectors. As described above, each of these speakers has a different voice quality, including an optimal speaker, an orthogonal speaker, and an opposite speaker.
For each speaker, the topic is "soccer" which matches the topic of the text information entered in the text box 31, and when a speaker ID is selected, a link to reference content CT2 featuring speakers with a variety of voice qualities is displayed as a reference URL 35.

For example, in the example of FIG. 18, five speakers are presented, and the user 10 can play back the voice of each speaker by pressing the play button 34. The user 10 can also check the voice of the reference content CT2 that was referenced to select that speaker ID by operating the reference URL 35.

Here, the advantage of proposing multiple speaker candidates for one piece of content CT1 will be considered.
For example, consider the case where this technology is applied to video content such as online lessons for elementary school students.

In the case of subjects such as Japanese, arithmetic, science, and social studies, instead of conducting a topic analysis on the text data TD of these four subjects together, a topic analysis can be performed on each subject, thereby obtaining a voice that is characteristic of each teacher.
When only the topic vectors of the text data TD are considered, for example, the voices of teachers of Japanese and social studies may be similar, and the user 10 may want to change the speaker for each subject to avoid monotony.

In such cases, a topic search is performed using the Japanese language and social studies topic vector TV, and orthogonal speakers and reverse speakers are selected from the reference data with the highest cosine similarity, thereby avoiding duplication of similar voices.

It is also possible to automatically split long pieces of text into multiple topics and change the speaker.
For example, even in the case of video content for Japanese language classes, there are various topics such as "novels,""criticisms," and "poetry." For example, I would like students to read "poetry" with emotion.
In such cases, it is possible to suggest speakers as a group by performing topic analysis on a paragraph-by-paragraph basis. Paragraph detection can be done by detecting indents and blank lines.

9. Configuration of information processing device
An example of the configuration of an information processing device 70 that can be used as the speech synthesis server 200, the content analysis server 300, the database 400, and the information terminal 100 in the above-mentioned TTS speaker suggestion system 1 will be described with reference to FIG.
The information processing device 70 can be configured as, for example, a dedicated workstation, a general-purpose personal computer, a mobile terminal device, or the like.

19 executes various processes according to programs stored in a ROM 72 or a non-volatile memory unit 74 such as an EEPROM (Electrically Erasable Programmable Read-Only Memory), or programs loaded from a storage unit 79 to a RAM 73. The RAM 73 also stores data necessary for the CPU 71 to execute various processes, as appropriate.
In the CPU 71, the functions of carrying out various controls and calculations shown in FIGS. 2, 3, 4 and 5 are realized by programs.

In addition, processors other than the CPU 71 may include a GPU (Graphics Processing Unit), a GPGPU (General-purpose computing on graphics processing units), an AI (artificial intelligence) processor, etc.

The CPU 71, ROM 72, RAM 73, and non-volatile memory unit 74 are interconnected via a bus 83. The input/output interface 75 is also connected to this bus 83.

An input unit 76 including an operator or an operating device is connected to the input/output interface 75. For example, the input unit 76 may be various operators or operating devices such as a keyboard, a mouse, a key, a dial, a touch panel, a touch pad, or a remote controller.
An operation by the user 10 is detected by the input unit 76 , and a signal corresponding to the input operation is interpreted by the CPU 71 .

The input/output interface 75 is also connected, either integrally or separately, to a display unit 77 such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) panel, and an audio output unit 78 such as a speaker.

The display unit 77 performs various displays as a user interface. The display unit 77 is, for example, a display device provided in the housing of the information processing device 70, or a separate display device connected to the information processing device 70.
The display unit 77 executes various image displays on the display screen based on instructions from the CPU 71. The display unit 77 also displays various operation menus, icons, messages, etc., that is, GUIs (Graphical User Interfaces), based on instructions from the CPU 71.

The input/output interface 75 may be connected to a storage unit 79 configured with a solid state drive (SSD) or a hard disk drive (HDD) or a communication unit 80 configured with a modem or the like.
The storage unit 79 can be used to store various data, and a database can be constructed in the storage unit 79.
For example, the retained speaker data unit 240 of the speech synthesis server 200 , the storage unit 340 of the content analysis server 300 , the storage unit 410 of the database 400 , etc. can be configured using the storage unit 79 .

The communication unit 80 performs communication processing via the network 600 .
For example, the network communication unit 150 of the information terminal 100 , the network communication unit 260 of the voice synthesis server 200 , the network communication unit 370 of the content analysis server 300 , and the network communication unit 440 of the database 400 can be configured using the communication unit 80 .

A drive 82 is also connected to the input/output interface 75 as required, and a removable recording medium 81 such as a flash memory, a memory card, a magnetic disk, an optical disk, or a magneto-optical disk is appropriately attached thereto.
The drive 82 allows data files such as image files and various computer programs to be read from the removable recording medium 81. The read data files are stored in the storage unit 79, and images and sounds contained in the data files are output on the display unit 77 and the sound output unit 78. In addition, the computer programs and the like read from the removable recording medium 81 are installed in the storage unit 79 as necessary.

In this information processing device 70, software can be installed via network communication by the communication unit 80 or via a removable recording medium 81. Alternatively, the software may be pre-stored in the ROM 72, the storage unit 79, etc.

The information terminal 100, the voice synthesis server 200, the content analysis server 300, and the database 400 can be configured by using such an information processing device 70. The configuration of the information terminal 100 in Fig. 2, the configuration of the voice synthesis server 200 in Fig. 3, the configuration of the content analysis server 300 in Fig. 4, and the configuration of the database 400 in Fig. 5 can be realized by the hardware configuration of the information processing device 70 in Fig. 19 and software installed therein.

10. Summary and Modifications
The above embodiment provides the following advantages.

The speech synthesis server 200 of the embodiment includes a speaker search unit 230 that selects speaker data from among a plurality of speaker data by performing a search based on the topic of the text data TD, and a speech synthesis unit 250 that generates synthetic speech of the text data TD using the speaker data selected by the speaker search unit 230 (see FIG. 3).
That is, speaker data with a voice quality corresponding to the topic (topic or genre) of the content of the text data TD is selected, and the reading voice of the text data TD is synthesized using the speaker data. This makes it possible to provide the user 10 with synthetic voice data AD using speaker data with a voice quality that matches the topic of the text data TD of the content CT1, rather than providing synthetic voice data AD with various voice qualities for the text data TD indiscriminately.

In the embodiment of the speech synthesis server 200, the speaker search unit 230 acquires TTS reference data RD selected based on the topic of the text data TD, and selects speaker data from among multiple speaker data based on the similarity between the voice features indicated in the TTS reference data RD and the voice features of the speaker data (see FIG. 11).
By acquiring the TTS reference data RD, it is possible to obtain information on speech features RDX that are generally considered to match the topic of the text data TD. Therefore, the speech synthesis server 200 can select speaker data appropriate for the topic of the text data TD to be processed from the speaker data held in the held speaker data section 240.

The voice synthesis server 200 of the embodiment transmits text data TD to an external database 400 and receives TTS reference data RD from the database 400 via the reference data acquisition unit 220 (see FIGS. 6 and 11).
By acquiring the TTS reference data RD from the database 400, the speech synthesis server 200 does not need to store a large number of TTS reference data RD. The speech synthesis server 200 can select appropriate speaker data from among the speaker data held by the server 200 based on the speech feature value RDX of the TTS reference data RD corresponding to the text data TD to be processed. In other words, the speech synthesis server 200 can select appropriate speaker data from among the speaker data stored therein without performing processes such as storing, adding, and managing the TTS reference data RD.

In this embodiment, the TTS reference data RD includes a topic vector RDT that serves as an index for classifying the topics of the reference content CT2 (an index for determining similarity/dissimilarity) (see FIG. 5).
This allows the database 400 to compare the topic vector RDT of the TTS reference data RD with the topic vector TV of the text data TD to be processed received from the speech synthesis server 200, and select the TTS reference data RD according to the topic of the text data TD. Therefore, it is possible to select the TTS reference data RD that is suitable for the topic of the text data TD to be processed.

The TTS reference data RD includes speech features obtained by extracting features from the speech data of the reference content CT2 (see FIG. 5).
By including the speech feature RDX in the TTS reference data RD, the speech synthesis server 200 can select speaker data with a voice quality similar to the speech feature RDX, which becomes speaker data with a voice quality that matches the topic of the text data TD.

The TTS reference data RD also includes information indicating the reference content CT2 used to create the TTS reference data RD (see FIG. 5). For example, the information indicating the reference content CT2 includes the URL (RDU) of the content.
This enables a user interface that allows the user 10 to view the reference content CT2, as shown in Figures 13 and 18. The user 10 can use the information about the reference content CT2 on which the synthesized voice by the voice synthesis server 200 is selected as a reference for content production.

The voice synthesis server 200 of the embodiment is equipped with a network communication unit 260 that performs processing to transmit information on the speaker data selected by the speaker search unit 230 and the synthetic voice generated by the voice synthesis unit 250 to the information terminal 100 that is the source of the text data (see Figure 3).
11, the voice synthesis server 200 transmits the speaker ID and the synthetic voice data AD to the information terminal 100 via the network communication unit 260. This makes it possible to provide a service in which synthetic voice suited to the content CT1 is suggested to the user 10 on a display screen such as that shown in FIG.

In the embodiment of the speech synthesis server 200, the network communication unit 260 performs a process of transmitting information about the reference content CT2 contained in the TTS reference data RD to the information terminal 100 that is the source of the text data TD (see FIG. 11).
This allows the reference URL 35 to be displayed on a display screen such as that shown in FIG. 13 or FIG. 18, providing a path for the user 10 to view the reference content CT2.

In the embodiment, an example is given in which the speaker search unit 230 of the speech synthesis server 200 selects speaker data for each of multiple TTS reference data RD selected based on the topic of the text data TD, based on the similarity between the speech features indicated in the TTS reference data RD and the speech features of the speaker data to be stored.
The speech synthesis server 200 can obtain a plurality of pieces of information on speech features RDX that match the topic of the text data TD by acquiring a plurality of TTS reference data RD (for example, TTS reference data RD#1, RD#x, RD#y in FIG. 16). Therefore, the speech synthesis server 200 can select speaker data based on each of the TTS reference data RD#1, RD#x, RD#y from the speaker data held in the held speaker data section 240, thereby selecting a plurality of speaker data that match the topic of the content CT1 and proposing the voice quality of each to the user 10.

In the embodiment, an example has been given in which the network communication unit 260 transmits information on speaker data selected based on a plurality of TTS reference data RD as information displayed in a list on the information terminal 100 .
For example, speaker IDs 32 and the like may be displayed as a list in the synthetic speech list 36 of Fig. 18. This allows the user 10 to preview a number of voice qualities that are selected as candidates based on whether they match the topic of the content CT1.

Such a plurality of TTS reference data RD are reference data further selected from the plurality of TTS reference data RD selected in descending order of similarity to the topic of the text data TD. An example is given in which the topic of the original reference content CT2 includes a first reference data having the highest similarity to the topic of the text data TD and one or more second reference data selected based on the similarity evaluation of the first reference data.
For example, multiple TTS reference data RD (RD#1 to RD#10) are selected as shown in Fig. 16. Among them, the first reference data (TTS reference data RD#1) that is determined to have the highest similarity and one or more second reference data (TTS reference data RD#x, RD#y) are selected. This makes it possible to select TTS reference data RD with various voice qualities that match the topic by using a similarity evaluation method.

In this case, the second reference data is reference data whose similarity is evaluated as being orthogonal or opposite in cosine similarity with respect to the speech feature of the first reference data.
As a result, the plurality of TTS reference data RD#1, RD#x, RD#y have speech features RDX with voice qualities that are dissimilar to each other, and the speech synthesis server 200 can provide the user 10 with speaker data with variations of different voice qualities based on these.

The number of the multiple reference data selected in descending order of similarity to the topic of the text data TD is set according to the variance of the speech features (see, for example, FIGS. 16 and 17).
In this way, if the multiple TTS reference data RD initially selected in descending order of similarity have roughly similar voice qualities, the number is increased to increase the variance, so that the TTS reference data RD#x, RD#y selected based on the similarity evaluation do not have a voice quality similar to that of the TTS reference data RD#1. In other words, by controlling the parameter for selecting TTS reference data RD according to the variance, a wide variety of voice qualities can be maintained in the end proposed to the user 10.

The TTS speaker suggestion system 1 of the embodiment includes a content analysis device (content analysis server 300) that analyzes reference content CT2 to generate TTS reference data RD including information on topics and information on speech features. The TTS speaker suggestion system 1 also includes a database 400 that stores the TTS reference data RD generated by the content analysis server 300 and selects the TTS reference data RD based on the topic of text data TD. The system further includes a speech synthesis device, a speech synthesis server 200 that includes a speaker search unit 230 and a speech synthesis unit 250 as described above.
In this TTS speaker suggestion system 1, the content analysis server 300 and the database 400 generate and accumulate TTS reference data RD based on various reference content CT2. The voice synthesis server 200 can use such information resources to provide synthetic voice that matches the text data TD received from the information terminal 100. The more the quality and quantity of the TTS reference data RD is improved, the more the voice synthesis server 200 can provide the user 10 with synthetic voice whose voice quality matches the text data TD.

The program of the embodiment is a program that causes, for example, a CPU, a DSP (digital signal processor), an AI processor, or an information processing device 70 including these to execute the process shown in FIG.
In other words, the program of the embodiment is a program that causes an information processing device to execute a speaker search process that selects speaker data from a plurality of speaker data by searching based on the topic of the text data TD, and a voice synthesis process that generates synthetic voice of the text data TD using the speaker data selected in the speaker search process.

With such a program, an information processing device serving as the voice synthesis server 200 of the embodiment can be realized in, for example, a computer device, a mobile terminal device, or other device capable of performing information processing.

Such a program can be recorded in advance in a HDD serving as a recording medium built into a device such as a computer device, or in a ROM within a microcomputer having a CPU.
Alternatively, the program may be temporarily or permanently stored (recorded) on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, a memory card, etc. Such removable recording media may be provided as so-called package software.
Furthermore, such a program can be installed in a personal computer or the like from a removable recording medium, or can be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.

Furthermore, such a program is suitable for the widespread provision of information processing devices 70 that constitute the voice synthesis server 200 of the embodiment. For example, by downloading the program to mobile terminal devices such as smartphones and tablets, imaging devices, mobile phones, personal computers, game devices, video devices, PDAs (Personal Digital Assistants), etc., these devices can be made into information processing devices 70 that function as the voice synthesis server 200 of the present disclosure.

Note that the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

The present technology can also be configured as follows.
(1)
a speaker search unit that selects speaker data from among the plurality of speaker data by performing a search based on a topic of the text data;
a voice synthesis unit that generates synthetic voice data of the text data using speaker data selected by the speaker search unit.
(2)
The information processing device described in (1) above, wherein the speaker search unit acquires reference data selected based on a topic of the text data, and selects speaker data from among multiple speaker data based on a similarity between voice features indicated in the reference data and voice features of the speaker data.
(3)
The information processing device according to (2) above, further comprising: transmitting the text data to an external database; and receiving the reference data from the database.
(4)
The reference data is
The information processing device according to (2) or (3) above, including a topic vector that is an index for classifying topics of the reference content used to create the reference data.
(5)
The reference data is
The information processing device according to any one of (2) to (4) above, including audio features obtained by extracting features of audio data of reference content used to create the reference data.
(6)
The reference data is
The information processing device according to any one of (2) to (5) above, further comprising information indicating reference content used to create the reference data.
(7)
The information processing device according to any one of (1) to (6) above, further comprising a communication unit that performs processing to transmit information on the speaker data selected by the speaker search unit and synthetic voice data generated by the voice synthesis unit to an information terminal that is a source of the text data.
(8)
the speaker search unit acquires reference data selected based on a topic of the text data, and selects speaker data from among a plurality of speaker data based on a similarity between a voice feature indicated in the reference data and a voice feature of the speaker data;
The information processing device according to (7) above, wherein the communication unit performs a process of transmitting information regarding reference content used to create the reference data, which is included in the reference data, to an information terminal that is a source of the text data.
(9)
The information processing device according to any one of (1) to (8) above, wherein the speaker search unit selects speaker data for each of a plurality of reference data selected based on a topic of the text data, based on a similarity between voice characteristics indicated in the reference data and voice characteristics of the speaker data.
(10)
a communication unit that performs processing to transmit information on the speaker data selected by the speaker search unit and the synthetic voice generated by the voice synthesis unit to an information terminal that is a source of the text data;
The information processing device according to (9) above, wherein information on the speaker data selected based on the plurality of reference data is transmitted as information displayed as a list on the information terminal.
(11)
The plurality of reference data includes
a piece of reference data further selected from a plurality of pieces of reference data selected in descending order of similarity to a topic of the text data;
a first reference data in which a topic of the original reference content has a highest similarity to a topic of the text data;
The information processing device according to (9) or (10) above, further comprising one or more second reference data selected based on a similarity evaluation with the first reference data.
(12)
The information processing device according to (11) above, wherein the second reference data is reference data whose similarity is evaluated as being orthogonal or opposite in cosine similarity based on the speech feature of the first reference data.
(13)
The information processing device according to (11) or (12), wherein the number of the multiple reference data selected in order of similarity to the topic of the text data is set according to the variance of the speech features.
(14)
a speaker search process for selecting speaker data from among a plurality of speaker data by searching based on a topic of the text data;
a speech synthesis process for generating synthetic speech of the text data using speaker data selected in the speaker search process;
The information processing method is executed by an information processing device.
(15)
a speaker search process for selecting speaker data from among a plurality of speaker data by searching based on a topic of the text data;
a speech synthesis process for generating synthetic speech of the text data using speaker data selected in the speaker search process;
A program for causing an information processing device to execute the above.
(16)
a content analysis device for analyzing the reference content to generate reference data including information on topics and information on speech features;
a database for storing the reference data generated by the content analysis device and for selecting reference data based on a topic of text data;
A voice synthesizer;
Equipped with
The speech synthesizer comprises:
a speaker search unit that acquires reference data selected from the database based on a topic of text data, and selects speaker data from among a plurality of speaker data based on a similarity between a voice feature indicated by the reference data and a voice feature of the speaker data;
a voice synthesis unit that generates synthetic voice data for the text data using speaker data selected by the speaker search unit.

1 TTS speaker suggestion system 70 Information processing device 71 CPU
100 Information terminal 200 Speech synthesis server 210 Text-phoneme symbol conversion unit 220 Reference data acquisition unit 230 Speaker search unit 240 Retained speaker data unit 250 Speech synthesis unit 260 Network communication unit 300 Content analysis server 400 Database 500 Content provider RDT Topic vector RDX Speech feature amount RDU Content URL
TD Text data AD Synthetic speech data RD TTS reference data TM Topic model TV Topic vector XV Speech feature CT1 Content (created by user)
CT2 Reference content (for analysis)

Claims (16)

a speaker search unit that selects speaker data from among the plurality of speaker data by performing a search based on a topic of the text data;
a voice synthesis unit that generates synthetic voice data of the text data using speaker data selected by the speaker search unit. The information processing device according to claim 1 , wherein the speaker search unit acquires reference data selected based on a topic of the text data, and selects speaker data from among a plurality of speaker data based on a similarity between voice characteristics indicated in the reference data and voice characteristics of the speaker data. The information processing apparatus according to claim 2 , further comprising: transmitting the text data to an external database; and receiving the reference data from the database. The reference data is
The information processing device according to claim 2 , further comprising a topic vector that is an index for classifying topics of the reference content used for creating the reference data. The reference data is
The information processing device according to claim 2 , further comprising: an audio feature quantity obtained by extracting a feature quantity of audio data of the reference content used to create the reference data. The reference data is
The information processing apparatus according to claim 2 , further comprising information indicating a reference content used to create the reference data. The information processing device according to claim 1 , further comprising a communication unit configured to transmit information on the speaker data selected by the speaker search unit and synthetic voice data generated by the voice synthesis unit to an information terminal that is a source of the text data. the speaker search unit acquires reference data selected based on a topic of the text data, and selects speaker data from among a plurality of speaker data based on a similarity between a voice feature indicated by the reference data and a voice feature of the speaker data;
The information processing device according to claim 7 , wherein the communication unit performs a process of transmitting information about a reference content used to create the reference data, the information being included in the reference data, to an information terminal that is a source of the text data. The information processing device according to claim 1 , wherein the speaker search unit selects speaker data for each of a plurality of reference data selected based on a topic of the text data based on a similarity between voice characteristics indicated in the reference data and voice characteristics of the speaker data. a communication unit that performs processing to transmit information on the speaker data selected by the speaker search unit and the synthetic voice generated by the voice synthesis unit to an information terminal that is a source of the text data;
The information processing apparatus according to claim 9 , wherein the information on the speaker data selected based on the plurality of reference data is transmitted as information to be displayed as a list on the information terminal. The plurality of reference data includes
a piece of reference data further selected from a plurality of pieces of reference data selected in descending order of similarity to a topic of the text data;
a first reference data in which a topic of the original reference content has a highest similarity to a topic of the text data;
The information processing apparatus according to claim 9 , further comprising one or more second reference data selected based on a similarity evaluation with the first reference data. The information processing apparatus according to claim 11 , wherein the second reference data is reference data whose similarity is evaluated as being orthogonal or opposite in cosine similarity with respect to the speech feature quantity of the first reference data. The information processing device according to claim 11 , wherein the number of the plurality of reference data selected in order of similarity to the topic of the text data is set according to a variance of the speech feature. a speaker search process for selecting speaker data from among a plurality of speaker data by searching based on a topic of the text data;
a speech synthesis process for generating synthetic speech of the text data using speaker data selected in the speaker search process;
The information processing method is executed by an information processing device. a speaker search process for selecting speaker data from among a plurality of speaker data by searching based on a topic of the text data;
a speech synthesis process for generating synthetic speech of the text data using speaker data selected in the speaker search process;
A program for causing an information processing device to execute the above. a content analysis device for analyzing the reference content to generate reference data including information on topics and information on speech features;
a database for storing the reference data generated by the content analysis device and for selecting reference data based on a topic of text data;
A voice synthesizer;
Equipped with
The speech synthesizer comprises:
a speaker search unit that acquires reference data selected from the database based on a topic of text data, and selects speaker data from among a plurality of speaker data based on a similarity between a voice feature indicated by the reference data and a voice feature of the speaker data;
a voice synthesis unit that generates synthetic voice data for the text data using speaker data selected by the speaker search unit.

Images