JP2018128500A - Formation device, formation method and formation program - Google Patents

Abstract

Speech enhancement that is optimal for speech recognition according to the environment. An acquisition unit 15a acquires observation signals at a plurality of points including an acoustic signal of a target speech that is a target of speech recognition and an acoustic signal of noise other than the target speech, and a speech enhancement unit 15d Using a predetermined beamformer wf for forming an acoustic signal beam for each frequency, an acoustic signal of an emphasized speech in which the acoustic signal of the target speech is emphasized among the observation signals is calculated, and the speech recognition unit 15e calculates The estimated phoneme probability distribution of the emphasized speech is estimated, a reference label indicating the phoneme is assigned to the emphasized speech, and the optimization unit 15f minimizes the difference between the reference label and the phoneme probability distribution of the emphasized speech. Thus, the beam former wf is optimized. [Selection] Figure 1

Description

  The present invention relates to a forming apparatus, a forming method, and a forming program.

  2. Description of the Related Art Conventionally, a technique for calculating a beam former that forms a beam that has been subjected to speech enhancement while suppressing noise before speech recognition has been disclosed (see Non-Patent Documents 1 and 2). Further, a technique for estimating a parameter for speech enhancement according to the environment is disclosed so that speech recognition can be performed more clearly (see Non-Patent Document 3).

T.Higuchi, N.Ito, T.Yoshioka, T.Nakatani, “ROBUST MVDR BEAMFORMING USING TIME-FREQUENCY MASKS FOR ONLINE / OFFLINE ASR IN NOISE”, 2016 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2016 March, pp.5210-5214 L.J.Griffiths, C.W.Jim, “An Alternative Approach to Linearly Constrained Adaptive Beamforming”, IEEE Transactions on antennas and propagation, vol.AP-30, NO.1, January 1982, pp.27-34 T.Higuchi, T.Yoshioka, T.Nakatani, “Optimization of Speech Enhancement Front-end with Speech Recognition-level Criterion”, Interspeech 2016, 2016, pp.3808-3812

  However, in the prior art, since speech enhancement and speech recognition are performed separately, speech enhancement is not always optimal for speech recognition. Also, even with the speech enhancement technique using the time-frequency mask described in Non-Patent Document 3, the improvement rate of the speech recognition rate was smaller than the improvement rate of the recognition rate by the beamformer.

  The present invention has been made in view of the above, and an object thereof is to perform speech enhancement that is optimal for speech recognition in accordance with the environment.

  In order to solve the above-described problems and achieve the object, the forming apparatus according to the present invention includes a plurality of points including an acoustic signal of a target speech that is a target of speech recognition and an acoustic signal of noise other than the target speech. Using the acquisition unit for acquiring the observation signal and a predetermined beamformer for forming a beam of the acoustic signal for each frequency, the acoustic signal of the emphasized speech that emphasizes the acoustic signal of the target speech among the observation signals A speech enhancement unit to calculate; a speech recognition unit that estimates a phoneme probability distribution of the calculated enhanced speech; and a reference label indicating a phoneme to the enhanced speech; and the reference label and the phoneme of the enhanced speech An optimization unit for optimizing the beamformer so as to minimize a difference from the probability distribution.

  According to the present invention, it is possible to perform speech enhancement optimal for speech recognition according to the environment.

FIG. 1 is a schematic diagram showing a schematic configuration of a forming apparatus according to an embodiment of the present invention. FIG. 2 is an explanatory diagram for explaining processing of the estimation unit of the present embodiment. FIG. 3 is an explanatory diagram for explaining the processing of the optimization unit of the present embodiment. FIG. 4 is a flowchart showing the formation processing procedure of this embodiment. FIG. 5 is a diagram illustrating a computer that executes a forming program.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited by this embodiment. Moreover, in description of drawing, the same code | symbol is attached | subjected and shown to the same part.

[Configuration of forming apparatus]
First, a schematic configuration of a forming apparatus according to the present embodiment will be described with reference to FIG. As shown in FIG. 1, the forming apparatus 1 according to the present embodiment is realized by a general-purpose computer such as a workstation or a personal computer, and includes an input unit 11, an output unit 12, a communication control unit 13, a storage unit 14, and a control unit. 15. The forming apparatus 1 executes a forming process to be described later, and forms a beam that has been subjected to speech enhancement of the target speech optimally for speech recognition.

  The input unit 11 is realized using an input device such as a keyboard or a mouse, and inputs various instruction information to the control unit 15 in response to an input operation by the operator. The output unit 12 is realized by a display device such as a liquid crystal display, a printing device such as a printer, an information communication device, a speaker, and the like. For example, after executing a forming process to be described later, an emphasis voice, a voice recognition result, and the like are sent to the operator. Output.

  The communication control unit 13 is realized by a NIC (Network Interface Card) or the like, and controls communication between an external device such as a server and the control unit 15 via a telecommunication line such as a LAN (Local Area Network) or the Internet.

  The storage unit 14 is realized by a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. In the storage unit 14, a processing program for operating the forming apparatus 1, data used during execution of the processing program, and the like are stored in advance, or temporarily stored for each processing. The storage unit 14 may be configured to communicate with the control unit 15 via the communication control unit 13.

  As illustrated in FIG. 1, the control unit 15 executes a processing program stored in a memory by an arithmetic processing unit such as a CPU (Central Processing Unit), so that an acquisition unit 15 a, a time-frequency analysis unit 15 b, and an estimation unit 15c, the speech enhancement unit 15d, the speech recognition unit 15e, and the optimization unit 15f.

  The acquisition unit 15a acquires observation signals at a plurality of points including an acoustic signal of a target speech that is a target of speech recognition and an acoustic signal of noise other than the target speech. Specifically, the acquisition unit 15a is installed at different points in M locations in a situation where an acoustic signal from a sound source of one target speech that is a target of speech recognition and an acoustic signal of background noise are mixed. A multi-channel observation signal consisting of M observation signals recorded by a microphone is acquired.

  The time-frequency analysis unit 15b performs short-time signal analysis such as short-time Fourier transform on the M observation signals acquired by the acquisition unit 15a, and performs the frequency (hereinafter referred to as the frequency of the same short-term section of a predetermined length). The observation signal is extracted every time). Moreover, the time frequency analysis part 15b produces | generates the observation vector which is an M-dimensional vertical vector using the extracted observation signal for every time frequency.

Here, since the target speech has sparsity, it is assumed that there is a point of time frequency of only noise that does not include the target speech (see Non-Patent Document 1). In this case, the observation vector y _{f, t} can be expressed by the following formula (1) or the following formula (2). Here, t is an integer of 1 to T and represents a time number. F is an integer of 0 to F and represents a frequency number.

  The steering vector is a vector whose component is a transfer characteristic from the sound source of the target speech or the noise sound source to each microphone, and includes spatial information of the sound source.

  The estimation unit 15c separates and estimates the probability distribution of the combination of signals that do not include the acoustic signal of the target speech from among the combinations of signals for each frequency in the same short period of the predetermined length of the observation signal. The steering vector including the spatial information of the target sound source is estimated for each frequency, and the beam former is calculated using the steering vector.

Specifically, first, the estimation unit 15c acquires the observation vector y _{f, t} from the time-frequency analysis unit 15b as a combination of signals for each frequency in the same short period of a predetermined length of the observation signal. Next, the estimation unit 15c classifies the observation vector into a cluster of target speech and noise and a cluster of only noise that does not include the target speech, and estimates a spatial correlation matrix corresponding to each cluster. In addition, the estimation unit 15c estimates the spatial correlation matrix of the target speech using this. From this spatial correlation matrix, a steering vector including the spatial information of the sound source of the target speech is derived.

  Here, with reference to FIG. 2, the process of the estimation part 15c is demonstrated. As shown in FIG. 2, the estimation unit 15c includes a parameter estimation unit 151, a mask estimation unit 152, a spatial correlation matrix calculation unit 153, a steering vector calculation unit 154, and a beamformer estimation unit 155.

First, the probability distribution of the observation vector y _{f, t} , as shown in the following equation (3), is a probability distribution of a cluster of target speech and noise (hereinafter also referred to as posterior probability) and a posterior probability of a cluster of noise only. Can be modeled and expressed as a mixture distribution.

  In this case, the parameter estimation unit 151 estimates each parameter (hereinafter referred to as a distribution parameter) of the above formula (3). At that time, the parameter estimation unit 151 uses the likelihood function shown in the following equation (4) as an objective function.

  That is, the parameter estimation unit 151 obtains a distribution parameter that locally maximizes the objective function shown in the above equation (4) as a distribution parameter of the mixed distribution that approximately represents the distribution of the observation vector.

  Therefore, in order to apply the EM (Expectation-Maximization) algorithm, the parameter estimation unit 151 defines a Q function representing the conditional expected value of the log likelihood function as shown in the following equation (5).

  The auxiliary parameter of the above equation (5) corresponds to a mask representing the degree to which the observation vector belongs to each cluster, and can be calculated as the following equation (6) in the E (expected value) step.

  In addition, the distribution parameter update equation is expressed by the following equation (7) and the following equation (M) in the M (maximization) step by partially differentiating the Q function shown in the equation (5) with respect to each parameter to 0. 8) is derived.

  The parameter estimation unit 151 updates the distribution parameters according to the above equations (7) and (8) in M steps. Further, the mask estimation unit 152 calculates the auxiliary parameter by the above equation (6) using the updated distribution parameter in the E step. The estimation unit 15c repeatedly performs the update of the distribution parameter and the calculation of the auxiliary parameter. Thereby, the parameter estimation unit 151 estimates a distribution parameter that locally maximizes the objective function shown in the above equation (4). Further, the mask estimation unit 152 estimates an auxiliary parameter, that is, a mask.

Here, by multiplying the observation signal by the auxiliary parameter λ ⁽ⁿ⁾ _{f, t} corresponding to the noise-only cluster, the noise-only observation signal is obtained. Therefore, the noise-only spatial correlation matrix can be obtained by the following equation (9).

  Therefore, the spatial correlation matrix calculation unit 153 can obtain the spatial correlation matrix of the target speech by subtracting the spatial correlation matrix of only noise from the spatial correlation matrix of the observation signal, as shown in the following equation (10).

  Next, the steering vector calculation unit 154 derives an eigenvector corresponding to the first eigenvalue by performing eigenvalue decomposition on the spatial correlation matrix of the target speech as the steering vector of the target speech.

In addition, the beamformer estimation unit 155 calculates a beamformer w _f that forms a beam that enhances the target speech, using the estimated steering vector of the target speech. Specifically, the beamformer estimation unit 155 calculates the beamformer w _f by minimizing the objective function shown in the following equation (12) under the condition shown in the following equation (11) (Non-Patent Document 2). reference).

The beam former w _f calculated here attenuates the power of the acoustic signal of noise in the other direction without attenuating the power of the acoustic signal in the steering vector direction including the spatial information of the sound source of the target speech. A beam that suppresses noise can be formed.

Returning to the description of FIG. Speech enhancement unit 15d, using a predetermined beamformer w _f for forming a beam of acoustic signals for each frequency f, to calculate the acoustic signal of enhanced speech sound signal of the target speech emphasized among the observed signal. Specifically, the speech enhancement unit 15d uses the beamformer w _f calculated using the steering vector of the target speech as an initial value, as shown in the following equation (13), the observation vector and the beamformer w _f Is used to form a beam of emphasized speech.

  The speech recognition unit 15e estimates the calculated probability distribution of the phoneme of the emphasized speech and assigns a reference label indicating the phoneme to the enhanced speech.

  Here, in the following description, an observation signal recorded by M microphones is expressed as shown in the following equation (14).

Also, the observation vector y _f, a _t, short-time discrete Fourier transform and short time discrete signal feature quantity of each time-frequency determined by applying the short signal analysis cosine transform or the like Y _{m, f,} using the _t Is expressed as the following equation (15).

  In this case, the speech recognition unit 15e performs the calculation represented by the following equation (16), and the phoneme probability distribution (hereinafter, the phoneme posterior probability) of the emphasized speech obtained by the above equation (13). (Also referred to as phoneme state posterior probability).

  Here, the emphasized speech is expressed by a vector represented by the following equation (17) using the enhanced speech at each frequency.

  Specifically, the speech recognition unit 15e receives the enhanced speech represented by the above formula (17) from the speech enhancement unit 15d, and performs linear computation and nonlinear computation multiple times using the initial values of the parameters learned in advance. Repeatedly, the posterior probability of the phoneme at each time expressed by the following equation (18) is output.

  Moreover, the speech recognition unit 15e assigns a binary reference label indicating the phoneme of the emphasized speech at each time, as represented by the following equation (19).

Optimization unit 15f is the difference between the phoneme probability distribution of the reference label and enhanced speech to minimize, to optimize the beamformer w _f. That is, the optimization unit 15f regards the network formed by the speech enhancement unit 15d and the speech recognition unit 15e as a network that outputs the phoneme state posterior probability of the enhanced speech when the observation vector is input, and optimizes the output.

  Specifically, the optimization unit 15f performs the following operation between the posterior probability of the phonemes at each time represented by the above formula (18) and the reference label of the phonemes at each time represented by the above formula (19). Using the cross entropy defined as shown in Equation (20) as an objective function, this objective function is minimized.

Here, by applying the steepest descent method, the update formula of the beamformer w _f is expressed as the following formula (21).

  In this case, as shown in the following equation (22), the gradient of the objective function can be calculated by applying the chain rule in the differential method and transforming it into an A × B shape.

  That is, the part A in the above equation (22) can be calculated by using a well-known method based on backpropagation applied to parameter estimation of the neural network. Further, the portion B of the above formula (22) can be calculated by the following formula (23) based on the above formula (13).

  Here, the processing of the optimization unit 15f will be described with reference to FIG. As illustrated in FIG. 3, the optimization unit 15 f includes a parameter initialization unit 156, a gradient calculation unit 157, a parameter update unit 158, and a convergence determination unit 159.

The parameter initialization unit 156 sets initial values of various parameters used for processing in the optimization unit 15f. For example, the parameter initialization unit 156 delivers the beam former w _f calculated by the estimation unit 15c to the gradient calculation unit 157 as an initial value. The parameter initializing unit 156 may hand over the unit vector in the gradient calculation section 157 as the initial value of the beamformer w _f. Further, the parameter initialization unit 156 passes the learning rate α used in the above equation (21) to the parameter update unit 158.

The gradient calculation unit 157 calculates the gradient represented by the above equation (22), and transfers the transferred initial value of the beamformer w _f and the calculated gradient to the parameter update unit 158. Parameter updating unit 158 receives the gradient learning rate α and a beamformer w _f, and calculates the updated value of the beamformer w _f by using equation (21), calculated beamformer w _f convergence determination unit Deliver to 159.

The convergence determination unit 159 determines whether or not a predetermined convergence condition is satisfied. Examples of the convergence condition include that the number of iterations of the update equation shown in the above equation (21) satisfies a predetermined number, or that the objective function shown in the above equation (17) converges. . If the convergence condition is not satisfied, the convergence determination unit 159 passes the beam former w _f received from the parameter update unit 158 to the gradient calculation unit 157. The gradient calculation unit 157 repeats the above processing using the beam former w _f received from the convergence determination unit 159 as an initial value. Thereby, the parameter update unit 158 calculates an update value of the beamformer w _f until a predetermined convergence condition is satisfied.

This means that the beam former w _f updated optimally for speech recognition is derived when a predetermined convergence condition is satisfied. In this case, the convergence determination unit 159 passes the updated value of the beamformer w _f directly or via the estimation unit 15c to the speech enhancement unit 15d.

Incidentally, speech enhancement unit 15d that has received the best updated beamformer w _f in speech recognition, with reference to the beamformer w _f, as shown in the equation (13), the optimum enhancement speech for speech recognition A beam is formed, and the output unit 12 realized by a speaker or the like outputs emphasized speech.

[Formation processing]
Next, the forming process of the forming apparatus 1 will be described with reference to FIG. FIG. 4 is a flowchart showing a forming process procedure of the forming apparatus 1. The flowchart of FIG. 4 is started, for example, at a timing when there is an operation input instructing the start of processing.

  First, the acquisition unit 15a acquires multi-channel observation signals recorded by microphones installed at a plurality of points including an acoustic signal of a target voice that is a target of voice recognition and an acoustic signal of noise other than the target voice. (Step S1).

Next, using the observation vector generated by the time-frequency analysis unit 15b from the observation signal, the estimation unit 15c estimates a steering vector including the spatial information of the sound source of the target speech (step S2). In addition, the estimation unit 15c calculates a beam former w _f that forms a beam of enhanced speech that enhances the acoustic signal in the steering vector direction, using the estimated steering vector.

Speech enhancement unit 15d, using the beamformer w _f calculated using the estimated steering vector, and calculates the acoustic signal of the enhanced speech (step S3).

  Next, the voice recognition unit 15e performs voice recognition of the calculated emphasized voice (step S4). That is, the speech recognition unit 15e estimates the phoneme probability distribution of the emphasized speech. In addition, the voice recognition unit 15e gives a reference label indicating a phoneme to the emphasized voice.

Optimization unit 15f is the difference between the phoneme probability distribution of the reference label and enhanced speech to minimize, to optimize the beamformer w _f. That is, the optimizing unit 15f, by deriving a beamformer w _f for optimizing speech recognition enhanced speech, to optimize the enhanced speech (step S5).

  Further, the output unit 12 outputs the optimized emphasized speech (step S6). Thereby, a series of forming processes is completed.

As described above, in the forming apparatus 1 of the present embodiment, the acquisition unit 15a includes a plurality of points including the acoustic signal of the target speech that is the target of speech recognition and the acoustic signal of noise other than the target speech. Obtain observation signals. The speech enhancement unit 15d may calculate the predetermined using beamformer w _f, acoustic signals of enhanced speech emphasizing an acoustic signal that purpose voice of observed signals for forming a beam of acoustic signals for each frequency To do. In addition, the speech recognition unit 15e estimates the calculated phoneme probability distribution of the emphasized speech, and assigns a reference label indicating the phoneme to the enhanced speech. Moreover, the optimization section 15f is the difference between the phoneme probability distribution of the reference label and enhanced speech to minimize, to optimize the beamformer w _f.

  Accordingly, the forming apparatus 1 can optimally form a beam that suppresses noise and emphasizes the voice of the target voice for voice recognition. Therefore, it is possible to perform speech enhancement optimal for speech recognition according to the environment. For example, high-accuracy voice recognition can be performed during operations such as smartphone operations and searches under noisy, automatic transcription of conversations and lectures, and the like.

Note that the estimation unit 15c separates the probability distribution of the combination of signals that do not include the acoustic signal of the target speech from among the combinations of signals for each frequency f in the same short period t of the observation signal with a predetermined length. by estimating estimates the steering vector containing the spatial information of the target speech sound for every frequency to calculate a beamformer w _f using the estimated steering vector. As a result, the beam former w _f that forms a beam for suppressing noise can be calculated as the initial value of the beam former w _f used for the processing of the optimization unit 15 _f .

Moreover, the optimization section 15f may not update all beamformer w _f for each frequency f. For example, only a part of the frequencies f may be updated according to the background noise condition or the like. Alternatively, among the components of the beamformer w _f for each frequency f, only a part of the components of the vector may be updated. Thereby, the processing load of the formation process 1 can be reduced.

[Example]
Using the forming apparatus 1 according to the above-described embodiment, in an environment where background noise exists such as in a bus or a cafe, a speaker reads a voice reading a sentence toward the tablet, and M = 6 attached to the tablet. An experiment was conducted for recording with a single microphone. Here, the learning rate α is set to 6 × 10 ³ . The initial value of the beamformer w _f has a value determined so as to maximize the likelihood function shown in the equation (4). In addition, the number of repetitions of the beamformer w _f update equation shown in the above equation (21) is 30 times.

In this case, the word recognition error rate when speech recognition was performed without using the forming apparatus 1 was 16.80%. This contrast, word recognition error rate in the case of performing speech recognition of the initial value due to enhanced speech before the beamformer w _f for performing a process by the optimizing unit 15f was 9.06%. In addition, the word recognition error rate when the speech recognition of the emphasized speech was performed by the beamformer updated by the optimization unit 15f was 8.89%. Thus, the effect of the forming process by the forming apparatus 1 of the present embodiment was confirmed.

[program]
It is also possible to create a program in which processing executed by the forming apparatus 1 according to the above embodiment is described in a language that can be executed by a computer. As an embodiment, the forming apparatus 1 can be implemented by installing a forming program for executing the forming process as package software or online software on a desired computer. For example, the information processing apparatus can function as the forming apparatus 1 by causing the information processing apparatus to execute the above forming program. The information processing apparatus referred to here includes a desktop or notebook personal computer. In addition, the information processing apparatus includes mobile communication terminals such as smart phones, mobile phones and PHS (Personal Handyphone System), and slate terminals such as PDA (Personal Digital Assistants). In addition, a terminal device used by a user can be a client, and the client can be implemented as a server device that provides services related to the above-described formation processing to the client. For example, the forming apparatus 1 is implemented as a server apparatus that provides a forming process service that receives an observation signal and outputs an emphasized voice. In this case, the forming apparatus 1 may be implemented as a Web server, or may be implemented as a cloud that provides services related to the above-described forming processing by outsourcing. Hereinafter, an example of a computer that executes a forming program that realizes the same function as the forming apparatus 1 will be described.

  As shown in FIG. 5, the computer 1000 that executes the forming program includes, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface. 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1031. The disk drive interface 1040 is connected to the disk drive 1041. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041. For example, a mouse 1051 and a keyboard 1052 are connected to the serial port interface 1050. For example, a display 1061 is connected to the video adapter 1060.

  Here, as shown in FIG. 5, the hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. Each table described in the above embodiment is stored in the hard disk drive 1031 or the memory 1010, for example.

  Further, the forming program is stored in the hard disk drive 1031 as a program module 1093 in which a command executed by the computer 1000 is described, for example. Specifically, a program module 1093 describing each process executed by the forming apparatus 1 described in the above embodiment is stored in the hard disk drive 1031.

  Data used for information processing by the forming program is stored as program data 1094 in, for example, the hard disk drive 1031. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the hard disk drive 1031 to the RAM 1012 as necessary, and executes the above-described procedures.

  Note that the program module 1093 and the program data 1094 related to the formation program are not limited to being stored in the hard disk drive 1031, but are stored in, for example, a removable storage medium and read out by the CPU 1020 via the disk drive 1041 or the like. May be. Alternatively, the program module 1093 and the program data 1094 related to the formation program are stored in another computer connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network), and are transmitted via the network interface 1070. It may be read by the CPU 1020.

  As mentioned above, although embodiment which applied the invention made | formed by this inventor was described, this invention is not limited with the description and drawing which make a part of indication of this invention by this embodiment. That is, other embodiments, examples, operational techniques, and the like made by those skilled in the art based on this embodiment are all included in the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Formation apparatus 11 Input part 12 Output part 13 Communication control part 14 Storage part 15 Control part 15a Acquisition part 15b Time frequency analysis part 15c Estimation part 15d Speech enhancement part 15e Speech recognition part 15f Optimization part

Claims (5)

An acquisition unit for acquiring observation signals at a plurality of points including an acoustic signal of a target speech that is a target of speech recognition and an acoustic signal of noise other than the target speech;
Using a predetermined beamformer for forming a beam of an acoustic signal for each frequency, a speech enhancement unit that calculates an acoustic signal of an enhanced speech in which the acoustic signal of the target speech is enhanced among the observation signals;
A speech recognition unit that estimates a probability distribution of the calculated phoneme of the emphasized speech and assigns a reference label indicating the phoneme to the emphasized speech;
An optimization unit for optimizing the beamformer so as to minimize a difference between the reference label and a phoneme probability distribution of the emphasized speech;
A forming apparatus comprising: Furthermore, by separating and estimating the probability distribution of the combination of signals that do not include the acoustic signal of the target speech among the combinations of signals for each frequency in the same short period of the predetermined length of the observation signal, Estimating a steering vector including spatial information of the sound source of the target speech for each frequency, and comprising an estimation unit that calculates a beamformer using the steering vector,
The speech enhancement unit calculates an acoustic signal of an enhanced speech in which the acoustic signal of the target speech is enhanced among the observation signals using the calculated beam former as an initial value. The forming apparatus as described.   The forming apparatus according to claim 1, wherein the optimization unit optimizes the beam former with respect to a part of frequencies or a part of components of a vector. A forming method executed by a forming apparatus,
An acquisition step of acquiring observation signals at a plurality of points including an acoustic signal of a target speech that is a target of speech recognition and an acoustic signal of noise other than the target speech;
Using a predetermined beamformer for forming a beam of an acoustic signal for each frequency, a speech enhancement step of calculating an acoustic signal of an enhanced speech in which the acoustic signal of the target speech is enhanced among the observation signals;
A speech recognition step of estimating the calculated probability distribution of the phoneme of the emphasized speech and adding a reference label indicating the phoneme to the emphasized speech;
An optimization step of optimizing the beamformer to minimize the difference between the reference label and the phonetic probability distribution of the enhanced speech;
The formation method characterized by including. An acquisition step of acquiring observation signals at a plurality of points including an acoustic signal of a target speech that is a target of speech recognition and an acoustic signal of noise other than the target speech;
Using a predetermined beamformer for forming a beam of acoustic signals for each frequency, a speech enhancement step of calculating an acoustic signal of enhanced speech in which the acoustic signal of the target speech is enhanced among the observed signals;
A speech recognition step of estimating the calculated probability distribution of the phoneme of the emphasized speech and adding a reference label indicating the phoneme to the emphasized speech;
An optimization step of optimizing the beamformer so as to minimize the difference between the reference label and the phoneme probability distribution of the enhanced speech;
A program for causing a computer to execute.

Images