JP2019066339A - Diagnostic device, diagnostic method and diagnostic system each using sound - Google Patents

Abstract

To provide diagnosis allowing for abnormality detection even when accuracy of sound source separation is not sufficient.SOLUTION: A diagnostic device performing diagnosis using sound is provided that comprises: a signal acquisition unit which acquires a sound signal being an electric signal acquired by converting sound and which outputs the sound signal; a preprocessing unit which converts the sound signal output by the signal acquisition unit to a frequency domain signal; a spatial correlation calculation unit which calculates a spatial correlation matrix on the basis of the frequency domain signal converted by the preprocessing unit; a spatial correlation abnormality detection unit which determines abnormality on the basis of the spatial correlation matrix calculated by the spatial correlation calculation unit; and an abnormality display unit which displays information related to the abnormality on the basis of the determination of the abnormality performed by the spatial correlation abnormality detection unit.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a sound diagnostic apparatus, a diagnostic method, and a diagnostic system.

  The state of machines and people often appear in sounds and vibrations. Therefore, diagnosis based on sounds and vibrations generated from machines and people is important in order to grasp the state of machines and people. However, in the diagnosis based on sound and vibration, there is a problem that the diagnosis can be wrong. The cause is roughly divided into two.

  One is noise derived from other than the diagnosis target, which is an external factor. The other is an internal factor, that is, the shake of the normal state of the diagnosis object itself, that is, the sound and the vibration are different even between the normal states.

  As a method for solving these problems, Patent Document 1 discloses “A flow line estimation device for walking sound in an indoor space consisting of a microphone array device and a sensor information integration device, and the sensor information integration device further displays the indoor space. Display unit and signal processing unit, connected to the microphone array device, and arranged in a pair of microphone arrays in the indoor space in a "ha" shape, and the walking sound of the indoor space is one microphone array The walking sound analog signal is recorded in the A / D conversion and AD converted to generate a walking sound digital signal. The sound source position and arrival direction are estimated from the walking sound digital signal using the MUSIC method (step 1). The sound source is separated (step 2), the features are extracted from the separated sound source separated, the likelihood of the acoustic model is calculated, and the abnormal sound The number of people walking and the flow line are detected by the particle filter (step 3), and the estimated walking flow line is displayed on the display unit of the sensor information integration device (step 4). It is described as "flow line estimation device of walking sound".

JP, 2014-191616, A

  The device described in Patent Document 1 detects an abnormal sound in step 3 for the sound subjected to the sound source separation in step 2. However, in such a cascade configuration, there is a high possibility that the abnormality detection in the latter stage is erroneous when the accuracy of the sound source separation in the former stage is insufficient.

  Therefore, an object of the present invention is to provide a diagnosis capable of detecting an abnormality even when the accuracy of sound source separation is insufficient.

  In order to solve the above problems, for example, the configuration described in the claims is adopted. The present application includes a plurality of means for solving the above-mentioned problems, and an example thereof is a diagnostic device for diagnosing by sound, which acquires a sound signal which is an electrical signal converted from sound, and generates a sound signal. A spatial correlation matrix for calculating a spatial correlation matrix based on a signal acquisition unit to output, a pre-processing unit to convert a sound signal output from the signal acquisition unit into a frequency domain signal, and a frequency domain signal converted by the pre-processing unit Based on the calculation unit and the spatial correlation matrix calculated by the spatial correlation calculation unit, information on the abnormality is displayed based on the spatial correlation abnormality detection unit that determines the abnormality and the determination of the abnormality by the spatial correlation abnormality detection unit And an abnormality display unit.

  According to the present invention, it is possible to provide a diagnosis capable of detecting an abnormality even when the accuracy of sound source separation is insufficient. Problems, configurations, and effects other than those described above will be apparent from the description of the embodiments below.

It is a figure which shows the example of a diagnostic device. It is a figure which shows the example of a diagnostic process. It is a figure which shows the example of the information displayed as abnormality detection mode. It is a figure which shows the example of a screen. FIG. 7 is a diagram showing an example of a diagnostic device of a second embodiment. FIG. 7 is a diagram showing an example of a diagnostic process of the second embodiment. FIG. 16 is a diagram showing an example of information displayed as an abnormality detection mode according to the second embodiment. FIG. 16 is a diagram showing an example of a diagnostic system of a third embodiment.

  Hereinafter, a preferred example of the mode for carrying out the present invention will be described as an example using the drawings.

  FIG. 1 is a diagram showing an example of a diagnostic device 100, which may be a general computer. The processor 121 and the memory 122 may be a processor and a memory of a general computer, and the processor 121 executes a program stored in the memory 122 or the storage unit 126.

  The signal input unit 123 is an electronic circuit that inputs a sound signal. When connected to a microphone to input a sound analog electrical signal, the signal input unit 123 includes an ADC (Analog-Digital Converter). When connected to an ADC external to the diagnostic device 100 to input a digital signal of sound, the signal input unit 123 may not include the ADC.

  The signal input unit 123 may convert an analog voltage of the input digital signal, convert a data format, or convert a sampling frequency. In addition, when a sound signal is input via a network, the signal input unit 123 may be a network interface. Even when the signal input unit 123 inputs a digital signal, the source of the digital signal is a microphone, so an example in which a microphone is connected to the signal input unit 123 will be described below.

  The display unit 124 is, for example, a liquid crystal display device, and displays display data generated by the processor 121. The display unit 124 may be a network interface, and display data may be displayed on another computer from the display unit 124 which is a network interface via the network.

  The input unit 125 is, for example, a keyboard and a mouse or a touch panel, and is a user interface to which information is input by a user operation. The input unit 125 may also input information from the control unit of the machine to be diagnosed. The input information is processed by the processor 121.

  The input unit 125 may also be a network interface, and the input unit 125 which is a network interface may receive information input by another computer via a network.

  The storage unit 126 is, for example, a hard disk drive, a solid state drive, or a flash memory, and stores programs and data. The program and data stored in the storage unit 126 may be transferred to the memory 122, and the program and data stored in the memory 122 may be transferred to the storage unit 126.

  For this reason, it does not matter which of the storage unit 126 and the memory 122 the program is stored. Therefore, although the program is described below as the program stored in the storage unit 126, it is read as the program stored in the memory 122 It is also good. Among the programs stored in the storage unit 126, the program shown in FIG. 1 will be described using FIG.

  Storage unit 126 may store other programs. For example, as the other programs, a program for controlling the whole from signal acquisition program 101a to abnormality detection mode input program 112a, and a computer which is diagnostic device 100 basically An operating system (OS) for operation may be stored.

  The storage unit 126 may further store information, and may store threshold information used for determination in program execution. Further, a database may be configured in the storage unit 126, and information may be accumulated in the database. Information on the input signal space correlation matrix, which will be described later with reference to FIG. 2, may be stored in the database.

  In addition to the examples illustrated in FIG. 1, the diagnostic device 100 may include other hardware, such as a network interface or a reader of a storage medium. The program and information stored in the storage unit may be input by a network interface (not shown) or may be input by a reader of a storage medium. Diagnostic device 100 may communicate with other devices via a network interface.

  FIG. 2 is a diagram showing an example of the diagnostic process. The signal acquisition unit 101 is a processor 121 that executes the signal acquisition program 101 a and a signal input unit 123. The signal acquisition unit 101 acquires sound signals from the M microphones as M channel analog signals, converts the M channel analog signals into M channel digital signals, and outputs the converted signals to the next preprocessing unit 102.

  The microphones may be linear, circumferential, or various other arrangements. However, in the present embodiment, in particular, it is desirable that they are not equally spaced. Since the frequencies that are good (without spatial aliasing and high in the accuracy of direction estimation) differ depending on the distance between the microphones, when the microphones are not equally spaced, signals can be efficiently acquired at various frequencies. Note that M is preferably an integer of 3 or more.

  The preprocessing unit 102 is a processor 121 that executes the preprocessing program 102 a. The pre-processing unit 102 divides the M channel digital signal into frames, multiplies the frame by a window function, performs a short time Fourier transformation on the signal after the window function multiplication, and generates an M channel frequency domain signal as an input signal. This information is output to the spatial correlation calculation unit 103, the per-sound source space correlation calculation unit 105, and the sound source separation unit 107.

  Here, if the frame size is N, the M channel frequency domain signal is a set of K × M complex numbers in which M complex numbers correspond to each of (N / 2 + 1) = K frequency bins.

  The input signal space correlation calculation unit 103 is a processor 121 that executes the input signal space correlation program 103a. The input signal space correlation calculation unit 103 calculates the input signal space correlation matrix for each frequency k based on the M channel frequency domain signal for each frequency k, and estimates the input signal space correlation matrix for each frequency k as the sound source presence direction cluster It outputs to the section 104 and the input signal space correlation abnormality detection section 108.

  Here, the spatial correlation matrix is a time average of a matrix of multiplication results of M channel frequency domain signal vectors x = [x_1,..., X_M] ^ T and x ^ H. However, ^ H represents conjugate transposition. The time average may be an arithmetic average during a certain T frame or may be a forgetting average.

  The abnormality detection mode input unit 112 is a processor 121 that executes the abnormality detection mode input program 112 a and an input unit 125. The abnormality detection mode input unit 112 receives an abnormality detection mode input by a user operation or the like. The mode of abnormality detection is, for example, (1) presence or absence of movement of noise sound source, (2) presence or absence of movement of target sound source in a normal state, and (3) normal operation state of a diagnosis target machine.

  Here, (1) the presence or absence of movement of the noise source is a mode relating to an abnormality detection display. (2) The presence or absence of movement of the target sound source in the normal state is a mode related to an abnormality detection display, and may be input from the control unit of the machine to be diagnosed. (3) The normal operating state of the diagnosis target machine is a mode for accumulating information at the normal time without detecting an abnormality, and may be input from the control unit of the diagnosis target machine.

  The input signal space correlation abnormality detection unit 108 is a processor 121 that executes the input signal space correlation abnormality detection program 108 a, and detects an abnormality based on the input signal space correlation for each frequency k.

  The input signal space correlation abnormality detection unit 108 calculates the degree to which the calculated input signal space correlation matrix for each frequency k is similar to the input signal space correlation matrix for each frequency k at the normal time stored in the database. If the calculated first similarity is higher than a preset first threshold, the result of determination as normal is output, and if the first similarity is low, the result of determination as abnormal is output.

  The input signal space correlation matrix for each frequency k at the normal time is (3) the input signal space correlation matrix for each frequency k accumulated according to the normal operating state of the diagnosis target machine input from the abnormality detection mode input unit 112 Is used.

  Anomaly detection via sound source separation, which will be described later, is a sound source when the directions between sound sources are too close, when there are multiple parts of the same type and the independence between sound sources is too low, or when the noise is too large. As the separation accuracy deteriorates, the abnormality detection accuracy also significantly decreases.

  However, even in those cases, since the input signal space correlation matrix changes due to an abnormality in the target sound, the input signal space correlation abnormality detection unit 108 can detect an abnormality even if the accuracy of the sound source separation deteriorates. It plays an effect.

  The input signal space correlation abnormality detection unit 108 compares the input signal space correlation matrix to be diagnosed with the input signal space correlation matrix at normal time, for example, by vectorizing a space correlation matrix of K frequency bins.

  That is, since the spatial correlation matrix is a Hermitian matrix, the comparison of the first similarity between vectors having K × M × (M−1) / 2 components in which only upper triangle and diagonal components are extracted I do. By reducing the number of dimensions in this manner, the influence of over-learning can be reduced, and the amount of calculation can be reduced.

  Let v be the vectorization of the input signal space correlation matrix to be diagnosed, and w be the vectorization of the normal input signal space correlation matrix. As the first similarity, for example, one obtained by multiplying the square of the Euclidean distance between the average vector of w and v by -1 can be used. In this case, the effect that abnormality detection can be performed at high speed can be expected.

  Also, as the first similarity, w can be fitted to a multivariate complex Gaussian distribution, and the log likelihood of a probability density function in which the multivariate complex Gaussian distribution as a fitting result generates v can be used. When the volume sensitivity differs greatly among multiple microphones, or when the installation intervals of multiple microphones differ significantly, using the above-mentioned simple Euclidean distance is likely to cause error in anomaly detection, but using a multivariate complex Gaussian distribution Since it is possible to absorb and learn the blurring of the object, it is possible to expect the effect that correct abnormality detection becomes possible.

  Also, as the first similarity, w can be fitted to a complex mixture Gaussian distribution, and the log likelihood of the probability density function in which the complex mixture Gaussian distribution of the fitting result generates v can be used. When there are multiple sound sources in the normal state, it can not be modeled with the above multivariate complex Gaussian distribution, so using the multivariate complex gaussian distribution is likely to make error detection abnormal, but using the complex mixed Gaussian distribution makes multiple sound sources Since it can be modeled, the effect that correct abnormality detection becomes possible can be expected.

  Besides, as a method of modeling w in a normal state, general 1 class support vector classifier, subspace method, local subspace method, k-means clustering, Deep Neural Network (DNN) autoencoder, Convolutional Neural Network (CNN) ) Autoencoder, Long Short Term Memory (LSTM) autoencoder, variational autoencoder (VAE), etc. may be used.

  The sound source existing direction cluster estimation unit 104 is a processor 121 that executes the sound source existing direction cluster estimation program 104 a. The sound source presence direction cluster estimation unit 104 estimates a sound source presence direction cluster in the T frame used in the calculation of the spatial correlation matrix based on the input signal space correlation matrix for each frequency k output from the input signal space correlation calculation unit 103. Do.

  First, based on the spatial correlation matrix, a frequency / direction histogram representing the magnitude of sound in each direction is estimated. As the estimation process, a general technique such as Minimum Variance Distortion Response (MVDR) beamformer or MUltiple SIgnal Classification (MUSIC) beamformer may be used.

  The frequency and direction histograms are integrated in the frequency direction to obtain a direction histogram. The integration process may be simply the sum of the values of the frequency and direction histograms, or may be the sum of logarithmic values of values obtained by adding a constant to the values of the frequency and direction histograms.

  If the magnitude of the sound in each direction of the direction histogram is larger than the preset fourth threshold, it is determined that the sound source is present in the corresponding direction, and if smaller than the fourth threshold, the sound source is detected in the corresponding direction. It determines that it does not exist. Then, there are a plurality of directions in which the sound source is present, and those directions in which the directions are sufficiently close are clustered.

  For clustering, general techniques such as aggregation clustering and k-means clustering may be used. Here, the number of clusters C is the number of sound sources. Each cluster c has a directional statistical sample mean direction and a directional statistical sample variance direction calculated with respect to the sound source presence direction belonging to the cluster, and von Mises defined by them It is determined by the distribution.

  The problem is solved by the use of directional statistical sample mean direction and directional statistical sample variance, as there is the problem that the errors are large if the frequency values or the mean or variance of the angular values are used.

  The per-sound source space correlation calculation unit 105 is a processor 121 that executes the per-sound source space correlation calculation program 105a, and the per-sound source space correlation matrix R_c (c = 1) based on the M channel frequency domain signal and the sound source presence direction cluster. ,..., C), and output the per-source space correlation matrix to the per-source separation filter updating unit 106 and the per-source spatial correlation abnormality detection unit 109.

  Specifically, first, direction estimation is performed independently for each frame and each frequency bin of the M channel frequency domain signal. At this time, a steering vector is previously calculated for each direction according to the microphone arrangement.

  For this calculation, for example, M. Togami, Y. Obuchi, and A. Amano, “Automatic speech recognition of human-symbiotic robot emiew,” in Human-Robot Interaction, Nilanjan Sarkar, Ed., Pp. 395-404 The process disclosed in I-tech Education and Publishing, 2007. may be used.

  It is assumed that the direction in which the inner product of the vector obtained by normalizing the M channel frequency domain signal and the steering vector is the source direction of the frame / frequency. The likelihood generated from the von Mises distribution whose source direction is linked to the aforementioned source presence direction cluster c is calculated, and if the likelihood is sufficiently high, the frame frequency is assigned to c.

  Then, the per-source spatial correlation matrix R_c is updated by time averaging of only the M channel frequency domain signal vector x of the frame / frequency assigned to c. The time averaging process is calculated using the arithmetic mean or forgetting mean of the product x ^ H of x and x ^ H as in the above-described input signal space correlation matrix.

  The per-sound source spatial correlation abnormality detection unit 109 is a processor 121 that executes the per-sound source space correlation abnormality detection program 109a, and the normal per-sound source signal space correlation matrix R_c for each calculated frequency k is stored in the database Calculate the degree of similarity with the per-source spatial correlation matrix for each frequency k, and if the calculated second similarity is higher than a preset second threshold, output the judgment result as normal. If the second similarity is low, the result of the judgment of abnormality is output.

  The per-sound source spatial correlation matrix for each frequency k at the time of normal operation is input from the abnormality detection mode input unit 112 (3) the per-sound source spatial correlation matrix for each frequency k accumulated according to the normal operating state of the diagnosis target machine Is used.

  As described above, the input signal space correlation abnormality detection unit 108 utilizes the fact that the input signal space correlation matrix changes due to the abnormality of the target sound even under the condition that the accuracy of the sound source separation deteriorates. However, since the input signal space correlation matrix changes only when noise that is not the target sound changes, the input signal space correlation abnormality detection unit 108 has a low accuracy of abnormality detection in the presence of noise.

  On the other hand, since the per-sound source space correlation abnormality detection unit 109 uses per-sound source space correlation decomposed for each of the target sound and noise, there is an effect that abnormality detection is possible even under the condition where noise is present.

  The comparison between the sound source spatial correlation matrix to be diagnosed and the normal sound source space correlation matrix at the normal time is performed, for example, as a vectorization of the spatial correlation matrix of K frequency bins, like the above-mentioned input signal space correlation anomaly detection unit 108. To carry out.

  That is, since the spatial correlation matrix is a Hermitian matrix, the comparison of the second similarity between vectors having K × M × (M−1) / 2 components in which only upper triangle and diagonal components are extracted I do. By reducing the number of dimensions in this manner, the influence of over-learning can be reduced, and the amount of calculation can be reduced.

  Let v be a vectorization of the per-source spatial correlation matrix to be diagnosed, and w be a vector of the per-source spatial correlation matrix in normal operation. As the second similarity, for example, one obtained by multiplying the square of the Euclidean distance between the average vector of w and v by -1 can be used. In this case, the effect that abnormality detection can be performed at high speed can be expected.

  Also, as the second similarity, w can be fitted to a multivariate complex Gaussian distribution, and the log likelihood of a probability density function in which the multivariate complex Gaussian distribution as a fitting result generates v can be used. When the volume sensitivity differs greatly among multiple microphones, or when the installation intervals of multiple microphones differ significantly, using the above-mentioned simple Euclidean distance is likely to cause error in anomaly detection, but using a multivariate complex Gaussian distribution Since it is possible to absorb and learn the blurring of the object, it is possible to expect the effect that correct abnormality detection becomes possible.

  Also, as the second similarity, w can be fitted to a complex mixture Gaussian distribution, and the log likelihood of the probability density function in which the complex mixture Gaussian distribution of the fitting result generates v can be used. When multiple sound sources exist in the same direction in normal times, modeling can not be performed with the above-described multivariate complex Gaussian distribution, so using the multivariate complex Gaussian distribution is likely to cause errors in anomaly detection, but using complex mixed Gaussian distribution Since a plurality of sound sources can be modeled, an effect that correct abnormality detection can be performed can be expected.

  Besides, as a method of modeling w in a normal state, a general one class support vector classifier, subspace method, local subspace method, k-means clustering, DNN autoencoder, CNN autoencoder, LSTM autoencoder, VAE etc. are used. It may be done.

  The sound source separation filter updating unit 106 is a processor 121 that executes the sound source separation filter update program 106a, and calculates the sound source separation filter based on the spatial correlation matrix of each sound source. The sound source separation filter is, for example, a general Generalized EigenValue (GEV) beamformer. GEV beamformer uses generalized eigenvectors e as a sound source separation filter when R_n is a spatial correlation matrix of noise, and R_x is a spatial correlation matrix of a target sound.

That is,
R_ne = λR_xe
However, when the sound source presence direction cluster c ′ is the target sound direction and the other clusters are the noise direction, R_x and R_n can be calculated as follows.

R_x = R_c '
R_n = Σ_ {c ≠ c '} R_c
In addition, since the scale of e is indefinite, let e 'which performed general normalization, such as Blind Analytic Normalization (BAN), be a final sound source separation filter.

  A general sound source separation filter such as MVDR beamformer may be used instead of GEV beamformer. Since these sound source separation filters are linear filters, there is an advantage that no distortion occurs in the sound source separation signal.

  The sound source separation unit 107 is a processor 121 that executes the sound source separation program 107a, performs sound source separation by applying a sound source separation filter to the M channel frequency domain signal, and outputs a sound source separation signal.

  The sound source separation signal abnormality detection unit 110 is a processor 121 that executes the sound source separation signal abnormality detection program 110a, and first calculates a feature amount vector based on the sound source separation signal. The feature quantity vector is composed of, for example, the power spectrum of the sound source separation signal, the amplitude cepstrum, and the mel frequency cepstrum coefficient (MFCC).

  Then, the sound source separation signal abnormality detection unit 110 calculates the degree to which the calculated feature quantity vector is similar to the normal feature quantity vector stored in the database, and the calculated third similarity is previously calculated. If it is higher than the set third threshold value, the judgment result that it is normal is outputted, and if the third similarity is low, the judgment result that it is abnormal is outputted.

  As the feature vector at the normal time, the feature vector stored in accordance with the normal operating state of the (3) diagnosis target machine input from the abnormality detection mode input unit 112 is used. Since the sound source separation signal is a sound from which noise is removed and only the target sound is extracted, it is possible to improve the accuracy of the abnormality detection compared to the case where the abnormality detection is performed on the input signal.

  The abnormality display unit 111 is a processor 121 that executes the abnormality display program 111a and the display unit 124. The abnormality display unit 111 has (1) presence / absence of movement of noise sound source and (2) purpose of normal state input from the abnormality detection mode input unit 112. According to the presence or absence of movement of the sound source, the determination result of the presence or absence of abnormality input from the input signal space correlation abnormality detection unit 108, the sound source space correlation abnormality detection unit 109, and the sound source separation signal abnormality detection unit 110 is displayed.

  FIG. 3 is a diagram showing an example of information displayed as the abnormality detection mode. The abnormality detection mode 301 includes (1) a mode 302 of presence / absence of movement of a noise source and (2) a mode 303 of presence / absence of movement of a target sound source in a normal state. Information 304 to be displayed is determined according to any of the four combinations of "none".

  Information 304 to be displayed is information as to which of the determination results of the input signal space correlation abnormality detection unit 108, the sound source space correlation abnormality detection unit 109, or the sound source separation signal abnormality detection unit 110 is to be displayed. Sometimes the results are displayed side by side.

  If the displayed information is the same regardless of the combination of the mode 302 and the mode 303, if it is displayed as abnormal, it is difficult for the user to know whether it is really abnormal or not. Such display switching has the effect of solving this problem.

  FIG. 4 is a diagram showing an example of the screen of the display unit 124. As shown in FIG. For example, if the determination result input from the input signal space correlation abnormality detection unit 108 indicates an abnormality, the abnormality display unit 111 may display the message 401 and the message 402, or the message 401 or the message 402 on the display unit 124. The message 401 and the message 402 convey that the sound source positions are different.

  The message 401 and the message 402, or the message 401 or the message 402 may be displayed on the display unit 124 also when the determination result input from the sound source space correlation abnormality detection unit 109 indicates an abnormality.

  When the determination result input from the sound source separation signal abnormality detection unit 110 indicates an abnormality, the abnormality display unit 111 displays the message 403 on the display unit 124, and determines that the sound is an abnormality based on the feature amount of the sound. Tell that.

  Which of the messages 401 and 402 and the message 403 is displayed on the display section 124 is as shown in the information 304 shown in FIG. 3, and the abnormality display section 111 has modes 302 and 303. Select according to the combination with

  The condition that the accuracy of the sound source separation deteriorates is when the directions of the sound sources are too close, or when there are multiple parts of the same type and the sound quality independence between the sound sources is too low, or the noise is too large. The spatial correlation matrix also changes under these conditions.

  The diagnostic processing based on sound and vibration in the present embodiment is, in addition to the sound source separation signal abnormality detection unit 110 via the sound source separation signal, the input signal space correlation abnormality detection unit 108 and abnormality detection not via the sound source separation signal. Since each sound source space correlation abnormality detection unit 109 is also provided, an effect that abnormality detection is possible even when the accuracy of the sound source separation is deteriorated is exhibited.

  A first threshold used by the input signal space correlation anomaly detection unit 108 for the first similarity comparison, and a second threshold used by the sound source space correlation anomaly detection unit 109 for the second similarity comparison; The third threshold used by the sound source separation signal abnormality detection unit 110 for the third similarity comparison may be a different value.

  These three threshold values are not directly comparable values as they are because they are different in the measure of the three similarities, but normalize the input signal spatial correlation that is the basis of the three similarities or the calculation of the three similarities. It may be a value that can be compared. When the three threshold values can be compared as described above, the first threshold value may be higher than the second threshold value, and the second threshold value may be higher than the third threshold value.

  When three thresholds are set in the diagnostic device 100, the three thresholds are compared with each other, the first threshold is higher than the second threshold, and the second threshold is higher than the third threshold. In addition to the case, a warning may be displayed to prompt resetting.

  In addition, a signal having a statistically sufficient number of samples is acquired including the final conclusion that it is abnormal and obtained statistically, and the occurrence probability that the first similarity is equal to or more than the first threshold is The first threshold and the second threshold are set such that the second similarity is lower than the occurrence probability that the second similarity is equal to or higher than the second threshold, and the occurrence probability that the second similarity is equal to or higher than the second threshold is The second threshold and the third threshold may be set such that the third similarity is lower than the occurrence probability that the third similarity is equal to or higher than the third threshold.

  Furthermore, a signal having a statistically sufficient number of samples is obtained and statistically processed, including the case where it is finally concluded that it is abnormal, and the probability density function of the first similarity and the second similarity are obtained. And the third similarity density probability density function are respectively normalized, the first threshold value is set to a value higher than the second threshold value, and the second threshold value is higher than the third threshold value. It may be set to

  In the second embodiment, an example of a diagnostic process that enables abnormality detection with higher accuracy than the first embodiment will be described even if the accuracy of sound source separation is insufficient and the number of sound sources is larger than the number of microphones. In the second embodiment, compared to the first embodiment, the frequency / direction power signal is calculated, signal separation is performed on the frequency / direction power signal, and the frequency / direction power signal after separation is used for abnormality detection. Is a different example.

  FIG. 5 is a diagram showing an example of a diagnostic device 500 of the second embodiment. The same components as those of the diagnostic device 100 shown in FIG. The storage unit 126 does not store the sound source presence direction cluster estimation program 104 a and the abnormality display program 111 a.

  Instead, the storage unit 126 includes a frequency / direction power calculation program 501a, a frequency / direction power signal separation program 502a, a frequency / direction power abnormality detection program 503a, a sound source presence direction cluster estimation program 504a, and an abnormality display program 505a. It is stored.

  FIG. 6 is a diagram illustrating an example of a diagnosis process of the second embodiment. The frequency / direction power calculation unit 501 is a processor 121 that executes the frequency / direction power calculation program 501a, and for each frame t, for every frame t, for every frequency k, based on the M channel frequency domain signal for every frequency k. Calculate the power X (t, k, d) for each d.

  Specifically, first, direction estimation is performed independently for each frame and each frequency bin of the M channel frequency domain signal. For this purpose, the technology disclosed in the above-mentioned "Automatic speech recognition of human-symbiotic robot emiew" or the like may be used. Then, steering vectors are calculated in advance for each direction d in accordance with the arrangement of the microphones.

  It is assumed that the direction d in which the inner product of the vector obtained by normalizing the M channel frequency domain signal and the steering vector is the highest is the sound source direction corresponding to the frame / frequency. Then, the power of the frame / frequency component is taken as the frequency / direction power X (t, k, d).

  The sound source existing direction cluster estimation unit 504 is a processor 121 that executes the sound source existing direction cluster estimation program 504a, and the power X (t for each frequency k and direction d for each frame t output by the frequency / direction power calculator 501). , K, d) to estimate a sound source presence direction cluster for each direction in the T frame. First, Y (k, d) obtained by integrating X (t, k, d) in T frame is calculated.

  Further, Y (k, d) is integrated in the frequency direction k = 1,..., K to obtain a direction histogram Z (d). The integration process may be simply the sum of the values of the frequency and direction histograms, or may be the sum of logarithmic values of values obtained by adding a constant to the values of the frequency and direction histograms.

  If the magnitude of the sound in each direction of the direction histogram is larger than the preset threshold, it is determined that the sound source is present in the corresponding direction, and if smaller than the preset threshold, the sound source is present in the corresponding direction It decides that it does not do. Then, there are a plurality of directions in which the sound source is present, and those directions in which the directions are sufficiently close are clustered.

  For clustering, general techniques such as aggregation clustering and k-means clustering may be used. Here, the number of clusters C is the number of sound sources. Each cluster c has a directional statistical sample mean direction and a directional statistical sample variance direction calculated with respect to the sound source presence direction belonging to the cluster, and the von Mises distribution defined by them Determined by

  Since there is a problem that the error is large when the frequency value of the angle value or the mean or the dispersion of the radian is used, this problem is solved by using the directional statistical sample mean and the directional statistical sample dispersion.

  When the time / frequency sparsity of the sound source is high, the direction histogram calculated by the sound source existing direction cluster estimation unit 504 has a sharp directivity as compared with the direction histogram of the sound source existing direction cluster estimation unit 104 of the first embodiment. There is.

  Therefore, when the time / frequency sparsity of the sound source is high, the sound source presence direction cluster estimation unit 504 has an effect that the estimation accuracy is higher than that of the sound source presence direction cluster estimation unit 104 of the first embodiment.

  The frequency / direction power signal separation unit 502 is a processor 121 that executes the frequency / direction power signal separation program 502a, and the power X for each frequency k and for each direction d for each frame t output by the frequency / direction power calculation unit 501. Signal separation is performed for (t, k, d), and frequency / direction power after separation is output.

First, X (t, k, d)) is converted into a matrix Q (t, a) in which the frequency k and the direction d are one axis. Specifically, the index a is a = d × K + k
Defined as
Q (t, a) = X (t, k, d)
Substitute.

  Next, signal separation is performed to extract frequency / direction power Q_b (t, a) corresponding to each base index b with Q (t, a) for T frames as an input. For signal separation, general supervised non-negative matrix factorization can be used.

  The basis is learned in advance with Q (t, a) in a normal state as an input. For basis learning, general learning methods such as multiplicative updating may be used. For initialization of basis learning, a general initialization method such as nonnegative independent component analysis may be used.

  The reason that Q (t, a) can be signal separated by supervised nonnegative matrix decomposition in this way is that if the sound of each base index b is uncorrelated, the frequency / direction power Q (t, a) and its component Q_b Uses the property that all values appearing in (t, a) are nonnegative and that in the normal state the frequency and direction power Q (t, a) is represented by a linear sum of a limited number of bases Because

  Note that, instead of supervised nonnegative matrix decomposition, Deep Neural Network (DNN) autoencoder, Convolutional Neural Network (CNN) autoencoder, Long Short Term Memory (LSTM) autoencoder or the like may be used.

Finally, the frequency and direction power after separation P (t, a) = Q (t, a) -Σ_bQ_b (t, a)
Calculate and output.

  This assumes that the component Σ_bQ_b (t, a) that can be expressed by the basis is a matrix that approximates Q (t, a) at the maximum within the range considered to be normal, and the approximation error P (t, a) Is assumed to be a component corresponding to abnormality.

  The frequency / direction power abnormality detection unit 503 is a processor 121 that executes the frequency / direction power abnormality detection program 503a, and the separated frequency / direction power P (t, a) is normally separated and stored in the database The degree of similarity with the subsequent frequency / direction power is calculated, and if the calculated similarity is higher than a preset threshold, the result of determination as normal is output, and if the similarity is low, determination as abnormal Output the result.

  The frequency / direction power after separation that is input according to the normal operation state of the diagnosis target machine (3) input from the abnormality detection mode input unit 112 is used as the frequency / direction power after separation during normal operation.

  As described above, the input signal space correlation abnormality detection unit 108 utilizes the fact that the input signal space correlation matrix changes due to the abnormality of the target sound even under the condition that the accuracy of the sound source separation deteriorates. However, since the input signal space correlation matrix changes only when noise that is not the target sound changes, the input signal space correlation abnormality detection unit 108 has a low accuracy of abnormality detection in the presence of noise.

  On the other hand, since the frequency / direction power abnormality detection unit 503 uses the frequency / direction power from which only the component corresponding to the abnormality is extracted, there is an effect that abnormality detection is possible even under the condition where noise is present. .

  The comparison between the frequency / direction power P (t, a) after separation of the frame t to be diagnosed and the frequency / direction power after separation in the normal state is, for example, a vector of K × D dimensions of P (t, a) Conduct as v. Where D is the number of discretized directions. The frequency / direction power after separation in the normal state is regarded as a vector v of K × D dimensions and is w.

  As the similarity, for example, one obtained by multiplying the square of the Euclidean distance between the average vector of w and v by -1 can be used. In this case, the effect that abnormality detection can be performed at high speed can be expected. Also, as the similarity, w can be fitted to a multivariate complex Gaussian distribution, and the log likelihood of a probability density function in which the multivariate complex Gaussian distribution as a fitting result generates v can be used.

  If the correlation is high between frequencies in a normal state, or if echo or reverberation is large, using the above-mentioned simple Euclidean distance is likely to cause error in anomaly detection, but using a multivariate complex Gaussian distribution will absorb these blurrings. Since the learning can be performed, an effect that correct abnormality detection can be performed can be expected.

  Also, as the similarity, w can be fitted to a complex mixture Gaussian distribution, and the log likelihood of the probability density function in which the complex mixture Gaussian distribution of the fitting result generates v can be used. When targeting a sound source that has multiple frequency patterns even during normal times, the above multivariate complex Gaussian distribution can not be modeled, so using multivariate complex Gaussian distribution is likely to make error detection abnormal, but complex mixture Gaussian If a distribution is used, a plurality of frequency patterns can be modeled, so that an effect that correct anomaly detection can be expected can be expected.

  Besides, as a method of modeling w in a normal state, a general one class support vector classifier, subspace method, local subspace method, k-means clustering, DNN autoencoder, CNN autoencoder, LSTM autoencoder, VAE etc. are used. It may be done.

  The abnormality display unit 505 is a processor 121 that executes the abnormality display program 505a and the display unit 124, and (1) presence / absence of movement of noise source and (2) purpose of normal state input from the abnormality detection mode input unit 112. Input signal space correlation abnormality detection unit 108, each sound source space correlation abnormality detection unit 109, sound source separation signal abnormality detection unit 110, input signal space correlation abnormality detection unit 108, and frequency / direction power abnormality according to the presence or absence of movement of the sound source The determination result of the presence or absence of abnormality input from the detection unit 503 is displayed.

  FIG. 7 is a diagram illustrating an example of information displayed as an abnormality detection mode according to the second embodiment. The abnormality detection mode 301, the mode 302, and the mode 303 are as described with reference to FIG. 3, and the information 704 to be displayed is different from the information 304 to be displayed as to which judgment result is to be displayed. It is.

  As in the example of FIG. 3, there is a problem that it is difficult for the user to know whether the information is displayed as abnormal if the displayed information is the same regardless of the combination of the mode 302 and the mode 303. Such display switching has the effect of solving this problem.

  In this embodiment, it is also possible to calculate the frequency / direction power signal, perform signal separation on the frequency / direction power signal, and use the separated frequency / direction power signal for abnormality detection.

  Signal separation for frequency / direction power signals is not based on phase-based beamforming between multiple microphones, but is a process based on differences in frequency / direction amplitude characteristics such as non-negative value matrix decomposition etc. Not limited by number. Therefore, even when the number of sound sources is larger than the number of microphones, it is possible to perform abnormality detection with higher accuracy than in the first embodiment.

  Diagnostic device 100 or diagnostic device 500 may be configured of a plurality of devices. FIG. 8 is a diagram showing an example of a diagnostic system configured by a plurality of devices. Although the following description will be made based on FIGS. 5 and 6, even in the case of FIGS. 1 and 2, the description will be based only on replacement of the corresponding programs or corresponding parts. Is omitted.

  Referring to FIG. 5, the signal analysis device 801 includes a frequency / direction power calculation program 501a, a frequency / direction power signal separation program 502a, and a sound source presence direction cluster estimation program 504a from the signal acquisition program 101a to the sound source separation program 107a. Is a computer (server) stored in the storage unit.

  In addition, the abnormality detection device 802 is a computer (server that stores the frequency / direction power abnormality detection program 503a and the abnormality display program 505a in the storage unit from the input signal space correlation abnormality detection program 108a to the sound source separation signal abnormality detection program 110a. ).

  Then, the signal analysis device 801 will be described based on FIG. 6 from the signal acquisition unit 101 to the sound source separation unit 107, the frequency / direction power calculation unit 501, the frequency / direction power signal separation unit 502, and the sound source presence direction cluster estimation. It is an apparatus provided with the unit 504.

  Further, the abnormality detection device 802 is a device provided with a frequency / direction power abnormality detection unit 503 and an abnormality display unit 505 from the input signal space correlation abnormality detection unit 108 to the sound source separation signal abnormality detection unit 110.

  As described with reference to FIGS. 2 and 6, the input signal space correlation calculation unit 103 outputs the input signal space correlation matrix, the sound source space correlation calculation unit 105 outputs the sound source space correlation matrix, and the sound source separation unit 107. Outputs a sound source separation signal, and the frequency / direction power signal separation unit 502 outputs the separated frequency / direction power. The signal analysis device 801 outputs these four pieces of information to the abnormality detection device 802 via the signal lines 811 to 814, respectively.

  When information is inputted from the signal analysis device 801 through the signal lines 811 to 814, the abnormality detection device 802 causes the input signal space correlation abnormality detection unit 108 and the sound source as described with reference to FIGS. Each space correlation abnormality detection unit 109, the sound source separation signal abnormality detection unit 110, and the frequency / direction power abnormality detection unit 503 calculate the degree of similarity for each information, and output the determination result to display the abnormality display unit. 505 displays the determination result.

  Here, each of the input signal space correlation abnormality detection unit 108, the individual sound source space correlation abnormality detection unit 109, the sound source separation signal abnormality detection unit 110, and the frequency / direction power abnormality detection unit 503 is used to calculate the similarity. The abnormality detection apparatus 802 inputs the information accumulated in the database from the normal model management apparatus 803 via the signal lines 821 to 824. Each threshold may be stored in the abnormality detection apparatus 802 or may be input from the normal model management apparatus 803.

  The normal model management device 803 is a computer (server) that accumulates information in a database, and the input signal space correlation matrix at normal time, the sound source space correlation matrix at normal time, the feature value vector at normal time, and the separation at normal time The subsequent frequency / direction powers are stored in the database in advance, and are output to the abnormality detection device 802 via the signal lines 821 to 824.

  In addition, the normal model management device 803 may obtain and store the information at the time of normal from the signal lines 811 to 814 outputted by the signal analysis device 801 in advance. For this purpose, the normal model management device 803 may machine-learn the information input via the signal lines 811 to 814.

  The abnormality detection mode input program 112 a may be stored in the storage unit of the abnormality detection device 802 or may be stored in the storage unit of the normal model management device 803. The abnormality detection mode input unit 112 may be included in the abnormality detection device 802 or in the normal model management device 803. Information on the abnormality detection mode may be transmitted from one of the abnormality detection device 802 and the normal model management device 803 to the other via the signal line 825.

  The signal analysis device 801, the abnormality detection device 802, and the normal model management device 803 may be connected by a network instead of the signal lines 811-814 and 821-825, and any two of the three devices may be integrated. May be one device.

  In addition, the signal analysis device 801 may be configured by a plurality of devices, that is, a plurality of computers by being divided in the middle of the flow of processing from the signal acquisition unit 101 to the sound source separation unit 107. By configuring the diagnostic system with a plurality of devices, hardware can be flexibly configured, and for example, devices can be assigned according to the processing load of each part.

  Furthermore, there may be a plurality of signal analysis devices 801 and abnormality detection devices 802, and information from the normal state may be distributed from one normal model management device 803 to a plurality of abnormality detection devices 802 via signal lines 821 to 824 or a network. . As a result, even when there are a plurality of diagnosis targets and a plurality of signal analysis devices 801, it is possible to unify and manage information at normal times.

  The present invention is not limited to the embodiments described above, but includes various modifications. For example, the embodiments described above are described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described.

  In addition, although each configuration described above has been described as the software implementation of the processor 121 that executes the program stored in the storage unit 126, a part or all of them may be implemented by hardware, for example, by designing with integrated circuits. You may

  Further, control lines and information lines (signal lines) indicate what is considered to be necessary for the description, and not all the control lines and information lines in the product are shown. In practice, almost all configurations may be considered to be mutually connected.

103 input signal space correlation calculation unit 104 sound source existing direction cluster estimation unit 105 per sound source space correlation calculation unit 106 sound source separation filter update unit 107 sound source separation unit 108 input signal space correlation abnormality detection unit 109 per sound source space correlation abnormality detection unit 110 sound source separation Signal abnormality detection unit

Claims (15)

A diagnostic device that diagnoses by sound, and
A signal acquisition unit that acquires a sound signal that is an electrical signal converted from sound and outputs the sound signal;
A pre-processing unit that converts a sound signal output from the signal acquisition unit into a frequency domain signal;
A spatial correlation calculation unit that calculates a spatial correlation matrix based on the frequency domain signal converted by the pre-processing unit;
A space correlation abnormality detection unit that determines an abnormality based on the space correlation matrix calculated by the space correlation calculation unit;
And an abnormality display unit for displaying information related to the abnormality based on the determination of the abnormality by the space correlation abnormality detection unit. The diagnostic device according to claim 1, wherein
A per-sound source processing unit that calculates a per-sound source spatial correlation matrix based on the frequency domain signal converted by the pre-processing unit;
A sound source processing unit that separates a sound source and generates a sound source separation signal based on the sound source space correlation matrix calculated by the sound source processing unit and the frequency domain signal converted by the preprocessing unit;
A sound source space correlation abnormality detection unit that determines an abnormality based on the sound source space correlation matrix calculated by the sound source processing unit;
A sound source separation signal abnormality detection unit that determines an abnormality based on the sound source separation signal generated by the sound source processing unit;
The abnormality display unit is
In addition to the determination of the abnormality by the spatial correlation abnormality detection unit, information on the abnormality is displayed based on the determination of the abnormality by the sound source space correlation abnormality detection unit and the determination of the abnormality by the sound source separation signal abnormality detection unit. A diagnostic device characterized by The diagnostic device according to claim 2, wherein
The sound source processing unit
A filter updating unit that calculates a sound source separation filter based on the sound source space correlation matrix calculated by the sound source processing unit;
A sound source separation unit that applies the sound source separation filter calculated by the filter updating unit to the frequency domain signal converted by the pre-processing unit and separates the sound source to generate a sound source separation signal is provided. Diagnostic device. The diagnostic device according to claim 3, wherein
The sound source processing unit
A direction cluster estimation unit that estimates a sound source presence direction cluster based on the spatial correlation matrix calculated by the spatial correlation calculation unit;
A per-sound source space correlation calculation unit for calculating a per-sound source space correlation matrix based on the frequency domain signal converted by the pre-processing unit and the sound source existing direction cluster estimated by the direction cluster estimation unit; Diagnostic equipment. The diagnostic device according to claim 4, wherein
It further has an input unit,
The abnormality display unit is
When the information indicating that there is no movement of the noise source and no movement of the target sound source in the normal state is obtained from the input unit, determination of abnormality by the spatial correlation abnormality detection unit, determination of abnormality by the spatial correlation abnormality detection unit for each sound source, And a diagnostic device for displaying information related to the abnormality based on the determination of the abnormality by the sound source separation signal abnormality detection unit. The diagnostic device according to claim 3, wherein
The sound source processing unit
A power calculator configured to calculate power of frequency components in each direction for each frequency based on the frequency domain signal converted by the pre-processor;
A power signal separation unit that separates the power calculated by the power calculation unit so as to remove a base learned in a normal state;
A direction cluster estimation unit that estimates a sound source presence direction cluster based on the power calculated by the power calculation unit;
An individual sound source space correlation calculation unit for calculating an individual sound source space correlation matrix based on the frequency domain signal converted by the pre-processing unit and the sound source presence direction cluster estimated by the direction cluster estimation unit;
The diagnostic device
And a power abnormality detection unit that determines an abnormality based on the power separated by the power signal separation unit.
The abnormality display unit is
Based on the determination of abnormality by the spatial correlation abnormality detection unit, the determination of abnormality by the sound source space correlation abnormality detection unit, the determination of abnormality by the sound source separation signal abnormality detection unit, and the abnormality determination by the power abnormality detection unit The diagnostic device characterized by displaying the information regarding. The diagnostic device according to claim 6, wherein
It further has an input unit,
The abnormality display unit is
When the information indicating that there is no movement of the noise source and no movement of the target sound source in the normal state is obtained from the input unit, determination of abnormality by the spatial correlation abnormality detection unit, determination of abnormality by the spatial correlation abnormality detection unit for each sound source, A diagnostic apparatus characterized by displaying information related to an abnormality based on the determination of an abnormality by the sound source separation signal abnormality detection unit and the determination of an abnormality by the power abnormality detection unit. A diagnostic method that a computer diagnoses by sound, and
The computer is
A storage unit in which the program is stored;
A processor that executes a program stored in the storage unit;
The processor is
Obtain and convert a sound signal, which is an electrical signal converted from sound
Convert the converted sound signal into a frequency domain signal,
Calculate the spatial correlation matrix based on the transformed frequency domain signal,
Calculate the per-source spatial correlation matrix based on the transformed frequency domain signal,
Based on the calculated per-source space correlation matrix and the transformed frequency domain signal, the source is separated to generate a source separation signal,
Determine the anomaly based on the calculated spatial correlation matrix,
Anomaly is determined based on the calculated per-source spatial correlation matrix,
An abnormality is determined based on the generated sound source separation signal,
A diagnostic method characterized by displaying information related to abnormality according to the judgment of abnormality based on the spatial correlation matrix, the judgment of abnormality based on the spatial correlation matrix for each sound source, and the judgment of abnormality based on the sound source separation signal. The diagnostic method according to claim 8, wherein
The processor is
Based on the calculated spatial correlation matrix, the sound source presence direction cluster is estimated, and the sound source space correlation matrix is calculated based on the transformed frequency domain signal and the estimated sound source presence direction cluster. Calculate the correlation matrix
By calculating the sound source separation filter based on the calculated sound source space correlation matrix, applying the calculated sound source separation filter to the converted frequency domain signal, and separating the sound source to generate the sound source separation signal , Separating the sound source. The diagnostic method according to claim 9, wherein
The processor is
When information is obtained that there is no movement of the noise source and no movement of the target sound source in the normal state, the determination of the abnormality based on the spatial correlation matrix, the determination of the abnormality based on the per-source spatial correlation matrix, and the abnormality based on the sound source separation signal A diagnostic method characterized by displaying information related to an abnormality according to the determination. The diagnostic method according to claim 8, wherein
The processor is
Based on the converted frequency domain signal, the power of the frequency component in each direction is calculated for each frequency, separated from the calculated power so as to remove the basis learned in the normal state, and based on the calculated power The sound source space correlation matrix is calculated by estimating the sound source presence direction cluster and calculating the sound source space correlation matrix based on the transformed frequency domain signal and the estimated sound source presence direction cluster,
Determine the anomaly based on the separated power,
According to the determination of the abnormality based on the spatial correlation matrix, the determination of the abnormality based on the spatial correlation matrix for each sound source, the determination of the abnormality based on the sound source separation signal, and the determination of the abnormality based on the power, displaying information about the abnormality. Diagnostic methods. The diagnostic method according to claim 11, wherein
The processor is
When information is obtained that there is no movement of the noise source and no movement of the target sound source in the normal state, determination of an abnormality based on the spatial correlation matrix, determination of an abnormality based on the spatial correlation matrix for each sound source, determination of an abnormality based on the sound source separation signal And displaying information related to the abnormality in accordance with the determination of the abnormality based on the power and the power. A diagnostic system that includes a plurality of computers and diagnoses by sound,
The first one of the plurality of computers is:
A signal acquisition unit that acquires a sound signal that is an electrical signal converted from sound and outputs the sound signal;
A pre-processing unit that converts a sound signal output from the signal acquisition unit into a frequency domain signal;
A spatial correlation calculation unit that calculates a spatial correlation matrix based on the frequency domain signal converted by the pre-processing unit;
A per-sound source processing unit that calculates a per-sound source spatial correlation matrix based on the frequency domain signal converted by the pre-processing unit;
A sound source processing unit that separates a sound source and generates a sound source separation signal based on the sound source-specific spatial correlation matrix calculated by the sound source-specific processing unit and the frequency domain signal converted by the preprocessing unit;
Transmitting the spatial correlation matrix calculated by the spatial correlation calculation unit, the sound source spatial correlation matrix calculated by the sound source processing unit, and the sound source separation signal generated by the sound source processing unit;
The second computer of the plurality of computers is
A spatial correlation abnormality detection unit that determines an abnormality based on the spatial correlation matrix received from the first computer;
A sound source space correlation abnormality detection unit that determines an abnormality based on the sound source space correlation matrix received from the first computer;
A sound source separation signal abnormality detection unit that determines an abnormality based on the sound source separation signal received from the first computer;
An abnormality display unit that displays information related to the abnormality based on the determination of the abnormality by the spatial correlation abnormality detection unit, the determination of the abnormality by the sound source space correlation abnormality detection unit, and the determination of the abnormality by the sound source separation signal abnormality detection unit; The diagnostic system characterized by having. The diagnostic system according to claim 13, wherein
The sound source processing unit of the first computer
A direction cluster estimation unit that estimates a sound source presence direction cluster based on the spatial correlation matrix calculated by the spatial correlation calculation unit;
An individual sound source space correlation calculation unit for calculating an individual sound source space correlation matrix based on the frequency domain signal converted by the pre-processing unit and the sound source presence direction cluster estimated by the direction cluster estimation unit;
The sound source processing unit of the first computer
A filter updating unit that calculates a sound source separation filter based on the per-sound source space correlation matrix calculated by the per-sound source space correlation calculation unit;
A sound source separation unit that applies the sound source separation filter calculated by the filter updating unit to the frequency domain signal converted by the pre-processing unit and separates the sound source to generate a sound source separation signal is provided. Diagnostic system. The diagnostic system according to claim 13, wherein
The sound source processing unit of the first computer
A power calculator configured to calculate power of frequency components in each direction for each frequency based on the frequency domain signal converted by the pre-processor;
A power signal separation unit that separates the power calculated by the power calculation unit so as to remove a base learned in a normal state;
A direction cluster estimation unit that estimates a sound source presence direction cluster based on the power calculated by the power calculation unit;
An individual sound source space correlation calculation unit for calculating an individual sound source space correlation matrix based on the frequency domain signal converted by the pre-processing unit and the sound source presence direction cluster estimated by the direction cluster estimation unit;
The sound source processing unit of the first computer
A filter updating unit that calculates a sound source separation filter based on the per-sound source space correlation matrix calculated by the per-sound source space correlation calculation unit;
A sound source separation unit that applies the sound source separation filter calculated by the filter updating unit to the frequency domain signal converted by the pre-processing unit and separates the sound source to generate a sound source separation signal;
The first computer is
The power signal separation unit transmits the separated power;
The second computer is
The apparatus further comprises a power abnormality detection unit that determines an abnormality based on the power received from the first computer.
The abnormality display unit of the second computer is:
Based on the determination of abnormality by the spatial correlation abnormality detection unit, the determination of abnormality by the sound source space correlation abnormality detection unit, the determination of abnormality by the sound source separation signal abnormality detection unit, and the abnormality determination by the power abnormality detection unit A diagnostic system characterized by displaying information on

Images