WO2020118659A1 - Structural defect detection system and structural defect detection method - Google Patents

Abstract

A structural defect detection system, comprising: a laser (1), a beam splitter (2), a beam expander (3), a semi-transparent mirror (7), a sound wave generator (4), a sound wave frequency adjuster (5), an imaging lens (6), a photoelectric sensor (9) and a computer (8), wherein the computer (8) controls the sound wave generator (4) to send a sound wave signal; the laser light emitted by the laser (1) forms an interference light path, and the interference light path forms a speckle interference field on the photoelectric sensor (9) to generate a speckle image; and the photoelectric sensor (9) transmits the speckle image to the computer (8) to perform defect detection of a detected object. In addition, further comprised is a structural defect detection method. By means of the technical solution, an assembly defect of complex and small electronic devices inside a consumer electronic product can be detected, and the structural defect detection method is a non-contact, high-precision, on-line and real-time non-destructive testing method.

Description

Structural defect detection system and structural defect detection method Technical field

The invention relates to the technical field of device detection, in particular to a structural defect detection system and a structural defect detection method.

Background technique

In the prior art, consumer electronic products such as mobile phones, computers, and various portable mobile devices require surface mount device mounting (SMD) for various microelectronic devices in the assembly and production process, Snap-fit, connector plug-in, laser welding plate, glue to glue various auxiliary materials. For example, the assembly of microelectronic devices is usually carried on the PCB board as the carrier inside the mobile phone. During the assembly of microelectronic devices, mechanical damage, fatigue, creep, and overheating can cause product assembly defects. Common assembly defects include debonding, deformation, crumpling, scratches, cracks, blowholes, and damage to electronic components. There are many types of assembly defects in microelectronic devices. Some assembly defects are visible on the surface, and some assembly defects are hidden inside and cannot be directly detected. Electronic products are broken or damaged due to these assembly defects. There was almost no harbinger before, so the destruction and damage were sudden, which created a safety hazard for users using electronic products. Therefore, it is very important to carry out non-destructive testing of complex and small electronic device assembly defects.

In the prior art, assembly defects on the surface of an object can be detected by a visual method, for example, by image processing technology, and an automatic optical inspection (Automatic Inspection (AOI) method) method is commonly used. However, the inventor has found through research that because the automatic optical inspection only inspects the appearance, it cannot see through the internal structure of the product, and therefore cannot see through the product, so it cannot detect all the actual defects of the product. Relatively speaking, X-ray inspection in the prior art can better detect and image internal assembly defects of products than automatic optical inspection. However, X-rays have radioactive hazards. At the same time, the inventors found through research that for complex multi-layer internal structures, penetrating X-rays are difficult to distinguish the three-dimensional structure inside the product, and there are areas in the product that are blocked. The assembly defects cannot be accurately located. The position of the X-ray inspection makes it impossible to achieve a good inspection effect when facing the defect inspection of the PCB board of microelectronic devices with various assembly methods.

Summary of the invention

In the technical solution of the present invention, an acoustic wave signal whose frequency continuously changes is applied to the surface of the measured object from all directions, and the structural defect of the measured object is subjected to forced vibration due to the action of simple harmonics, as the frequency of the acoustic wave signal continues Increase, when the frequency of a certain acoustic wave signal is equal to or close to the natural frequency of the internal defect part of the measured object, resonance will occur at the defect, and the amplitude of vibration will be the largest at this time, so a large Off-surface displacement.

Based on this, in order to solve the technical problems in the prior art, a structural defect detection system is specifically proposed.

The structural defect detection system includes a laser, a beam splitter, a beam expander, a semi-transparent mirror, an acoustic wave generator, an acoustic wave frequency adjuster, an imaging lens, a photoelectric sensor, and a computer;

Wherein the acoustic wave generator is connected to the acoustic wave frequency adjuster, the acoustic wave frequency adjuster is connected to the computer; the photoelectric sensor is connected to the computer;

Specifically, the computer sends a frequency control signal to the sonic frequency regulator, and the frequency control signal is transmitted to the sonic wave generator after the digital-to-analog (D/A) conversion of the sonic frequency regulator; the sonic wave generator sends out frequency control Acoustic signal corresponding to the signal;

Wherein, the laser light emitted by the laser passes through a beam splitter, a beam expander, an imaging lens, and a semi-transparent mirror to form an interference optical path;

Specifically, the interference optical path includes that the laser emits laser light, and the laser beam is first split by the beam splitter to form object light and reference light; the object light becomes parallel after being expanded by the beam expander Light is projected onto the object to be measured; diffuse reflection light is generated on the surface of the object to be measured, and the transmission of the diffuse reflection light through the imaging lens and the half mirror is received by the photoelectric sensor; the reference light passes through The transflective mirror is received by the photoelectric sensor after being reflected; the optical path of the structural defect detection system is adjusted so that the optical paths of the object light and the reference light are equal; the reference light passes through the transflective mirror After the reflection, the diffuse reflected light is simultaneously projected on the photoelectric sensor to form a speckle interference field; the speckle interference field is digitized by the photoelectric sensor to generate a speckle image;

The photoelectric sensor transmits the generated speckle image to the computer; the computer calculates the phase change of the speckle image to obtain the vibration waveform distribution of the measured object under the excitation of sound waves of different frequencies; further, a computer can also be used Calculate the phase difference diagram of the measured object under the excitation of sound waves of different frequencies.

In one embodiment, the photoelectric sensor is a CCD photoelectric sensor or a CMOS photoelectric sensor;

In one embodiment, the acoustic wave generator is a voltage-controlled acoustic wave generator, including a power amplifier and a speaker.

In addition, in order to solve the technical problems in the prior art, a structural defect detection method is specifically proposed.

The structural defect detection method includes a training phase and a detection phase;

The training phase includes:

Define multiple different detection states, classify different defects of multiple measured object samples according to the defined detection states, and the results of the classification are used as the output characteristics of the output layer of the neural network; for each type of defect, obtain 1000 or more images The phase difference map of the sample of the measured object constitutes a training data set for training a neural network, and there is a correspondence between the detection state in the training data set and the phase difference map;

Extract multiple phase difference maps and their corresponding detection states from the training data set; use the phase difference maps as input features of the neural network, and use the detection states as output features of the neural network, using the The input feature and the output feature train the neural network to obtain a neural network model of the relationship between the phase difference map of the measured object and the defect of the measured object;

The detection phase includes:

Obtain the phase difference map of the measured object;

Input the acquired phase difference map as input features into the neural network for detection;

The output layer of the neural network outputs a detection state, and the detection state is an output feature of the neural network.

In one embodiment, the defined detection status includes no defects, pores, deformation, and other defects.

In one embodiment, the neural network is a deep neural network based on deep learning.

In an embodiment, obtaining the phase difference map specifically includes:

The sound wave generator sends out sound wave signals of different frequencies;

The laser light emitted by the laser forms an interference optical path; the interference optical path forms a speckle interference field on the photoelectric sensor. The speckle interference field is digitally processed by the photoelectric sensor to generate a speckle image and transmitted to the computer;

When the sound wave frequency changes, the phase map of the surface deformation of the measured object under different sound wave frequencies obtained by the phase shift method at different sound wave frequencies is subtracted between the phase maps to obtain the surface of the measured object at different sound wave frequencies Deformed phase difference diagram.

In an embodiment, the interference optical path includes: the laser light emitted by the laser is first split by a beam splitter to form object light and reference light; the object light is expanded by the beam expander to become parallel light and projected onto the measured object On the object; diffuse reflection light is generated on the surface of the measured object, and the diffuse reflection light is received by the photoelectric sensor through the imaging lens and the transflective mirror; the reference light is reflected by the transflective mirror, and the The diffuse reflected light is simultaneously projected on the photoelectric sensor to form a speckle interference field.

In one embodiment, during the training phase, the number of phase difference maps of multiple samples of the object to be tested is greater than or equal to 1000 for each type of defect; the number of phase maps is greater than or equal to 3.

The implementation of the embodiments of the present invention will have the following beneficial effects:

In the present invention, the speckles obtained by the stimulated vibration of the internal defects of the measured object can be precisely measured and calculated by the speckle obtained by the interference of the coherent laser beam irradiated on the surface of the measured object, without the need to phase the speckle image Reconstruct the real image of the object, but directly calculate the vibration displacement of the interference speckle with the frequency of the sound wave signal, which can indirectly solve the forced vibration waveform of the measured object under the active stress of the sound wave. The intensity and frequency of the sound wave can be set to a more suitable range to adapt to various structural defects. Different material properties, structural distribution and different ways of connecting and assembling different devices will produce different vibration signal distributions. The vibration signal distribution can be A high-precision speckle image is used to measure the phase difference map, and then an artificial neural network is used to detect and identify the defect occurrence area and defect type. The technical solution of the present invention is a non-contact, high-precision, online, real-time Non-destructive testing method.

BRIEF DESCRIPTION

The drawings required in the embodiments or the description of the prior art will be briefly introduced below.

In order to more clearly explain the embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings required in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, without paying any creative work, other drawings can be obtained based on these drawings.

1 is a schematic diagram of a structural defect detection system in the present invention;

2 is a schematic diagram of acquiring a phase difference graph when the sound wave frequency is changed in the present invention;

3 is a schematic diagram of defect detection using a neural network in the present invention.

detailed description

The technical solutions in the embodiments of the present invention will be described clearly and completely in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention.

As shown in FIG. 1, the present invention discloses a structural defect detection system, which includes: a laser 1, a beam splitter 2, a beam expander 3, a semi-transparent mirror 7, an acoustic wave generator 4, an acoustic wave Frequency adjuster 5, imaging lens 6, photoelectric sensor 9, computer 8; wherein, the sound wave generator 4 is connected to the sound wave frequency adjuster 5, the sound wave frequency adjuster 5 is connected to the computer 8; The photoelectric sensor 9 is connected to the computer 8; wherein, the sound wave generator includes a power amplifier and a speaker.

The laser light source of the laser 1 emits coherent laser light; the coherent laser light is first split by the beam splitter 2 to form object light and reference light; the object light is expanded by the beam expander 3 to become parallel light and projected onto the object to be measured As a result, diffusely reflected light containing deformation or vibration information of the measured object is received by the photo sensor 9 after passing through the imaging lens;

Adjusting the optical path of the structural defect detection system so that the optical paths of the object light and the reference light are equal; wherein, after the reference light is reflected by the semi-transparent mirror 7 and the diffuse reflected light is simultaneously projected on the photoelectric sensor 9 to form interference, And form a speckle interference field; the speckle interference field is digitized by the photoelectric sensor 9 to generate a grayscale image, and the grayscale image is a speckle image;

In one embodiment, the photoelectric sensor is a CCD photoelectric sensor or a CMOS photoelectric sensor, and the photoelectric sensor uses a non-imaging method to detect the speckle image obtained by interference at high speed, without the need for a complicated optical lens to perform speckle Imaging

Wherein, after the photoelectric sensor 9 collects the speckle image, the speckle image is transmitted to the computer 8 connected to the photoelectric sensor 9; the computer 8 is used to calculate the phase change of the speckle image, so that the measured object is different Vibration waveform distribution under frequency sound wave excitation;

The vibration waveforms of various regions of the measured object are mainly affected by two aspects: first, it is related to the frequency of the sound wave excited by the sound wave generator and the resonance caused by the defect of the measured object; second, it is related to the material of the measured object itself and various micro The connection mode of the electronic device is related; the above information can be used to effectively judge and detect whether there are any defects such as deformation, wrinkling, cracks, etc. during the assembly process.

The computer 8 sends a frequency control signal to the sound wave frequency regulator 5, and the frequency control signal is transmitted to the sound wave generator 4 after the digital-to-analog (D/A) conversion of the sound wave frequency regulator 5; the sound wave generator 4 sends out Acoustic signal corresponding to frequency control signal.

In an embodiment, the acoustic wave generator 4 is a voltage-controlled acoustic wave generator, including a power amplifier and a speaker; the acoustic wave generator 4 can generate the acoustic wave signal required by the control signal, and the acoustic wave signal is amplified by the power amplifier It can meet the frequency response and sound intensity required by the defect detection task, and send it to the speaker to emit the corresponding sound wave, so as to realize the continuous frequency broadband sound wave scanning.

In the present invention, the principle that the internal structure defects of the measured object can be detected is: the sound wave generator outputs a broadband sound wave signal whose frequency continuously changes, and the sound wave signal acts on the surface of the measured object from different directions. The internal defect site is subjected to forced vibration due to the action of simple harmonics. As the frequency of the acoustic wave signal continues to increase, when the frequency of a certain acoustic wave signal is equal to or close to the natural frequency of the internal defect site of the measured object, the defect site There will be resonance phenomenon, at this time the vibration amplitude is the largest, so there will be a large amplitude of off-surface displacement in the surface defect part of the object; at the same time, considering the structural safety of each object to be tested, the excitation of the sound wave intensity and excitation Effectively design the position to avoid the damage of the measured object caused by the excessive sound wave intensity. The above method is called broadband sound wave scanning excitation.

In the process of broadband sound wave scanning excitation, the photoelectric sensor continuously captures the speckle image formed by the interference of the object light and the reference light, and calculates the package phase distribution map of the object under the deformation state through the multiple speckle images, The image processing technology is used to detect and identify the defect area and the non-defect area of the measured object from the phase distribution change diagram reflecting the defect information, thereby performing fast and accurate defect detection processing on the measured object.

During the deformation of the measured object, it is necessary to take multiple speckle images by changing the phase of the reference optical path of the structural defect detection system; wherein, the number of the speckle images is greater than or equal to 3; The phase distribution of the surface of the measured object at the current excitation sound wave frequency. The following formula gives the phase formula in the calculation of the phase shift method, where φ(x, y) is the phase distribution and m is the number of speckle images.

As shown in FIG. 2, when the frequency of the sound wave emitted by the sound wave generator changes, at different sound wave frequencies, for example, the frequency of f1, f2, f3, the phase diagram of the sound wave of different frequencies obtained by the phase shift method, the phase The subtraction between the graphs can obtain the phase difference map of the surface deformation of the measured object at different sound wave frequencies f. The abnormality of the phase difference map on the distribution of different regions can determine whether the defect exists and the approximate location of the defect.

However, the automatic identification of defects, including the existence of defects, the location of defects, and the determination of related features through phase difference maps carrying defect information, cannot well characterize defect detection targets and complete defect detection tasks.

In the technical solution of the present invention, a method for classifying, training, and detecting a phase difference map that can indirectly reflect structural defects using a neural network is disclosed; wherein, the neural network is a deep neural network based on deep learning; The deep neural network based on deep learning can obtain hierarchical visual features from the input phase difference map through unsupervised and supervised learning methods, thereby providing a more effective defect detection scheme.

Obtain the phase difference graph:

In the technical solution of the present invention, the phase difference map needs to be obtained in both training and detection stages:

The computer sends a frequency control signal to the sound wave frequency regulator, and the frequency control signal is transmitted to the sound wave generator after being converted by the digital-to-analog (D/A) of the sound wave frequency regulator; the sound wave generator sends out the sound wave signal corresponding to the frequency control signal ;

At the same time, the laser light source of the laser emits coherent laser light; the coherent laser light is first split by the beam splitter to form the object light and the reference light; the object light is expanded by the beam expander to become parallel light and projected onto the object to be measured, thereby , The diffuse reflected light containing the deformation or vibration information of the measured object is received by the photoelectric sensor after passing through the imaging lens;

Adjust the optical path so that the optical paths of the object light and the reference light are equal; where the reference light is reflected by the transflective mirror, and the object light is simultaneously projected on the photoelectric sensor to form an interference and form a speckle interference field; the speckle interference The field is digitized by the photoelectric sensor to generate a grayscale image, and the grayscale image is a speckle image;

After the photoelectric sensor collects the speckle image, the speckle image is transmitted to the computer connected to the photoelectric sensor; the phase change of the speckle image is calculated by the computer to obtain the vibration waveform of the measured object under the excitation of sound waves of different frequencies distributed;

During the process of broadband vibration excitation, the photoelectric sensor continuously captures the speckle image formed by the interference between the object light and the reference light, and calculates the package phase distribution map of the measured object in the deformed state through the multiple speckle images ;

During the deformation of the measured object, it is necessary to take multiple speckle images by changing the phase of the reference optical path of the structural defect detection system; where the number of speckle images is greater than or equal to 3; then, the computer calculates the current The phase distribution of the surface of the measured object at the frequency of the excitation sound wave, the following formula gives the phase formula in the calculation of the phase shift method, where φ(x, y) is the phase distribution, and m is the number of sampled speckle images:

When the frequency of the sound wave emitted by the sound wave generator changes, the phase map of the surface deformation of the object under different sound waves obtained by the phase shift method at different sound wave frequencies is obtained by subtracting the phase maps to obtain different sound wave frequencies The phase difference graph of the surface deformation of the measured object.

During the training phase:

First, define multiple different detection states, and classify different defects of multiple measured object samples according to the defined detection state, and the classification result is used as the output feature of the output layer of the neural network, that is, the output feature is the detection state; For each type of defect, a phase difference map of 1000 or more samples of the measured object is obtained respectively to form a training data set for training a neural network, and there is a correspondence between the detection state in the training data set and the phase difference map;

Among them, the definition of non-defect, stomatal, deformation, and others is the detection state, that is, the output characteristics include no defect, stomatal, deformation, and others;

Next, extract multiple phase difference maps and their corresponding detection states from the training data set; use the phase difference maps that indirectly reflect structural defects as the input features of the neural network input layer, and use the detection states as the output features of the neural network output layer, Using the input features and output features to train the neural network, a neural network model of the relationship between the phase difference map of the measured object and the defects of the measured object is obtained.

In the detection stage:

Obtain the phase difference map of the measured object;

Input the acquired phase difference map into the input layer of the neural network for detection, wherein the phase difference map is an input feature of the neural network;

The output layer of the neural network outputs a detection state, wherein the detection state is an output characteristic of the neural network; the detection state includes no defects, pores, deformation, and others.

For the technical solution of the present invention, for different applications and different types of product defects, the external excitation method can be changed. In addition to the broadband acoustic wave scanning excitation loading in the embodiment of the present invention, thermal loading, Vacuum loading, electromagnetic excitation loading and other methods, through the analysis of deformation, to obtain information about defects. The defect characteristics and distribution of some of the measured objects are very complicated. The parameters loaded by external excitation sources, such as loading time, intensity, and uniformity, are all parameters that can be adjusted by the structural defect detection system.

The implementation of the embodiments of the present invention will have the following beneficial effects:

The technical scheme of the present invention can detect assembly defects of complex and small electronic devices inside consumer electronic products, and is a non-contact, high-precision, online, and real-time non-destructive detection method.

The technical solution of the present invention stimulates the measured object to induce forced vibration through active sound waves, can detect the deformation, vibration, impact, stiffness and strength of various construction machinery and equipment, and controls the product quality inspection and optimization of the production process parameters It is an advantageous detection tool.

The technical solution of the present invention can realize rapid and online detection of laser welding, glue bonding quality, and bonding quality of composite materials by monitoring the displacement change and deformation of objects with the excitation source, effectively characterizing and evaluating the structure Of the bonding quality, and evaluate the relationship between loading time and deformation.

The technical solution of the present invention can analyze the structural characteristics of objects, thereby making it more widely used, such as the monitoring of the operating status of industrial production equipment, the monitoring of gas leakage, the monitoring of the deformation of machined parts, etc. The above monitoring results can be used for Industrial big data analysis in the Internet of Things industry.

The above disclosure is only preferred embodiments of the present invention, and of course it cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope of the present invention.

Claims (10)

A structural defect detection system characterized by, The structural defect detection system includes a laser, a beam splitter, a beam expander, a semi-transparent mirror, an acoustic wave generator, an acoustic wave frequency adjuster, an imaging lens, a photoelectric sensor, and a computer; Wherein the acoustic wave generator is connected to the acoustic wave frequency regulator, the acoustic wave frequency regulator is connected to the computer; the photoelectric sensor is connected to the computer; the computer transmits the frequency control signal to the acoustic wave A frequency regulator, the acoustic wave frequency regulator transmits the frequency control signal to the acoustic wave generator; the acoustic wave generator emits a corresponding acoustic wave signal; Wherein, the laser light emitted by the laser forms an interference optical path after passing through the beam splitter, the beam expander, the imaging lens, and the transflective mirror; the interference optical path forms a dispersion on the photoelectric sensor Speckle interference field, the speckle interference field is digitally processed by the photoelectric sensor to generate a speckle image; The photoelectric sensor transmits the generated speckle image to the computer for defect detection of the measured object. The structural defect detection system according to claim 1, wherein: The photoelectric sensor is a CCD photoelectric sensor or a CMOS photoelectric sensor. The structural defect detection system according to claim 1, wherein: The sound wave generator is a voltage-controlled sound wave generator, including a power amplifier and a speaker. The structural defect detection system according to claim 2 or 3, characterized in that The interference optical path includes that the laser light emitted by the laser is first split by the beam splitter to form object light and reference light; the object light is expanded by the beam expander to become parallel light and projected onto the On the object to be measured; diffuse reflection light is generated on the surface of the object to be measured, and the transmission of the diffuse reflection light through the imaging lens and the half mirror is received by the photoelectric sensor; the reference light passes through the half mirror After being reflected by the half mirror, the diffuse reflected light is simultaneously projected on the photoelectric sensor to form the speckle interference field. A structural defect detection method, characterized in that The structural defect detection method includes a training phase and a detection phase; The training phase includes: Define multiple different detection states, and classify different defects of multiple measured object samples according to the defined detection state, and the results of the classification are used as the output characteristics of the output layer of the neural network; for each type of defect, multiple pieces of measured objects are obtained separately The phase difference map of the sample constitutes a training data set for training the neural network, and there is a correspondence between the detection state in the training data set and the phase difference map; Extract multiple phase difference maps and their corresponding detection states from the training data set; use the phase difference maps as input features of the neural network, and use the detection states as output features of the neural network, using the The input feature and the output feature train the neural network to obtain a neural network model of the relationship between the phase difference map of the measured object and the defect of the measured object; The detection phase includes: Obtain the phase difference map of the measured object; Input the acquired phase difference map as input features into the neural network for detection; The output layer of the neural network outputs a detection state, and the detection state is an output feature of the neural network. The structural defect detection method according to claim 5, characterized in that The defined detection status includes no defects, pores, deformation, and other defects. The structural defect detection method according to claim 5, characterized in that The neural network is a deep neural network based on deep learning. The method for detecting structural defects according to claim 5 or 6 or 7, wherein Obtaining the phase difference graph specifically includes: The sound wave generator sends out sound wave signals of different frequencies; The laser light emitted by the laser forms an interference optical path; the interference optical path forms a speckle interference field on the photoelectric sensor. The speckle interference field is digitized by the photoelectric sensor to generate a speckle image and transmitted to the computer; when the sound wave frequency When changes occur, acquire multiple speckle images; Using the plurality of speckle images, a phase shift method is used to calculate a plurality of phase diagrams of the surface deformation of the measured object under different frequency sound wave excitation, and the plurality of phase diagrams are subtracted to obtain the measured sound waves at different sound wave frequencies. The phase difference diagram of the surface deformation of the object. The structural defect detection method according to claim 8, wherein: The interference light path includes: the laser light emitted by the laser is first split by a beam splitter to form object light and reference light; the object light is expanded by the beam expander to become parallel light and projected onto the object to be measured; Diffuse reflection light is generated on the surface of the object, and the diffuse reflection light is received by the photoelectric sensor through the imaging lens and the semi-transparent mirror successively; after the reference light is reflected by the semi-transparent mirror, the diffuse reflection light is simultaneously projected A speckle interference field is formed on the photoelectric sensor. The structural defect detection method according to claim 8, wherein: In the training phase, the number of phase difference maps of multiple samples of the object to be tested is greater than or equal to 1000 for each type of defect; the number of phase maps is greater than or equal to 3.

Images