TWM668513U - An auxiliary identification system using ultrasound imaging and artificial intelligence to detect fetal brain abnormalities - Google Patents

Abstract

An auxiliary identification system for detecting fetal brain abnormalities using artificial intelligence using ultrasound images can be used when the ultrasound images are blurred and the local anatomical features are less obvious. The auxiliary identification system and method using artificial intelligence technology to process fetal ultrasound images can effectively improve the recognition efficiency of the convolution neural network for important fetal development key anatomical positions, further reduce the burden of clinical physicians' interpretation, and improve the accuracy of fetal development abnormality interpretation.

Description

An auxiliary identification system using ultrasound imaging and artificial intelligence to detect fetal brain abnormalities

This invention belongs to the field of auxiliary diagnostic measurement technology. Specifically, it is an auxiliary identification system that uses ultrasound images to detect fetal brain abnormalities with artificial intelligence. It can improve the ability to identify the anatomical locations of important developmental key points of the fetus, thereby reducing the burden of clinical physicians' interpretation and improving the accuracy of fetal developmental abnormality interpretation.

According to the press release, ultrasound imaging is one of the most basic medical imaging diagnostic techniques. Due to the convenience and non-invasiveness of ultrasound examination, ultrasound is currently the most popular and effective imaging diagnostic tool for fetal structural abnormalities before birth, especially for fetal brain abnormalities. The incidence of fetal brain abnormalities before birth is 1.4-1.6 out of 1,000, accounting for 3-6% of fetal stillbirth. Therefore, prenatal fetal ultrasound examination is very important for the early diagnosis of fetal structural abnormalities.

However, the quality of ultrasound images is often difficult to control. The resolution of ultrasound images is relatively fuzzy, the fetal position and body posture are difficult to find and identify, and the shaking during ultrasound image shooting produces ultrasound image afterimages and ultrasound image noise. As a result, ultrasound images are often not clear enough, affecting the interpretation of clinical physicians. In addition, for obstetricians and gynecologists, fetal ultrasound image identification requires a long learning curve. Professional interpretation of subtle fetal brain abnormalities also requires professional training. Therefore, not all obstetricians and gynecologists specialize in ultrasound operation and interpretation.

In recent years, deep learning technology using convolutional networks as its main architecture has achieved good recognition performance in many medical application fields. For example, Taiwan’s patent publication No. I802486, “Method for interpreting pediatric renal ultrasound images using artificial intelligence”, publication No. I810498, “Intelligent analysis device for liver tumors”, and publication No. I811129, “System and method for auxiliary identification of target detection in pediatric congenital heart ultrasound images”, all assist clinical physicians in making interpretations in different fields and reduce their burden.

In other words, if we can use artificial intelligence deep learning technology to assist clinical physicians in capturing the anatomical location of fetal developmental abnormalities in the details of fetal ultrasound images, it will be able to greatly reduce the burden of clinical physicians' interpretation and reduce the risk of missed fetal developmental abnormalities. Detecting fetal developmental abnormalities as early as possible and referring pregnant women to medical centers for more detailed examinations as soon as possible will be the problem that this creation will focus on and the key point to be solved.

In view of the above needs, the creators of this invention believe that further development is necessary. Therefore, with many years of experience in related technologies and product design and manufacturing, they have conducted research and creation on the above shortcomings and actively sought solutions. After continuous efforts in research and trial production, they have developed an auxiliary identification system that uses ultrasound images to detect fetal brain abnormalities with artificial intelligence, in order to solve the inconvenience and trouble in interpretation caused by the poor resolution quality of fetal ultrasound images.

Therefore, the main purpose of this invention is to provide an auxiliary identification system that uses ultrasound images to detect fetal brain abnormalities with artificial intelligence, so as to effectively process fetal ultrasound images when the ultrasound images are blurred and the local anatomical features are less obvious, and enhance the performance of the convolutional neural network in identifying key anatomical locations of important fetal development.

Secondly, another main purpose of this creation is to provide an auxiliary identification system that uses ultrasound images to detect fetal brain abnormalities with artificial intelligence. It can be used for automatic identification, measurement and abnormal development detection of fetal ultrasound images, effectively reducing the burden of clinical physicians' interpretation and improving the accuracy of developmental abnormality interpretation, so that medical personnel can monitor and intervene in real time.

To this end, this invention mainly uses the following technical means to specifically achieve the above-mentioned purposes and functions, including a hierarchical anatomical region detection module, a key tissue detection module, a key anatomical landmark detection module, a biometric measurement module and an abnormal development interpretation module;

The hierarchical anatomical region detection module receives a fetal cerebellum plane ultrasound image as input, and captures a local region image after hierarchical detection from top to bottom. One or more ROI images with obvious image features are gradually reduced from top to bottom. The fetal ultrasound image analysis process can be represented by a fetal ultrasound image analysis tree data structure in a highly reliable and stable manner. Separate ROI images layer by layer, and these ROI images contain one or more key tissues, but the ROI images do not have to be completely aligned with the key tissue images. The definition of the ROI images needs to take into account that when the convolutional network detects the ROI image boundary, each ROI region can have enough and reliable image biological anatomical features to provide to the convolutional network so that it can accurately and automatically detect this ROI image;

The key tissue detection module is connected to the hierarchical anatomical region detection module, and its main function is to find all key tissues in the local range by using an object detection method in the local ROI image captured by the hierarchical anatomical region detection module;

The key anatomical marker point detection module is connected to the key tissue detection module, and uses the key tissue position information that has been found to find all the key anatomical marker points in the local ROI image using a key point detection method;

In addition, the biological measurement module is connected to the key anatomical landmark detection module, which measures the distance between key anatomical landmarks or uses image segmentation technology to measure important biological development characteristics;

Furthermore, the abnormal development judgment module is connected to the biometrics module, and judges whether a specific abnormal development risk condition occurs according to the biological development characteristic values calculated by the biometrics module and in accordance with medical standards or machine learning methods.

In one embodiment of the present invention, the fetal ultrasound image analysis tree T is a tree structure, and the root node of the image analysis tree T represents the entire original fetal ultrasound input image; one or more branch nodes are expanded from the root node, each branch node represents a local ROI image, and this local ROI image is automatically detected and captured by a specific object detection convolutional network from the local ROI image corresponding to the upper node, and each branch node can also expand one or more branch nodes, and the upper branch node ROI image includes the lower branch node ROI image, thereby generating a tree structure of the upstream and downstream image analysis relationship, and the end of each branch node has the image analysis tree T One or more terminal branch nodes, each of which can be further divided into one or more key tissues or key anatomical landmarks, wherein each key tissue is detected from a local ROI image corresponding to the terminal branch node through a specific object detection or image segmentation algorithm, and each key anatomical landmark is detected from a local ROI image corresponding to the terminal branch node through a specific key point detection algorithm.

In one embodiment of the present invention, the hierarchical anatomical region detection module is for a fetal cerebellum plane ultrasound input image, and the first-level node of the fetal ultrasound image analysis tree corresponding to the fetal ultrasound image is a cranial ROI image. I1, and the second layer node of the skull ROI image I1 includes a transparent septum ROI image I1,1 and a posterior cervical thickness ROI image I1,2, wherein the transparent septum ROI image I1,1 includes a key tissue transparent septum T1, and the third layer node of the posterior cervical thickness ROI image I1,2 includes a cerebello-medullary cistern ROI image I1,2,1, and the cerebello-medullary cistern ROI image I1,2,1 includes a key tissue cerebellum T2, and three groups of key anatomical landmarks, including cerebellum transverse diameter anatomical landmarks P1, P2, cerebello-medullary cistern diameter anatomical landmarks P3, P4, and posterior cervical thickness anatomical landmarks P5, P6.

In one embodiment of the present invention, in the fetal cerebellum plane ultrasound input image, the hierarchical anatomical region detection module has obvious image features of the skull, cerebellum and septum pellucidum, while the image features of the local images that are relatively unclear and difficult to identify independently are the cerebellomedullary cistern, posterior neck thickness and six anatomical landmarks closely related to biological developmental characteristics, including cerebellum transverse diameter anatomical landmarks P1, P2, cerebellomedullary cistern diameter anatomical landmarks P3, P4, and posterior neck thickness anatomical landmarks P5, P6, which uses the skull as the local image with obvious features, first performs skull ROI local image recognition, excludes the external skull image to reduce the complexity of subsequent analysis, and limits the scope of subsequent analysis to the skull ROI image I1. The next step is to use the cerebellum as the local image with obvious features, combined with the posterior neck skin and occipital bone with obvious boundary features, and combine the two image features to identify the posterior neck thickness ROI image I1,2 and the cerebellomedullary cistern ROI image I1,2,1, so as to improve the accuracy of local image recognition with unclear features.

In one embodiment of the present invention, the biometrics module calculates the cerebellar transverse diameter by the distance between the aforementioned automatic cerebellar transverse diameter anatomical landmarks P1 and P2, and further verifies the reliability of the measurement data of P1, P2 and the cerebellar transverse diameter according to the detected position of the boundary frame of the key tissue cerebellum T2. The biometrics module also calculates the cerebellar medullary cistern diameter by the distance between the aforementioned automatic cerebellar medullary cistern diameter anatomical landmarks P3 and P4, and further verifies the reliability of the measurement data of P1, P2 and the cerebellar transverse diameter according to the detected position of the boundary frame of the key tissue cerebellum T2. The reliability of the measurement data of P3, P4 and the cerebellomedullary cistern diameter is checked by the bounding box position of the cerebellum ROI image I1,2,1 and the key tissue cerebellum T2, and the biometric measurement module also calculates the posterior cervical thickness through the distance between the aforementioned posterior cervical thickness anatomical landmarks P5 and P6, and further checks the reliability of the measurement data of P5, P6 and the posterior cervical thickness according to the detected bounding box position of the cerebellomedullary cistern ROI image I1,2,1 and the posterior cervical thickness ROI image I1,2.

Through the concrete realization of the aforementioned technical means, this creation can greatly enhance its practicability, increase its added value, and improve its economic benefits.

In order to enable you, the Review Committee, to further understand the composition, features and other purposes of this creation, the following are some preferred embodiments of this creation, and are described in detail with the help of diagrams, so that those familiar with the technical field can implement them in detail.

The attached drawings illustrate specific embodiments of the invention and its components. All references to front and back, left and right, top and bottom, upper and lower, and horizontal and vertical are only for the convenience of description and do not limit the invention or its components to any position or spatial direction. The dimensions specified in the drawings and descriptions may be changed according to the design and requirements of the specific embodiments of the invention without departing from the scope of the patent application of the invention.

The invention is an auxiliary identification system for detecting fetal brain abnormalities by using ultrasound images and artificial intelligence. Please refer to FIG1 . The system comprises a hierarchical anatomical region detection module (10), a key tissue detection module (20), a key anatomical landmark detection module (30), a biometric measurement module (40) and an abnormal development interpretation module (50). The key tissue detection module (20) is connected to the hierarchical anatomical region detection module (10), the key anatomical landmark detection module (30) is connected to the key tissue detection module (20), and the biometric measurement module (40) is connected to the abnormal development interpretation module (50). 0) is connected to the key anatomical landmark detection module (30), and the abnormal development judgment module (50) is connected to the biometric measurement module (40) to input a fetal cerebellum plane ultrasound image (P1) into the hierarchical anatomical region detection module (10). After passing through the key tissue detection module (20), the key anatomical landmark detection module (30), the biometric measurement module (40) and the abnormal development judgment module (50), a fetal ultrasound examination report (100) can be generated to assist clinical physicians in identifying whether the fetal brain growth is abnormal.

Please refer to FIG. 2. According to the main embodiment of the present invention, the important biological development features that need to be measured in the fetal cerebellum plane ultrasound image (P1) include but are not limited to the biparietal diameter of the skull, the biparietal diameter of the cerebellum, the cisterna magna, and the nuchal fold. The previous method of using deep learning technology to measure the aforementioned important biological development features is mainly to detect the minimum bounding boxes (BB) containing these biological development features, such as the skull bounding box is BB1, the cerebellum bounding box is BB2, the cisterna magna bounding box is BB3, and the nuchal fold bounding box is BB4.

Please also refer to what is disclosed in Figures 3 and 4. In the embodiment of the present invention, for the input image of the fetal cerebellum plane ultrasound image (P1), the image features of the skull and cerebellum are more obvious, and the recognition effect of the skull boundary box BB1 and the cerebellum boundary box BB2 using deep learning object detection technology is better; however, the image features of the cisterna magna and the posterior cervical thickness (nuchal fold) are less obvious, lacking a good recognition reference, resulting in poor recognition effect of the cisterna magna boundary box BB3 and the posterior cervical thickness boundary box BB4. As shown in Figure 3, the precision-recall curve of the recognition results of the skull boundary frame BB1 and the cerebellum boundary frame BB2 is significantly higher than the average of the four boundary frame recognitions, and the curve is closer to the upper right; while the precision-recall curve of the recognition results of the cerebellomedullary cistern boundary frame BB3 and the posterior neck thickness boundary frame BB4 is significantly higher than the average of the four boundary frame recognitions, and the curve is closer to the lower left. As shown in Figure 4, from the F1 curve, it can be clearly found that the F1 score of the recognition results of the skull boundary frame BB1 and the cerebellum boundary frame BB2 is higher, while the F1 score of the recognition results of the cerebellomedullary cistern boundary frame BB3 and the posterior neck thickness boundary frame BB4 is lower.

Please refer to FIG. 5 . In the embodiment of the present invention, the hierarchical anatomical region detection module (10) uses a method of gradually reducing the local region of interest (hereinafter referred to as ROI) image from top to bottom, and gradually eliminates images irrelevant to the biological development characteristics required to be measured to reduce the complexity of subsequent analysis. Finally, only the smallest local ROI image containing biological development characteristics is retained to improve the recognition efficiency of biological development characteristics. Therefore, the process of gradually reducing the local ROI image from top to bottom can be represented by a data structure of an image parse tree for fetal ultrasound biometry to represent the analysis process of fetal ultrasound images. A fetal ultrasound image analysis tree T is a tree structure, in which the root node of the image analysis tree T represents the entire original fetal ultrasound input image; one or more branch nodes are expanded from the root node, each branch node represents a local ROI image, and this local ROI image is automatically detected and captured from the local ROI image corresponding to the upper node through a specific object detection convolutional network; each branch node can also expand one or more branch nodes, and the upper branch node ROI image includes the lower branch node ROI image, thereby generating a tree structure of upstream and downstream image analysis relationships; the terminal branch nodes are the image analysis tree T The terminal branch node is the most terminal branch node. Each terminal branch node can be further divided into one or more key tissues or key anatomical landmarks. Each key tissue is detected by a specific object detection or image segmentation algorithm from the local ROI image corresponding to the terminal branch node. Each key anatomical landmark is detected by a specific key point detection algorithm from the local ROI image corresponding to the terminal branch node. The selection of local ROI images is based on the principle that they must contain tissues with obvious image features, so that the object detection and key point detection algorithms can have sufficient volume image features to achieve good recognition performance.

Please also refer to FIG. 6. In the embodiment of the present invention, for a fetal cerebellum plane ultrasound image (P1), the pre-trained cranial ROI detection model is first used to detect the bounding box of the cranial ROI image I1 from the fetal cerebellum plane ultrasound image (P1), and exclude other ultrasound images not related to the brain. In the cranial ROI image I1, the tissues with obvious anatomical features are the transparent septum and cerebellum, while the tissues that are relatively difficult to detect are the cerebellomedullary cistern and the posterior neck thickness. However, the cerebello-medullary cistern and posterior neck thickness are very important biometric points for diagnosing fetal structural abnormalities. In order to improve the detection accuracy of the convolutional network of the cerebello-medullary cistern and posterior neck thickness, the cerebellum, whose anatomical features are obviously easy to detect, is used in conjunction with the edge image of the posterior neck thickness to detect the posterior neck thickness ROI image I1,2 area. Similarly, since the location of the cerebello-medullary cistern is not easy to identify independently, two new methods are used to improve the detection performance of the cerebello-medullary cistern: First, through top-down layer-by-layer analysis, the search range for identifying the cerebello-medullary cistern is reduced from the entire ultrasound image to the posterior neck thickness ROI image I1,2 area, which greatly reduces the complexity of identifying the cerebello-medullary cistern. Next, the cerebellum with obvious anatomical features and the obvious posterior border of the skull are used to detect the cerebellomedullary cistern ROI image I1,2,1. After the search target area is greatly reduced to the cerebellomedullary cistern ROI image I1,2,1, the key point detection technology is further used to detect the cerebellum transverse diameter anatomical landmarks P1, P2 and the cerebellomedullary cistern diameter anatomical landmarks P3, P4 in the cerebellomedullary cistern ROI image I1,2,1. Similarly, the posterior cervical thickness anatomical landmarks P5 and P6 only need to be searched in the area that has been reduced to the posterior cervical thickness ROI image I1,2. The anatomical tissues of the septum pellucidum T1 and cerebellum T2 only need to be searched in the septum pellucidum ROI image I1,1 and the cerebellomedullary cistern ROI image I1,2,1, respectively, whose search ranges have been reduced.

Furthermore, please refer to Figure 7. In an embodiment of the present invention, for a fetal cerebellum plane ultrasound image (P1), the four branch node ROI images I1, I1,1, I1,2 and I1,2,1 corresponding to the fetal ultrasound image analysis tree in Figure 6 in this input image are shown in the figure. The branch node ROI images are selected based on the principle that the image features are obvious so that the corresponding four object detection models can accurately identify the upper, lower, left and right frame positions of the ROI image boundary box. For example, I1 can refer to the upper and lower left and right contours of the fetal skull as the basis for I1 boundary detection; ROI image I1,1 can refer to the clear septum cavity and the obvious features of the anterior horns of the bilateral ventricles in the fetal brain for boundary detection; the right side boundary of ROI image I1,2 of posterior cervical thickness (nuchal fold) can be located according to the outer skin boundary of the skull, but the left side and upper and lower sides of the posterior cervical thickness do not have enough image features to provide the object detection model for positioning. Therefore, this work proposes to use the cerebellum with obvious features to provide the left side and upper and lower boundary reference positioning features of ROI image I1,2, so that the ROI image I1,2 of posterior cervical thickness (nuchal fold) is included in the cerebellum area to improve the posterior cervical thickness (nuchal fold). Similarly, in this ultrasound image example, although the right boundary of the cerebellomedullary cistern ROI image I1,2,1 has the occipital bone for boundary reference positioning, it also lacks the image features for left side and top and bottom positioning. Therefore, the cerebellar cistern ROI image I1,2,1 is also included in the cerebellum region to provide left side and top and bottom boundary reference positioning features. After completing the detection of the four branch node ROI images I1, I1,1, I1,2 and I1,2,1 from top to bottom, for the identification of the anatomical tissues of the transparent septum T1 and the cerebellum T2, the respective search ranges are limited to the local areas of the transparent septum ROI image I1,1 and the cerebellomedullary cistern ROI image I1,2,1, which can greatly reduce the complexity of the detection problem and improve the efficiency of the identification of these two anatomical tissues.

Please refer to FIG8 . In the embodiment of the present invention, for a fetal cerebellum ultrasound image (P1), the process of gradually analyzing and unfolding the image from top to bottom corresponding to the fetal ultrasound image analysis tree in FIG6 is shown in FIG8 . The leftmost figure is the input fetal cerebellum ultrasound image (P1), and the boundary frame of the skull ROI image local image I1 of the pre-trained skull ROI image detection model is shown by the dotted line in the figure. After the search range is limited to the ROI image I1 local image, the pre-trained septum pellucidum ROI image detection model and the posterior cervical thickness ROI image detection model are used to perform the septum pellucidum ROI image I1,1 and the posterior cervical thickness ROI image I1,2 local image detection respectively. After the search range is limited to the I1,2 local image, the pre-trained cisterna magna ROI image detection model is used to perform local image detection of the cisterna magna ROI image I1,2,1. Finally, the anatomical tissue septum pellucida T1 is detected from the septum pellucida ROI image I1,1 using the pretrained septum pellucida detection model; the anatomical tissue cerebellum T2 is detected from the cistern ROI image I1,2,1 using the pretrained cistern diameter and cerebellar transverse diameter key point detection models; the cerebellar transverse diameter anatomical landmarks P1 and P2, as well as the cistern diameter anatomical landmarks P3 and P4 are detected from the cistern ROI image I1,2,1; the posterior cervical thickness anatomical landmarks P5 and P6 are detected from the posterior cervical thickness ROI image I1,2 using the pretrained posterior cervical thickness key point detection model.

In the embodiment of the present invention, each branch node in the fetal ultrasound image analysis tree represents a local ROI image, which is automatically detected and captured from the local ROI image corresponding to the upper node through the corresponding pre-trained object detection convolutional network model. The object detection convolutional network model can use the YOLO series object detection model, the R-CNN series object detection model, the SSD series object detection model, the RatinaNet series object detection model, or the EfficientDet object detection model, but is not limited thereto.

In the embodiment of the present invention, each terminal branch node in the fetal ultrasound image analysis tree is the terminal branch node of the image analysis tree T, and each terminal branch node can be further divided into one or more key tissues or key anatomical landmarks. The key tissue detection module (20) automatically detects and captures each key tissue local image based on the corresponding pre-trained object detection convolutional network model or image segmentation algorithm. The object detection convolutional network model may use a YOLO series object detection model, an R-CNN series object detection model, an SSD series object detection model, a RatinaNet series object detection model, or an EfficientDet series object detection model, but not limited thereto; and the image segmentation algorithm may use a U-Net series object detection model, a DeepLab series object detection model, or a PSPNet series object detection model, but not limited thereto. The key anatomical landmark detection module (30) detects each key anatomical landmark from the local ROI image corresponding to the terminal branch node through the corresponding pre-trained key landmark detection model. The key point detection model may use DeepLabCut, HRNet, AlphaPose, CenterNet, YOLOv8 or Key.Net, but is not limited thereto.

In another embodiment of the present invention, the biometric measurement module (40) calculates the Euclidean distance between the two paired markers Pi and Pj according to the key anatomical markers detected by the key anatomical marker detection module (30) to obtain the biometric value. For example, the transverse diameter of the cerebellum is obtained by calculating the Euclidean distance between the anatomical markers P1 and P2; the diameter of the cerebellomedullary cisterna is obtained by calculating the Euclidean distance between the anatomical markers P3 and P4; and the posterior neck thickness is obtained by calculating the Euclidean distance between the anatomical markers P5 and P6.

In the embodiment of the present invention, the abnormal development judgment module (50) judges the risk of developmental abnormality according to whether the key tissues are detected and the various biometric values calculated by the biometric module (40), and whether the various biometric values exceed the normal value range of the various biometric statistics, and finally generates the fetal ultrasound examination report (100) according to the estimated risk values of various developmental abnormalities.

From the above system architecture and description, it can be seen that the existing fetal ultrasound image analysis technology mainly classifies the entire ultrasound image, or directly uses object detection technology to detect important anatomical features of fetal ultrasound images. However, the anatomical features in ultrasound images are often not specific enough due to the volume features of the image, resulting in poor performance when directly using object detection for identification. Through the development of the main technology of this creation, this creation has the following functions and features, such as:

1. This work proposes a fetal ultrasound image analysis tree, which uses a top-down approach to gradually reduce the local ROI image, and step by step eliminates images that are irrelevant to the final required biological development characteristics to reduce the complexity of subsequent analysis. Finally, only the smallest local ROI image containing biological development characteristics is retained, thereby improving the recognition efficiency of biological development characteristics.

2. The selection of local ROI images in this work is based on the principle that they must include tissues with obvious image features, so that the object detection and key point detection algorithms can have sufficient convolution image features to achieve good recognition performance. For example, the cerebellum, which has obvious anatomical features and is easy to detect, is used in conjunction with the edge image of the posterior neck thickness to detect the posterior neck thickness ROI image area.

3. Each terminal branch node in the fetal ultrasound image analysis tree of this invention is the terminal branch node of the image analysis tree T. Each terminal branch node can be further divided into one or more key tissues or key anatomical landmarks, which can effectively detect key tissues and key anatomical landmarks with minimal complexity, rather than searching for key tissues and key anatomical landmarks in the entire fetal ultrasound image.

In summary, it can be understood that this creation is a new creation with excellent creativity. In addition to effectively solving the problems faced by practitioners, it also greatly improves the efficacy. In the same technical field, there is no same or similar product creation or public use. At the same time, it has improved efficacy. Therefore, this creation has met the requirements of "novelty" and "progressiveness" for new patents, and an application for a new patent is filed in accordance with the law.

10: Hierarchical anatomical region detection module 20: Key tissue detection module 30: Key anatomical landmark detection module 40: Biometric measurement module 50: Abnormal development interpretation module 100: Fetal ultrasound examination report P1: Fetal cerebellum plane ultrasound image

Figure 1: is a schematic diagram of the auxiliary identification system architecture of the present invention. Figure 2: is a schematic diagram of the important biological developmental characteristics that need to be measured in the planar ultrasound image of the fetal cerebellum in the present invention, including the biparietal diameter, cerebellum diameter, cisterna magna and nuchal fold, and their corresponding minimum bounding boxes (BB). Figure 3: is a schematic diagram of the precision-recall curve of the recognition results of various tissues in the fetal cerebellum planar ultrasound image with different recognition difficulties in the embodiment of the present invention. The image features of the brain and cerebellum are more obvious, and the convolution network is easier to recognize, and its Precision-Recall curve is closer to the upper right corner; the image features of the cisterna magna and the nuchal fold are less obvious, and the convolution network is more difficult to recognize, and its Precision-Recall curve is farther from the upper right corner. Figure 4: is a schematic diagram showing the F1 scores of the recognition results of different recognition difficulties of various tissues in the fetal cerebellum plane ultrasound image in the present invention. Figure 5: is a typical structural diagram of the fetal ultrasound image parse tree [image parse tree for fetal ultrasound biometry] proposed in the present invention. Figure 6: is a structural schematic diagram of the fetal cerebellum plane ultrasound input image parse tree drawn in the present invention. Figure 7: is a schematic diagram of the ultrasound image corresponding to the fetal cerebellum plane ultrasound input image parse tree drawn in the present invention, the region of interest (ROI) in the local ultrasound image, the key tissues and the key anatomical landmarks. Figure 8: is a schematic diagram of the top-down analysis process of the fetal cerebellum plane ultrasound input image drawn in the present invention embodiment.

10: Hierarchical anatomical region detection module

20: Key organization detection module

30: Key anatomical landmark detection module

40: Biometric measurement module

50: Abnormal development interpretation module

100: Fetal ultrasound examination report

P1: Planar ultrasound image of fetal cerebellum

Claims (5)

An auxiliary identification system for detecting fetal brain abnormalities by artificial intelligence using ultrasound images, comprising a hierarchical anatomical region detection module, a key tissue detection module, a key anatomical landmark detection module, a biometric measurement module and an abnormal development interpretation module; The hierarchical anatomical region detection module detects a fetal cerebellum plane ultrasound image inputted therein, and captures a local region image after hierarchical detection from top to bottom, and selects one or more ROI images with obvious image features. The process of gradually reducing the local ROI image from top to bottom can be represented by a data structure of a fetal ultrasound image analysis tree to analyze the fetal ultrasound image in a highly reliable and stable manner. Layer-separated ROI images, and these ROI images contain one or more key tissues, but the ROI images do not have to be completely aligned with the key tissue images, and the definition of the ROI images must take into account that when the convolutional network is detecting the ROI image boundary, each ROI region can have enough and reliable image biological anatomical features to provide to the convolutional network so that it can accurately and automatically detect this ROI image; The key tissue detection module is connected to the hierarchical anatomical region detection module, and its main function is to find all key tissues in the local range of the ROI image captured by the hierarchical anatomical region detection module by using the object detection method; The key anatomical landmark detection module is connected to the key tissue detection module, and it uses the key point detection method to find all key anatomical landmarks in the local ROI image in conjunction with the key tissue position information that has been found; The biological measurement module is connected to the key anatomical landmark detection module, and it measures the important biological development characteristics by measuring the distance between key anatomical landmarks or image segmentation technology; Furthermore, the abnormal development judgment module is connected to the biometrics module, and judges whether a specific abnormal development risk condition occurs according to the biological development characteristic values calculated by the biometrics module and in accordance with medical standards or machine learning methods. As described in claim 1, an auxiliary identification system for detecting fetal brain abnormalities using artificial intelligence using ultrasound images, wherein the fetal ultrasound image analysis tree T is a tree structure, and the root node of the image analysis tree T represents the entire original fetal ultrasound input image; one or more branch nodes are expanded from the root node, each branch node represents a local ROI image, and this local ROI image is automatically detected and captured from the local ROI image corresponding to the upper node through a specific object detection convolutional network, and each branch node can also expand one or more branch nodes, and the upper branch node ROI image includes the lower branch node ROI image, thereby generating a tree structure of upstream and downstream image analysis relationships, and each branch node has the image analysis tree T at the very end. One or more terminal branch nodes, each of which can be further divided into one or more key tissues or key anatomical landmarks, wherein each key tissue is detected from a local ROI image corresponding to the terminal branch node through a specific object detection or image segmentation algorithm, and each key anatomical landmark is detected from a local ROI image corresponding to the terminal branch node through a specific key point detection algorithm. As described in claim 2, an auxiliary identification system for detecting fetal brain abnormalities using ultrasound images with artificial intelligence, wherein the hierarchical anatomical region detection module has a first-level node of a fetal ultrasound image analysis tree corresponding to a fetal cerebellum plane ultrasound input image as a skull ROI image. I1, and the second layer node of the skull ROI image I1 includes a transparent septum ROI image I1,1 and a posterior cervical thickness ROI image I1,2, wherein the transparent septum ROI image I1,1 includes a key tissue transparent septum T1, and the third layer node of the posterior cervical thickness ROI image I1,2 includes a cerebello-medullary cistern ROI image I1,2,1, and the cerebello-medullary cistern ROI image I1,2,1 includes a key tissue cerebellum T2, and three groups of key anatomical landmarks, including cerebellum transverse diameter anatomical landmarks P1, P2, cerebello-medullary cistern diameter anatomical landmarks P3, P4, and posterior cervical thickness anatomical landmarks P5, P6. As described in claim 3, an auxiliary identification system for detecting fetal brain abnormalities using ultrasound images with artificial intelligence, wherein the hierarchical anatomical region detection module detects, in the fetal cerebellum plane ultrasound input image, the local images with obvious image features are the skull, cerebellum and septum pellucidum, and the local images with relatively unclear image features that are difficult to identify independently are the cerebellomedullary cistern, posterior neck thickness, and six anatomical landmarks closely related to biological developmental characteristics, including cerebellum transverse diameter anatomical landmarks P1, P2, cerebellomedullary cistern diameter anatomical landmarks P3, P4, and posterior neck thickness anatomical landmarks P5, P6, which uses the skull as the local image with obvious features, first performs skull ROI local image recognition, excludes the external skull image to reduce the complexity of subsequent analysis, and limits the scope of subsequent analysis to the skull ROI image I1. The next step is to use the cerebellum as the local image with obvious features, combined with the posterior neck skin and occipital bone with obvious boundary features, and combine the two image features to identify the posterior neck thickness ROI image I1,2 and the cerebellomedullary cistern ROI image I1,2,1, so as to improve the accuracy of local image recognition with unclear features. As described in claim 4, an auxiliary identification system for detecting fetal brain abnormalities using ultrasound images with artificial intelligence, wherein the biometric measurement module calculates the cerebellar transverse diameter through the distance between the aforementioned automatic cerebellar transverse diameter anatomical landmarks P1 and P2, and further verifies the reliability of the measurement data of P1, P2 and the cerebellar transverse diameter based on the detected boundary frame position of the key tissue cerebellum T2, and the biometric measurement module also calculates the cerebellar medullary cistern diameter through the distance between the aforementioned automatic cerebellar medullary cistern diameter anatomical landmarks P3 and P4, and further verifies the reliability of the measurement data of P1, P2 and the cerebellar transverse diameter. The reliability of the measurement data of P3, P4 and the diameter of the cerebellomedullary cistern is checked based on the detected bounding box positions of the cerebellum magna ROI image I1,2,1 and the key tissue cerebellum T2, and the biometrics module also calculates the posterior neck thickness through the distance between the aforementioned posterior neck thickness anatomical landmarks P5 and P6, and further checks the reliability of the measurement data of P5, P6 and the posterior neck thickness based on the detected bounding box positions of the cerebellum magna ROI image 11,2,1 and the posterior neck thickness ROI image I1,2.