CN116137026A - Medical image processing device, medical image processing method and storage medium - Google Patents

Abstract

Embodiments disclosed in the present specification and drawings relate to a medical image processing apparatus, a medical image processing method, and a storage medium. One of the problems to be solved by the embodiments disclosed in the present specification and the drawings is to obtain a more accurate morphological measurement result. The medical image processing device according to the embodiment includes a control unit. The control unit executes a 1 st process including at least two different kinds of processes for the medical image data. The control unit further executes the 1 st processing for a spatially reduced range based on the processing result of the 1 st processing.

Description

Medical image processing device, medical image processing method and storage medium

This application claims priority based on Chinese Patent Application No. 202111351852.1 filed on November 16, 2021 and Japanese Patent Application No. 2022-166028 filed on October 17, 2022, the entire contents of which are cited here.

technical field

The embodiments disclosed in the specification and drawings relate to a medical image processing device, a medical image processing method, and a storage medium.

Background technique

The heart valve is the basic and important structure of the heart. The heart valve refers to the valve between the atrium and the ventricle or between the ventricle and the artery. Valve disease can lead to death.

Accurate valve morphology information is critical for the treatment of valve diseases such as transcatheter aortic valve implantation (TAVI). One of the important clinical decisions in TAVI is the selection of the correct implant device and the corresponding size, and the accuracy of these selections is largely dependent on factors such as valve commissures, nadir, aortic root, aortic valve leaflets, etc. Accurate extraction of key anatomical points and structures. However, because the valve is located inside the heart, its size is small, and it is always in a state of opening and closing under normal circumstances, and its shape and shape change greatly, so it is very difficult to accurately extract valve information. In addition, it is more difficult to extract information from unhealthy valves such as valve calcification and incomplete valves due to irregular shapes. Therefore, there is a need to provide more accurate solutions to extract the morphological information of valves.

Here, due to different diseases and individual differences, the shape and morphology of valves will vary greatly, so it is a challenging task to accurately extract the morphological information of valves. Most of the current algorithms directly extract the target information from the original volume region at one time, instead of repeating from coarse to fine, and do not use any auxiliary information for refinement. Most automatic valve information extraction is aimed at a single task of morphology, for example, some automatic algorithms only detect anatomical landmarks or only segment the aortic root or aortic valve cusp. Therefore, they cannot obtain accurate morphological information, especially for complex cases such as calcified valves and valve defects. In addition, some existing technologies propose to perform segmentation or detection from coarse to fine, but these technologies only use one segmentation or detection algorithm, and there is no solution in the prior art to combine multiple tasks into one workflow.

Part of these prior art problems can be considered to be that, for example, keypoint detection and object segmentation are performed separately, and the relationship between the acquired keypoint processing and segmentation processing is not considered. Therefore, if the key point detection process can be combined with the segmentation process to generate more useful auxiliary information or provide more accurate morphological information, it will be of great help to accurate valve measurement in treatment planning.

Except heart valve, other complex human organs or complex parts of organs may also produce the above-mentioned technical problems.

Contents of the invention

One of the technical problems to be solved by the embodiments disclosed in this specification and the accompanying drawings is to obtain more accurate morphological measurement results. However, the technical problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above-mentioned technical problems. The technical problems corresponding to the respective effects brought about by the respective configurations described in the embodiments described later may also be defined as other technical problems.

A medical image processing apparatus according to an embodiment includes a control unit. The control unit executes first processing including at least two different types of processing on the medical image data. In addition, the control unit re-executes the first processing on the spatially reduced range based on the processing result of the first processing.

Invention effect

According to the medical image processing device, the medical image processing method, and the storage medium according to the embodiments, more accurate morphological measurement results can be obtained.

Description of drawings

FIG. 1 is a block diagram showing an example of the configuration of a medical image processing apparatus according to the first embodiment.

FIG. 2 is a flowchart showing processing executed by the medical image processing apparatus according to the first embodiment.

3 is a diagram showing a flow and processing results of an example of heart valve detection processing executed by the medical image processing apparatus according to the first embodiment.

4A is a diagram for explaining an example of the first processing executed in step S100 by the first processing function in the medical image processing apparatus according to the first embodiment.

4B is a diagram for explaining an example of the first processing executed in step S200 by the second processing function in the medical image processing apparatus according to the first embodiment.

4C is a diagram for explaining an example of adjusted first processing executed in step S300 by the first processing function in the medical image processing apparatus according to the first embodiment.

5A is a diagram for explaining a modified example of enhancement processing executed by a first processing function in the medical image processing apparatus according to the first embodiment.

5B is a diagram for explaining a modified example of enhancement processing executed by the first processing function in the medical image processing apparatus according to the first embodiment.

6A is a diagram for explaining a modified example of information enhancement processing executed by a second processing function in the medical image processing apparatus according to the first embodiment.

6B is a diagram for explaining a modified example of information enhancement processing performed by the second processing function in the medical image processing apparatus according to the first embodiment.

7A is a flowchart illustrating an example of correction processing executed by the medical image processing apparatus according to the first embodiment.

7B is a schematic diagram for explaining a processing procedure of correction processing performed by the medical image processing apparatus according to the first embodiment.

7C is a schematic diagram for explaining a processing procedure of correction processing performed by the medical image processing apparatus according to the first embodiment.

8 is a flowchart showing an example of heart valve detection processing executed by the medical image processing apparatus according to the second embodiment.

9 is a flowchart showing an example of heart valve detection processing executed by the medical image processing apparatus according to the third embodiment.

10 is a flowchart showing an example of lung detection processing executed by the medical image processing apparatus according to the fourth embodiment.

11 is a diagram showing a processing flow and processing results of an example of processing executed by the medical image processing apparatus according to the fifth embodiment.

Detailed ways

Hereinafter, embodiments of the medical image processing device, the medical image processing method, and the storage medium according to the present application will be described in detail with reference to the drawings. In addition, the medical image processing apparatus, the medical image processing method, and the storage medium related to the present application are not limited by the embodiments described below. In addition, in the following description, the same component will be given the same code|symbol and the overlapping description will be omitted.

(first embodiment)

First, the medical image processing apparatus according to the first embodiment will be described. The medical image processing device of the present application may exist in the form of medical image diagnostic devices such as ultrasonic diagnostic devices and MRI (Magnetic Resonance Imaging: magnetic resonance imaging) imaging devices, and may also exist independently in the form of workstations and the like.

FIG. 1 is a block diagram showing an example of the configuration of a medical image processing apparatus 1 according to the first embodiment.

As shown in FIG. 1 , the medical image processing apparatus 1 according to this embodiment mainly includes a control unit 10 and a memory 20. The ultrasonic probe, display, input/output interface, device main body, etc. shown in the figure, the control unit 10 and the memory 20 are communicably connected to the ultrasonic probe, display, input/output interface, and device main body. Since the structures and functions of the ultrasonic probe, the display, the input and output interfaces, and the main body of the device are well known to those skilled in the art, detailed descriptions thereof are omitted.

The memory 20 stores various data necessary for the medical image processing apparatus 1 to perform processing, such as raw volume data to be processed, various data used and generated by the ultrasonic diagnostic apparatus, image data for display, and the like.

The control unit 10 controls the medical image processing apparatus 1 as a whole, and executes various processes to be executed by the medical image processing apparatus 1 .

The control unit 10 includes a first processing function 100 and a parameter setting function 300, wherein the first processing function 100 executes first processing including at least two different types of processing on the medical image data to be processed. The parameter setting function 300 sets parameters of the first processing executed by the first processing function 100 . The control unit 10 of the present invention executes the first processing including at least two different types of processing on the medical image data through the first processing function 100, and executes the processing result of the first processing that narrows the processing range spatially. In the adjusted first process, an image representing the target area in the medical image data is obtained. The adjustment to spatially narrow the processing range is realized by, for example, parameter setting by the parameter setting function 300 .

In addition, the control unit 10 of the present invention may further include a second processing function 200. Before the control unit 10 executes the adjusted first processing, the second processing function 200 executes the second processing on the processing result of the executed first processing. , further performing the adjusted first processing on the processing result of the second processing, and obtaining an image representing the target area in the medical image data according to the adjusted result of the first processing.

Next, the processing performed by the medical image processing apparatus 1 according to this embodiment will be described in detail using FIG. 2 .

FIG. 2 is a flowchart showing processing executed by the medical image processing apparatus 1 according to the first embodiment.

As shown in FIG. 2 , first, in step S1 , medical image data to be processed is input to the medical image processing apparatus 1 .

Next, in step S100, the first processing function 100 of the control unit 10 executes a first processing including at least two different types of processing. Although not shown in the drawings, in step S100, before performing the first processing , the control unit 10 may first set the parameters of the first processing through the parameter setting function 300 . Here, as an example of the "first processing including at least two different types of processing", FIG. 2 shows a plurality of types of processing on medical image data consisting of processing A to processing N. As shown in FIG. The order of processing A to processing N is fixed, but the processing content and number of processing A to N are just examples, and some of processing A to processing N may be completely the same processing, or may include the same processing with different parameters . Other embodiments are possible as long as the first processing of the present invention includes at least two different types of processing for medical images.

Next, in step S200, the second processing function 200 of the control unit 10 executes the second processing on the processing result of the first processing. Here, as the second processing, FIG. and fused information enhancement processing, the information enhancement processing is just an example of the second processing, and the second processing of the present invention may also have other implementation manners.

Step S200 is a preferred implementation, so although step S200 is performed, step S200 is not necessary and can be omitted, that is, step S300 can be followed after step S100 is performed. In this case, in step S300 , the control unit 10 uses the first processing function 100 to execute the adjusted first processing in which the processing range is spatially narrowed for the processing result of the first processing. The adjustment of the first processing is realized, for example, by parameter adjustment performed by the parameter setting function 300. The so-called adjusted first processing is specifically the adjusted first processing after the processing range is narrowed in space, ensuring that the The adjusted first treatment can obtain more accurate results than the unadjusted first treatment. In Fig. 2, the adjusted first processing is represented as processing A'~processing N'. Compared with processing A~processing N', processing A'~processing N' narrows the processing range in space. Realization, of course, can also be achieved by, for example, fine-tuning the algorithm or updating the input data. The order of processing A' to processing N' is the same as that of processing A to processing N.

In addition, in the case of executing the preferred embodiment of step S200, in step S300, the control unit 10 executes the adjusted first process with the processing range narrowed in space on the processing result of the second process through the first processing function 100, That is to say, the control unit 10 first performs information enhancement processing on the result of the first processing through the second processing function 200 in step S200, and then performs information enhancement processing on the result of the first processing after the information enhancement processing through the first processing function in step S300. , perform the above-mentioned adjusted first processing in which the processing range is narrowed spatially.

After step S300, the control unit 10 obtains an image representing the target area in the medical image data based on the adjusted processing result of the first processing. Processing A to processing N (or processing A' to processing N') correspond to combined processing consisting of multiple processing, as described above, steps S100 and S300 have been performed twice from coarse to fine Combined processing, so that more accurate morphological measurement results can be obtained.

Next, in step S400, the control unit 10 of the medical image processing apparatus 1 of the present invention can also activate the second processing function 200 again, execute the information enhancement process again on the result of step S300, and activate the first processing function 100 again. , re-execute the adjusted first processing on the result of the information enhancement processing again, the re-adjusted first processing can be represented as processing A" to processing N", and can also be based on the result of the re-executed second processing And so on, such as parameter adjustment, so as to narrow the processing range spatially compared with the first two processings before. The process of repeatedly executing the second treatment and the first treatment represented by step S400 can be repeated as many times as required to form an iterative process, thereby obtaining more accurate morphological measurement results, so the processing process corresponding to S400 in Figure 2 is marked with an ellipsis Shows.

The setting of the parameters for the first treatment can not only be carried out according to the requirements of the processing results from coarse to fine, such as through previous experiments, feedback, etc., but also can be based on, for example, repeated execution of the second treatment and repetition of the first treatment. Other factors such as the number of times should be adjusted appropriately. In addition, the adjustment of the first processing can be realized by adjusting the parameters, but the parameter adjustment is only an example, and the adjustment of the first processing can also be realized by, for example, fine-tuning the algorithm, updating the input data, and any combination of these factors.

As mentioned above, through steps S100-S300, the combined processing consisting of multiple processings has been performed twice from coarse to fine, so that more accurate morphological measurement results can be obtained, so the processing of step S400 is not necessary, and can be omitted.

Next, the control unit 10 of the medical image processing apparatus 1 of the present invention may also use the information obtained in the previous steps to correct the obtained image representing the target area in step S500 . The correction is to make the obtained image more suitable for subsequent processing, which is not necessary for the present invention.

Next, in step S600, the medical measurement is completed using the corrected data.

According to the present invention, more accurate morphological measurement results are obtained by performing combined processing consisting of multiple processings on medical image data from coarse to fine.

The processing of the medical image processing apparatus according to this embodiment will be described in detail below using FIGS. 3 to 7C and taking a heart valve as an example.

3 is a diagram showing a flow and processing results of an example of heart valve detection processing executed by the medical image processing apparatus according to the first embodiment.

As shown in FIG. 3 , first in step S1 , an original volume region of medical image data of the heart to be processed is input to the medical image processing apparatus 1 .

Next, in step S100 , as an example of the first processing, the first processing function 100 of the control unit 10 executes combined processing consisting of two different processings S101 and S102 . That is, as the first processing, landmark detection processing is executed on medical image data (step S101 ), and segmentation processing of a target region is executed based on the result of the landmark detection processing (step S102 ).

Specifically, in step S101, the control unit 10 performs rough detection of key points for the original volume region of the medical image data of the heart. For example, the first processing function 100 of the control unit 10 can roughly locate the anatomical key points defined by the user, such as Junction, cusp, left coronary ostium and right coronary ostium.

In step S102 , the control unit 10 executes segmentation processing on the processing target region using the key points detected in step S101 . For example, in S102, the first processing function 100 of the control unit 10 segments the aortic root by applying the roughly positioned anatomical key points obtained in step S101 to obtain a mask image of the aortic root. The mask image includes, for example, the sinus of Valsalva (SOV), the left ventricular outflow tract (Left Ventricular OutflowTract: LVOT), the ascending aorta (Ascending Aorta: AAO) and related junctions.

The specific anatomical key points in step S101 and the mask image of the aortic root in step S102 are just examples, not fixed, and can be customized as required.

Next, in step S200, the second processing function 200 of the control unit 10 executes information enhancement processing. As an example of the second processing, the second processing function 200 uses the rough anatomical key points obtained in step S101 and the aortic root segmentation results obtained in step S102 to perform region limitation and select a more accurate VOI (body of interest: volume of interest). Area limitation is an example of information enhancement processing. For example, area limitation can be cropping the segmentation result image obtained in step S100. Of course, information enhancement processing is not limited to cropping. The details of information enhancement processing will be described later using FIG. 6A and Figure 6B explains in detail.

In step S200, more operations can be performed, such as checking whether the obtained key points and the mask image are consistent. In addition, as mentioned above, step S200 is not necessary and can be omitted.

Next, in step S300, the control unit 10 causes the first processing function 100 to execute the adjusted first processing with the spatially narrowed processing range on the result of the second processing, and obtains the representation according to the adjusted result of the first processing The target area in the medical image data is an image of a heart valve. That is, as the adjusted first processing to be executed again, the control unit 10 executes key point accurate detection processing (step S301) on the result of the second processing, that is, the information-enhanced medical image data through the first processing function 100 (step S301), and based on the key point As a result of point precise detection processing, more precise segmentation is performed, such as segmentation processing of the aortic valve (step S302).

Specifically, in step S301, the control unit 10 utilizes the new morphological information processed in step S200 through the first processing function 100 to locate more accurate anatomical key points; The new morphological information after the information enhancement processing and the precise anatomical key points obtained in step S301 can be used to obtain, for example, a mask image of the aortic valve including three valve leaflets by performing segmentation processing of the aortic valve, for example.

As mentioned above, the processing performed in step S300 is the same processing content as the first processing, but the processing parameters may be adjusted due to the results of the second processing, so as to narrow the processing range spatially and ensure that the processing parameters passed through step S300 A more accurate result than step S100 can be obtained. Details about the processing procedures of step S300 and step S100 will be described in detail later using FIG. 4 .

So far, the medical image processing device according to this embodiment has performed two combined processes consisting of key point detection and segmentation from coarse to fine through steps S100-S300, thereby obtaining a more accurate segmented image of the aortic valve and More accurate morphological measurements.

Figure 3 also shows step S400 as a preferred embodiment after step S300 with an ellipsis, that is, after step S300, in step S400, the control part 10 can execute the information enhancement as the second processing again through the second processing function 200 processing, and for the result of the re-executed information enhancement processing, re-execute the combination of key point detection and segmentation as the first processing, and such re-execution can be repeated many times, and by continuously performing more refined processing, we can obtain An image that more accurately represents the object area.

Next, in step S500 , the control unit 10 of the medical image processing apparatus 1 of the present invention uses the information obtained in the previous steps to correct the obtained image of the heart valve. The correction is to make the obtained heart valve image more suitable for subsequent processing such as measurement, which is not necessary in the present invention. The details of this correction processing will be described in detail later using FIGS. 7A to 7C .

Next, in step S600, the control unit 10 uses the corrected heart valve image to measure, for example, some key items required for TAVI surgery planning, such as the length of the free side of the valve leaflet, the length of the joint between the valve and the vessel wall, and the geometry. Height, etc., to complete medical measurements such as for TAVI.

The details of the processing procedures of step S100 , step S200 , and step S300 are further described below using FIGS. 4A to 4C .

4A is a diagram for explaining an example of a processing procedure executed in step S100 by the first processing function in the medical image processing apparatus according to the first embodiment.

First, in step S101, the first processing function 100 executes rough key point detection processing. This processing can be realized by conventional deep learning neural network such as 3D Spatial Configuration Network (SCN). The input of this deep learning neural network is, for example, the original body region ((a) in Fig. 4A ), which contains the image of the whole heart, and the output is, for example, important clinical key points in the anatomical morphology of the heart ((a in Fig. 4A ). b)) For example, when the target area is the aortic valve, it can be the nadir point of the left coronary valve (LCC Nadir point), the nadir point of the right coronary valve (RCC Nadir point), the nadir point of the non-coronary valve (NCCNadir point), none Coronary valve-left coronary valve commissure point (N-L Commissure point), right coronary valve-left coronary valve commissure point (R-L Commissure point), no coronary valve-right coronary valve commissure point (N-R commissure point), subright coronary ostium At least a part of the 8 clinical key points (Right coronary ostium) and left coronary ostium (Left coronary ostium). Through the rough detection of key points in this step, the area where the heart valve structure is located can be roughly located. Therefore, the settings of the parameters of the deep learning neural network used in this step do not require high detection accuracy, but must have fast time performance. Only in this way can the volume of interest (VOI) be narrowed down quickly and ready for the next operation. Of course, the key point detection processing described above is just an example, and any method may be used as long as a predetermined key point can be detected from the image data to be processed. For example, the method may be that the user manually specifies the position of the key point by using the GUI.

Next, in step S102 , the first processing function 100 uses key points to perform key point-guided aortic root segmentation processing. This processing can be realized by a conventional deep learning neural network (such as 3DUNET). With this deep learning network, it is possible to divide input data into a processing target area and a region other than the processing target area. In the present embodiment, each pixel constituting the input image data is divided into pixels corresponding to the aortic root and other pixels. Through the rough detection of key points in step S101 , a more accurate and smaller VOI area can be located, and segmentation based on this can improve the performance of segmentation. The input of the deep learning neural network in step S102 is the medical image data obtained based on the rough detection of key points, and the output is the mask image of the aortic root inside the VOI region ((d) in FIG. 4A ). Of course, the above division is just an example, and any method may be used as long as the target area can be segmented from the image data to be processed. For example, a method may be used in which the user manually designates all pixels corresponding to the aortic root using a GUI, or a known image segmentation method such as binarization or graph cut processing may be used.

Here, between the rough key point detection processing in step S101 shown in (b) in FIG. 4A and the aortic root segmentation processing in step S102 shown in (d) in FIG. (c) shows the strengthening treatment. That is to say, according to the present invention of the first embodiment, the first processing function 100 may perform enhancement processing on the medical image data based on the result of the key point detection processing, and perform segmentation processing after performing the enhancement processing.

(c) in FIG. 4A shows an example of cropping medical image data based on key points detected by key point detection processing as an enhancement process. In the example shown in FIG. 4A , the input of the deep learning neural network in step S102 is the medical image data trimmed using the bounding box defined based on rough key points, and the output is the cropped VOI area. Aortic root mask image.

Figure 4A uses the cropping process as an example to illustrate the enhancement process between the key point detection process and the segmentation process in the first process, but the enhancement process is not limited to the cropping process, and the details of the enhancement process will be described later using Figure 5 Be explained.

4B is a diagram for explaining an example of the procedure of information enhancement processing executed in step S200 by the second processing function 200 in the medical image processing apparatus according to the first embodiment.

In step S200, the second processing function 200 uses the rough key point detection result or the aortic root segmentation result in step S100 to perform information enhancement processing, for example, by applying the area limit of ROI (region of interest: region of interest) (hereinafter referred to as ROI limit) to focus on the more precise VOI area of the result of step S100. (a) in FIG. 4B shows an example in which rough key point detection results are used to limit ROIs, a bounding box based on rough anatomical key points is calculated, and the result of step S100 is cropped according to the bounding box. (b) in FIG. 4B shows an example of using the aortic root segmentation result to limit the ROI, calculate a bounding box based on the root mask, and crop the image based on the bounding box.

Since the boundary of the aortic root mask image is close to some anatomical key points, it is better to use the bounding box of the root mask image to extend outward to a larger area to prevent the loss of information of some key points. However, the range of the aortic root mask image is smaller, so it is better to use the aortic root mask region for clipping than bounding boxes calculated based on coarse anatomical keypoints.

FIG. 4B shows an example of information enhancement processing performed with ROI limitation as the second processing function, but information enhancement processing is not limited to ROI limitation. The details of information enhancement processing will be described later using FIGS. 6A and 6B .

FIG. 4C is a diagram for explaining an example of adjusted first processing executed in step S300 by the first processing function 100 in the medical image processing apparatus 1 according to the first embodiment.

First, key point precise detection processing is performed in step S301. This process can use the same neural network as that used in the key point rough detection process in step S101, but the input VOI is updated, and the model parameters can be updated through the parameter setting function 300 to narrow down the space The processing range adapts to the requirements of accurate key point detection. The input of the neural network is the image of the sub-volume area cropped by using the precise VOI area information in step S200. It can be seen from FIG. 4B that the input shown here is the result of area limitation using the rough key point detection results , here, of course, you can also input the result of area limitation using the segmentation result of the aortic root. The output of the neural network is the same as that of step S101, which is still the above-mentioned important clinical key points ((b) in Fig. 4C) such as the anatomical morphology information of the heart. network model parameters, the precise detection of key points in step S301 can accurately locate the region where the heart valve structure is located through the key points.

Next, in step S302 , the first processing function 100 performs precise keypoint-guided heart valve segmentation processing. Similarly, this process can use the same neural network as the aortic root segmentation process in step S102, but the input VOI is updated, and the model parameters can be updated through the parameter setting function 300 to narrow the processing range spatially , adapted for more accurate heart valve segmentation. The output of this process is a mask image of the heart valve inside the VOI region ((d) in FIG. 4C ). In addition, each pixel constituting the input image data in step S102 is divided into two parts: the pixel corresponding to the aortic root and the other pixels, but in this step S302, each pixel constituting the input image data is divided into two parts: Four parts are pixels corresponding to the right coronary lobe, pixels corresponding to the left coronary lobe, pixels corresponding to no coronary lobe, and other pixels.

It is the same as strengthening processing that can also be provided between the key point coarse detection processing and the aortic root segmentation processing in step S100, and the strengthening processing can also be provided between the key point accurate detection processing and heart valve segmentation processing in step S300 (in Fig. 4C (c)). The strengthening process can follow the strengthening process in step S100, and its details will not be repeated.

A modified example of the strengthening process will be described below using FIGS. 5A and 5B .

5A and 5B are diagrams for explaining modifications of the enhancement processing performed by the first processing function 100 in the medical image processing apparatus according to the first embodiment.

Specifically, as an enhanced process, not only can the key points detected in step S101 or S301 be used to provide a bounding box for cropping the image to be processed by the neural network for segmentation processing (hereinafter referred to as the segmentation neural network), but also It can be used in other ways to improve the performance of segmentation neural networks. For example, as shown in Figure 5A, the key point heat map can be generated first for the key points detected by the key point detection process, and then the heat map can be used as another input channel of the segmentation neural network and input together with the image to be processed by the segmentation neural network into the segmentation neural network. The modified example shown in FIG. 5A is mainly used in the inference process of the deep learning of the segmentation neural network. Due to the increase of the channels for segmentation neural network processing, anatomical key point information can be added to the segmentation neural network, so that the result of segmentation processing can contain more information about anatomical key points, and the performance of segmentation processing is improved.

In addition, as shown in FIG. 5B , after generating the heat map of the key points detected by the key point detection process, the heat map can be used as the internal loss function of the segmentation neural network, for example, a ratio can be provided for the vicinity of the key point area. Other regions are given greater weight to reduce under-segmentation, thereby improving the performance of the segmentation process. The modified example shown in FIG. 5B is mainly used in the training process of the deep learning of the segmentation neural network.

The above-mentioned strengthening treatment is not necessary. In addition, for the enhancement processing of the present invention, the cutting processing in FIG. 4A and the modified examples in FIG. 5A and FIG. 5B can be arbitrarily selected or used in combination. That is, in this embodiment, the enhancement processing includes at least one of the following: cropping the medical image data based on the key points detected by the key point detection processing; using the key point heat map detected by the key point detection processing as channel input Segment the neural network; use the key point heatmap detected by the key point detection process as the loss function of the segmentation neural network.

A modified example of the information enhancement process will be described below using FIGS. 6A and 6B .

6A and 6B are diagrams for explaining a modified example of information enhancement processing performed by the second processing function 200 in the medical image processing apparatus according to the first embodiment.

In the information enhancement processing according to this embodiment, geometric transformation or geometric constraints can be applied to make the processing target easier to identify. Figure 6A shows an example of information enhancement processing of the results of aortic valve segmentation through geometric transformation by taking section transformation as an example. The aorta has the shape of three leaflets when viewed in the cross-sectional direction, so when it is difficult to identify the target after cropping the existing cross-sectional images such as axial, sagittal, or coronal, you can use the existing The obtained key points are transformed into cross-sectional images such as the aorta that are easier to identify valves by transforming existing cross-sectional images such as axial or sagittal or coronal through geometric transformation-based cross-sectional transformations, thereby obtaining The effect of information enhancement.

Figure 6B shows another example of information enhancement processing of the results of aortic valve segmentation through geometric constraints by taking shape constraints as an example. The morphological characteristics of calcified valves and valves with only a single phase of systole/diastole are obtained in advance through valve labels, etc., as shown on the left, middle, and right sides of Figure 6B, which respectively reflect normal valves, calcified valves, and single-phase valves. The respective shape-restricted template images of the phase valves, according to which of the normal valves, calcified valves, and single-phase phase valves to be processed, input the corresponding shape-constrained template images into the network model to improve training performance and avoid unnecessary over-segmentation. By applying the shape restriction to the result of step S100, for example, the effect of information enhancement can be obtained, which is ready for subsequent processing such as step S300.

In addition, as the information enhancement processing of step S200, traditional grayscale information constraints, morphological constraints, etc. can also be used, as long as the existing features obtained in step S100 are used for information extraction, enhancement and fusion, so that after processing in step S200 The morphological information can guide subsequent processing such as step S300 to improve accuracy. In a word, the information enhancement processing in step S200 can be realized by ROI restriction, geometric transformation, geometric constraint, gray level information constraint, shape constraint and so on.

In this embodiment, the modified examples of the ROI restriction in FIG. 4B and the information enhancement processing in FIGS. 6A and 6B can also be selected arbitrarily or used in combination. In the present invention, the second processing function 200 executes information enhancement processing by executing at least one of ROI restriction, geometric transformation, geometric constraint, grayscale information constraint, and shape constraint.

Next, details of the correction processing described above will be described with reference to FIGS. 7A to 7C .

FIG. 7A is a flowchart illustrating an example of correction processing executed by the medical image processing apparatus 1 according to the first embodiment. 7B and 7C are schematic views for explaining the procedure of correction processing executed by the medical image processing apparatus 1 according to the first embodiment.

In this embodiment, for example, after the key point detection and heart valve segmentation in step S300 are completed, accurate key points, a mask image of the aortic root and, for example, an aortic valve mask image with three leaflets are obtained. The invention can then use these information to perform the correction process in step S500, so as to correct the shape of the heart valve, for example.

As shown in FIG. 7A , for example, the correction processing of the present invention may include three steps: first, in step S501, the control unit 10 extracts the largest connected domain from the segmentation result to prepare for subsequent processing, and the subsequent processing will be based on the The extracted maximum connected domain is the processing object. Next, since there may be holes in the segmentation result image, especially near the heart valves, in step S502 , the holes on the heart valves in the largest connected domain are filled. In addition, since steps S100 and S300 perform two separate segmentation operations, the segmentation results of the aortic root and the heart valve may not be uniform. For example, the rim (insertion region) of the heart valve may be under-segmented so that, for example, inside the sinus of Valsalva, the rim (the insertion region) of the heart valve may not be connected to the vessel wall. This detachment between the valve rim and the vessel wall Obviously, it does not conform to clinical knowledge. Therefore, in step S503, the edge of the valve is extended to the vessel wall, so as to unify the segmented images of the root of the aorta and the valve.

7B and 7C are schematic diagrams for explaining the procedure of correction processing performed by the medical image processing apparatus according to the first embodiment, showing the holes in step S502 in axial, sagittal, and coronal cross-sectional views, respectively. The process of padding and edge extension in step S503.

The extraction of the above-mentioned maximum connected domain in the correction process can be realized by the traditional method of finding the maximum connected domain in the prior art, hole filling, and the expansion of the segmentation result image can also be realized by the traditional method of the prior art such as image processing, the details of which are in This will not be repeated here.

As described above, according to the medical image processing apparatus 1 of the first embodiment, by repeatedly executing combination processing consisting of a plurality of types of processing on medical image data, within each processing unit, the previous feature extraction is at least partially affected. The next feature extraction is guided and assisted for better performance. Therefore, by repeatedly applying multiple feature extractions, accurate morphological information can be gradually generated from coarse to fine, especially for complex objects such as heart valves, as well as complex situations such as calcified and incomplete morphological structures of heart valves. with better performance.

In addition, by performing information enhancement processing between two processing units to modify the morphological information, it is possible to ensure that the processing accuracy is gradually improved from coarse to fine.

The first embodiment has been described above, but the technology disclosed in the present invention can be implemented in various forms other than the above-mentioned first embodiment.

(second embodiment)

In the first embodiment described above, an example has been described where the first processing executed by the first processing function 100 includes two types of processing, key point detection and segmentation. However, the embodiment is not limited thereto, and for example, the first treatment may include three or more treatments. The second embodiment of the present invention in which the first processing includes three types of processing will be described below using FIG. 8 as an example.

8 is a flowchart showing an example of heart valve detection processing executed by the medical image processing apparatus 1 according to the second embodiment. In the description of the second embodiment, differences from the above-mentioned first embodiment will be mainly described. In addition, in the description of the second embodiment, the same reference numerals are assigned to the same configurations as those of the above-mentioned first embodiment, and description thereof will be omitted.

According to the medical image processing device according to the second embodiment, the first processing function 100 not only performs key point detection processing (S101, S301) and segmentation processing (S102, S302), but also performs classification processing of calcified valves (S103, S303), namely In other words, in the present embodiment, the first processing executed by the first processing function 100 is combined processing consisting of three types of processing: key point detection, calcified valve classification, and segmentation.

Compared with the first embodiment, the workflow of the second embodiment adds calcified valve classification processing, and the control unit 10 uses the first processing function 100 to determine whether the processing object is a calcified valve according to the processing results of the key point detection processing before the segmentation processing. Or normal valve for classification. This added process can help with more accurate segmentation. For example, users can train two separate segmentation models for calcified valves and normal valves, and feed data into different segmentation models according to the classification results. Thus, in addition to the effects of the first embodiment, the second embodiment can also achieve beneficial technical effects of preventing data imbalance problems during training and obtaining better segmentation effects.

Figure 8 does not show the repeated steps corresponding to step S400 in the first embodiment, but it is obvious that the workflow in the second embodiment can also be repeated one or more times after step S300 as in the first embodiment The 2nd treatment and 1st treatment from coarse to fine to get better results.

(third embodiment)

In addition, in the above-mentioned first embodiment, an example in which the first processing function 100 executes combined processing consisting of two types of processing, the key point detection processing and the segmentation processing, has been described. However, the embodiment is not limited thereto, and for example, the first processing function 100 may execute a combined processing of key point detection processing and processing other than segmentation processing. A third embodiment of the present invention in which the first processing function 100 executes combined processing other than key point detection processing and segmentation processing will be described below using FIG. 9 as an example.

FIG. 9 is a flowchart showing an example of heart valve detection processing executed by the medical image processing apparatus 1 according to the third embodiment. In the description of the third embodiment, differences from the above-mentioned first and second embodiments will be mainly described. . In addition, in the description of the third embodiment, the same reference numerals are assigned to the same configurations as those of the above-mentioned first and second embodiments, and description thereof will be omitted.

According to the medical image processing apparatus 1 of the third embodiment, the first processing function 100 executes first processing (S100, S300) including segmentation processing (S104, S304) and blood flow estimation processing (S105, S305). For example, when image processing is performed on the mitral valve which is the atrioventricular valve between the left ventricle and the left atrium, the medical image processing apparatus 1 according to the third embodiment first inputs dynamic image data of one cardiac cycle in step S10, Next, the control unit 10 uses the first processing function 100 as the first processing of step S100. In step S104, the input dynamic image data is first subjected to segmentation processing of the left ventricle, and a mask image of the left ventricle of one cardiac cycle is segmented. Blood flow estimation processing is then performed in step S105. Since, for example, the mitral valve is located between the left ventricle and the left atrium, the blood flow velocity near the mitral valve has characteristics of large changes, so the region where the mitral valve is located can be located by estimating the blood flow velocity. The blood flow estimation process in step S105 may be a rough estimate to ensure the time performance of the process, for example, only the blood flow velocity of the entire left ventricle in the diastolic period of the ventricle is estimated. For example, during diastole, the mitral valve opens, and the direction of blood flow is from the left atrium to the left ventricle. Therefore, the blood flow velocity around the mitral valve changes greatly at this time, and the position of the mitral valve can be located more quickly by estimating the blood flow velocity. Area.

Next, according to the medical image processing apparatus 1 of the third embodiment, the control unit 10 executes positioning processing in step S202 as the second processing by the second processing function 200 . Specifically, the control unit 10 uses the second processing function 200 to locate the region with the largest change in the blood flow velocity in the left ventricle in the left ventricle segmentation result according to the blood flow velocity estimation result in step S105, which can be regarded as including the mitral flap structure.

Then, in the adjusted first processing (step S300) performed again, the combined processing of segmentation and blood flow estimation is also carried out, but the processing range is narrowed down compared with step S100, and the processing range is more accurate. deal with. Specifically, first in step S304, the control unit 10 uses the first processing function 100 to perform mitral valve segmentation processing on the image data narrowed by the positioning in step S202. Cube division. Then, different from the rough estimation of the blood flow velocity of the entire left ventricle in step S105, in step S305, the control unit 10 uses the first processing function 100 to only estimate the blood flow velocity near the mitral valve according to the result of mitral valve segmentation Perform precise estimation and more accurately locate the region where the mitral valve is located based on the blood velocity estimation results.

The method for estimating the blood flow velocity is not limited, and it can be realized by any known method. For example, when the input medical image data is ultrasonic data, the fluid information of the blood flow can be obtained from the ultrasonic image based on the Doppler method. When the input medical image data is an MRI image, fluid information can be acquired by performing known four-dimensional fluid analysis (4D flow analysis) on the MRI image. When the input medical image data is a multi-phase contrast CT (Computed Tomography: computed tomography) image, fluid information can be obtained by analyzing changes in CT values at various positions in the multi-phase contrast CT image. In addition, for example, for a circuit module that simulates the circulatory dynamics of a biological body (such as a human body) constructed based on a known windbox model (Windkessel model) or a pulse wave propagation model, in step S105, according to the shape information of the left ventricle or left atrium For fluid information, in step S305, in addition to the left ventricle and left atrium, fluid information is also obtained based on the shape of the mitral valve.

According to the present invention according to the third embodiment, more accurate morphological measurement results can be obtained similarly to the first embodiment by performing combined processing consisting of segmentation processing and blood flow estimation processing on medical image data from coarse to fine.

Fig. 9 does not show the repeated steps corresponding to step S400 of the first embodiment, the correction step corresponding to S500 and the measurement step corresponding to S600, but it is obvious that the workflow of the third embodiment can be as in the first embodiment These steps are also carried out as in the embodiment to obtain better performance and measurement results.

(fourth embodiment)

In the above-mentioned first to third embodiments, the cases where the heart, the aortic valve, and the mitral valve are taken as objects have been described as examples. However, the embodiment is not limited thereto, and other cardiac valves such as the tricuspid valve, or other than the heart, such as the lungs, may be used as targets. The fourth embodiment in which the lungs are treated will be described below using FIG. 10 as an example.

FIG. 10 is a flowchart showing an example of lung detection processing executed by the medical image processing apparatus 1 according to the fourth embodiment. In the description of the fourth embodiment, differences from the above-mentioned first to third embodiments will be mainly described. . In addition, in the description of the fourth embodiment, the same reference numerals are assigned to the same configurations as those of the above-mentioned first to third embodiments, or description thereof will be omitted.

According to the medical image processing apparatus 1 of the fourth embodiment, the control unit 10 still executes the first processing (S100, S300) including the key point detection processing (S106, S306) and the segmentation processing (S107, S307) through the first processing function 100. In addition, in this embodiment, according to the anatomical and morphological characteristics of the lungs, the control unit 10 repeats the second processing and the first processing (S400) successively through three iterations after the first processing (S100, S300) twice. to obtain more accurate morphological measurements.

First, in step S30 , the original volume region of the medical image data of the lungs to be processed is input to the medical image processing apparatus 1 .

Next, in step S100, the control unit 10 uses the first processing function 100 to perform key point detection processing of the main bronchi on the medical image data (step S106), and then the control unit 10 uses the first processing function 100 , based on the key points of the main bronchus detected in step S106, a rough segmentation process of the lungs is performed, for example, a segmentation process of the left lung and the right lung is performed (step S107). The main bronchus comes from the first-level airway and is divided into parts of the left lung and the right lung. Therefore, as the first processing, the left lung and the right lung can be quickly located and detected through the key point detection processing and segmentation processing of the main bronchus.

Next, in step S203, the control unit 10 uses the second processing function 200 to perform positioning and information enhancement of the lung parenchyma region as the second processing, and select a more accurate VOI for subsequent precise processing. Step S203 may adopt the same processing as step S200, and the details thereof will not be repeated here.

Next, in step S300, as the adjusted first processing to be executed again, the control unit 10 uses the first processing function 100 to perform accurate key point processing with a spatially narrowed processing range according to the processing results of steps S100 and S203. Detection processing and segmentation processing, for example, by combining main bronchi and lung regions, detection processing of key points of bronchioles from second or higher airways is performed (step S306 ). After that, the control unit 10 uses the first processing function 100 to perform precise segmentation based on the key points of the bronchioles detected in step S306 to obtain mask images of five lung lobes (step S307 ).

Then, the control unit 10 repeats the second processing and the first processing in sequence (S400 in FIG. 10 ).

Specifically, as the re-executed second processing, in step S204, the control unit 10 uses the second processing function 200 to perform positioning and information enhancement processing on the lung lobe region according to the anatomical morphology of the lung. Here, the lung lobe region can be located and corrected according to the clinical features, for example, the lung lobe region can be corrected according to the characteristic that the bronchioles belonging to one lung lobe cannot be segmented into other lung lobes.

Subsequently, as the readjusted first processing to be performed again, in step S110, the control unit 10 uses the first processing function 100 to carry out spatial further More precise keypoint detection processing and segmentation processing with reduced processing range. For example, considering that each lung lobe is an independent system, so the pulmonary blood vessels of one lung lobe will not be detected in other lung lobes, so firstly in step S111, the control unit 10 uses the first processing function 100 to obtain the result based on step S204 Each lung lobe detects the key points of the pulmonary vessels, and then performs segmentation of the lung segments based on the detected key points of the pulmonary vessels in step S112.

Next, in step S500 , the control unit 10 uses the information obtained in the previous steps to correct the obtained image of the lung segment. Similar to the first embodiment, the correction process is to make the obtained lung image more suitable for the subsequent measurement process (not shown in FIG. 10 ), for example, and is not essential to the present invention.

According to the present invention according to the fourth embodiment, a more accurate morphological measurement result can also be obtained by performing combined processing consisting of multiple types of processing on the medical image data from coarse to fine with the lung as an object.

(fifth embodiment)

In addition, for example, in the above-mentioned first to fourth embodiments, it is assumed that the control unit 10 executes the first processing including at least two different types of processing on the medical image data, and executes the spatial reduction on the processing result of the first processing. Although the first processing after the adjustment of the processing range is used, the embodiments of the technology disclosed in the present application are not limited thereto.

For example, the control unit 10 may execute a first type of processing multiple times while adjusting to narrow the processing range in space, and then, based on the processing result, execute a second type of processing different from the first type of processing. As a result of the processing of the two types of processing, the first type of processing adjusted to further narrow the processing range in space is executed.

Hereinafter, an example of such a case will be described as a fifth embodiment. In addition, in the description of the fifth embodiment, the differences from the above-mentioned embodiment will be mainly described, and the detailed description of the content overlapping with the above-mentioned embodiment will be omitted.

In the present embodiment, the control unit 10 has a first processing function 100 , a second processing function 200 , and a parameter setting function 300 as in the above-mentioned embodiment.

Specifically, the control unit 10 uses the first processing function 100 to execute the first type of processing on the medical image data, and re-executes the first type of processing on the spatially narrowed range of the processing result of the first type of processing. Furthermore, the control unit 10 uses the first processing function 100 to execute a second type of processing that is different from the first type of processing for the processing result of the first type of processing that has been narrowed in space. Based on the processing results of the first type of range and the processing results of the second type of processing, the range is further narrowed in space, and the first type of processing is executed again, thereby extracting the target area.

FIG. 11 is a diagram showing a processing flow and processing results of an example of processing executed by the medical image processing apparatus 1 according to the fifth embodiment.

In addition, an example will be described here in which the first type of processing is segmentation processing and the second type of processing is key point detection processing. In addition, an example will be described here where the medical image data is medical image data of the heart and the target region is the aortic valve.

As shown in FIG. 11 , first, in step S1 , medical image data of the heart to be processed is input to the medical image processing apparatus 1 . For example, a CT image of the heart is input to the medical image processing apparatus 1 as medical image data.

Next, in step S121 , the control unit 10 uses the first processing function 100 to perform segmentation processing of the aortic root (Aortic Root) on the medical image data of the heart input in step S1 . In addition, although not shown, in step S121 , the control unit 10 may set the parameters of the division processing through the parameter setting function 300 before executing the division processing.

At this time, the control unit 10 roughly extracts the region of the aortic root by performing segmentation processing on the medical image data of the heart using the first processing function 100 .

For example, the control unit 10 executes the segmentation process of the aortic root using the deep learning neural network through the first processing function 100 . The input of this deep learning neural network is, for example, medical image data including the whole heart, and the output is, for example, a group of coordinates of pixels corresponding to a general area of the aortic root. Here, as a deep learning neural network, for example, UNet-based processing based on the nnUNet framework can be used.

Next, in step S122 , the control unit 10 uses the first processing function 100 to execute adjusted division processing in which the processing range is spatially narrowed based on the processing result of the division processing executed in step S121 . Here, the adjustment of the division processing is realized, for example, by adjusting the parameters of the parameter setting function 300 .

At this time, the control unit 10 executes the adjusted division processing in which the processing range is spatially narrowed by the first processing function 100, thereby extracting the main body more accurately than the division processing executed in step S121. The area at the root of the artery.

Specifically, the control unit 10 uses the first processing function 100 to cut out only the periphery of the aortic root region in the medical image data based on the rough aortic root region extracted in step S121, and the cut out Region performs a segmentation process to more precisely extract the region of the aortic root.

For example, the control unit 10 executes the segmentation process of the aortic root using the deep learning neural network through the first processing function 100 . The input of this deep learning neural network is, for example, an image cut out from the medical image data of the heart around only the region of the aortic root, and the output is, for example, a coordinate group of pixels corresponding to the region of the aortic root. Here, as a deep learning neural network, for example, UNet-based processing based on the nnUNet framework can be used.

Next, in step S221 , the control unit 10 uses the second processing function 200 to execute correction processing on the processing result of the division processing executed in step S122 .

Specifically, the control unit 10 uses the second processing function 200 to execute correction processing for correcting the region of the aortic root extracted in step S122 by image processing. In addition, this correction processing is for making the processing result of the division processing more suitable for the subsequent processing, and it is not an essential processing in this embodiment.

Next, in step S123 , the control unit 10 uses the first processing function 100 to execute key point detection processing on the processing result of the division processing executed in step S122 .

Specifically, the control unit 10 uses the first processing function 100 to clip only the area around the aortic root region in the medical image data based on the region of the aortic root corrected in step S221, and the clipped Region performs a keypoint detection process, detecting clinically important keypoints of the anatomical morphology of the aorta.

For example, the control unit 10 detects the nadir point of the left coronary valve (LCC Nadir point), the nadir point of the right coronary valve (RCC Nadir point), and the non-coronary valve The lowest point (NCC Nadir point), no coronary valve-left coronary valve junction (N-L Commissure point), right coronary valve-left coronary valve junction (R-L Commissure point), no coronary valve-right coronary valve junction (N-R commissure point) point), right coronary ostium, and left coronary ostium.

Here, the nadir point (Nadir point) refers to a point on the valve cusp that is closest to the left ventricular outflow tract (Left Ventricular Outflow Tract: LVOT). In addition, a commissure point refers to a point at which the valve cusps meet each other. In addition, the inferior coronary ostium point (coronary ostium point) refers to the respective entrances of the right coronary artery (Right Coronary Artery: RCA) and the left coronary artery (Left Coronary Artery: LCA) located on the proximal side. ) points.

For example, the control unit 10 uses the first processing function 100 to execute key point detection processing using a deep learning neural network. The input of the deep learning neural network is, for example, an image of the region of the aortic root, and the output is, for example, the nadir of the left coronary valve, the nadir of the right coronary valve, the nadir of the non-coronary valve, the non-coronary valve-left coronary valve juncture, the right coronary Coordinates of valve-left coronary valve commissure point, non-coronary valve-right coronary valve commissure point, right coronary ostium inferior edge point, left coronary artery inferior edge point. Here, as the deep learning neural network, for example, Spatial Configuration-Net (SCN: Spatial Configuration Network) can be used.

Next, in step S124, the control unit 10 uses the first processing function 100 to further perform spatially further Adjusted segmentation processing for narrowing the processing range extracts the region of the aortic valve cusp. Here, the adjustment of the division processing is realized, for example, by adjusting the parameters of the parameter setting function 300 .

At this time, the control unit 10 uses the first processing function 100 to perform an adjusted segmentation process that further narrows the processing range spatially based on the key points extracted in step S123, thereby extracting from the aorta extracted in step S122 The region of the root extracts the region of the aortic cusp.

Specifically, the control unit 10 cuts out only the periphery of the aortic root region in the medical image data based on the region of the aortic root corrected in step S221 using the first processing function 100 , and performs Segmentation processing whereby the region of the aortic valve cusp is extracted.

For example, the control unit 10 executes segmentation processing through the first processing function 100, thereby extracting right coronary valve (Right Coronary Cusp: RCC), left coronary valve (Left Coronary Cusp: LCC), non-coronary valve (Non Coronary Cusp: NCC) of the regions.

For example, the control unit 10 uses the first processing function 100 to perform segmentation processing using a deep learning neural network. The input of this deep learning neural network is, for example, an image cut out only around the area of the aortic root in the medical image data of the heart, and the output is, for example, pixels corresponding to each area of the right coronary valve, left coronary valve, and non-coronary valve coordinate group. Here, as a deep learning neural network, for example, UNet-based processing based on the nnUNet framework can be used.

Next, in step S222, the control unit 10 uses the second processing function 200 to perform correction processing on the processing results of the segmentation processing executed in steps S122 and S124 and the processing results of the key point detection processing executed in step S123.

Specifically, the control unit 10 uses the second processing function 200 to combine the region of the aortic root extracted in step S122, the region of the aortic valve cusp extracted in step S124, and the region detected in step S123. Correction processing in which the positions of key points are corrected by image processing. In addition, the correction processing is used to make the processing results of the segmentation processing and key point detection processing more suitable for subsequent processing such as measurement processing, and is not an essential processing in this embodiment.

Next, in step S600 , the control unit 10 performs morphological measurement based on the processing results of the segmentation processing and the processing results of the key point detection processing.

Specifically, the control unit 10 performs morphological measurement similarly to the above-described embodiment based on the aortic root region, aortic valve cusp region, and key point positions corrected in step S222.

In addition, in the above-mentioned example, after the medical image data is segmented in step S121, the adjusted segment processing for spatially narrowing the processing range is performed once in step S122, but the present embodiment is not limited thereto. For example, in step S122 , the control unit 10 may sequentially execute the division processing a plurality of times while adjusting the processing results of the division processing executed immediately before to spatially narrow the processing range. Thereby, a more accurate morphological measurement result can be obtained.

In addition, in the above-mentioned example, the key point detection process and the segmentation process are each performed once in steps S123 and S124, but this embodiment is not limited to this. For example, the control unit 10 may use the first processing function 100 to repeatedly execute the key point detection processing and the segmentation processing of the further spatially reduced range in sequence, thereby extracting the region of the aortic valve cusp. Thereby, a more accurate morphological measurement result can be obtained.

In addition, in the above example, through the key point processing in step S123, the nadir of the left coronary valve, the nadir of the right coronary valve, the nadir of no coronary valve, the junction of no coronary valve and left coronary valve, The right coronary valve-left coronary valve coaptation point, no coronary valve-right coronary valve coaptation point, right coronary ostium inferior edge point, left coronary artery inferior edge point, these eight feature points, but this embodiment is not limited thereto. For example, in step S123 , it is not necessary to detect all of the eight feature points as key points, as long as at least feature points required for subsequent segmentation processing and measurement are detected as key points. That is, the control unit 10 may execute the key point detection process through the first processing function 100, thereby detecting the nadir of the left coronary valve, the nadir of the right coronary valve, the nadir of the non-coronary valve, and the nadir of the non-coronary valve-left coronary valve as key points. At least one of coronary valve juncture, right coronary valve-left coronary valve juncture, no coronary valve-right coronary valve juncture, inferior border of right coronary ostium, inferior border of left coronary ostium.

In addition, in the above example, regions of the aortic root, right coronary valve, left coronary valve, and non-coronary valve are extracted through the segmentation processing in steps S121, S122, and S124, but the present embodiment is not limited thereto. For example, in steps S121 , S122 , and S124 , it is not necessary to extract all of these four regions, but only to extract at least the regions required for subsequent segmentation processing and measurement. In other words, the control unit 10 may perform segmentation processing using the first processing function 100 to extract at least one region of the aortic root, right coronary valve, left coronary valve, and non-coronary valve.

In addition, in the above-mentioned example, the case where the target area is the aortic valve has been described, but this embodiment is not limited thereto. The same applies to the case of the target area. Furthermore, the present embodiment is also similarly applicable to a case where, for example, a target area other than a heart valve is used, such as the lung.

In addition, in the above example, it is assumed that the second type of processing is key point detection processing and the first type of processing is segmentation processing, but this embodiment is not limited thereto. For example, as in the third embodiment, when the target area is the mitral valve, the second type of processing may be blood flow estimation processing, and the first type of processing may be left ventricle or mitral valve segmentation processing.

As described above, according to the medical image processing apparatus 1 according to the fifth embodiment, after executing the first type of processing a plurality of times while adjusting to narrow the processing range spatially, the same processing result as the first type is executed based on the processing result. Different second types of processing, based on the processing results of the second type of processing, execute the adjusted first type of processing to further narrow the processing range in space, and can generate accurate morphology step by step from coarse to fine In particular, better performance can be obtained in complicated situations such as complex objects such as heart valves and calcified morphological structures and incomplete morphological structures of heart valves. Therefore, according to the fifth embodiment, more accurate morphological measurement results can be obtained.

In addition, the image processing, key point detection, segmentation, deep learning neural network training and reasoning described in the above embodiments can all be implemented in conventional ways in the prior art, and detailed descriptions are omitted here.

In addition, the above-mentioned first to fourth embodiments have been described for the purpose of specifying the spatial position of an object, but the technology disclosed in this application is not limited thereto, and may be used for specifying time information or concentration information. For example, when specifying a specific cardiac phase, as the first process, a process of specifying a target phase based on the motion of a heart valve and a process of specifying a target phase based on a blood flow state may be performed.

In addition, the above-mentioned fifth embodiment has also been described for the purpose of specifying the spatial position of an object, but the technology disclosed in this application is not limited thereto, and may be used for specifying time information or concentration information. For example, when specifying a specific cardiac phase, as the first process, a process of specifying the target phase from the motion of the heart valve may be performed, and as the second process, a process of specifying the target phase from the blood flow state may be performed.

In addition, the control unit 10 described in the above-mentioned embodiment is realized by a processor, for example. In this case, the processing function of the control unit 10 is stored in the memory 20 in the form of a program executable by a computer, for example. Furthermore, the control unit 10 realizes processing functions corresponding to the respective programs by reading and executing the respective programs stored in the memory 20 . In other words, the control unit 10 has each processing function shown in FIG. 1 in a state where each program is read.

In addition, the control unit 10 described in the above-mentioned embodiment is not limited to being realized by a single processor, but may be configured by combining a plurality of independent processors, and each processing function is realized by each processor executing a program. In addition, each processing function of the control unit 10 may be appropriately distributed or integrated into a single or a plurality of processing circuits for implementation. In addition, each processing function of the control part 10 can also be realized by only hardware, such as a circuit, only software, or a mixture of hardware and software. In addition, although the example in which the program corresponding to each processing function is memorize|stored in the single memory 20 was demonstrated here, embodiment is not limited to this. For example, a program corresponding to each processing function may be distributed and stored in a plurality of memories, and the control unit 10 may be configured to read and execute each program from each memory.

The technology disclosed in this application can be realized not only as the above-mentioned medical image processing device, but also as a medical image processing method and a medium storing a medical image processing program.

In addition, the medical image processing apparatus according to the present application may be installed in a medical image diagnosis apparatus, or may be a case where the medical image processing apparatus executes processing independently. In such a case, the medical image processing apparatus has a processing circuit that executes the same processing as the above-mentioned first processing function, second processing function, and parameter setting function, and a memory that stores programs and various information corresponding to each function. . Further, the processing circuit acquires three-dimensional medical image data from a medical imaging diagnostic device such as an ultrasonic diagnostic device or an image storage device via a network, and executes the above-mentioned processing using the acquired medical image data. Here, the processing circuit is, for example, a processor that realizes a function corresponding to each program by reading and executing the program from the memory.

The term "processor" used in the above description refers to, for example, a CPU (Central Processing Unit: Central Processing Unit), a GPU (Graphics Processing Unit: Graphics Processing Unit), or an application-specific integrated circuit (Application Specific Integrated Circuit : ASIC, application-specific integrated circuit), programmable logic device (for example, simple programmable logic device (Simple Programmable Logic Device: SPLD), compound programmable logic device (Complex Programmable Logic Device: CPLD), and field programmable gate array ( Field Programmable Gate Array: FPGA)) and other circuits. The processor realizes the function by reading and executing the program stored in the memory. In addition, instead of storing the program in the memory, the program is directly loaded into the circuit of the processor. In this case, the processor realizes the function by reading and executing the program loaded in the circuit. In addition, each processor of the present embodiment is not limited to the case where the processor is configured as a single circuit, but a plurality of independent circuits are combined to form a single processor to realize its functions.

In addition, each component of each device illustrated in the description of the above-mentioned embodiment is functional and conceptual, and does not necessarily have to be physically configured as illustrated. That is, the specific form of the distribution/integration of each device is not limited to the illustrated form, and all or a part thereof may be configured by functionally or physically dispersing/integrating in arbitrary units according to various loads or usage conditions, etc. . Furthermore, all or any part of each processing function performed by each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as wired logic hardware.

In addition, the processing methods described in the above-mentioned embodiments can be realized by executing a processing program prepared in advance by a computer such as a personal computer or a workstation. This processing program can be distributed via a network such as the Internet. In addition, the processing program can also be recorded to the hard disk, floppy disk (Flexible Disk: FD), CD (Compact Disk)-ROM (ReadOnly Memory), MO (Magneto-optical Disk), DVD ((Digital Versatile Disk), USB (Universal In computer-readable non-transitory recording media such as Flash memory such as Serial Bus) memory and SD (Secure Digital) card memory, it is executed by reading from the non-transitory recording medium by the computer.

In addition, among the processes described in the above-mentioned embodiments, all or part of the processes described as being performed automatically may be performed manually, or all or part of the processes described as being performed manually may be performed manually. A part is performed automatically by known methods. In addition, information including processing procedures, control procedures, specific names, various data, and parameters shown in the above text or drawings can be changed arbitrarily except when specifically described.

In addition, various data handled in this specification are typically digital data.

According to at least one embodiment described above, more accurate morphological measurement results can be obtained.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and do not limit the scope of the invention. These embodiments can be implemented in other various forms, and various omissions, substitutions, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the present invention described in the claims and its equivalent scope.

Regarding the above embodiments, the following supplementary notes are disclosed as one aspect and optional features of the invention.

(Note 1)

A medical image processing device, comprising a control unit, the control unit executes a first type of processing on medical image data; 1 type of processing; for the processing result of the first type of processing in the spatially narrowed range, execute a second type of processing different from the first type; The processing result of the first type of processing and the processing result of the second type of processing in the range are further narrowed in space, and the first type of processing is executed again, thereby extracting the target area.

(Note 2)

Alternatively, the first type of processing may be segmentation processing, and the second type of processing may be key point detection processing.

(Note 3)

Alternatively, the control unit may execute the key point detection process and the segmentation process using a deep learning neural network.

(Note 4)

The control unit may perform correction processing on the processing result of the first type of processing in the spatially narrowed range before executing the second type of processing.

(Note 5)

Alternatively, the medical image data may be medical image data of a heart; and the target area may be a heart valve.

(Note 6)

It may also be that the target area is the aortic valve; the control unit extracts at least one area of the aortic root, right coronary valve, left coronary valve, and non-coronary valve by performing the segmentation process; Key point detection processing, as key points to detect the lowest point of the left coronary valve, the lowest point of the right coronary valve, the lowest point of the non-coronary valve, the non-coronary valve-left coronary valve joint, the right coronary valve-left coronary valve joint, the non-coronary valve Valve - at least one of the right coronary valve coaptation point, the inferior edge of the right coronary ostium, and the inferior edge of the left coronary ostium.

(Note 7)

The control unit may extract the target area by sequentially and repeatedly executing the second type of processing and the first type of processing for the further spatially narrowed range.

(Note 8)

A medical image processing method, comprising: a step in which a control unit executes a first type of processing on medical image data; re-executing the first type of processing on a spatially narrowed range based on a processing result of the first type of processing The step of processing; for the processing result of the first type of processing in the spatially narrowed range, the step of executing a second type of processing different from the first type; The processing results of the first type of processing and the processing results of the second type of processing in the narrowed range are further narrowed in space, and the first type of processing is executed again, thereby extracting the target area step.

(Note 9)

A computer-readable storage medium storing a program for causing a computer to execute: a process of performing a first type of processing on medical image data; The process of executing the first type of processing in a narrowed range; performing a second type of processing different from the first type of processing for the processing result of the first type of processing in the spatially narrowed range The step of: performing the first step on the range that is further narrowed in space based on the processing result of the first type of processing and the processing result of the second type of processing in the spatially narrowed range Type of processing, thereby extracting the process of the target area.

(Additional Note 10)

A medical image processing device, comprising a control unit, the control unit executes a first type of processing on medical image data; and then executes the first type of processing on a range narrowed based on a processing result of the first type of processing. processing; performing a second type of processing different from the first type for the processing result of the first type of processing in the narrowed range; for the first type of processing based on the narrowed range Based on the results of the processing and the processing results of the second type of processing, the range is further narrowed, and the first type of processing is executed again, thereby extracting the target area.

Claims (14)

1. A medical image processing device, comprising a control unit, the control unit, performing first processing including at least two different kinds of processing on the medical image data; and The first processing is re-executed for the range that has been spatially narrowed based on the processing result of the first processing. 2. The medical image processing device according to claim 1, the control unit, after performing said first processing on said medical image data, performing a second processing on the processing result of the first processing; and The first processing for the spatially reduced range is executed on the processing result of the second processing, and an image representing the target area in the medical image data is obtained based on the processing result of the first processing. 3. The medical image processing device according to claim 1 or 2, The processing order of the different type of processing in the first processing for the spatially narrowed range is the same as the processing of the different type of processing in the first processing for the medical image data in the same order. 4. The medical image processing device as claimed in claim 2, the control unit, As the first processing, key point detection processing is performed on the medical image data, and segmentation processing of the object region is performed based on a processing result of the key point detection processing; As the second processing, information enhancement processing is performed on the processing result of the first processing. 5. The medical image processing device as claimed in claim 2, The control unit further repeatedly executes the second processing and the first processing for the spatially reduced range sequentially after executing the first processing for the spatially reduced range, An image representing the object area is thereby obtained. 6. The medical image processing device as claimed in claim 2, The control section also executes correction processing on the image representing the target area. 7. The medical image processing device as claimed in claim 4, The control unit executes the information enhancement process by performing at least one of area limitation of the ROI, that is, a region of interest, geometric transformation, geometric constraint, constraint based on grayscale information, and shape constraint. 8. The medical image processing device as claimed in claim 4, The control section executes the key point detection processing and the segmentation processing using a deep learning neural network. 9. The medical image processing device according to claim 8, The control unit executes the segmentation process after performing an enhancement process on the medical image data based on a processing result of the key point detection process, The strengthening treatment includes at least one of the following: performing cropping processing on the medical image data based on the key points detected by the key point detection processing; The key point heat map detected by the key point detection process is input as a channel into the deep learning neural network; The key point heat map detected by the key point detection process is used as the loss function of the deep learning neural network. 10. The medical image processing device according to claim 4, The medical image data is image data of a heart, and the target area is a heart valve. 11. The medical image processing device according to claim 10, the subject area is the aortic valve, In the key point detection process, detect the lowest point of the left coronary valve, the lowest point of the right coronary valve, the lowest point of the non-coronary valve, the non-coronary valve-left coronary valve junction, the right coronary valve-left coronary valve junction, the non-coronary valve- At least one of the right coronary valve coaptation point, the lower edge of the right coronary ostium, and the lower edge of the left coronary ostium is used as a key point. 12. A medical image processing method, comprising: The control unit executes a first processing step including at least two different types of processing on the medical image data; and the control unit re-executes the first processing step on a spatially narrowed range based on a processing result of the first processing. 13. A computer-readable storage medium storing a program that causes a computer to execute: a step of performing first processing including at least two different types of processing on the medical image data; 14. A medical image processing device comprising a control unit, the control unit, performing first processing including at least two different kinds of processing on the medical image data; and The first processing is re-executed for the narrowed range based on the processing result of the first processing.

Images