TWI769370B - Focus detection apparatus and method thereof - Google Patents

Abstract

A focus detection apparatus and a method thereof are provided. In the method, a medical image at a first location is obtained, and a first focus is identified on the medical image of the first location. Another medical image at a second location is obtained, and a second focus is identified on the medical image of the second location. Then, whether the second focus is the first focus is determined. If the second focus is the first focus, the first and second focus is combined as a single focus to be presented. Accordingly, every medical image can be determined whether is the same focus as the previously identified focus in order, so as to provide more complete information for instant aided detection technology.

Description

Lesion detection device and method thereof

The present invention relates to a medical image detection, and more particularly, to a real-time lesion detection device and a method thereof.

Nowadays, Computer Aided Detection (CADe) systems have been widely used in clinical practice to automatically identify lesions such as tumors, masses or calcifications to assist doctors in their diagnosis and treatment. Generally speaking, the computer-aided detection technology needs to obtain all the scanned images of the target part of an object first, and then identify the lesions from all the scanned images. With the advancement of technology, some computer-aided detection systems can even identify lesions in real time during the process of scanning objects. However, the existing real-time identification technology can only know whether there is a lesion, but cannot immediately determine whether the newly identified lesion and the previously identified lesion are the same lesion, thus affecting the doctor's diagnosis and treatment judgment. It can be seen that it is necessary to improve the lesion detection technology for medical images.

In view of this, the present invention provides a lesion detection device and a method thereof, which can sequentially determine whether every other medical image is the same lesion, thereby providing a real-time detection technology with relatively complete data.

The lesion detection method of the present invention includes the following steps. Obtain the medical image of the first position. There is a first lesion in the medical image identifying the first position. A medical image of a second location is obtained, and the first location is different from the second location. There is a second lesion in the medical image identifying the second position. Determine whether the second lesion is the first lesion. If the second lesion is the first lesion, the first lesion and the second lesion are combined into a single lesion for presentation.

In an embodiment of the present invention, the above-mentioned determining whether the second lesion is the first lesion includes the following steps. According to the distance from the second lesion to the centroid of the position of the lesion group to which the first lesion belongs, the second lesion is determined as the first lesion.

In an embodiment of the present invention, the above-mentioned determining that the second lesion is the first lesion includes the following steps. Obtain the first distances from the second lesions to the centroid of the positions of several lesion groups, and the first lesion belongs to one of those lesion groups, and the centroid of the position of each lesion group is the position of all the lesions contained in each of them. on the center of gravity. It is judged whether the smallest first distance among these first distances is smaller than the distance threshold value. If the minimum first distance is not less than the distance threshold, the second lesion is added to a new lesion group different from those of the lesion group. If the minimum first distance is smaller than the distance threshold, the second lesion is added to the kth lesion group corresponding to the minimum first distance, where k is a positive integer. Obtain the second distance from all the lesions included in the k-th lesion group to the centroid of the position of the k-th lesion group. It is determined whether the largest second distance among those second distances is less than the distance threshold value. If the maximum second distance is less than the distance threshold, the grouping is ended.

In an embodiment of the present invention, after judging whether the largest second distance among those second distances is smaller than the distance threshold, the following steps are further included. If the maximum second distance is not less than the distance threshold, move the yth lesion corresponding to the maximum second distance out of the kth lesion group, and update the position center of gravity of the kth lesion group, where y is a positive integer.

In an embodiment of the present invention, obtaining the medical image of the first position includes the following steps. Scan and record medical images with a scanner. The position of the scanner is recorded as the position of the medical image.

In an embodiment of the present invention, the above-mentioned method for detecting a lesion further includes the following steps. If the scanner is not in contact, the recording of the medical image and the display of the movement track of the scanner will be suspended.

In an embodiment of the present invention, the above-mentioned method for detecting a lesion further includes the following steps. According to the three-dimensional position information corresponding to the medical images of several different positions, the movement track formed by the corresponding positions of the medical images of each different positions is presented in a three-dimensional manner. A three-dimensional virtual object is formed, and the positions of all suspicious lesions are presented on the three-dimensional virtual object.

In an embodiment of the present invention, the above-mentioned method for detecting a lesion further includes the following steps. The virtual part range is generated according to the position information corresponding to the medical images at several different positions.

In an embodiment of the present invention, the above-mentioned method for detecting a lesion further includes the following steps. Obtain past lesion information and present the past lesion information.

The lesion detection device according to the embodiment of the present invention includes a storage device and a processor. The memory records several modules and medical images. The processor is coupled to the storage, and accesses and loads those modules recorded by the storage. Those modules include the lesion identification module and the same lesion judgment module. The lesion identification module identifies a first lesion in the medical image at the first position, and a second lesion in the medical image at the second position, and the first position is different from the second position. The same lesion determination module determines whether the second lesion is the first lesion, and if the second lesion is the first lesion, the first lesion and the second lesion are combined into a single lesion for presentation.

Based on the above, the lesion detection device and the method according to the embodiments of the present invention can sequentially determine whether the currently identified lesion is the same lesion as the previously identified lesion during the scanning process. In addition, combined with the pause function, 3D rendering, virtual part range rendering and past lesion information rendering, it will be able to provide a rich and complete scanning assistance technology.

In order to make the above-mentioned features and advantages of the present invention more obvious and easy to understand, the following embodiments are given and described in detail with the accompanying drawings as follows.

FIG. 1 is a block diagram of components of a lesion detection device 1 according to an embodiment of the present invention. Referring to FIG. 1 , the lesion detection apparatus 1 includes at least but not limited to a storage 110 , a display 130 and a processor 150 . The lesion detection device 1 may be a computer host, a server, or even a scanner for real-time medical imaging (eg, a probe based on ultrasound technology, an optical imaging lens, or any mobile probe, etc.).

The storage 110 may be any type of fixed or removable random access memory (RAM), read only memory (ROM), flash memory, conventional hard disk drive, solid state drive A solid-state drive or similar device is used to record the lesion identification module 111, the same lesion determination module 112, the position processing module 113, the three-dimensional presentation module 114, the site marking module 115, and the past information providing module Group 116 and other software programs, two-dimensional or three-dimensional medical images (eg, automated breast ultrasound (ABUS), tomosynthesis, magnetic resonance imaging (MRI) and other images), lesions , information on lesion location, lesion group, past lesion information and other information. The aforementioned modules, data, files and information will be described in detail in subsequent embodiments.

The display 130 may be a display of various display technologies such as Liquid-Crystal Display (LCD), Light-Emitting Diode (LED), and Organic Light-Emitting Diode (OLED).

The processor 150 is connected to the storage 110 and the display 130, and can be a central processing unit (CPU), or other programmable general-purpose or special-purpose microprocessors (Microprocessors), digital signal processors (DSPs), Programmable controller, application specific integrated circuit (ASIC) or other similar element or a combination of the above. In the embodiment of the present invention, the processor 150 is used to execute all operations of the lesion detection apparatus 1 , and can access and execute the modules recorded in the above-mentioned storage 110 .

In order to facilitate the understanding of the operation process of the embodiment of the present invention, the following will describe in detail the process of detecting the lesion of the medical image by the lesion detection apparatus 1 in the embodiment of the present invention by referring to several embodiments. Hereinafter, the method according to the embodiment of the present invention will be described in conjunction with various elements and modules of the lesion detection device 1 . Each process of the method can be adjusted according to the implementation situation, and is not limited to this.

FIG. 2 is a flow chart illustrating a method for detecting a lesion according to an embodiment of the present invention. Please refer to FIG. 2 , after capturing network packets, uploading by users, obtaining medical images through external or built-in storage media (eg, flash drives, CD-ROMs, external hard drives, etc.), or even directly through external or built-in scanners ( For example, an ultrasound probe, a camera, a video camera, etc.) scan a specific object (eg, a person or other animals) to record a medical image, and store the medical image in the storage 110 . It is worth noting that each medical image will correspond to position information (for example, the position of the scanner relative to the object to be scanned (for example, the distance and angle to a specific part, custom coordinates, etc.), or the medical image corresponds to the object to be scanned. Scanning a specific area on an object, etc.), this location information will be used as the location of the medical image (may be a certain point or area).

First, the lesion identification module 111 obtains the medical image at the first position (step S210 ), which means that the scanner records and obtains the medical image at the first position relative to the object to be scanned, or the medical image is for the object to be scanned. the first position. Next, the lesion identification module 111 identifies whether there is a first lesion in the medical image of the first position. The lesion identification module 111 can identify the lesions by using, for example, texture feature analysis, classifier, deep learning technology or any other computer-aided diagnosis technology, which is not limited in the embodiment of the present invention. And if the lesion identification module 111 identifies that the medical image of the first location has a first lesion (step S220 ), the location of the first lesion is recorded. For example, FIG. 3A is an example showing that there is a lesion FF in the image block FL corresponding to the first position. In addition, assuming that the medical image at the first position is the first analyzed medical image, the lesion identification module 111 will classify the medical image into the first lesion group.

Next, the lesion identification module 111 obtains the medical image at the second position in the same or similar manner as in step S210 (that is, the scanner records and obtains the medical image at the second position relative to the object to be scanned, or the medical image is For the second position on the object to be scanned) (step S230), and in the same or similar manner as step S220, it is judged whether the medical image of the second position has a second lesion (step S240), and the position of the second lesion is recorded, This second lesion is classified as a non-clustered lesion (ie, not yet belonging to any clustered lesion, and may be temporarily stored in a queue or other staging area). For example, in FIG. 3B , the second position next to the image block FL identified in FIG. 3A corresponds to the image block SL where the presence of the lesion SF is detected. It should be noted that the second position is different from the aforementioned first position, and the medical image at the second position is the next image following the medical image at the first position. That is, the scanner does not generate other scanned medical images from the first position to the second position.

Next, the same lesion determination module 112 determines whether the second lesion is the first lesion identified by the medical image at the first position (step S250 ). In one embodiment, the same lesion determination module 112 determines that the second lesion is the first lesion according to the distance from the second lesion to the center of gravity of the position of the lesion group to which the first lesion belongs. For example, referring to FIG. 4 , which is a flowchart of lesion grouping, the same lesion determination module 112 obtains ungrouped lesions (eg, second lesions) from the queue or other temporary storage areas (step S410 ). The same lesion determination module 112 then calculates to obtain the first distance _Di ( 1≤i≤K , i and K are both positive integers, K is the lesion The number of groups, for example, the first distance from the ungrouped lesions to the centroid of the position of the third lesion group is D ₃ ) (step S420 ). The center of gravity of each lesion group is the center of gravity of all the lesions contained in each group. The same lesion determination module 112 will determine whether the minimum distance _Dk ( 1≤k≤K , _k is a positive integer) among the first distances _D1 ~ DK is smaller than the distance threshold value D _T (ie, _min (D1, D ₂ , ···, D _K )=D _k <D _T ?) (step S425). If the minimum first distance _Dk is not smaller than the distance threshold value D _T , the same lesion determination module 112 marks the ungrouped lesion as an invasive lesion and adds it to the new lesion group, and K is incremented by one (step S430 ), and End the grouping operation for this lesion (step S480) (if there are other lesions that have not been grouped in the queue, return to step S410). Assuming that the second lesion is classified into the new lesion group, the same lesion determination module 112 will display the first lesion and the second lesion respectively (ie, the second lesion is not the first lesion).

On the other hand, if the minimum first distance _Dk is smaller than the distance threshold D _T , the same lesion determination module 112 adds the ungrouped lesions to the kth lesion group corresponding to the minimum first distance _Dk (step S440 ). Assuming that the first lesion belongs to the kth lesion group, the first and second lesions both belong to the same lesion group (ie, the second lesion is the first lesion), and the same lesion determination module 112 will display the kth lesion through the display 130 . The first lesion and the second lesion are combined into a single lesion for presentation (step S260). It should be noted that the combined display here may be the combination of the outlines of the two lesions and the presentation in the same visual style. For example, referring to FIGS. 3A to 3C at the same time, the same lesion determination module 112 merges the lesions FF and SF identified in FIGS. 3A and 3B into a lesion PF, and displays the combined lesion PF on the display 130 (as shown in FIG. 3C ). Show). It should be noted that the contour of the lesion PF may be adjusted according to the contours of the lesions FF and SF, and the shape of the lesion shown in Figures 3A-3C is only a convenient example.

Since the medical images can be judged sequentially one by one whether it is the same lesion as the previously identified lesion, it is applied to the real-time scanning technology. During the scanning process of the scanner moving, the lesion detection device 1 can instantly identify a plurality of lesions belonging to the same lesion. Combined display, thereby improving the efficiency of diagnosis. In addition, even if the medical image is not obtained from the scanner in real time (eg, downloaded through the network, imported from a pen drive, etc.), the embodiments of the present invention can still be applied.

In addition, in order to improve the accuracy of the same lesion determination, please refer to FIG. 4 , if a lesion is added to the existing lesion group (in this example, the kth lesion group), the same lesion determination module 112 will calculate to obtain the kth lesion The second distance D _k,j ( 1 ≤j≤Y , j and Y are positive integers, and Y is the kth lesion The number of lesions in the group, for example, the second distance from the 5th lesion in the kth lesion group to the position center of gravity of the kth lesion group is Dk _,5 ) (step S450), and determine those second distances _{Dk ,1} ~ D _k,Y whether the largest second distance D _k,y (1 ≤ y≤Y , y is a positive integer) is less than the distance threshold D _T (ie, max(D _k,1 , D _k,2 , ..., D _k,Y )=D _k,y <D _T ?) (step S455). If the maximum second distance D _k,y is smaller than the distance threshold D _T , the same lesion determination module 112 ends the grouping (step S480 ) (if there are other ungrouped lesions in the queue, return to step S410 ). And if the maximum second distance Dk _,y is not smaller than the distance threshold value D _T , the same lesion determination module 112 removes the yth lesion corresponding to the maximum second distance Dk _,y out of the kth lesion group (step S460 ). ) (Y minus one) (in other embodiments, the same lesion determination module 112 may directly remove all the lesions whose second distance is greater than the distance threshold _DT from the kth lesion group), and update the k The location center of gravity of the lesion group. The yth lesion removed in step S460 will be re-identified as an ungrouped lesion (step S470), and the same lesion determination module 112 will recalculate the second distance _Dk of the kth lesion group based on the updated position center of gravity, ₁ to D _k,Y (return to step S420).

It should be noted that, in addition to the aforementioned determination of the same lesion based on the distance of the center of gravity of the lesion group, in other embodiments, the same lesion determination module 112 may also directly determine the second lesion and another adjacent and nearest lesion. Merge, or merge with another lesion with the highest texture feature similarity within a specific distance range, and so on, to be adjusted by the user according to their needs.

In addition to the aforementioned functions of instant determination of lesions, the lesion detection device 1 also has other functions. In one embodiment, the lesion detection device 1 is combined with the application of a separate or built-in scanner, and the position processing module 113 can synchronously present the movement track of the scanner through the display 130, and based on image recognition and sensing on the scanner A scanner (eg, infrared, switch, proximity sensor, etc.) is used to determine whether the scanner contacts or leaves the object surface (eg, skin surface, etc.). If the position processing module 113 detects that the scanner is not in contact with the surface of the object, it will suspend the recording of the medical image by the scanner and the display of the movement track of the scanner.

In one embodiment, in order to enhance the visual experience and improve the accuracy of auxiliary diagnosis, the three-dimensional presentation module 114 obtains the three-dimensional position information corresponding to several medical images at different positions (for example, in the object to be scanned) according to the position processing module 113. coordinates, relative positions, etc.) on the display 130, and the movement track formed by the corresponding position of the medical image of each different position (ie, the movement track during the scanning process of the scanner) is presented through the display 130 in a three-dimensional manner. In addition, the three-dimensional presentation module 114 can also form a three-dimensional virtual object according to a specific part of the object to be measured, and based on the identification result of the lesion identification module 111 , and through the display 130 , all suspicious lesion positions are displayed on the three-dimensional virtual object.

In one embodiment, the part marking module 115 marks the position information of the scanner at a specific position on the object to be scanned, and generates a virtual part range according to the position information corresponding to the medical images of several different positions. For example, referring to FIG. 5 , which is a screen 500 displayed by the display 130 , the part marking module 115 calculates the virtual part range PA according to the appearance of the target part (the breast in this example) based on the four positioning points L1 obtained by the scanner positioning. It is worth noting that this embodiment does not limit the number of positioning points, as long as it is greater than two.

Besides, the existing real-time scanning technology usually only presents the medical image obtained by the current scanning. Medical image archiving and communication system (Picture Archiving and Communication System, PACS), hospital information system (Hospital Information System, HIS), radiology information system (Radiology Information System, RIS), local database, etc.), flash drive, etc.) or storage The past lesion information of the object to be scanned (eg, past lesion location, image, image report, pathology report, etc.) is obtained from the device 110 , and the past lesion information is simultaneously displayed on the currently acquired or scanned medical image through the display 130 . For example, please refer to FIG. 6 , which is a screen 600 displayed by the display 130 , the screen 600 presents the current focus CF identified by the focus identification module 111 , the past focus PF in the past lesion information, and the past recorded time information D1 . In this way, the diagnostic staff can easily compare the current and past lesions, effectively improving the accuracy of judgment.

For the lesion information integration, the processor 150 displays the subject's past lesion information on the interface through the display 130 before/when the user reads the image, and the loaded past lesion information can be automatically compared with the current lesion. In addition, in response to the user's selection of a specific area, the past information providing module 116 may load medical images and related lesion information of the selected area and the patient at different times. If combined with the multi-screen browsing mode (the display 130 simultaneously displays multiple medical images), the user can quickly browse the multiple recorded medical images. The past information providing module 116 can also provide different medical images sequentially displayed in the scanning direction, for example, browsing from the nipple radially to the 12 o'clock direction. Alternatively, the past information providing module 116 can provide the scanned image plane to view the image, for example, to browse and play in a sagittal view.

To sum up, the lesion detection device and method according to the embodiments of the present invention can sequentially determine whether to merge the same lesion or not, and when applied to the real-time scanning technology, the same lesion can be displayed in real time with the movement of the scanner, improving the Efficiency of auxiliary diagnosis. In order to provide a rich and complete scanning assistance technology, the embodiments of the present invention further combine a pause function, three-dimensional presentation, virtual part range presentation, and past lesion information presentation.

Although the present invention has been disclosed above by the embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, The protection scope of the present invention shall be determined by the scope of the appended patent application.

1: Lesion detection device 110: Storage 111: Lesion identification module 112: The same lesion judgment module 113: Position processing module 114: 3D rendering module 115: Part Marking Module 116: Past information supply module 130: Display 150: Processor S210~S260, S410~S480: Steps FL, SL: Image block FF, SF, PF, CF: lesions L1: Anchor point PA: virtual part range 500, 600: Screen D1: Past record time information

FIG. 1 is a block diagram of components of a lesion detection device according to an embodiment of the present invention. FIG. 2 is a flowchart of a lesion detection method according to an embodiment of the present invention. 3A-3C are an example illustrating the same lesion judgment analysis. FIG. 4 is a flowchart of lesion grouping according to an embodiment of the present invention. FIG. 5 is an example illustrating a virtual part range. FIG. 6 is an example illustrating past lesion information.

S210~S260: Steps

Claims (10)

A lesion detection method, comprising: obtaining a medical image of a first position; identifying that a first lesion exists in the medical image of the first position; obtaining a medical image of a second position, wherein the first position is different from the second position position; identifying the presence of a second lesion in the medical image at the second position; determining whether the second lesion is the first lesion, comprising: determining the second lesion as the first lesion according to the distance from the second lesion to the first lesion a first lesion; and if the second lesion is the first lesion, the first lesion and the second lesion are combined and displayed as a single lesion. The lesion detection method according to item 1 of the claimed scope, wherein the step of judging whether the second lesion is the first lesion comprises: according to the center of gravity of the position of the second lesion to the lesion group to which the first lesion belongs distance to determine the second lesion as the first lesion. The lesion detection method according to item 2 of the scope of the application, wherein the step of determining the second lesion as the first lesion comprises: obtaining the center of gravity of the position of the second lesion to a plurality of lesion groups that have not yet been added. The first distance of , wherein the first lesion belongs to one of the lesion groups, and the positional center of gravity of each of the lesion groups is the positional center of gravity of all the lesions included in each of them; Determine whether a minimum first distance among the first distances is less than a distance threshold; if the minimum first distance is not less than the distance threshold, add the second lesion to a new lesion different from the lesion groups group and mark it as a invasive lesion; if the minimum first distance is less than the distance threshold, add the second lesion to the kth lesion group corresponding to the minimum first distance, where k is a positive integer; obtain the The second distance from all the lesions in the k-th lesion group except the trauma group to the centroid of the position of the k-th lesion group; judging whether a largest second distance among the second distances is less than the distance threshold; and if the largest second distance If the second distance is smaller than the distance threshold, the grouping is ended. The lesion detection method according to item 3 of the scope of application, wherein after the step of judging whether a maximum second distance among the second distances is smaller than the distance threshold value, further comprising: if the maximum second distance is not smaller than the distance threshold If the distance threshold is set, the yth lesion corresponding to the maximum second distance is moved out of the kth lesion group, and the position center of gravity of the kth lesion group is updated, where y is a positive integer. The lesion detection method as described in item 1 of the claimed scope, wherein the step of obtaining the medical image of the first position includes: scanning and recording the medical image through a scanner; and recording the position of the scanner as a The location of the medical image. The lesion detection method as described in item 5 of the patent application scope further includes: If the scanner is not in contact, the recording of the medical image and the display of the movement track of the scanner are suspended. The lesion detection method described in item 1 of the scope of the patent application further comprises: according to the three-dimensional position information corresponding to the medical images at a plurality of different positions, the corresponding positions of the medical images at the different positions are presented in a three-dimensional manner. moving trajectory; and forming a three-dimensional virtual object, and presenting all suspicious lesion positions on the three-dimensional virtual object. The lesion detection method described in item 1 of the scope of the patent application further comprises: generating a virtual part range according to the position information corresponding to the medical images of a plurality of different positions. The lesion detection method described in item 1 of the scope of the application further comprises: acquiring at least one past lesion information; and presenting the at least one past lesion information. A lesion detection device, comprising: a storage, recording a plurality of modules and at least one medical image; and a processor, coupled to the storage, and accessing and loading the modules recorded in the storage , the modules include: a lesion identification module for identifying a first lesion in a medical image at a first position, and identifying a second lesion in a medical image at a second position, wherein the first position is different from the first position two locations; and a same lesion determination module to determine whether the second lesion is the first lesion If the second lesion is the first lesion, the first lesion and the second lesion are combined and displayed as a single lesion, wherein the same lesion determination module is further based on the relationship between the second lesion and the first lesion. The distance determines that the second lesion is the first lesion.

Images