HK1240361B - Method and apparatus for providing reference information - Google Patents

Description

Method and device for providing reference information

Technical Field

The present invention relates to the field of electronic technology, and in particular to a method and device for providing reference information.

Background Art

With the development of artificial intelligence, more and more hospitals have begun using robots to serve patients. For example, robots used for ultrasound scanning of the human body quickly create three-dimensional images of the body's scanned area and then control the ultrasound probe to automatically scan and generate corresponding pathological images. Doctors then analyze the pathological images and arrive at a diagnosis.

Although the diagnosis results issued by doctors are more authoritative and reliable, for doctors who are not very experienced, relying solely on pathological images cannot accurately and quickly give diagnosis results, resulting in uncertainty in the diagnosis time.

Therefore, in order to improve the efficiency of diagnosis of patients, it is necessary to provide a solution that can provide diagnostic reference for the diagnosing party.

Summary of the Invention

The purpose of the present invention is to provide a method and device for providing reference information, aiming to provide a solution that can provide diagnostic reference for diagnosing parties, so as to improve the efficiency of diagnosis of patients.

The present invention is implemented as follows: a method for providing reference information, which is used to provide reference information to a diagnostic party after an ultrasonic scanner completes a scan. The reference information acquisition method includes:

obtaining an ultrasonic scan image generated by performing an ultrasonic scan on the affected area;

performing image processing on the ultrasonic scanning image to obtain a corresponding three-dimensional image;

fusing the three-dimensional image with the magnetic resonance imaging image corresponding to the affected area to obtain a target image;

If an index image class related to the target image exists in the control model library, the diagnostic information of the index image class is loaded.

Correspondingly, another object of the present invention is to provide a device for providing reference information, which is used to provide reference information to the diagnosis party after the ultrasound scanner completes the scan. The reference information acquisition device includes:

an acquisition unit, configured to acquire an ultrasonic scanning image generated after performing an ultrasonic scan on the affected area;

a three-dimensional image generating unit, configured to perform image processing on the ultrasonic scanning image to obtain a corresponding three-dimensional image;

an image fusion unit, configured to fuse the three-dimensional image with the magnetic resonance imaging image corresponding to the affected area to obtain a target image;

The first execution unit is configured to load diagnostic information of an index image class if an index image class related to the target image exists in the reference model library.

In the above scheme, an ultrasonic scanning image generated by ultrasonic scanning of the affected part is obtained; the ultrasonic scanning image is processed to obtain a corresponding three-dimensional image; the three-dimensional image is fused with the magnetic resonance image corresponding to the affected part to obtain a target image; if an index image class related to the target image exists in the reference model library, the diagnostic information of the index image class is loaded, thereby providing a diagnostic reference for the diagnosing party when diagnosing the patient, thereby improving the diagnostic efficiency of the patient.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flowchart of a method for providing reference information provided by an embodiment of the present invention;

FIG2 is a schematic structural diagram of a device for providing reference information provided by an embodiment of the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

The purpose of the present invention is to provide a method and device for providing reference information, aiming to provide a solution that can provide diagnostic reference for diagnosing parties, so as to improve the efficiency of diagnosis of patients.

In all embodiments of the present invention, the reference information provided is provided to the diagnosis party after the ultrasound scanner completes the scan. That is, the ultrasound scanner generates ultrasound scan images and provides the diagnosis party with corresponding reference information to improve the efficiency of diagnosis.

The following is a detailed description of the implementation of the present invention with reference to the accompanying drawings:

FIG1 shows an implementation flow of a method for providing reference information according to an embodiment of the present invention. For ease of description, only parts related to the embodiment are shown.

As shown in FIG1 , a method for providing reference information includes:

S110: Acquire an ultrasonic scanning image generated by performing an ultrasonic scan on the affected area.

In step S110 , the ultrasonic scan image is a pathological image used to describe the affected area.

It should be noted that the ultrasonic scanning of the affected part may specifically be: if the affected part is already within the preset identification area, a three-dimensional contour scan of the affected part is performed to obtain an outer contour image of the affected part; and the ultrasonic scanner is controlled to perform an ultrasonic scan of the affected part according to the outer contour image.

It can be understood that the preset area is the overlapping scanning area when the three-dimensional scanner and the ultrasonic scanner are working. When the patient's affected area is within the preset area of the scanning device, it can be determined as the body to be scanned, and then the three-dimensional scanner can be controlled to perform a three-dimensional contour scan on it.

Among them, each three-dimensional scanner can be a 3D scanner, which simultaneously projects or raster projects multiple light rays on the object to be scanned in a preset area, and then collects images of multiple external surfaces of the object to be scanned, and completes three-dimensional contour scanning by collecting information on the external surfaces of part or all areas of the organism at different angles.

Furthermore, if the patient's affected part is already within the preset identification area, performing a three-dimensional contour scan on the patient's affected part to obtain an outer contour image of the patient's affected part includes: acquiring multiple sets of data scanned by the three-dimensional scanner; and performing three-dimensional contour splicing on the multiple sets of data to obtain an outer contour image of the patient's affected part.

For example, when the subject to be scanned, i.e., a patient's head, is within a predetermined area, a 3D scanner performs a 3D contour scan of the head. Multiple sets of data acquired by the 3D scanner are then spliced together using a human body modeling reference library. This library, which includes accurate 3D information about various parts and organs of the human body, can be used to generate a complete human body model, i.e., a patch model. Further parametric digital processing of the patch model can be performed to determine the precise parameters of the human head.

S120: Perform image processing on the ultrasonic scanning image to obtain a corresponding three-dimensional image.

In step S120, image enhancement is performed to improve the appearance of the image, making it more suitable for machine analysis and processing. Essentially, it selectively enhances certain information in the image while suppressing other information to increase the image's "readability." Filtering and denoising aims to remove or reduce noise in the image, such as speckle noise and Gaussian noise, to reduce the impact of speckle noise on the image.

Image segmentation and edge contour extraction are the basis for three-dimensional reconstruction and display. The effect of segmentation and the accuracy of the contour directly affect the accuracy of the model after three-dimensional reconstruction. Segmentation and edge contour extraction can help doctors extract objects of interest, such as pathological tissues, so that the diagnosis can perform qualitative and quantitative analysis of the pathological tissues, thereby improving the accuracy and scientificity of the diagnosis.

It should be noted that step S120 may specifically include: performing image enhancement and image filtering processing on the ultrasonic scanning image to obtain a first image; performing image segmentation on the first image to obtain multiple slice images; marking the contour lines of each slice image in the multiple slice images; and connecting and recombining the contour lines of adjacent slice images according to the object contour lines on the multiple slice images to obtain the corresponding three-dimensional image information.

After the first image is segmented, each of the multiple slice images is marked with contour lines, that is, after the ultrasonic scanning image that has undergone image enhancement and image filtering is segmented, the edge contour of each slice image is extracted.

S130: Fusing the three-dimensional image with the magnetic resonance imaging image corresponding to the affected area to obtain a target image.

In step S130 , the MRI image is related to the affected part but not to the patient, that is, the MRI image is a preset MRI image of the affected part.

It should be noted that image fusion refers to the fusion processing of multiple source images of the same scene or target obtained by multiple image sensors according to actual needs, so as to obtain clear, more comprehensive and reliable image information of the required targets in the same scene, making it suitable for visual perception or computer processing.

Step S130 may specifically include: performing format conversion on the MRI image and/or the three-dimensional image to superimpose the MRI image and the three-dimensional image; and adjusting the direction or size of the MRI image and/or the three-dimensional image so that feature points on the three-dimensional image coincide with feature points on the MRI image, thereby obtaining a target image.

It's important to note that before image fusion, image registration is performed. This includes both rigid and non-rigid registration. Rigid registration maps points in the source image to corresponding points in the target image. Image registration also involves image conversion and positioning. Image conversion involves converting the format, 3D orientation, and size of images from the same or different imaging devices.

It can be understood that image positioning is repeated positioning based on image content, which can quickly achieve image registration.

S140: If an index image class related to the target image exists in the reference model library, the diagnostic information of the index image class is loaded.

In step S140, the reference model library includes index image classes and corresponding diagnostic results for different patients. It is understood that the information to be loaded from the reference model library may include diagnostic information for multiple index image classes. Specifically, an index image class is a collection of images related to the target image. The loaded diagnostic information is the diagnostic information corresponding to the images within the index image class.

Furthermore, step S140 may specifically include: acquiring feature points in the target image; determining the index image class from the reference model library according to the feature points; and loading diagnostic information of the index image class.

It should be noted that the feature points are used to characterize the pathological content or affected area of the target image, and the feature points may be points on the scanned images of different tissue structures.

For example, the feature points include a feature point A on a bone and a feature point B on a tumor tissue, and the information carried by the feature points may be the distance between the feature point A and the feature point B.

The index image class is determined from the comparison model library according to the feature point, that is, the index image class having the same feature point as the feature point is screened out from the comparison model library according to the feature point.

For another example, the feature points include feature point C on the tumor tissue and feature point D on the tumor tissue, and the target ultrasonic scanning image with the same feature points as the feature points is screened out from the control model library. That is, based on feature point C or feature point D, the index image class with the same feature point C or feature point D on the corresponding ultrasonic scanning image is screened out from the index image class in the control model library.

Furthermore, after step S130, the method may further include the following steps: if the reference model library does not contain an index image class related to the target image, the target image and the diagnosis result determined by the diagnosis party are stored as new diagnosis information in the reference model library.

If the index image class related to the target image does not exist in the reference model library, the target image and the diagnosis result determined by the diagnosing party are stored as new diagnosis information in the reference model library, thereby enriching the content of the reference model library.

In the above scheme, an ultrasonic scanning image generated by ultrasonic scanning of the affected part is obtained; the ultrasonic scanning image is processed to obtain a corresponding three-dimensional image; the three-dimensional image is fused with the magnetic resonance image corresponding to the affected part to obtain a target image; if an index image class related to the target image exists in the reference model library, the diagnostic information of the index image class is loaded, thereby providing a diagnostic reference for the diagnosing party when diagnosing the patient, thereby improving the diagnostic efficiency of the patient.

Please continue to refer to Figure 2. The modules included in an apparatus 100 for providing reference information are used to execute the steps in the embodiment corresponding to Figure 1. Please refer to Figure 1 and the relevant descriptions in the embodiment corresponding to Figure 1 for details, which will not be repeated here.

As shown in FIG2 , a device 100 for providing reference information includes: an acquisition unit 10, a three-dimensional image generation unit 20, an image fusion unit 30, and a first execution unit 30. Specifically:

The acquisition unit 10 is configured to acquire an ultrasonic scanning image generated by performing ultrasonic scanning on the affected area, and send the ultrasonic scanning image to the three-dimensional image generation unit 20 .

For example, the acquisition unit 10 acquires an ultrasonic scan image generated by performing an ultrasonic scan on the affected area, and transmits the ultrasonic scan image to the three-dimensional image generation unit 20 .

The three-dimensional image generating unit 20 is configured to receive the ultrasonic scanning image sent by the acquiring unit 10 , perform image processing on the ultrasonic scanning image to obtain a corresponding three-dimensional image, and send the three-dimensional image to the image fusion unit 30 .

For example, the three-dimensional image generating unit 20 receives the ultrasonic scanning image sent by the acquiring unit 10 , performs image processing on the ultrasonic scanning image to obtain a corresponding three-dimensional image, and sends the three-dimensional image to the image fusion unit 30 .

Furthermore, the three-dimensional image generation unit 20 includes: an image enhancement and filtering unit, an image segmentation unit, a marking unit, and an image generation unit. Specifically:

The image enhancement and filtering unit is used to perform image enhancement and image filtering on the ultrasonic scanning image to obtain a first image.

For example, the image enhancement and filtering unit performs image enhancement and image filtering on the ultrasonic scanning image to obtain a first image.

The image segmentation unit is used to perform image segmentation on the first image to obtain a plurality of slice images.

For example, the image segmentation unit performs image segmentation on the first image to obtain a plurality of slice images.

A marking unit is configured to mark a contour line on each of the plurality of slice images.

For example, the marking unit performs contour line marking on each of the plurality of slice images.

The image generating unit is configured to connect and reorganize the contour lines of adjacent slice images according to the object contour lines on the plurality of slice images to obtain the corresponding three-dimensional image information.

For example, the image generation unit connects and reorganizes the contour lines of adjacent slice images according to the object contour lines on the multiple slice images to obtain the corresponding three-dimensional image information.

The image fusion unit 30 is configured to receive the three-dimensional image sent by the three-dimensional image generation unit 20 , fuse the three-dimensional image with the magnetic resonance image corresponding to the affected area to obtain a target image, and send the target image to the first execution unit 40 .

For example, the image fusion unit 30 receives the 3D image sent by the 3D image generation unit 20 , fuses the 3D image with the MRI image corresponding to the affected area to obtain a target image, and sends the target image to the first execution unit 40 .

Furthermore, the image fusion unit 30 includes: a format conversion unit and an image adjustment unit. Specifically:

A format conversion unit is used to perform format conversion on the nuclear magnetic resonance image and/or the three-dimensional image so as to superimpose the nuclear magnetic resonance image and the three-dimensional image.

For example, the format conversion unit performs format conversion on the nuclear magnetic resonance image and/or the three-dimensional image to superimpose the nuclear magnetic resonance image and the three-dimensional image.

The image adjustment unit is used to adjust the direction or size of the nuclear magnetic resonance image and/or the three-dimensional image so that the feature points on the three-dimensional image coincide with the feature points on the nuclear magnetic resonance image, thereby obtaining a target image.

For example, the image adjustment unit adjusts the direction or size of the MRI image and/or the 3D image so that feature points on the 3D image coincide with feature points on the MRI image, thereby obtaining a target image.

The first execution unit 40 is configured to receive the target image sent by the image fusion unit 30 and load the diagnostic information of the index image class if an index image class related to the target image exists in the reference model library.

For example, the first execution unit 40 receives the target image sent by the image fusion unit 30, and if an index image class related to the target image exists in the reference model library, loads the diagnostic information of the index image class.

Furthermore, the first execution unit includes: a feature point acquisition unit, a determination unit, and an information loading unit. Specifically:

The feature point acquisition unit is used to acquire feature points in the target image.

For example, the feature point acquisition unit acquires feature points in the target image.

A determination unit is used to determine the index image class from the comparison model library according to the feature points.

For example, the determining unit determines the index image class from the comparison model library according to the feature points.

An information loading unit is used to load the diagnostic information of the index image class.

For example, the information loading unit loads the diagnostic information of the index image class.

As a preferred embodiment of the present invention, the reference information acquisition device further includes: a second execution unit.

The second execution unit is configured to store the target image and the diagnosis result determined by the diagnosis party as new diagnosis information in the reference model library if the reference model library does not contain the index image class related to the target image.

For example, if the reference model library does not contain the index image class related to the target image, the second execution unit stores the target image and the diagnosis result determined by the diagnosis party as new diagnosis information in the reference model library.

If the index image class related to the target image does not exist in the reference model library, the target image and the diagnosis result determined by the diagnosing party are stored as new diagnosis information in the reference model library, thereby enriching the content of the reference model library.

In the above scheme, an ultrasonic scanning image generated by ultrasonic scanning of the affected part is obtained; the ultrasonic scanning image is processed to obtain a corresponding three-dimensional image; the three-dimensional image is fused with the magnetic resonance image corresponding to the affected part to obtain a target image; if an index image class related to the target image exists in the reference model library, the diagnostic information of the index image class is loaded, thereby providing a diagnostic reference for the diagnosing party when diagnosing the patient, thereby improving the diagnostic efficiency of the patient.

Those skilled in the art will understand that the steps or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment, and the aforementioned storage medium includes: ROM, RAM, disk or optical disk, etc. Various media that can store program codes.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims (10)

1. A method for providing reference information, used to provide reference information to a diagnostician after an ultrasound scanner has completed a scan, characterized in that the reference information acquisition method includes: Acquire ultrasound scan images generated after ultrasound scanning of the affected area; The ultrasonic scan image is processed to obtain the corresponding three-dimensional image; The three-dimensional image is fused with the magnetic resonance image corresponding to the affected area, so that the feature points on the three-dimensional image and the feature points on the magnetic resonance image coincide to obtain a target image. The target image includes at least one feature point, which is used to characterize the pathological content or affected area of the target image. If an indexed image class related to the feature point in the target image exists in the comparison model library, then the diagnostic information of the indexed image class is loaded. 2. The method as described in claim 1, characterized in that, the step of image processing the ultrasonic scan image to obtain a corresponding three-dimensional image includes: The ultrasonic scan image is subjected to image enhancement and image filtering processing to obtain a first image; The first image is segmented to obtain multiple slice images; Each of the plurality of slice images is marked with a contour line; The contour lines of adjacent slice images are connected and recombined based on the object contour lines on the multiple slice images to obtain the corresponding three-dimensional image information. 3. The method as described in claim 1, characterized in that, the step of fusing the three-dimensional image with the MRI image corresponding to the affected area, such that the feature points on the three-dimensional image and the feature points on the MRI image coincide to obtain the target image, includes: The format of the nuclear magnetic resonance image and/or the three-dimensional image is converted so that the nuclear magnetic resonance image and the three-dimensional image are superimposed; The orientation or size of the nuclear magnetic resonance image and/or the three-dimensional image is adjusted so that the feature points on the three-dimensional image coincide with the feature points on the nuclear magnetic resonance image, thereby obtaining the target image. 4. The method as described in claim 1, characterized in that, if an indexed image class related to the feature points in the target image exists in the comparison model library, then loading the diagnostic information of the indexed image class includes: Obtain feature points in the target image; The indexed image class is determined from the reference model library based on the feature points; Load the diagnostic information of the indexed image class. 5. The method according to any one of claims 1 to 4, characterized in that, after the step of image fusion of the three-dimensional image and the MRI image corresponding to the affected area, such that the feature points on the three-dimensional image and the feature points on the MRI image coincide to obtain the target image, the method further includes: If there is no indexed image class in the reference model library that is related to the feature points in the target image, then the target image and the diagnostic result determined by the diagnostic party are stored as new diagnostic information in the reference model library. 6. An apparatus for providing reference information, used to provide reference information to a diagnostician after an ultrasound scanner has completed a scan, characterized in that the reference information acquisition device comprises: The acquisition unit is used to acquire the ultrasound scan image generated after performing an ultrasound scan on the affected area. A three-dimensional image generation unit is used to perform image processing on the ultrasonic scanning image to obtain a corresponding three-dimensional image; An image fusion unit is used to fuse the three-dimensional image with the magnetic resonance image corresponding to the affected area, so that the feature points on the three-dimensional image and the feature points on the magnetic resonance image coincide to obtain a target image. The target image includes at least one feature point, which is used to characterize the pathological content or affected area of the target image. The first execution unit is configured to load diagnostic information of the index image class if an index image class related to the feature points in the target image exists in the comparison model library. 7. The apparatus of claim 6, wherein the three-dimensional image generation unit comprises: An image enhancement and filtering unit is used to perform image enhancement and image filtering processing on the ultrasonic scanning image to obtain a first image; An image segmentation unit is used to segment the first image to obtain multiple slice images; A marking unit is used to mark the outline of each of the plurality of slice images; The image generation unit is used to connect and reassemble the contour lines of adjacent slice images based on the object contour lines on the multiple slice images to obtain the corresponding three-dimensional image information. 8. The apparatus of claim 6, wherein the image fusion unit comprises: A format conversion unit is used to convert the format of the nuclear magnetic resonance image and/or the three-dimensional image so as to overlay the nuclear magnetic resonance image and the three-dimensional image; An image adjustment unit is used to adjust the orientation or size of the nuclear magnetic resonance image and/or the three-dimensional image so that the feature points on the three-dimensional image coincide with the feature points on the nuclear magnetic resonance image, thereby obtaining a target image. 9. The apparatus of claim 6, wherein the first execution unit comprises: A feature point acquisition unit is used to acquire feature points in the target image; A determining unit is configured to determine the indexed image class from the reference model library based on the feature points; The information loading unit is used to load the diagnostic information of the indexed image class. 10. The apparatus according to any one of claims 6 to 9, wherein the reference information acquisition apparatus further comprises: The second execution unit is used to store the target image and the diagnostic result determined by the diagnostic party as new diagnostic information in the comparison model library if there is no indexed image class related to the target image in the comparison model library.