Disclosure of Invention
The invention aims to provide an artery and vein classification method and device for OCT-A images, which can solve the problem that the existing retinal vessel artery and vein classification algorithm can not realize the classification of micro-vessels.
According to a first aspect of the present invention, there is provided an arteriovenous classification method of an OCT-a image, comprising:
acquiring fundus color photographs and OCT-A images, respectively carrying out topology estimation on segmentation results of the fundus color photographs and the OCT-A images, and obtaining a topology tree set;
after the eye bottom color photograph and the OCT-A image are subjected to rough matching, key point matching is carried out, so that topology trees between the two modal images are matched, and topology information is obtained;
artery and vein classification is carried out on blood vessels in a near vision disc area of fundus color photograph according to an dominance set theory;
and (3) performing global artery and vein classification by adopting a label propagation algorithm according to the topology information to obtain an artery and vein classification result of the OCT-A blood vessel.
Further, "obtaining fundus color illumination and OCT-A image, respectively performing topology estimation on segmentation results of fundus color illumination and OCT-A image, and obtaining a topology tree set" specifically includes:
dividing fundus color illumination and OCT-A images by a morphological refinement algorithm to obtain a vascular structure with unit pixel width;
detecting key nodes of the vascular structure, and disconnecting the segmented image at bifurcation points and intersection points in the key nodes to obtain a plurality of independent vascular segments;
performing central line fitting on the blood vessel segments by adopting cubic spline fitting to obtain central line information of each segment of blood vessel;
establishing an undirected graph of a retinal blood vessel network according to the central line information, extracting characteristic vectors of vessel segments, and establishing a weighted undirected graph of retinal blood vessels;
extracting a video disc area of fundus color illumination according to an automatic segmentation algorithm, determining a starting point of a blood vessel tree according to the video disc area, and determining a starting point of an OCT-A image;
after virtual points are introduced to connect the starting points of all the vessel trees, an independent vessel tree is obtained through a minimum spanning tree algorithm, and a topology tree set is obtained.
Further, "after coarse matching of the fundus color photograph and the OCT-a image, performing key point matching to match a topology tree between two modality images, the obtaining of topology information" specifically includes:
performing rough matching on fundus color illumination and OCT-A images according to a registration algorithm;
intercepting the coincident part of the OCT-A image in the fundus color illumination segmentation result after rough matching and extracting key nodes, and intercepting the coincident part of the fundus color illumination in the OCT-A image segmentation result after rough matching and extracting the key nodes;
matching key nodes under two modes through a Gaussian regression process;
and completing topology tree matching according to consistency of key nodes contained in the topology tree, and obtaining topology tree information.
Further, "classifying arteries and veins of a near vision disc region blood vessel of fundus color photograph according to the dominance set theory" specifically includes:
selecting points in a blood vessel section within a preset multiplying power video disc range as clustering objects, extracting feature vectors from each point, and obtaining a point set;
clustering the selected point sets according to the advantage sets and classifying all the points into two types;
defining arteries and veins according to the average value of the brightness information of the midpoints of each class;
and counting the label distribution of the middle points of each blood vessel segment and performing active vein marking on the blood vessel segments.
Further, "global artery and vein classification is performed by adopting a label propagation algorithm according to topology information, and an artery and vein classification result of the OCT-A blood vessel is obtained" specifically comprises:
the label of the initial vessel section is transmitted downwards through the topology information, so that the retinal vessel is completely classified;
the arteriovenous classification of OCT-a vessels was obtained from a complete classification of retinal vessels.
According to a second aspect of the present invention, there is provided an arteriovenous classification device of OCT-a image, comprising:
the acquisition module is used for: acquiring fundus color photographs and OCT-A images, respectively carrying out topology estimation on segmentation results of the fundus color photographs and the OCT-A images, and obtaining a topology tree set;
a first processing module: after the eye bottom color photograph and the OCT-A image are subjected to rough matching, key point matching is carried out, so that topology trees between the two modal images are matched, and topology information is obtained;
and a classification module: artery and vein classification is carried out on blood vessels in a near vision disc area of fundus color photograph according to an dominance set theory;
and a second processing module: and (3) performing global artery and vein classification by adopting a label propagation algorithm according to the topology information to obtain an artery and vein classification result of the OCT-A blood vessel.
Further, the obtaining module specifically includes:
a first dividing unit: dividing fundus color illumination and OCT-A images by a morphological refinement algorithm to obtain a vascular structure with unit pixel width;
a first detection unit: detecting key nodes of the vascular structure, and disconnecting the segmented image at bifurcation points and intersection points in the key nodes to obtain a plurality of independent vascular segments;
a first processing unit: performing central line fitting on the blood vessel segments by adopting cubic spline fitting to obtain central line information of each segment of blood vessel;
a first extraction unit: establishing an undirected graph of a retinal blood vessel network according to the central line information, extracting characteristic vectors of vessel segments, and establishing a weighted undirected graph of retinal blood vessels;
a second processing unit: extracting a video disc area of fundus color illumination according to an automatic segmentation algorithm, determining a starting point of a blood vessel tree according to the video disc area, and determining a starting point of an OCT-A image;
a third processing unit: after virtual points are introduced to connect the starting points of all the vessel trees, an independent vessel tree is obtained through a minimum spanning tree algorithm, and a topology tree set is obtained.
Further, the first processing module specifically includes:
a first matching unit: performing rough matching on fundus color illumination and OCT-A images according to a registration algorithm;
a second extraction unit: intercepting the coincident part of the OCT-A image in the fundus color illumination segmentation result after rough matching and extracting key nodes, and intercepting the coincident part of the fundus color illumination in the OCT-A image segmentation result after rough matching and extracting the key nodes;
a second matching unit: matching key nodes under two modes through a Gaussian regression process;
a first acquisition unit: and completing topology tree matching according to consistency of key nodes contained in the topology tree, and obtaining topology tree information.
Further, the classification module specifically includes:
a third extraction unit: selecting points in a blood vessel section within a preset multiplying power video disc range as clustering objects, extracting feature vectors from each point, and obtaining a point set;
a first classification unit: clustering the selected point sets according to the advantage sets and classifying all the points into two types;
a first defining unit: defining arteries and veins according to the average value of the brightness information of the midpoints of each class;
a first statistical unit: and counting the label distribution of the middle points of each blood vessel segment and performing active vein marking on the blood vessel segments.
Further, the second processing module specifically includes:
a fourth processing unit: the label of the initial vessel section is transmitted downwards through the topology information, so that the retinal vessel is completely classified;
a second acquisition unit: the arteriovenous classification of OCT-a vessels was obtained from a complete classification of retinal vessels.
The beneficial effects of the invention are as follows: 1. the artery and vein classification method of the blood vessel in the OCT-A image is provided for the first time, an effective tool is provided for evaluating the structural change of the retina microvasculature, and the early screening of vascular diseases is also possible. 2. The OCT-A image is guided by fundus illumination for the first time, and the blood vessel artery and vein classification of the OCT-A is realized by se:Sub>A topological information transmission mode, so that the problem that the blood vessel classification cannot be carried out due to the lack of color contrast information in the OTC-A image is solved. 3. The topological information transfer method is put forward for the first time, and key points are matched by using a Gaussian regression process, so that the matching between topological subgraphs is carried out on the basis of a topological structure, and the connection between fundus color photograph blood vessels and OCT-A blood vessel trees is realized. 4. The problem that fundus color photograph blood vessels and OCT-A blood vessels cannot be completely overlapped due to registration accuracy errors is solved through matching among topology trees, accuracy of pixel-level label propagation is compensated through information among the topology trees, and meanwhile robustness of OCT-A blood vessel classification is improved. 5. Preliminary experiments are performed in a batch of paired fundus color photographs and OCT-A images, and the results show that the algorithm used in the invention can accurately classify OCT-A blood vessels.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 shows a flow of an arteriovenous classification method of an OCT-a image according to an embodiment of the present invention, including:
s11, acquiring fundus color illumination and OCT-A images, and respectively carrying out topology estimation on segmentation results of the fundus color illumination and the OCT-A images to obtain a topology tree set.
The execution subject of the method may be a server.
And S12, performing key point matching after performing rough matching on the fundus color photograph and the OCT-A image, so that topology trees between the two modal images are matched, and topology information is obtained.
S13, classifying arteries and veins of the blood vessels of the near vision disc area of the fundus color photograph according to the theory of the dominance set.
S14, performing global artery and vein classification by adopting a label propagation algorithm according to the topology information to obtain an artery and vein classification result of the OCT-A blood vessel.
In the embodiment of the present specification, the technical solution for achieving the above object of the present invention is as follows: 1) Performing topology estimation on segmentation results of the eye fundus color illumination and the OCT-A image by using a graph theory-based method respectively, and establishing a complete topology tree set; 2) Performing coarse registration on the eye fundus color photograph and the OCT-A image by adopting a registration algorithm, and then performing key point matching by using a Gaussian regression process on the basis of the coarse registration to realize matching of topological trees between two modal images, thereby realizing topology information transmission; 3) Artery and vein classification is carried out on blood vessels in the region close to the optic disc of fundus color photograph by adopting an advantage set theory; 4) Global arteriovenous classification is realized by adopting a label propagation algorithm according to topology information, so that arteriovenous classification of OCT-A blood vessels is realized.
As a preferred embodiment, the "obtaining fundus color photograph and OCT-a image, performing topology estimation on the segmentation results of fundus color photograph and OCT-a image, respectively, to obtain a topology tree set" specifically includes: dividing fundus color illumination and OCT-A images by a morphological refinement algorithm to obtain a vascular structure with unit pixel width; detecting key nodes of the vascular structure, and disconnecting the segmented image at bifurcation points and intersection points in the key nodes to obtain a plurality of independent vascular segments; performing central line fitting on the blood vessel segments by adopting cubic spline fitting to obtain central line information of each segment of blood vessel; establishing an undirected graph of a retinal blood vessel network according to the central line information, extracting characteristic vectors of vessel segments, and establishing a weighted undirected graph of retinal blood vessels; extracting a video disc area of fundus color illumination according to an automatic segmentation algorithm, determining a starting point of a blood vessel tree according to the video disc area, and determining a starting point of an OCT-A image; after virtual points are introduced to connect the starting points of all the vessel trees, an independent vessel tree is obtained through a minimum spanning tree algorithm, and a topology tree set is obtained.
In the embodiments of the present description, the present invention converts the visual topology estimation into a graph optimization problem, i.e. a complex vascular network is represented by a graph G containing points and edges, and then the topology estimation is implemented in the graph G using a minimum spanning tree algorithm. The specific operation flow is as follows:
1) Establishing a diagram: adopting a morphological refinement algorithm on the basis of the segmented image, and iteratively removing pixels outside the blood vessel to obtain a blood vessel structure with a width of only one pixel; the key nodes are then detected using a sliding window of 3*3, where the detected points are divided into: the method comprises the steps of terminating nodes (only one adjacent pixel exists in a neighborhood), connecting points (two adjacent pixels exist in the neighborhood), bifurcation points (three adjacent pixels exist in the neighborhood), crossing points (the number of the adjacent pixels in the neighborhood is greater than three), disconnecting a segmented image at the bifurcation points and the crossing points to obtain a plurality of independent blood vessel segments, and then performing center line fitting by adopting three spline fitting to obtain center line information of each blood vessel segment. And taking the starting point and the end point of each segment of blood vessel as a point V in the graph theory, connecting adjacent points according to the neighborhood information to form a side E in the graph theory, and thus establishing an undirected graph G (V, E) of the retinal blood vessel network. Then, feature vectors of the vessel segment are extracted, including direction, gray scale, contrast, and diameter information (only direction and diameter information is extracted as its feature vector for OCT-a because OCT-a lacks color information), and feature vector clustering is performed at neighboring point set Vi (i=1, 2, …, N) using the dominant set to reduce feature vector dimension, and this is taken as similarity measure W between two points. A weighted undirected graph G (V, E, W) of retinal blood vessels is created.
2) Determining a starting point, extracting a video disc region by adopting an automatic segmentation algorithm for fundus illumination, taking a set of intersection points of the edge of the video disc region and blood vessels as a starting point S of a blood vessel tree, deleting points and edges falling in the video disc range, wherein the number of the intersection points is the number of retinal blood vessel trees in principle; for OCT-a images, points that fall within the pixels of the image edge are taken as starting points.
3) Topology estimation, introducing virtual points, connecting with the starting points of each vessel tree, converting the vessel topology estimation problem into a minimum graph problem with the virtual points as the starting points, calculating a minimum graph by adopting a minimum spanning tree algorithm, and finally removing the virtual points to obtainAnd (3) an independent vessel tree, so as to complete the topological estimation of the fundus color photograph vessel and the OCT-A vessel.
As a preferred embodiment, "after coarse matching of an fundus color photograph and an OCT-a image, performing key point matching to match a topology tree between two modality images, obtaining topology information" specifically includes: performing rough matching on fundus color illumination and OCT-A images according to a registration algorithm; intercepting the coincident part of the OCT-A image in the fundus color illumination segmentation result after rough matching and extracting key nodes, and intercepting the coincident part of the fundus color illumination in the OCT-A image segmentation result after rough matching and extracting the key nodes; matching key nodes under two modes through a Gaussian regression process; and completing topology tree matching according to consistency of key nodes contained in the topology tree, and obtaining topology tree information.
In the embodiment of the specification, the registration algorithm is used for realizing the rough registration of fundus color illumination and OCT-A images, but due to the limitation of registration accuracy, the complete matching of retinal blood vessels and OCT-A blood vessels can not be realized, and the pixel-level guidance can not be realized, so the invention provides the topology information transfer algorithm to realize the matching of vessel topology trees under two image modes. The operation method is as follows: firstly, intercepting the coincident part of OCT-A images in fundus color illumination segmentation results according to registration results, extracting the key nodes mentioned above, and extracting the key nodes in the OCT-A segmentation results; then matching key nodes in the two modes by adopting a Gaussian regression process; and finally, matching the topology tree under two modes according to the consistency of the key nodes contained in the topology tree.
Specifically, to find a set of corresponding points in the fundus illumination and OCT-a images, we first define points where the coincident feature points between the two modalities are considered to be a set of matches within six pixel errors. Thus selecting a set of corresponding points according to our definition in the result of the coarse registration. Wherein->Represents the key point in fundus color photograph, +.>The key points corresponding to fundus color illumination in OCT-A are represented, and l represents the number of point pairs. Then the set +.>Using a gaussian regression process as a set of training sets, estimation is performed by fitting the position coordinates between the corresponding pointsCorresponding mean value in Gaussian function>Sum of variances->The calculation method is as follows:
wherein K represents a vectorK is a kernel function defining a mapping consisting of affine transformation and non-linear mapping,/->Representing noise variance->Representing a symmetric matrix of size L×L, wherein the element calculation formula is +.>,Representation->Is a matrix of size L x D where D is the size of the feature vector, and since only the coordinate information of the key point is used in the present invention, d=2.
Then, the coordinates of the projections of all key points in the eye bottom map in the OCT-A image are calculated according to the trained model, and then a new model is calculated according to the definition of the matching pointsMatching point pairs to obtain more complete corresponding setWill be->Fitting by using Gaussian regression process as training set to obtain +.>Iterating in this way, finally when the set of matching points +.>Exit the cycle at steady state, resulting in +.>I.e. the final set of matching points, in principle m is equal to the number of key points in the corresponding retinal image portion of the OCT-a image.
After obtaining the vessel topology information and the set of corresponding points in the two modes, matching between the topology trees is realized by judging the coincidence degree of key points contained between the two groups of vessel trees. Specifically, for the topology tree in any fundus color photograph, the collection of key points contained in the topology tree is countedWherein F represents fundus illumination, i represents corresponding topology tree, and then +.f. by comparing the set of key points contained in the vessel tree contained in OCT-A image>Wherein O represents the OCT-A image. The two vessel trees with the highest overlap ratio are considered to be matched vessel trees. So far, the matching and the fusion of the vessel tree in the two modal images are realized.
As a preferred embodiment, "classifying arteries and veins of a near-vision disc region blood vessel of fundus color photograph according to the dominance set theory" specifically includes: selecting points in a blood vessel section within a preset multiplying power video disc range as clustering objects, extracting feature vectors from each point, and obtaining a point set; clustering the selected point sets according to the advantage sets and classifying all the points into two types; defining arteries and veins according to the average value of the brightness information of the midpoints of each class; and counting the label distribution of the middle points of each blood vessel segment and performing active vein marking on the blood vessel segments.
In the embodiment of the present specification, since the arteries and veins of the fundus color photograph blood vessel have differences in color and caliber and are obvious in the near vision disc region, an accurate clustering result can be obtained relatively easily, and meanwhile, since the arteries of the retinal blood vessel have higher oxygen content than veins, a very strong visual difference is shown in the image, and the visual difference can be used as a key index for distinguishing the arteries and veins. Therefore, the invention provides an artery and vein classification method for realizing the blood vessels in the near-optic disc region by using the advantage set, wherein the blood vessels in the near-optic disc region are firstly gathered into two types by virtue of advantages, and then the brightness information in the two types of blood vessels is extracted as an index for distinguishing the blood vessel types, so that the artery and vein classification of the blood vessels is realized. The specific operation steps are as follows:
firstly, selecting points in a blood vessel section within a range of 1.5 to 2 times of a video disc as clustering objects, and extracting feature vectors containing information such as color, contrast, blood vessel width and the like from each point; then clustering the selected point set by using the advantage set, and dividing all the points into two types; taking the average value of the brightness information of the points in each class as the basis of the classification labels, wherein a group of points with large brightness are defined as arteries, and a group of points with small brightness are defined as veins; and finally, counting the label distribution of the middle points of each section of the blood vessel, defining the classification labels of the blood vessel sections by adopting a voting mechanism, and if the number of the points of the blood vessel sections, which are labeled as arteries, is more than the number of the points of the blood vessel sections, which are labeled as veins, then considering the blood vessel sections as arteries, otherwise, marking the blood vessel sections as veins.
As a preferred embodiment, "global artery and vein classification by using a label propagation algorithm according to topology information, obtaining an arteriovenous classification result of an OCT-a blood vessel" specifically includes: the label of the initial vessel section is transmitted downwards through the topology information, so that the retinal vessel is completely classified; the arteriovenous classification of OCT-a vessels was obtained from a complete classification of retinal vessels.
In the embodiment of the present disclosure, in obtaining the topology of the complete retinal blood vessel and the arteriovenous classification label of the starting blood vessel segment in the fundus color photograph, the present invention uses a label propagation algorithm to implement global arteriovenous classification, specifically, we consider that all the blood vessel segments in the same blood vessel tree are consistent with the label of the starting blood vessel segment of the blood vessel tree, so that the complete classification of the retinal blood vessel can be implemented by transmitting the topology information obtained by the label of the starting blood vessel segment obtained by the above steps downward. OCT-a vessel arteriovenous classification is also included.
Fig. 2 shows a structure of an arteriovenous classification device of an OCT-a image according to an embodiment of the present invention, including:
the acquisition module 21: acquiring fundus color photographs and OCT-A images, respectively carrying out topology estimation on segmentation results of the fundus color photographs and the OCT-A images, and obtaining a topology tree set;
the first processing module 22: after the eye bottom color photograph and the OCT-A image are subjected to rough matching, key point matching is carried out, so that topology trees between the two modal images are matched, and topology information is obtained;
classification module 23: artery and vein classification is carried out on blood vessels in a near vision disc area of fundus color photograph according to an dominance set theory;
the second processing module 24: and (3) performing global artery and vein classification by adopting a label propagation algorithm according to the topology information to obtain an artery and vein classification result of the OCT-A blood vessel.
As a preferred embodiment, the acquisition module specifically includes: a first dividing unit: dividing fundus color illumination and OCT-A images by a morphological refinement algorithm to obtain a vascular structure with unit pixel width; a first detection unit: detecting key nodes of the vascular structure, and disconnecting the segmented image at bifurcation points and intersection points in the key nodes to obtain a plurality of independent vascular segments; a first processing unit: performing central line fitting on the blood vessel segments by adopting cubic spline fitting to obtain central line information of each segment of blood vessel; a first extraction unit: establishing an undirected graph of a retinal blood vessel network according to the central line information, extracting characteristic vectors of vessel segments, and establishing a weighted undirected graph of retinal blood vessels; a second processing unit: extracting a video disc area of fundus color illumination according to an automatic segmentation algorithm, determining a starting point of a blood vessel tree according to the video disc area, and determining a starting point of an OCT-A image; a third processing unit: after virtual points are introduced to connect the starting points of all the vessel trees, an independent vessel tree is obtained through a minimum spanning tree algorithm, and a topology tree set is obtained.
As a preferred embodiment, the first processing module specifically includes: a first matching unit: performing rough matching on fundus color illumination and OCT-A images according to a registration algorithm; a second extraction unit: intercepting the coincident part of the OCT-A image in the fundus color illumination segmentation result after rough matching and extracting key nodes, and intercepting the coincident part of the fundus color illumination in the OCT-A image segmentation result after rough matching and extracting the key nodes; a second matching unit: matching key nodes under two modes through a Gaussian regression process; a first acquisition unit: and completing topology tree matching according to consistency of key nodes contained in the topology tree, and obtaining topology tree information.
As a preferred embodiment, the classification module specifically includes: a third extraction unit: selecting points in a blood vessel section within a preset multiplying power video disc range as clustering objects, extracting feature vectors from each point, and obtaining a point set; a first classification unit: clustering the selected point sets according to the advantage sets and classifying all the points into two types; a first defining unit: defining arteries and veins according to the average value of the brightness information of the midpoints of each class; a first statistical unit: and counting the label distribution of the middle points of each blood vessel segment and performing active vein marking on the blood vessel segments.
As a preferred embodiment, the second processing module specifically includes: a fourth processing unit: the label of the initial vessel section is transmitted downwards through the topology information, so that the retinal vessel is completely classified; a second acquisition unit: the arteriovenous classification of OCT-a vessels was obtained from a complete classification of retinal vessels.
Furthermore, those skilled in the art will appreciate that while some embodiments herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.
Those of ordinary skill in the art will appreciate that: the above embodiments are only for illustrating the technical solution of the present invention, not for limiting the same, and although the present invention has been described in detail with reference to the above embodiments, it should be understood by those of ordinary skill in the art that: it is still possible to modify the technical solutions described in the foregoing embodiments or to make equivalent substitutions for some or all of the technical features thereof, without departing from the essence of the corresponding technical solutions from the scope of the present invention defined by the claims.