CN118016251A - Pathological image big data annotation display method and application thereof - Google Patents
Pathological image big data annotation display method and application thereof Download PDFInfo
- Publication number
- CN118016251A CN118016251A CN202410424278.5A CN202410424278A CN118016251A CN 118016251 A CN118016251 A CN 118016251A CN 202410424278 A CN202410424278 A CN 202410424278A CN 118016251 A CN118016251 A CN 118016251A
- Authority
- CN
- China
- Prior art keywords
- data
- annotation
- visible area
- labels
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000001575 pathological effect Effects 0.000 title claims abstract description 43
- 238000001914 filtration Methods 0.000 claims abstract description 16
- 238000007781 pre-processing Methods 0.000 claims abstract description 15
- 238000009877 rendering Methods 0.000 claims abstract description 9
- 230000003993 interaction Effects 0.000 claims abstract description 7
- 238000002372 labelling Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 12
- 230000007170 pathology Effects 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 12
- 230000005540 biological transmission Effects 0.000 claims description 11
- 230000001788 irregular Effects 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 6
- 238000003860 storage Methods 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 2
- 238000005096 rolling process Methods 0.000 claims description 2
- 238000012544 monitoring process Methods 0.000 abstract 1
- 230000000007 visual effect Effects 0.000 description 11
- 238000012545 processing Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 230000007246 mechanism Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 230000000877 morphologic effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004091 panning Methods 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
Landscapes
- Health & Medical Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Engineering & Computer Science (AREA)
- Epidemiology (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
- Processing Or Creating Images (AREA)
Abstract
The application provides a pathological image big data annotation display method and application thereof, which are characterized in that parameters of a visible area are acquired and updated in real time by monitoring and responding to the operations of amplifying, shrinking, moving and scrolling of a pathological image in a browser by a user, and a request is sent to the rear end; the back-end preprocessing module calculates marking boundary information, removes overlapping point marking and screens data outside a visible area through the filtering module, and only transmits the required marking information to the front-end rendering, so that smooth display and interaction are realized, and browsing efficiency and user experience are greatly improved.
Description
Technical Field
The application relates to the technical field of pathology, in particular to a pathology image big data annotation display method and application thereof.
Background
With the widespread use of artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) technology in the medical field and the rapid development of digital pathology (Digital Pathology), the accuracy and complexity of pathological image analysis has been significantly improved. Today, pathologists and researchers are increasingly refining and scaling the need for labeling information in images as they browse and parse pathological slice images. For example, to achieve accurate diagnosis and in-depth research, it is necessary to superimpose rich metadata, structured information, and various types of labeling markers, including but not limited to lesion areas, cell types, tissue structure boundaries, etc., on high-resolution digital pathology images.
However, existing pathology image browsers and related software tools have significant limitations in dealing with the presentation and interaction of large scale annotation data. When the number of annotations increases dramatically, the prior art often fails to provide an efficient and smooth user experience. In particular, some browsers can only effectively support a limited number of small-scale labels, and when a large data volume label requirement is met, the performance of the browser is obviously reduced, and even serious interface response hysteresis can occur. The method mainly originates from the fact that the prior art generally adopts the method of loading all the annotation data into the browser memory at one time and rendering in real time, so that the page loading time is overlong, and after a large amount of annotation data are pushed, views can not be effectively rendered and updated timely due to the limitation of the processing capacity of the browser, and finally the real-time, seamless and efficient browsing and operation of the user on the extremely rich annotation information of the pathological image are influenced.
Therefore, a new technical scheme is urgently needed, and the performance bottleneck of the prior art on large data volume annotation display can be overcome while high-definition display of pathological images is ensured, so that the severe requirements of increasing clinical diagnosis and scientific research on intelligent and high-efficiency annotation browsing of digital pathological images are met.
Disclosure of Invention
The embodiment of the application provides a pathological image big data annotation display method and application thereof, aiming at the problems of performance bottleneck and the like of big data annotation display in the prior art.
The invention has the core technology that the visual area of the browser is changed along with the enlargement, the shrinkage and the movement of the browser when the browser reads a film, so that the parameter level, the coordinates of the upper right corner of the visual area, the height of the visual area and the width of the visual area can be acquired in the visual area, the height and the width are uploaded to a back-end service, the back-end service processes the original marking information through a preprocessing module, and the preprocessing data is returned to the front end through a filtering module.
In a first aspect, the present application provides a pathological image big data annotation display method, the method comprising the steps of:
S00, acquiring request parameters when a user performs zooming-in, zooming-out or moving operation on a pathological image in a browser, wherein the request parameters at least comprise: hierarchical information, upper right corner coordinates of the visible area, and height and width of the visible area;
s10, sending the request parameters to a back-end service;
S20, after receiving the request parameters, the back-end service preprocesses the original annotation information through a preprocessing module, wherein the preprocessing comprises the following steps: determining boundary information of the irregular graph marks and caching;
s30, new labeling data enter two filters of a filtering module, wherein:
Judging whether the labels are single points or not under the current zoom level by the first filter according to the level information, and if so, performing de-duplication display on the labels overlapped at the same position;
the second filter judges whether the label is positioned in the visible area according to the parameters of the visible area, and only retains the label data positioned in the visible area;
and S40, the filtered marking data is sent to a front-end browser by a back-end service, and the front-end browser renders and displays the marking information in real time according to the filtered data.
Further, the preprocessing includes: and adding the minimum value and the maximum value of the x coordinate and the y coordinate in each marked coordinate point array to the original data to form a new marked array, and caching.
Further, the step S00 further includes: and acquiring a rolling event of the user, updating the visible area parameter when the user rolls the page, and triggering a subsequent annotation information refreshing process.
Further, the step S20 further includes: the size and shape of the label are dynamically adjusted according to the scaling of the image, and the proportion of the label is kept consistent with that of the actual pathological tissue structure.
Further, in step S30, the first filter identifies and merges overlapping single-point labels by using a hierarchical traversal algorithm, and records and displays the label quantity statistical information.
Further, in step S30, the second filter adopts a polygon clipping algorithm to determine and screen out the complex shape label completely located in the visible rectangular area.
Further, the method further comprises a buffer optimization step, wherein the processed annotation data applicable to the current window is buffered in the client browser between the step S20 and the step S40.
In a second aspect, the present application provides a pathological image big data annotation display device, including:
The user interaction module is used for capturing the amplifying, shrinking and moving operations performed by a user in the process of browsing the pathological image, generating and sending request parameters comprising the level information, the upper right corner coordinates of the visible area and the height and width of the visible area;
the back-end service module receives the request parameters and further comprises:
The preprocessing unit is used for pre-calculating and storing boundary information of irregular graph annotation, and forming a new annotation array by adding minimum and maximum x and y coordinates of the coordinate point array in original annotation data;
A filtration unit comprising: the first sub-filter is responsible for judging and removing labels overlapped at the same position under the current zoom level according to the level information; the second sub-filter screens out the labeling data positioned in the visible area based on the parameters of the visible area;
The data transmission module is used for transmitting the marking data processed by the filtering unit from the front-end service module to the front-end browser;
and the front-end rendering module is used for receiving and rendering and displaying the annotation information on the pathological image in real time according to the filtered annotation data, so that a smooth large-data-volume annotation display function is realized.
In a third aspect, the application provides an electronic device comprising a memory in which a computer program is stored and a processor arranged to run the computer program to perform the above-described pathology image big data annotation display method.
In a fourth aspect, the present application provides a readable storage medium having stored therein a computer program comprising program code for controlling a process to execute a process comprising a pathology image big data annotation display method according to the above.
The main contributions and innovation points of the application are as follows: 1. compared with the prior art, the method and the device can intelligently screen and transmit the required annotation data according to the actual view range and the zoom level of the user, and greatly reduce the data quantity received and rendered by the front end, thereby effectively solving the problems of browser display blocking and slow loading speed caused by large data quantity annotation;
2. Compared with the prior art, the method and the device have the advantages that the consistency and the accuracy between the labeling information and the pathological images are improved by introducing the dynamic adjustment mechanism of the labeling size, so that doctors can better understand and read the morphological characteristics of the focus part intuitively and accurately.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the other features, objects, and advantages of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a view of a pathological image without any added labels;
FIG. 2 is a view of a pathological image without any labels added;
FIG. 3 is a view of a reader style zoom-in and zoom-out layer structure according to an embodiment of the present application;
FIG. 4 is a browser and backend services interaction flow according to an embodiment of the present application;
FIG. 5 is a diagram of two example (A, B) labels of an image strip that is magnified by a browser's view to a certain level, in accordance with an embodiment of the present application;
FIG. 6 is a diagram of a case where the scaled-down callout A, B appears as a single point, according to an embodiment of the present application;
FIG. 7 is a diagram of the original json file size, in accordance with an embodiment of the present application;
FIG. 8 is a diagram of the size of data after processing according to an embodiment of the present application;
Fig. 9 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of the present specification. Rather, they are merely examples of apparatus and methods consistent with aspects of one or more embodiments of the present description as detailed in the accompanying claims.
It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than described in this specification. Furthermore, individual steps described in this specification, in other embodiments, may be described as being split into multiple steps; while various steps described in this specification may be combined into a single step in other embodiments.
Example 1
The embodiment of the application provides a pathological image big data annotation display method, and particularly relates to a method shown in fig. 4, which mainly comprises the following key steps:
① Front-end interaction and parameter acquisition:
when a user performs zooming-in, zooming-out or moving operation on a pathological image in a browser, the system acquires current visual area parameters in real time, wherein the current visual area parameters comprise a hierarchy (namely, a zoomed-in or zoomed-out hierarchy), the right upper corner coordinates of a visual area and the height and width of the visual area. These parameters are sent to the backend service for dynamic processing and delivery of the corresponding annotation information.
Preferably, not only are the zoom-in, zoom-out and movement operations monitored, but also the occurrence of scrolling events is of particular concern during the user's browsing of the pathology images. When a user scrolls a page, the system automatically captures the change of the scroll position, updates the parameters of the visual area in real time, and immediately sends an update request to the back-end service, so as to re-acquire and display the labeling information of the corresponding area. By adding the processing of the scrolling event, the continuous and smooth display of pathological image annotation along with page scrolling is realized, so that a user can view complete and accurate annotation information without repeatedly and manually adjusting views when viewing a longer pathological image sequence.
Wherein, the specific mathematical model and steps of the coordinate transformation are as follows:
the fifth layer has a coordinate of A (800:800), the scale is 0.5, and the total of 5 layers is 5 layers, and the fourth layer has a coordinate of 400, 400;
the coordinate value multiplied by the zoom magnification level index (total level-current level);
The formula is: x=x scaling power ≡ (total level-current level);
y=y scaling power ≡ (total level-current level);
② And a pretreatment module:
For each complex irregular graph annotation (such as A and B), a preprocessing module firstly calculates minimum and maximum x and y values in a coordinate point array contained in each annotation, and adds the boundary information into original annotation data to form a new annotation array. Therefore, in the subsequent requests, the boundaries do not need to be recalculated each time, but the boundaries are directly read from the cache, so that the data processing efficiency is improved.
In fig. 5, the labels a and B are formed by connecting coordinate point arrays, so as to form an irregular graph, respectively find the minimum and maximum values of x, y in the coordinate point arrays in the labels, add the minimum and maximum values into the original data to form a new label array, transmit the new label array into a filtering module, and buffer the new data, so that the maximum value and the minimum value do not need to be calculated when the new data is requested again. For example, the a label: [ { x1, y1}, { x2, y2},.+ -. Xmin, ] +xmin, ymin, xmax, ymax= [ { x1, y1}, { x2, y2},...
The caching mechanism is to directly calculate and combine the marking information after AI analysis into new data to be stored in a database or NoSql, and the problem of data consistency is avoided because the marking information is only processed once and is not repeatedly calculated.
Preferably, an adaptive adjustment function of labeling size and shape is added in the preprocessing module, and based on the scaling of the image, the coordinate point of each label is correspondingly scaled to ensure that the label always keeps consistent proportion with the corresponding pathological tissue structure. By introducing a dynamic adjustment mechanism of the labeling size, the consistency and the accuracy between the labeling information and the pathological image are improved, and a doctor is helped to understand and read the morphological characteristics of the focus part more intuitively and accurately.
③ And a filtering module:
(1) Filter one: for handling the situation where the original irregular figure may appear as a single point due to scaling at different zoom levels. Whether the display is a single point is judged by calculating the size of the region marked under a specific hierarchy. For example, at a certain zoom level, if the area sizes of the labels a and B are both shrunk to zero, it means that they are displayed in superposition as a point in the current view. And further judging whether the two points are positioned at the same position by comparing the minimum coordinates of the two points after the adjustment of the corresponding scaling factors, and if so, displaying only one point to avoid visual overlapping and confusion.
The minimum and maximum coordinates of the labeling data for labeling a and B as assumed in fig. 3 and 5 are: a is [ [50,50], [60,60] ] B is [ [80:80], [100,100] ], the visible area shows a first layer, 5 layers in total, each layer is enlarged or reduced by 0.5 times, whether the point is determined by whether the area size is 0 or 1, the 4 th round of 0.5 is marked with the area size of [60-50,60-50 ]. Times.0.5 and is equal to 0, and the 4 th round of 0.5 is also marked with the B is equal to [100-80,100-80 ]. Times.0.5, so that the point can be judged in the case of the upper layer shrinkage, and whether two points are at the same position or not is judged by multiplying the zoom level by the minimum coordinates such as: a = [50,50] = [0,0] to the power of 4 of 0.5, B = [80,80] = [0,0] to the power of 4 of 0.5, so that the positions displayed on the first layer of the two marks are the same, one of the positions can be filtered, and only the point C is displayed as shown in fig. 6.
Preferably, in the first filter, the overlapped single-point labels are subjected to deep search and merging processing by adopting a hierarchical traversal algorithm, and meanwhile, the total number of the merged single-point labels is displayed on an interface, so that a user can know the overall label distribution condition conveniently. Unnecessary repeated labeling display in the view is eliminated, clear labeling quantity statistics is provided, and the key points of the illness state can be evaluated and mastered quickly by a user. The following is a preferred embodiment:
1. Acquiring and sorting labeling data: firstly, the system acquires all irregular graph annotation data related to the current pathological image from an original annotation database, wherein the data exists in the form of a coordinate point array.
2. Identifying overlapping single-point labels: after the zooming-in or zooming-out operation is carried out, the system detects the display state of each label under the current level through an algorithm and judges whether overlapping single-point labels exist or not. This step typically involves calculating the position of each annotation on a scaled projection onto a pixel of the screen, and if multiple annotations are found to be projected at the same pixel location, then they are considered to be likely to overlap.
3. The hierarchical traversal algorithm applies: and traversing all the possibly overlapped single-point labels by using a hierarchical traversal algorithm, and establishing an association relation map. For each overlapping point, the algorithm searches other adjacent marking points with the same positions, and records the original marking object to which the marking points belong.
4. Combining and counting: and selecting a representative label for displaying the detected overlapping single points, and combining and hiding the rest overlapping labels under the current level. Meanwhile, the system records and accumulates the number of the overlapped single-point labels.
5. Display and feedback: in the front-end interface, only the processed non-overlapped or combined single-point labels are displayed, and the total number of the overlapped single-point labels under the current level is displayed at a proper position (such as a tool-tip or label statistics area), so that a user is helped to understand and effectively manage a large amount of label information, and diagnosis and analysis efficiency is improved.
(2) And a second filter: to ensure smooth display and not waste bandwidth resources, filter two would exclude annotation data that is not within the current browser's viewable area. By doubling the viewable area (to accommodate possible panning operations), new origin coordinates (ox, oy) and enlarged widths, heights (owidth, oheight) are set, and then it is determined whether each annotated boundary falls within the enlarged viewable area according to the coordinate relationship. Only labels falling within this range will be filtered out and returned to the front-end browser for display. If the translation image can stably display the label, the visible area is doubled, and the origin coordinates are:
(ox, oy) = (x-width, y-height) with an aspect of owidth =width×2 and oheight =height×2, and determining whether the formula is in the visible region is as follows:
ox < minx < ox+ owidth and oy < miny < oy+ oheight
Preferably, in the second filter, a polygon clipping algorithm is used to accurately determine whether each complex shape label is completely located in the visible rectangular region, so as to ensure that only complex label data related to the current view is returned to the front end. The filtering module improved by the polygon clipping algorithm can screen out the marking data to be displayed more accurately, reduce invalid data transmission and remarkably improve the performance and fluency of the browser when displaying a large number of complex marks. The following is a preferred embodiment:
1. Obtaining parameters of a visible area: first, the system receives the view area parameters from the front end browser, including the hierarchical information, the upper right corner coordinates of the view area, the height and width of the view area.
2. Positioning complex labeling: for each labeling of complex shapes, consider it as a polygon, whose coordinate points are defined by an array. The system compares these polygon labels to the current viewable area.
3. Applying a polygon clipping algorithm: and adopting a polygon clipping algorithm to judge the intersection of each complex labeling polygon and the current visible rectangular area. The algorithm can accurately calculate the portion of the marked polygon that is visible within the current window.
4. Screening labels within the viewable area: for each annotation, if its visible portion reaches a threshold (e.g., fully or partially visible) as compared to the entire annotation area, the annotation is considered to be within the visible area, leaving the data for the annotations.
5. Rejecting labels outside the visible area: complex annotations that are not screened by the polygon clipping algorithm, i.e., portions thereof that are not visible within the current window, will not be sent to the front end for display, thereby reducing unnecessary data transmission and rendering effort.
6. Transmitting the screened data to the front end: and finally, the system sends the complex annotation data which is screened and confirmed to be displayed in the current visible area to the front-end browser, so that the front-end is ensured to only render and display the annotation information related to the current viewing content of the user, and the browsing and operating fluency is improved.
Wherein no abnormal data exists, because the annotation is removed without the annotation information and is not displayed in the viewable area if too small or too large. Since the sampling is basically 0.5 or quarter times at the time of image production and is not an integer, the filtering algorithm of the invention is not affected, and if the new coordinates calculated by scaling are decimal places, the new coordinates are rounded off, and the latter decimal places are removed.
The two layers of filtering greatly reduce the annotation data to go to the browser of the client, and only the really needed data is returned for display each time, the core idea is that when the annotation is reduced, a plurality of annotations are displayed on the same coordinate, only one of the annotations is displayed, when the annotation is enlarged, only the data in the visible area of the current browser is returned, and the data does not need to be displayed and pushed to the browser, so that the problems of large data annotation on the browser display and slow loading are solved.
Preferably, after preprocessing and filtering are completed, the annotation data suitable for the current window display is stored in a local cache of the client, and when the user returns to the same view range again, the annotation data is preferentially read from the cache, and the server is not frequently requested, so that the network transmission burden is reduced. Description of effects: by introducing a caching mechanism, the network communication times and delay are greatly reduced, the response speed and the overall experience of a user when browsing, switching and zooming pathological images are remarkably improved, and the method is particularly outstanding in a low-bandwidth environment.
And it has been verified that by the method of the present invention, it is possible to return a json file of several tens megabits to the desired content only by several tens kB, as shown in fig. 7 and 8.
The above-mentioned polygon clipping algorithm is a technology for processing geometric figures in computer graphics, and is mainly used for determining a portion of one polygon (two-dimensional or three-dimensional) inside or intersecting another geometric object (usually another polygon or a rectangular window). The purpose of such algorithms is to cut out parts of the source polygon within the target region, or to calculate an intersection, union or difference between two or polygons.
Example two
Based on the same conception, the application also provides a pathological image big data annotation display device, which comprises:
The user interaction module is used for capturing the amplifying, shrinking and moving operations performed by a user in the process of browsing the pathological image, generating and sending request parameters comprising the level information, the upper right corner coordinates of the visible area and the height and width of the visible area;
the back-end service module receives the request parameters and further comprises:
The preprocessing unit is used for pre-calculating and storing boundary information of irregular graph annotation, and forming a new annotation array by adding minimum and maximum x and y coordinates of the coordinate point array in original annotation data;
A filtration unit comprising: the first sub-filter is responsible for judging and removing labels overlapped at the same position under the current zoom level according to the level information; the second sub-filter screens out the labeling data positioned in the visible area based on the parameters of the visible area;
The data transmission module is used for transmitting the marking data processed by the filtering unit from the front-end service module to the front-end browser;
and the front-end rendering module is used for receiving and rendering and displaying the annotation information on the pathological image in real time according to the filtered annotation data, so that a smooth large-data-volume annotation display function is realized.
Example III
This embodiment also provides an electronic device, referring to fig. 9, comprising a memory 404 and a processor 402, the memory 404 having stored therein a computer program, the processor 402 being arranged to run the computer program to perform the steps of any of the method embodiments described above.
In particular, the processor 402 may include a Central Processing Unit (CPU), or an application specific integrated circuit (ApplicationSpecificIntegratedCircuit, abbreviated as ASIC), or may be configured as one or more integrated circuits that implement embodiments of the present application.
The memory 404 may include, among other things, mass storage 404 for data or instructions. By way of example, and not limitation, memory 404 may comprise a hard disk drive (HARDDISKDRIVE, abbreviated HDD), a floppy disk drive, a solid state drive (SolidStateDrive, abbreviated SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Memory 404 may include removable or non-removable (or fixed) media, where appropriate. Memory 404 may be internal or external to the data processing apparatus, where appropriate. In a particular embodiment, the memory 404 is a Non-Volatile (Non-Volatile) memory. In particular embodiments, memory 404 includes Read-only memory (ROM) and Random Access Memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (ProgrammableRead-only memory, abbreviated PROM), an erasable PROM (ErasableProgrammableRead-only memory, abbreviated EPROM), an electrically erasable PROM (ElectricallyErasableProgrammableRead-only memory, abbreviated EEPROM), an electrically rewritable ROM (ElectricallyAlterableRead-only memory, abbreviated EAROM) or a FLASH memory (FLASH), or a combination of two or more of these. The RAM may be a static random access memory (StaticRandom-access memory, abbreviated SRAM) or a dynamic random access memory (DynamicRandomAccessMemory, abbreviated DRAM) where the DRAM may be a fast page mode dynamic random access memory 404 (FastPageModeDynamicRandomAccessMemory, abbreviated FPMDRAM), an extended data output dynamic random access memory (ExtendedDateOutDynamicRandomAccessMemory, abbreviated EDODRAM), a synchronous dynamic random access memory (SynchronousDynamicRandom-access memory, abbreviated SDRAM), or the like, where appropriate.
Memory 404 may be used to store or cache various data files that need to be processed and/or used for communication, as well as possible computer program instructions for execution by processor 402.
The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any of the pathology image big data annotation display methods in the above embodiments.
Optionally, the electronic apparatus may further include a transmission device 406 and an input/output device 408, where the transmission device 406 is connected to the processor 402 and the input/output device 408 is connected to the processor 402.
The transmission device 406 may be used to receive or transmit data via a network. Specific examples of the network described above may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, simply referred to as a NIC) that can connect to other network devices through the base station to communicate with the internet. In one example, the transmission device 406 may be a Radio Frequency (RF) module, which is configured to communicate with the internet wirelessly.
The input-output device 408 is used to input or output information.
Example IV
The present embodiment also provides a readable storage medium having stored therein a computer program including program code for controlling a process to execute the process including the pathology image big data annotation display method according to the first embodiment.
It should be noted that, specific examples in this embodiment may refer to examples described in the foregoing embodiments and alternative implementations, and this embodiment is not repeated herein.
In general, the various embodiments may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto. While various aspects of the invention may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.
Embodiments of the invention may be implemented by computer software executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets, and/or macros can be stored in any apparatus-readable data storage medium and they include program instructions for performing particular tasks. The computer program product may include one or more computer-executable components configured to perform embodiments when the program is run. The one or more computer-executable components may be at least one software code or a portion thereof. In addition, in this regard, it should be noted that any blocks of the logic flows as illustrated may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or memory blocks implemented within the processor, magnetic media such as hard or floppy disks, and optical media such as, for example, DVDs and data variants thereof, CDs, etc. The physical medium is a non-transitory medium.
It should be understood by those skilled in the art that the technical features of the above embodiments may be combined in any manner, and for brevity, all of the possible combinations of the technical features of the above embodiments are not described, however, they should be considered as being within the scope of the description provided herein, as long as there is no contradiction between the combinations of the technical features.
The foregoing examples illustrate only a few embodiments of the application, which are described in greater detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that various modifications and improvements can be made without departing from the spirit of the application, which are within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.
Claims (10)
1. The pathological image big data annotation display method is characterized by comprising the following steps:
s00, acquiring request parameters when a user performs zooming-in, zooming-out or moving operation on a pathological image in a browser, wherein the request parameters at least comprise: hierarchical information, upper right corner coordinates of the visible area, and height and width of the visible area;
s10, sending the request parameters to a back-end service;
S20, after receiving the request parameters, the back-end service preprocesses the original annotation information through a preprocessing module, wherein the preprocessing comprises the following steps: determining boundary information of the irregular graph marks and caching;
s30, new labeling data enter two filters of a filtering module, wherein:
Judging whether the labels are single points or not under the current zoom level by the first filter according to the level information, and if so, performing de-duplication display on the labels overlapped at the same position;
the second filter judges whether the label is positioned in the visible area according to the parameters of the visible area, and only retains the label data positioned in the visible area;
and S40, the filtered marking data is sent to a front-end browser by a back-end service, and the front-end browser renders and displays the marking information in real time according to the filtered data.
2. The pathological image big data annotation display method of claim 1, wherein the preprocessing includes: and adding the minimum value and the maximum value of the x coordinate and the y coordinate in each marked coordinate point array to the original data to form a new marked array, and caching.
3. The pathological image big data annotation display method of claim 1, wherein the step S00 further comprises: and acquiring a rolling event of the user, updating the visible area parameter when the user rolls the page, and triggering a subsequent annotation information refreshing process.
4. The pathological image big data annotation display method of claim 1, wherein the step S20 further comprises: the size and shape of the label are dynamically adjusted according to the scaling of the image, and the proportion of the label is kept consistent with that of the actual pathological tissue structure.
5. The method for displaying large data labels of pathological images according to claim 1, wherein in step S30, the first filter uses a hierarchical traversal algorithm to identify and merge overlapping single-point labels, and records and displays statistical information of the number of labels.
6. The method for displaying large data labels of pathological images according to claim 5, wherein in step S30, the second filter adopts a polygonal clipping algorithm to judge and screen out complex shape labels completely located in the visible rectangular region.
7. The method for displaying large data labels of pathological images according to any one of claims 1 to 6, further comprising a buffer optimization step, wherein the label data processed and applicable to the current window is buffered in the client browser between the step S20 and the step S40.
8. A pathological image big data annotation display device, characterized by comprising:
The user interaction module is used for capturing the amplifying, shrinking and moving operations performed by a user in the process of browsing the pathological image, generating and sending request parameters comprising the level information, the upper right corner coordinates of the visible area and the height and width of the visible area;
The back-end service module receives the request parameters and further comprises:
The preprocessing unit is used for pre-calculating and storing boundary information of irregular graph annotation, and forming a new annotation array by adding minimum and maximum x and y coordinates of the coordinate point array in original annotation data;
A filtration unit comprising: the first sub-filter is responsible for judging and removing labels overlapped at the same position under the current zoom level according to the level information; the second sub-filter screens out the labeling data positioned in the visible area based on the parameters of the visible area;
The data transmission module is used for transmitting the marking data processed by the filtering unit from the front-end service module to the front-end browser;
and the front-end rendering module is used for receiving and rendering and displaying the annotation information on the pathological image in real time according to the filtered annotation data, so that a smooth large-data-volume annotation display function is realized.
9. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, the processor being arranged to run the computer program to perform the pathology image big data annotation display method according to any of claims 1 to 7.
10. A readable storage medium, characterized in that the readable storage medium has stored therein a computer program comprising program code for controlling a process to execute a process comprising the pathological image big data annotation display method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410424278.5A CN118016251B (en) | 2024-04-10 | 2024-04-10 | Pathological image big data annotation display method and application thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410424278.5A CN118016251B (en) | 2024-04-10 | 2024-04-10 | Pathological image big data annotation display method and application thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118016251A true CN118016251A (en) | 2024-05-10 |
| CN118016251B CN118016251B (en) | 2024-07-16 |
Family
ID=90954487
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410424278.5A Active CN118016251B (en) | 2024-04-10 | 2024-04-10 | Pathological image big data annotation display method and application thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118016251B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8200669B1 (en) * | 2008-08-21 | 2012-06-12 | Adobe Systems Incorporated | Management of smart tags via hierarchy |
| WO2015195191A1 (en) * | 2014-06-20 | 2015-12-23 | Apple Inc. | Graphical representation generation for multiple points of interest |
| CN105740275A (en) * | 2014-12-10 | 2016-07-06 | 北京神州泰岳软件股份有限公司 | Map display method and device |
| CN109657017A (en) * | 2018-11-05 | 2019-04-19 | 北京市天元网络技术股份有限公司 | It polymerize the methods of exhibiting and device of point set in map |
| CN109977179A (en) * | 2019-02-19 | 2019-07-05 | 中电海康集团有限公司 | A kind of poly- scattered method for visualizing of grid dividing realization magnanimity atural object mark point of layering |
| CN115588004A (en) * | 2022-11-11 | 2023-01-10 | 杭州迪英加科技有限公司 | A method for storing and displaying pathological image AI analysis results |
-
2024
- 2024-04-10 CN CN202410424278.5A patent/CN118016251B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8200669B1 (en) * | 2008-08-21 | 2012-06-12 | Adobe Systems Incorporated | Management of smart tags via hierarchy |
| WO2015195191A1 (en) * | 2014-06-20 | 2015-12-23 | Apple Inc. | Graphical representation generation for multiple points of interest |
| CN105740275A (en) * | 2014-12-10 | 2016-07-06 | 北京神州泰岳软件股份有限公司 | Map display method and device |
| CN109657017A (en) * | 2018-11-05 | 2019-04-19 | 北京市天元网络技术股份有限公司 | It polymerize the methods of exhibiting and device of point set in map |
| CN109977179A (en) * | 2019-02-19 | 2019-07-05 | 中电海康集团有限公司 | A kind of poly- scattered method for visualizing of grid dividing realization magnanimity atural object mark point of layering |
| CN115588004A (en) * | 2022-11-11 | 2023-01-10 | 杭州迪英加科技有限公司 | A method for storing and displaying pathological image AI analysis results |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118016251B (en) | 2024-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8564623B2 (en) | Integrated data visualization for multi-dimensional microscopy | |
| US20220319095A1 (en) | Three-dimensional model and material merging method, device, terminal, storage medium and rendering method | |
| JP5111095B2 (en) | Image reconstruction method with content awareness | |
| CN106126527B (en) | A kind of device and corresponding method of control electronic map scaling | |
| CN106775225A (en) | The method that across document seamless roam browses PDF maps | |
| CN112540713B (en) | Video preview progress bar scaling method, system, device and storage medium | |
| CN111414104A (en) | Electronic map local display method and device | |
| CN111091497A (en) | Map vector line and plane thinning method, intelligent terminal and storage medium | |
| CN114913074A (en) | A superpixel seamless stitching method and system for deep network semantic segmentation | |
| CN105302433A (en) | Medical image display apparatus, method for controlling the same | |
| CN118016251B (en) | Pathological image big data annotation display method and application thereof | |
| CN110211039A (en) | A kind of image processing method and its device | |
| CN110248147B (en) | Image display method and device | |
| CN112991808B (en) | Parking space display method and device for parking area and electronic equipment | |
| JP3369734B2 (en) | Three-dimensional computer-aided design apparatus and method | |
| JP2728411B2 (en) | Graphic data display method and apparatus | |
| CN111755101A (en) | A method and device for displaying pathological slices | |
| CN115588004A (en) | A method for storing and displaying pathological image AI analysis results | |
| CN110851752B (en) | Image transmission method and device | |
| CN112328924B (en) | Method, electronic equipment, medium and system for realizing picture viewer by web side | |
| CN118588250B (en) | AI labeling data rendering method and device for digital pathological section | |
| JP4169265B2 (en) | Information search method and information search apparatus | |
| US12112027B2 (en) | System and method for displaying high-resolution liver cancer pathological image based on image pyramid | |
| CN119917687B (en) | Three-dimensional scene data loading display method and electronic equipment | |
| US20210056702A1 (en) | Dynamically change tracker speed, switch crop rectangles, and display invisible corners via zoom-loupes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |