TWI757861B - Collective tracking and counting device and method - Google Patents

Abstract

A collective tracking and counting device and a collective tracking and counting method are disclosed. The collective tracking and counting device includes an optical imaging device, an image sensor, an analyzing chip and a display. The optical imaging device includes a light source emitting lights to a sample including moving objects, so that the image sensor can continuously sense sample images at different times for the analyzing chip to analyze and generate an analyzing result for the display to display. The collective tracking and counting method includes steps of: (a) providing video sequences of the sample; (b) using the frame differencing technology to detect the moving objects in the sample; (c) accumulating a total differential footprint trajectory (DFT) area over time; and (d) generating characteristic parameters of the sample.

Description

Collective tracking and counting device and method

The present invention relates to moving objects, and more particularly to a collective tracking and counting device and method.

In recent years, low fertility has become one of the serious problems faced by many countries in the world. Take Taiwan as an example. In 2019, Taiwan's fertility rate was 1.15, ranking second to last among the 200 countries in the world, only higher than South Korea. Since fertility is often related to the quality of male sperm, sperm problems related to fertility have become an increasingly important research project in modern society.

When examining couples for infertility problems at a medical facility, the first test is usually a semen analysis. Generally speaking, Total Motile Sperm Count (TMSC) and average curve velocity (Curvilinear Velocity, VCL) are usually two important characteristic parameters for judging sperm quality. Among them, the size of the motile sperm count was highly correlated with the natural pregnancy rate, and sperm samples with a faster average curve velocity tended to contain more healthy sperm.

Medical institutions often use Computer Assisted Sperm Analyzer (CASA, Computer Assisted Sperm Analyzer) to determine the number of motile sperm. The computer-assisted sperm motility analyzer tracks the trajectory of each sperm individually using traditional Point Tracking Technology and is capable of evaluating dozens of different sperm parameters. In order to avoid misjudgment due to the high concentration of semen samples resulting in the overlapping of many sperm, the World Health Organization (WHO) recommends that when using a computer-assisted sperm motility analyzer, the sperm concentration of the semen sample should be diluted to less than 50×106 /ml. However, diluting the sperm concentration of the semen sample not only causes inconvenience to the user, but also increases the inaccuracy of the measurement with each dilution.

In addition, as shown in FIG. 1, if a sperm video is taken as an example, the traditional point tracking technology includes the following steps: First, after inputting a series of frame sequences of the sperm video (step S10), the traditional point tracking technology will track a single frame in a single frame. The background is ignored and only interesting objects such as sperm are recognized (step S12). Next, the conventional point tracking technology classifies the objects and selects only individual spermatozoa of a certain size in a single frame, while ignoring other impurities in the frame that are too large or too small (step S14 ).

Afterwards, the conventional point tracking technology will track the object, identify the sperm with the closest position coordinates between the two frames as the same sperm, and record the movement track coordinates of all the individual sperm between the two frames (step S16 ). It should be noted that when the concentration of the semen sample is high and many sperms overlap, step S16 requires more calculations to distinguish individual sperms without confusion. Then, the conventional point tracking technology records the coordinate positions of all individual sperms between multiple frames, and obtains a series of coordinate points of all individual sperm moving trajectories (step S18 ). Finally, the traditional point tracking technology will calculate the velocity of all individual spermatozoa according to these coordinate points, and obtain the total number and average velocity of motile spermatozoa (step S19).

It can be seen from the above that the traditional point tracking technology still has problems such as long time consumption and few objects that can be analyzed in practical applications, which need to be overcome urgently. Therefore, if a simple sperm motility test device can be developed, it can not only detect important characteristic parameters such as the number of motile sperm (TMSC) and average velocity (VCL) at the same time, but also provide sperm samples without diluting the samples. Preliminary test, which is convenient for subjects to test the quality of their sperm.

In view of this, the present invention proposes a collective tracking and counting device and method to overcome the problems encountered in the prior art.

One embodiment according to the present invention is a collective tracking and counting device. In this embodiment, the collective tracking and counting device includes an optical imaging device, an image sensor, an analysis chip, and a display. The optical imaging device includes a light source for emitting light to a sample including a moving object to generate an image of the sample. The image sensor is arranged relative to the optical imaging device, and is used to continuously sense sample images at different times. The analysis chip is coupled to the image sensor for analyzing sample images at different times to generate analysis results. The display is coupled to the analysis chip for displaying analysis results.

In one embodiment, the analysis wafer adopts the differential trajectory method to analyze the sample images at different times to generate analysis results.

In one embodiment, the differential track area accumulated over time for the moving objects in the sample is obtained by analyzing the wafer.

In one embodiment, the analysis result includes at least one characteristic parameter related to the motion characteristics of the moving objects in the sample.

In one embodiment, the at least one characteristic parameter is Total Motile Sperm Count (TMSC) and/or Curvilinear Velocity (VCL).

Another embodiment according to the present invention is a collective tracking and counting method. In this embodiment, the collective tracking and counting method includes the following steps: (a) providing a video sequence of the sample; (b) detecting moving objects in the sample using frame difference technology; (c) accumulating a total difference track over time area; and (d) generating at least one characteristic parameter of the sample.

In one embodiment, the at least one characteristic parameter is related to motion characteristics of the moving objects in the sample.

In one embodiment, the at least one characteristic parameter is motile sperm count (TMSC) and/or mean velocity of curve (VCL).

In one embodiment, the video sequence of the sample includes a plurality of sample images continuously sensed at different times, and the step (c) uses the differential trajectory method to analyze the sample images.

In one embodiment, the sample images are imaged by a light source emitting light to the moving objects in the sample.

Compared with the prior art, the collective tracking and counting device and method of the present invention utilize an image sensor to continuously record images of moving objects in a sample at different times, and then present the moving objects in the sample in real time through an analysis chip according to a differential trajectory method. The collective movement trajectory of the sample is further calculated, and the characteristic parameters such as the number and average speed of the moving objects in the sample are further calculated, so as to determine the activity level of the sample.

Therefore, the collective tracking and counting device and method of the present invention do not need to use complex point tracking technology and can effectively solve the problem of overlapping moving objects in the sample, so it can be applied to the analysis of high-density samples, and has the advantages of simple steps, The advantages of low cost and wide application range should have quite high market potential.

The advantages and spirit of the present invention can be further understood from the following detailed description of the invention and the accompanying drawings.

Reference will now be made in detail to the exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Elements/components using the same or similar numbers in the drawings and the embodiments are intended to represent the same or similar parts.

A specific embodiment according to the present invention is a collective tracking and counting method. Unlike the point tracking method used in the prior art, which is more complicated, the collective tracking and counting method proposed in the present invention can simultaneously analyze the characteristic parameters of more than 3,500 moving objects contained in a single screen through relatively simple steps, such as activities Sperm count (TMSC) and mean velocity of curve (VCL).

In this embodiment, the collective tracking and counting method of the present invention adopts the frame differencing technique to subtract any two adjacent frames in time before and after the sample image, and calculates the difference between the corresponding pixel positions between the two frames. After subtracting the luminance values, the absolute values are obtained, and all pixel positions whose absolute values of differences are greater than a preset threshold are recorded. These pixel positions can then be defined as the differential footprint (DF) of the moving object. These differential footprints can be accumulated over time into Differential Footprint Trajectory (DFT). As for the number and velocity of moving objects, it can be reflected on the cumulative graph of the differential track area over time.

Please refer to FIGS. 2A to 2C . FIGS. 2A to 2C are respectively collective differential trajectory diagrams of semen samples with different motile sperm counts under a 10x objective lens. Among them, the motile sperm count (TMSC) in Figure 2A was 130; the motile sperm count (TMSC) in Figure 2B was 46; the motile sperm count (TMSC) in Figure 2C was 6. It should be noted that, Figures 2A to 2C respectively include differential trajectories of a plurality of spermatozoa, and a line represents the differential trajectory of a single spermatozoa, from dark gray to white sequentially representing the sperm movement trajectory from 0 to 10 seconds.

Next, please refer to FIG. 3 , which is a flowchart illustrating the collective tracking and counting method in this embodiment. As shown in Figure 3, the collective tracking and counting method consists of the following steps:

Step S20: providing the video sequence of the sample;

Step S22: using the frame difference technology to detect the moving object in the sample;

Step S24: Accumulate the total differential track area over time; and

Step S26: Generate at least one characteristic parameter of the sample.

In practical applications, the "frame difference technique" in step S22 is to subtract the luminance values of the corresponding pixels of any two adjacent frames before and after time, and then take the absolute value. If it is greater than a predetermined threshold, it is determined that it is It is a "moving object", otherwise it is judged as a "non-moving object". In this way, step S22 can remove all stationary objects and backgrounds, and only leave differential footprints of "moving objects" (eg, motile sperm).

In addition, step S24 is to accumulate the differential footprints of all moving sperms in the continuous frame into a collective movement trajectory (ie, differential trajectory) through the "differential trajectory method", and accumulate the collective movement trajectory area (ie, the total differential trajectory area) over time. ). Finally, step S26 calculates the collective information of all moving objects in the sample (such as characteristic parameters such as the number of motile sperm and average velocity) according to the accumulated value of the collective moving track area (ie, the total differential track area) over time.

From the above, it can be seen that the differential trajectory method adopted by the collective tracking and counting method of the present invention does not need to identify, classify and track individual moving objects, and what it wants to present instantly is the collective moving trajectories of all moving objects in the sample, rather than separate moving objects. The respective moving trajectories of each moving object are judged. Therefore, even if the moving objects in the sample overlap each other, it will not affect each other. Therefore, the collective tracking and counting method of the present invention can quickly analyze high-density samples without diluting the sample.

After comparing FIG. 3 and FIG. 1 , it can be clearly known that after the video sequence of the sample is provided, the prior art needs to go through four steps S12 to S18 to obtain the characteristic parameters of the sample, while the present invention only needs to go through two steps. The characteristic parameters of the sample can be obtained from S22 to S24, so the collective tracking and counting method proposed in the present invention can indeed effectively simplify the process steps.

Next, please refer to FIGS. 4A to 4D . 4A is a schematic diagram showing a circular object with a diameter d moving at a rate v in seven consecutive frames from time t0 to t6; FIG. 4B is a diagram showing the differential footprint of a circular object between any two adjacent frames; FIG. 4C is a graph showing the differential trajectory of the circular object; FIG. 4D is a graph showing the cumulative area of the differential trajectory of the circular object over time.

As shown in FIG. 4A , the present invention can continuously record the moving object with the diameter d in the sample through the optical imaging device and the image sensor (see FIG. 6 ) with the time ti (i=0 to 6) to the right at the speed v There are seven consecutive frames that move squarely, and each frame includes a plurality of pixels, and each pixel records a luminance value (0 to 255).

As shown in FIG. 4B , the luminance values of corresponding pixel positions between any two frames adjacent in time (eg, two frames corresponding to i=0 and i=1) can be subtracted, and then absolute values can be obtained. When there is no change in the brightness of the corresponding pixel positions between two adjacent frames, the subtraction is zero. The areas marked with gray-scale color on these pixel positions form the differential footprint of the moving object and when the brightness change is greater than the set threshold, the corresponding pixel positions are marked with a certain gray-scale color to form differential footprint images DF1 to DF1 to DF6.

The larger the total area of the above differential footprints, the more moving objects in the sample or the faster the moving objects. As shown in FIG. 4C , the differential footprints of all moving objects in the sample can be superimposed one by one to form differential trajectory images DFT1 to DFT6 . Over time, each differential trace extends so that the total area of the differential trace increases over time. It should be noted that these differential trajectories can reflect the moving trajectories of all moving objects in the sample. For example, longer differential trajectories represent faster object movement rates, and more differential trajectories represent more moving objects.

As shown in Fig. 4D, a turning point can be obtained on the curve of the accumulation of differential track areas over time, and the total number and average velocity of all moving objects in the sample can be obtained according to the coordinate position of this turning point, without the need for the prior art. Likewise, track each moving object in the sample separately.

Next, as shown in FIG. 5 , the analysis time (about 0.6 seconds) taken by the present invention to analyze the sample by the differential trajectory method is significantly shorter than the analysis time (3.4 seconds to 6.7 seconds) taken by the traditional point tracking method to analyze the sample, and The present invention can analyze 1 to 3,500 moving objects by using the differential trajectory method, which is significantly more than that of the traditional point tracking method, which can only analyze 1 to 200 moving objects. In other words, compared with the traditional point tracking method, the differential trajectory method adopted in the present invention takes less time and can analyze more moving objects. It should be noted that the calculation time spent in analyzing a sample by the differential trajectory method is independent of the number of moving objects contained in the sample, so it can quickly analyze a high-density sample without diluting the sample.

Another embodiment according to the present invention is a collective tracking and counting device. Different from the complex point tracking method used in the prior art, the collective tracking and counting device proposed in the present invention can simultaneously analyze the characteristic parameters of more than 3,500 moving objects included in a single screen through relatively simple steps, such as activities Sperm count (TMSC) and mean velocity of curve (VCL).

Please refer to FIG. 6 , which is a schematic diagram of the collective tracking and counting device in this embodiment. As shown in FIG. 6 , the collective tracking and counting device 1 includes an optical imaging device 10 , an image sensor 12 , an analysis wafer 14 and a display 16 . The optical imaging device 10 includes a light source 100 for emitting light to the sample SP including the moving object MO to generate an image of the sample. The image sensor 12 is disposed relative to the optical imaging device 10 for continuously sensing sample images at different times. The analysis chip 14 is coupled to the image sensor 12 for analyzing sample images at different times to generate analysis results. The display 16 is coupled to the analysis chip 14 for displaying analysis results.

In practical applications, the analysis chip 14 uses the differential trajectory method to analyze the sample images at different times, so as to obtain the differential trajectory areas accumulated over time of the moving objects MO in the sample SP. The analysis result may include at least one characteristic parameter related to the motility characteristics of the moving objects MO in the sample SP, such as, but not limited to, motile sperm count (TMSC) and/or mean velocity of curve (VCL).

Compared with the prior art, the collective tracking and counting device and method of the present invention utilize an image sensor to continuously record images of moving objects in a sample at different times, and then present the moving objects in the sample in real time through an analysis chip according to a differential trajectory method. The collective movement trajectory of the sample is further calculated, and the characteristic parameters such as the number and average speed of the moving objects in the sample are further calculated, so as to determine the activity level of the sample.

Therefore, the collective tracking and counting device and method of the present invention do not need to use complex point tracking technology and can effectively solve the problem of overlapping moving objects in the sample, so it can be applied to the analysis of high-density samples, and has the advantages of simple steps, The advantages of low cost and wide application range should have quite high market potential.

S10~S19: Steps S20~S26: Steps ti: Time d: Diameter v: Velocity DF1~DF6: Differential footprint image b _R : First position b _F : Second position DFT1~DFT6: Differential track image

The accompanying drawings of the present invention are described as follows: FIG. 1 is a flowchart illustrating a conventional point tracking object tracking and counting method. 2A to 2C are respectively gray-scale color-coded differential trajectories of semen samples with different motile sperm counts. FIG. 3 is a flowchart illustrating a collective tracking and counting method according to a preferred embodiment of the present invention. FIG. 4A is a schematic diagram showing a circular object with a diameter d moving at a rate v in seven consecutive frames from time t0 to t6. FIG. 4B is a graph showing the differential footprint of a circular object between any two adjacent frames. FIG. 4C is a differential trajectory diagram of a circular object. FIG. 4D is a graph showing the accumulation of differential track areas of a circular object over time. FIG. 5 is a graph showing the comparison between the analysis time spent by the differential trajectory method of the present invention and the conventional point tracking method and the number of moving objects that can be analyzed. FIG. 6 is a schematic diagram illustrating a collective tracking and counting device according to another preferred embodiment of the present invention.

S20~S26: Steps

Claims (9)

A collective tracking and counting device, comprising: an optical imaging device including a light source for emitting a light to a sample and the sample including a plurality of moving objects; an image sensor disposed relative to the optical imaging device, for continuously sensing a plurality of sample images at different times; an analysis chip, coupled to the image sensor, for analyzing the sample images at different times to generate an analysis result of the sample; and a display, The analysis chip is coupled to display the analysis result; wherein, the analysis chip adopts the differential trajectory method to analyze the sample images at different times to generate the analysis result. The collective tracking and counting device as described in claim 1, wherein the analysis wafer analyzes to obtain differential track areas accumulated over time of the moving objects in the sample. The collective tracking and counting device as described in claim 1, wherein the analysis result includes at least one characteristic parameter related to the movement characteristics of the moving objects in the sample. The collective tracking and counting device according to claim 3, wherein the at least one characteristic parameter is Total Motile Sperm Count (TMSC) and/or Curvilinear Velocity (VCL). A collective tracking and counting method, comprising the steps of: (a) providing a video sequence of a sample; (b) detecting a plurality of moving objects in the sample from the video sequence using frame difference techniques; (c) accumulating the total differential track area of the moving objects over time; and (d) generating at least one characteristic parameter of the sample. The collective tracking and counting method as described in claim 5, wherein the at least one characteristic parameter is related to the movement characteristics of the moving objects in the sample. The collective tracking and counting method as described in claim 6, wherein the at least one characteristic parameter is Total Motile Sperm Count (TMSC) and/or Curvilinear Velocity (VCL). The collective tracking and counting method as described in claim 5, wherein the video sequence of the sample includes a plurality of sample images continuously sensed at different times, and step (c) adopts a differential trajectory method for these samples. Sample images were analyzed. The collective tracking and counting method as described in claim 8, wherein the sample images are imaged by a light source emitting a light to the moving objects in the sample.

Images