CN115345819A - Gastric cancer image recognition system, device and application thereof - Google Patents

Abstract

The present invention relates to a gastric cancer image recognition system, device and application thereof. The system includes a data input module, a data preprocessing module, an image recognition model building module and a lesion recognition module; at the same time, the system can realize self-training, thereby accurately recognizing gastric cancer lesion in the image.

Description

A gastric cancer image recognition system, device and application thereof

technical field

The invention belongs to the field of medicine, and more specifically relates to the technical field of utilizing an image recognition system to realize pathological image recognition.

Background technique

Although the incidence of gastric cancer has gradually decreased since 1975, there were still nearly 1 million new cases in 2012, making it the fifth most common malignancy in the world. In terms of mortality, gastric cancer is the third leading cause of cancer death in the world.

The prognosis of gastric cancer largely depends on its divergence. Studies have shown that the 5-year survival rate of early gastric cancer is almost more than 90%, while the survival rate of advanced gastric cancer is lower than 20%. Therefore, early detection and regular follow-up in high-risk cancer populations are the most effective means to reduce the incidence of gastric cancer and improve the survival rate of patients.

Due to the high rate of misdiagnosis and missed diagnosis of gastric cancer (especially superficial flat lesions) by ordinary white-light endoscopy, various endoscopic diagnostic techniques have emerged as the times require. However, the application of these endoscopic devices requires not only superb operating skills, but also considerable economic support. Therefore, there is an urgent need to develop an easy-to-obtain, economical, practical, safe and reliable diagnostic technique for discovering and diagnosing early gastric cancer and precancerous lesions.

Contents of the invention

In the long-term medical practice, in order to reduce various problems caused by artificial endoscopic diagnosis, the inventor used machine learning technology to obtain a system that can be used for gastric cancer diagnosis after repeated development, repeated optimization and training. Strict image screening and preprocessing further improves the efficiency of training. The diagnostic system of the present invention can very accurately identify cancer lesion sites in pathological images (such as gastroscopy pictures and real-time images), and its recognition rate has even surpassed that of internal medicine experts.

The first aspect of the present invention provides a gastric cancer image recognition system, which includes:

A, data input module, for inputting the image that comprises gastric cancer lesion, described image is preferably endoscopic image;

b. The data preprocessing module is used to receive the image from the data input module, and accurately frame the lesion of gastric cancer. The part inside the frame is defined as a positive sample, and the part outside the frame is defined as a negative sample, and output Coordinate information and/or lesion type information of the lesion; preferably, before frame selection, the module also desensitizes the image in advance to remove the patient's personal information;

Preferably, the frame selection can generate a rectangular frame or square frame containing the lesion site; the coordinate information is preferably the coordinate information of the points in the upper left corner and lower right corner of the rectangular frame or square frame;

Also preferably, the frame selection site is determined by the following method: 2n endoscopists perform frame selection in a "back-to-back" manner, that is, 2n people are randomly divided into n groups, 2 people/group, and all images are randomly divided into n parts, and Randomly assign physicians to each group for frame selection; when the frame selection is completed, compare the frame selection results of the two physicians in each group, evaluate the consistency of the frame selection results between the two physicians, and finally determine the frame selection site, of which n is a natural number between 1-100, such as 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 70, 80, 90 or 100;

Further preferably, the criteria for evaluating the consistency of the frame selection results between the two physicians are as follows:

For each lesion picture, compare the overlapping area of the boxing results of the two doctors in each group. If the area of the overlapping part of the parts selected by the two doctors in each group (that is, the intersection) is greater than the area covered by the union of the two doctors 50% of the area, it is considered that the frame selection results of the two doctors are consistent, and the diagonal coordinates corresponding to the above intersection, that is, the coordinates of the upper left corner and the lower right corner, are saved as the final location of the target lesion;

If the area of the overlapping part (that is, the intersection) is less than 50% of the area covered by the union of the two, it is considered that the frame selection results of the two doctors are quite different, and such lesion pictures are selected separately, and all participating doctors The 2n physicians in the frame selection work discuss together to determine the final location of the target lesion;

C, image recognition model construction module, can receive the image after data preprocessing module process, be used for building and training the image recognition model based on neural network, described neural network is preferably a convolutional neural network;

d. The lesion recognition module is used to input the image to be checked into the trained image recognition model, and determine whether there is a lesion and/or the location of the lesion in the image to be checked based on the output result of the image recognition model.

In one embodiment, the image recognition model building block includes a feature extractor, a candidate region generator and an object recognizer, wherein:

The feature extractor is used to perform feature extraction on the image from the data preprocessing module to obtain a feature map, preferably, the feature extraction is performed by a convolution operation;

The candidate region generator is used to generate several candidate regions based on the feature map;

The target recognizer calculates the classification score of the candidate region, the score indicates the probability that the region belongs to the positive sample and/or the negative sample; at the same time, the target recognizer can propose an adjustment value for the border position of each region , so as to adjust the frame position of each region, and then accurately determine the lesion position; preferably, a loss function (Loss function) is used in the training of the classification score and the adjusted value;

Also preferably, when performing the training, a gradient descent method based on mini-batch is used, that is, a mini-batch containing a plurality of positive and negative candidate regions is generated for each training picture; Sample 256 candidate regions until the ratio of positive candidate regions to negative candidate regions is close to 1:1, and then calculate the corresponding mini-batch loss function; if the number of positive candidate regions in a picture is less than 128, use negative candidates area to fill this mini-batch;

Further preferably, the learning rate of the first 50,000 mini-batches is set to 0.001, and the learning rate of the last 50,000 mini-batches is set to 0.0001; the momentum item is preferably set to 0.9, and the weight decay is preferably set to 0.0005.

In another embodiment, wherein the feature extractor can perform feature extraction on an input image of any size and/or resolution, the image can be the original size and/or resolution, or can be changed in size and / or the input image after resolution to obtain a multi-dimensional (eg 256-dimensional or 512-dimensional) feature map;

Specifically, the feature extractor includes X convolutional layers and Y sampling layers, wherein the i-th (i is between 1-X) convolutional layers include Q _i size m*m*p _i Convolution kernel, where m*m represents the pixel value of the length and width of the convolution kernel, p _i is equal to the number of convolution kernels Q _i-1 of the previous convolution layer, in the i-th convolution layer, the convolution kernel Convolute the data from the upper level (such as the original image, the i-1th convolutional layer, or the sampling layer) with a step size L; each sampling layer contains 1 moving with a step size 2L, the size is The 2L*2L convolution kernel performs convolution operation on the image input by the convolution layer; among them, after feature extraction by the feature extractor, the feature map of Qx dimension is finally obtained;

Wherein X is between 1-20, for example, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19 or 20 ; Y is between 1-10, such as 1, 2, 3, 4, 5, 6, 7, 8, 9 or 10; m is between 2-10, such as 2, 3, 4, 5, 6, 7 , 8, 9 or 10; p is between 1-1024, Q is between 1-1024, and the values of p or Q are respectively 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 32, 64, 128, 256, 512 or 1024.

In another embodiment, wherein the candidate region generator sets a sliding window in the feature map, the size of the sliding window is n×n, such as 3×3; the sliding window slides along the feature map, and for the sliding window For each position, there is a corresponding relationship between its center point and the corresponding position in the original image, and k candidate regions with different scales and aspect ratios are generated in the original image centering on the corresponding position; wherein, if k The candidate regions have x types (eg, 3 types) of different scales and aspect ratios, then k=x ² (eg, k=9).

In another embodiment, the target recognizer includes an intermediate layer, a classification layer and a border regression layer, wherein the intermediate layer is used to map the data of the candidate regions formed by the sliding window operation, and is a multi-dimensional (such as 256 dimensions or 512 dimension) vector;

The classification layer and the frame regression layer are respectively connected to the middle layer. The classification layer is used to determine whether the target candidate area is the foreground (ie positive sample) or the background (ie negative sample), and the frame regression layer is used to generate the x coordinates and The y coordinate, and the width w and height h of the candidate area.

A second aspect of the present invention provides a recognition device for gastric cancer images, including a storage unit storing diagnostic images of gastric cancer, an image preprocessing program, and a trainable image recognition program, preferably also including a computing unit and a display unit;

The device can use an image recognition program that includes images of gastric cancer lesions for training (preferably supervised training), so that the trained image recognition program can identify the gastric cancer lesion in the image to be checked;

Preferably, the image to be inspected is an endoscopic photo or a real-time image.

In one embodiment, wherein the image preprocessing program accurately frames the lesion of gastric cancer in the gastric cancer diagnostic image, the part within the frame is defined as a positive sample, and the part outside the frame is defined as a negative sample, And output the location coordinate information and/or lesion type information of the lesion; preferably, before the frame selection, the image is desensitized in advance to remove the patient's personal information;

Preferably, the frame selection can generate a rectangular frame or a square frame containing the lesion; the coordinate information is preferably the coordinate information of the points in the upper left corner and the lower right corner;

Also preferably, the frame selection site is determined by the following method: 2n endoscopists perform frame selection in a "back-to-back" manner, that is, 2n people are randomly divided into n groups, 2 people/group, and all images are randomly divided into n parts, and randomly Allocate physicians to each group for frame selection; when the frame selection is completed, compare the frame selection results of the two physicians in each group, and evaluate the consistency of the frame selection results between the two physicians, and finally determine the frame selection site, where n is a natural number between 1 and 100, such as 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 70, 80, 90 or 100;

Further preferably, the criteria for evaluating the consistency of the frame selection results between the two physicians are as follows:

For each lesion image, compare the overlapping area of the frame selection results of the two physicians in each group, if the area of the overlapping part of the parts selected by the two physicians in each group (that is, the intersection) is greater than the area covered by the union of the two physicians 50% of the area, it is considered that the frame selection results of the two doctors are consistent, and the diagonal coordinates corresponding to the above intersection are saved as the final location of the target lesion;

If the area of the overlapping part (that is, the intersection) is less than 50% of the area covered by the union of the two, it is considered that the frame selection results of the two doctors are quite different, and such lesion pictures are selected separately, and all participating doctors The 2n physicians working on frame selection work together to discuss and determine the final location of the target lesion.

In another embodiment, the image recognition program is a trainable image recognition program based on a neural network, and the neural network is preferably a convolutional neural network; preferably, the image recognition program includes a feature extractor, a candidate region generator and object recognizer, where:

The feature extractor is used to perform feature extraction on the image to obtain a feature map, preferably, the feature extraction is performed through a convolution operation;

The candidate region generator is used to generate several candidate regions based on the feature map;

The target recognizer calculates the classification score of the candidate region, the score indicates the probability that the region belongs to the positive sample and/or the negative sample; at the same time, the target recognizer can propose an adjustment value for the border position of each region , so as to adjust the frame position of each region, so as to accurately determine the lesion position; preferably, a loss function (Loss function) is used in the training of the classification score and the adjusted value;

In another embodiment, when performing the training, a mini-batch-based gradient descent method is used, that is, a mini-batch containing multiple positive and negative candidate regions is generated for each training picture. Then randomly sample 256 candidate regions from each picture until the ratio of positive candidate regions and negative candidate regions is close to 1:1, and then calculate the loss function of the corresponding mini-batch. If the number of positive candidate regions in a picture is less than 128, fill the mini-batch with negative candidate regions;

Preferably, the learning rate of the first 50,000 mini-batches is set to 0.001, and the learning rate of the last 50,000 mini-batches is set to 0.0001; the momentum item is preferably set to 0.9, and the weight decay is preferably set to 0.0005.

In another embodiment, wherein the feature extractor can perform feature extraction on an input image of any size and/or resolution, the image can be the original size and/or resolution, or can be changed in size and / or the input image after resolution to obtain a multi-dimensional (eg 256-dimensional or 512-dimensional) feature map;

Specifically, the feature extractor includes X convolutional layers and Y sampling layers, wherein the i-th (i is between 1-X) convolutional layers include Q _i size m*m*p _i Convolution kernel, where m*m represents the pixel value of the length and width of the convolution kernel, p _i is equal to the number of convolution kernels Q _i-1 of the previous convolution layer, in the i-th convolution layer, the convolution kernel Convolute the data from the upper level (such as the original image, the i-1th convolutional layer, or the sampling layer) with a step size L; each sampling layer contains 1 moving with a step size 2L, the size is The 2L*2L convolution kernel performs convolution operation on the image input by the convolution layer; among them, after feature extraction by the feature extractor, the feature map of Qx dimension is finally obtained;

Wherein X is between 1-20, for example, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19 or 20 ; Y is between 1-10, such as 1, 2, 3, 4, 5, 6, 7, 8, 9 or 10; m is between 2-10, such as 2, 3, 4, 5, 6, 7 , 8, 9 or 10; p is between 1-1024, Q is between 1-1024, and the values of p or Q are respectively 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 32, 64, 128, 256, 512 or 1024.

In another embodiment, wherein the candidate region generator sets a sliding window in the feature map, the size of the sliding window is n×n, such as 3×3; the sliding window slides along the feature map, and for the sliding window For each position, there is a corresponding relationship between its center point and the corresponding position in the original image, and k candidate regions with different scales and aspect ratios are generated in the original image centering on the corresponding position; wherein, if k The candidate regions have x types (eg, 3 types) of different scales and aspect ratios, then k=x ² (eg, k=9).

In another embodiment, the target recognizer includes an intermediate layer, a classification layer and a border regression layer, wherein the intermediate layer is used to map the data of the candidate regions formed by the sliding window operation, and is a multi-dimensional (such as 256 dimensions or 512 dimension) vector;

The classification layer and the frame regression layer are respectively connected to the middle layer. The classification layer is used to determine whether the target candidate area is the foreground (ie positive sample) or the background (ie negative sample), and the frame regression layer is used to generate the x coordinates and The y coordinate, and the width w and height h of the candidate area.

The third aspect of the present invention provides the use of the system of the first aspect or the device of the second aspect of the present invention in the prediction and diagnosis of gastric cancer and/or gastric precancerous lesions.

The fourth aspect of the present invention provides the use of the system of the first aspect or the device of the second aspect of the present invention in the identification of gastric cancer images or lesion parts in gastric cancer images.

The fifth aspect of the present invention provides the use of the system of the first aspect or the device of the second aspect of the present invention in the real-time diagnosis of gastric cancer and/or gastric precancerous lesions.

The sixth aspect of the present invention provides the use of the system of the first aspect or the device of the second aspect of the present invention in real-time identification of gastric cancer images or lesion parts in gastric cancer images.

After a long period of exploration, the inventor found that the lesion of gastric cancer has its own characteristics, that is, the lesion is not obvious enough, and the boundary with the surrounding tissue is not clear enough, so the difficulty of image recognition model training is compared with that of conventional tasks (such as recognizing objects in daily life). It is more difficult, and a little carelessness will make the training difficult to converge and lead to failure. In the present invention, the inventor improved the training method of the neural network-based image recognition model (such as accurately defining the target lesion position in the training image by frame selection, improving the recognition accuracy of the image recognition model, etc.), thereby A recognition system (and/or device) for intelligently and efficiently recognizing gastric cancer lesions in endoscopic pictures is obtained, and its recognition rate is higher than that of ordinary endoscopists. The real-time diagnosis system enhanced by machine learning can also monitor and identify gastrointestinal lesions and their location and probability in real time, which can greatly improve the detection rate of gastric cancer for ordinary doctors, reduce the misdiagnosis rate, and provide a better diagnosis for gastric cancer. Safe and reliable technology.

Description of drawings

Figure 1 contains endoscopic images of gastric cancer lesion sites

Figure 2 Schematic diagram of frame selection process

Fig. 3 is the lesion site of gastric cancer identified by the image recognition system of the present invention.

specific implementation plan

Unless otherwise specified, terms used in this disclosure have their ordinary meanings as understood by those of ordinary skill in the art. The following are the meanings of some terms in this disclosure. If there is any inconsistency with other definitions, the following definitions shall prevail.

definition

The term "gastric cancer" refers to malignant tumors derived from gastric mucosal epithelial cells, including early gastric cancer and advanced gastric cancer.

The term "module" refers to a set of functions capable of achieving a specific effect. The module can be executed only by a computer, or manually, or both.

Get lesion data

The key role of the step of obtaining lesion data is to obtain sample materials for deep learning.

In one embodiment, the acquisition process may specifically include the steps of collection and preliminary screening.

The "acquisition" refers to searching and collecting all endoscopic diagnostic images of all patients with gastric cancer in all endoscopic databases according to the standard of "diagnosed as gastric cancer", for example, in the folder to which the patients diagnosed as "stomach cancer" belong All pictures, that is, all stored pictures of a patient during the entire endoscopy process, so it may also include gastroscopy pictures other than target lesions, for example, the patient was diagnosed with benign ulcers, polyps, etc., but the files under his name The folder also contains pictures stored in various parts of the esophagus, gastric fundus, gastric body, and duodenum during the examination process.

The "preliminary screening" is a step of screening the collected pathological images of patients with gastric cancer, which can be specifically described by an experienced endoscopist based on the relevant content in the "endoscopy findings" and "pathological diagnosis" in the case to carry out. Because the pictures used for deep learning network must be of clear quality and accurate features, otherwise it will lead to more difficult learning or inaccurate recognition results. Therefore, the modules and/or steps of the primary screening of lesion data can select pictures with clear gastric cancer lesion sites from a set of examination pictures.

The important thing is that the initial screening will combine the histopathological results of the patient's biopsy, that is, the description of the atrophy site in the "pathological diagnosis", to accurately locate the lesion, and at the same time take into account the clarity of the picture, shooting angle, magnification, etc., and try to choose those with high definition , Moderate magnification, endoscopic images that can see the whole picture of the lesion.

Through the preliminary screening, it can be ensured that the pictures input into the training set are all high-quality images containing certain lesion parts, thereby improving the feature accuracy of the input training data set, so that the artificial intelligence network can better summarize and summarize the atrophy Image features of lesions to improve diagnostic accuracy.

Lesion Data Preprocessing

The preprocessing is to complete the process of accurately frame-selecting the lesion of gastric cancer, the part within the frame is defined as a positive sample, and the part outside the frame is defined as a negative sample, and the location coordinate information and lesion type information of the lesion are output .

In one embodiment, all or part of the lesion data preprocessing is realized by an "image preprocessing program".

The term "image preprocessing program" refers to a program that can realize the frame selection of the target area in the image, thereby marking the type and range of the target area.

In one embodiment, the image preprocessing program can also desensitize the image to remove personal information of the patient.

In one embodiment, the image preprocessing program is a software written in a computer programming language capable of performing the aforementioned functions.

In another embodiment, the image preprocessing program is software capable of performing a frame selection function.

In a specific embodiment, the software that performs the frame selection function can import the picture to be processed into the software, and display the picture on the operation interface. At this time, the operator (for example, a doctor) who implements frame selection only needs to select the target lesion to be framed. Drag the mouse along the direction from upper left to lower right (or other diagonal directions) to form a rectangular frame or square frame covering the target lesion, and at the same time, the background generates and stores the upper left corner and lower right corner of the rectangular frame Accurate coordinates for unique positioning.

In order to ensure the accuracy of preprocessing (or frame selection), the present invention further strengthens the quality control of frame selection, which is also an important guarantee that the method/system of the present invention can obtain greater accuracy. The specific method is as follows: select 2n bits (For example, 6, 8, 10, etc.) Endoscopists select frames in a "back-to-back" manner, that is, 2n people are randomly divided into n groups, 2 people/group, and all the selected training images are also randomly divided into n and randomly assigned to each group of physicians for frame selection; when the frame selection was completed, compare the frame selection results of the two physicians in each group, and evaluate the consistency of the frame selection results between the two physicians, and finally determine the frame selection parts.

In one embodiment, the consistency evaluation standard is: for the same lesion picture, comparing the frame selection results of 2 physicians in each group, that is, comparing the overlapping areas of the rectangular frames determined by the diagonal coordinates, we stipulate that if two If the area of the overlapped part of the rectangular frames (i.e. the intersection) is greater than 50% of the area covered by the union of the two, it is considered that the frame selection results of the two physicians are consistent, and the diagonal coordinates corresponding to the above intersection are saved for the final location of the target lesion. Conversely, if the area of the overlapped part of the two rectangular frames (that is, the intersection) is less than 50% of the area covered by the union of the two, it is considered that the frame selection results of the two doctors are quite different, and such lesion pictures will be It will be individually selected by the software background, and in the later stage, all physicians involved in the frame selection work will jointly discuss and determine the final location of the target lesion.

Image Recognition Model

The term "image recognition model" refers to an algorithm built on the principles of machine learning and/or deep learning, and may also be referred to as a "trainable image recognition model" or "image recognition program".

In one embodiment, the program is a neural network, preferably a convolutional neural network; in another embodiment, the neural network is based on LeNet-5, RCNN, SPP, Fast-RCNN and / or the convolutional neural network of the Faster-RCNN architecture; where the faster-RCNN can be regarded as a combination of Fast-RCNN and RPN, in one embodiment, based on the faster-RCNN network.

The image recognition program includes at least the following layers: the original image feature extraction layer, the candidate area selection layer and the target recognition layer, and the trainable parameters are adjusted through the preset algorithm.

The term "original image feature extraction layer" refers to a layer or a combination of layers that can extract the original image information in multiple dimensions through mathematical calculations on the input image to be trained. Said layers may actually represent a combination of several different functional layers.

In one embodiment, the original image feature extraction layer can be based on ZF or VGG16 network.

The term "convolutional layer" refers to the network layer responsible for performing convolution operations on the original input image or the image information processed by the sampling layer in the original image feature extraction layer to extract information. The convolution operation actually slides on the input image through a convolution kernel of a specific size (such as 3*3) with a certain step size (such as 1 pixel), and the image is moved during the movement of the convolution kernel. It is realized by multiplying the corresponding weights of the pixels on the convolution kernel and adding all the products together to obtain an output. In image processing, an image is often represented as a vector of pixels, so a digital image can be regarded as a discrete function in a two-dimensional space, for example, expressed as f(x,y), assuming that there is a two-dimensional convolution operation function C (u, v), the output image g(x, y)=f(x, y)*C(u, v) will be generated, and the image blurring and information extraction can be realized by convolution.

The term "training" refers to the repeated self-adjustment of the parameters of the trainable image recognition program by inputting a large number of manually labeled samples, so as to achieve the expected purpose, that is, to realize the recognition of the lesion in the gastric cancer image.

In one embodiment, the present invention is based on faster-rcnn network, and adopts following end-to-end training method in step S4:

(1) Use the model pre-trained on ImageNet to initialize the parameters of the target candidate region generation network (RPN), and fine-tune the network;

(2) Also use the pre-trained model on ImageNet to initialize the Fast R-CNN network parameters, and then use the region proposal extracted by the RPN network in (1) for training;

(3) Use the Fast R-CNN network of (2) to reinitialize the RPN, fix the convolutional layer to fine-tune the RPN network, and only fine-tune the cls and/or reg layers of the RPN;

(4) Fix the convolutional layer of Fast R-CNN in (2), use the region proposal extracted by RPN in (3) to fine-tune the Fast R-CNN network, and only fine-tune the fully connected layer of Fast R-CNN.

The term "candidate area selection layer": refers to the layer or layer combination that selects a specific area on the original image for classification recognition and frame regression through an algorithm. Similar to the feature extraction layer of the original image, the layer can also represent A combination of several different layers.

In one embodiment the region candidate selection layer is directly connected to the original input layer.

In one embodiment, the candidate region selection layer is directly connected to the last layer of the original image feature extraction layer.

In one embodiment, the "candidate region selection layer" may be based on RPN.

The term "target recognition layer" and the term "sampling layer" can sometimes be called a pooling layer. Its operation is similar to a convolutional layer, except that the convolution kernel of the sampling layer only takes the maximum value, average value, etc. of the corresponding position (maximum pooling, average pooling).

The term "feature map", also called feature map, refers to a small-area, high-dimensional multi-channel image obtained after the original image feature extraction layer performs convolution operations on the original image. As an example, the feature map can be a scale of 51* 39 of the 256-channel images.

The term "sliding window" refers to a window of small size (such as 2*2, 3*3) generated on the feature map, moving along each position of the feature map, although the size of the feature map is not large, but due to the feature The graph has been subjected to multiple layers of data extraction (such as convolution), so a larger view can be achieved with a smaller sliding window on the feature map.

The term "candidate area" can also be called candidate window, target candidate area, reference box, bounding box, and it can also be used interchangeably with anchor or anchor box in this article.

In one embodiment, the sliding window is first used to locate a position of the feature map. For this position, k rectangular or square windows with different areas and different proportions are generated, for example, 9, and are anchored at the center of the position, so also It is called anchor or anchor box, and based on the relationship between each sliding window in the feature map and the center position of the original image, a candidate area is formed. The candidate area can essentially be considered as the sliding window moved on the last convolutional layer ( 3*3) corresponds to the area of the original image.

In one embodiment of the present invention, k=9, comprise the following steps when generating candidate region:

(1) First generate 9 kinds of anchor boxes according to different areas and aspect ratios, and the anchor boxes do not change with the feature map or the size of the original input image;

(2) For each input image, calculate the center point of the original image corresponding to each sliding window according to the image size;

(3) Establish the mapping relationship between the sliding window position and the original image position based on the above calculation.

The term "intermediate layer" means that after using the sliding window to form the target candidate area, the feature map is further mapped into a multi-dimensional (such as 256-dimensional or 512-dimensional) vector, and this layer can be regarded as a new level. The present invention referred to as the middle layer. The middle layer is connected to the classification layer and the window regression layer.

The term "classification layer" (cls_score), a branch of the connection with the output of the intermediate layer, which can output 2k scores, corresponding to two scores of k target candidate regions, one of which is the foreground (ie positive sample) Score, one is the background (ie negative sample) score, this score can judge whether the target candidate area is the real target or the background. Therefore, for each sliding window position, the classification layer can output the probability of belonging to the foreground (ie positive samples) and background (ie negative samples) from high-dimensional (eg 256-dimensional) features.

Specifically, in one embodiment, when the IOU (intersection-over-union ratio) of the candidate area and any ground-truth box (real sample boundary, that is, the boundary of the object to be recognized in the original image) is greater than 0.7, it can be considered as Positive samples or positive labels, when the IOU between the candidate area and any ground-truth box is less than 0.3, it is considered to be the background (ie negative samples), and a class label is assigned to each anchor. Among them, the IOU indicates the degree of overlap between the candidate area and the ground-truth box from the above mathematics, and its calculation method is as follows:

IOU＝(A∩B)/(A∪B)

The classification layer can output a k+1-dimensional array p, representing the probability of belonging to k classes and backgrounds. For each RoI (Region of Interesting), the discrete probability distribution is output, and p is calculated by the fully connected layer of the k+1 class using softmax. The mathematical expression is as follows:

p=(p ₀ , p ₁ , . . . , p _k )

The term "windowed regression layer" (bbox_pred), another branch of the connection to the output of the intermediate layer, is juxtaposed with the classification layer. This layer can output the parameters that the 9 anchors correspond to the window should be panned and zoomed at each position. Corresponding to k target candidate areas respectively, each target candidate area has 4 border position adjustment values, these 4 border position adjustment values refer to the x _a coordinate, y _a coordinate of the upper left corner of the target candidate area and the target candidate area Adjustment value for height h _a and width w _a . The role of this branch is to fine-tune the position of the target candidate area to make the final result position more accurate.

The window regression layer can output the displacement of the bounding box regression, and output a 4*K-dimensional array t, indicating the parameters that should be translated and scaled when they belong to the k category. The mathematical expression is as follows:

是指相对于object proposal尺度不变的平 移，

是指对数空间中相对于object proposal的高与宽。k represents the index of the category,

Refers to the scale-invariant translation relative to the object proposal,

Refers to the height and width relative to the object proposal in logarithmic space.

In one embodiment, the present invention implements the simultaneous training of the classification layer and the window regression layer through a loss function (Loss function), which is composed of classification loss (ie classification layer softmax loss) and regression loss (ie L1 loss) according to a certain composed of specific gravity. :

The calculation of softmax loss requires the calibration results and prediction results of the candidate area corresponding to the ground truth; the calculation of regression loss requires three sets of information:

(1) Predict the coordinates x, y and width and height w, h of the center of the candidate area;

(2) Position coordinates x _a , y _a and width and height w _a , h _a of each center point in the 9 anchor point reference boxes around the candidate area.

(3) The center point position coordinates x*, y* and width and height w*, h* corresponding to the ground truth.

The way to calculate regression loss and total Loss is as follows:

t _x =(xx _a )/w _a , t _y =(yy _a )/h _a ,

t _w =log(w/w _a ), t _h =log(h/h _a ),

Among them, p _i is the probability that the anchor is predicted to be the target.

有两个数值，

等于0为负标签，

等于1是正标签。

has two values,

Equal to 0 for negative labels,

Equal to 1 is a positive label.

t _i represents a vector set of 4 parameterized coordinates of the predicted candidate region.

表示postive anchor对应的ground truth包围盒的坐标向量。

Indicates the coordinate vector of the ground truth bounding box corresponding to the postive anchor.

In one embodiment, when the loss function is trained, a gradient descent method based on mini-batch is used, that is, a mini-batch comprising a plurality of positive and negative candidate regions (anchor) is generated for each training picture. Then randomly sample 256 anchors from each picture until the ratio of positive anchors to negative anchors is close to 1:1, and then calculate the corresponding mini-batch loss function (Loss function). If the number of positive anchors in a picture is less than 128, the mini-batch is filled with negative anchors.

In a specific embodiment, the learning rate of the first 50,000 mini-batches is set to 0.001, and the learning rate of the last 50,000 mini-batches is set to 0.0001; the momentum item is preferably set to 0.9, and the weight decay is preferably set to 0.0005.

After the above training, the trained deep learning network is used to identify endoscopic images of target lesions. In one embodiment, the classification score is set to 0.85, that is, the deep learning network confirms that the lesions with a lesion probability exceeding 85% will be marked, so that the picture is judged as positive; on the contrary, if no lesion is detected in a picture Suspicious lesion area, then this picture is judged as negative.

Example

1. Waiver of informed consent statement:

(1) This study only uses the endoscopic pictures and relevant clinical data obtained in the previous clinical diagnosis and treatment by the Gastroenterology Endoscopy Center of Beijing Friendship Hospital to conduct retrospective observation and research, and will not affect the patient's condition, treatment, prognosis or even life safety. any effect;

(2) The main researcher alone completes all data collection work, and immediately applies special software to erase personal information on all pictures after the picture data collection is completed, so as to ensure the follow-up physician screening, frame selection and artificial intelligence programming In the process of expert input training, debugging and testing, the patient's private information was not leaked;

(3) In the electronic medical record query system of the Gastroenterology Endoscopy Center, there are no entries such as "contact information" or "home address" to display, that is, the system has not entered the patient's contact information, so this study cannot be traced back to the inclusion Patients signed informed consent.

2. Pathological image acquisition

Inclusion criteria :

(1) From January 1, 2013 to June 10, 2017, endoscopic examinations (including electronic gastroscopy, electronic colonoscopy, ultrasonic endoscopy, electronic chromoendoscopy, and magnifying endoscopy) were accepted at the Digestive Endoscopy Center of Beijing Friendship Hospital endoscopy and chromoendoscopy) patients;

(2) Patients diagnosed with "gastric cancer" under the microscope (including but not distinguishing early gastric cancer and advanced gastric cancer);

Exclusion criteria :

(1) Gastrointestinal malignant tumors involving extensive or unclear sites;

(2) Those who only have malignant tumors of the pancreaticobiliary system;

(3) Patients with other systemic malignancies at the same time;

(2) The endoscopic picture is not clear and/or the shooting angle does not meet the requirements.

3. Experimental process and results

(1) Data collection: The researchers searched from the electronic medical record system of the Gastroenterology Endoscopy Center of Beijing Friendship Hospital who received endoscopic examinations (including electronic gastroscopy, electronic Colonoscopy, ultrasonography, electronic chromoendoscopy, magnifying endoscopy and chromoendoscopy), and the endoscopic pictures and related clinical findings of patients diagnosed with "gastric cancer" (including but not distinguishing between early gastric cancer and advanced gastric cancer) material;

(2) Erase personal information: Immediately after the collection is completed, all pictures will be erased from personal information.

(3) Image screening: all processed images are refined, and the endoscopic images corresponding to cases with clear pathological results confirmed as gastric cancer are screened out, and the target lesion is finally screened out in each case according to the pathological site of the biopsy A total of 3774 pictures with clear parts and less background interference;

(4) Construct a test data set: a total of 100 test pictures, including 50 "gastric cancer" (early gastric cancer and advanced gastric cancer) confirmed by pathological results, and randomly collected in the database. Stomach images confirmed by pathological results 50 endoscopic pictures of "non-neoplastic lesions" (including benign gastric ulcers, polyps, stromal tumors, lipomas, and heterotopic pancreas). Specific operations include:

First randomly select 50 pictures from all gastric cancer pictures screened out in step (3);

In addition, the "non-neoplastic lesions" of the stomach confirmed by pathological results were randomly collected in the database

(Including benign gastric ulcer, polyp, stromal tumor, lipoma, heterotopic pancreas) 50 endoscopic pictures, and immediately delete the personal information of the above 50 pictures;

(5) Build a training data set: from the gastric cancer pictures screened in step (3), exclude the pictures randomly selected for building the test data set in step (4), and the remaining 3724 pictures are used for deep learning network training, thus forming the training data set;

(6) Frame target lesion: 6 endoscopists randomly divided 6 people into 3 groups in a "back-to-back" manner, 2 people/group; all the training pictures after screening were randomly divided into 3 parts, and randomly assigned to each group Physician checks. The implementation of the lesion frame selection step is based on self-written software, which can display the image on the operation interface after the image to be processed is imported into the software. Drag the mouse in the lower right direction to form a rectangular frame covering the target lesion, and at the same time the background generates and stores the exact coordinates of the upper left corner and lower right corner of the rectangular frame for unique positioning.

After the frame selection is completed, compare the frame selection results of the 2 physicians in each group. For the same lesion picture, compare the overlapping areas of the rectangular frames determined by the diagonal coordinates. If the area (that is, the intersection) is greater than 50% of the area covered by the union of the two, it is considered that the frame selection results of the two physicians are consistent, and the diagonal coordinates corresponding to the above intersection are saved as the final target lesion. position. On the contrary, if the area of the overlapping part of the two rectangular frames (that is, the intersection) is less than 50% of the area covered by the union of the two, it is considered that the frame selection results of the two doctors are quite different, and such lesion pictures will be discarded. The software background (or manual marking) is selected individually, and in the later stage, all physicians involved in the frame selection work will jointly discuss and determine the final location of the target lesion.

(7) Input training: input all the above-mentioned frame-selected pictures into the convolutional neural network based on faster-rcnn for training, and test the two network structures of ZF and VGG16; the training adopts an end-to-end method;

The ZF network has 5 convolutional layers, 3 fully connected layers and a softmax classification output layer, and the VGG16 network has 13 convolutional layers, 3 fully connected layers and a softmax classification output layer. Under the framework of Faster-RCNN , ZF and VGG16 models are the basic CNNs used to extract training image features.

During training, a gradient descent method based on mini-batch is used, that is, a mini-batch containing multiple positive and negative candidate regions (anchor) is generated for each training picture. Then randomly sample 256 anchors from each picture until the ratio of positive anchors to negative anchors is close to 1:1, and then calculate the corresponding mini-batch loss function (Loss function). If the number of positive anchors in a picture is less than 128, the mini-batch is filled with negative anchors.

Set the learning rate of the first 50,000 mini-batches to 0.001, and set the learning rate of the last 50,000 mini-batches to 0.0001; the momentum item is preferably set to 0.9, and the weight decay is preferably set to 0.0005.

The loss function (Loss Function) used during training is as follows:

In the above formula, i represents the index of the anchor in each batch, p _i represents the probability of whether the anchor is the target (Object); p _i * is the real label of the anchor: when the anchor is Object, the label is 1, otherwise the label is 0. t _i is a 4-dimensional vector representing the parameterized coordinates of the bounding box, and t _i * represents the label of the bounding box parameterized coordinates used in the bounding box regression prediction.

(8) Test and result statistics: Using the test data set (including 50 pictures of gastric cancer and 50 pictures of gastric "non-tumor lesions"), the artificial intelligence system and gastroenterologists with different seniority were tested, compared and evaluated The sensitivity, specificity, accuracy, consistency and other indicators of the two in diagnosis were analyzed statistically. In the test, when the trained deep learning network is used to identify endoscopic pictures of target lesions, the classification score is set to 0.85, that is, only lesions with a lesion probability of more than 85% confirmed by the deep learning network will be marked. It is judged as positive; on the contrary, if no suspicious lesion area is detected in a picture, then this picture is judged as negative.

The result is as follows:

Based on the platform of the National Clinical Research Center for Digestive Diseases, a total of 89 participating physicians had overall sensitivity fluctuations in the range of 48% to 100% in the diagnosis of gastric cancer endoscopic lesions, with a median of 88% and an average sensitivity of 87%. %; the specificity fluctuates in the range of 10% to 98% (the median is 78%, the average specificity is 74%), and the accuracy rate fluctuates in the range of 51% to 91% (the median is 82%, the average accuracy is 80%) %). The recognition sensitivity of the deep learning network model diagnosis is 90%, the specificity is 50%, and the accuracy rate is 70%. Therefore, in the diagnosis of gastric cancer based on gastroscopy pictures, the sensitivity of artificial intelligence is higher than that of the general physicians, but the specificity is lower than the median level, and the accuracy is slightly lower than the median level of physicians, but considering the depth The learning network model diagnostic model has excellent stability in identification, but different doctors have great fluctuations and instability in specificity and accuracy, so the use of artificial intelligence to identify lesions can still effectively exclude individual differences in doctors The diagnostic bias brought by it has a good application prospect.

Among them, sensitivity is also called sensitivity (sensitivity, SEN), also known as true positive rate (true positive rate, TPR), that is, the percentage of the actual disease that is correctly diagnosed by the diagnostic criteria.

Specificity, also known as specificity (specificity, SPE), or true negative rate (true negative rate, TNR), reflects the ability of a screening test to identify non-patients.

Accuracy rate = total number of correctly identified individuals/total number of identified individuals.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. All modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included within the protection scope of the present invention.

Claims (16)

1. A gastric cancer image recognition system, comprising:

a. the data input module is used for inputting an image containing a gastric cancer lesion part, wherein the image is preferably an endoscope image;

b. the data preprocessing module is used for receiving the image from the data input module, precisely framing the lesion part of the gastric cancer, defining the part inside the framing as a positive sample and defining the part outside the framing as a negative sample, and outputting coordinate information and/or lesion type information of the lesion part; preferably, before the frame selection, the module also carries out desensitization treatment on the image in advance to remove personal information of the patient;

preferably, the frame selection can generate a rectangular frame or a square frame containing the lesion; the coordinate information is preferably coordinate information of points at the upper left corner and the lower right corner of the rectangular frame or the square frame;

further preferably, the boxed location is determined by the following method: the 2n endoscopic physicians select the images in a back-to-back mode, namely 2n endoscopic physicians randomly divide the 2n endoscopic physicians into n groups and 2 endoscopic physicians/groups, simultaneously randomly divide all the images into n images and randomly distribute the images to all the endoscopic physicians for selecting the images; when the frame selection is completed, comparing the frame selection results of each group of two physicians, and evaluating the consistency of the frame selection results between the two physicians, and finally determining a frame selection part, wherein n is a natural number between 1 and 100, such as 1, 2,3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 70, 80, 90 or 100;

further preferably, the criterion for evaluating the consistency of the results of the frame selection between two physicians is as follows:

aiming at each lesion picture, comparing the overlapping area of the framing results of two doctors in each group, if the area (namely intersection) of the overlapping parts of the parts respectively framed and selected by the two doctors in each group is more than 50% of the area covered by the union of the two doctors, considering that the framing judgment results of the two doctors have good consistency, and storing the diagonal coordinates corresponding to the intersection, namely the coordinates of the points at the upper left corner and the lower right corner as the final positioning of the target lesion;

if the area of the overlapped part (namely the intersection) is less than 50% of the area covered by the union of the two, the frame selection judgment results of the two doctors are considered to have larger difference, the lesion pictures are selected independently, and all 2n doctors participating in the frame selection work discuss and determine the final position of the target lesion together;

c. the image recognition model building module can receive the image processed by the data preprocessing module and is used for building and training an image recognition model based on a neural network, and the neural network is preferably a convolutional neural network;

d. and the lesion recognition module is used for inputting the image to be detected into the trained image recognition model and judging whether a lesion and/or the position of the lesion exist in the image to be detected based on the output result of the image recognition model.

2. The system of claim 1, the image recognition model building module comprising a feature extractor, a candidate region generator, and a target identifier, wherein:

the feature extractor is used for performing feature extraction on the image from the data preprocessing module to obtain a feature map, and preferably, the feature extraction is performed through a convolution operation;

the candidate region generator is used for generating a plurality of candidate regions based on the feature map;

the target identifier calculating a classification score for the candidate region, the score being indicative of a probability that the region belongs to the positive sample and/or the negative sample; meanwhile, the target recognizer can provide an adjustment value for the frame position of each region, so that the frame position of each region is adjusted, and the position of a focus is accurately determined; preferably, a Loss function (Loss function) is used in the training of the classification score and the adjustment value;

it is also preferable that the training is performed by a mini-batch based gradient descent method, that is, a mini-batch containing a plurality of positive and negative candidate regions is generated for each training picture; then randomly sampling 256 candidate regions from each picture until the proportion of the positive candidate region to the negative candidate region is close to 1, and then calculating a loss function of the corresponding mini-batch; if the number of the positive candidate areas in one picture is less than 128, filling the mini-batch with the negative candidate areas;

further preferably, the learning rate of the first 50000 mini-batch is set to 0.001, and the learning rate of the last 50000 mini-batch is set to 0.0001; the momentum term is preferably set to 0.9 and the weight attenuation is preferably set to 0.0005.

3. The system of claim 2, wherein the feature extractor can perform feature extraction on an input image of any size and/or resolution, the image may be an original image size and/or resolution, or an input image with changed size and/or resolution, to obtain a feature map in multiple dimensions (e.g., 256 dimensions or 512 dimensions);

specifically, the feature extractor comprises X volumesA packed layer and Y sampling layers, wherein the ith (i is between 1-X) packed layer contains Q _i Size of m x m p _i Where m x m represents the pixel values of the length and width of the convolution kernel, p _i Number of convolution kernels equal to the last convolution layer Q _i-1 In the ith convolutional layer, the convolutional kernel performs convolutional operation on the data from the previous stage (such as the original image, the (i-1) th convolutional layer or the sampling layer) by a step length L; each sampling layer comprises 1 convolution kernel which moves by step length 2L and has the size of 2L x 2L, and the convolution operation is carried out on the image input by the convolution layer; after feature extraction is carried out by the feature extractor, a Qx-dimensional feature map is finally obtained;

wherein X is between 1-20, e.g., 1, 2,3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, or 20; y is between 1 and 10, e.g. 1, 2,3, 4, 5, 6, 7, 8, 9 or 10; m is between 2 and 10, such as 2,3, 4, 5, 6, 7, 8, 9 or 10; p is between 1 and 1024, Q is between 1 and 1024, and the value of p or Q is, for example, 1, 2,3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 32, 64, 128, 256, 512, or 1024, respectively.

4. A system according to claim 2 or 3, wherein the candidate region generator sets a sliding window in the feature map, the sliding window having a size of n x n, such as 3 x 3; enabling the sliding window to slide along the feature map, enabling a central point of each position where the sliding window is located to have a corresponding relation with a corresponding position in the original image, and generating k candidate regions with different scales and aspect ratios in the original image by taking the corresponding position as a center; wherein k = x if the k candidate regions have x (e.g. 3) different scales and aspect ratios ² (e.g., k = 9).

5. The system according to any one of claims 2-4, wherein the target identifier further comprises an intermediate layer, a classification layer and a bounding box regression layer, wherein the intermediate layer is used for mapping data of candidate regions formed by sliding window operations, and is a multi-dimensional (e.g. 256-dimensional or 512-dimensional) vector;

the classification layer and the frame regression layer are respectively connected with the intermediate layer, the classification layer is used for judging whether the target candidate region is a foreground (positive sample) or a background (negative sample), and the frame regression layer is used for generating an x coordinate and a y coordinate of a central point of the candidate region and a width w and a height h of the candidate region.

6. A stomach cancer image recognition device comprises a storage unit for storing a stomach cancer diagnosis image, an image preprocessing program and a trainable image recognition program, and preferably further comprises an arithmetic unit and a display unit;

the device can be trained (preferably supervised training) by using an image recognition program containing images of gastric cancer lesions, so that the trained image recognition program can recognize gastric cancer lesion parts in an image to be detected;

preferably, the image to be detected is an endoscopic photograph or a real-time image.

7. The apparatus according to claim 6, wherein the image preprocessing program precisely frames a lesion site of gastric cancer in the gastric cancer diagnostic image, the portion inside the frame is defined as a positive sample, and the portion outside the frame is defined as a negative sample, and outputs position coordinate information and/or lesion type information of the lesion; preferably, before the frame selection, desensitization treatment is carried out on the image in advance to remove personal information of the patient;

preferably, the frame selection can generate a rectangular frame or a square frame containing the lesion site; the coordinate information is preferably coordinate information of points at the upper left corner and the lower right corner;

also preferably, the boxed site is determined by the following method: the 2n endoscopic physicians select the images in a back-to-back mode, namely 2n endoscopic physicians are randomly divided into n groups, 2 endoscopic physicians/group, all the images are randomly divided into n parts at the same time, and the n parts are randomly distributed to all the endoscopic physicians for selecting the images; when the frame selection is completed, comparing the frame selection results of each group of two physicians, evaluating the consistency of the frame selection results between the two physicians, and finally determining a frame selection part, wherein n is a natural number between 1 and 100, such as 1, 2,3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 70, 80, 90 or 100;

further preferably, the criterion for evaluating the consistency of the results of the frame selection between two physicians is as follows:

aiming at each lesion image, comparing the overlapping area of the framing result of each group of 2 doctors, if the overlapping area (namely intersection) of the parts respectively framed by the two doctors in each group is more than 50% of the area covered by the union of the two doctors, considering that the framing judgment result of the 2 doctors has good consistency, and storing the diagonal coordinate corresponding to the intersection as the final positioning of the target lesion;

if the area of the overlapped part (i.e. the intersection) is less than 50% of the area covered by the union of the two, the framing judgment results of 2 doctors are considered to be greatly different, such lesion pictures are separately selected, and all 2n doctors participating in the framing work discuss the final position of the target lesion together.

8. The apparatus of claim 6 or 7, the image recognition program being a trainable neural network based image recognition program, the neural network preferably being a convolutional neural network; preferably, the image recognition program comprises a feature extractor, a candidate region generator and an object recognizer, wherein:

the feature extractor is configured to perform feature extraction on the image to obtain a feature map, and preferably, the feature extraction is performed by a convolution operation;

the candidate region generator is used for generating a plurality of candidate regions based on the feature map;

the target identifier calculating a classification score for the candidate region, the score being indicative of a probability that the region belongs to the positive sample and/or the negative sample; meanwhile, the target recognizer can provide an adjustment value for the frame position of each region, so that the frame position of each region is adjusted, and the position of a focus is accurately determined; preferably, a Loss function (Loss function) is used in the training of the classification score and the adjustment value.

9. The apparatus according to any one of claims 6 to 8, wherein in the training, a mini-batch based gradient descent method is used, i.e. a mini-batch comprising a plurality of positive and negative candidate regions is generated for each training picture. Then 256 candidate regions are randomly sampled from each picture until the ratio of positive candidate region to negative candidate region approaches 1, and then the corresponding mini-batch penalty function is calculated. If the number of the positive candidate areas in one picture is less than 128, filling the mini-batch with the negative candidate areas;

preferably, the learning rate of the first 50000 mini-batchs is set to be 0.001, and the learning rate of the last 50000 mini-batchs is set to be 0.0001; the momentum term is preferably set to 0.9 and the weight attenuation is preferably set to 0.0005.

10. The apparatus according to claim 8 or 9, wherein the feature extractor is capable of performing feature extraction on an input image of any size and/or resolution, the image may be an original image size and/or resolution, or an input image after the size and/or resolution is changed, so as to obtain a feature map of multiple dimensions (for example, 256 dimensions or 512 dimensions);

specifically, the feature extractor comprises X convolutional layers and Y sampling layers, wherein the ith convolutional layer (i is between 1-X) comprises Q _i Size of m x p _i Wherein m x m represents the pixel values of the length and width of the convolution kernel, p _i Number of convolution kernels equal to the last convolution layer Q _i-1 In the ith convolutional layer, the convolutional kernel performs convolutional operation on the data from the previous stage (such as the original image, the (i-1) th convolutional layer or the sampling layer) by a step length L; each sampling layer comprises 1 convolution kernel which moves by step length 2L and has the size of 2L x 2L, and the convolution operation is carried out on the image input by the convolution layer; after feature extraction is carried out by the feature extractor, a Qx-dimensional feature map is finally obtained;

wherein X is between 1-20, e.g., 1, 2,3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, or 20; y is between 1 and 10, e.g. 1, 2,3, 4, 5, 6, 7, 8, 9 or 10; m is between 2 and 10, such as 2,3, 4, 5, 6, 7, 8, 9 or 10; p is between 1 and 1024, Q is between 1 and 1024, and the value of p or Q is, for example, 1, 2,3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 32, 64, 128, 256, 512, or 1024, respectively.

11. The apparatus according to any of claims 8 to 10, wherein the candidate region generator sets a sliding window in the feature map, the sliding window having a size of n x n, such as 3 x 3; enabling the sliding window to slide along the feature map, enabling a central point of each position where the sliding window is located to have a corresponding relation with a corresponding position in the original image, and generating k candidate regions with different scales and aspect ratios in the original image by taking the corresponding position as a center; wherein k = x if the k candidate regions have x (e.g. 3) different scales and aspect ratios ² (e.g., k = 9).

12. The apparatus of any one of claims 8 to 11, wherein the target identifier further comprises an intermediate layer, a classification layer and a bounding box regression layer, wherein the intermediate layer is used for mapping data of candidate regions formed by sliding window operations, and is a multi-dimensional (e.g. 256-dimensional or 512-dimensional) vector;

and the classification layer and the frame regression layer are respectively connected with the intermediate layer, the classification layer is used for judging whether the target candidate region is a foreground (namely a positive sample) or a background (namely a negative sample), and the frame regression layer is used for generating an x coordinate and a y coordinate of a central point of the candidate region and a width w and a height h of the candidate region.

13. Use of the system according to any one of claims 1 to 5 or the device according to any one of claims 6 to 12 for the prediction and diagnosis of gastric and/or pre-gastric cancer lesions.

14. Use of the system according to any one of claims 1 to 5 or the device according to any one of claims 6 to 12 in gastric cancer images or identification of lesion sites in gastric cancer images.

15. Use of the system according to any one of claims 1 to 5 or the device according to any one of claims 6 to 12 for real-time diagnosis of gastric and/or pre-gastric cancer lesions.

16. Use of the system according to any one of claims 1 to 5 or the device according to any one of claims 6 to 12 for real-time identification of gastric cancer images or lesion sites in gastric cancer images.

Images