RU2770634C1 - Method for modelling mammary glands using a neural network - Google Patents

Abstract

FIELD: medicine.

SUBSTANCE: proposed method for modelling the mammary glands is applied in medicine in the field of plastic surgery. The method is implemented by means of implants selected individually and taking into account both the preferences of the patient and their concomitant pathologies. The method includes taking the necessary measurements of the parameters of the human body from a photo using an iOS application, on the basis whereof the user of the application, either the patient or the doctor, orders breast implants satisfying the parameters set by the patient and restoring the natural appearance of the mammary glands of the desired shape and size. The method excludes identification of the patient or their personal information allowing their original image to be recovered. When taking measurements, the reference distance is retrieved from the photo uploaded to the application using a neural network determining the object in the photo. The iris of the human eye, displayed on the screen and determined by the neural network (NN) with markers at the corners of the iris, can act as the object. The photo of a person is therein aligned on the screen of the working device with the application, and measurements of the required parameters are taken and recalculated in real time.

EFFECT: method allows for more accurate measurement of both the mammary gland and the implant, selection of appropriate implants satisfying the requirements of the patient and their health, and improvement in the quality of intervention during implantation.

1 cl, 5 dwg

Description

This method is applicable in the field of medicine and is a highly productive technological process in the manufacture of implants for their subsequent implantation.

Satisfaction with the result of mammoplasty depends on the correct choice of the implant. According to statistics, the vast majority of women are unhappy with their breasts. They are not satisfied with the shape of the breast, the size of the breast. The determining factor in the ability to improve the appearance of the breast to the size and shape desired by the patient is the most objective determination of the size of the woman's breasts and the corresponding determination of the size and shape of the implant.

An incorrectly chosen endoprosthesis, not only will it look unnatural, but health problems in the future are not excluded, ranging from back pain and the onset of fatigue, and even up to the appearance of numbness zones in certain areas of the chest.

The question always arises before specialists and patients - does the volume of implants affect the size of the breast. However, this is not quite true. Experts know that when implanting an endoprosthesis, it is necessary to take into account three main parameters: projection - height from the base; base width and volume.

The clinician needs to consider four main categories of profile, which is the ratio of projection to base width.

For example, low profile, medium profile, high profile and extra high profile (very voluminous chest).

The higher the height and the narrower the base, the higher the profile of the breast implant, which should be taken into account when choosing an endoprosthesis. The profile determines how far the breast will visually protrude after mammoplasty.

Only a specialist can determine what shape and size will suit a woman. In this case, factors such as the structure of the body, including the structure of the chest, are taken into account. These factors are important in mammoplasty for obtaining high aesthetics, maintaining health, reducing the risks of engraftment and side effects. In addition, it is important for the surgeon to take into account the volume of subcutaneous fat, sagging of the mammary glands, as well as the volume of muscle tissue. This is necessary, first of all, in order to minimize negative factors and increase the aesthetic and visual appearance.

At the same time, it is very important to remember that the endoprosthesis does not replace the own mammary gland, but complements it, and the size of the implant will give different results in patients with initially different, for example, bust sizes.

It is also necessary to take into account the fact that the size of the implant is affected by the elasticity of the skin and the volume of soft tissues, since insufficient stretching prevents the installation of an endoprosthesis of the desired size. Therefore, the correct selection of the size of implants will be of decisive importance.

To date, there are many ways to model the breast using various measurement methods necessary for the implantation of endoprosthesis of the relevant tissues and organs (mammary glands, muscle tissue).

For example, an article found at aif.ru/boostbookymammoplastika.html talks about methods of mammoplasty by lipofilling, which consists in breast augmentation by transplanting the patient's own fat cells. The main condition of this surgical intervention is concomitant liposuction. In slender patients, there is nowhere to take excess fatty tissue from. But the peculiarity of adipose tissue is low survival rate with strong compression, therefore, in one procedure, breast enlargement in this way can only be done by one size. Then, when the skin is sufficiently stretched and the injected tissue takes root, the procedure can be repeated. With this method, to begin with, markings are applied to the patient's body, which helps to introduce fat symmetrically. Then, using a special apparatus, liposuction is performed, after which, through pinpoint punctures, the taken fat is injected into the mammary gland and distributed as needed. The incisions in the area of liposuction are sealed or sutured. Punctures of the skin of the mammary glands are treated with antiseptics.

The disadvantage of this procedure is that the transplanted adipose tissue is partially absorbed, resulting in a loss of volume. In the chest, there may also be seals, depressions and bumps that require additional correction.

Implantation mammoplasty is also known, in which breast examination and measurements are also carried out to determine which implants are suitable for the patient. The disadvantages of this method are complications, but they can occur in case of improper selection of implants and unskilled work of the surgeon.

There is also a correction of breast asymmetry, which can occur as a result of spinal curvature, tumor removal, improper breastfeeding, hormonal disorders. The disadvantage of this method is that with a curvature of the spine and hormonal disruptions, breast asymmetry may return. Requires treatment of underlying diseases.

The most accurate way is 3D breast modeling. It is known that thanks to computer modeling of the breast, a woman can get an idea of how her breasts will look like with different types of implants, compare options and make a choice. 3D modeling allows you to try on implants of various sizes and shapes, which means making an informed choice. The result of such operations directly depends not only on the qualifications of the surgeon, but also on the type of implant chosen, its shape and size.

3D modeling of the mammary glands is a prediction of the result of an operation, followed by a demonstration on a computer of a realistic three-dimensional image of the body with various breast implants (both in shape and size). Such modeling is possible, for example, using the BodyNova program, which uses the development of 20 Mpi / 3D scanning and takes into account the individual features of the anatomy, soft tissues of the body during modeling, which makes it possible to predict the result of a future operation. This information is available on the Internet at https://grandmed.ru/plasticheskaya_hirurgiya/plastika_grudi/modelirovanie_grudi/3d_breast_model/.

There is a method (patent for invention 2480172) breast plastic surgery, which also refers to plastic surgery. The known method is carried out by performing the installation of the implant based on the calculation of the new position of the inframammary fold according to the formula LVC(F)=LVC(50-60)+ΔL. The length of the lower ventral curvature of the implant is measured from 50-60% of the height of the implant, depending on its shape. The thickness of the tissues in the lower pole of the chest is multiplied by π/2. The method makes it possible to determine the postoperative level of the inframammary fold, depending on the selected implant and the characteristics of the patient's tissues, and to improve the aesthetic results.

Known biomeasuring method of Professor Heden (Sweden), the so-called "Method Akademikliniken".

(1) Heden P: Breast augmentation with anatomical, high-cohesive silicon gel implant.

In: Spear SL, ed. Surgery of the breast∧principles and art, 2nd ed. Philadelphia: Lippincott Williams &Wilkins; 2006:1344-1367.

(2) Measurement and planning of augmentation mammoplasty using the Akademikliniken method. Guidelines for plastic surgeons. Allergan - Family Health. 2010

The method consists in the fact that the new distance from the nipple to the postoperative level of the inframammary fold is calculated as the sum of the value of the lower curvature of the implant and the thickness of the breast tissue in the lower pole. However, the calculations of Prof. Heden are not entirely accurate, since they assume that an increase in the circumference with an increase in radius occurs by an amount equal to the difference in the radii of the smaller and larger circles, which is incorrect. An error in calculations can lead to incomplete achievement of a good aesthetic result, manifested in the fact that the postoperative scar is located outside the inframammary fold and becomes noticeable. Also, such a mistake can lead to incorrect implant placement (excessively high or low in relation to the nipple-areolar complex).

A known method of RECOGNITION OF ANATOMICAL CHARACTERISTICS AND DIMENSIONAL ANALYSIS OF THE VOLUME OF THE BREAST TO ENSURE BREAST SURGERY (see patent for invention No. 2604710). The known method relates to diagnostic tools and the formation of a three-dimensional image of the upper body. The system comprises a three-dimensional imaging device, an associated computer analysis device, including one or more computing modules designed to determine an open continuous curve, to determine the parameters of a three-dimensional image of the chest wall from the three-dimensional image data, to determine a part of the chest wall from the data of the specified open curve and for determining the initial breast volume, and a display or reporting device associated with the computer analysis device. In the second embodiment of the system, the computer analysis device additionally contains one or more computing modules designed to determine the chest wall from the data of open curves, to determine the initial volume of one breast.

A known METHOD FOR FORMING INFRAMAMMAR FOLD IN TWO-STAGE BREAST RECONSTRUCTION, which refers to reconstructive surgery and includes marking the future inframammary fold with the patient in a vertical position. A skin incision and soft tissue dissection are performed up to the expander capsule, the capsule is dissected and the expander is removed, and the anterior leaf of the capsule is removed. The posterior leaf of the capsule is incised above the lower transitional fold of the expander pocket parallel to the preoperative marking of the formed inframammary fold. Dissection and mobilization of soft tissues downwards are performed until a mobile thoraco-epigastric flap of sufficient size is obtained, necessary for the formation of ptosis of the reconstructed mammary gland. The mobilized posterior leaf of the capsule is traction upward to the level corresponding to the preoperative marking of the formed inframammary fold, which entails simultaneous traction of the mobilized thoraco-epigastric flap adjacent to the mobilized posterior leaf of the capsule. The mobilized back leaf of the capsule is fixed to the remaining upper part of the back leaf of the capsule by suturing with non-absorbable suture material. The method allows to create a natural shape of the reconstructed breast with a pronounced inframammary fold, aesthetically acceptable ptosis, with no umbilical skin, and also allows the surgeon to observe the formed inframammary fold and the degree of ptosis before suturing, which prevents the need to re-suture (see patent for invention No. 2567790).

A known method for recognizing anatomical characteristics and dimensional analysis of breast volume for breast surgery (see application for invention No. 2014107796) by automatically determining breast volume based on three-dimensional contour image data of the upper body, containing: (a) a three-dimensional imaging device designed for forming said three-dimensional contour image of said surface of said upper body; (b) a computer analysis device associated with said three-dimensional imaging device and including one or more computing modules for determining the rate of change of contours of said three-dimensional image of said upper body; (c) wherein said computer analysis device further comprises one or more computational modules for determining an open continuous curve representing a submammary fold of a three-dimensional breast image from said three-dimensional image data by analyzing said rate of change of said contours of said three-dimensional image, wherein said an open curve representing the fold of the breast, passes over the surface of the specified three-dimensional image and defines the lower part of the chest; a display or reporting device associated with said computer analysis device and designed to display or provide reports containing data of said initial breast volume. However, this method is laborious and involves the use of several special-purpose devices.

There is a method (see patent for invention No. 2675088) for the selection of breast implants for augmentation mammoplasty, in which the parameters of the chest and mammary glands of the patient are measured, the width, height and volume of the implants are determined, followed by the selection of suitable implants from the catalogs of manufacturers. At the same time, the patient's height, weight, chest circumference at the level of the submammary folds, the distance from the middle of the nipple (SS) to the submammary fold (CMC) are measured. Then, the width and height of suitable implants are determined according to Table 1, given in the description, where OGK is the circumference of the chest in cm, W is the width of the implant in cm, H is the height of the implant in cm. with the arch length of implants that have already been previously selected in width and height from the catalog, and exclude those implants in which the arch length is more than or less than the length of the SS-CMC segment by more than 1.0 cm. In a particular case, after determining the width and height future implants, their parameters are refined using additional measurements of the width of the base and the height of the mammary glands. However, this method is less informative and gives a large error rate in the measurement and, accordingly, in the selection of the implant.

A known method of augmentation mammoplasty in combination with mastopexy (RF patent No. 2361526, A61V 17/00, bull. No. 20, 20.07.2009), which consists in marking anatomical formations by applying the axis of the mammary gland, the line of the submammary fold, the parasternal line, position of the second and third ribs, anterior axillary line, marks of points of pexia of the mammary gland, mastopexy and endoprosthesis. At the same time, the line of the submammary fold is additionally marked from the parasternal to the anterior axillary line below the mammary gland, a new place of the areola is noted as a projection of the finger on the anterior surface of the chest when it is installed in the submammary fold on the line of the axis of the mammary gland - point (A), (Fig. 1) retreat down from the lower edge of the areola by 2 cm and put a point (B), draw an oval connecting the point (A) and the point (B), which defines the outer boundaries of de-epidermization (S). Deep-epidermization is performed around the nipple-areolar complex, after which the skin and subcutaneous tissue are dissected along the periphery from the deep-epidermization zone, towards the upper squares to the level of the second rib, and along the width - to the medial and lateral sides by 3 cm and down by 3 cm ( R). Fixation sutures are applied passing through the upper edge of the deep-epidermised skin, the upper edge of the gland's own tissue and the fascia of the pectoral muscles, with fixation sequentially, namely, respectively, at point (G) - at the intersection of the line of the axis of the mammary gland and the second intercostal space, at point (H) and point (I) located 3 cm medial and lateral to point (G) along the second intercostal space; then, 2 cm away from the lower edge of the areola, along the axis of the mammary gland, the own tissue of the gland is cut up to 4 cm long (C-D), and deep to the retromammary tissue, a bed for the endoprosthesis is formed in the retromammary space, the boundaries of which are the lines: submammary fold (L-N) , anterior axillary (K-L), parasternal (M-N) and third rib projections. An endoprosthesis (R) is installed in the formed bed, the own breast tissue is sutured over the endoprosthesis with separate sutures, the skin around the areola is sutured with a purse-string intradermal suture. The method allows to reduce the trauma of surgery and improve the cosmetic effect of the treatment.

However, this method has a drawback: it does not allow the formation of a submammary fold with significant ptosis and tubularity of the mammary glands, and the retromammary location of the prosthesis increases the risk of capsular contracture, contouring of the prosthesis edges and early ptosis of the gland under the influence of the gravity of the implant and its own tissues.

There is a known method (patent for invention 2718660) of radical or prophylactic subcutaneous mastectomy with one-stage reconstruction of a ptotic mammary gland, in which, prior to surgery, a zone of periareolar reduction of excess skin of the mammary gland is marked on the patient's body. The skin in the reduction zone is de-epidermised around the areola. In the upper outer quadrant of the gland, along the outer border of the reduction zone, an arcuate access is performed, from which a subcutaneous mastectomy and implant installation are performed. The outer de-epidermised edge of the skin is connected to the edge of the areola with sutures. In a particular case, when de-epidermization of the skin in the reduction zone, physiological saline is used for its preliminary infiltration. EFFECT: method ensures the preservation of the nipple and areola during subcutaneous mastectomy with simultaneous reconstruction of the ptotic mammary gland, allows minimizing postoperative scars by placing them only along the border of the dark part of the areola, thereby achieving a good aesthetic effect.

A known method of three-dimensional modeling with breast augmentation. (Vorstenbosch, J. &Islur; correlation of prediction and actual result of 3D modeling in cloud-based breast augmentation. Aesth Plast Surg (2017) 41: 481. Received: 27 December 2016 / Accepted: 13 February 2017). Taken from the Internet at crisalix/com/ru/publications.

Received: 27 December 2016 / Accepted: 13 February 2017

Springer Science + Business Media New York and International Society of Aesthetic Plastic Surgery 2017.

The proposed method allows us to simplify, improve the accuracy of the selection of implants with subsequent improvement in aesthetics in the correction of the mammary glands. The objective of the invention is to improve the aesthetic result of breast plastic surgery by correctly calculating the position of the implant to accurately determine the postoperative level, depending on the selected implant and patient characteristics, and thereby reducing the number of postoperative complications and aesthetically unacceptable results.

The technical result of the task is achieved by the fact that after choosing the implant, depending on the anthropometric characteristics of the patient and taking into account her wishes, using the proposed method, the preoperative shape of the breast is calculated, followed by modeling of the implants.

The method works on the basis of an application, where the main platform is the iOS operating system and the additional platform is the Android operating system.

Before choosing an implant, the required parameters of the human body are measured from a photograph. Based on the measurement results, the application user orders breast implants.

The main platform of the application is iOS, the secondary platform is Android. Supported Versions: iOS 9.0 and above when using Apple TestFlight. It does not require publication in the AppStore. The working application runs entirely on the smartphone (locally). It is allowed to transfer to the network images that do not allow identifying a person or information from which it is possible to restore the original full image of a person. The application must provide access to the gallery of photos stored on the user's phone. At the same time, the application opens photos in all popular formats (jpg, png) and takes measurements, as well as saves their results to a photo file without changing the photo itself. When opening a photo with already existing measurement results, they should be displayed on the screen.

The measurement technique is carried out as follows. The reference distance is extracted from the photo using a neural network that determines an object in the photo (the iris of the human eye) with known dimensions, based on the known horizontal length of 12.5 mm, the division value of one pixel in millimeters is calculated. The allowable error in calculating the boundaries of the iris, set by the application, is 7%. With source code, the application is developed using the git versioning system. Repository address

https://gitlab.com/deepsystm/boobsmeasurement.git.

Any code transfer is carried out only through this repository. Access rights will be granted to all developers who need it. The interface and use of the application is possible both by the patient himself and by his attending physician. The logic of the work consists in the phased implementation of the following actions, which are reflected in the images (see Fig. 1-5).

In FIG. 1 shows the claimed method, Fig. 2-5 illustrate the steps of the claimed method in detail.

Screen 1 opens a file (see Fig. 2).

At the same time, we see that the list of previously opened files is displayed on the screen. Moreover, if the program is opened for the first time, then the New tab opens by default and there is a standard file selection component. If the Saved Items list is not empty, it opens first by default.

With a long tap on a photo in the list, you can select one or more photos and:

• share it (forward),

• delete,

• delete all (separate button).

If, when opening a photo, the program determines that it is not suitable for taking a measurement, a modal is displayed with text explaining what needs to be corrected in the photo so that it becomes valid. If a photo with data already saved is opened, screens 2-3 are skipped.

Next, go to screens 2 and 3, which allow you to adjust the boundaries of the iris of the eyes.

The screen displays an enlarged human left eye and a square of iris borders (RO) defined by a neural network (NN) with markers in each corner for more accurate definition of borders. It is necessary to offer text on the screen that will help the user quickly understand what is required of him (see Fig. 3). When the changes are made, the user presses the Next button and performs a similar operation on the right eye, then also presses Next and gets to screen No. 4 (see Fig. 3).

All screens (except screen #1) show an X button in the upper right corner. Pressing it closes the current screen and goes to Screen #1. If in the course of work the user has performed actions that can be saved, the program asks whether or not to save the measurements before closing. Then the data is sent to the server for training POST

https://flat.digital/telemetry/pmplastic/:payloadID

{payload:FILE}

payloadID - a unique photo hash generated on the program side.

Then we proceed to work on screen No. 4, on which we carry out the necessary measurements. When opening screen No. 4, a grid of dotted lines is displayed on the photo (see Fig. 4).

S1 - the number of pixels between the centers of the pupils

When working with screen No. 4, all lines are built to the full width and height of the screen. Wherein:

Line 1 - is built vertically relative to the screen through the center of an invisible segment connecting two pupils;

Line 2 and Line 3 - are built vertically relative to the screen at a distance S * 2 on opposite sides of Line 1;

Line 4 and Line 5 - are built at a distance of 7 cm (the corresponding number of pixels is calculated) on opposite sides of Line 1;

Line 6 (2 pcs.) - is built horizontally relative to the screen at a distance S1 * 6 from the center of an invisible segment connecting two pupils. The line is divided into two independent lines - the place of division is line 1. Made due to the possible vertical asymmetry of the chest in women;

Line 7 - is built horizontally relative to the screen at a distance S1 * 3 from the center of the invisible segment connecting the two pupils.

The photo of a person is aligned in the middle in proportions as in the screenshot below. Limits on the right and left edges are set in size S1 * 5 from the center line. The intersection of two lines in the measurement area is highlighted with a black dot. Only lines can be moved in the measurement area, points cannot be moved. Moving the line recalculates all dimensions.

The measurement area itself can be changed - when you click on it, markers appear that allow you to move the boundaries of the measurement area (if they are not set accurately by the NS). Lines are displayed as dotted lines when first opened. In order for the line to be active, the user needs to touch it anywhere and move it more precisely if necessary. An active line is displayed as a solid fill, an inactive line is displayed as a dotted fill. When clicking anywhere in the text field of the measurement results, all results should be copied to the clipboard with a pop-up message showing about it. When you click on the Save button, all measurement results are saved to a file, in the text field under the Save button, "Last save, just now" is written.

For a more accurate setting of points, a person’s photo can be enlarged / reduced (zoom). The maximum possible reduction of the photo in width is to place it within the borders of the screen, you do not need to zoom less. When the photo is zoomed, the lines also scale with the photo, but remain of an acceptable thickness (thin) to move them.

Parts of the photo that are outside the measurement area should be darkened (10-20% transparency, slight darkening). Measurements are carried out (measured segments in pixels) according to the following formulas:

The display of the value of the measured segments is carried out in centimeters. The measurement is recalculated in real time as the user moves the line.

The final stage is the testing of the functionality, which is carried out on the basis of a set of 10 photographs, from which the measurement results are already known. The measurement result obtained using the developed application is compared with the results of a real measurement. Based on the results of the comparison, a conclusion is made about the performance of the application.

Thus, the proposed method allows us to most accurately carry out the necessary measurements and create an implant model that fully corresponds to the parameters chosen by the patient and, at the same time, recreates a complete imitation of the natural mammary gland during corrective intervention.

Claims (1)

A method for modeling mammary glands using a neural network, which includes taking the necessary measurements of human body parameters from a photograph using an iOS application, on the basis of which the application user orders breast implants that meet the parameters required by the patient and restore the natural appearance of the mammary glands of the desired shape and size, excluding at the same time, identification of the patient or information about him, which allows to restore the original image of the patient, and the reference distance is extracted from the photo loaded into the application using a neural network that determines the object in the photo - the iris of the human eye, displayed on the screen and a certain neural network with archers in the corners of the iris, at the same time, the photo of a person is aligned on the screen of the working device with this application, and the measurement of the necessary parameters is carried out and recalculated in real time.

Images