BR102024004817A2 - METHOD AND SYSTEM FOR GENERATING A MODEL FOR USE IN A VIRTUAL EXTRACTION PROCEDURE OF A TARGETED EXTRACTION OBJECT IN A PATIENT - Google Patents

Abstract

A computer-implemented method for automatically generating a three-dimensional model for use in a virtual extraction of a targeted extraction object from a patient, the extraction object comprising an object in the anatomy of a patient, the method being based on a surface scan of an anatomical region of the patient's oral cavity and a volumetric density scan of the anatomical region, having the steps of receiving a first data set of labeled surface scan segments, receiving a second data set of labeled volumetric density scan segments, cross-assembling them into a common 3D coordinate system, receiving identification of the targeted extraction object, identifying the volumetric density 3D model and the surface 3D model associated with the volumetric density scan segment label that corresponds to the identified targeted extraction object, and generating a third data set comprising a 3D model of a socket, a 3D model of portions of a dental crown, a 3D model of portions of a targeted extraction tooth, and/or a 3D model of the extraction contour.

Description

[001] Digital software tools for anatomical treatment planning and/or prosthetic restoration design rely on accurate scan data of anatomical objects of interest, along with neighboring anatomical structures such as bones, tissues, nerve tracts, organs, and other anatomical structures/objects/features, as well as artificial structures such as surgical implants, prosthetic abutments, other anchorage systems, artificial analogues of natural structures, among other natural and artificially placed anatomical structures.

[002] In the field of digital dental technology, the scan data of anatomical structures are generally determined optically or radiologically. Optical digitizers for three-dimensional measurement directly from intraoral surface structures or from extraoral impressions of oral surface structures are widespread and cost-effective. Typically, the surface data are represented by means of a surface mesh comprising triangular elements, which are routinely saved and exchanged between systems in STL or similar surface definition formats.

[003] Radiological scanners, such as digital volume tomographs (DVTs) or computed tomography (CT) scanners, use X-rays to generate volume data sets of anatomical structures. Surface representations can also be determined from these data sets by applying thresholding methods in which the intensity values (measured in Hounsfield units) of individual scan elements (voxels) are analyzed to determine whether they exceed or fall below certain thresholds. Radiologically dense structures (hereinafter referred to as “volumetric density” structures or objects), such as teeth, can be identified in this way and the boundary surface of the identified volumetric density scan structures can be modeled as triangulated surface data (e.g., a triangular surface mesh) and again saved and exchanged between systems in STL or similar surface definition formats.

[004] Computations with MRI scan data are more complex, in which contour analysis methods are preferably applied, which work on the basis of the gradients of adjacent scan elements. However, also in these cases, volumetric structures or objects such as masticatory mucosa, gingiva and other predominantly soft tissue tissues can be identified and their boundary surface modeled as triangulated surface data for further processing.

[005] With the described analysis methods, triangulated surface data of desired surfaces, such as, for example, visible surfaces, or boundary surfaces, such as, for example, surfaces between objects or other invisible surfaces between anatomical objects and/or structures in the scan data can be obtained for all modalities (imaging devices). In the context of the present invention, the volumetric density scan data can, for example, comprise all of the above-mentioned data types, regardless of how they were obtained, but also other algorithmically and/or interactively determined structures, such as preparation lines, segmentation lines and anatomical landmarks, as well as desired contact points for nodes in virtual extraction procedures, virtual dental restoration procedures, virtual prosthesis design and placement procedures, etc.

[006] Optical surface scan data is typically preferred for the evaluation or computation of virtual dental restorations due to the ability to achieve high accuracy without exposing the patient to high doses of radiation.

[007] In the context of the present description, the term dental restoration parts includes any type of object that can be manufactured for the treatment of esthetics and/or dental defects. Examples include intracoronal restorations, extracoronal restorations, partial crowns, crowns, telescopic crowns, bridges, veneers, implant prosthetic abutments, partial dentures and prostheses. Furthermore, in the context of the present description, for the sake of simplicity, dental replacement parts are also covered by the term dental restoration parts. The term virtual dental restoration (hereinafter more briefly referred to as dental restoration) is to be understood as including appropriate electronic representations of dental restorations, i.e. digital three-dimensional representations, preferably surface representations, of such dental restoration parts with sufficient accuracy. For example, in order to manufacture a dental restoration part, a CAD/CAM data set of the corresponding virtual dental restoration can be forwarded to a manufacturing machine.

[008] When treating a dental defect, the teeth are usually prepared, i.e. cavities, old filling material or defective tooth parts are removed. What remains for each tooth is the remaining tooth structure, the surface of which is divided into a prepared part (cavity) and an unprepared part. The boundary line between the unprepared tooth surface and the prepared tooth surface is called the preparation line. In addition to the preparation lines, which are present only in the case of prepared teeth, each tooth also has at least one segmentation line that separates a tooth from extra-odontological structures such as masticatory mucosa and/or bone and/or adjacent teeth. It follows from these definitions that segmentation lines and/or preparation lines delimit unprepared tooth surfaces. In the context of the present description, such boundary lines could be placed on the scan data, predominantly in an interactive manner, through the use of a graphical user interface, through which algorithmic assistance, based on the analysis of the surface curvatures of the scan data, is typically used in their determination.

[009] Because of the complex structure of the scan data and the multitude of functional aesthetic criteria that must be met—fit to opposing dentition, possibly taking into account jaw movement, fit to adjacent teeth taking into account contact points, fit to preparation lines for the purpose of achieving an ideal margin fit, compliance with minimum material strengths for the purpose of achieving satisfactory mechanical stability, consideration of desired tooth shapes in the anterior region of the tooth, etc.—the process of designing a custom dental restoration, even with the assistance of an interactive computer-aided design tool, requires a human with knowledge and experience in dental restoration design, as well as training in the use of such tools.

[0010] Adding to the complexity of designing prostheses for clients, prostheses are often designed to replace an anatomical object that has been removed from the existing oral situation. For example, the designer may be tasked with designing a temporary or permanent prosthesis to be placed on a prosthetic abutment attached to an implant that is placed in the socket of an extracted object. A designer can achieve better accuracy by basing the design of the prosthesis on the patient's actual anatomical contours in the socket left where the extracted object was previously seated. Thus, often after the extraction of the anatomical object, the area had to be optically rescanned before sending the surface model to the designer.

[0011] There is a growing demand for more immediacy in dental treatments. In particular, for procedures that can be completed in one or fewer office visits than would have been possible in the past, innovative ways to shorten the pre-planning of dental treatment are being sought.

[0012] In the situation where an anatomical object is targeted for extraction, methodologies are sought to pre-plan the design of prostheses such that they can be ready to be installed in immediate position in place of the extracted anatomical objects in the patient's oral cavity. For example, it may be desirable to pre-plan the placement of an implant and pre-plan the design of a temporary prosthetic abutment such that immediately after tooth extraction (i.e., during the same office visit), an implant can be placed and a prosthetic abutment installed. It would also be desirable for the prosthetic abutment to be designed and manufactured, e.g., by additive or subtractive manufacturing, e.g., 3D printing or milling, respectively, and installed in the patient's mouth during the same office visit to accommodate urgent situations as much as possible.

[0013] Ideally, a prosthesis should be designed to fit and conform to the contours of the socket created after the extraction of the anatomical object. However, modern prosthesis design tools, such as CAD/CAM systems, only receive surface data (represented as a 3D triangular mesh, typically in STL format) and therefore lose information about any hidden structures, such as bone or other internal structures and contours. Consequently, there is no information available in prosthesis design tools regarding the socket profile of the extracted anatomical object (or objects), but designers typically estimate the socket profile of the extracted anatomical object when designing the prosthesis to fit in its place. This can lead to poor fit or misalignment of the so-designed prostheses when placed in the patient's oral cavity and can, in turn, lead to patient discomfort or other treatment-related problems.

[0014] It is an object of the present invention to alleviate at least some of the disadvantages of known procedures for designing and placing dental restorations or prostheses. In particular, it is an object of the present invention to provide methods and systems that help improve the pre-planning of dental restoration or prosthesis procedures using CAD/CAM tools.

[0015] The object is solved by means of a computer-implemented method for automatically generating a three-dimensional model for use in a virtual extraction of a targeted extraction object from a patient according to independent claim 1, as well as a system for automatically generating a three-dimensional model for use in a virtual extraction of a targeted extraction object from a patient according to independent claim 14.

[0016] Dependent claims 2 to 13 represent embodiments of the invention.

[0017] According to the invention, a computer-implemented method automatically generates, by means of one or more computer processors, a three-dimensional model for use in a virtual extraction of a targeted extraction object from a patient, the targeted extraction object comprising an object in the anatomy of a patient. The method is based on a surface scan of an anatomical region of the patient's oral cavity, as well as a volumetric density scan of the same or similar anatomical region, wherein the anatomical region includes at least the targeted extraction object. The method comprises: receiving a first set of labeled surface scan segment data, each surface scan segment comprising a three-dimensional (3D) surface model of a corresponding object recognized and segmented from surface scan data of the surface scan, and each surface scan segment having an associated label identifying the surface scan segment, and receiving a second set of labeled volumetric density scan segment data, each volumetric density scan segment comprising a three-dimensional (3D) volumetric density model of a boundary surface of a corresponding object recognized and segmented from volumetric density scan data of the volumetric density scan, and each volumetric density scan segment having an associated label identifying the volumetric density scan segment.

[0018] Recognition and segmentation of objects in the surface scan data of the surface scan and/or in the volumetric density scan data of the volumetric density scan may be based on or assisted by means of automatic data analysis, such as, for example, analysis by means of a suitably trained neural network capable of distinguishing objects and anatomical structures from one another in the surface scan data and/or in the volumetric density scan data.

[0019] In one embodiment, the associated labels of each surface scan segment and/or each volumetric density scan segment identify the respective surface scan segments and the respective volumetric density scan segments as corresponding to a tooth, a portion of a tooth, a prosthetic tooth, a portion of masticatory mucosa tissue, a portion of bone, an implant, or another object or structure present in the oral cavity and/or anatomical region of the patient.

[0020] In the invention, labeled surface scan segments from the first data set are cross-assembled in a common 3D coordinate system to labeled volumetric density scan segments from the second data set. In one embodiment, cross-assembly may be implemented by using correspondingly labeled surface scan segments and volumetric density scan segments. Alternatively, or in addition, cross-assembly may be implemented by determining surface scan segments and volumetric density scan segments of similar shape and volume.

[0021] In one embodiment, cross-shape assembly may include scaling, translational, and rotational transformations applied to the surface scan segments and/or volumetric density scan segments, for the purpose of achieving the best possible registration of the surface scan segments and volumetric density scan segments in the common 3D coordinate system. In one embodiment, cross-shape assembly parameters, such as scaling factors, translational and/or rotational transformation parameters for up to 3 spatial dimensions may be stored and used for the purpose of "overlaying" separately maintained labeled surface scan segments and labeled volumetric density scan segments, or portions thereof, when necessary, in the following procedure.

[0022] In one embodiment, cross-assembling includes generating a fifth dataset of labeled segments, each segment comprising a combined three-dimensional (3D) model representing an anatomical object, wherein the combined three-dimensional (3D) model includes labeled segments from both the labeled surface scan segments and the labeled volumetric density scan segments. In some applications, portions of the labeled surface scan segments and/or the labeled volumetric density scan segments may, however, be removed from the combined three-dimensional (3D) model representing an anatomical object.

[0023] For purposes of the following description, where co-represented portions of the identified volumetric density 3D model and the identified surface 3D model are being determined, this determination may be based on the operation of cross-assembling labeled surface scan segments from the first data set and labeled volumetric density scan segments from the second data set, whereby an association of the labeled surface scan segments from the first data set and labeled volumetric density scan segments from the second data set may be established and the association information stored in a transient or non-transient storage component. Alternatively, determination of co-represented portions in the identified volumetric density 3D model and the identified surface 3D model may also be determined in a separate operation, wherein the association of correspondingly labeled surface scan segments from the first data set and volumetric density scan segments from the second data set with each other is established, and optionally the association information is stored in a transient or non-transient storage component.

[0024] In the invention, one or more computer processors receive the targeted extraction object. This includes receiving a selection of a targeted extraction object that may be manually selected, or selection assisted by a suitably trained neural network capable of identifying objects likely to be targeted for extraction, such as, for example, damaged or modified objects, anatomical location or orientation of objects, or unhealthy objects, in the surface scan data and/or in the volumetric density scan data.

[0025] In one embodiment, the targeted extraction object is a tooth or a portion of a tooth, such as, for example, the root of the tooth.

[0026] In the invention, both the 3D volumetric density model associated with the volumetric density scan segment label as well as the 3D surface model associated with the surface scan segment label that correspond to the targeted extraction object are identified.

[0027] The method of the invention may include generating a third data set comprising a model comprising a 3D model of a socket being an equivalent of the identified volumetric density 3D model minus portions of the identified volumetric density 3D model co-represented in the identified surface 3D model. In one embodiment, the 3D model of a socket (socket model) comprises a three-dimensional model of the dental socket of a patient seating the target extraction object.

[0028] For the purposes of this description, it is assumed that the 3D model of an alveolus substantially corresponds to portions of the identified volumetric density 3D model.

[0029] In one embodiment, the 3D model of a socket is generated by determining portions of the identified volumetric density 3D model not co-represented in the identified surface 3D model. This means that in a case where the targeted extraction object is a tooth, the socket model is derived by determining those portions of the identified volumetric density 3D model that do not have any identified surface 3D model co-represented, which would generally include at least a portion of the root of the tooth.

[0030] In an alternative embodiment, the 3D model of a socket is generated by determining portions of the identified volumetric density 3D model co-represented in the identified surface 3D model, and determining a difference between the determined co-represented portions and the volumetric density 3D model by removing the determined co-represented portions from the volumetric density 3D model.

[0031] In one embodiment, the determined portions of the identified volumetric density 3D model are stored in a computer-readable data storage component, which may be a transient or non-transient storage component, including network storage components.

[0032] The method of the invention may further include generating a third data set comprising a model comprising at least a portion of the identified volumetric density 3D model co-represented in the identified surface 3D model and/or at least a portion of the identified surface 3D model co-represented in the identified volumetric density 3D model. The third data set will thus include a representation of at least parts of the visible portions of the targeted extraction object, such as, for example, a tooth crown or a prosthesis.

[0033] The method of the invention may further include generating a third data set comprising a model comprising at least a portion of the identified volumetric density 3D model minus portions of the identified volumetric density 3D model co-represented in the identified surface 3D model. The third data set will thus include a representation of at least parts of the invisible portions of the targeted extraction object, such as, for example, a tooth root, an implant and/or a prosthetic abutment.

[0034] The method of the invention may further include generating a third data set comprising a model comprising at least a subset of the first data set minus portions of the identified 3D surface model co-represented in the identified volumetric density 3D model.

[0035] In one embodiment, the method may include generating a fourth data set by removing portions of the identified 3D surface model co-represented in the identified 3D volumetric density model from a copy of at least a subset of the first data set. The third data set will thus include a representation of the visible portions of at least a portion of the patient's oral cavity without the visible portions of the targeted extraction object as defined by the identified 3D surface model.

[0036] The third data set generated by the invention thus includes a representation of at least parts of the patient's anatomical region, including or excluding a representation of the targeted extraction object. This helps to improve pre-planning of dental prosthetic or restorative procedures using CAD/CAM tools as used by the prosthesis designer and can result in improved fit and alignment of the prosthesis within the patient's oral cavity. Additionally, the information provided in the third data set is suitable for improving the design of the prosthesis such that it can be ready to be immediately installed in place of the extracted anatomical objects in the patient's oral cavity, by potentially having an implant placed, a temporary custom procedure or permanent prosthetic abutment installed, and a prosthesis fitted during one and the same office visit.

[0037] In one embodiment, the method comprises determining a boundary of the identified 3D surface model and generating a cut line from the determined boundary. This may include smoothing, straightening, or smoothing the boundary of the identified 3D surface model. It may also include determining an intersection curve between the surface definition of the identified 3D surface model and the boundary surface of the 3D volumetric density model. The cut line thus defined is then projected onto the identified 3D volumetric density model and may be used to generate the sixth data set comprising the identified 3D surface model and/or the identified 3D volumetric density model on the side of the cut line that includes substantially all portions of the identified 3D volumetric density model that are co-represented in the identified 3D surface model. The sixth data set may thus include a representation of at least parts of the visible portions of the targeted extraction object having an enhanced boundary representing essentially the line of demarcation between the visible portion of the targeted extraction object in the patient and the invisible portions thereof.

[0038] In one embodiment, the method further comprises augmenting any of the first, second, third, fourth, fifth, and/or sixth datasets, or the identified 3D surface model and/or the identified 3D volumetric density model when there is a void in a surface defined by the dataset or model, respectively, and thereby filling such void with generated surface information. The generated surface information may be generated by interpolating the boundaries of the void and/or based at least in part on standard anatomical modeling, which may be determined by means of a suitably trained neural network capable of generating corresponding sections of anatomical objects and structures. The generated surface information can thus provide a constant and continuous progression of the surface information existing in the data set (or sets) or model (or models), respectively, and, at best, represent an authentic representation of the surfaces or boundary surfaces of the patient's anatomical objects or structures at positions where the data set (or sets) or model (or models) are incomplete.

[0039] In one embodiment, the method comprises storing any of the third, fourth, fifth, sixth, and/or seventh data sets, and/or merged or cross-assembled versions thereof, in a computer-readable data storage component, which may be a transient or non-transient storage component, including a networked storage component. The information may be stored in data exchange formats or surface definition formats such as STL for import and use by the CAD/CAM system in the process of designing a prosthesis.

[0040] The invention also relates to a system for automatically generating a three-dimensional model for use in a virtual extraction of a targeted extraction object from a patient, wherein the system comprises one or more computer processing units configured to load and execute computer-readable instructions that, when executed, configure one or more computer processing units to implement the method of the invention and its embodiments as described hereinbefore.

[0041] In one embodiment of the system of the invention, the system includes an image creation application and an electronic display apparatus for displaying graphical representations of at least any one or more of the first, second, third, fourth, fifth, sixth, and/or seventh datasets and/or merged or cross-assembled versions, or the identified 3D surface model and/or the identified 3D volumetric density model to the user for visualization and modification.

[0042] The imaging application, in combination with the electronic display apparatus, is configured to automatically update the display of the graphical representations of the first, second, third, fourth, fifth, sixth, and/or seventh data sets and/or merged or cross-assembled versions thereof, or the identified 3D surface model and/or the identified 3D volumetric density model whenever such data sets and/or models are updated and information is added to or removed from them.

[0043] In addition to displaying the graphical representations of the first, second, third, fourth, fifth, sixth, and/or seventh data sets and/or merged or cross-assembled versions thereof, or the identified 3D surface model and/or the identified 3D volumetric density model, the imaging application also provides display controls responsive to user input for modifying the display of the graphical representation of the graphical representations of the first, second, third, fourth, fifth, sixth, and/or seventh data sets and/or merged or cross-assembled versions thereof, or the identified 3D surface model and/or the identified 3D volumetric density model, such as, for example, to allow modification of the viewing angle, rotation, or tilt of the displayed information.

[0044] In addition to the display controls, the imaging application also provides selection controls that, upon activation, allow the user to select at least one of the objects graphically represented in the display. The imaging application is enabled, upon receiving a selection operation using the selection controls by the user of the imaging application, to highlight labeled surface scan image segments and/or volumetric density scan image segments in the graphical representation of the first, second, third, fourth, fifth, sixth, and/or seventh data sets and/or merged or cross-assembled versions thereof, or the identified surface 3D model and/or the identified volumetric density 3D model, corresponding to the user's selection on the electronic display apparatus.

[0045] The image creation application additionally provides a virtual extraction control which, upon activation, performing for each graphically represented object selected by the user using the selection, controls the possibility of removing the graphical representation of the targeted extraction object from the graphical representation displayed on the electronic display device and supplements it with the graphical representation of the alveolus model.

[0046] The invention also relates to a computer-readable medium including computer-readable instructions that, when executed by means of one or more computer processing units, configure the one or more computer processing units to implement at least one component of a system for automatically generating a three-dimensional model for use in a virtual extraction of a targeted extraction object from a patient.

[0047] The invention also relates to a computer program, said computer program embodying computer-readable instructions which, when executed by means of one or more computer processing units, configure the one or more computer processing units to implement at least one component of a system for automatically generating a three-dimensional model for use in a virtual extraction of a targeted extraction object from a patient.

[0048] The invention also relates to the use of the third, fourth, fifth, sixth and/or seventh data sets and/or merged or cross-assembled versions thereof, in a virtual extraction procedure of a targeted extraction object from a patient.

[0049] The invention also relates to third, fourth, fifth, sixth and/or seventh data sets, and/or merged or cross-assembled versions thereof, in a prosthetic design CAD/CAM software tool for generating a prosthetic design that anatomically represents a patient's targeted extraction object. Such use may be prior to and/or during an office visit, preferably a single office visit, of a patient on whom an actual extraction procedure of the targeted extraction object is performed.

[0050] These uses allow for the reduction of the number of patient visits from potentially many to just a single visit by providing the ability to conduct accurate pre-planning of the extraction procedure, as well as design and manufacture anatomically correct prostheses and, as appropriate, prosthetic abutments either prior to or during the procedure. At the time of extraction of the targeted extraction object from the patient's anatomy, installation of the prosthetic abutment/prosthesis can be conducted in situ and immediately following extraction of the targeted extraction object.

[0051] The disclosure provides devices, systems, graphical user interfaces, computer-enabled tools, methods and processes for generating, displaying, exporting/saving and using anatomically accurate virtual 3D models of an anatomical cavity or alveolus (hereinafter referred to as "alveolus") configured to conformably seat an object (such as a tooth or implant) in an oral cavity of a mammal (human or other, hereinafter referred to as "patient").

[0052] A virtual 3D model, as used in the context of the description, is a three-dimensional digital representation of a physical object (or socket). Virtual 3D models are often represented as point clouds or triangular (or other polygonal) meshes, as commonly known and used in the field of computer graphics. A virtual 3D model of a socket (or simply "socket model") implemented in accordance with the described embodiments is a virtual 3D model that represents in a sufficiently accurate manner the anatomy of a real socket of a patient that seats an associated object, and is determined and generated based on a surface scan of an anatomical region of the patient's anatomy (such as the oral cavity) and a volumetric density scan of the anatomical region.

[0053] In both surface scanning and volumetric density scanning, the scanned anatomical region includes at least the object seated in the actual socket. In some use cases, one purpose behind generating the socket model is that the actual object seated in the patient's actual socket is targeted for extraction (hereinafter, “targeted extraction object”). For example, the object may comprise a tooth (hereinafter, “targeted extraction tooth”) or an implant (hereinafter, “targeted extraction implant”) targeted for extraction.

[0054] Having access to, and corresponding understanding of, a virtual socket model specific to the patient's situation may enhance patient treatment planning, and in particular diagnosis and treatment planning related to the patient's anatomy. It may enhance, for example, dental treatment planning, including treatment planning involving the extraction of a tooth, placement of an implant, a prosthetic abutment, and/or a prosthesis such as an artificial crown. It may further enhance treatment workflow by reducing the number of steps and patient visits required to design an anatomically accurate prosthesis, such as, for example, a temporary healing prosthetic abutment or a permanent prosthetic abutment.

[0055] Each socket model generated as described herein substantially conforms to the contours of the extrinsic surface of the actual object it sits upon in the patient's anatomy. For example, a tooth socket model according to embodiments of the invention is generated to conform to the contours of a patient's actual tooth root-based volumetric density scan data (via CBCT scanning, for example) of the anatomical region (including the tooth of interest) of interest.

[0056] The invention is further described by way of the following detailed description in combination with the drawing. In the following detailed description, like reference numerals denote like elements, unless otherwise indicated.

[0057] Although the following detailed description describes certain embodiments of the invention in greater detail, it is noted that features described in the context of only one of the embodiments are nevertheless intended to be available also in the context of or in combination with any other embodiment of the invention, whether or not described in the detailed description, except where such a combination of features would lead to non-significant results. In no way is the following detailed description intended to limit the invention to the specific embodiments and combinations of features thereof; rather, the invention is exclusively limited and defined by the appended claims.

[0058] In the drawing, Figure 1 is a flow diagram outlining a first aspect of the method according to an embodiment of the present disclosure; Figure 2 is an architectural diagram of the system according to an embodiment of the present disclosure; Figures 3A to 3N illustrate an exemplary embodiment according to the present disclosure of the method of generating a socket model according to the invention; Figure 4 is a flow diagram outlining a second aspect of the method according to an embodiment of the present disclosure; Figures 5A to 5D depict a first exemplary graphical user interface implementation of various aspects of the present disclosure; Figure 6 is a flow diagram outlining a third aspect of the method according to an embodiment of the present disclosure; Figures 7A to 7J depict a second exemplary graphical user interface implementation of various aspects of the present disclosure; Figure 8 is a flowchart outlining a fourth aspect of the method according to an embodiment of the present disclosure; Figures 9A through 9T depict a second exemplary graphical user interface implementation of various aspects of the present disclosure.

[0059] One aspect of the invention is the generation of a three-dimensional virtual socket model that sufficiently accurately models an anatomical socket by seating an object in an anatomical region within the patient's oral cavity region. One embodiment comprises a method for generating such a socket model.

[0060] Figure 1 is a flowchart of a method 100, being executed on a computer-enabled device, such as the system 200 of Figure 2, for generating a 3D socket model that accurately represents the anatomy of an actual socket in the patient's anatomy that seats an associated object, such as a tooth. The method 100 includes a step 101 of receiving each of a surface scan and a volumetric density scan (both stored in scan data 220 of the system 200) of an anatomical area of the patient's oral cavity, wherein the scanned anatomical region includes at least the targeted object, in this example in the form of at least a portion of the actual object sitting in the patient's actual socket.

[0061] The method 100 further includes a step 102 of receiving a first set of labeled surface scan segment data (or segments) and a second set of labeled volumetric density scan segment data (or segments) (both stored in segment data 222), comprising individual segments corresponding to recognized objects and/or features, via image recognition and segmentation processing on the surface scan and the volumetric density scan of the anatomical region of interest. Steps 101 and 102 may occur outside of the execution of the application 215a, for example, via a remote service 212.

[0062] For example, surface scan data 220 may be generated via a remote surface scan service 240. A surface scan service 240 may do any or any combination of the following: collecting surface scan data collected from a surface digitizer (such as an optical digitizer or camera), converting the surface scan data into a three-dimensional model that can be accessed and read by the processor(s) 201 of the system 200, manipulated or converted into another format if necessary to prepare it for display on an electronic display 209, and saving it to a computer-readable file that can be received by the system 200 and stored in local memory 220 reserved for scan data.

[0063] As another example of a remote service, volumetric density scan data 220 may be generated via a remote volumetric density scanning service 240. A volumetric density scanning service 240 may do any or any combination of the following: collecting volumetric density scan data collected from a volumetric density scanner (using volumetric scanning equipment, such as a cone-beam computer tomography (CBCT) scanner), converting the volumetric density scan data into a three-dimensional model that can be accessed and read by the processor(s) 201 of the system 200, manipulated or converted to another format if necessary to prepare it for display on an electronic display 209, and saving it to a computer-readable file that can be received by the system 200 and stored in local memory 220 reserved for scan data.

[0064] Another remote service 212 may include a segmentation service 244. The segmentation service receives scan data (surface scan (IOS) or volumetric density scan (CBCT or CT) data) and uses image processing, extraction, and classification to segment received scan images containing the anatomical area of interest into multiple identified objects and associating a classification marker with each segment. The segmentation service 244 may provide the segments in the form of individual virtual segment 3D models, wherein each individual virtual segment 3D model (hereinafter also referred to simply as a "segment model") is a digital 3D representation of the actual anatomical part or feature of the patient's anatomy. In one embodiment, the segmentation service 244 provides each segment as an individual digital 3D model, preferably (but not limited to) in STL format as a triangular mesh or a point cloud.

[0065] Each of the remote services 212 returns data to the system 200 via the network adapter(s), where it is stored in the appropriate mass storage 206 (scan data 200 or segment data 222).

[0066] One or both of steps 101 and 102 may be performed outside of the execution of application 215a, for example, by another process or application stored in local memory 204 and executed by local processor (or local processors) 201 of system 200. In other embodiments, one or both of steps 101 and 102 may alternatively, or also, be performed as an integral execution process within application 215a.

[0067] Because the received surface scan segments and the volumetric density scan segments are generally obtained using different modalities (and therefore different (and typically independent) scanning machines/equipment), the scan data produced by each scanning modality is collected and saved according to a 3D coordinate system native to the particular scanning machine/equipment that collected the data. In these situations, it becomes important to align the resulting scan data from each scanner into a common 3D coordinate system such that similar objects from each scan are combined and displayed as occupying the same space (which they should, since each represents the same object).

[0068] The surface scan segments (labeled) and the volumetric density scan segments (labeled) are cross-assembled into a common 3D coordinate system (step 103). The result of cross-assembling into a common 3D coordinate system is that for each pair of surface scan and volumetric density scan segments that correspond to the same patient object of the scanned anatomical area of interest, there must exist one or a plurality of respective points that “match” (i.e., the points originating from each segment in the segment pair must substantially or exactly match) in the 3D space of the common 3D coordinate system. These matching points correspond to a corresponding point on the actual object/feature of the patient's actual anatomy. The matching points will only be present for areas of the patient's anatomy that the particular scanning modality was able to capture.

[0069] Accordingly, since the surface scan data 220 includes only visible surface image data and the volumetric scan data 220 includes both surface and subsurface image data, point matching can only occur for points of the volumetric density scan segment (or segments) that correspond to visible points on the surface of the scanned object (since the surface scan segment (or segments) does not contain subsurface data points). It will be appreciated that, although ideally the points of each corresponding (or co-represented) pair of surface scan segment and associated volumetric density scan segment should match exactly in the common 3D coordinate system, the points may only “substantially match,” meaning match within a margin of error, due to differences in accuracy between the scanning embodiments, as well as differences in the resolution and generation accuracy of the scan segment 3D models generated for each scanning embodiment.

[0070] However, cross-assembly should result in the surface scan segments corresponding to (or co-represented in) scanned objects occupying nearly identical space in the 3D coordinate system as similar regions of the associated object as represented via the corresponding volumetric density scan segments.

[0071] At a step 104, a selection indicating one or more targeted extraction objects is received.

[0072] The method further includes a step 106 of generating, for each targeted extraction object, a 3D model of a socket that conforms to the extrinsic surface boundary of the portion of the targeted extraction object that is seated within the patient's actual socket. This involves identifying the 3D volumetric density model associated with the label of the volumetric density scan segment corresponding to the identified targeted extraction object and identifying the 3D surface model associated with the label of the surface scan segment corresponding to the identified targeted extraction object, and then generating—for each respective targeted extraction object—a 3D model of the socket based on the corresponding (or co-represented) identified surface scan segment model and the identified volumetric density scan segment model.

[0073] In one embodiment, step 106 is implemented via a step 106a of determining co-represented portions of the identified volumetric density model that do not occupy substantially the same 3D space as the identified surface model, and a step 106b of configuring the portions determined in step 106a as the 3D model of the socket. In a step 107, the generated socket model(s) may then be saved to a file and/or exported to a saved file for later use or distribution.

[0074] Referring to Figure 2, method 100 is executed on a computer-enabled system 200 and comprises computer-readable instructions stored in computer-readable memory 202 that, when executed by a computer processor 201, perform the steps of the method. In embodiments, the method is implemented within a computer-implemented application 215a (or “tool”) as computer-executable instructions stored in memory 202 of system 200, or is accessible via a remote service, through a device-enabled network adapter 211 of system 200.

[0075] Application 215a includes computer-readable instructions that, when executed by means of a local or remote computer processor (not shown, but executing services 112), perform the steps of method 100. Application 215a may operate in connection with an electronic display 209. In one embodiment, application 215a includes a graphical user interface (GUI) that is displayed and presented on electronic display 209.

[0076] The GUI may include a 3D model viewing pane for displaying a three-dimensional model of the patient's anatomical area of interest. The model displayed in the viewing pane is based on a surface scan and a volumetric density scan of the patient's area of interest. The GUI includes various user controls to allow the user to direct the application to perform various operations, such as, but not limited to: selecting and loading scan records of a patient, selecting and manipulating display views, selecting content for display in the GUI, selecting objects of interest and/or identifiers of objects of interest that may be included in a patient's scan data, selecting and viewing identifiers, descriptions, and images of implants, prostheses, materials, etc. in connection with planning dental treatment for a patient, and so forth.

[0077] One or more such controls comprise an object selection tool that allows the user to select an object that is present in the patient's scan data as a targeted extraction object. One or more such controls comprise a socket model generation tool that generates an independent 3D model of the socket in which the targeted extraction object is seated. Alternatively, or in addition, one or more such controls comprise a virtual tooth extraction tool that, when activated by a user, operates to extract the selected object from the area of interest.

[0078] Alternatively, or in addition, one or more such controls comprise an extraction tool that generates a data set comprising a model of at least a portion of an exposed object (such as the crown of a tooth) based on at least a portion of an identified volumetric density 3D model, volumetric density 3D model co-represented in an identified surface 3D model, and/or at least a portion of an identified surface 3D model co-represented in an identified volumetric density 3D model.

[0079] Alternatively, or in addition, one or more of these controls comprises a further extraction tool that generates a data set comprising a model of at least a portion of a hidden object (such as the root of a tooth) based on at least a portion of an identified 3D volumetric density model minus portions of the identified 3D volumetric density model co-represented in an identified 3D surface model.

[0080] Alternatively, or in addition, one or more such controls comprise a tool that generates a data set comprising a model of at least a subset of the surface scan minus portions of an identified surface 3D model co-represented in an identified volumetric density 3D model.

[0081] Figures 3A-3N illustrate an exemplary embodiment of an aspect of the present invention in which a three-dimensional socket model is generated from three-dimensional surface scan segments (the 3D surface model of a corresponding object recognized and segmented from surface scan data of the surface scan) and volumetric density scan segments (the 3D volumetric density mode of a boundary surface of a corresponding object recognized and segmented from volumetric density scan data of the volumetric density scan) associated with an object. In the illustrated embodiment, the object is a tooth and the generated socket model is a dental socket that is equivalent to the portion of the extrinsic shape of the portion of the tooth that lies below the masticatory mucosa line, i.e., the root of the tooth.

[0082] Figure 3A shows an example of a 3D surface boundary model 1a generated based on a volumetric density scan. The individual structures in model 1a correspond to the actual structures of the patient's oral situation. As shown, the 3D surface boundary model 1a incorporates anatomical structure representations of the masticatory mucosa tissue 2, bone 3, and actual teeth of a patient (shown labeled as 11, 12, 13, 14, 15, 16, and 17, according to the International Dental Federation (FDI) notation (a tooth numbering system commonly used in the dental industry), among other individual teeth (not labeled). In one embodiment, the surface boundary model 1a is generated from the volumetric 3D structures represented in the volumetric density scan and extracted through image recognition and segmentation techniques, such as thresholding the intensity values (measured in Hounsfield units) of individual scan elements in the upstream radiographs. In one embodiment, the surface boundary model 1a comprises a point cloud, a triangular or other polygonal mesh, or other digital 3D model.

[0083] Figure 3B shows an example of a 3D surface model 1b generated based on a surface scan of the same anatomical area as in Figure 3A. The individual structures in model 1b correspond to actual structures in the patient's oral situation. As shown, 3D surface model 1b incorporates anatomical structure representations of a patient's actual masticatory mucosal tissue 2 and teeth (shown labeled 11, 12, 13, 14, 15, 16, and 17, according to FDI notation). Since the bone in the patient's oral cavity lies beneath the masticatory mucosal surfaces and teeth, typically no bony information would be present in surface scan model 1b.

[0084] Because the surface scan and the volumetric density scan are obtained from the scan of the same oral areas of interest, both models 1a and 1b include respective models of anatomical structures that represent certain same anatomical structure (or same anatomical structures) (e.g., teeth 11, 12, 13, 14, 14, 15, 16, and 17, and masticatory mucosa 2) of the patient. Model 1b comprises a point cloud, a triangular or other polygonal mesh, or other digital 3D model, generated from the 3D surface structures represented in the surface scan.

[0085] As noted in Figures 3A and 3B, both the volumetric density scan model 1a and the surface scan model 1b are surface models of (broadly the same) anatomical area. The 3D surface models 1a and 1b do not include representations of the internal anatomy. This means that neither model contains information about the alveoli or other structure beneath the visible exterior surfaces of the objects in the model. In the illustrative example, where the object is a tooth, this means that it is not possible to determine the anatomy of the dental alveolus that seats the tooth, because the alveolus is not visible in either the 3D volumetric density scan model (Figure 3A) or the 3D surface scan model (Figure 3B).

[0086] In both models 1a in Figure 3A and 1b in Figure 3B, only the crown portions of the teeth are modeled; neither scan model includes the anatomy of the root or socket that seats the tooth. While surface detail is useful when planning dental treatment for a patient, for planning surgical treatment, and for designing prosthetics for the patient, the lack of subsurface information available in models 1a, 1b pertaining to the patient's anatomy below the visible surfaces of the patient's oral cavity may impede accurate planning and design.

[0087] To facilitate an accurate 3D alveolus model generation, in one embodiment, each of the respective volumetric density scan data and the surface scan data from which the respective 3D surface models 1a and 1b are generated are submitted to a segmentation application. The segmentation application may be a remote service 244 or may be a local application (stored in local memory 204 and executed via one or more processors 201). A segmentation processor processes each of the received surface scan data and volumetric density scan data to automatically recognize (via an image recognition function), extract, and categorize individual recognized objects into labeled categories or classes (via a segmentation function).

[0088] For example, in an embodiment where the object is a tooth in a patient's oral cavity, the segmentation processor receives each of the intraoral surface scan data and the CBCT or other volumetric density scan data and processes each of the scan data sets to recognize and label recognized objects such as teeth, masticatory mucosa, individually identified bones, and potentially other objects such as fillings, implants, etc. that are recognized in the received scan data. The segmentation processor tags recognized objects with corresponding object type labels associated with the object type (or classification) of the recognized object.

[0089] For example, the segmentation processor may recognize an object in the 3D model or scan data that corresponds to a type of tooth 16 and assign the recognized object an object type label of “16” (or any unique label that classifies the recognized object as being of the unique object type corresponding to that patient's actual tooth 16). The segmentation processor recognizes and classifies (i.e., “labels”) recognized data segments in the scan data into a plurality of individual segments corresponding to various recognized object types.

[0090] Preferably, the individual segments comprise a 3D surface model representing the actual scanned object (e.g., a scanned tooth, portion of masticatory mucosa, bone, implant, etc.). In one embodiment, each segment corresponds to an individual object in the patient's oral cavity and is labeled as such with an associated label. Each segment comprises an independent 3D model, represented as a three-dimensional triangular (or other polygonal) mesh.

[0091] Referring to Figure 3C, a segmentation processor may recognize representations of individual teeth 11, 12, 13, 14, 15, 16, and 17 in the patient's volumetric density scan, and may segment each recognized representation of any of the teeth 11, 12, 13, 14, 15, 16, and 17, masticatory mucosae 2, and bone 3, into corresponding independent segments 11a, 12a, 13a, 14a, 15a, 16a, 17a, 2a, and 3a, collectively forming a segmented volumetric density scan surface model 10a. Each segment is converted to an independent surface mesh, e.g., a 3D triangular mesh, and each segment may be independently selected (e.g., when presenting the segmented model in a graphical user interface (GUI) as discussed hereinafter.

[0092] Notably, because segments 11a-17a, 2a, 3a were extracted from the volumetric scan data, each segment includes the full information available from the volumetric density scan. This means that objects (such as nerve canals) and portions of objects (such as tooth roots) that cannot be visually represented in a surface scan because they are below the visible surfaces of the inside and outside of the patient's mouth are still modeled in the volumetric density scan segments. Each volumetric density scan segment includes full object information (based on what is visually represented in the volumetric scan data) even below the surfaces of the scanned anatomical area of the patient. Thus, each of the teeth 11a-17a includes root information, which is clearly visible in the segmented model.

[0093] In a similar manner, with reference to Figure 3D, a segmentation processor may recognize representations of individual teeth 11, 12, 13, 14, 15, 16, and 17 in the patient surface scan and may segment each recognized representation of any of the teeth 11, 12, 13, 14, 15, 16, and 17 and masticatory mucosae 2 into a corresponding independently selectable segment 11b, 12b, 13b, 14b, 15b, 16b, and 17b and 2b, collectively forming a segmented surface scan surface model 10b. Each segment is converted into an independent surface mesh, e.g., a 3D triangular mesh.

[0094] Although the segmented model 10a includes all segments of the scanned anatomical area of the patient, each segment is an independently selectable 3D model of the corresponding scanned object. Accordingly, each segment 11b through 17b and 2b may be viewed alone, for example, as shown in Figure 3H (which shows tooth segment 16b alone corresponding to tooth 16 of the patient). Tooth segment 16b includes only the portion of tooth 16 that is present in the surface scan. Thus, tooth segment 16b represents only the crown of tooth 16, since only the crown (the portion of tooth 16 that is above the masticatory mucosa line) is visible during the surface scan.

[0095] Figure 3E shows the segmented volumetric density scan surface model 10a and the segmented surface scan surface model 10b cross-mounted in a common three-dimensional coordinate system. Independent imaging systems are typically used to capture each of the surface scan data and volumetric density scan data. For example, an intra-oral scanner (IOS) may be used to capture the surface scan of the patient's oral area of interest, while a CBCT scanner may be used to capture the volumetric density scan of the patient. Both scans are valuable in providing important information and together complement each other to provide a more complete picture of the patient's actual oral situation. Surface scanners (such as optical scanners) can capture very high resolution details of the visual topography of the patient's dentition, but they can only capture surface details and not internal details.

[0096] In contrast, volumetric density scans (such as CT or CBCT scans) can capture internal volumetric and density details of the patient's dentition, such as jawbone dimensions and density, complete teeth (including roots), and nerve pathways. Together, surface scanning and volumetric density scanning can form the foundation of a dental treatment planning and prosthesis manufacturing process.

[0097] Because independent imaging systems capture image data relative to the specific 3D coordinate system of the imaging system capturing the scan, in order to cross-assemble the scans into a single viewing pane having its own 3D coordinate system, both scans must be aligned with each other. This process is often referred to as scan combination or registration. Methods exist for aligning 3D meshes into a single 3D coordinate system.

[0098] In one embodiment, each of the surface scan data and the volumetric density scan data is segmented into 3D triangular mesh segments corresponding to individual teeth and jaws, followed by determining the keypoint of the corresponding tooth segments from each surface scan and volumetric density scan for each tooth, followed by aligning the keypoints in a common 3D coordinate system. This process can be performed, for example, using CoDiagnostix™ dental implant planning software offered by Dental Wings, Inc. (a Straumann Group company).

[0099] Notably in Figure 3E, the positions of the respective segments (11a, 12a, 13a, 14a, 15a, 16a, 17a, 2a and 11b, 12b, 13b, 14b, 15b, 16b and 17b and 2b) of the respective segmented models 10a and 10b correspond to the same respective real anatomical structures (11, 12, 13, 14, 15, 16, 17 and 2) in the patient's mouth. As can be seen, it is very important that the segments corresponding to portions of the same real anatomical structure from each type of scan (e.g., internal or surface) be assembled in the same three-dimensional coordinate system. When they are properly cross-assembled, the segments representing the same real anatomical structure substantially coincide, as shown.

[00100] In one embodiment, a segmentation processor processes the scan data to identify and classify portions of the scan data into individual segments classified into anatomical structure types based on a labeled training data set that includes multiple situations of each of the anatomical structure types. In one embodiment, the segmentation processor is a trained convolutional neural network (CNN) that was trained on a large dataset of scanned images obtained from a large number of different people with different anatomical structure situations that include, or are missing, different teeth, masticatory mucosa, bones, and other natural and artificial anatomical structures (e.g., implants, prostheses, etc.).

[00101] Figure 3F illustrates the problem encountered when a dental scan segment 16b (see Figure 3D) is removed from the scanned surface. As illustrated, the removal of the dental crown segment 16b results in a hole 16c in the model 10b where the crown 16b used to be. This is expected since the 3D segmented surface scan model 10b is generated based only on the surface scan data, which does not contain bone or other subsurface information, such as tooth root information. Thus, when the dental segment 16b is selected from the segmented surface scan model 10b (see Figure 3D) and removed, there is no socket information available, and the surface scan model 10b has only a hole 16c where the dental segment 16b was prior to removal.

[00102] Figure 3G shows the 3D segmented surface scan model after tooth segment 16b has been removed and tooth segment 16a from the 3D segmented volumetric density scan model (from Figure 3C) has been comonted into the same 3D coordinate system. Figure 3H shows tooth segment 16a separated and apart from the other segments of volumetric density scan model 10b. Tooth segment 16a includes a crown portion 16ac and a root portion 16ar. Notably, the root portion 16ar of tooth 16 includes a trunk 16at and three individual roots (a lingual root 16ar_l, a mesiobuccal root 16ar_mbs (not visible in Figure 3H), and a distal buccal root 16ar_dbr). Although tooth 16 may have 3 individual roots, other teeth may have only one root, or may have two or more roots. For simplicity, the trunk and individual roots of any given tooth may be collectively referred to herein as “the root” of the tooth. Only the crown 16ac is visible above the masticatory mucosa line on a surface scan. The root portion 16ar is not visible above the masticatory mucosa line to the naked eye or to the cameras on an intraoral scanner.

[00103] Figure 3I shows individual tooth segment 16b (from the 3D segmented surface scan model 10b of Figure 3G). As noted, tooth segment 16B represents only the crown of tooth 16, since only the crowns of the teeth are visible to the surface scanning cameras (because they are above the masticatory mucosa line and can be seen both with the naked eye and also through the camera lens (or lenses).

[00104] Notably, since both the surface scan and the volumetric density scan visually represent the same area of interest, both scans include surface information pertaining to the same actual corresponding anatomical structures (assuming each scan scanned the same areas). This means that for visible surfaces, such as the crowns of teeth, both the surface scan and the volumetric density scan will each include surface information or surface boundary information, respectively, pertaining to the crowns of the teeth.

[00105] Surface scans using optical sensors tend to produce much higher resolution images, resulting in highly detailed 3D surface models. Volumetric density scans typically use modalities that cannot be as accurate or would be medically unsafe to be as accurate as an optical scan. For example, volumetric density scans generated using X-ray technologies, including CT or CBCT modalities, are based on X-ray radiation, and although highly accurate images can be obtained using a high dose of X-rays, doing so would be medically unsafe for the patient. Consequently, CT and CBCT modalities used on patients must be set to very low levels of X-ray radiation to make them safe for humans. The disadvantage is that the accuracy of the images is lower. Consequently, crown surface data from surface scanning will generally include higher detail than crown surface data from volumetric density scanning.

[00106] In order to generate a 3D model of the patient's oral situation after a tooth has been removed, the application retains the root portion 16ar of the volumetric density scan tooth segment 16a and removes the crown portion 16ac. To do so, the application determines the masticatory mucosa line around the volumetric density scan tooth segment 16B based on points along the lower edge of the surface scan tooth segment 16B.

[00107] Figure 3J shows the surface scan tooth segment 16a and the volumetric density scan tooth segment 16b mounted in a cross-mounted fashion (both displayed within the same 3D coordinate system). As shown, the masticatory mucosa line 16Bgl is the set of points corresponding to the lower edge of the surface scan tooth segment 16b. Since the application knows the position of the masticatory mucosa line 16bgl, it calculates the crown portion 16ac as all points on the same side of the masticatory mucosa line (also called a cut line) that the surface scan tooth segment 16b lies within the application-specific 3D coordinate system.

[00108] Simply put, the application removes all points from the volumetric density scan tooth segment 16a that coincide with or substantially lie within the same volume of the area of the 3D coordinate system in which the surface scan tooth segment 16b (i.e., the crown) lies, i.e., the corresponding portions of the volumetric density scan tooth segment and the surface scan tooth segment are co-represented. Even more simply put, the root portion 16ar is obtained by subtracting 16b from 16a (and removing any outliers if necessary).

[00109] Figure 3K illustrates the root 16ar after removal of the crown portion 16ac from the volumetric density scan tooth segment 16a. Because this is a surface model, only the exterior points of the segment are present in the 3D model; therefore, when the crown portion 16ac is removed from the segment 16a, the interior 16ar_interior is empty. The shape and form of the root portion 16ar is defined only by points on the extrinsic surface of the tooth root 16ar, as defined from the surface model of the individual segments obtained from the volumetric density scan.

[00110] Accordingly, the contours of the interior of the root follow the contours of the exterior surface of the tooth root itself. The remaining root 16ar may thus be displayed together with the surface model 10a (with the crown segment 16a removed) of Figure 3F to generate a 3D surface model 10d representing the oral situation of the patient with the tooth 16 extracted. This is shown in Figures 3L, 3M and 3N. Figure 3L presents the 3D model 10d substantially along the horizontal plane with a view from the lingual side to show the orifice 16c and the contour of the alveolus 16s (represented by 16br as extracted from 16b). Figure 3M shows the model 10d from another orientation, along the same horizontal plane as Figure 3l, viewed from the posterior side of the model. The contour of the alveolus is more visible from this viewpoint. Figure 3N shows yet another view of the 10d model looking into the socket in which tooth 16 was virtually extracted. The contours of socket 16s are visible, showing where two of the three individual root tips were seated prior to the virtual extraction of tooth 16.

[00111] The socket 10S includes the contours in which the trunk 16Bt and each of the individual roots were seated prior to the virtual tooth extraction. As illustrated, the socket 16s follows the contours of the root of the extracted tooth, including a portion of trunk socket 16at and three individual root sockets including lingual root socket 16sr_l, mesiobuccal root socket 16sr_mbs, and distal buccal root socket 16sr_dbr, corresponding to lingual root 16ar_l, mesiobuccal root 16ar_mbs, and distal buccal root 16ar_dbr, respectively.

[00112] Because the volumetric density scan segment model of the tooth includes only the extrinsic (boundary) surfaces of the tooth object, it contains no information about the interior of the tooth itself. That is, for segmentation, the segmentation processor produces a 3D mesh of the extrinsic surfaces of the tooth without including any modeling of the inside of the tooth. For closed objects, such as a tooth, the 3D mesh model is also a closed triangular mesh (the number of triangular-shaped edges and facets associated with any given vertex is equal). Thus, because the inside of the cut root segment 16ar (i.e., the segment 16 with the crown portion 16ac removed) is empty, the surfaces on the inside of the root segment 16ar follow the same contours as the surfaces on the outside of the root segment 16ar. That is, the inside surfaces are merely the same exterior walls of root segment 16ar, but viewed from the inside of the walls.

[00113] Removing the crown portion 16ac from the volumetric density scan segment 16a produces an open mesh (i.e., there is at least one vertex in the mesh m such that the number of edges associated with the vertex exceeds the number of triangular shaped facets associated with that vertex). An edge facet, as used herein, is a facet whose number of adjacent facets (that share an edge) does not equal the number of edges of the facet. In the context of removing the crown portion 16ac from the volumetric density scan segment 16a, the remaining portion of the volumetric density scan segment 16a, i.e., the cut root segment 16ar, includes a set of edge facets along the margin line (where the crown meets the masticatory mucosa line), making it an open mesh. Since there is no information on the inside of the 16ar cut root segment, the surface on the inside of the open mesh is identical to the outside surface of the 16ar cut root segment.

[00114] A second aspect of the present disclosure relates to a virtual object extraction tool that incorporates the 3D socket model generation methodology described hereinbefore (e.g., in connection with FIGS. 3A-3N). Such a virtual object extraction tool may be implemented in various anatomical treatment planning and design software tools.

[00115] For example, without limitation, in various embodiments, the virtual object extraction tool is determined and generated in connection with any and all virtual tooth extraction tools in a dental treatment planning software tool, an implant planning software tool, and/or a dental design and manufacturing planning (CADCAM) software tool. For example, in a dental tooth extraction operation, the innovative devices, systems, graphical user interfaces, computer-enabled tools, methods, and processes presented herein enable a virtual tooth extraction of a patient's tooth to be performed virtually on an electronic display prior to the actual extraction of the patient's tooth.

[00116] The virtual tooth extraction tool allows the dental professional to visualize a highly accurate model that accurately represents the contours of the actual socket that seats the object and that will remain and be exposed after the object is removed from the patient's mouth. The socket model can be generated and displayed (along with the surrounding area of the patient's oral cavity) after the virtual removal of the targeted extraction tooth. The contours of the socket are accurately represented and the inside of the socket conforms to the extrinsic shape, below the masticatory mucosa line (i.e., margin line), of the exterior surface of the patient's actual tooth (including the root(s)).

[00117] The socket model can be used to perform a virtual object extraction, whereby a targeted extraction object (such as a tooth) can be removed from a 3D model of the anatomical area of interest and a 3D virtual socket model is added to the model (or made available for display with the model). Displaying the 3D model with the object removed and the virtual socket made visible is in itself instructive for use in the actual extraction of objects from a patient's anatomy. It is additionally useful in subsequent treatment planning, such as dental implant planning and prosthetic design planning, discussed hereinafter in this document.

[00118] Referring to FIG. 4, a virtual object extraction method 400 may comprise a step 401 of receiving a surface scan and a volumetric density scan of an anatomical area of interest and a step 402 of receiving surface scan segments and volumetric density scan segments corresponding to objects recognized in the surface scan and volumetric density scan of the anatomical area of interest. Steps 401 and 402 may occur outside of execution of the application or may be an integral part of the application, similar to the discussion of steps 101 and 102 in connection with the method of FIG. 1.

[00119] The method 400 may further comprise a step 403 of cross-assembling the surface scan segments and the volumetric density scan segments into a common 3D coordinate system. In a step 404, the method comprises displaying on an electronic display at least the cross-assembled segments associated with the targeted extraction object and adjacent cross-assembled segments of interest. In a step 405, the method comprises receiving a selection of one or more target extraction objects. In a step 406, the method comprises identifying the surface scan segments and the volumetric density scan segments associated with the selected objects of interest. In a step 407, the method comprises, for each selected target extraction object, generating a 3D honeycomb model corresponding to the selected target extraction object.

[00120] The method 400 may further comprise a step 408 of removing from the display each of the identified surface scan segments and volumetric density scan segments corresponding to the selected target extraction object (or selected target extraction objects), and a step 409 of displaying the generated 3D socket model (or generated 3D socket models) on the electronic display. The method 400 may further comprise a step 409 of saving and/or exporting a 3D surface model representing the extracted tooth model including the generated 3D socket model (or generated 3D socket models) and relevant segments of the anatomical area of interest, but without the segments corresponding to the selected target extraction object (or selected target extraction objects).

[00121] The functionality of any of steps 401, 401, 403, 405, 406, and 407 may be implemented as described in connection with respective steps 101, 102, 103, 104, 105, and 106 of FIG. 1, or may be implemented by means of a call to a 3D model generation tool (215a in FIG. 2) that performs the functionality of the relevant steps of the method of FIG. 1 and returns the generated 3D socket model (or 3D models of generated sockets) to the tool executing method 400.

[00122] Method 400 may be implemented as a computer-enabled virtual object extraction application 215b running on a system such as system 200 shown in FIG. 2. Various aspects of virtual object extraction application 215b may be implemented as computer instructions that may be stored in local memory 204 of system 200 and that, when executed by processor (or processors) 201, may implement features and aspects of method 400.

[00123] The computer instructions may include instructions implementing a graphical user interface (GUI), which may include a user-facing graphical user interface environment (or “front-end”) and an underlying operational GUI (or “back-end”) that executes functionality to perform operations indicated by user input received via GUI environment controls. The front-end GUI includes input controls through which user control, input, and data are received via input device (or devices) 207 and through which patient data may be loaded into the environment. The front-end GUI further includes output display areas, such as user control elements and display panels, in which user controls, various views of data and scan models, and other information are displayed and output via output device (or devices) 208, 209 based on GUI control settings and user input to such controls.

[00124] FIGURES 5A-5D illustrate an exemplary embodiment of a graphical user interface (GUI) display environment 500 during various steps in a dental treatment planning workflow. The GUI display environment may be displayed on an electronic display 209 of a system 200 implementing a dental treatment planning application 215b. The GUI display environment 500 includes control(s) 501 (not shown individually) to allow a user to select and load scan data of a patient 220 into mass storage 206 of the system 200 or into an external memory device (not shown) accessible through the system 200 or that is accessible from a remote service 212 via network adapter(s) 211.

[00125] In the context of the described aspects of the invention, the scan data 220 includes surface scan data and volumetric density scan data of an anatomical area of interest of a patient. In one embodiment, both the surface scan and the volumetric density scan are obtained before the dental treatment planning application is activated; in other embodiments, one or both of the surface scan data and volumetric density scan data are obtained in combination with the use of the application 215b.

[00126] For example, a remote service 212 may include an optical scanner application in communication with an optical scanner and placing optical scan data in communication with the dental treatment planning application during or after completion of an intraoral scan of areas of interest in the patient's oral situation. Similarly, a remote service 212 may include a volumetric density scanner application in communication with a volumetric density scanner and placing volumetric density scan data in communication with the application 215b during or after completion of an intraoral scan of areas of interest in the patient's oral situation.

[00127] The application 215a manages the display of graphical content in the GUI display environment 500, including monitoring user input for graphical controls received through the user input device (or devices), such as a mouse, keyboard, joystick, voice recognition, etc. The user input may correspond to actions to be taken, such as invoking various application functions specific to the substantive features of the application or GUI functions to change the layout or content of the features displayed on the display. More particularly, the front-end GUI displays user input controls and monitors user input associated with the functional controls. Upon receipt of user input associated with a user input control, the GUI invokes an appropriate function corresponding to the particular user input control and the type and content of the user input. The GUI also responds to back-end process (or processes) that communicate with the back-end GUI, which in turn communicates with the front-end GUI to display, remove from display, and/or modify the display of information on the electronic display.

[00128] Such user-selected functions may result in, without limitation, displaying, removing from displaying, and modifying the display of: models, views, segments, and/or annotations, and displaying, removing from displaying, and updating the appearance of various user controls and information displayed in the GUI display environment 500, and receiving and returning information to facilitate substantive functional features of the application 215b, including, but not limited to: substantive treatment assessment, substantive treatment planning, virtual performance of treatment or operations (such as tooth extraction, implant placement, prosthetic design and placement, etc.).

[00129] Referring to Figure 5A, the GUI display environment 500 includes global controls 501, such as file management, generic display controls, and other controls that are generic to the GUI display environment. For example, the controls 501 may include file selection controls, file save/export controls, view pane format controls, etc. The GUI display environment 500 also includes patient data-specific controls 502, such as individual dental model and arch controls.

[00130] The GUI display environment 500 includes functional controls 503, including a tooth extraction control 512. The tooth extraction control 512 is generally shown as a single control, but may comprise a plurality of controls, such as guided dialog pop-up display panels or other well-known interactive GUI techniques for displaying information and prompts for information and for receiving user input. The GUI 500 also includes and displays at least one display panel 503 for displaying therein a three-dimensional model of a selected patient's 3D anatomical model of the patient's oral situation (or selected portions thereof), as obtained from the patient's scan data and as selected via selection controls on the controls 502.

[00131] Controls 501 include one or more controls (not shown) that, when selected, allow user selection of a patient's surface and volumetric density scan data 220 from local, mass, or remote memory to mass memory 206 of system 200. In one embodiment, when patient scan data 220 is initially loaded, GUI display environment 500 may display one or a plurality of view panels 503 (only one shown) for the purpose of presenting on the display an overview of the patient's oral situation. In Figure 5A, view panel 503 is shown displaying a 3D model 10A of the volumetric density scan. Environment 500 may also include multiple additional views of the patient's oral situation based on the volumetric density scan data. For example, environment 500 may include a panoramic viewing panel, an axial viewing panel, cross-sectional viewing panels, and a tangential viewing panel (not shown).

[00132] An important purpose of the virtual tooth extraction tool, accessible via control 512, is to virtually represent the patient's oral situation following the virtual removal of one or more teeth or other objects selected for extraction. For example, in a dental treatment application 215b, when a tooth is virtually removed and displayed within the GUI display environment 500, the resulting displayed 3D model should include a representation of the tooth socket that becomes visible to the naked eye following removal of the target extraction tooth.

[00133] Previously, dental treatment planning applications and dental prosthesis design software (such as dental CAD/CAM) were not capable of providing detailed accuracy of the socket of a removed tooth because the surface models of the object generated from scan data did not include subsurface information. In contrast to prior art dental treatment planning and/or dental prosthesis computer-aided design (CAD) and computer-aided manufacturing (CAM) tools, the virtual tooth extraction tool extracts information from both a surface scan and a volumetric density scan of the patient's dentition, and uses the information corresponding to the same targeted tooth from both types of scans to automatically determine and provide a highly accurate anatomically based representation of the socket left behind after tooth removal. Application 215b provides a virtual tooth extraction tool (activated via control 512) that automatically determines the contours of the socket of the selected tooth that is targeted for removal and generates a virtual 3D model of the socket for display, saving, and/or extraction.

[00134] As shown in Figure 5A, a user may activate the virtual tooth extraction tool by selecting the tooth extraction control 512 by moving, via a mouse (not shown), a graphical cursor 520 over the control 512 and clicking the mouse on the control 512. Figure 5B shows an embodiment of a pop-up dialog box 513 that is displayed in the GUI environment 500 when the Tooth Extraction control 512 is activated. As illustrated, the dialog box may include a tooth selection map that allows a user of the application 215b to select one or more individual teeth for virtual tooth extraction.

[00135] In one embodiment, the user may click on individual teeth in the chart to select such teeth as a targeted extraction tooth. The user may optionally select, by selecting a respective check box or radio button or other such selection feature, an instruction to save the socket model generated by the tool after generating the socket and/or an instruction to extract the tooth extraction model (which contains the model in the preview pane 503 minus the target extraction tooth (or target extraction teeth) plus the socket models generated for such target extraction tooth (or target extraction teeth). When the user has finished selecting the target extraction tooth (or target extraction teeth) and the save/extraction options, the user may click the Select button 517 to invoke the tooth extraction tool.

[00136] Figure 5C shows a posterior view (looking down the maxillary arch towards the front of the patient) of the 3D tooth extraction model 10D, which more conveniently shows the 3D socket model 16S corresponding to the socket from which tooth 16 is extracted. The socket model 16S is shown together with the 3D surface scan model with tooth 16 extracted. Figure 5D shows the same 10D model from the inferior view (from the bottom looking up towards the maxilla). As illustrated, tooth 16 is missing, but the interior 16c of the socket 16S is visible and follows the contours of the root (or roots) of the extracted tooth 16.

[00137] Virtual tooth extraction is often a precursor to the placement of a dental implant. Figure 6 thus shows a flowchart of an exemplary method 600 of a third aspect of the present disclosure, namely for generating a surface model with a tooth extraction model with an implant placed in the socket of the virtually extracted tooth. According to method 600, in a step 601, a dental implant planning application, such as an application 215c running on a system 200 of Figure 2, obtains a virtual model of a patient's dentition based on both a volumetric density scan and a surface scan of the area of interest of the patient's dentition.

[00138] In one embodiment, at a step 602a, the dental implant planning application 215c places, under control of user input received through GUI controls, a virtual implant in the virtual model of the patient's dentition. At a step 603a, the dental implant planning application 215c performs, upon activation by a user of a tooth extraction control, a virtual tooth extraction of the targeted tooth for replacement by the implant, resulting in the generation of an extraction tooth socket model and a tooth extraction model in which the targeted extraction tooth is removed from the virtual model of the patient's dentition and replaced by the generated extraction tooth socket model. Since the virtual implant is already in the virtual model of the patient's dentition, the tooth extraction model includes both the extraction tooth socket model and the virtually placed implant model.

[00139] In an alternative embodiment, steps 602a and 603a are reversed. In this alternative embodiment, in a step 602b the tooth extraction is performed first to generate the tooth extraction model. In a step 603b, the virtual implant is placed in the tooth extraction model. The virtual tooth extraction model may then be saved and/or exported to a file (step 604). Optionally, a surface scan crown segment is exported in the same 3D coordinate system as the exported tooth extraction model (step 605).

[00140] Optionally, at a step 606, the exported tooth extraction model and dental crown segment file(s) may be sent to a prosthesis design tool for import and design of an implant prosthesis.

[00141] Implant placement planning is often accomplished by pre-planning the selection and virtual placement of an implant on a virtual model of the patient's dentition. Virtual implant planning and placement can be accomplished using a dental implant planning tool, several of which are commercially available. One example is the CoDiagnostix® dental implant planning software offered by Dental Wings, Inc. (a Straumann Group company). This dental implant planning software includes software-implemented tools, including a graphical user interface for importing and displaying internal and surface scans of a patient, selecting dental implants, and placing the selected dental implant (or selected dental implants) on a model of the patient's dentition generated based on the imported scan (or scans).

[00142] For example, Figure 7A depicts a GUI display environment 700 generated by an application (such as application 215c in Figure 2) in which surface and volumetric density scans of a patient have been imported and loaded into memory 206 of system 200. Figure 7A depicts the patient's dental situation after a dental professional has selected and virtually placed an implant. Techniques for virtually placing the implant in a virtual model of a patient's dentition are known in the art, for example through use of CoDiagnostix® dental implant planning software. In Figure 7A, a virtual implant pin 710 has been placed and is shown in various types of views in corresponding display panels of the graphical environment 700. In the example shown, the implant pin 710 is shown virtually placed in a cross-sectional view (display panel 703d, an axial view (display panel 703c), a panoramic view (display panel 703b), a tangential view (display panel 703e), and a 3D view (display panel 703a).

[00143] As shown in the various views in panels 703a-703e, implant placement is depicted by placement of an implant post or screw 710, which is the base portion of the completed implant. A completed implant includes an implant post 710, a prosthetic abutment (not shown) that attaches to the implant post 710, and a prosthetic abutment for a dental prosthesis or restoration (also not shown), which may be a crown, bridge, or denture.

[00144] In the initial planning stage, only the implant post 710 needs to be virtually placed. The implant planning software application provides a virtual implant placement guide (or guides) 711, which do not correspond to a physical component - they are merely visual indicators that assist the dental professional in placing the implant at the correct angle. In Figure 7A, the virtual guides 711 appear in the 3D visualization panel 703a as a long cylindrical rod with a central axis coinciding with the central axis of the implant post 710 and having a diameter corresponding to the diameter of the prosthetic abutment attachment socket on the inside of the implant post. Preferably, the cylinder of the virtual implant placement guide 711 extends along the central axis above the occlusal plane of the teeth, such that the length of the cylinder is much greater than the diameter of the cylinder. Preferably, tab 711 is displayed in a contrasting color to the colors used in the 3D model and other view panes, such that the application user can immediately see the tab in relation to the content in each view pane.

[00145] The GUI display environment 700 includes a tooth extraction control 712, accessed in the exemplary embodiment by selecting a control from the display controls of the visualization panel 702 that corresponds to the portion of the patient's dentition in which the implant under consideration is placed. In the embodiment shown, the dental professional selects the lower arch control, right-clicks on it to open a context menu, and selects a tooth extraction control 712 from the context menu. The tooth extraction control may be selected to instruct the dental treatment planning application to automatically perform a virtual tooth extraction (using the principles described earlier in this document). There are many ways to implement the control that invokes the automated virtual tooth extraction tool - the key is to provide one or more controls that allow the user to invoke the tooth extraction workflow.

[00146] Figure 7B illustrates a pop-up window 713 that appears in the GUI display environment 700 when the user selects the Tooth Extraction control 712. The pop-up window presents various user input options and controls for obtaining inputs required by the virtual tooth extraction tool, including a tooth selection control 718, a mode control 714. In the embodiment shown in Figure 7B, the tooth selection control 718 displays a set of selectable tooth icons corresponding to the teeth in the selection portion of the patient's dentition as selected by the user in Figure 7A. The user (i.e., the dental professional) may select the tooth corresponding to the tooth with the virtual implant post 710 placed. In the example, the user selects tooth 35 corresponding to the lower left arch tooth in which the virtual implant prosthetic abutment 710 is placed. For mode, the user selects “Mode: Cut Socket” 715 from the “Options” drop-down menu 714, checks the check box control 716 to indicate that the extracted tooth should be saved as a separate file for future planning, and invokes the virtual tooth extraction tool by clicking the “Extract” control 717.

[00147] Figure 7C shows a 3D surface model of the patient's dentition in viewing pane 703a. Upon completion, the virtual tooth extraction tool adds two 3D model files to the list of scan models and available 3D models in controls section 702 of GUI display environment 700. The user may select these files to be displayed in pane 403a. One file is a dentition model with extracted tooth (indicated at 721 in the file list). The extracted tooth model is created by means of the virtual tooth extraction tool in accordance with the techniques described in connection with Figures 1 and 4, the extracted tooth model being a 3D surface model of the patient's dentition (including the previously placed implant post 310) with the selected tooth 35 removed from the model and a socket generated and included in the model in its place.

[00148] The second file is a model of an extracted tooth (indicated at 722) in the file list and is a model of tooth 35 where the crown has been removed from the root.

[00149] Once these files have been created, the user may click on the “Plan” menu 723 in the GUI display environment 700, as shown in FIG. 7D, and select a Virtual Plan Export control 724. In the next step, shown in FIG. 7E, the user may then select the export file format in a format selection control 727 from a pop-up window 726. In the example, the user selects an STL format option 728 and clicks “Next” 729 to move to the next menu. In the next step shown in FIG. 7F, the user selects a button control 732 to activate an option to export the scans or segmentations of the selected model without further processing. Upon clicking the “Next” button 733 in FIG. 7G, the tooth extraction file 731 is selected from the Export File selection pop-up window 730. Clicking the “Next” button 732 brings up the implant selection pop-up window 735 shown in FIG. 7H, which provides a scan body selection control 736. The scan body selection control 736 includes a scan selection control 737 that displays and allows selection of a suitable scan body from a set of possible scan body types. The user may select a scan body from the scan bodies menu to add the selected scan body to the model for export. The user then clicks the “Next” button 738. In Figure 7I, the user can select additional options, such as selecting the 3D coordinate system to which the model should be exported and selecting whether the exported objects should be exported as individual files sharing a coordinate system. Clicking the Export Schedule line 741 invokes the export function based on the selected options and parameters as selected by the user in the previous screens. The export files are saved to a known location in computer-readable memory.

[00150] In Figure 7J, the same process can be followed to export the surface scan crown segment of the virtually extracted tooth (and, optionally, the opposing tooth crown segment - that is, the tooth crown segment of the tooth in the opposite jaw that meets the targeted extracted tooth in the occlusal plane when the jaws are closed). It is important to ensure that the surface scan tooth segments are exported in the same coordinate system as the exported virtual tooth extraction model.

[00151] The resulting exported files comprise a 3D model of the patient's dentition with a virtual tooth extraction of the tooth in which the intended implant replacement will be placed. In place of the extracted tooth is a virtual socket in which the virtual tooth previously sat. Included in the model is the virtual implant post placed in the socket in which the dental professional placed it during the implant planning process.

[00152] The next step in the implant planning process is the design of a dental prosthesis or restoration, such as a temporary or permanent prosthetic abutment, crown, bridge, or dentures. Figure 8 is a flowchart illustrating an exemplary process for designing an anatomically correct prosthesis, which is especially useful for designing temporary prosthetic abutments for placement in the patient's mouth while the tissue around the surgical site heals after implant placement.

[00153] In accordance with aspects of this disclosure, the design of temporary prosthetic abutments based on a virtual tooth extraction, as set forth above, allows for the design of a customized prosthetic abutment (or other prosthesis) based on the patient's anatomy determined from the virtual tooth extraction. This means that the temporary prosthetic abutment (or other prosthesis) can be designed and manufactured even prior to implant placement surgery, so that it can be ready for placement immediately after (or shortly after) implant post placement. Because the virtual tooth extraction generates an anatomically correct tooth socket specific to the anatomy of the patient's extracted tooth, there is no need to scan the patient's actual socket after the tooth is actually extracted for the purpose of profiling the surface of the socket.

[00154] This saves significant time, as typically such scanning must be delayed long enough until some initial healing has occurred, as bleeding at the surgical site can interfere with obtaining a highly accurate surface scan of the socket.

[00155] The process described in FIG. 8 includes obtaining a virtual tooth extraction model based on both a volumetric density scan and a surface scan of the area of interest of the patient's dentition (step 801) and obtaining a surface scan crown segment of the virtually extracted tooth (step 802). In one embodiment, the virtual tooth extraction model may be obtained in accordance with the methods and systems described above. The obtained surface scan crown segment is imported into the model and used as the crown of the denture (i.e., the portion of the denture that will be visible above the masticatory mucosa line). In one embodiment, the design is electronically designed in a CAD/CAM tool that provides a clone function to clone a virtual diagnostic wax-up of the crown of the original tooth.

[00156] In a GUI of the denture design tool, the virtual tooth extraction model with the virtual implant placed therein is displayed to allow a user to view the model (step 804). User controls are provided through the GUI to allow the user to design the lower portion of the denture such that, at least in some portions, the design follows the contours of the inside of the extraction socket (step 805).

[00157] The final design is a prosthesis design having an upper portion that substantially conforms to the surface scanned crown segment and a lower portion having an intrinsic surface that substantially conforms to the inner surface of the tooth extraction socket of the Virtual Tooth Extraction model. The prosthesis design may be exported to one or more electronic files in a format usable for subsequent manufacturing (step 806). A physical prosthesis may then be manufactured (step 807) based on the electronic file(s). In one embodiment, the electronic files comprise or are converted to 3D printing instructions for printing by means of a 3D printer. In another embodiment, the electronic file(s) comprise an STL file.

[00158] FIGURES 9A-9R depict a GUI display environment 900 for a prosthetic design application, such as a computer-aided design (CAD) or computer-aided manufacturing (CAM) tool. In one embodiment, the prosthetic design tool is an application 215d that operates on a system such as 200 described previously in FIG. 2.

[00159] Figure 9A is a GUI display environment 900 displayed on an electronic display 209 and having user input controls and display areas as discussed hereinafter. To design a prosthesis, a user initiates a new case by clicking on control 901 (Figure 9A), entering case information (e.g., in text boxes, the case ID, patient ID, and dental professional ID to be associated with the new case), and selecting and loading 3D models generated from patient scan data into memory for use by application 215b. In the example, a dental crown is to be designed. The user enters the names of the virtual tooth extraction files (in this case, as the lower teeth model input) and the extracted tooth file (as the lower diagnostic wax-up) and uploads the files to the system (e.g., by clicking the “Save” button (Figure 9B). The application displays the 3D model from the selected Virtual Tooth Extraction file(s) in a viewing pane 903 (Figure 9C). If necessary, the application provides controls in environment 900 to clean up the scan (e.g., to fill holes where the scan data is incomplete, smooth scan lines, remove noise, etc.). The application may then provide tools to adjust the orientation of the model(s) to the plane of occlusion, if necessary (Figure 9D).

[00160] Before proceeding with design, the user marks the tooth positions on the displayed model to indicate to the application 215 the position (or positions) of the tooth or teeth for which a prosthesis will be designed and which teeth are adjacent to the tooth/teeth for which the prosthesis will be designed (see FIG. 9E). Next, in FIG. 9F, the user selects the platform (implant manufacturer, implant type, and connection) and scans the body. These selections must match the implant and scan bodies selected in the dental implant planning application and on which the virtual tooth extraction files were generated.

[00161] Once the setup is complete, the user can proceed with the design of the prosthesis. In Figure 6J, the surface scan crown segment from the 3D model has been exported as a single segment in a 3D coordinate system matching the virtual tooth extraction model. Because the surface scan crown model was generated from an optical scan of the patient's original tooth (prior to the actual extraction of the real tooth), the surface scan crown segment can be used as a digital diagnostic wax-up model without having to rescan the patient's mouth. Since the surface scan crown segment has been exported as a single segment matching the same 3D coordinate system as the virtual tooth extraction model, the surface scan crown segment of the virtually extracted tooth can be mounted on the display panel with the virtual tooth extraction model and can be used directly by the 215d application as the upper portion of the prosthetic crown. Upon importing the surface scan crown segment file as the diagnostic wax-up file (Fig. 9B), the designer may select a Clone Diagnostic Wax-up control (Fig. 9G) to instruct the prosthetic design application 215d to clone the diagnostic wax-up model into the diagnostic wax-up model file to serve as the surface of the prosthetic crown. Once the Clone Diagnostic Wax-up tool clones the diagnostic wax-up model as the prosthesis, the user may then perform fine-tuning of the prosthetic shape via the snapping, formatting, and sculpting controls available in the GUI display environment 900. The environment 900 also includes controls for rotating and changing the view of the model displayed in the viewing pane 903a such that the user may view the prosthesis 904 from all angles. For example, in Fig. 9H, the model 905 is rotated such that the prosthesis 904 may be viewed from the buccal side. This allows the designer to make adjustments to the prosthesis by visualizing the prosthesis in situ within the virtual tooth extraction model 905.

[00162] Once the visible surfaces of the crown are designed based on the extracted surface scan crown segment obtained during the tooth extraction process, the user can then design the lower portion of the denture. The application may provide controls for the designer to enter restoration specifications (see Figure 9I), such as material type, color, as well as whether or not to output (e.g., output an STL file, issue an order placement (which may be a direct communication with a remote manufacturing facility).

[00163] In FIG. 9J, the virtual tooth extraction model 905 is displayed in the visualization pane 903a. Since the example case in FIGS. 9A-9T is a new case for designing a prosthetic design for an implant, and the virtual tooth extraction model 905 was exported with the implant placed in the model (e.g., using the process described in connection with FIGS. 7A-7J), the virtual tooth extraction model 905 includes a virtual implant post. In this example, a temporary prosthetic abutment is selected and automatically virtually connected to the implant post and displayed as shown. The user can then select the cementum gap thickness (FIG. 9K), set the material thickness (FIG. 9L), and display the prosthesis 904 within the model 905, which includes the contours of the socket (or “emergence profile”) (FIG. 9M). The user can also then turn off the display of the 905 model to display only the prosthesis (Figure 9N).

[00164] Figure 9O shows the model 905 turned back on for display (see the on/off buttons for displaying various models in Figure 9O) and rotated to obtain a good external side view of the socket 910. With the anatomy transparency level set to low (for the purpose of viewing the implant prosthetic abutment and the prosthetic abutment from within the socket 910), it is clear that it is possible to design an anatomically correct prosthesis 904 that conforms to the patient's oral anatomy by using the contours of the socket to guide the design of the lower portion of the crown of the prosthesis to fit against the contours of the inside of the socket 910. In this regard, the prosthetic design application provides controls for adjusting the shape and fit of the lower portion of the crown.

[00165] Once the bottom of the prosthetic crown is shaped to conform to the contours of the socket 910, the designer can proceed to specify the prosthetic shell. The application can automatically calculate the proximal distance information between the prosthetic surfaces and the teeth on either side of the prosthesis (when it is virtually attached to the virtual implant post 906) (Figure 9P) and generate a shell surface (viewed from the buccal side in Figure 9Q and looking down on the occlusal plane in Figure 9R). Figure 9S shows the final prosthetic design within the model with both the upper and lower jaws with the anatomy model partially set to transparent for the purpose of partially seeing the implant post 906, the prosthetic abutment, and the prosthetic crown. The user can easily verify the placement and shape of the prosthesis by visually inspecting the model. Figure 9S. Figure 9T shows the same model and orientation as Figure 9S, but with the anatomy in full opacity. The workflow will continue to follow the usual steps before sending the crown into production.

[00166] Upon completion of the prosthetic design, the design files may be exported and used to manufacture the design. In one embodiment, the exported design files may be sent to a manufacturing facility or a remote manufacturing service. In one embodiment, the exported prosthetic design files may be used to generate 3D printer instructions for submission to a 3D printer that is responsive to such instruction to 3D print the prosthesis.

[00167] The aspects and embodiments of the invention described above provide multiple advantages in visualization of the virtual socket, anatomical treatment planning, and anatomical prosthetic design. According to one advantage, anatomical treatment professionals and prosthetic designers can plan treatment and design anatomically accurate prostheses based on an accurate three-dimensional model of the socket that is the site of treatment planning and prosthetic design and that models the specific anatomy of the patient's socket.

[00168] The model may be generated prior to extraction of the patient's anatomical object, which allows for accurate treatment planning and prosthesis design prior to or simultaneously (i.e., in parallel) with the actual surgical extraction of the patient's anatomical object. This means that the patient may visit the treating practitioner in as little as a single visit. During just one or two office visits, the patient's anatomical area of interest, including the anatomical object targeted for extraction, may be scanned. The scan data may be imported into a digital treatment planning tool that includes a virtual socket model generation tool and/or virtual object extraction tool, each of which generates a virtual socket model that is included in the display of a virtual anatomical model of the patient's anatomical area of interest (i.e., the area that includes the targeted extraction object and adjacent anatomical objects or features).

[00169] With the virtual socket model included in the displayed virtual anatomical model, the treating practitioner can more accurately virtually place an implant or other treatment body into the virtual socket model, design and print 3D printable surgical guides, and export the virtual anatomical model and virtual socket model for use in a separate prosthetic design software tool to design the anatomically accurate prosthesis. If the treating practitioner has access to immediate prosthetic manufacturing equipment, the prosthesis can be manufactured while the patient is still in the office. Otherwise, the prosthesis can be sent to a laboratory for manufacturing and the patient can return to the office when the anatomical area surrounding the implant has healed sufficiently to secure the prosthesis to the implant.

Claims (18)

1. A computer-implemented method (100) for automatically generating, by means of one or more computer processors, a three-dimensional (10D) model for use in a virtual extraction (602b, 603a) of a targeted extraction object from a patient, the targeted extraction object comprising an object in the anatomy of a patient, the method based on each of a surface scan (1b) of an anatomical region of the patient's oral cavity and a volumetric density scan (1a) of the anatomical region, the anatomical region including at least the targeted extraction object, the method comprising: receiving a first set of labeled surface scan segment data, each surface scan segment comprising a three-dimensional (3D) surface model (10b) of a corresponding object recognized and segmented from surface scan data of the surface scan, and each surface scan segment having an associated label identifying the surface scan segment; receiving a second set of labeled surface scan segment data labeled volumetric density scans, each volumetric density scan segment comprising a three-dimensional (3D) volumetric density model (10a) of a boundary surface of a corresponding object recognized and segmented from volumetric density scan data of the volumetric density scan, and each volumetric density scan segment having an associated label identifying the volumetric density scan segment; cross-assembling in a common 3D coordinate system labeled surface scan segments (10b) from the first data set to labeled volumetric density scan segments (10a) from the second data set; receiving identification of the targeted extraction object (16); identifying the 3D volumetric density model (16a) associated with the label of the volumetric density scan segment corresponding to the identified targeted extraction object; identifying the 3D surface model (16b) associated with the label of the surface scan segment corresponding to the identified targeted extraction object; generating a third a dataset comprising a model comprising: a) a 3D model of a alveolus (16s) being an equivalent of the identified volumetric density 3D model (16a) minus portions of the identified volumetric density 3D model (16a) co-represented in the identified surface 3D model (16b); b) at least a portion of the identified volumetric density 3D model (16a) co-represented in the identified surface 3D model (16b) and/or at least a portion of the identified surface 3D model (16b) co-represented in the identified volumetric density 3D model (16a); c) at least a portion of the identified volumetric density 3D model (16a) minus portions of the identified volumetric density 3D model (16a) co-represented in the identified surface 3D model (16b); and/or (d) at least a subset of the first dataset (10b) minus portions of the identified 3D surface model (16b) co-represented in the identified 3D volumetric density model (16a). 2. Method (100) according to claim 1, characterized in that the generation step comprises: determining portions of the identified volumetric density 3D model (16a) not co-represented in the identified surface 3D model (16b); and storing, in a computer-readable data storage component, the determined portions of the identified volumetric density 3D model (16a) as the 3D model of the alveolus (16s). 3. Method (100) according to claim 1, characterized in that the generation step comprises: determining portions of the identified 3D volumetric density model (16a) co-represented in the identified 3D surface model (16b); determining a difference between the determined co-represented portions and the identified 3D volumetric density model (16a) by removing the determined co-represented portions from the identified 3D volumetric density model (16a); and storing, in a computer-readable data storage component, the determined difference as the 3D model of the alveolus (16s). 4. Method (100) according to any one of the preceding claims, characterized in that the generation step comprises: generating a fourth data set by removing portions of the identified 3D surface model (16b) co-represented in the identified 3D volumetric density model (16a) from a copy of at least a subset of the first data set (10b); and storing, in a computer-readable data storage component, the fourth data set. 5. The method (100) of any preceding claim, wherein labeled surface scan segments from the first data set are cross-assembled to correspondingly labeled volumetric density scan segments from the second data set. 6. The method (100) of any preceding claim, wherein cross-assembling the first dataset of labeled surface scan segments (10b) to the second dataset of labeled volumetric density scan segments (10a) includes generating a fifth dataset of labeled segments, each segment comprising a combined three-dimensional (3D) model representing an anatomical object. 7. Method (100) according to any one of the preceding claims, characterized in that it comprises: determining a boundary of the identified 3D surface model (16b) and generating a cutting line (16bgl) from it; and projecting the cutting line onto the identified 3D volumetric density model (16a). 8. Method (100) according to claim 7, characterized by the fact that it comprises generating a sixth data set comprising the identified surface 3D model (16b) and/or the identified volumetric density 3D model (16a) on the side of the cut line (16bgl) that includes substantially all portions of the identified volumetric density 3D model (16a) that are co-represented in the identified surface 3D model (16b). 9. Method (100) according to any one of the preceding claims, characterized in that it comprises generating a seventh data set from the first, second, third, fourth, fifth and/or sixth data set or from the identified 3D surface model and/or from the identified 3D volumetric density model in which a void in a surface defined by means of the data set or model, respectively, is increased by filling the void with generated surface information. 10. Method (100) according to claim 9, characterized by the fact that it includes generating the surface information generated by means of interpolation of the void boundaries and/or based on standard anatomical modeling. 11. Method (100) according to any one of the preceding claims, characterized in that the target extraction object is a tooth and the socket model (165) comprises a three-dimensional model of the socket of a patient that seats the target extraction object (16). 12. Method (100) according to any one of the preceding claims, characterized in that the associated labels of each surface scan segment and/or each volumetric density scan segment identify the surface scan segment or the volumetric density scan segment, respectively, as corresponding to a tooth, a portion of a tooth, a dental prosthesis, a portion of masticatory mucosa tissue, a portion of bone, an implant or another object or structure present in the patient's oral cavity and/or anatomical region. 13. Method (100) according to any one of the preceding claims, characterized in that it comprises storing any of the third, fourth, fifth, sixth and/or seventh data sets and/or merged or cross-assembled versions thereof in a computer-readable data storage component. 14. A system (200) for automatically generating a three-dimensional model for use in a virtual extraction of a targeted extraction object from a patient, the system comprising one or more computer processing units (201) configured to load and execute computer-readable instructions that, when executed, configure the one or more computer processing units (201) to implement the method (100) as defined in any one of claims 1 to 13. 15. Computer readable media comprising computer readable instructions that, when executed by one or more computer processing units, configure one or more computer processing units to implement a method as defined in any one of claims 1 to 13. 16. A computer program, said computer program incorporating computer-readable instructions that, when executed by one or more computer processing units, configure one or more computer processing units to implement a method as defined in any one of claims 1 to 13. 17. Use of the third, fourth, fifth, sixth and/or seventh data sets as defined in any one of claims 1 to 13, and/or merged or cross-assembled versions thereof, characterized in that it is for a virtual extraction procedure of a targeted extraction object from a patient. 18. Use of the third, fourth, fifth, sixth and/or seventh data sets as defined in any one of claims 1 to 13, and/or merged or cross-assembled versions thereof, characterized in that it is for a prosthetic design CAD/CAM software tool to generate a prosthetic design that anatomically represents a patient's targeted extraction object, the use being prior to, during and/or during a single office visit when an actual extraction procedure of the targeted extraction object is performed on the patient.