CN106256331A - Systems and methods for navigating through the airway in a virtual bronchoscopic view - Google Patents

Abstract

Systems and methods for navigating through an airway in a virtual bronchoscope view are disclosed. Specifically, systems and methods are disclosed for displaying a virtual bronchoscope view while navigating an airway through the virtual bronchoscope. The method includes determining a first position and a first direction at the first position, storing the first position and the first direction in a memory, displaying a first virtual camera view corresponding to the first position, determining a second position corresponding to an airway moving through a virtual bronchoscope, storing the second position in the memory, displaying a second virtual camera view corresponding to the second position, determining a second direction based on the first position and the second position, storing the second direction in the memory, determining a third position corresponding to further movement through the virtual bronchoscope, and determining whether the further movement is in a forward direction or a backward direction.

Description

Systems and methods for navigating through the airway in a virtual bronchoscopic view

technical field

The present disclosure relates to systems and methods for displaying medical images in a dynamic and changing manner. More specifically, the present disclosure relates to systems and methods for dynamically displaying medical images during path planning during forward and backward movement of a virtual bronchoscope based on the position and orientation of a virtual camera being navigated through the airways of the lungs .

Background technique

Path planning visualization techniques are rapidly growing in various fields of medicine. Visualization techniques help minimize the size of incisions, treat a patient's disease non-invasively during surgery, and non-invasively navigate inside a patient to identify and treat targeted lesions. However, visualizations can pose unintended risks if incorrect information is displayed. For example, when navigating a virtual camera view through the airways of the lungs in a backward or withdrawn direction, the view presented to the clinician may make it difficult for the clinician to undo previously taken steps and return to a previous position. Also, the user will find the virtual camera view navigating outside of the airways of the lungs.

Contents of the invention

In one aspect, the disclosure features a method for displaying a virtual bronchoscope view while navigating through an airway of the virtual bronchoscope. The method includes determining a first position and a first direction at the first position, storing the first position and the first direction in memory, displaying a first virtual camera view corresponding to the first position, determining a position corresponding to moving through the virtual a second position of the airway of the bronchoscope, storing the second position in memory, displaying a second virtual camera view corresponding to the second position, determining a second direction based on the first position and the second position, storing the second direction In memory, a third position corresponding to further movement through the virtual bronchoscope is determined, and whether the further movement is in a forward direction or a backward direction is determined.

If it is determined that the further movement is in a forward direction, the method includes displaying a third virtual camera view corresponding to a third position. And if it is determined that the movement is in a backward direction, the method includes retrieving the stored first direction and the stored second direction from memory, determining a smoothing vector based on the stored first direction and the stored second direction, based on the smoothing The vector obtains a smoothed virtual camera view, and displays the smoothed virtual camera view.

In an embodiment, determining whether said further movement is in a forward direction or a backward direction comprises determining a next pointer position and a current pointer position on a virtual bronchoscope screen, calculating coordinates of the next pointer position and the current pointer position The difference between the coordinates of . If the calculated difference is positive, it is determined that the further movement is in the backward direction. And if the calculated difference is negative, it is determined that the further movement is in a forward direction.

In an embodiment, the first position, the second position and the third position are determined based on the position of a pointer or cursor on the screen displaying the virtual camera view. In yet another embodiment, the method further includes determining a third direction based on the second position and the third position if the further movement is determined to be in a forward direction, and storing the third direction in memory.

In an embodiment, determining the smoothing vector includes determining a first vector based on the first position and a first orientation, determining a second vector based on the first position and a second orientation, and averaging the first vector and the second vector to obtain the smoothed vector .

In an embodiment, the spline is a second order spline or a fourth order spline. In an embodiment, the method further comprises receiving input from a user to change the first virtual camera view, the second virtual camera view, the third virtual camera view, or the smoothed virtual camera view, and storing the changed first virtual camera view, second virtual camera view Virtual camera view, third virtual camera view or smooth virtual camera view. In still other embodiments, determining the smoothing vector includes determining a vector having a direction between the first direction and the second direction.

In another aspect, the disclosure features an apparatus for displaying a virtual bronchoscope view while navigating through an airway of the virtual bronchoscope. The apparatus includes a network interface configured to receive position information of the navigation instrument from at least one position sensor of the navigation instrument, the position information including the physical location; store a plurality of virtual camera views of the virtual bronchoscope, instructions, a first position, at a first A first orientation at a location, a second location, and a memory of a second orientation at the second location; a processor configured to execute instructions. The instructions, when executed by the processor, cause the processor to determine whether the airway moving through the virtual bronchoscope is in an anterior direction or a posterior direction. If the movement is determined to be in a forward direction, the instructions further cause the processor to determine a third position corresponding to the airway moved through the virtual bronchoscope, and determine the third direction based on the second position and the third position. If the movement is determined to be in a backward direction, the instructions further cause the processor to retrieve the first direction and the second direction from memory and determine a smoothing vector based on the first direction and the second direction. The apparatus also includes a display configured to dynamically display on the screen an image of the smoothed virtual camera view corresponding to the determined smoothing vector.

Any of the above-described aspects and embodiments of the present disclosure may be combined without departing from the scope of the present disclosure.

Description of drawings

Objects and features of the presently disclosed systems and methods will become apparent to those of ordinary skill in the art when the description of the various embodiments is read with reference to the accompanying drawings, in which:

FIG. 1 is a schematic diagram of a computing device for path planning according to an embodiment of the present disclosure;

Figure 2 is a diagram illustrating the four phases of path planning according to the present disclosure;

3 is a flowchart illustrating a method for dynamically determining and displaying virtual camera views for forward and backward movement within a lung airway, according to an embodiment of the present disclosure;

4 is a flowchart illustrating a method of determining a forward or backward direction within a lung airway, according to an embodiment of the present disclosure;

5 and 6A-6C are graphical illustrations of views when navigating within a lung airway, according to embodiments of the present disclosure; and

7 is a graphical illustration of steps taken to navigate through a lung airway, according to some embodiments of the present disclosure.

detailed description

The virtual bronchoscopic (VB) view enables the user to interact with the virtual camera (or the user's point of view) and to modify both the position and orientation of the virtual camera view inside the airways of the lungs. Moving backwards in the VB view can be achieved by adjusting the virtual camera (or user viewpoint) on one of the planar views (transverse, sagittal, coronal) and then walking back in a straight line in the VB view. However, this approach would involve at least two views, and it might be difficult to undo the actual step and return to the previous position. Additionally, users will find a virtual camera to navigate outside the airway.

According to an embodiment of the present disclosure, forward movement including turns is recorded by storing each individual step in a stack, and when the virtual camera moves backwards, its backward steps are taken from the stack, thus maintaining Both the actual position and orientation of the virtual camera. Forward movement is performed by moving in a straight line. Lateral movement is performed by manipulating a data input device such as a mouse pointer at a location on the VB view, and the center of the VB view is updated to that location in an animated manner to allow for a better user experience.

When the virtual camera view turns (to the right or left) while moving forward in response to moving an on-screen cursor or pointer and/or selection buttons by a user operating a user input device such as a mouse or touchpad, After the linear movement has been performed, the steering takes place. The software stores one step or several steps forward and then steps with turnarounds in a stack. The steps of turning are performed with a single animation to provide smooth turning. The animation is calculated based on the virtual camera's current position, current orientation, new rotation axis, and delta (ie, change in left or right direction) in the view's 2D x-axis.

In order to provide a similar user experience when moving in the backward direction, systems and methods according to the present disclosure after the cursor or pointer has been moved by the user via the operation of a data input device such as a mouse, touchpad, trackball or touchscreen , to record the position of the cursor or pointer on the display device. As discussed herein, the x-coordinate and y-coordinate of the position of the cursor or pointer on the display device are stored in the stack and used to determine the forward and backward movement of the virtual camera. When the virtual camera moves backwards, the current position and orientation at the current position, along with the previous position and orientation are taken from the stack. During or before the backward movement, an average direction vector or other smooth vector optimized for backward navigation is computed for two adjacent steps in the stack (eg, steps i and i-1). In some embodiments, a smoothing vector is applied to the first vector and the second vector based on a spline or Lanczos algorithm. In the backward movement, a virtual camera view corresponding to the calculated mean direction vector or other smoothing vector is used. This allows for a smoother animation when performing a turn while moving backwards.

Referring now to FIG. 1 , the present disclosure is generally directed to a path planning system 10 and method for planning a path through a patient's anatomical lumen network for use during surgery. Path planning system 10 may include a computing device 100, such as, for example, a laptop computer, desktop computer, tablet computer, or other similar device having a display 102, memory 104, one or more processors 106, and/or generally Other types of parts found in computing devices. Display 102 may be touch sensitive and/or voice activated, enabling display 102 to function as both an input device and an output device. Alternatively, a keyboard 113, mouse 114 or other data entry devices may be employed.

Memory 104 includes any non-transitory, computer-readable storage medium for storing data and/or software executable by processor 106 and controlling the operation of computing device 100 . In an embodiment, memory 104 may include one or more solid-state storage devices, such as flash memory chips. In an alternative embodiment, memory 104 may be a mass storage device connected to processor 106 through a mass storage controller (not shown) and a communication bus (not shown).

Although descriptions of computer-readable media contained herein refer to solid-state storage, those skilled in the art will recognize that computer-readable storage media can be any available media that can be accessed by processor 106 . That is, computer-readable storage media includes non-transitory, volatile and non-volatile, and Removable and non-removable media. For example, computer readable storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROM, DVD or other optical storage, magnetic tape cartridges, magnetic tape, magnetic disk storage or other magnetic storage devices, or can be used to Any other medium that stores desired information and that is accessible by computing device 100 .

Computing device 100 may also include network module 108 that connects to a distributed network or the Internet via wired or wireless connections for sending and receiving data to and from other sources. For example, computing device 100 may receive computed tomography (CT) images of a patient from a server, such as a hospital server, Internet server, or other similar server, for use during path planning. Patient CT images may also be provided to computing device 100 via memory 104, which may be a removable memory.

Path planning module 200 includes a software program stored in memory 104 and executed by processor 106 of computing device 100 . As will be described in more detail below, the path planning module 200 guides the clinician through a series of steps to develop a path plan for later use in a medical procedure. The path planning module 200 is in communication with a user interface module 202 for displaying visual interactive functions to the clinician on the display 102 and for receiving clinician input.

As used herein, the term "clinician" refers to any medical professional (e.g., physician, surgeon, nurse, etc.) or physician involved in the planning, execution, monitoring, and/or supervision of the use of the systems, methods, and devices described herein. Other users of the path planning system 10 involved in the medical procedure of an embodiment.

Referring now to FIG. 2 , in an embodiment, path planning using path planning module 200 may be performed in four distinct phases. In a first stage S1, the clinician selects patients for path planning. In the second phase S2, the clinician adds the target. In a third stage S3, the clinician creates a path to the target. Finally, in a fourth stage S4, the clinician reviews and accepts the plan, and can export the plan for use in the medical procedure. The clinician may repeat either or both of the second stage S2 and the third stage S3 as desired to select additional goals and/or create additional pathways for a particular patient. For example, a clinician can select additional goals and can create a route to each goal. The clinician may also or alternatively create multiple paths to the same target. Once the path plan is generated, a virtual bronchoscopic view (shown in FIGS. 6A-6C ) can be displayed, which allows the clinician to navigate within the patient's virtual airway based on the path plan.

FIG. 3 is a flowchart illustrating a method 300 for dynamically displaying a virtual bronchoscopic view based on position and orientation information of a virtual camera, according to an embodiment of the present disclosure. At steps 305-310, the location within the virtual bronchoscopic view on the display 102 pointed to by the data input device 112 is displayed. In some embodiments, the location is provided in a known airway, such as the patient's trachea or entry point. The location pointed on the display 102 by the data input device 112 is obtained as 2D x-coordinates and y-coordinates. At step 315, the virtual location is utilized in order to generate a virtual camera view at the virtual location. As the clinician advances through the patient's airway in the virtual bronchoscopic view, the clinician clicks or selects different locations within the airway to move forward and backward.

At step 310, when a virtual location on display 102 pointed to by data input device 112 (eg, by using a pointer or cursor) reaches location Li (as further described below with respect to FIG. 7 _{), view direction D i is} utilized. Based on the location L _i , the one or more processors 106 determine a view direction D _{i corresponding to the location L i .} The location _Li is a 2D coordinate having an x-coordinate and a y-coordinate relative to the position of the airway on the display 102 .

The view direction D _i can be expressed as a vector with magnitude and angle Θ _i The angle Θ _i can be defined as the current vector The angle of and the previous vector The difference between the angles, as shown in Figure 7. For example, for a _first vector extending from a first location L1 to a _second location L2 and to be orthogonal to the first vector The direction of the second vector extending from the second location L ₂ second vector The angle Θ ₂ is 90°.

At step 315 , once the location L _i and view direction D _i are determined, the one or more processors 106 retrieve a virtual camera view C _i from memory 104 based on the location L _i and view direction D _i . The virtual camera view _Ci is a 2D virtual image to be displayed in the virtual bronchoscope window 600 (shown in FIGS. 6A , 6B and 6C). The virtual camera view _Ci is the view of the interior of the airway from the point of view of the tip of the virtual camera. Examples of virtual camera views C _i are shown in FIGS. 6A-6C .

At step 320, the one or more processors 106 store the first position L _i and the view direction D _i in the memory 104, eg, in a stack or lookup table at step S _i . An example of a look-up table (LUT) is shown below.

Table 1

As used herein, the term position refers to coordinate values and data indicating the position of a pointer or cursor on display 102 . The view direction refers to the angular difference between the view along a line from the current position (first vector) and the view along a line from a previous position (second vector), as further shown in FIG. 7 .

At step 325, the virtual camera view _Ci is displayed in the virtual bronchoscope window 600 on the display 102, as shown in FIGS. 6A-6C. In some embodiments, the user can change and update the virtual camera view C _i by clicking a button on the mouse 114 or pressing a key or combination of keys on the keyboard 113 . For example, the user can change orientation, thereby displaying a new virtual camera view in a different view direction.

Move forward

Steps 330 - 355 of FIG. 3 illustrate movement of the virtual camera in a forward direction through the airway. At step 330 , the next position L _i+1 and the next view direction D _i+1 of the virtual camera are obtained based on the position of the pointer or cursor on the display 102 . The next view direction D _i+1 may be determined by determining the direction of a vector extending from location L _i to the next location L _i+1 . At step 335, a determination is made as to whether the virtual camera has moved in a previous direction from position _Li to the next position _Li+1 , as described below with reference to FIG. 4 .

If it is determined that the virtual camera moved from position L _i to the next position L _i+1 in the forward direction, the method proceeds to step 340 . At step 340, once the next position L _i+1 and the view direction D _i are both determined, the one or more processors 106 retrieve the values from memory based on the next position L _i+1 and the next view direction D _i+1 In step 104, the next virtual camera view C _i+1 is obtained.

At step 345 , the one or more processors 106 store the next position L _i+1 and the next view direction D _i+1 in memory 104 at next step S _i+1 . At step 350 , the next virtual camera view C _i+1 is displayed in the virtual bronchoscope window 600 on the display 102 . At step 355, the one or more processors 106 set the current position L _{i+ 1 at the next step S i+1} as the current step S _i before returning to step 330 to determine the next position L _i+1 .

move backward

In some embodiments, if at step 335 it is determined that the virtual camera is not moving in a forward direction, then the method proceeds to step 360 . At step 360, it is determined whether the virtual camera is moving in a backward direction. Step 360 is used to confirm that the virtual camera is moving in the backward direction. Movement in the backward direction can be defined as moving to or near a previously visited location. If at step 360 it is determined that the movement is not in a backward direction, then the method returns to step 335 to determine whether the movement of the virtual camera is in a forward direction.

If at step 360 it is determined that the movement is in the backward direction, then the method proceeds to step 370 . Based on the next position L _i+1 in the backward direction, the one or more processors 106 access a look-up table in memory 104 to determine the step previously stored in the look-up table corresponding to the next position L _i+1 . For example, if at the next location L _i+1 the coordinates are x _i+1 , y _i+1 , the processor will access the lookup table shown above in Table 1 and determine the previous locations L _i and L _{i- 1} corresponds to steps S _i and S _i-1 , respectively. Once steps S _i and S _i-1 are determined, the one or more processors 106 obtain the view direction D _i at step S _i and the view direction D at step S _i-1 from Table 1 at step 370 _i-1 . Thus, for example, if it is determined that the movement of the virtual camera is in the backward direction and the position corresponds to step S5, the one or more processors 106 obtain the view direction D4 and the view direction D4 from the lookup table shown in Table _{1 . D3} .

At step 375, the one or more processors 106 calculate a smoothing vector V based on the view direction D _i−1 and the view direction D _i . As shown in Table 1 above, the view directions D _i-1 and D _i correspond to vector angles Θ _i-1 and Θ _i , respectively. In some embodiments, the smoothing vector is a new vector with angle ΘN _i+1 that bisects vector angles Θ _i−1 and Θ _i . Thus, for backward motion, a new view direction at position L _i+1 is created with a new angle equal to Θ _i /2 by applying a smoothing vector to angle Θ _i of view direction D _i .

In an embodiment, the smoothing vector V is a vector based on (1) the first or previous vector defined by position L _i-1 and direction D _i-1 and (2) the first vector defined by position L _i and direction D _i Two or the current vector, and optimize for backward navigation such that, for example, the camera view does not leave the airway and/or the animation of the camera view looks smooth. The smoothing vector V may be a vector between the first vector and the second vector, such as a vector that is an average or a weighted average of the first vector and the second vector. The smoothing vector V may alternatively be determined by using a smoothing algorithm such as a spline or Lanczos algorithm. The splines can be quadratic (splines of degree two) or cubic (splines of degree four).

After determining a new view direction DN _i+1 at position L _i+1 based on the smoothing vector, the one or more processors 106 determine a new virtual camera view DN _i +1 at position L _i+1 at step 380 ₁ . In some embodiments, location L _i+1 may be location L _i-1 or a location close to location L _i-1 . Therefore, instead of using the original virtual camera view C _i+1 at location L _i+ 1 , the one or more processors 106 obtain the original virtual camera view C _i+1 from the memory 104 and pass The line of sight direction D _i+1 changes the virtual camera view C _i+1 , and generates a new virtual camera view CN _i+1 . At step 385 , a new virtual camera view DN _i+1 is displayed on the display 102 in the virtual bronchoscope window. After step 385, the one or more processors 106 set the current position L _i+1 for step S _i +1 as step S in step 390 before returning to step 330 to determine the next position L _{i+1 i} .

Turning now to FIG. 4 , a flowchart illustrating a method 400 for determining the forward and backward motion of steps 335 and 360 of FIG. 3 is depicted in more detail. Using the current position L _i and the previous position L _i-1 of the pointer or cursor on the display 102 , a determination can be made in which direction the virtual camera is moving forward or backward at the current position L _i .

At step 405 , the one or more processors 106 _obtain the 2D x-coordinates and y-coordinates of the pointer's current location Li on the display 102 . Next, at step ₄₁₀ , the _one or more processors 106 obtain the 2D x-coordinates and y-coordinate.

At step 415, the one or more processors 106 determine whether the difference between the y-coordinate value of the next location _Li+1 of the pointer on the display ₁₀₂ and the y-coordinate value of the location Li is less than zero. If it is determined at step 415 that the difference between the y-coordinate value of the next position L _i+1 and the y-coordinate value of position L _i is less than zero, then the one or more processors 106 determine that the virtual camera is in a forward direction move.

If at step 415, it is determined that the difference between the y-coordinate value of position L _i+1 of the pointer on display 102 and the y-coordinate value of position L _i is not less than zero, then method 400 proceeds to step 420, where it is determined that Whether the difference between the y-coordinate value of position L _i+1 and the y-coordinate value of position L _i is greater than zero. If it is determined at step 415 that the difference between the y-coordinate value of the next location _Li+1 and the _y -coordinate value of location Li is greater than zero, then the one or more processors 106 determine that the virtual camera is in a backward direction move.

Referring now to FIGS. 5 and 6A-6C, a 3D map window 500 for a virtual bronchoscope is shown. When the goals and paths are identified by the computing device 100, the clinician may wish to review the paths in a navigation preview mode. 5 illustrates a navigation preview mode of the planning phase in which computing device 100 shows a 3D map window 500 and a virtual bronchoscope window 600 on the screen of display 102 ( FIGS. 6A-6C ), according to an embodiment of the disclosure.

3D map window 500 shows a 3D map and virtual bronchoscope window 600 shows a virtual bronchoscope video image. 3D map window 500 displays and overlays path 505 to target 550 and current position indicator 507 . In the navigation preview mode, the display 102 shows the virtual bronchoscope window 600 as a fly-through view from the trachea to the target 550 .

The virtual bronchoscope window 600 also shows a path 660 towards the target 550 for review. The current position indicator 507 moves in the 3D map window 500 based on and according to the current position shown in the virtual bronchoscope window 600 . In one aspect, the route 660 or 505 may not be displayed based on a display option that the clinician may set between displaying the route or not displaying the route.

The virtual bronchoscope window 600 includes a slider 670 for transparency. By moving slider 670, the opacity of the virtual bronchoscopic video image can be changed from opaque to transparent. However, the transparency state of the virtual bronchoscope is not synchronized with the 3D map shown in the 3D map window 500 .

As shown in FIG. 5 , the airway channel 501 contains three camera views at three respective locations 510 , 520 and 530 . View directions 510a, 520a, and 530a are displayed in virtual bronchoscope window 600 as the virtual camera navigates to target 550 at each location 510, 520, and 530, respectively. Each view direction 510a, 520a, and 530a corresponds to the video image of the virtual bronchoscope shown in Figures 6A, 6B, and 6C, respectively. As the virtual camera advances from position 510 to 530, the view seen by the user is changed, as shown in FIGS. 6A-6C respectively.

In FIG. 6A , a user is able to see a path 660 containing locations 510 , 520 , and 530 , along with the bifurcated branches of airway passage 501 , in virtual bronchoscope window 600 . As the virtual camera approaches location 520 (FIG. 6B), the view shown in virtual bronchoscope window 600 is approaching the bifurcation branch. Once the virtual camera reaches position 530 (FIG. 6C), the airway passage is displayed at position 530. During forward movement, the user utilizes an input device such as a mouse, keyboard, or touchpad to move a pointer or cursor and select forward locations along path 660 , such as locations 510 , 520 and 530 . Thus, when the user notices the forking branch, the user can select a location along path 660 to enter the forking branch, such as location 530 . When each location is selected during forward movement, the virtual camera centers the view at that location. During backward movement, when the user moves the pointer or cursor and selects a backward position along path 660, the systems and methods of the present disclosure change and smooth the view at the backward position instead of displaying in the backward position The virtual camera view is centered, thereby providing the user with a camera view that prevents the view that the user sees in the virtual bronchoscope window 600 from being outside the airway. For example, for a user who chooses to move backwards in the airway of FIG. 6C , the systems and methods of the present disclosure would prevent the view seen by the user from being displayed as if the virtual camera was outside the airway at the bifurcated branch of the airway.

FIG. 7 is a graphical illustration 700 of airways and vectors associated with movement within the airways of a virtual bronchus view. Graphical diagram 700 shows vectors and smoothed vectors at different locations within the airway, such as locations 510 , 520 , and 530 of FIG. 6 . As shown in FIG. 7 , the airway channel 501 contains five locations L ₀ -L ₄ (705-740) and corresponding viewing directions D ₀ -D ₄ (705a-740a). Each view direction 705a-740a is shown as a solid line vector extending from a location 705-740.

In the case of forward movement, as the virtual camera advances from position _L0 to position L4, the virtual camera views as seen by the user at each position _705-740 are in directions along view directions 705a-740a, respectively be shown.

In the case of moving backwards, eg, traversing positions 740-730-720, each view direction displayed at the previous position (eg, 730a and 720a) is changed by a smooth vector. For example, for a smooth vector that is the average of the current vector and the previous vector at the current location, the changed view direction 730c is shown by a dashed line and is shown as the bisector of 730a and 730b. The changed view direction 730c is created as the bisector of the normal view direction 730a at location L ₃ 730 and the view direction at the previous location L ₂ 720 , shown by dashed line 730b extending from location L ₃ 730 .

Similarly, for backward movement from position L3 ₇₃₀ to position L2 ₇₂₀ , the changed view direction 720 is shown by dashed lines and is shown as the difference between the normal view direction _720a of L2 720 and the view direction _of the previous position L1 710. The bisector, which is generally shown by dashed line 720b extending from location L ₂ 720 .

Although the disclosure has been described in terms of specific illustrative embodiments, it will be apparent to those skilled in the art that various rearrangements and substitutions may be made without departing from the spirit of the disclosure. For example, the determination of the smoothing vector is shown above as being performed online or dynamically during the movement of the virtual camera. It is contemplated that in other embodiments, the smoothing vector may be determined off-line or prior to initiating the navigation mode at predetermined locations throughout the airway. Such other embodiments may be beneficial when more sophisticated methods are utilized to determine smoothing vectors or other vectors that optimize the virtual camera view when moving in a backward direction. The scope of the disclosure is defined by the appended claims.

In addition to the above, reference is also made to the following commonly assigned application which describes image processing and user interface updating features, among other features described in relation to the system described herein: Filed Jul. 2, 2014 U.S. Provisional Patent Application No. 62/020,240, entitled "System And Method For Navigating Within The Lung"; U.S. Provisional Patent Application No. 62/020,240, filed July 2, 2014, entitled "Real-Time Automatic Registration Feedback" .62/020,220; U.S. Provisional Patent Application No. 62/020,177, filed July 2, 2014, entitled "Methods for Marking Biopsy Location"; filed July 2, 2014, entitled "Unified Coordinate System For Multiple CT Scans Of Patient Lungs," U.S. Provisional Patent Application No. 62/020,242; U.S. Provisional Patent Application No. 62/020,245, filed July 2, 2014, by Klein et al., entitled "Alignment CT" ; U.S. Provisional Patent Application No. 62/020,250, filed July 2, 2014, entitled "Algorithm for Fluoroscopic Pose Estimation"; Patent Application No. 62/020,253; U.S. Provisional Patent Application No. 62/020,261, filed July 2, 2014, entitled "Lung And Pleura Segmentation"; filed July 2, 2014, entitled " U.S. Provisional Patent Application No. 62/020,258 for Cone View–A Method Of ProvidingDistance And Orientation Feedback While Navigating In 3D; filed July 2, 2014, entitled "Dynamic 3D Lung Map View for ToolNavigation Inside the Lung" U.S. Provisional Patent Application No. 62/020,262; U.S. Provisional Patent Application No. 62/020,262, filed July 2, 2014, entitled "System and Method for Segmentation of Lung" 020,261; U.S. Provisional Patent Application No. 62/020,257, filed July 2, 2014, entitled "Automatic Detection Of Human LungTrachea"; all references to processing DICOM images, detecting trachea, navigating in the lungs, and displaying DICOM images and aspects of processing images to provide enhanced clarity and performance for analysis, diagnosis and treatment systems related to lung therapy planning and navigation, among others. The contents of all above-cited applications are hereby incorporated by reference.

Claims (20)

CLAIMS 1. A method for displaying a virtual bronchoscope view while navigating through the airway of the virtual bronchoscope, the method comprising: determining a first location and a first direction at the first location; storing the first position and the first orientation in memory; displaying a first virtual camera view corresponding to the first location; determining a second location corresponding to the airway moved through the virtual bronchoscope; storing the second location in memory; displaying a second virtual camera view corresponding to the second location; determining a second direction based on the first location and the second location; storing the second orientation in memory; determining a third location corresponding to further movement through the virtual bronchoscope; and determining whether the further movement is in a forward direction or in a backward direction; If it is determined that the further movement is in the forward direction: then displaying a third virtual camera view corresponding to the third location; and If the movement is determined to be in the backward direction: then retrieving the stored first direction and the stored second direction from the memory; determining a smoothing vector based on the stored first direction and the stored second direction; obtaining a smoothed virtual camera view based on the smoothing vector; and Displays the smoothed virtual camera view. 2. The method of claim 1, wherein determining whether the further movement is in a forward direction or a backward direction comprises: Determine the next and current pointer positions on the virtual bronchoscope screen; Calculate the difference between the coordinates of the next pointer position and the coordinates of the current pointer position; If the calculated difference is positive, determining that the further movement is in a backward direction; and If the calculated difference is negative, it is determined that the further movement is in a forward direction. 3. The method of claim 1, wherein the first location, the second location, and the third location are determined based on the location of a pointer or cursor on a screen displaying the virtual camera view. 4. The method of claim 1, further comprising, if it is determined that the further movement is in a forward direction: then determining a third direction based on the second location and the third location; and Store the third orientation in memory. 5. The method of claim 1, wherein determining a smoothing vector comprises: determining a first vector based on the first position and the first orientation; determining a second vector based on the first position and the second orientation; and Average the first vector and the second vector to obtain a smoothed vector. 6. The method of claim 1, wherein determining a smoothing vector comprises: determining a first vector based on the first position and the first orientation; determining a second vector based on the first position and the second orientation; and Applies a spline or Lanxesian algorithm to the first and second vectors to obtain smoothed vectors. 7. The method of claim 6, wherein the spline is a second order spline or a fourth order spline. 8. The method of claim 1, further comprising: receiving input from a user to change the first virtual camera view, the second virtual camera view, the third virtual camera view, or the smoothed virtual camera view; and The changed first virtual camera view, second virtual camera view, third virtual camera view or smoothed virtual camera view is stored. 9. The method of claim 1, wherein determining a smoothing vector comprises determining a vector having a direction between a first direction and a second direction. 10. An apparatus for displaying a view of a virtual bronchoscope while navigating through the airway of the virtual bronchoscope, comprising: a network interface configured to receive location information of the navigation instrument from at least one location sensor of the navigation instrument, the location information including the physical location; a memory storing a plurality of virtual camera views of the virtual bronchoscope, instructions, a first position, a first direction at the first position, a second position, and a second direction at the second position; processor configured to execute instructions, wherein when the instruction is executed by the processor, the processor: Determine whether the airway moving through the virtual bronchoscope is in an anterior or posterior direction; If it is determined that the movement is in the forward direction: then determining a third location corresponding to the airway moved through the virtual bronchoscope; and determining a third direction based on the second location and the third location; and If the movement is determined to be in the backward direction: the first direction and the second direction are then retrieved from storage; and determining a smoothing vector based on the first direction and the second direction; and A display configured to dynamically display on the screen an image of the smoothed virtual camera view corresponding to the determined vector smoothing. 11. The apparatus of claim 10, wherein the first location, the second location, and the third location are determined based on a location of a pointer on the screen. 12. The apparatus of claim 10, wherein the instructions, when executed by the processor, further cause the processor to: determining a first pointer location and a second pointer location on a screen displaying the virtual camera view; calculating the difference between the coordinates of the first pointer location and the coordinates of the second pointer location; If the calculated difference is positive, it is determined that the movement is in the backward direction; and If the calculated difference is negative, it is determined that the movement is in the forward direction. 13. The apparatus of claim 10, wherein the instructions, when executed by the processor, further cause the processor to: determining a first vector based on the first position and the first orientation; determining a second vector based on the first position and the second orientation; and Average the first vector and the second vector to obtain a smoothed vector. 14. The apparatus of claim 10, wherein the instructions, when executed by the processor, cause the processor to determine a smoothing vector having a direction between the first direction and the second direction. 15. The apparatus of claim 10, wherein the instructions, when executed by the processor, cause the processor to: determining a first vector based on the first position and the first orientation; determining a second vector based on the first position and the second orientation; and Applies splines to the first and second vectors to obtain smoothed vectors. 16. The apparatus of claim 15, wherein the spline is a second order spline or a fourth order spline. 17. A method for displaying a virtual bronchoscope view while navigating through the airway of the virtual bronchoscope, the method comprising: determining a first location and a first direction at the first location; storing the first position and the first orientation in memory; displaying a first virtual camera view corresponding to the first location; determining a second position and a second orientation corresponding to the airway moving through the virtual bronchoscope; determining a smoothing vector based on the stored first and second directions; store the smoothing vector in memory; displaying a second virtual camera view corresponding to the second location; determining a third location corresponding to the airway moved further through the virtual bronchoscope; and determining whether the further movement is in a forward direction or in a backward direction; If it is determined that the further movement is in the forward direction: then displaying a third virtual camera view corresponding to the third location; and If it is determined that the movement is in the backward direction: then retrieve the smoothing vector stored in memory; obtaining a smoothed virtual camera view based on the retrieved smoothing vector; and Displays the smoothed virtual camera view. 18. The method of claim 17, wherein determining a smoothing vector comprises: determining a first vector based on the first position and the first orientation; determining a second vector based on the first position and the second orientation; and Average the first vector and the second vector to obtain a smoothed vector. 19. The method of claim 17, wherein determining a smoothing vector comprises: determining a first vector based on the first position and the first orientation; determining a second vector based on the first position and the second orientation; and Applies a spline or Lanxesian algorithm to the first and second vectors to obtain smoothed vectors. 20. The method of claim 19, wherein the spline is a second order spline or a fourth order spline.