Disclosure of Invention
The invention aims to provide a surgical instrument rollback control method, a readable storage medium and a surgical robot system, which are used for solving the problem that the existing surgical instrument is easy to collide with tissues or mutually collide when being withdrawn.
In order to solve the technical problems, the invention provides a surgical instrument rollback control method, which comprises the steps of performing key point dispersion according to the form of the surgical instrument before rollback, and reconstructing according to the discrete key points to obtain spline lines;
Constructing a cost function based on the distance between each key point and the spline line when the surgical instrument is in a form in a rollback process;
finding out the instruction speed of the elbow and wrist joint when the minimum value is obtained by meeting the time derivative of the cost function;
And integrating the command speed to obtain a command position of the elbow and wrist joint, and controlling the surgical instrument to retract based on the command speed and the command position.
Optionally, the cost function includes a weight of each of the keypoints, the weight of each of the keypoints being set according to a mode type, the mode type being set based on an end type of the surgical instrument.
Optionally, the mode types include an end optimization mode and a global optimization mode;
When the mode type is an end optimization mode, the weight of the key points at the far end is not smaller than that of the key points at the near end;
and when the mode type is a global optimization mode, the weights of all the key points are the same.
Optionally, after obtaining the commanded position of the elbow and wrist joint, the surgical instrument retraction control method further includes:
filtering the command speed and the command position;
Evaluating the filtered cost function, if the cost function exceeds a preset threshold, feeding back abnormal information, and stopping controlling the surgical instrument to fall back;
and if the cost function is within the preset threshold, controlling the surgical instrument to retract according to the filtered instruction speed and the filtered instruction position.
To solve the above technical problem, the present invention also provides a readable storage medium having stored thereon a program which, when executed, implements the steps of the surgical instrument retraction control method as described above.
In order to solve the technical problems, the invention also provides a surgical robot system which is characterized by comprising a driving module and an elbow and wrist joint optimization module, wherein the elbow and wrist joint optimization module comprises a cost optimization module;
the cost optimization module is configured to perform key point dispersion according to the form of the surgical instrument before rollback, and obtain spline lines according to the key point reconstruction after dispersion; when the surgical instrument is in a form in a rollback process, constructing a cost function based on the distance between each key point and the spline line, finding out an instruction speed of the elbow and wrist joint when the minimum value of the derivative of the cost function with respect to time is obtained;
the drive module controls retraction of the surgical instrument based on the commanded speed and the commanded position.
Optionally, the elbow and wrist joint optimization module further includes a mode selection module, where the mode selection module is configured to select a mode type, and configure different weights for each of the key points in the cost function according to the selected mode type.
Optionally, the mode types include an end optimization mode and a global optimization mode;
When the mode type is an end optimization mode, the weight of the key points at the far end is not smaller than that of the key points at the near end;
and when the mode type is a global optimization mode, the weights of all the key points are the same.
Optionally, the elbow and wrist joint optimization module further comprises an optimization verification module, wherein the optimization verification module is configured to filter the instruction speed and the instruction position after the instruction position of the elbow and wrist joint is obtained by the cost optimization module;
And if the cost function is within the preset threshold, the driving module controls the surgical instrument to fall back according to the instruction speed and the instruction position filtered by the optimization verification module.
Optionally, the surgical robot system further comprises an instrument exit triggering module and a telescopic joint planning module;
the exit triggering module is used for triggering the elbow and wrist joint optimizing module and the telescopic joint planning module;
the telescopic joint planning module is used for planning joint track of the positions and the speeds of the telescopic joints.
In summary, in the surgical instrument rollback control method, the readable storage medium and the surgical robot system, the surgical instrument rollback control method comprises the steps of performing key point dispersion according to a form of a surgical instrument before rollback, reconstructing according to the key points after the dispersion to obtain a spline line, constructing a cost function based on the distance between each key point and the spline line when the surgical instrument is in the form of rollback, finding out an instruction speed of the elbow wrist joint when the minimum value of the derivative of the cost function with respect to time is found out, integrating the instruction speed to obtain an instruction position of the elbow wrist joint, and controlling the surgical instrument to rollback based on the instruction speed and the instruction position.
With such configuration, the surgical instrument can be retracted along a path formed according to the form before retraction, so as to effectively reduce the possibility of collision during retraction of the surgical instrument, and the surrounding environment is not required to be perceived, and additional image support is not required.
Detailed Description
The invention will be described in further detail with reference to the drawings and the specific embodiments thereof in order to make the objects, advantages and features of the invention more apparent. It should be noted that the drawings are in a very simplified form and are not drawn to scale, merely for convenience and clarity in aiding in the description of embodiments of the invention. Furthermore, the structures shown in the drawings are often part of actual structures. In particular, the drawings are shown with different emphasis instead being placed upon illustrating the various embodiments.
The terms "proximal" and "distal" are defined herein with respect to a surgical instrument having an end for insertion into a human body and an end extending outside the body and connected to a robotic arm. The term "proximal" refers to the end of the surgical instrument that extends beyond the body, and the term "distal" refers to the end of the surgical instrument that is closer to the body. Alternatively, in a manual or hand-operated application scenario, the terms "proximal" and "distal" are defined herein with respect to an operator, such as a surgeon or clinician. The term "proximal" refers to a location closer to the operator, and the term "distal" refers to a location closer to the surgical instrument and thus farther from the operator.
The invention aims to provide a surgical instrument rollback control method, a readable storage medium and a surgical robot system, which are used for solving the problem that the existing surgical instrument is easy to collide with tissues or mutually collide when being withdrawn. The following description refers to the accompanying drawings.
Referring to fig. 1, there is exemplarily shown a surgical robot system including a doctor console 10 and a patient-side cart 20, the doctor console 10 and the patient-side cart 20 being configured in a master-slave control relationship, i.e., the doctor console 10 is a master end, the patient-side cart 20 is a slave end, and an operation of the doctor console 10 can be mapped to the patient-side cart 20, thereby realizing master-slave teleoperation to perform a surgery. Further, the patient end trolley 20 includes a robotic arm 21 and an end instrument assembly 22 mounted on and connected to the robotic arm 21.
The surgical robotic system shown in fig. 1 is a single-hole surgical robotic system, and the end instrument assembly 22 performs a surgical procedure through only one puncture opening in the patient's body surface. Referring specifically to fig. 2, an end instrument assembly 22 mounted on and connected to a mechanical arm 21 is shown, the end instrument assembly 22 specifically includes a punch card 23 and a plurality of surgical instruments 24, the punch card 23 is used for penetrating a puncture hole on a body surface of a patient, the puncture hole has a through inner hole, and a distal end of the surgical instrument 24 extends into the patient through the inner hole of the punch card 23. Surgical instruments 24 include endoscopes and implement instruments (e.g., arc shears, graspers, etc.).
Referring to fig. 2 and 3, the surgical instrument 24 has a generally serpentine-shaped deployed configuration after extending into the patient through the bore of the punch 23. Optionally, the surgical instrument 24 includes an elbow and wrist joint 242 and a telescopic joint 241. During surgery, the telescopic joint 241 is used to adjust the overall advancement and retraction of the surgical instrument 24, and the elbow and wrist joint 242 is used to adjust the end position and instrument pose of the surgical instrument 24.
Referring to fig. 4, in some application scenarios, for example, after tissue dissociation is completed, a passive instrument needs to be replaced with an active instrument to perform an electro-cutting and electro-coagulation operation, and at this time, the surgical instrument 24 needs to be retracted to a position near the uppermost end of the punch card 23, so as to facilitate removal of the surgical instrument 24. The dashed lines in fig. 4 indicate the configuration of the surgical instrument 24 prior to retraction. With continued reference to fig. 3, since the surgical instrument 24 is deployed in a generally serpentine-shaped configuration within the patient, it is susceptible to collisions with other surgical instruments 24 or tissue within the patient when retracted (as moved in the direction of the arrow in fig. 3).
Based on this, an embodiment of the present invention provides a surgical instrument retraction control method, including:
Step S1, performing the dispersion of the key points 31 according to the form of the surgical instrument 24 before the retraction, and reconstructing the spline 32 according to the key points 31 after the dispersion. Referring to fig. 5a and 5b, keypoints 31 are disposed discretely along the axial direction of the surgical instrument 24 at intervals, and the keypoints 31 are consolidated with the surgical instrument 24, the keypoints 31 moving with the movement of the surgical instrument 24. When the surgical instrument 24 receives the retraction instruction, the coordinate values of each key point 31 at the current time are recorded, as shown in fig. 5 a. Further, spline lines 32 in space can be reconstructed by spatial spline interpolation based on the coordinate values of the respective key points 31 at this time, as shown in fig. 5 b. It will be appreciated that the collision of the surgical instrument 24 with surrounding tissue or other surgical instrument 24 may be effectively reduced provided that the surgical instrument 24 is retracted along the spline 32.
Step S2, constructing a cost function based on the distance between each key point and the spline line when the surgical instrument 24 is in the form of a rollback process. As shown in fig. 5c, in order to make the morphology of the surgical instrument 24 during the retraction as much as possible within the contour range of the morphology before the retraction, a response evaluation index may be implemented by constructing a cost function, so as to evaluate the difference between the morphology during the retraction and the contour range of the morphology before the retraction. The cost function can be constructed and calculated by the distance between each key point 31 and the spline 32 in the rollback process.
And step S3, finding the instruction speed of the elbow joint 242 when the minimum value is obtained by the derivative of the cost function with respect to time. After the cost function is defined, the real-time control requirement can be met while optimizing the cost function by searching the instruction speed of the elbow joint 242.
Step S4, integrating the command speed to obtain a command position of the elbow and wrist joint 242, and controlling the surgical instrument 24 to retract based on the command speed and the command position. In practice, the commanded velocity of the wrist joint 242 found at each time during retraction of the surgical instrument 24 is integrated to obtain the commanded position of the wrist joint 242.
Referring to fig. 6, steps S1 to S4 are optimizing the retraction motion trajectory of the elbow and wrist joint 242, and each control step of the elbow and wrist joint 242 performs the motion according to the command speed and command position obtained by optimizing the steps S1 to S4, so that the surgical instrument 24 can be enabled to substantially follow the path formed according to the form before retraction when retracting, the possibility of collision generated when retracting the surgical instrument 24 can be effectively reduced, and the surrounding environment does not need to be perceived, and no additional image support is required.
The construction of the cost function in step S2 and the specific steps for finding the commanded velocity of the wrist joint 242 in step S3 will be further described with reference to the accompanying drawings.
Referring to fig. 7, assuming that the number of keypoints 31 is N, the sequence numbers of the keypoints 31 from the far-end to the near-end are arranged from small to large, the cost function can be defined as:
Where dist (A, spline b) represents the shortest distance from point A to spline b, as shown in FIG. 7. q is the generalized coordinate vector of the joint of the surgical instrument 24 (including the telescopic joint 241 and the elbow wrist joint 242), spline is spline 32, P i represents the coordinates of the ith keypoint 31, and w i is the weight of the ith keypoint 31 in the cost function (which is a constant scalar).
It should be noted that the cost function shown here is only an exemplary example and is not a limitation on the way the cost function is constructed. Those skilled in the art can construct different cost functions according to the prior art and the actual requirements.
Finding the commanded velocity of elbow wrist joint 242Can be converted into a solution to the mathematical problem of giving the current telescopic joint 241 position q 1, velocityThe current position q ew, of the elbow wrist joint 242 is foundSo that
Wherein, As a derivative of the cost function with respect to time, f constriant is a constraint vector for the surgical instrument 24 (e.g., instrument joint angle constraint),Maximum speed allowed for elbow wrist 242.
The present embodiment aims at searchingThe specific method of (a) is not limited, and for example, a common algorithm in the art such as "gradient method", "steepest descent method" or "genetic algorithm" may be used.
Further, the commanded velocity of the wrist 242 is determined for each instant during retraction of the surgical instrument 24The integration is performed to obtain the commanded position of elbow joint 242, as shown in FIG. 8.
Optionally, the cost function includes a weight w for each of the keypoints 31, the weight w for each of the keypoints 31 being set according to a mode type, the mode type being set based on an end type of the surgical instrument 24.
The surgical instrument 24 includes a plurality of different types, particularly where there are different injuries to the distal end (i.e., tip) of the surgical instrument 24, and the focus of the back-off control optimization is different for different injuries of the surgical instrument 24. Optionally, the mode types include an end optimization mode and a global optimization mode.
If the distal end of the surgical instrument 24 is injurious (e.g., an arc shears or the like) which may be harmful to the human body, then a tip optimization mode is employed where the weight w of the keypoint 31 at the distal end is not less than the weight w of the keypoint 31 at the proximal end. So for i=1, 2..n-1 needs to meet w i≥wi+1, andFor example, w 1 =1 is preferable, and the weights of the remaining keypoints 31 are all set to 0. In the tip optimization mode, the focus of the retraction control optimization is on the trajectory of the distal end (i.e., tip) of the surgical instrument 24 to ensure that the retraction trajectory of the injurious distal portion does not exceed the contour of the morphology of the surgical instrument 24 prior to retraction as much as possible.
If the distal end of the surgical instrument 24 is not or relatively less invasive (e.g., grasper, endoscope, etc.), then a global optimization mode is employed where the weight w of all of the keypoints 31 is the same. w i = 1/N, i = 1,2,..n. In the global optimization mode, the focus of the retraction control optimization is on the overall configuration of the instrument so that the overall configuration does not exceed the contour of the morphology of the surgical instrument 24 prior to retraction as much as possible.
Optionally, after obtaining the commanded position of the elbow wrist joint 242, the surgical instrument retraction control method further includes:
Step S5, filtering the instruction speed and the instruction position;
step S6, evaluating the filtered cost function, if the cost function exceeds a preset threshold, feeding back abnormal information, and stopping controlling the surgical instrument 24 to retract;
And if the cost function is within the preset threshold, controlling the surgical instrument 24 to retract according to the filtered command speed and the command position.
And step S5 and step S6 are used for filtering and smoothing the optimization results of the steps S1-S4, and verifying the filtering and smoothing results. For each control step, instruction filtering is performed for the instruction speed and the instruction position, the filtered cost function is reevaluated, the preset threshold can be set according to requirements, if the filtered cost function exceeds the preset threshold, the optimization is abnormal, and at the moment, the movement of the telescopic joint 241 and the elbow joint 242 can be stopped, and the control of the retraction of the surgical instrument 24 is stopped. Further, the filter can be reset, and the operator is prompted that the path optimization fails in a mode of image, force sense, sound and the like so as to perform optimization again or manually execute the return operation.
Based on the surgical instrument rollback control method, the embodiment of the invention also provides a surgical robot system which comprises a driving module and an elbow and wrist joint optimization module, wherein the elbow and wrist joint optimization module comprises a cost optimization module;
The cost optimization module is configured to perform dispersion of key points 31 according to a form of the surgical instrument 24 before retraction, reconstruct spline lines 32 according to the discrete key points 31, construct a cost function based on distances between the key points 31 and the spline lines 32 when the surgical instrument 24 is in the form during retraction, find an instruction speed of the elbow wrist joint 242 when a minimum value is obtained by a derivative of the cost function with respect to time is found, integrate the instruction speed to obtain an instruction position of the elbow wrist joint 242, and control retraction of the surgical instrument 24 by the driving module based on the instruction speed and the instruction position.
Optionally, the elbow and wrist joint optimization module further includes a mode selection module, where the mode selection module is configured to select a mode type, and configure different weights for each of the keypoints 31 in the cost function according to the selected mode type. Further, the pattern types include an end optimization mode and a global optimization mode, when the pattern type is the end optimization mode, the weight of the key point 31 at the far end is not smaller than the weight of the key point 31 at the near end, and when the pattern type is the global optimization mode, the weights of all the key points 31 are the same.
Optionally, the elbow and wrist joint optimization module further includes an optimization verification module configured to filter the instruction speed and the instruction position after the cost optimization module obtains the instruction position of the elbow and wrist joint 242, evaluate the filtered cost function, if the cost function exceeds a preset threshold, the optimization verification module feeds back abnormal information, the driving module stops controlling the surgical instrument 24 to roll back, and if the cost function is within the preset threshold, the driving module controls the surgical instrument 24 to roll back according to the instruction speed and the instruction position after the cost optimization verification module filters.
Optionally, the surgical robot system further comprises an instrument exit triggering module and a telescopic joint planning module, wherein the exit triggering module is used for triggering the elbow and wrist joint optimizing module and the telescopic joint planning module, and the telescopic joint planning module is used for planning the joint track of the position and the speed of the telescopic joint 241.
The exit trigger module may issue instrument exit instructions in a variety of ways including, but not limited to, performing single click, double click, long press, slide, etc. operations on the physician's console 10 or patient end trolley 20 via interactive on-screen or physical buttons. After receiving the instrument withdrawal instruction, the elbow and wrist joint optimizing module and the telescopic joint planning module begin to execute joint track planning on the position and the speed of the telescopic joint 241, and the elbow and wrist joint optimizing module optimizes the retraction track of the elbow and wrist joint 242 according to the position and the speed of the telescopic joint 241, so that the driving module drives the surgical instrument 24 to retract along the optimized track.
Optionally, the telescopic joint planning module performs the following two steps after receiving the instrument exit instruction:
Step SA1, recording the current command position of the telescopic joint 241, wherein the command position of the telescopic joint 241 is the starting point of the path planner, the final position of the path planner is a preset retraction position in the program, and the surgical instrument 24 can be conveniently replaced at the preset retraction position. Specifically, if the predetermined fallback position is lower than the current commanded position, the current commanded position may be directly configured as the predetermined fallback position.
And step SA2, carrying out path planning of maximum acceleration and maximum speed limitation. It will be appreciated that the path planning of the telescopic joint 241 is limited by the maximum speed and maximum acceleration. In some embodiments, the complete path plan is shown in FIG. 9, and the path includes three phases, an acceleration phase, a constant velocity phase, and a deceleration phase. In the acceleration stage, the speed is increased, the acceleration amplitude is required to be smaller than or equal to the maximum acceleration, the command speed in the constant speed stage is kept at the maximum speed, the command position is linearly increased, the command acceleration is zero, and in the deceleration stage, the speed is reduced, and the acceleration amplitude is required to be smaller than or equal to the maximum acceleration. Specifically, in other embodiments, when the path start point and the path end point are closely spaced, the maximum speed may not be reached, and at this time, there is no constant speed stage, and only the acceleration stage and the deceleration stage are reserved.
The embodiment of the invention also provides a readable storage medium, on which a program is stored, which when executed implements the steps of the surgical instrument rollback control method as above.
In summary, in the surgical instrument rollback control method, the readable storage medium and the surgical robot system, the surgical instrument rollback control method comprises the steps of performing key point dispersion according to a form of a surgical instrument before rollback, reconstructing according to the key points after the dispersion to obtain a spline line, constructing a cost function based on the distance between each key point and the spline line when the surgical instrument is in the form of rollback, finding out an instruction speed of the elbow wrist joint when the minimum value of the derivative of the cost function with respect to time is found out, integrating the instruction speed to obtain an instruction position of the elbow wrist joint, and controlling the surgical instrument to rollback based on the instruction speed and the instruction position. With such configuration, the surgical instrument can be retracted along a path formed according to the form before retraction, so as to effectively reduce the possibility of collision during retraction of the surgical instrument, and the surrounding environment is not required to be perceived, and additional image support is not required.
It should be noted that the above embodiments may be combined with each other. The above description is only illustrative of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention, and any alterations and modifications made by those skilled in the art based on the above disclosure shall fall within the scope of the present invention.