[go: up one dir, main page]

CN107932514A - Airborne equipment based on Robot Visual Servoing control mounts method - Google Patents

Airborne equipment based on Robot Visual Servoing control mounts method Download PDF

Info

Publication number
CN107932514A
CN107932514A CN201711348364.9A CN201711348364A CN107932514A CN 107932514 A CN107932514 A CN 107932514A CN 201711348364 A CN201711348364 A CN 201711348364A CN 107932514 A CN107932514 A CN 107932514A
Authority
CN
China
Prior art keywords
msub
mrow
mtd
robot
mfrac
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201711348364.9A
Other languages
Chinese (zh)
Inventor
潘辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin Jinhang Computing Technology Research Institute
Original Assignee
Tianjin Jinhang Computing Technology Research Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin Jinhang Computing Technology Research Institute filed Critical Tianjin Jinhang Computing Technology Research Institute
Priority to CN201711348364.9A priority Critical patent/CN107932514A/en
Publication of CN107932514A publication Critical patent/CN107932514A/en
Pending legal-status Critical Current

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls
    • B25J9/1694Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697Vision controlled systems
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00Controls for manipulators
    • B25J13/08Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls
    • B25J9/1602Programme controls characterised by the control system, structure, architecture
    • B25J9/1607Calculation of inertia, jacobian matrixes and inverses

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Automation & Control Theory (AREA)
  • Human Computer Interaction (AREA)
  • Manipulator (AREA)

Abstract

The invention discloses a kind of airborne equipment based on Robot Visual Servoing control to mount method, it includes the following steps:Step 1:Bomb hoist device people's end view picture is obtained, extracts characteristics of image;Step 2:Vision controller is designed according to characteristics of image;Step 3:The amount of exercise in each joint of robot is calculated according to robot kinematics;Step 4:Control law is generated, controls bomb truck movement that carry thing is moved at aircraft hanging point according to control law and is mounted.The present invention can effectively improve air weapon and mount efficiency;The present invention is applied widely, is suitable for the automatic of a variety of carry things of heterogenous and mounts.

Description

Airborne equipment hanging method based on robot vision servo control
Technical Field
The invention belongs to the technical field of pattern recognition and artificial intelligence, and relates to a robot vision servo control-based airborne equipment mounting method, which is mainly applied to mounting various missiles and bombs for combat aircrafts, can save manpower and material resources, and remarkably improve the efficiency of mounting various airborne weapons on a fighter plane.
Background
The robot vision servo control is an important technology in the field of pattern recognition and artificial intelligence, the robot has the capability of perceiving the external environment by introducing the vision sensor, the important application of multi-sensor information fusion and integration technology in the robot technology is realized, and compared with the robot control technology based on the traditional non-vision sensor, the robot vision servo control has higher flexibility and higher precision, has stronger environment self-adaption capability and can better complete various tasks.
The robot vision servo system provides external information for the robot through the vision sensor, so that the robot is controlled to adjust the pose in real time, accurate tracking or positioning is realized, and corresponding work is completed.
The current ground service personnel mainly still rely on the manpower to accomplish when hanging up various types of string loading thing for the warplane, need lean on the naked eye to observe the position, constantly manual adjustment string loading thing position and gesture, waste time and energy, efficiency is not high, and this is very likely to control the victory or defeat of a battle when the war, consequently, the research utilizes the automatic each type of string loading thing that has important practical meaning and engineering value for the warplane of string loading of missile vehicle-mounted robot.
Disclosure of Invention
Objects of the invention
The purpose of the invention is: a quick and effective automatic loading method of an airborne weapon based on a robot vision servo control method is provided, and the loading work of the loaded object can be quickly finished by using a robot.
(II) technical scheme
In order to solve the technical problem, the invention provides a robot vision servo control-based airborne equipment hanging method, which comprises the following steps:
the method comprises the following steps: acquiring an image of the tail end of the missile hooking robot, and extracting image characteristics;
step two: designing a visual controller according to the image characteristics;
step three: calculating the motion amount of each joint of the robot according to the kinematics of the robot;
step four: and generating a control law, and controlling the missile loading vehicle to move to the position of the aircraft hanging point for hanging according to the control law.
In the first step, an image of the tail end of the missile hanging robot is obtained, and feature points under a camera coordinate system are extractedcP=[Xc,Yc,Zc]TImage coordinates [ u, v ]]TAccording to the imaging model haveWherein f isxAnd fyIs the equivalent focal length of the camera; dotcP is varied in the vision sensor coordinate system, assuming that the point moves at a speed in the camera coordinate systemThen there isWhereincV=[cvx,cvy,cvz,]TcΩ=[cωx,cωy,cωz,]TAre respectively ascP linear and angular velocities in the camera coordinate system.
In the second step, derivation is carried out on the image coordinates in the first step, and then the following relation is obtained according to the second step:
wherein,referred to as the image jacobian matrix,cΨ=[cvx cvy cvz cωx cωy cωz]T
in the third step, the motion amount of each joint of the robot is calculated according to the kinematics of the robot;
converting the tail end characteristic points of the missile hanging robot manipulator into a tail end coordinate system of the robot manipulator under a camera coordinate system to obtain the motion amount of each joint,
whereinJrobotIs a generalized Jacobian matrix of the robot, S: (ePc) Is composed ofePcThe elements of (a) make up an anti-symmetric matrix,namely the required motion amount of each joint of the missile hooking robot.
In the fourth step, the amount of exercise of each joint of the missile hooking robot is given according to the third stepAnd designing a control law, and controlling the tail end of the robot to mount the mounted object to a specified position.
(III) advantageous effects
The robot vision servo control-based airborne equipment hanging method can effectively improve airborne weapon hanging efficiency; the invention has wide application range and is suitable for automatic hanging of various hung objects of various machines.
Drawings
Fig. 1 is a system block diagram of an airborne weapon automatic hanging method based on robot visual servo according to the invention.
Fig. 2 is a schematic structural diagram of an automatic missile hanging vehicle based on the method provided by the invention.
Detailed Description
In order to make the objects, contents, and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.
And acquiring a current hanging point image by using one camera, extracting characteristics, comparing the characteristics with the characteristics of a reference image, and continuously adjusting the pose of the hanging object according to a visual control law until the hanging object reaches a proper hanging position, wherein the image characteristics acquired by a visual sensor are completely the same as the characteristics of the reference image, so that the hanging work is completed.
The method needs teaching movement of the missile hanging robot for the first time, obtains a reference image when reaching a hanging point, and can control the robot to carry out hanging by taking the reference image as a reference when hanging the type of hung object later, and specifically comprises the following steps:
the method comprises the following steps: acquiring an image of the tail end of the missile hooking robot, and extracting image characteristics;
acquiring an end image of the missile hanging robot, and extracting feature points under a camera coordinate systemcP=[Xc,Yc,Zc]TImage coordinates [ u, v ]]TAccording to the imaging model haveWherein f isxAnd fyIs the equivalent focal length of the camera; dotcP is varied in the vision sensor coordinate system, assuming that the point moves at a speed in the camera coordinate systemThen there isWhereincV=[cvx,cvy,cvz,]TcΩ=[cωx,cωy,cωz,]TAre respectively ascP linear velocity and angular velocity under the camera coordinate system;
step two: designing a visual controller according to the image characteristics;
and (3) deriving the image coordinates in the first step, and obtaining the following relation according to the second step:
wherein,referred to as the image jacobian matrix,cΨ=[cvx cvy cvz cωx cωy cωz]T
step three: calculating the motion amount of each joint of the robot according to the kinematics of the robot;
converting the tail end characteristic points of the missile hanging robot manipulator into a tail end coordinate system of the robot manipulator under a camera coordinate system to obtain the motion amount of each joint,whereinJrobotIs a generalized Jacobian matrix of the robot, S: (ePc) Is composed ofePcThe elements of (a) make up an anti-symmetric matrix,namely the required motion amount of each joint of the missile hooking robot.
Step four: and generating a control law, and controlling the missile loading vehicle to move to the position of the aircraft hanging point for hanging according to the control law.
The amount of exercise of each joint of the missile hooking robot is given according to the third stepAnd designing a control law, and controlling the tail end of the robot to mount the mounted object to a specified position.
In conclusion, the invention realizes the method for automatically mounting various mounted objects on the airplane by using the robot vision servo control method based on the vision images, and can greatly improve the mounting efficiency.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.

Claims (5)

1. A robot vision servo control-based airborne equipment hanging method is characterized by comprising the following steps:
the method comprises the following steps: acquiring an image of the tail end of the missile hooking robot, and extracting image characteristics;
step two: designing a visual controller according to the image characteristics;
step three: calculating the motion amount of each joint of the robot according to the kinematics of the robot;
step four: and generating a control law, and controlling the missile loading vehicle to move to the position of the aircraft hanging point for hanging according to the control law.
2. The robot-vision-servo-control-based airborne equipment hanging method according to claim 1, wherein in the first step, an end image of the missile hanging robot is obtained, and feature points under a camera coordinate system are extractedcP=[Xc,Yc,Zc]TImage coordinates [ u, v ]]TAccording to the imaging model haveWherein f isxAnd fyIs the equivalent focal length of the camera; dotcP is varied in the vision sensor coordinate system, assuming that the point moves at a speed in the camera coordinate systemThen there isWhereincV=[cvx,cvy,cvz,]TcΩ=[cωx,cωy,cωz,]TAre respectively ascP linear and angular velocities in the camera coordinate system.
3. The robot vision servo control-based airborne equipment hanging method according to claim 2, wherein in the second step, derivation is carried out on the image coordinates in the first step, and then according to the relationship in the second step, the following is obtained:
<mrow> <mfenced open = "[" close = "]"> <mtable> <mtr> <mtd> <mover> <mi>u</mi> <mo>&amp;CenterDot;</mo> </mover> </mtd> </mtr> <mtr> <mtd> <mover> <mi>v</mi> <mo>&amp;CenterDot;</mo> </mover> </mtd> </mtr> </mtable> </mfenced> <mo>=</mo> <mfenced open = "[" close = "]"> <mtable> <mtr> <mtd> <mfrac> <msub> <mi>f</mi> <mi>x</mi> </msub> <msub> <mi>Z</mi> <mi>c</mi> </msub> </mfrac> </mtd> <mtd> <mn>0</mn> </mtd> <mtd> <mrow> <mo>-</mo> <mfrac> <mi>u</mi> <msub> <mi>Z</mi> <mi>c</mi> </msub> </mfrac> </mrow> </mtd> <mtd> <mrow> <mo>-</mo> <mfrac> <mrow> <mi>u</mi> <mi>v</mi> </mrow> <msub> <mi>f</mi> <mi>y</mi> </msub> </mfrac> </mrow> </mtd> <mtd> <mrow> <msub> <mi>f</mi> <mi>x</mi> </msub> <mo>+</mo> <mfrac> <msup> <mi>u</mi> <mn>2</mn> </msup> <msub> <mi>f</mi> <mi>x</mi> </msub> </mfrac> </mrow> </mtd> <mtd> <mrow> <mo>-</mo> <mfrac> <mrow> <msub> <mi>f</mi> <mi>x</mi> </msub> <mi>v</mi> </mrow> <msub> <mi>f</mi> <mi>y</mi> </msub> </mfrac> </mrow> </mtd> </mtr> <mtr> <mtd> <mn>0</mn> </mtd> <mtd> <mfrac> <msub> <mi>f</mi> <mi>y</mi> </msub> <msub> <mi>Z</mi> <mi>c</mi> </msub> </mfrac> </mtd> <mtd> <mrow> <mo>-</mo> <mfrac> <mi>v</mi> <msub> <mi>Z</mi> <mi>c</mi> </msub> </mfrac> </mrow> </mtd> <mtd> <mrow> <mo>-</mo> <msub> <mi>f</mi> <mi>y</mi> </msub> <mo>-</mo> <mfrac> <msup> <mi>v</mi> <mn>2</mn> </msup> <msub> <mi>f</mi> <mi>y</mi> </msub> </mfrac> </mrow> </mtd> <mtd> <mrow> <mo>-</mo> <mfrac> <mrow> <mi>u</mi> <mi>v</mi> </mrow> <msub> <mi>f</mi> <mi>x</mi> </msub> </mfrac> </mrow> </mtd> <mtd> <mrow> <mo>-</mo> <mfrac> <mrow> <msub> <mi>f</mi> <mi>y</mi> </msub> <mi>u</mi> </mrow> <msub> <mi>f</mi> <mi>x</mi> </msub> </mfrac> </mrow> </mtd> </mtr> </mtable> </mfenced> <mfenced open = "[" close = "]"> <mtable> <mtr> <mtd> <mrow> <msub> <mmultiscripts> <mi>v</mi> <mi>c</mi> </mmultiscripts> <mi>x</mi> </msub> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msub> <mmultiscripts> <mi>v</mi> <mi>c</mi> </mmultiscripts> <mi>y</mi> </msub> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msub> <mmultiscripts> <mi>v</mi> <mi>c</mi> </mmultiscripts> <mi>z</mi> </msub> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msub> <mmultiscripts> <mi>&amp;omega;</mi> <mi>c</mi> </mmultiscripts> <mi>x</mi> </msub> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msub> <mmultiscripts> <mi>&amp;omega;</mi> <mi>c</mi> </mmultiscripts> <mi>y</mi> </msub> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msub> <mmultiscripts> <mi>&amp;omega;</mi> <mi>c</mi> </mmultiscripts> <mi>z</mi> </msub> </mrow> </mtd> </mtr> </mtable> </mfenced> <mo>=</mo> <msub> <mi>J</mi> <mrow> <mi>i</mi> <mi>m</mi> <mi>a</mi> <mi>g</mi> <mi>e</mi> </mrow> </msub> <mmultiscripts> <mi>&amp;Psi;</mi> <mi>c</mi> </mmultiscripts> </mrow>
wherein,referred to as the image jacobian matrix,cΨ=[cvx cvy cvz cωx cωy cωz]T
4. the robot vision servo control based on-board equipment hanging method according to claim 3, characterized in that in the third step, the motion amount of each joint of the robot is calculated according to the robot kinematics;
converting the tail end characteristic points of the missile hanging robot manipulator into a tail end coordinate system of the robot manipulator under a camera coordinate system to obtain the motion amount of each joint,whereinJrobotIs a generalized Jacobian matrix of the robot, S: (ePc) Is composed ofePcThe elements of (a) make up an anti-symmetric matrix,namely the required motion amount of each joint of the missile hooking robot.
5. The robot-vision-servo-control-based on-board equipment hanging method according to claim 4, wherein in the fourth step, the amount of movement of each joint of the hanging ammunition robot is given according to the third stepDesigning a control law, controlling the tail end of the robot to mount the mounted object to a specified positionLocation.
CN201711348364.9A 2017-12-15 2017-12-15 Airborne equipment based on Robot Visual Servoing control mounts method Pending CN107932514A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711348364.9A CN107932514A (en) 2017-12-15 2017-12-15 Airborne equipment based on Robot Visual Servoing control mounts method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711348364.9A CN107932514A (en) 2017-12-15 2017-12-15 Airborne equipment based on Robot Visual Servoing control mounts method

Publications (1)

Publication Number Publication Date
CN107932514A true CN107932514A (en) 2018-04-20

Family

ID=61944275

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711348364.9A Pending CN107932514A (en) 2017-12-15 2017-12-15 Airborne equipment based on Robot Visual Servoing control mounts method

Country Status (1)

Country Link
CN (1) CN107932514A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108927807A (en) * 2018-08-14 2018-12-04 河南工程学院 A kind of robot vision control method based on point feature
CN108985184A (en) * 2018-06-26 2018-12-11 齐霄强 A kind of automatic carry system of multipurpose aircraft store Combinations and hanging method
CN109648555A (en) * 2018-12-18 2019-04-19 湖北航天技术研究院总体设计所 A kind of automatic transferring tool arm control method based on Multi-sensor Fusion
CN113268089A (en) * 2021-04-08 2021-08-17 成都立航科技股份有限公司 Method for adjusting pose of hanging object outside hanging vehicle

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6278906B1 (en) * 1999-01-29 2001-08-21 Georgia Tech Research Corporation Uncalibrated dynamic mechanical system controller
US6681151B1 (en) * 2000-12-15 2004-01-20 Cognex Technology And Investment Corporation System and method for servoing robots based upon workpieces with fiducial marks using machine vision
JP2011093014A (en) * 2009-10-27 2011-05-12 Ihi Corp Control device of hand-eye bin picking robot
CN106017463A (en) * 2016-05-26 2016-10-12 浙江大学 Aircraft positioning method based on positioning and sensing device
CN106960440A (en) * 2017-04-20 2017-07-18 力源液压系统(贵阳)有限公司 Based on subject image characteristic point pixel space position combination control method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6278906B1 (en) * 1999-01-29 2001-08-21 Georgia Tech Research Corporation Uncalibrated dynamic mechanical system controller
US6681151B1 (en) * 2000-12-15 2004-01-20 Cognex Technology And Investment Corporation System and method for servoing robots based upon workpieces with fiducial marks using machine vision
JP2011093014A (en) * 2009-10-27 2011-05-12 Ihi Corp Control device of hand-eye bin picking robot
CN106017463A (en) * 2016-05-26 2016-10-12 浙江大学 Aircraft positioning method based on positioning and sensing device
CN106960440A (en) * 2017-04-20 2017-07-18 力源液压系统(贵阳)有限公司 Based on subject image characteristic point pixel space position combination control method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
刘卫朋: "焊接机器人焊接路径识别与自主控制方法研究及应用", 《中国博士学位论文全文数据库》 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108985184A (en) * 2018-06-26 2018-12-11 齐霄强 A kind of automatic carry system of multipurpose aircraft store Combinations and hanging method
CN108985184B (en) * 2018-06-26 2022-04-29 齐霄强 Automatic mounting system and method for multipurpose aircraft plug-in
CN108927807A (en) * 2018-08-14 2018-12-04 河南工程学院 A kind of robot vision control method based on point feature
CN108927807B (en) * 2018-08-14 2020-08-07 河南工程学院 Robot vision control method based on point characteristics
CN109648555A (en) * 2018-12-18 2019-04-19 湖北航天技术研究院总体设计所 A kind of automatic transferring tool arm control method based on Multi-sensor Fusion
CN113268089A (en) * 2021-04-08 2021-08-17 成都立航科技股份有限公司 Method for adjusting pose of hanging object outside hanging vehicle

Similar Documents

Publication Publication Date Title
Zheng et al. Image-based visual servoing of a quadrotor using virtual camera approach
CN108453738B (en) A control method for autonomous aerial grasping operation of quadrotor aircraft based on Opencv image processing
CN107544550B (en) Unmanned aerial vehicle automatic landing method based on visual guidance
CN105353772B (en) A kind of Visual servoing control method in UAV Maneuver target locating
CN107932514A (en) Airborne equipment based on Robot Visual Servoing control mounts method
CN109471454B (en) Terminal guidance segment entering method of micro operation aircraft with designated attack inclination angle
García Carrillo et al. Stabilization and trajectory tracking of a quad-rotor using vision
CN109115035B (en) A kind of carrier rocket grade return phase precise guidance method based on trajectory forming
CN109895099A (en) A kind of flight mechanical arm visual servo grasping means based on physical feature
CN107729808A (en) A kind of image intelligent acquisition system and method for power transmission line unmanned machine inspection
CN103365297A (en) Optical flow-based four-rotor unmanned aerial vehicle flight control method
CN104062977A (en) Full-autonomous flight control method for quadrotor unmanned aerial vehicle based on vision SLAM
CN106054924A (en) Unmanned aerial vehicle accompanying method, unmanned aerial vehicle accompanying device and unmanned aerial vehicle accompanying system
Choi et al. Tracking an unknown moving target from uav: Extracting and localizing an moving target with vision sensor based on optical flow
CN109178345A (en) A kind of holder direction and celestial body posture cooperative control method for aerial tracking of maneuvering target
CN105739544B (en) Course following method and device of holder
CN115097856A (en) Target tracking dynamic feedback control method for quad-rotor unmanned aerial vehicle based on navigation vector field
CN109592057B (en) Vision servo-based aerial refueling machine oil receiving implementation method
CN106855690A (en) Robot of space rope system reclaims the self-adaptation control method of target after closely arresting
Kuswadi et al. Gun turret automatic weapon control system design and realization
CN108945536A (en) A kind of spacecrafts rendezvous experiment porch based on rotor craft
Bouzerzour et al. Robust vision-based sliding mode control for uncooperative ground target searching and tracking by quadrotor
Zhu et al. Vision/GPS-based docking control for the UAV autonomous aerial refueling
Darma et al. Visual servoing quadrotor control in autonomous target search
Wu et al. Intelligent explosive ordnance disposal UAV system based on manipulator and real-time object detection

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20180420