CN107598928A - Camera and robot control system and its automatic adaptation method based on semantic model - Google Patents

Abstract

The invention relates to a camera and robot control system based on a semantic model and an automatic adaptation method thereof. A semantic model server module and a semantic model client module are respectively established in the camera controller and the robot controller, and the semantic model server module The semantic modeling of the module and the analysis of the client module of the semantic model realize the automatic adaptation of the industrial camera and the robot control system. The invention can reprogram the controller to modify the functions and commands of the camera without the need for the robot to stop working when the camera function is upgraded and changed, which increases the flexibility of the robot system, saves working time, and is plug-and-play.

Description

Camera and Robot Control System and Its Automatic Adaptation Method Based on Semantic Model

technical field

The invention relates to a semantic model-based industrial camera and robot control system and an automatic adaptation method thereof, belonging to the field of robot control.

Background technique

In the context of the industry 4.0 era, with the deepening of China's 2025 strategy, the robot industry market is showing explosive growth momentum. This is inseparable from the machine vision system. Machine vision is equivalent to the extension of human vision on the machine. Machine collaboration becomes a reality. If the robot industry wants to achieve real intelligence and automation, it must have the ability to accurately observe things in order to be able to judge things well. As the core of the machine vision system, the importance of industrial cameras is self-evident.

In the traditional machine vision solution, the robot controller needs to perform fixed programming work on the communication configuration and functions of the camera. When there are a large number of random combinations of camera functions, extremely tedious robot programming work is required; when the camera function is upgraded and changed, the robot controller needs to Stop working and reprogram the functions and configurations to fix. If you change to a different camera, you need to perform specific programming for this camera according to the functions and configurations of the camera manufacturer, and the software between different cameras does not have reusability. Traditional solutions affect efficiency, are not flexible, and do not conform to the concept of intelligent manufacturing.

"Semantic" refers to the ability of machines to research and collect information and understand machine language with less human intervention. It is currently widely used in the Internet field, and it is widely used in web and HTML transmission. In the field of industrial control, there are different data formats and communication modes of each device. This will result in a system of data collection, requiring engineers to know different data, format meanings, etc., wasting a lot of resources. Semantics solves this problem, allowing devices to be described in a unified manner, speak the same language, and achieve better interconnection.

In response to the demand for intelligence in the field of robot control, based on semantic modeling, the adaptive method of industrial cameras and robot control systems has been studied, semantic description information, automatic adaptation of server and client, breaking through the cumbersome fixed programming of traditional vision, and realizing Software reuse, plug and play, and interconnection are of great significance to the development of China's intelligent manufacturing.

Contents of the invention

Aiming at the cumbersome reprogramming work of a large number of communication configuration processes and camera replacement or function update in the existing vision scheme, the present invention provides a semantic model-based industrial camera and robot control system and its automatic adaptation method. Based on the semantic model module, through semantic description, automatic adaptation, plug and play, and synchronous update are realized to improve work efficiency.

The technical solution adopted by the present invention to achieve the above purpose is: based on the automatic adaptation method of the camera and the robot control system based on the semantic model, the semantic model server module and the semantic model client module are respectively established in the camera controller and the robot controller. The terminal module realizes the automatic adaptation of the industrial camera and the robot control system through the semantic modeling of the semantic model server module and the analysis of the semantic model client module, including the following steps:

The semantic model server module carries out semantic description according to camera parameters and camera control commands, writes them into the elements and sub-elements of the address space, and then forms the parameter organization structure and service list into an XML file;

The semantic model client module accesses the semantic model server module in the camera controller and obtains the XML file, parses the camera parameters and camera control commands; sends the required camera parameters and camera control commands corresponding to the semantic commands to camera controller;

The semantic model server module gets the semantic command, the camera controller executes the operation corresponding to the semantic command, and feeds back the robot information; the robot controller performs control according to the robot information.

The camera parameters include camera shooting functions, shooting parameter adjustments, and camera parameter information.

The parameter organization structure includes correspondence between elements and sub-elements; the service list includes contents of elements and sub-elements.

The semantic description according to the camera parameters and control commands, and the writing into the elements and sub-elements of the address space include the following steps:

Write the camera shooting function, shooting parameter adjustment, and camera parameter information into the address space as elements respectively; the classification of each element is regarded as the sub-element under the element; the action corresponding to a certain sub-element, that is, the camera control command, is used as the sub-element under the sub-element The next level of child elements.

The camera controller performs the corresponding command operation and feeds back the robot information including the following steps:

The camera controller executes the shooting action according to the semantic command, and obtains the position information corresponding to the shooting result.

After feeding back the robot information, the robot controller calculates the motion trajectory of the robot according to the position information, and controls the robot to perform the grasping action according to the motion trajectory.

The camera and robot control system based on the semantic model establishes a semantic model server module and a semantic model client module in the camera controller and the robot controller respectively, including:

Semantic model server module, used to carry out semantic description according to camera parameters and camera control commands, write them into the elements and sub-elements of the address space, and then form the parameter organization structure and service list into XML files; get the semantic commands, make The camera controller executes the operation corresponding to the semantic command and feeds back the robot information to the robot controller;

The semantic model client module is used to access the semantic model server module in the camera controller and obtain the XML file, parse the camera parameters and camera control commands; convert the required camera parameters and camera control commands to the corresponding semantic The command is sent to the semantic model server module of the camera controller.

The present invention has the following advantages and beneficial effects:

1. The automatic adaptation of the present invention is not aimed at a certain type of robot controller and industrial camera, and its adaptation method is applicable to all robot controllers and industrial cameras equipped with the semantic model module of the present invention.

2. The automatic adaptation feature of the present invention can replace the manual modification of camera functions and communication commands with the teaching pendant without interrupting the work of the robot controller when the camera function is upgraded and changed, which increases the flexibility of the robot system. Save working time, plug and play.

3. The semantic model module of the present invention uses a service-oriented technology, does not depend on a specific hardware platform and operating system, and the software has reusability and a wide range of applications.

Description of drawings

Fig. 1 is the embodiment block diagram that the hardware environment composition of the present invention is applied;

Fig. 2 is a module diagram of a controller semantic model of the present invention;

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

See Figure 1, the industrial camera and robot control system based on the semantic model, which mainly involves the robot controller, camera controller, and robot body.

The robot controller is an X86 architecture platform running the QNX real-time operating system, and its hardware part includes a motion controller and a servo driver, and supports EtherCAT bus communication. The software part includes semantic model server module, control algorithm module, sensor module, motion control module; control algorithm module, including basic algorithm library, functional algorithm library and application algorithm library, used for algorithm application of robot control; sensor module, including Sensor hardware drivers and data conversion models for robot-sensor interaction. Motion control module, including dynamics module and kinematics module, is used to analyze commands and control robot operation.

The camera controller includes a semantic model client module, a communication configuration module, and a visual function module;

Referring to Figure 2, the core points of automatic adaptation in this patent are the semantic model client module and server module. The semantic model server module is a service-oriented module based on semantics, which defines a model that integrates address space and information, and uses semantic modeling to model all the parameters, operation methods, history, events and other information related to a specific object Semantically describe objects, establish relationships with each other, and form XML files of parameter organization structure and service list, which can represent complex data structures and processes, establish network communication servers, and complete data exchange based on TCP binary protocol. The semantic modeling described above is based on data modeling, adding a semantic description layer and describing in the form of "data-semantics-relationship". The semantic model client module is a service-oriented module based on semantics. It establishes a client for network communication, accesses the server module through ID, and obtains parameter organization structure and service list based on semantic attribute information and XML file analysis.

Taking the robot controller and industrial camera system as an example, the specific process includes: the industrial camera controller establishes a semantic model server module, and the parameter information of the camera such as: shooting function, shooting parameter adjustment, basic camera information and other parameters The semantic modeling method carries out semantic description and writes it into the elements of the address space. The elements include hierarchical relationships such as element layer and sub-element layer. The relationship between each element is established to form an XML file of parameter organization structure and service list. The robot controller establishes a semantic model client module, and the client initiates a handshake to the server to establish a connection. Obtain the parameter organization structure and service list XML, analyze the information of elements and sub-elements in the address space based on the semantic model, and finally obtain all the data of the camera's information model, so as to achieve the purpose of automatic adaptation. According to the adaptation result, the robot controller sends the corresponding control command to the camera controller. The camera controller executes the sent command and feeds back the position information to the robot controller. The robot controller combines the control algorithm module, The motion control module controls the robot body to execute grabbing commands.

An example of the automatic adaptation method of the present invention will be described below, and the specific steps are as follows:

Step 1: The camera controller establishes a semantic model server module, based on semantic modeling, adjusts camera shooting functions (such as shooting shape, recognition color, storage function) and shooting parameters (such as exposure time, focal length, shooting mode) ), the basic information of the camera (such as manufacturer, date of manufacture, maximum resolution) and other parameters are semantically described, and written into the elements and sub-elements of the address space; examples of functions and control commands: for example, the camera can capture images of different shapes and Different colors, semantically describe the color and shape at the element layer, semantically describe the shape including circle, square, etc., and the color including red, yellow, etc. at the respective sub-element layer, and control commands such as shooting a yellow circle Commands such as shape and red square are semantically described in the next sub-element layer. Then the parameter organization structure and service list are formed into an XML file.

Step 2: The robot controller sets up the semantic model client module, initiates a handshake to the server, establishes communication, obtains the parameter organization structure and service list XML, analyzes the information of elements and sub-elements in the address space based on the semantic model, and finally obtains the camera All the data in the information model corresponds to step 1. For example, after access, the controller knows that the camera can capture different shapes and colors, as well as various control commands, so that the controller and the camera can achieve the purpose of automatic adaptation.

Step 3: According to the functional requirements, the robot controller, for example, shoots a yellow circle, and the semantic model client module sends the semantic command of shooting a yellow circle obtained by analyzing the semantics to the camera controller. After the camera controller obtains the command, the visual The function module starts to execute algorithms such as search and matching (there is a template such as a yellow circle in the camera controller, and it keeps searching. When the captured yellow circle happens to be a match, it will calculate the position information of the yellow circle and send it to the robot. controller), when the yellow circle is photographed, its position information is sent to the robot controller.

Step 4: The robot control calls the control algorithm module to solve the calculation according to the position information, and obtains the execution trajectory of the robot, and the motion control module controls the robotic arm to perform the grasping task.

To sum up, the method of the present invention proposes a system and an automatic adaptation method based on a semantic model to meet the requirements of automatic adaptation of industrial cameras and robot control systems. This method provides convenience for the robot controller to flexibly control industrial cameras, plug and play, does not depend on specific models of controllers and cameras, its software is reusable, saves working time, is conducive to the expansion of multi-camera systems, and can realize interconnection Interoperable ad hoc decision making.

Claims (7)

1. The automatic adaptation method of the camera and the robot control system based on the semantic model is characterized in that: the semantic model server module and the semantic model client module are respectively established in the camera controller and the robot controller, and through the semantic model The semantic modeling of the server module and the analysis of the client module of the semantic model realize the automatic adaptation of the industrial camera and the robot control system, including the following steps: The semantic model server module carries out semantic description according to camera parameters and camera control commands, writes them into the elements and sub-elements of the address space, and then forms the parameter organization structure and service list into an XML file; The semantic model client module accesses the semantic model server module in the camera controller and obtains the XML file, parses the camera parameters and camera control commands; sends the required camera parameters and camera control commands corresponding to the semantic commands to camera controller; The semantic model server module gets the semantic command, the camera controller executes the operation corresponding to the semantic command, and feeds back the robot information; the robot controller performs control according to the robot information. 2. The automatic adaptation method of a camera and a robot control system based on a semantic model according to claim 1, wherein the camera parameters include camera shooting functions, shooting parameter adjustments, and camera parameter information. 3. The semantic model-based automatic adaptation method between a camera and a robot control system according to claim 1, wherein the parameter organization structure includes correspondence between elements and sub-elements; the service list includes contents of elements and sub-elements. 4. The automatic adaptation method of a camera and a robot control system based on a semantic model according to claim 1, wherein the semantic description is carried out according to the camera parameters and control commands, and the elements and sub-elements written into the address space include the following steps : Write the camera shooting function, shooting parameter adjustment, and camera parameter information into the address space as elements respectively; the classification of each element is regarded as the sub-element under the element; the action corresponding to a certain sub-element, that is, the camera control command, is used as the sub-element under the sub-element The next level of child elements. 5. the automatic adaptation method of camera and robot control system based on semantic model according to claim 1, described camera controller carries out corresponding command operation, and feedback robot information comprises the following steps: The camera controller executes the shooting action according to the semantic command, and obtains the position information corresponding to the shooting result. 6. According to the automatic adaptation method of the camera and the robot control system based on the semantic model according to claim 1, after feeding back the robot information, the robot controller obtains the robot motion trajectory according to the position information solution, and controls the robot to perform grabbing according to the motion trajectory action. 7. The camera and robot control system based on the semantic model is characterized in that a semantic model server module and a semantic model client module are respectively established in the camera controller and the robot controller, including: Semantic model server module, used to carry out semantic description according to camera parameters and camera control commands, write them into the elements and sub-elements of the address space, and then form the parameter organization structure and service list into XML files; get the semantic commands, make The camera controller executes the operation corresponding to the semantic command and feeds back the robot information to the robot controller; The semantic model client module is used to access the semantic model server module in the camera controller and obtain the XML file, parse the camera parameters and camera control commands; convert the required camera parameters and camera control commands to the corresponding semantic The command is sent to the semantic model server module of the camera controller.