CN111866464B - A remote control system of agricultural tractor based on virtual reality technology - Google Patents

Abstract

The invention provides a remote control system for agricultural tractors based on virtual reality technology. Including scene recognition module, status monitoring module, data transmission module, cloud scene simulation module, AR remote control module. The beneficial effects of the present invention are: through scene recognition, the driving image is a comprehensive map in six directions, thereby comprehensively acquiring the scene elements of the agricultural tractor, determining the driving scene, and realizing comprehensive monitoring of the operating environment. The purpose of state monitoring is to monitor the real-time state of the agricultural tractor and judge the data of the agricultural tractor. During data transmission, according to the number of different agricultural tractors, different data transmission methods are adopted, which is convenient to improve the comprehensiveness of the efficiency of data transmission. A simulation of the scene in Yundon. Based on the technology of cloud big data, it can reflect the real state of agricultural tractors, and then through the AR remote control module, the real-time state of agricultural tractors can be observed remotely, and real-time monitoring can be performed to realize real-time remote control.

Description

A remote control system for agricultural tractor based on virtual reality technology

technical field

The invention relates to the technical field, in particular to a remote control system for agricultural tractors based on virtual reality technology.

Background technique

At present, tractors, as high-horsepower equipment, are generally used for agricultural transportation and agricultural planting. The development level of agricultural equipment directly affects the technical level and economic benefits of my country's agricultural sector. A large number of agricultural equipment is used in modern agricultural production to improve agricultural productivity and the commercialization rate of agricultural products, ensure the safety of food production and increase farmers' income, and the most agricultural equipment is tractors.

when the tractor is in use. There are many varieties of agricultural products, complex scene conditions, and huge requirements for the status, performance, life, and cost of tractor products. With the development of Internet technology, scene simulation technology, and sensing technology, the realization of Agricultural scene simulation and remote control have promoted the high automation of agricultural work. In recent years, with the continuous development of virtual reality technology and the establishment of an immersive control system, remote control has become a new development trend.

SUMMARY OF THE INVENTION

The present invention provides a remote control system for agricultural tractors based on virtual reality technology to solve the above situation.

A remote control system for agricultural tractors based on virtual reality technology, characterized in that it includes:

Scene recognition module: used to obtain the driving image of the running agricultural tractor, and extract the image elements to determine the driving scene;

Status monitoring module: used to determine the driving status through the sensing equipment installed on the agricultural tractor;

Data transmission module: used to determine a data transmission mode, and according to the data transmission mode, transmit the driving scene and driving state to the cloud scene simulation module;

Cloud scene simulation module: used to construct a simulation scene of the agricultural tractor according to the driving scene and driving state, and generate a real-time simulation video;

AR remote control module: used to display the real-time simulation video through the AR device, determine simulation parameters, optimize the driving scene of the agricultural tractor according to the simulation parameters, and generate control instructions.

As an embodiment of the present invention, the scene recognition module includes:

Camera unit: used to obtain scene images through the camera equipment installed on the agricultural tractor; wherein,

There are no less than 5 camera devices; wherein,

No less than two camera devices with a camera angle of 120° are installed on both sides of the rear of the agricultural tractor;

No less than two camera devices with a camera angle of 180° are installed on both sides of the middle of the agricultural tractor;

No less than one camera device with a camera angle of 180° is installed on both sides of the front of the agricultural tractor;

Element extraction unit: for judging whether the image elements in the scene image have an impact on the driving behavior of the agricultural tractor according to the scene image, and extracting the image element when there is an impact;

The image elements include road surface elements, obstacle elements, traffic indication elements and weather elements;

Scene judgment unit: used to compare the image elements with historical driving scenes to determine the driving scene; wherein,

When the image element does not exist in the historical driving scene, the image element is transmitted to the user terminal, the driving scene is determined, and the driving scene is saved.

As an embodiment of the present invention, the state monitoring module includes:

Sensor unit: used to collect state data according to the preset sensing device on the agricultural tractor; wherein,

The sensing device includes: temperature sensing device, speed sensing device, position sensor, liquid level sensor, energy consumption sensor, speed sensor, acceleration sensor, ray radiation sensor, thermal sensor, vibration sensor and humidity sensor;

Signal strength unit: used to preset verification data and a time axis between the agricultural tractor and the AR remote control module, and determine the strength of the signal according to the transmission distance of the verification data on the time axis;

Data processing unit: used to determine the correlation between the state data, and determine the dynamic state model according to the correlation;

Threshold unit: used to obtain the equipment parameters of the agricultural tractor, and set a threshold model according to the equipment parameters;

State judging unit: used to compare the threshold model with the dynamic state model, determine the driving difference, calculate the state loss value of the agricultural tractor under the driving difference according to the signal strength, and determine the driving state.

As an embodiment of the present invention, the data transmission module includes:

Transmission path judgment unit: used to obtain the data transmission mode and verify whether the data transmission channel is synchronous transmission;

Mode setting unit: used to set the data transmission mode according to the strength of the signal strength in the driving scene, wherein,

Described data transmission single-channel data transmission mode and multiple-channel data transmission mode;

Mode selection unit: configured to determine a data transmission priority according to the driving scene and driving state, and determine the data transmission mode according to the priority.

As an embodiment of the present invention, the cloud scene simulation module includes:

Data acquisition unit: used to determine scene data and state data according to the driving scene and driving state;

Scene processing unit: used to use the scene data as a data source, construct a scene simulation model, and determine a scene video;

Status processing unit: used to use the status data as a data source, and substitute it into the preset simulated agricultural tractor to determine the status video;

Fusion unit: used to fuse the state video into the scene video to determine the simulation video;

Real-time processing unit: used to determine difference data according to the data acquisition unit and the fusion unit, and substitute the difference data into the simulation video according to the type of the difference data to determine the real-time simulation video.

As an embodiment of the present invention, the AR remote control module includes:

Device control unit: used to obtain the simulation video and push the simulation video to the AR device;

Parameter extraction unit: used to determine a simulation program package and a simulation language according to the simulation video, determine the simulation object parameters of the agricultural tractor according to the simulation program package, and determine the simulation of the agricultural tractor according to the simulation language. Structural parameters;

Equipment control unit: used to generate object control parameters according to the simulation object parameters, generate an object control window according to the simulation structure parameters and the object control parameters, and receive the first control behavior input by the user;

Autonomous control unit: used to process the simulation object parameters through an ant colony optimization algorithm to obtain autonomous control parameters and generate a second control behavior;

Instruction generation unit: used for judging the control intention according to the first control behavior and the second control behavior, and generating control instructions; wherein,

The priority of the first control behavior is higher than the second control behavior;

Instruction implementing unit: used to send the control instruction to the agricultural tractor, determine the control information, and execute the control operation.

As an embodiment of the present invention, determining the signal strength by the signal strength unit includes the following steps:

： Step 1: Based on the agricultural tractor and the AR remote control module, determine the channel feature set and the base station feature set between the agricultural tractor and the AR remote control module

:

表示第

个信道；所述

表示第

个基站的特征；所述

，表示共有

个信道，共有

个基站； Among them, the

means the first

channel; the

means the first

characteristics of base stations; the

, indicating a common

channels, a total of

base stations;

： Step 2: Substitute the channel characteristics and base station characteristics into the relationship model to determine the implementation probability of any channel and any base station

:

；

;

表示信道特征均值；所述

表示算基站特征均值；所述

表示第

个 信道与第

个基站的实施概率； Among them, the

represents the channel feature mean; the

means to calculate the mean value of base station characteristics; the

means the first

channel and

the implementation probability of each base station;

： Step 3: Determine the implementation capability of the channel according to the implementation probability

:

；

;

表示第

个信道的传输能力； Among them, the

means the first

The transmission capacity of each channel;

： Step 4: Build a strength model based on the implementation capabilities

:

；

;

表示信号强度； Among them, the

Indicates signal strength;

，当所述

时，表示信号强度强；当所述

时，表示信 号强度弱。 Set intensity threshold

, when the

When the signal strength is strong; when the

when the signal strength is weak.

As an embodiment of the present invention, the scene recognition module further includes a cloud control unit and an AI recognition unit; wherein

The cloud control unit includes a cloud server, and the cloud server is used to build an AI quantitative model, an AI pattern recognition model and an AI analysis model inside the AI recognition unit through big data technology and a general AI model; the cloud server also uses It is used to build a dedicated communication channel between the AI recognition unit and the agricultural tractor through the cloud network;

The AI recognition unit includes an AI recognition server, and the AI recognition server is used for receiving scene images and generating a full-scene stereo space; wherein,

The AI recognition server does the following:

importing the received scene image into the AI analysis model to obtain image elements;

importing the image elements into the AI quantitative model to generate a corresponding quantitative analysis mode; Medium

The quantitative analysis mode includes at least ground analysis, traffic signal analysis and business type analysis;

The quantitative analysis pattern is imported into the AI pattern recognition model, and the AI pattern recognition model determines historical driving data and real-time driving data according to the quantitative analysis pattern.

As an embodiment of the present invention, the AR remote control module further includes:

Path optimization unit: used to optimize the driving trajectory of the agricultural tractor according to the real-time simulation video; wherein,

The driving trajectory optimization steps of the agricultural tractor include:

Obtaining an initial simulation trajectory path, and reducing the error rate and convergence speed of the simulation trajectory path based on multi-model control, and determining a first simulation optimized trajectory path; wherein,

The multi-model control includes gain control, sliding mode control and artificial intelligence control;

Setting an expected change value of the error rate change, and judging whether the difference between the change value of the error rate and the expected change value is higher than a preset expected change threshold;

When the change value of the error rate is greater than the preset expected change threshold value, adopt the differential control method to stabilize the change value of the error rate, and determine the target simulation optimization trajectory path;

When the change value of the error rate is smaller than the preset expected change threshold, proportional model control is used first to increase the change value of the error rate, and when the change value of the error rate is greater than the preset expected change threshold , using the differential control method to stabilize the change value of the error rate, and determine the target simulation optimization trajectory path.

As an embodiment of the present invention, the differential control method includes:

；

;

表示预设期望变化阈值；所述

表示第

时刻的误差率的变化值； 所述

表示误差率的误差系数；所述

表示误差率的实际变化值； Among them, the

represents the preset expected change threshold; the

means the first

the change value of the error rate at the moment; the

is the error coefficient representing the error rate; the

represents the actual change value of the error rate;

时，表示误差率的变化值稳定； when

When , it means that the change value of the error rate is stable;

， 滑模控制对所述误差率的变化值的影响参数

和人工智能控制对所述误差率的变化值的 影响参数

，构建比例控制模型： The proportional model control includes: respectively determining the influence parameters of the gain control on the change value of the error rate

, the influence parameter of sliding mode control on the change value of the error rate

and artificial intelligence control the influence parameters on the change value of the error rate

, build a proportional control model:

；

;

， 滑模控制对所述误差率的变化值的影响参数

和人工智能控制对所述误差率的变化值的 影响参数

，当所述

大于

时，表示可以采用微分控制的方式稳定误差率的变化值。 According to the proportional control model, the influence parameter of the gain control on the change value of the error rate is adjusted

, the influence parameter of sliding mode control on the change value of the error rate

and artificial intelligence control the influence parameters on the change value of the error rate

, when the

more than the

When , it means that the variation value of the error rate can be stabilized by means of differential control.

The beneficial effects of the present invention are: through scene recognition, the driving image is a comprehensive map in six directions, thereby comprehensively acquiring the scene elements of the agricultural tractor, determining the driving scene, and realizing comprehensive monitoring of the operating environment. The purpose of state monitoring is to monitor the real-time state of the agricultural tractor and judge the data of the agricultural tractor. During data transmission, according to the number of different agricultural tractors, different data transmission methods are adopted, which is convenient to improve the comprehensiveness of the efficiency of data transmission. A simulation of the scene in Yundon. Based on the technology of cloud big data, it can reflect the real state of agricultural tractors, and then through the AR remote control module, the real-time state of agricultural tractors can be observed remotely, and real-time monitoring can be performed to realize real-time remote control.

Other features and advantages of the present invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and drawings.

The technical solutions of the present invention will be further described in detail below through the accompanying drawings and embodiments.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification, and are used to explain the present invention together with the embodiments of the present invention, and do not constitute a limitation to the present invention.

In the attached image:

FIG. 1 is a system composition diagram of a remote control system for an agricultural tractor based on virtual reality technology in an embodiment of the present invention.

Detailed ways

The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, but not to limit the present invention.

As shown in Figure 1, a remote control system for agricultural tractors based on virtual reality technology includes:

Scene recognition module: used to obtain the driving image of the running agricultural tractor, and extract the image elements to determine the driving scene; the driving image is a comprehensive map of six directions, so that the driving scene can be comprehensively determined, and the driving scene includes at least five agricultural tractors. A comprehensive picture of each direction. Image elements such as the ground, where there are dirt roads, cement roads, asphalt roads, and lawns, land, and mountains without roads. Other elements include, obstacle elements, such as people near agricultural tractors, stones, trees, and wires poles, traffic signs, etc.

Status monitoring module: used to determine the driving status through the sensing equipment installed on the agricultural tractor; mainly used to obtain the status of the tractor, such as temperature status, humidity status, diesel liquid level status, speed status, etc.

Data transmission module: used to determine the data transmission mode, and according to the data transmission mode, transmit the driving scene and driving state to the cloud scene simulation module; the data transmission module is mainly used when there is only one agricultural tractor. Single-channel data transmission, when there are multiple agricultural tractors, the multi-channel, multi-channel synchronous data transmission method is adopted.

Cloud scene simulation module: used to construct a simulation scene of the agricultural tractor according to the driving scene and driving state, and generate a real-time simulation video; mainly used for simulating the running scene of the agricultural tractor, and then the simulated status video of the agricultural tractor, Includes state track and scene pages.

AR remote control module: used to display the real-time simulation video through the AR device, determine simulation parameters, optimize the driving scene of the agricultural tractor according to the simulation parameters, and generate control instructions. It is used to predict and optimize the predicted behavior of agricultural tractors based on the simulation video, and provide the optimal predicted behavior based on the business problem to be solved.

The beneficial effects of the present invention are: through scene recognition, the driving image is a comprehensive map in six directions, thereby comprehensively acquiring the scene elements of the agricultural tractor, determining the driving scene, and realizing comprehensive monitoring of the operating environment. The purpose of state monitoring is to monitor the real-time state of the agricultural tractor and judge the data of the agricultural tractor. During data transmission, according to the number of different agricultural tractors, different data transmission methods are adopted, which is convenient to improve the comprehensiveness of the efficiency of data transmission. A simulation of the scene in Yundon. Based on the technology of cloud big data, it can reflect the real state of agricultural tractors, and then through the AR remote control module, the real-time state of agricultural tractors can be observed remotely, and real-time monitoring can be performed to realize real-time remote control.

As an embodiment of the present invention, the scene recognition module includes:

Camera unit: used to obtain scene images through the camera equipment installed on the agricultural tractor; wherein,

There are no less than 5 camera devices; wherein,

No less than two camera devices with a camera angle of 120° are installed on both sides of the rear of the agricultural tractor;

No less than two camera devices with a camera angle of 180° are installed on both sides of the middle of the agricultural tractor;

No less than one camera equipment with a camera angle of 180° is installed on both sides of the front of the agricultural tractor; the camera equipment can be a snap camera, a video camera, etc.; no less than five, for the purpose of comprehensively monitoring the tractor. The set camera angle has overlapping parts for each camera device and adjacent devices, and there will be no shortage of camera images, resulting in missing image elements.

Element extraction unit: used to determine whether the image elements in the scene image have an impact on the driving behavior of the agricultural tractor according to the scene image, and when there is an impact, extract the image element; the definition of the impact is that for the agricultural tractor The behavior will produce positive or negative effects, and with positive or negative effects, determine image elements.

The image elements include road surface elements, obstacle elements, traffic indication elements and weather elements;

Scene judgment unit: used to compare the image elements with historical driving scenes to determine the driving scene; wherein, the historical driving scene, that is, similar situations that have been encountered, there are many similar situations in the agricultural field.

When the image element does not exist in the historical driving scene, the image element is transmitted to the user terminal, the driving scene is determined, and the driving scene is saved. The user terminal is a behavior of manually judging the driving scene, so that the recognition behavior will not affect the determination of the scene when the machine cannot recognize it.

As an embodiment of the present invention, the state monitoring module includes:

Sensor unit: used to collect state data according to the preset sensing device on the agricultural tractor; wherein, the sensor is mainly used to obtain the state of the agricultural tractor.

The sensing device includes: temperature sensing device, speed sensing device, position sensor, liquid level sensor, energy consumption sensor, speed sensor, acceleration sensor, ray radiation sensor, thermal sensor, vibration sensor and humidity sensor;

Signal strength unit: used to preset verification data and time axis between the agricultural tractor and the AR remote control module, and determine the signal strength according to the transmission distance of the verification data on the time axis; The preset verification data is repeatedly transmitted in the time period. According to the transmission distance of the verification data transmission on the time axis, that is, the time for a round trip, and then the signal strength is judged according to the time, so that the signal strength can be improved when the signal strength is low. strength.

Data processing unit: used to determine the correlation between the state data, and determine the dynamic state model according to the correlation; used to determine the relationship between the collected data, and then generate a dynamic state model to represent the operation of the agricultural tractor state model.

Threshold unit: used to obtain the equipment parameters of the agricultural tractor, and set the threshold model according to the equipment parameters; any equipment has a limit value, so the threshold value can be more convenient and intuitive to judge the state threshold of the agricultural tractor, and then adjust the correct value. Farm tractor for adjustment.

State judging unit: used to compare the threshold model with the dynamic state model, determine the driving difference, calculate the state loss value of the agricultural tractor under the driving difference according to the signal strength, and determine the driving state. It is mainly used to judge whether the agricultural tractor is in a normal driving state or has a fault, which is convenient for predicting the fault of the agricultural tractor.

As an embodiment of the present invention, the data transmission module includes:

Transmission path judgment unit: used to obtain the data transmission mode and verify whether the data transmission channel is synchronous transmission; synchronous transmission means that the monitoring information and the user's observation of the state of the agricultural tractor through the AR device are in a synchronous state. What is the actual scene behavior, the AR device displays what behavior.

Mode setting unit: used to set the data transmission mode according to the strength of the signal strength in the driving scene, wherein,

Described data transmission single-channel data transmission mode and multiple-channel data transmission mode;

It is used to prevent untimely data transmission. When the signal is strong, single-channel data transmission is used to reduce the waste of communication resources. When the signal is weak, multiple channels of synchronous transmission are used to complement data to prevent data loss.

Mode selection unit: configured to determine a data transmission priority according to the driving scene and driving state, and determine the data transmission mode according to the priority.

As an embodiment of the present invention, the cloud scene simulation module includes:

Data acquisition unit: used to determine scene data and state data according to the driving scene and driving state; the scene data is mainly data of elements in the scene, such as data of obstacles (stones on the road, people and other obstacles). The state data is mainly the operating state parameters of the agricultural tractor, such as: speed data, temperature data, liquid level data and so on.

Scene processing unit: used to use the scene data as a data source to construct a scene simulation model and determine a scene video; the scene simulation model is a scene frame based on simulation software in the prior art.

Status processing unit: used to use the status data as a data source, and substitute it into the preset simulated agricultural tractor to determine the status video;

Fusion unit: used to fuse the state video into the scene video to determine the simulation video;

Real-time processing unit: used to determine difference data according to the data acquisition unit and the fusion unit, and substitute the difference data into the simulation video according to the type of the difference data to determine the real-time simulation video.

The beneficial effects of the above technical solutions are: through the fusion of the status video and the scene video, the behavior of the agricultural tractor can be comprehensively determined for remote monitoring of the agricultural tractor, and there will be no data missing, and the user can remotely observe the real-time behavior of the agricultural tractor in person. status for easy manual control.

As an embodiment of the present invention, the AR remote control module includes:

Device control unit: used to acquire the simulation video and push the simulation video to the AR device; the AR device, such as an AR helmet or AR eyes, is convenient for users to observe.

Parameter extraction unit: used to determine a simulation program package and a simulation language according to the simulation video, determine the simulation object parameters of the agricultural tractor according to the simulation program package, and determine the simulation of the agricultural tractor according to the simulation language. Structural parameters; the simulation package defines the logic of the policy and the data parameters of the simulation. The simulation language defines the technical architecture during simulation.

Equipment control unit: used to generate object control parameters according to the simulation object parameters, generate an object control window according to the simulation structure parameters and the object control parameters, and receive the first control behavior input by the user; the first control behavior is mainly: The user's control behavior has a higher priority.

Autonomous control unit: used to process the simulation object parameters through the ant colony optimization algorithm to obtain the autonomous control parameters and generate the second control behavior; after the user control, the control benchmark is determined, and the present invention uses the ant colony optimization algorithm to adjust the control instructions. Carry out optimization and determine the best regulation method.

Instruction generation unit: used for judging the control intention according to the first control behavior and the second control behavior, and generating control instructions; wherein,

The priority of the first control behavior is higher than the second control behavior;

Instruction implementing unit: used to send the control instruction to the agricultural tractor, determine the control information, and execute the control operation. The agricultural tractor is pre-set with a signal receiving and identification device, which can parse and clear the control information according to the control instructions, and then execute the implementation.

As an embodiment of the present invention, determining the signal strength by the signal strength unit includes the following steps:

： Step 1: Based on the agricultural tractor and the AR remote control module, determine the channel feature set and the base station feature set between the agricultural tractor and the AR remote control module

:

表示第

个信道；所述

表示第

个基站的特征；所述

，表示共有

个信道，共有

个基站； Among them, the

means the first

channel; the

means the first

characteristics of base stations; the

, indicating a common

channels, a total of

base stations;

： Step 2: Substitute the channel characteristics and base station characteristics into the relationship model to determine the implementation probability of any channel and any base station

:

；

;

表示信道特征均值；所述

表示算基站特征均值；所述

表示第

个 信道与第

个基站的实施概率； Among them, the

represents the channel feature mean; the

means to calculate the mean value of base station characteristics; the

means the first

channel and

the implementation probability of each base station;

： Step 3: Determine the implementation capability of the channel according to the implementation probability

:

；

;

表示第

个信道的传输能力

； Among them, the

means the first

transmission capacity of a channel

;

： Step 4: Build a strength model based on the implementation capabilities

:

；

;

表示信号强度； Among them, the

Indicates signal strength;

，当所述

时，表示信号强度强；当所述

时，表示信 号强度弱。 Set intensity threshold

, when the

When the signal strength is strong; when the

when the signal strength is weak.

The principle and beneficial effects of the above technical solutions are: the present invention determines the signal strength according to the base station data and channel data between the agricultural tractor and the AR remote control module. There are base stations of different operators in the base station, but the channel does not have the difference of the operator, only the difference of the channel stability and the transmission amount. Through the channel feature set and the base station feature set, the invention determines the implementation probability when each channel cooperates with the base station based on the relationship model, and then determines the implementation capability according to the transmission capability of the channel, and finally substitutes the strength model of the signal strength to determine the final signal strength.

As an embodiment of the present invention, the scene recognition module further includes a cloud control unit and an AI recognition unit; wherein

The cloud control unit includes a cloud server, and the cloud server is used to build an AI quantitative model, an AI pattern recognition model and an AI analysis model inside the AI recognition unit through big data technology and a general AI model; the cloud server also uses It is used to build a dedicated communication channel between the AI recognition unit and the agricultural tractor through the cloud network; the example of the present invention realizes the scene through the three steps of the constructed AI quantitative model, AI pattern recognition model and AI analysis model isomorphic precise analysis, quantitative analysis and instruction generation. Intelligent and accurate analysis and generation of data.

The AI recognition unit includes an AI recognition server, and the AI recognition server is used for receiving scene images and generating a full-scene stereo space; wherein,

The AI recognition server does the following:

importing the received scene image into the AI analysis model to obtain image elements;

importing the image elements into the AI quantitative model to generate a corresponding quantitative analysis mode; Medium

The quantitative analysis mode includes at least ground analysis, traffic signal analysis and business type analysis;

The quantitative analysis pattern is imported into the AI pattern recognition model, and the AI pattern recognition model determines historical driving data and real-time driving data according to the quantitative analysis pattern. The invention realizes the accurate transformation of the imagined image into the driving data, extracts the historical data, and realizes the intelligent judgment of the scene through the AI technology intelligent identification technology and the big data technology data analysis and calculation function.

As an embodiment of the present invention, the AR remote control module further includes:

Path optimization unit: used to optimize the driving trajectory of the agricultural tractor according to the real-time simulation video; wherein,

The steps of optimizing the driving trajectory of the agricultural tractor include:

Obtaining an initial simulation trajectory path, and reducing the error rate and convergence speed of the simulation trajectory path based on multi-model control, and determining a first simulation optimized trajectory path; wherein,

The multi-model control includes gain control, sliding mode control and artificial intelligence control;

In multi-model control, the error rate of the simulated trajectory path is reduced through multiple control models, and the convergence rate is an iterative adjustment sequence of errors to reduce the rate of change of errors.

Set the expected change value of the error rate change, and judge whether the difference between the error rate change value and the expected change value is higher than the preset expected change threshold; and the expected change value of the error rate change value is the preset change condition threshold.

When the change value of the error rate is greater than the preset expected change threshold value, adopt the differential control method to stabilize the change value of the error rate, and determine the target simulation optimization trajectory path;

The differential control method is to stabilize the change value of the optimized error. The scale model is to increase the change value of the error, thereby realizing the regulation of the change rate of the error. When the change rate of the error is larger, it is easier to adjust it; because the present invention builds an adjustment model based on the change value of the error rate , therefore, the smaller the change value is, the less easy it is to adjust, and the larger the change value, the more in line with the model of the present invention, which is convenient for optimization and adjustment.

When the change value of the error rate is smaller than the preset expected change threshold, proportional model control is used first to increase the change value of the error rate, and when the change value of the error rate is greater than the preset expected change threshold , using the differential control method to stabilize the change value of the error rate, and determine the target simulation optimization trajectory path.

As an embodiment of the present invention, the differential control method includes:

；

;

表示预设期望变化阈值；所述

表示第

时刻的误差率的变化值； 所述

表示误差率的误差系数（即，实际农用拖拉机和仿真的农用拖拉机之间的误差系 数）；所述

表示误差率的实际变化值； Among them, the

represents the preset expected change threshold; the

means the first

the change value of the error rate at the moment; the

the error coefficient representing the error rate (ie, the error coefficient between the actual farm tractor and the simulated farm tractor); the

represents the actual change value of the error rate;

时，表示误差率的变化值稳定； when

When , it means that the change value of the error rate is stable;

，滑模控制对所述误差率的变化值的影响参数

和人工智能控制对所述误差率的变化值的 影响参数

，构建比例控制模型： The proportional model control includes: respectively determining the influence parameters of the gain control on the change value of the error rate

, the influence parameter of sliding mode control on the change value of the error rate

and artificial intelligence control the influence parameters on the change value of the error rate

, build a proportional control model:

；

;

， 滑模控制对所述误差率的变化值的影响参数

和人工智能控制对所述误差率的变化值的 影响参数

，当所述

大于

时，表示可以采用微分控制的方式稳定误差率的变化值。 According to the proportional control model, the influence parameter of the gain control on the change value of the error rate is adjusted

, the influence parameter of sliding mode control on the change value of the error rate

and artificial intelligence control the influence parameters on the change value of the error rate

, when the

more than the

When , it means that the variation value of the error rate can be stabilized by means of differential control.

In the above technical solutions, the differential control method is mainly an optimal control, mainly to adjust the change value of the error rate, and the proportional control model is to adjust the change value of the error rate to a state that can be adjusted.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, optical storage, and the like.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block in the flowcharts and/or block diagrams, and combinations of flows and/or blocks in the flowcharts and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in one or more of the flowcharts and/or one or more blocks of the block diagrams.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions An apparatus implements the functions specified in a flow or flows of the flowcharts and/or a block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in one or more of the flowcharts and/or one or more blocks of the block diagrams.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit and scope of the invention. Thus, provided that these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims (6)

1. a kind of agricultural tractor remote control system based on virtual reality technology, is characterized in that, comprises: Scene recognition module: used to obtain the driving image of the running agricultural tractor, and extract the image elements to determine the driving scene; Status monitoring module: used to determine the driving status through the sensing equipment installed on the agricultural tractor; Data transmission module: used to determine a data transmission mode, and according to the data transmission mode, transmit the driving scene and driving state to the cloud scene simulation module; Cloud scene simulation module: used to construct a simulation scene of the agricultural tractor according to the driving scene and driving state, and generate a real-time simulation video; AR remote control module: used to display the real-time simulation video through the AR device, and determine simulation parameters, optimize the driving scene of the agricultural tractor according to the simulation parameters, and generate control instructions; wherein, The AR remote control module includes: Device control unit: used to obtain the simulation video and push the simulation video to the AR device; Parameter extraction unit: used to determine a simulation program package and a simulation language according to the simulation video, determine the simulation object parameters of the agricultural tractor according to the simulation program package, and determine the simulation of the agricultural tractor according to the simulation language. Structural parameters; Equipment control unit: used to generate object control parameters according to the simulation object parameters, generate an object control window according to the simulation structure parameters and the object control parameters, and receive the first control behavior input by the user; Autonomous control unit: used to process the simulation object parameters through an ant colony optimization algorithm to obtain autonomous control parameters and generate a second control behavior; Instruction generation unit: used for judging the control intention according to the first control behavior and the second control behavior, and generating control instructions; wherein, The priority of the first control behavior is higher than the second control behavior; Instruction implementing unit: used to send the control instruction to the agricultural tractor, determine the control information, and execute the control operation; The status monitoring module includes: Sensor unit: used to collect state data according to the preset sensing device on the agricultural tractor; wherein, The sensing device includes: temperature sensing device, speed sensing device, position sensor, liquid level sensor, energy consumption sensor, speed sensor, acceleration sensor, ray radiation sensor, thermal sensor, vibration sensor and humidity sensor; Signal strength unit: used to preset verification data and a time axis between the agricultural tractor and the AR remote control module, and determine the strength of the signal according to the transmission distance of the verification data on the time axis; Data processing unit: used to determine the correlation between the state data, and determine the dynamic state model according to the correlation; Threshold unit: used to obtain the equipment parameters of the agricultural tractor, and set a threshold model according to the equipment parameters; State judging unit: used to compare the threshold model with the dynamic state model, determine the driving difference, calculate the state loss value of the agricultural tractor under the driving difference according to the signal strength, and determine the driving state. 2. a kind of agricultural tractor remote control system based on virtual reality technology according to claim 1, is characterized in that, scene recognition module comprises: Camera unit: used to obtain scene images through the camera equipment installed on the agricultural tractor; wherein, There are no less than 5 camera devices; wherein, No less than two camera devices with a camera angle of 120° are installed on both sides of the rear of the agricultural tractor; No less than two camera devices with a camera angle of 180° are installed on both sides of the middle of the agricultural tractor; No less than one camera device with a camera angle of 180° is installed on both sides of the front of the agricultural tractor; Element extraction unit: for judging whether the image elements in the scene image have an impact on the driving behavior of the agricultural tractor according to the scene image, and extracting the image element when there is an impact; The image elements include road surface elements, obstacle elements, traffic indication elements and weather elements; Scene judgment unit: used to compare the image elements with historical driving scenes to determine the driving scene; wherein, When the image element does not exist in the historical driving scene, the image element is transmitted to the user terminal, the driving scene is determined, and the driving scene is saved. 3. a kind of agricultural tractor remote control system based on virtual reality technology according to claim 1, is characterized in that, data transmission module comprises: Transmission path judgment unit: used to obtain the data transmission mode and verify whether the data transmission channel is synchronous transmission; Mode setting unit: used to set the data transmission mode according to the strength of the signal strength in the driving scene, wherein, The data transmission modes include: single-channel data transmission mode and multiple-channel data transmission mode; Mode selection unit: configured to determine a data transmission priority according to the driving scene and driving state, and determine the data transmission mode according to the priority. 4. a kind of agricultural tractor remote control system based on virtual reality technology according to claim 1, is characterized in that, cloud scene simulation module comprises: Data acquisition unit: used to determine scene data and state data according to the driving scene and driving state; Scene processing unit: used to use the scene data as a data source, construct a scene simulation model, and determine a scene video; Status processing unit: used to use the status data as a data source, and substitute it into the preset simulated agricultural tractor to determine the status video; Fusion unit: used to fuse the state video into the scene video to determine the simulation video; Real-time processing unit: used to determine difference data according to the data acquisition unit and the fusion unit, and substitute the difference data into the simulation video according to the type of the difference data to determine the real-time simulation video. 5. The remote control system for agricultural tractors based on virtual reality technology according to claim 1, wherein the scene recognition module comprises a cloud control unit and an AI recognition unit; wherein, The cloud control unit includes a cloud server, and the cloud server is used to build an AI quantitative model, an AI pattern recognition model and an AI analysis model inside the AI recognition unit through big data technology and a general AI model; the cloud server also uses It is used to build a dedicated communication channel between the AI recognition unit and the agricultural tractor through the cloud network; The AI recognition unit includes an AI recognition server, and the AI recognition server is used for receiving scene images and generating a full-scene stereo space; wherein, The AI recognition server does the following: importing the received scene image into the AI analysis model to obtain image elements; The image elements are imported into the AI quantitative model, and the corresponding quantitative analysis mode is generated; wherein, The quantitative analysis mode includes at least ground analysis, traffic signal analysis and business type analysis; The quantitative analysis pattern is imported into the AI pattern recognition model, and the AI pattern recognition model determines historical driving data and real-time driving data according to the quantitative analysis pattern. 6. a kind of agricultural tractor remote control system based on virtual reality technology according to claim 1, is characterized in that, described AR remote control module also comprises: Path optimization unit: used to optimize the driving trajectory of the agricultural tractor according to the real-time simulation video; wherein, The driving trajectory optimization steps of the agricultural tractor include: Obtaining an initial simulation trajectory path, and reducing the error rate and convergence speed of the simulation trajectory path based on multi-model control, and determining a first simulation optimized trajectory path; wherein, The multi-model control includes gain control, sliding mode control and artificial intelligence control; Setting an expected change value of the error rate change, and judging whether the difference between the change value of the error rate and the expected change value is higher than a preset expected change threshold; When the change value of the error rate is greater than the preset expected change threshold value, adopt the differential control method to stabilize the change value of the error rate, and determine the target simulation optimization trajectory path; When the change value of the error rate is smaller than the preset expected change threshold, proportional model control is used first to increase the change value of the error rate, and when the change value of the error rate is greater than the preset expected change threshold , using the differential control method to stabilize the change value of the error rate, and determine the target simulation optimization trajectory path.

Images