CN116408814A - Mobile robot system - Google Patents

Abstract

The present invention relates to mobile robotic systems. The robot system includes: a mobile robot, the mobile robot includes an environment detector, the environment detector includes a variety of sensors, and an abnormal cause reasoning device, when two or more sensors detect abnormalities, according to the various sensors The collected data is used to determine the cause of the abnormality through the neural network.

Description

mobile robot system

technical field

The invention relates to the field of robots, in particular to a mobile robot system including a mobile robot.

Background technique

With the continuous improvement of robot performance, the application range of mobile robots has been continuously expanded, not only in industry, agriculture, medical treatment, service and other industries, but also in the field of environmental detection.

Environmental detection is an important security application function of mobile robots. By carrying environmental detectors on mobile robots, mobile monitoring of general environmental variables, odors, toxic gases, special gases, etc. in a large area can be realized, which is beneficial to Early detection of potential safety hazards to ensure the safety of personnel and parks.

Contents of the invention

However, the environmental detectors carried by the existing mobile robots usually rely on the data collection of a single sensor, which has a high false positive rate and lack the combined application of data from multiple sensors.

If the environmental detector contains multiple sensors, in the case of multiple sensors of different types, multiple sensors may detect abnormalities at the same time. At this time, if the data of each sensor is analyzed separately, it may not be possible Determine exactly what caused it.

For example, when there is combustion smoke, the smoke sensor, PM2.5 sensor, dust sensor, illuminance sensor, etc. will all detect abnormalities. At this time, it is difficult to pinpoint exactly what is causing these sensor anomalies.

The invention provides a robot system capable of comprehensively analyzing the collected data of various sensors to accurately determine the cause of the abnormality.

The present invention provides a robot system, comprising: a mobile robot, the mobile robot includes an environment detector, and the environment detector includes multiple sensors; and an abnormal reason reasoning device, when two or more sensors detect an abnormal Collect data and judge the cause of the abnormality through the neural network.

In the above-mentioned robot system, the collected data is a two-dimensional matrix, each of which represents the sampling data of each sensor in the environment detector,

Each column of the two-dimensional matrix represents the sampled data at each time point.

In the robotic system mentioned above, the neural network adopts convolutional neural network for image classification.

In the above robot system, the convolutional neural network for image classification uses a 1×3 convolution kernel in the convolution layer and a 1×2 convolution kernel in the pooling layer.

In the aforementioned robotic system, a convolutional neural network for image classification employs translation-invariant pooling layers.

In the above-mentioned robot system, various sensors belong to different categories, and various sensors include: environmental variable sensors and gas sensors, and various sensors also include at least one of odor sensors, living biosensors, and human proximity sensors A sort of.

In the above robot system, the environmental variable sensors include: air pressure sensor, temperature sensor, humidity sensor, air pressure altitude sensor, air quality sensor, PM1.0/PM2.5/PM10 sensor, ultraviolet intensity sensor, light intensity sensor, electromagnetic radiation At least one of an intensity sensor, a noise intensity sensor, a precipitation state sensor, a rain sensor, a wind speed sensor, and a wind direction sensor.

In the above robot system, the gas sensors include: smoke sensor, carbon monoxide sensor, natural gas sensor, butane sensor, liquefied petroleum gas sensor, ammonia sensor, hydrogen sulfide sensor, carbon dioxide sensor, oxygen sensor, ozone sensor, formaldehyde sensor, volatile At least one of the organic gas sensor and the dust sensor.

In the above robot system, the various sensors further include: at least one of a position sensor, a magnetic direction sensor, a robot attitude sensor, and an image sensor.

In the above robot system, the environment detector also includes:

a control unit, controlling the data collection of the sensor;

an air duct communicating with the external air of the mobile robot so that the environmental detector is in contact with the external air; and

a fan for applying wind force to the air duct,

Wherein, the control unit also controls the switch and wind power of the fan.

In the above robot system, the environment detector further includes: a storage unit for storing the data collected by the sensor unit,

The environmental detector is a two-layer PCB circuit board, including an upper board and a lower board. Various sensors, air ducts, fans, storage parts, and control parts are divided into two groups and arranged on the upper board and the lower board respectively.

In the above robot system, the upper board is arranged on the surface of the two-layer PCB that is in direct contact with the external environment, and the sensors and fans that need to be in direct contact with the external environment among the various sensors are arranged on the upper board.

In the present invention, a variety of environmental data can be collected by setting various sensors such as environmental variable sensors and gas sensors in the environmental detector, and when two or more of these various sensors are abnormal, the Based on these data, the cause of the abnormality is analyzed. Therefore, it is possible to accurately find out the cause of the abnormality, which is conducive to taking targeted countermeasures.

Description of drawings

Fig. 1 is a schematic diagram showing a robot system involved in the present invention;

(a) of FIG. 2 is a side view showing the mobile robot 10, and (b) of FIG. 2 is a front view showing the mobile robot 10;

Fig. 3 is the schematic diagram showing the circuit connection structure of an example of environment detector;

(a) and (b) of Fig. 4 show the example that has the environment detector 11 of double-layer PCB circuit board structure;

FIG. 5 is a schematic diagram showing an example of a neural network model constituting the abnormality cause inferring means.

Detailed ways

Hereinafter, a mobile robot according to the present invention and a robot system including the robot will be described with reference to the drawings. In the following description, the same or similar reference numerals are assigned to the same or similar components.

FIG. 1 is a schematic diagram showing a robot system according to the present invention.

As shown in FIG. 1 , the robot system involved in the present invention includes: a mobile robot 10 and an abnormal cause inference device 20 .

<Mobile robot>

FIG. 2( a ) is a side view showing the mobile robot 10 , and FIG. 2( b ) is a front view showing the mobile robot 10 .

As shown in (a) and (b) of FIG. 2 , the mobile robot includes an environment detector 11 that can sense environmental factors.

Here, the environment detector 11 includes multiple types of sensors, each of which belongs to a different type. Wherein, there may be only one sensor of each type, or there may be multiple sensors.

Here, due to the wide variety of sensors used in environmental detection, some sensors have certain commonality. Therefore, a variety of sensors can be divided into different categories such as environmental variable sensors, gas sensors, odor sensors, living biosensors, human proximity sensors, position sensors, magnetic direction sensors, robot attitude sensors, and image sensors.

Environmental variable sensors can include: air pressure sensor, temperature sensor, humidity sensor, air pressure altitude sensor, air quality sensor, PM1.0/PM2.5/PM10 sensor, ultraviolet intensity sensor, light intensity sensor, electromagnetic radiation intensity sensor, noise intensity sensor , at least one of a precipitation state sensor, a rain sensor, a wind speed sensor, and a wind direction sensor.

Environmental variable data such as air pressure, temperature and humidity, air quality, light, noise, wind direction and speed, precipitation and rainfall can be obtained by carrying environmental variable sensors, making the mobile robot 10 suitable for environmental monitoring such as weather.

Gas sensors can include: smoke sensor, carbon monoxide sensor, natural gas sensor, butane sensor, liquefied petroleum gas sensor, ammonia sensor, hydrogen sulfide sensor, carbon dioxide sensor, oxygen sensor, ozone sensor, formaldehyde sensor, volatile organic gas sensor, dust at least one of the sensors.

The data of various gases can be obtained through the gas sensor, so that the mobile robot 10 can be applied to environmental monitoring such as dangerous gas monitoring and disaster monitoring.

The odor sensor may include an odor sensor. By carrying an odor sensor, the mobile robot 10 can be applied to environmental monitoring such as environmental pollution detection.

The human body proximity sensor may include at least one of a pyroelectric sensor and a human body movement sensor. By being equipped with a human body proximity sensor, the mobile robot 10 can also effectively recognize the approach of a person.

The various environment detectors 11 may also include: at least one of a position sensor, a magnetic orientation sensor, a robot attitude sensor, and an image sensor. By carrying these sensors, the mobile robot 10 can not only monitor the environment, but also collect position data, image data, etc. corresponding to the environment data.

The mobile robot 10 of the present application can be widely used in various application scenarios requiring environmental monitoring by being equipped with various sensors. For example, it can be applied to a patrol-type mobile robot. By carrying various types of sensors on the patrol-type mobile robot, various parameters in the target scene can be collected while patrolling.

In this application, the environment sensor 11 may further include a control unit and a storage unit. The control part can control the data collection of the sensor. The storage unit may be used to store data collected by the sensor unit. The control unit may include a processor such as a CPU (Central Processing Unit, central processing unit). The storage unit may include ROM (Read Only Memory, read only memory), RAM (Random Access Memory, random access memory), and the like. The processor of the control unit can read codes of various programs stored in the storage unit or the like, and execute various processes. Various programs may be acquired from other server devices via a network, or may be stored in a storage medium and read via a drive device. Each function of the control unit may be configured by a dedicated logic circuit or an ASIC (Application Specific Integrated Circuit) in which a dedicated logic circuit is integrated. In addition, the control unit may be configured by combining with a programmable logic controller (PLC). In addition, the control unit and the storage unit may not be located in the environment sensor 11 , but may be located in other parts of the mobile robot 10 .

In addition, the environment detector 11 may also include: an air duct and a fan. The air duct communicates with the external air of the mobile robot 10 so that the environment detector 11 is in contact with the external air. The fan applies wind force to the air duct. The fan may be any fan such as an intake turbo fan. The above-mentioned control unit can control the switch and wind power of the fan. The environment detector 11 may also include a fan driving circuit for driving a fan.

In addition, the environment probe 11 may further include a communication unit that enables the mobile robot to communicate with external equipment such as an abnormality cause reasoning device and an external server. In addition, the communication unit may not be located in the environment sensor 11 but may be located elsewhere in the mobile robot 10 .

In addition, the environment detector 11 may further include a power supply unit, which can receive an input of an external power supply to supply power to the environment detector 11 .

FIG. 3 is a schematic diagram showing an example of a circuit connection structure of the environment sensor 11 .

In the example of the environment sensor 11 shown in FIG. 3 , the environment sensor 11 includes a plurality of sensors, a fan, a fan drive circuit, a control unit, a storage unit, a power supply unit, and the like.

Among them, a variety of sensors include: temperature sensor, humidity sensor, air pressure sensor, air pressure altitude sensor, air quality sensor, light sensor, ultraviolet intensity sensor, carbon dioxide sensor, oxygen concentration sensor, IMU solid-state gyroscope, magnetic direction sensor, wind speed and direction sensor , PM1.0/PM2.5/PM10 sensor, noise intensity sensor, precipitation status sensor, rain sensor, odor sensor, combustible gas sensor, carbon monoxide sensor, dust sensor, dense fog sensor, formaldehyde sensor, organic volatile gas sensor, living organisms sensor, pyroelectric sensor.

In the case that the environmental detector 11 includes a large number of sensors, how to configure multiple sensors on the smaller environmental detector 11 so as to better cooperate with multiple sensors and fans and other equipment has also become a problem to be solved. In the embodiment of the present invention, a double-layer PCB circuit board structure is designed. That is, the environment sensor 11 includes an upper board and a lower board, and the upper board and the lower board are laminated. The upper board and the lower board can also be the front and back of a PCB circuit board.

A plurality of sensors, air ducts, fans, storage units, control units and other components of the environment detector 11 are divided into two groups and arranged on the upper board and the lower board. Wherein, the upper plate is arranged on the surface directly in contact with the external environment. Among the multiple sensors, the sensors that need to be in contact with the external environment and the fan are arranged on the upper board. Accordingly, even when a plurality of sensors are mounted, the environmental sensor 11 can be downsized and its performance can be improved.

Specifically, (a) and (b) of FIG. 4 show an example of the environment detector 11 having a double-layer PCB circuit board structure.

In the example shown in (a) and (b) of FIG. 4 , the upper board of the environmental detector 11 includes gas sensors, and the gas in the external environment of the mobile robot 10 is sucked in by a fan, and passes through each gas sensor uniformly. Therefore, the gas sensor can be in good contact with the external gas, which is beneficial to improve the detection accuracy.

The upper board of the environmental detector 11 also includes an illuminance sensor, an ultraviolet intensity sensor, etc. among the environmental variable sensors. These sensors directly collect external ambient light conditions, so they also need to be in contact with the outside.

Human body proximity sensors such as pyroelectric sensors are also preferably arranged on the upper plate, so that external signals can be received more accurately, which is conducive to improving detection accuracy.

The lower board of the environmental detector 11 includes: circuits such as a control unit, a storage unit, a communication unit, and a power supply unit. These circuits do not need to be in contact with the external environment, and in order to prolong their working life, they are preferably isolated from the external environment. Therefore, for These parts can also be sealed.

Air pressure sensors, temperature and humidity sensors, noise sensors, GPS sensors, etc. can also collect data without direct contact with the external environment. Therefore, they can be randomly arranged on the upper or lower board according to the design space of the circuit board. In the example shown in (a) and (b) of FIG. 4, it is arrange|positioned on the lower plate.

As mentioned above, by dividing the multiple sensors and circuits in the environment detector 11 into two types that need to be in direct contact with the external environment and those that do not need to be in direct contact, they are divided into two groups and arranged on the upper and lower boards of the environment detector 11 respectively. On-board, so as to be able to take into account the detection accuracy of sensors, etc., and the life of sensors and circuits.

<Anomaly cause reasoning device>

Hereinafter, the abnormal cause inference device 20 will be described.

In the actual application of a mobile robot 10 equipped with an environment detector 11 containing multiple sensors, due to the complexity of the environment, multiple sensors may produce concurrent responses to a specific abnormal state at the same time, and the collection of multiple sensors The data all exceed the safety threshold, that is, multiple sensors are abnormal at the same time, so multiple sensors may send out alarms at the same time, so that the security management personnel cannot determine the cause of the abnormality.

For example, when foggy combustible gas or gas pipeline leakage is detected, smoke sensors, PM2.5 sensors, and combustible gas sensors may all issue abnormal alarms. For example, when there is combustion smoke, the smoke sensor, PM2.5 sensor, dust sensor, illuminance sensor, etc. will all detect abnormalities. For example, when a septic tank gas leak occurs, the ammonia sensor, hydrogen sulfide sensor, odor sensor, combustible gas sensor, and air quality sensor may all be abnormal. For example, when garbage odor is detected, the ammonia sensor, hydrogen sulfide sensor, odor sensor, and air quality sensor may have numerical abnormalities at the same time.

In order to solve such problems, in this embodiment, as shown in Figure 5, a neural network-based abnormality reasoning device is constructed, and the data collection results of various sensors are input into the neural network of the abnormality reasoning device, Output the abnormal cause judgment result.

That is, when two or more sensors detect an abnormality, the abnormality cause reasoning device 20 determines the cause of the abnormality based on the collected data of multiple sensors and through a neural network.

The abnormality reason reasoning device 20 may be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof. For example, the device for inferring abnormal causes may be implemented by a combination of software and hardware by any suitable electronic device such as a desktop computer, a tablet computer, a smart phone, or a server equipped with a processor. The form of the computer is shown in FIG. 1, but is not limited thereto.

The processor of the abnormality cause reasoning device 20 can execute abnormality determination processing.

Abnormal cause inference device 20 may also include a memory (not shown), a communication module (not shown), and the like.

The communication module of the abnormality cause inference device may support establishment of a direct (eg, wired) communication channel or a wireless communication channel between the abnormality cause inference device and an external electronic device, and perform communication via the established communication channel. For example, the communication module enables the abnormality cause inference device 20 to communicate with the mobile robot 10 via a network.

In addition, the abnormality cause inferring device 20 may further include an output unit such as a display, a microphone, and a speaker for displaying the abnormality cause as the abnormality judgment result, or issuing a warning.

<input data>

The input data of the abnormal cause inference device 20 is the data collected by the environment detector 11 of the mobile robot 10 . The environment detector 11 can collect data at a constant sampling frequency. Since the environment detector 11 is installed on the mobile robot 10, the sampling frequency is not too high, for example, it is preferably about 10 Hz.

The collected data may be two-dimensional matrix data, wherein each row represents the sampled data of each type of environmental detectors that detected anomalies, and each column represents the sampled data at each time point when the data is collected at a constant sampling frequency. In other words, each row represents a different kind of sensor, and each column represents a different time point.

For example, taking gas pipeline leakage as an example, at this time, it is assumed that the environment detector 11 mounted on the mobile robot 10 includes n types of sensors. There is one sensor of each kind, that is, n sensors. These sensors are sampled at a sampling frequency of 10 Hz, and each sampling time is 10 s, and 100 data are sampled. The sensor type, number, sampling frequency, sampling time, sampling quantity, etc. are just examples, and are not limited thereto. For example, there may be multiple sensors of each type. In this case, data processing such as averaging may be performed on the values of the multiple sensors as the collected data of this type of sensor. In the example shown in Figure 5, it is assumed that the environmental detectors include smoke concentration sensors, combustible gas concentration sensors, PM2.5 sensors, PM10 sensors, dust concentration sensors, air quality sensors, carbon dioxide concentration sensors, carbon monoxide concentration sensors, ammonia sensors , Hydrocarbon concentration sensor, odor index sensor, illuminance sensor, noise sensor, etc.

At this point, the collected data of each sensor is:

1) The data set collected by the smoke density sensor: A=[a1, a2...a100];

2) The data set collected by the combustible gas concentration sensor: B=[b1, b2...b100];

3) The data set collected by the PM2.5 sensor: C=[c1, c2...c100];

4) The data set collected by the PM10 sensor: D=[d1, d2...d100]

...

Then, combine the data collected by these sensors to form a two-dimensional matrix data=[A; B; C; D; ...; N]. Specifically, as follows:

Among them, each row indicates which kind of sensor it is. For example, in the example shown in FIG. 5 , the first row of the input data as two-dimensional matrix data may represent the sampling data of the smoke concentration sensor, the second row may represent the sampling data of the combustible gas concentration sensor, and so on.

The subscript in each column indicates which one of the sampled data, for example, the number of the 100 sampled data. Since the sampling time of each sampled data is different, it can also be understood as the sampled data of different sampling times. For example, a1 represents the sampling data at the first time point, a2 represents the sampling data at the second time point, and so on.

In addition, since the robot in the present invention is a mobile robot, the sampling data may also be sampling data at different sampling positions when the mobile robot is moving.

In this application, the data for each sensor is one-dimensional. In order to apply the classification ability of the neural network model, these one-dimensional data are combined into two-dimensional data here. However, the data correlation of each dimension is small and has a certain degree of independence.

<model building>

In the state of the art, Convolutional Neural Networks (CNN) can be used to solve the problem of classifying images into different classes.

However, in this application, when two or more sensors detect abnormalities, the reasoning device 20 determines the cause of the abnormality based on the data collected by various sensors and through a neural network. Image classification has similar characteristics, so it is attempted to use the image classification convolutional neural network model to realize the abnormal cause reasoning device 20 .

As mentioned above, since the input data can adopt n×100×1 dimensional data, that is, two-dimensional data. Therefore, such input data can be applied to image classification convolutional neural network models. Therefore, the image classification convolutional neural network model can be used to determine the cause of sensor abnormality.

However, since the input data of this application is different from the image data, some improvements have been made on the basis of the traditional image classification convolutional neural network model in order to be more suitable for the application scenario of this application.

As mentioned above, in this application, a certain degree of independence needs to be maintained among the data of each dimension.

However, in the usual image classification convolutional neural network model, for example, a 3×3 or 5×5 or 1×1 convolutional layer or a 2×2 pooling layer is used. If such a model is used directly, it will The different input data is removed, and the dimensionality of the data will be reduced. For example, if a 2×2 pooling layer is used, the data of 10 sensors will become data of 5 sensors after pooling, which will change the data source and fail to meet the requirement of maintaining the independence of data in each dimension in this application. Require.

Therefore, in this application, the hidden layer of the image classification convolutional neural network includes a convolutional layer and a pooling layer, and a 1×3 convolution kernel is used in the convolutional layer, and a 1×2 convolutional layer is used in the pooling layer. nuclear.

By using a 1×3 convolution kernel in the convolution layer and a 1×2 convolution kernel in the pooling layer, the independence between each dimension can be guaranteed.

In addition, the image data in the usual image classification convolutional neural network is not time-dependent. However, the input data of this application is sampling data sampled at different time points, therefore, the input data has a correlation with time.

Therefore, in this application, in the image classification convolutional neural network, a translation invariant pooling layer can be used. By adopting the translation-invariant pooling layer, it is possible to maintain some change information during the downsampling process and avoid smoothing the outliers through the pooling layer operation, thereby ensuring the trend and change point of the curve after pooling. Therefore, it is beneficial to keep the change trend of each dimension after the pooling layer consistent with the original data.

In this application, if the convolutional layer uses a 1×3 convolutional kernel and the pooling layer uses a 1×2 convolutional kernel, and a translation-invariant pooling layer is used, the data of each dimension can be maintained. Independence, but also to maintain the continuity of information, so that the analysis of abnormal causes can be more accurate.

In addition, in order to increase the generalization ability of the model, noise can also be introduced into the data of each dimension, thereby increasing randomness.

<output data>

The output data of this model are various causes of sensor abnormality.

In the example shown in Figure 5, the output data may include: gas pipeline leakage, combustion smoke, water mist, oily smoke, vehicle exhaust, septic tank gas leakage, domestic garbage corruption, toilet odor, decoration odor, etc. The output data can be the probability of each abnormal cause, and the most probable cause can be found out according to the probability value.

With the abnormality cause reasoning device 20 configured in this way, when two or more sensors detect abnormality, it is possible to determine the cause of the abnormality and reduce the misjudgment rate of the abnormality cause.

Above, although the embodiments and specific examples of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention. fall within the scope defined by the claims.

Claims (12)

1. A robotic system, comprising:

a mobile robot comprising an environmental probe including a plurality of sensors; and

and an abnormality cause inference unit configured to judge a cause of the abnormality through a neural network based on the acquired data of the plurality of sensors when the abnormality is detected by two or more sensors.

2. The robotic system of claim 1, wherein,

the acquired data is a two-dimensional matrix,

each row of the two-dimensional matrix represents sampled data for each sensor in the environmental detector,

each column of the two-dimensional matrix represents sample data for each point in time.

3. The robotic system of claim 2, wherein,

the neural network adopts an image classification convolutional neural network.

4. The robotic system of claim 3, wherein,

the image classification convolutional neural network adopts a convolution kernel of 1 multiplied by 3 in a convolution layer, and adopts a convolution kernel of 1 multiplied by 2 in a pooling layer.

5. The robotic system of claim 3, wherein,

the image classification convolutional neural network adopts a pooling layer with translational invariance.

6. The robotic system as claimed in any one of claims 1-5, wherein,

the plurality of sensors includes: an environmental variable sensor and a gas sensor,

and, the plurality of sensors further include at least one of an odor sensor, a living body biosensor, and a human body proximity sensor.

7. The robotic system of claim 6, wherein,

the environmental variable sensor includes: at least one of an air pressure sensor, a temperature sensor, a humidity sensor, an air pressure height sensor, an air quality sensor, a PM1.0/PM2.5/PM10 sensor, an ultraviolet intensity sensor, an illumination intensity sensor, an electromagnetic radiation intensity sensor, a noise intensity sensor, a rainfall state sensor, a rainfall sensor, a wind speed sensor and a wind direction sensor.

8. The robotic system of claim 6, wherein,

the gas sensor includes: at least one of a smoke sensor, a carbon monoxide sensor, a natural gas sensor, a butane sensor, a liquefied petroleum gas sensor, an ammonia sensor, a hydrogen sulfide sensor, a carbon dioxide sensor, an oxygen sensor, an ozone sensor, a formaldehyde sensor, a volatile organic gas sensor and a dust sensor.

9. The robotic system of claim 6, wherein,

the plurality of sensors further comprises: at least one of a position sensor, a magnetic orientation sensor, a robot gesture sensor and an image sensor.

10. The robotic system of claim 6, wherein,

the environment detector further comprises:

a control unit for controlling data collection of the sensor;

the air duct is communicated with the external air of the mobile robot, so that the environment detector is in contact with the external air; and

a fan for applying wind power to the air duct,

wherein, the control part also controls the switch and wind power of the fan.

11. The robotic system of claim 10, wherein,

the environment detector further comprises: a storage part for storing the data collected by the sensor,

the environment detector is a two-layer PCB circuit board, comprising an upper board and a lower board,

the plurality of sensors, the air duct, the fan, the storage portion, and the control portion are arranged on the upper plate and the lower plate, respectively, in two groups.

12. The robotic system of claim 11, wherein,

the upper board is arranged on the surface which is in direct contact with the external environment of the two-layer PCB circuit board,

the fan and the sensor of the plurality of sensors, which are required to be in direct contact with the external environment, are disposed on the upper plate.

Images