KR101786516B1 - Moving robot and controlling method thereof - Google Patents

Abstract

The present invention relates to a mobile robot, and more particularly, to a vacuum cleaner that performs autonomous traveling, including a main body, a driving unit for providing a driving force for moving the main body, a camera for generating an image related to the external environment of the main body, A sensor, a laser sensor that scans an object existing around the main body and generates laser scan information, a controller that corrects information related to the position and path of the main body using at least one of the image and the laser scan information .

Description

[0001] MOVING ROBOT AND CONTROLLING METHOD THEREOF [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a mobile robot and a control method thereof, and more particularly, to a vacuum cleaner that performs autonomous traveling and a control method thereof.

In general, robots have been developed for industrial use and have been part of factory automation. In recent years, medical robots, aerospace robots, and the like have been developed, and household robots that can be used in ordinary homes are being developed.

A representative example of the domestic robot is a robot cleaner, which is a type of household appliance that sucks and cleanes dust or foreign matter around the robot while traveling in a certain area by itself. Such a robot cleaner is generally equipped with a rechargeable battery and has an obstacle sensor capable of avoiding obstacles during traveling, so that it can run and clean by itself.

In recent years, research has been actively carried out to utilize the robot cleaner in various fields such as health care, smart home, and remote control by moving away from the cleaning operation by merely autonomously moving the cleaning area by the robot cleaner.

Specifically, the conventional robot cleaner includes only an infrared sensor, an ultrasonic sensor, an RF sensor, an optical sensor, or a camera sensor for acquiring a two-dimensional image in order to detect information related to an obstacle. Therefore, it is difficult to obtain accurate obstacle information in the conventional robot cleaner.

In particular, conventional robot cleaners generally detect obstacles by using two-dimensional image information obtained by a two-dimensional camera sensor. By using such two-dimensional image information, the distance between the obstacle and the robot body and the three- It is difficult to detect.

In addition, the conventional robot cleaner extracts feature points from two-dimensional image information to detect obstacle information. When the two-dimensional image information is formed such that feature point extraction is disadvantageous, the accuracy of the detected obstacle information is remarkably degraded.

In order to solve such a problem, there is a need for a robot cleaner equipped with a laser sensor.

The robot cleaner equipped with the laser sensor can detect the position of the robot cleaner or correct the path of the robot cleaner by using the laser scan information generated by the laser sensor. Further, the robot cleaner equipped with the laser sensor can detect information related to an obstacle or a wall existing around the main body of the robot cleaner.

However, the position of the robot cleaner detected based only on the laser scan information has a problem that the accuracy thereof is remarkably low. That is, since the laser sensor can only scan an object within a specific distance from the main body of the robot cleaner, it can not generate laser scan information when the robot cleaner is located in an environment such as a wide area or a long corridor.

In addition, the robot cleaner can not recognize the position or the position of the robot cleaner at the time when the laser scan information is generated only by the laser scan information.

Disclosure of Invention Technical Problem [8] The present invention provides a vacuum cleaner and a control method thereof, which can autonomously travel by recognizing a position of a robot or adjusting a path by using a laser sensor and a camera sensor installed in a robot cleaner.

It is also an object of the present invention to provide a vacuum cleaner that performs autonomous travel for recognizing the position of a robot or adjusting a path based on at least one of a visual pattern existing in the travel environment and a form of the travel environment and a control method thereof will be.

It is also an object of the present invention to provide an image forming apparatus and an image forming apparatus that compares an image generated by a camera sensor with laser scanning information generated by a laser sensor even if a slip occurs during traveling of the robot cleaner, And a control method thereof.

It is another object of the present invention to provide a vacuum cleaner and a control method therefor that can perform autonomous travel to adjust position information and route information of a cleaner by using a camera sensor and a laser sensor together when the robot cleaner circulates along a closed curve .

It is also an object of the present invention to provide a robot cleaner which is capable of detecting a return path based on an image pattern extracted from an image photographed by a camera sensor and a topographic shape extracted from laser scanning information generated by a laser sensor, And to provide a control method of the vacuum cleaner.

According to another aspect of the present invention, there is provided a vacuum cleaner comprising: a main body; a driving unit for providing driving force for moving the main body; a camera sensor for generating an image related to an external environment of the main body; And a controller for modifying information related to a position and a path of the main body using at least one of a laser sensor that scans an object existing around the main body and generates laser scan information and the image and the laser scan information, .

The controller may operate the camera sensor and the laser sensor at predetermined time intervals to generate an image related to the external environment of the main body and generate laser scan information.

In addition, the controller detects the position of the cleaner main body based on the image and laser scan information generated at predetermined time intervals, and matches the detected position with the image and the laser scan information And controls the storage unit to be stored in the database.

Thereafter, the controller compares the database and the image and laser scan information generated at the current point of time, and corrects the information related to the current position and path of the robot, at a position determined to return.

In one embodiment, the controller extracts information related to an image pattern included in the external environment using an image generated by the camera sensor.

In one embodiment, the controller extracts information related to a topographic shape formed by an object existing around the main body, using laser scanning information generated by the laser sensor.

In one embodiment, the communication device further includes a communication unit for receiving a signal related to the operation of the vacuum cleaner. When the communication unit receives a signal for returning the vacuum cleaner to the start of travel, The control unit controls the driving unit to return the driving unit to the driving unit.

In one embodiment, the image processing apparatus further includes a storage unit configured to store the image, the information related to the image pattern, the laser scan information, the information related to the topographic shape, and the database formed of the image and the position information on which the laser scan information is generated Wherein the control unit operates the camera sensor and the laser sensor at predetermined intervals, and generates the image generated by the camera sensor, the information related to the image pattern extracted from the image, and the information generated by the laser sensor Information related to the geographical shape extracted from the laser scan information, and information related to the location where the camera sensor and the laser sensor are operated are updated in the database.

In one embodiment, the control unit compares the database with the information related to the extracted image pattern and the geographical feature during the return to the driving start point, and sets information related to the route for returning to the driving start point .

In one embodiment, the control unit may include information related to the image pattern extracted at one point during the return to the driving start point, information related to the image pattern corresponding to the one point among the information related to the image pattern stored in the database And corrects the route for returning to the starting point of travel if the vehicle does not coincide,

In one embodiment, the control unit may include information related to the geographical shape extracted at one point during the return to the driving start point, information related to the geographical shape corresponding to the one point among information related to the geographical shape stored in the database, The route for returning to the travel start point is corrected.

In one embodiment, when the image pattern extracted at one point and the image pattern stored in the database coincide with each other during the return to the driving start point, , Compares the topographic form extracted at the current position of the main body, and corrects the route for returning to the starting point of travel based on the comparison result.

In one embodiment, if the laser scanning information generated at one point and the laser scanning information stored in the database coincide with each other during the return to the driving start point, Pattern is compared with an image pattern extracted at a current position of the main body, and a path for returning to the start point of travel is corrected based on the comparison result.

In one embodiment, the control unit includes a local path correction module and a global path correction module, wherein the local path correction module detects a plurality of frames related to the position of the robot using the laser scan information obtained at a plurality of points in time, .

In one embodiment, the local path correction module selects any one of the plurality of generated frames and acquires an image corresponding to the selected frame.

In one embodiment, the selected frame is a frame generated at a current position of the main body.

In one embodiment, the local path correction module performs the location recognition on the position corresponding to the selected frame based on the image corresponding to the selected frame.

In one embodiment, the local path correction module extracts a feature point from the image, generates a descriptor corresponding to the image based on the extracted feature point, and calculates a position corresponding to the selected frame based on the generated descriptor And determines whether or not to return to the home page.

In one embodiment, if it is determined that the position corresponding to the selected frame corresponds to a position corresponding to another frame, the global path correction module may perform a laser scan Information and the laser scanning information of the other frame are matched with each other.

In one embodiment, the global path correction module modifies the path of the cleaner based on a result of matching the laser scan information of the selected frame with the laser scan information of the other frame.

According to the present invention, it is possible to acquire the position of the cleaner more precisely by using the laser scan information together with the image of the periphery of the cleaner, and it is possible to travel in a planned route, thereby improving convenience of the cleaner user.

In addition, according to the present invention, in the case where the cleaner detects the current position or determines the running direction, by using two or more different sensors without depending only on one of the camera sensor and the laser sensor, It is possible to overcome the disadvantages of the sensor.

In addition, according to the present invention, when the cleaner detects the current position or determines the traveling direction, the camera sensor and the laser sensor are used together to reduce the influence of errors generated in the camera sensor and the laser sensor, respectively .

According to the present invention, in the case of a vacuum cleaner traveling along a closed curve, the position or path information of the vacuum cleaner is adjusted each time the closed curve is circulated one time, thereby causing a slip between the vacuum cleaner and the floor, It is possible to accurately detect the position and path information of the cleaner.

1A is a block diagram illustrating components of a mobile robot according to an embodiment of the present invention.
1B is a block diagram showing more detailed components of the sensing unit 140 included in the mobile robot of the present invention.
1C to 1E are conceptual diagrams showing the appearance of a mobile robot according to the present invention.
2A to 2C are conceptual diagrams showing an embodiment of a mobile robot having a camera sensor and a laser sensor according to the present invention.
FIG. 3A is a block diagram showing more detailed components of the control unit 180 included in the mobile robot of the present invention.
3B is a flowchart illustrating a method of detecting or correcting the position information of the robot using the image and the laser scan information according to the present invention.
3C is a conceptual diagram showing laser scanning information obtained by the mobile robot of the present invention at different points in time.
FIG. 3D is a conceptual diagram showing an embodiment in which the mobile robot of the present invention acquires a plurality of images or laser scanning information while traveling.
FIG. 3E is a flowchart illustrating a method of performing location recognition using the image of the mobile robot of the present invention.
FIG. 3F is a conceptual diagram illustrating a method in which the mobile robot of the present invention matches different laser scanning information.
4A to 4D are conceptual diagrams illustrating a method of determining a current position of a mobile robot and correcting a path using the camera sensor and the laser sensor according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following, FIG. 1A, components of a mobile robot according to an embodiment of the present invention will be described in detail.

1A, a mobile robot according to an embodiment of the present invention includes a communication unit 110, an input unit 120, a driving unit 130, a sensing unit 140, an output unit 150, a power source unit 160 ), A memory 170, and a controller 180, or a combination thereof.

At this time, it is needless to say that the components shown in FIG. 1A are not essential, so that a robot cleaner having components having more components or fewer components can be implemented. Hereinafter, each component will be described.

First, the power supply unit 160 includes a battery that can be charged by an external commercial power supply, and supplies power to the mobile robot. The power supply unit 160 supplies driving power to each of the components included in the mobile robot, and supplies operating power required for the mobile robot to travel or perform a specific function.

At this time, the controller 180 senses the remaining power of the battery, and when the remaining power is insufficient, controls the battery 180 to move to a charge connected to the external commercial power source, and receives the charging current from the charging stand to charge the battery. The battery may be connected to the battery sensing unit so that the battery remaining amount and the charging state may be transmitted to the control unit 180. The output unit 150 can display the battery remaining amount on the screen by the control unit.

The battery may be located at the bottom of the center of the robot cleaner, or may be located at either the left or right side. In the latter case, the mobile robot may further include a balance weight to eliminate weight biases of the battery.

Meanwhile, the driving unit 130 includes a motor. By driving the motor, the left and right main wheels can be rotated in both directions to rotate or move the main body. The driving unit 130 can advance the main body of the mobile robot forward, backward, leftward, rightward, curved, or rotated in place.

Meanwhile, the input unit 120 receives various control commands from the user for the robot cleaner. The input unit 120 may include one or more buttons, for example, the input unit 120 may include an OK button, a setting button, and the like. The OK button is a button for receiving a command for confirming the detection information, the obstacle information, the position information, and the map information from the user, and the setting button is a button for receiving a command for setting the information from the user.

Also, the input unit 120 may include an input reset button for canceling the previous user input and receiving the user input again, a delete button for deleting the preset user input, a button for setting or changing the operation mode, A button for receiving an input, and the like.

The input unit 120 may be installed on the upper portion of the mobile robot using a hard key, a soft key, a touch pad, or the like. In addition, the input unit 120 may have a form of a touch screen together with the output unit 150.

On the other hand, the output unit 150 can be installed on the upper portion of the mobile robot. Of course, the installation location and installation type may vary. For example, the output unit 150 may display a battery state, a traveling mode, and the like on a screen.

In addition, the output unit 150 can output the state information of the mobile robot, for example, the current state of each configuration included in the mobile robot, detected by the sensing unit 140. [ The output unit 150 may display the external status information, the obstacle information, the position information, the map information, and the like detected by the sensing unit 140 on the screen. The output unit 150 may include any one of a light emitting diode (LED), a liquid crystal display (LCD), a plasma display panel, and an organic light emitting diode (OLED) As shown in FIG.

The output unit 150 may further include sound output means for audibly outputting an operation process or an operation result of the mobile robot performed by the control unit 180. [ For example, the output unit 150 may output a warning sound to the outside in accordance with the warning signal generated by the control unit 180.

On the other hand, the communication unit 110 is connected to the terminal device and / or another device (referred to as "home appliance" in this specification) located in a specific area and a communication method of one of wired, wireless, And transmits and receives signals and data.

The communication unit 110 can transmit and receive data with other devices located in a specific area. In this case, the other device may be any device capable of connecting to a network and transmitting / receiving data. For example, the device may be an air conditioner, a heating device, an air purifier, a lamp, a TV, The other device may be a device for controlling a door, a window, a water supply valve, a gas valve, or the like. The other device may be a sensor for detecting temperature, humidity, air pressure, gas, or the like.

Accordingly, the control unit 180 can transmit the control signal to the other device through the communication unit 110, so that the other device can operate according to the received control signal. For example, when the other device is an air conditioner, it is possible to turn on the power or perform cooling or heating for a specific area according to a control signal, and in the case of a device for controlling a window, It can be opened at a certain rate.

In addition, the communication unit 110 may receive various status information from at least one other device located in a specific area. For example, the communication unit 110 may set the temperature of the air conditioner, whether the window is opened or closed, The current temperature of the specific area sensed by the temperature sensor, and the like.

Accordingly, the control unit 180 may generate the control signal for the other device according to the status information, the user input through the input unit 120, or the user input through the terminal apparatus.

At this time, the communication unit 110 may be connected to at least one of other wireless communication systems such as radio frequency (RF) communication, Bluetooth, infrared communication (IrDA), wireless LAN (LAN), Zigbee At least one communication method may be employed, and thus the other device and the mobile robot 100 may constitute at least one network. At this time, the network is preferably the Internet.

The communication unit 110 can receive a control signal from the terminal device. Accordingly, the control unit 180 can perform control commands related to various operations according to the control signal received through the communication unit 110. [ In addition, the communication unit 110 may transmit state information, obstacle information, location information, image information, map information, and the like of the mobile robot to the terminal device.

Meanwhile, the memory 170 stores a control program for controlling or driving the robot cleaner and data corresponding thereto. The memory 170 may store audio information, image information, obstacle information, location information, map information, and the like. Also, the memory 170 may store information related to the traveling pattern.

The memory 170 mainly uses a nonvolatile memory. Here, the non-volatile memory (NVM) is a storage device capable of continuously storing information even when power is not supplied. Examples of the storage device include a ROM, a flash memory, A storage device (e.g., a hard disk, a diskette drive, a magnetic tape), an optical disk drive, a magnetic RAM, a PRAM, and the like.

1B, the sensing unit 140 includes an external signal sensor 141, a front sensor 142, a cliff sensor 143, a lower camera sensor 144, an upper camera sensor 145, And a three-dimensional camera sensor 146.

The external signal detection sensor 141 can sense an external signal of the mobile robot. The external signal detection sensor 141 may be, for example, an infrared ray sensor, an ultrasonic sensor, a radio frequency sensor, or the like.

The mobile robot can receive the guidance signal generated by the charging signal using the external signal detection sensor 141 and confirm the position and the direction of the charging base. At this time, the charging base can transmit a guidance signal indicating a direction and a distance so that the mobile robot can return. That is, the mobile robot can receive the signal transmitted from the charging station, determine the current position, set the moving direction, and return to the charging state.

In addition, the mobile robot can detect a signal generated by a remote control device such as a remote controller or a terminal by using an external signal detection sensor 141.

The external signal detection sensor 141 may be provided on one side of the inside or outside of the mobile robot. For example, the infrared sensor may be installed in the mobile robot or in the vicinity of the lower or upper camera sensor 145 or the three-dimensional camera sensor 146 of the output unit 150.

On the other hand, the front sensor 142 may be installed at a predetermined distance along the outer circumferential surface of the mobile robot in front of the mobile robot, specifically, the side surface of the mobile robot. The front sensing sensor 142 is disposed on at least one side of the mobile robot to sense an obstacle ahead of the mobile robot. The front sensing sensor 142 senses an object, particularly an obstacle, To the control unit 180. That is, the front sensing sensor 142 may sense protrusions existing on the moving path of the mobile robot, household appliances, furniture, walls, wall corners, and the like, and may transmit the information to the controller 180.

The front sensor 142 may be, for example, an infrared sensor, an ultrasonic sensor, an RF sensor, a geomagnetic sensor, or the like. The mobile robot may use one type of sensor as the front sensor 142, The above sensors can be used together.

Ultrasonic sensors, for example, can typically be used to detect distant obstacles in general. The ultrasonic sensor includes a transmitter and a receiver. The controller 180 determines whether an obstacle is present or not based on whether the ultrasonic wave radiated through the transmitter is reflected by an obstacle or the like and is received by the receiver, The distance to the obstacle can be calculated.

Also, the controller 180 can detect the information related to the size of the obstacle by comparing the ultrasonic waves emitted from the transmitter and the ultrasonic waves received by the receiver. For example, the control unit 180 can determine that the larger the obstacle is, the more ultrasonic waves are received in the receiving unit.

In one embodiment, a plurality (e. G., Five) of ultrasonic sensors may be installed along the outer circumferential surface on the front side of the mobile robot. At this time, preferably, the ultrasonic sensor can be installed on the front side of the mobile robot alternately with the transmitting part and the receiving part.

That is, the transmitting unit may be disposed to be spaced left and right from the front center of the main body, and one or two transmitting units may be disposed between the receiving units to form a receiving area of the ultrasonic signal reflected from the obstacle or the like. With this arrangement, the receiving area can be expanded while reducing the number of sensors. The angle of origin of the ultrasonic waves can maintain an angle range that does not affect different signals to prevent crosstalk. Also, the receiving sensitivity of the receiving units may be set differently.

In addition, the ultrasonic sensor may be installed upward by a predetermined angle so that the ultrasonic wave emitted from the ultrasonic sensor is outputted upward, and the ultrasonic sensor may further include a predetermined blocking member to prevent the ultrasonic wave from being radiated downward.

As described above, the front sensing sensor 142 can use two or more kinds of sensors together. Accordingly, the front sensing sensor 142 can be any one of an infrared sensor, an ultrasonic sensor, and an RF sensor Can be used.

For example, the front sensing sensor 142 may include an infrared sensor in addition to the ultrasonic sensor.

The infrared sensor may be installed on the outer surface of the mobile robot together with the ultrasonic sensor. The infrared sensor can also detect the obstacles existing on the front or side and transmit the obstacle information to the controller 180. That is, the infrared sensor senses protrusions existing on the moving path of the mobile robot, house furniture, furniture, wall surface, wall edge, and the like, and transmits the information to the control unit 180. Therefore, the mobile robot can move within a specific area without collision with the obstacle.

On the other hand, the cliff sensor 143 (or Cliff Sensor) can detect obstacles on the floor supporting the main body of the mobile robot by mainly using various types of optical sensors.

That is, the croft detection sensor 143 is provided on the rear surface of the floor mobile robot, but it may be installed in a different position depending on the type of the mobile robot. The obstacle detection sensor 143 is located on the back surface of the mobile robot and detects an obstacle on the floor. The obstacle detection sensor 143 may include an infrared sensor including an emitting unit and a light receiving unit, an ultrasonic sensor, An RF sensor, a position sensitive detector (PSD) sensor, and the like.

For example, any one of the cliff detection sensors 143 may be installed in front of the mobile robot, and the other two cliff detection sensors 143 may be installed relatively behind.

For example, the cliff detection sensor 143 may be a PSD sensor, but it may be composed of a plurality of different kinds of sensors.

The PSD sensor uses a semiconductor surface resistance to detect the shortest path distance of incident light at one p-n junction. The PSD sensor includes a one-dimensional PSD sensor for detecting light in only one direction and a two-dimensional PSD sensor for detecting a light position on a plane, all of which can have a pin photodiode structure. The PSD sensor is a type of infrared sensor that measures the distance by measuring the angle of the infrared ray reflected from the obstacle after transmitting the infrared ray by using the infrared ray. That is, the PSD sensor uses the triangulation method to calculate the distance to the obstacle.

The PSD sensor includes a light emitting unit that emits infrared rays to an obstacle, and a light receiving unit that receives infrared rays that are reflected from the obstacle and is returned to the obstacle. When an obstacle is detected by using the PSD sensor, a stable measurement value can be obtained irrespective of the reflectance and the color difference of the obstacle.

The control unit 180 can detect the depth of the cliff by measuring the infrared angle between the infrared light emission signal emitted by the cliffing sensor 143 toward the ground and the reflection signal reflected by the obstacle.

On the other hand, the control unit 180 can determine whether to pass or not according to the ground state of the detected cliff by using the cliff detection sensor 143, and determine whether the cliff passes or not according to the determination result. For example, the control unit 180 determines whether a cliff is present or not through the cliff detection sensor 143, and then passes through the cliff only when the reflection signal is sensed through the cliff detection sensor 143.

As another example, the control unit 180 may determine the lifting phenomenon of the mobile robot using the deterioration sensor 143.

On the other hand, the lower camera sensor 144 is provided on the back side of the mobile robot, and acquires image information on the lower side, that is, the bottom surface (or the surface to be cleaned) during the movement. The lower camera sensor 144 is also referred to as an optical flow sensor in other words. The lower camera sensor 144 converts the downward image input from the image sensor included in the sensor to generate image data of a predetermined format. The generated image data can be stored in the memory 170.

The lower camera sensor 144 may further include a lens (not shown) and a lens controller (not shown) for adjusting the lens. It is preferable to use a pan-focus lens having a short focal length and a deep depth as the lens. The lens control unit includes a predetermined motor and moving means for moving the lens back and forth to adjust the lens.

Also, one or more light sources may be installed adjacent to the image sensor. The at least one light source irradiates light onto a predetermined area of the bottom surface, which is photographed by the image sensor. That is, when the mobile robot moves in a specific region along the bottom surface, a certain distance is maintained between the image sensor and the bottom surface when the bottom surface is flat. On the other hand, when the mobile robot moves on the bottom surface of a nonuniform surface, it is distant by a certain distance due to the unevenness and obstacles on the bottom surface. At this time, one or more light sources may be controlled by the control unit 180 to adjust the amount of light to be irradiated. The light source may be a light emitting device capable of adjusting the amount of light, for example, an LED (Light Emitting Diode).

Using the lower camera sensor 144, the control unit 180 can detect the position of the mobile robot regardless of the slip of the mobile robot. The control unit 180 may compute and analyze the image data photographed by the lower camera sensor over time to calculate the moving distance and the moving direction, and calculate the position of the mobile robot on the basis of the movement distance and the moving direction. By using the image information of the lower portion of the mobile robot using the lower camera sensor 144, the controller 180 can perform a correction that is robust against slippage with respect to the position of the mobile robot calculated by other means.

On the other hand, the upper camera sensor 145 is installed to face upward or forward of the mobile robot, and can photograph the surroundings of the mobile robot. When the mobile robot includes a plurality of upper camera sensors 145, the camera sensors may be formed on the upper or side surface of the mobile robot at a predetermined distance or at a certain angle.

The upper camera sensor 145 may further include a lens for focusing the subject, an adjusting unit for adjusting the camera sensor, and a lens adjusting unit for adjusting the lens. The lens uses a wide angle lens so that all surrounding areas, for example, all areas of the ceiling, can be photographed even at a predetermined position. For example, a lens whose angle of view is a certain angle, for example, 160 degrees or more.

The laser sensor 147 is attached to one side or a part of the body of the mobile robot to scan the omnidirectional side of the body, thereby generating laser scanning information related to an object existing within the measurement range from the body. That is, the laser sensor 147 can detect the distance between the object and the main body by outputting the light source to the objects located around the main body.

Specifically, the laser sensor 147 may include a laser transmission unit, a laser detection unit, a signal processing unit, and a signal communication unit. The laser sensor 147 can be classified into a time of flight (TOF) method and a phase-shift method according to a modulation method of a laser signal.

When the laser sensor 147 is in the TOF mode, the laser sensor 147 emits a pulse signal to measure the time at which the reflected pulse signals from the objects within the measurement range arrive at the receiver, You can measure the distance between objects.

When the laser sensor 147 is of the phase-shift type, the laser sensor 147 emits a laser beam having a specific frequency continuously modulated, and measures a phase change amount of a signal reflected back from an object within the measurement range , The distance between the mobile robot and objects within the measurement range can be measured.

For example, the laser sensor 147 may be a Lidar sensor.

2A to 2C, an embodiment of a mobile robot having a camera sensor and a laser sensor according to the present invention will be described.

As shown in Fig. 2A, the mobile robot 100 can move along a predetermined path 201 within an arbitrary operating area 1000. [0050] As shown in Fig.

For example, the predetermined path 201 may be a path that moves along the wall defining the working area 1000. In yet another example, the preset path 201 may form a closed curve. Therefore, when the mobile robot 100 travels along the preset path 201, it can return to the point where the traveling has started.

Referring to FIG. 2B, the mobile robot 100 may acquire an image related to the periphery of the mobile robot 100 using the camera sensor at specific time intervals while traveling along the path 201 shown in FIG. 2A . As shown in FIG. 2B, each time the mobile robot 100 is located at the plurality of points 202a, an image related to at least one of front, side, rear, upper, and lower sides of the main body can be obtained.

The mobile robot 100 also uses the laser sensor at specific time intervals along the path 201 shown in Fig. 2A to scan the main body of the mobile robot 100 with laser scanning information related to objects existing within the measurement range Can be obtained. That is, each time the mobile robot 100 is positioned at the plurality of points 202a, it is possible to control the laser sensor 147 to scan an object existing within the measurement range with respect to the main body, .

The mobile robot 100 may set the operation cycle of the camera sensor and the laser sensor to the same or set the operation cycle of the camera sensor to a multiple of the operation cycle of the laser sensor, It can be set to multiples.

That is, as shown in FIG. 2A, the mobile robot 100 travels along a closed curve that defines the operation area 1000, and uses the camera sensor at a predetermined time interval until the mobile robot 100 returns to the start of travel, ) Of the image. The mobile robot 100 travels along a closed curve defining the operation area 1000 and uses the laser sensor 147 at predetermined time intervals until returning to the start of travel, Related laser scanning information can be obtained.

FIG. 2B illustrates an embodiment of running along a closed curve defining an operating region 1000, and detecting position information for each of a plurality of points 202a.

That is, when the mobile robot 100 travels along the closed curve and travels from the travel start point Pi to the travel end point Pf, the control unit 180 determines whether or not it is revisited for each of the plurality of points 202a can do. For example, when the mobile robot 100 circulates along the closed curve, the control unit 180 may determine that the starting point Pi and the ending point Pf are the same point.

Referring to FIG. 2B, the controller 180 of the mobile robot 100 may scan the plurality of points 202a using at least one of the laser scanning information obtained for each of the plurality of points 202a and the encoder and the wheel sensors. The corresponding position information can be detected. In addition, the control unit 180 accumulates information related to the moving direction or the moving distance of the mobile robot 100, and can detect the path information based on the accumulated information.

However, if the controller 180 detects a slip on the driving wheels of the mobile robot 100 or an error occurs in the encoder and the wheel sensor in the process of detecting the position information and the route information, May occur.

That is, when an error occurs in the information related to the moving direction or the moving distance of the mobile robot 100 at any one of the plurality of points 202a, the position information or the laser scanning information detected at the point after the error occurs Error is reflected.

2B, when the control unit 180 compares the first image 203a with the second image 203b, it is determined that the driving start point Pi and the driving end point Pf are substantially the same point , There may be an error between the position information corresponding to the travel start point Pi and the position information corresponding to the travel end point Pf.

2C, when the first image 203a and the second image 203b are compared with each other, the control unit 180 determines that the driving start point Pi and the driving end point Pf are substantially the same It is possible to determine whether the position information corresponding to the driving start point Pi and the driving end point Pf match the laser scanning information.

If it is determined that there is an error between the position information corresponding to the driving start point Pi and the laser scanning information and the position information corresponding to the driving end point Pf and the laser scanning information, It is possible to match the position information corresponding to the starting point Pi and the traveling end point Pf with the laser scanning information. Accordingly, the control unit 180 can modify the route information 202b corresponding to the plurality of points 202a.

In the following FIG. 3A, the more detailed components of the control unit 180 included in the mobile robot of the present invention will be described.

The control unit 180 may include a local path correction module 181 and a global path correction module 182. [ The local path correction module 181 corrects the path or position information of the mobile robot 100 with respect to the local area and the global path correction module 182 can perform correction on the entire travel path. Embodiments related to the local path correction module 181 and the global path correction module 182 will be described separately below. The functions performed by the local path correction module 181 and the global path correction module 182, however, (180) executes.

The global path correction module 182 may also include at least one of a key frame management unit 182a, a place recognition unit 182b, a place matching unit 182c, and a mapping unit 182d.

3B, a method of modifying the position or path of the mobile robot 100 by the control unit 180 including the local path correction module 181 and the global path correction module 182 will be described.

The local path correction module 181 may generate a plurality of frames related to the position of the robot using the laser scan information obtained at a plurality of points of view (S301).

In addition, the local path correction module 181 can select any one of the generated frames (S302), and obtain an image corresponding to the selected frame (S303).

For example, the frame selected by the local path correction module 181 may be a frame generated at the current position of the mobile robot 100 or at the current position of the cleaner main body.

As another example, the frame selected by the local path correction module 181 may be the most recently generated frame.

The global path correction module 182 may perform the location recognition on the location corresponding to the selected frame based on the image corresponding to the selected frame (S304). That is, when the local path correction module 181 selects a frame generated at the current position, the global path correction module 182 can perform the position recognition on the current position based on the image photographed at the current position .

The global path correction module 182 may use the location recognition result to match the selected frame with information associated with the recognized location (S305).

The global path correction module 182 can modify the position information of the mobile robot 100 included in the plurality of frames using the matching result (S306).

3C and 3D, an embodiment in which the local path correction module 181 performs local path correction is described.

More specifically, the local path correction module 181 of the control unit 180 determines whether or not the current position of the mobile robot 100 is coincident with the current position of the mobile robot 100, using co- registration between a plurality of laser scan information obtained by the laser sensor 147. [ The position can be estimated.

The local path correction module 181 can control the laser sensor 147 so that the laser sensor 147 scans the periphery of the main body of the mobile robot 100 at a predetermined time interval to acquire a plurality of laser scan information.

3C, the local path correction module 181 calculates the first laser scan information 310a obtained at the first point-in-time t-1 and the second laser scan information 310a obtained at the second point- The Euclidean distance between the two points 310a and 310b can be detected. In addition, the local path correction module 181 may set the detected two points as a corresponding point.

Next, the local path correction module 181 can detect information related to the translational momentum and the rotational momentum that make the positions of the set points coincident. The local path correction module 181 calculates the movement amount of the first laser scanning information 310a by using the translational momentum and the rotational momentum when the distance between the coincidence point of the first laser scanning information 310a and the coincidence point of the second laser scanning information 310b decreases below the reference value Information related to the displacement of the robot 100 can be detected. Thus, the local path correction module 181 can detect position information related to the current point of the mobile robot 100 using the position information related to the point where the mobile robot 100 starts to move and the detected displacement.

For example, the local path correction module 181 can estimate the position of the mobile robot 100 using an ICP (Iterative Closest Point) algorithm.

3D, the local path correction module 181 of the mobile robot 100 may generate a plurality of frames 320a, 320b, 320c, 320d, and 320e while traveling within the operation area 1000. [

Hereinafter, a frame means a unit of data formed by an image and laser scan information corresponding to each of a plurality of points. In addition, some of the plurality of frames may be set as key frames.

3D, the local path correction module 181 photographs an image related to the external environment of the main body while the mobile robot 100 travels within the operation area 1000, By scanning objects and generating laser scan information, a plurality of frames 320a, 320b, 320c, 320d, and 320e can be generated. In addition, the controller 180 may set the frame 320a and the frame 320e as key frames.

When the mobile robot 100 generates a plurality of frames in this manner, the local path correction module 181 can calculate the position change amount of the mobile robot 100 using one frame and the frame generated next have. The local path correction module 181 can sequentially calculate the position change amount for a plurality of frames.

3D, when calculating the position change amount between the frame 320d and the frame 320e, the errors generated when calculating the position change amount in the previous frame are accumulated, and the local path correction module 181 When the position of the mobile robot 100 is finally detected with respect to the frame 320e, a relatively large error may occur.

In the present invention, as a method for minimizing such an error, the position change amount can be calculated by comparing the laser scan information corresponding to the key frame with the laser scan information of the current frame.

That is, as shown in FIG. 3D, the local path correction module 181 determines whether or not the key frame 320a and the current key frame 320c are present after setting any one key frame 320a, It is possible to calculate the position change amount of the mobile robot 100 by comparing the frames.

In one embodiment, the local path correction module 181 may set the current frame to the next key frame if the distance between the set key frame and the current frame increases beyond the reference distance.

In another embodiment, the local path correction module 181 may set the last generated frame to the next key frame if a predetermined number of frames are generated after an arbitrary key frame is set.

3E, an embodiment in which the global path correction module 182 modifies the position information or the path information of the mobile robot 100 will be described.

The location recognition unit 182b of the global path correction module 182 can perform the location recognition for the current position of the mobile robot 100 using the camera sensor.

Specifically, the place recognizing unit 182b can perform a similarity check by comparing an image generated at a current camera sensor with a database formed by including a plurality of images related to a plurality of positions.

The place recognition unit 182b can extract a plurality of feature points from the image generated by the current camera sensor (S314).

For example, the location recognition unit 182b may use an open source such as OpenFABMAP to perform location recognition using any image.

The place recognition unit 182b can generate a descriptor corresponding to the image generated by the current camera sensor based on the extracted feature point (S324).

That is, the location recognizing unit 182b may perform clustering of the extracted minutia points and generate a descriptor corresponding to each of the plurality of clusters generated as the clustering result.

The place recognizing unit 182b judges whether or not to return to a location corresponding to the generated descriptor by using a database formed by a plurality of place information, image information corresponding to the place information, and the generated descriptor (S334).

If it is determined that the mobile robot 100 first visits the location corresponding to the generated descriptor, the location recognition unit 182b may update the generated descriptor to the database (S344).

When it is determined that the mobile robot 100 is to return to the position corresponding to the generated descriptor in the place recognizing unit 182b, the location matching unit 182c compares the laser scan information of the frame stored in the database, (Step S305). Hereinafter, the function of this place matching unit 182c is defined as a place match or place match.

That is, when the location matching unit 182c determines that the location corresponding to the selected frame matches the location corresponding to another frame, the location matching unit 182c determines that the selected frame The laser scanning information of the other frame can be matched with the laser scanning information of the other frame.

In addition, the mapping unit 182d included in the global path correction module 182 can modify the path of the cleaner based on the matching result between the laser scan information of the selected frame and the laser scan information of the other frame.

Specifically, in Fig. 3F, an embodiment in which the place matching unit 182c performs the place matching will be described.

If it is determined that the first frame generated by the mobile robot 100 at the travel start point and the n-th frame generated at the travel end point are the same place as shown in Fig. 3F, the place matching unit 182c The laser scan information 330a of the first frame and the laser scan information 330b of the nth frame can be matched.

When the location matching unit 182c determines that the mobile robot 100 is to return to the position corresponding to the generated descriptor in the place recognizing unit 182b, the laser scanning information 330a of the first frame, Information relating to the constraint between the laser scanning information 330b of the frame can be detected. For example, the information related to the constraint condition may include a relative displacement value between the laser scan information 330a of the first frame and the laser scan information 330b of the nth frame.

When the constraint condition is set as above, the place matching unit 182c can modify the information included in the plurality of frames generated before the n-th frame. Referring to FIG. 3F, the location matching unit 182c may modify the laser scanning information or the position information corresponding to a part of the first to n-th frames.

Accordingly, the control unit 180 or the local path correction module 181 of the mobile robot 100 can generate the position information 340a related to a plurality of frames primarily on the move.

If it is determined by the place recognizing unit 182b that the position corresponding to the first frame is the same as the position corresponding to the Nth frame, the place matching unit 182c compares the laser scan information 330a of the first frame The laser scanning information 330b of the Nth frame can be matched.

The mapping unit 182d may modify the primarily generated position information 340a based on the matching result performed by the place matching unit 182c so that the position information 340b related to the plurality of frames is obtained, Lt; / RTI > That is, the mapping unit 182d can modify the information related to the path that the mobile robot 100 has traveled.

The position information 340b generated secondarily by the mapping unit 182d in Fig. 3F may correspond to some point in the closed curve defining the operating region 1000 of the mobile robot 100 shown in Fig. 2A.

4A to 4D, a method of determining the current position of the mobile robot and correcting the path using the camera sensor and the laser sensor according to the present invention will be described.

Referring to FIG. 4A, the mobile robot 100 can acquire image and laser scan information at predetermined time intervals while starting to travel at the starting point P1. That is, the control unit 180 operates the camera sensor and the laser sensor at predetermined time intervals while the main body of the mobile robot 100 moves along the first travel path M1, And generate laser scanning information formed by an object located in the periphery of the main body.

The control unit 180 can extract information related to the image pattern included in the external environment of the main body of the mobile robot 100 using the image generated by the camera sensor.

The control unit 180 may extract information related to the topographic form formed by the objects existing around the main body using the laser scanning information generated by the laser sensor.

For example, the operation area 400 may include a first room ROOM 1 and a second room ROOM 2, and the first door 401 and the second door may have a first door 401 and a second door 402, respectively. A second door 403 may be provided. In addition, a pattern member 402 having a recognizable image pattern may be attached to one side of the first door 402. In this case, the travel starting point P1 may be located inside the first room.

4A, the communication unit 110 of the mobile robot 100 can receive a return signal including an instruction to return to the travel start point P1 during traveling, and the point at which the return signal is received And is defined as a return point P2.

That is, the communication unit 110 can receive a signal related to the operation of the mobile robot or the vacuum cleaner. When the communication unit receives a signal for returning the mobile robot or the vacuum cleaner to the start point, The driving unit 130 may be controlled to return to the driving start point.

The memory 170 stores the image captured by the camera sensor, the information related to the image pattern recognized from the image, the laser scan information generated by the laser sensor, the periphery of the mobile robot 100 extracted from the laser scan information, And information related to the topographic shape of the laser scan information and position information on the image and the laser scan information are generated.

The control unit 180 may operate the camera sensor at a predetermined period and update the database with information related to the image generated by the camera sensor and the image pattern extracted from the image every predetermined period.

The control unit 180 may operate the laser sensor at a predetermined cycle and update the database with information related to the laser scan information generated by the laser sensor and the geometric shape extracted from the laser scan information at the predetermined period have.

In addition, the control unit 180 may update the database with information related to the position of the mobile robot 100 at the time the camera sensor and the laser sensor are operated.

The control unit 180 also compares the database with the information related to the extracted image pattern and the geographical feature during the return to the start point P1 to determine the information related to the path for returning to the start point P1 Can be set.

The control unit 180 determines whether information related to the image pattern extracted at one point during the return to the travel start point P1 and information related to the image pattern corresponding to the one point among the information related to the image pattern stored in the database is If they do not match, the route for returning to the travel start point P1 can be modified.

In addition, the control unit 180 may store information related to the geographical shape extracted at one point in the course of returning to the travel start point P1 and information related to the geographical shape corresponding to the one point among the information related to the geographical shape stored in the database It is possible to modify the route for returning to the travel start point P1.

On the other hand, the controller 180 may determine whether any one of the image patterns extracted at one point and the image patterns stored in the database coincide with each other during the return to the driving start point P1. If it is determined that there is a match, the controller 180 compares the topographical shape corresponding to any one of the image patterns with the topographical shape extracted from the current position of the main body, and based on the comparison result, The path to be updated can be modified.

In addition, the controller 180 can determine whether any one of the laser scan information generated at one point and the laser scan information stored in the database coincide with the return to the start point P1. The control unit 180 compares the image pattern corresponding to any one of the laser scan information with the image pattern extracted from the current position of the main body and returns to the start point of the travel based on the comparison result The path to be updated can be modified.

4A, while the mobile robot 100 moves along the second travel route M2 in order to return from the return point P2 to the travel start point P1, The path of the mobile robot 100 can be determined using the image generated by the sensor and the laser scan information generated by the laser sensor together.

That is, while moving along the second travel path M2 of the mobile robot 100, the controller 180 can detect information related to the topographic form from the laser scan information and extract the image pattern from the image .

For example, referring to FIG. 4A, information related to the topographic form may include information related to the location of the first door 401 and the second door 403, the type of wall forming the first and second rooms, .

For example, referring to FIG. 4A, the extracted image pattern may be an image pattern displayed on one side of the pattern member 401.

4B and 4A, when the mobile robot 100 moves along the first travel path M1, the position of the pattern member 402 moves from the first door 401 to the second door 403 ).

In this case, if the control unit 180 sets a path for returning to the travel start point P1 using only the camera sensor, when the mobile robot 100 passes through the third point P3, the control unit 180 moves It is possible to generate an error in detecting the position information of the robot 100.

That is, the control unit 180 recognizes the image pattern displayed on the pattern member 402 attached to the second door 403 from the image photographed at the third point P3 while returning along the third travel path M3 And the second room ROOM 2 is mistaken as the first room ROOM 1 according to the recognition result.

Meanwhile, the control unit 180 of the mobile robot 100 according to the present invention sets the return path by using the camera sensor and the laser sensor together, so that the error as shown in FIG. 4B can be prevented.

That is, even if the control unit 180 recognizes the image pattern displayed on the pattern member 402 attached to the second door 403 when the mobile robot 100 passes through the third point P3, The second room ROOM 2 can be distinguished from the first room ROOM 1 in consideration of the laser scanning information corresponding to the third room P3.

4C and FIG. 4A, the third door 404 is added after the mobile robot 100 moves along the first travel route M1.

In this case, when the control unit 180 sets a path for returning to the travel start point P1 using only the laser sensor, when the mobile robot 100 passes through the fourth point P4, the control unit 180 moves It is possible to generate an error in detecting the position information of the robot 100.

That is, from the laser scanning information generated at the fourth point P4 during the return along the third travel path M3, the control unit 180 determines whether the topological information And the second room ROOM 2 is mistaken as the first room ROOM 1 according to the geographical shape.

On the other hand, the control unit 180 of the mobile robot 100 according to the present invention sets the return path by using the camera sensor and the laser sensor together, thereby preventing the error as shown in FIG. 4C.

That is, even when the control unit 180 extracts information related to the topography formed by the second door 403 and the third door 404 when the mobile robot 100 passes through the fourth point P4, The second room ROOM 2 may be distinguished from the first room ROOM 1 in consideration of the image corresponding to the fourth point P4 or the image pattern recognized from the image.

4D and FIG. 4A, when the mobile robot 100 moves along the first travel path M1, the position of the pattern member 402 moves from the first door 401 to the second door 403 ), And the third statement 404 is added.

4D, even if the control unit 180 of the present invention uses the camera sensor and the laser sensor together, the image pattern and the laser scan information recognized at the fifth point P5 are moved along the first travel path M1 It may be similar to the recognized image pattern and laser scan information after leaving the first room ROOM 1 during movement.

Accordingly, in accordance with an embodiment of the present invention, the control unit 180 sets the position and path information of the mobile robot 100 using the camera sensor and the laser sensor, It is preferable to correct the signal by using an external signal detection sensor 141, a front detection sensor 142, a cliff detection sensor 143 or the like included in the sensing unit 140.

According to the present invention, it is possible to acquire the position of the cleaner more precisely by using the laser scan information together with the image of the periphery of the cleaner, and it is possible to travel in a planned route, thereby improving convenience of the cleaner user.

In addition, according to the present invention, in the case where the cleaner detects the current position or determines the running direction, by using two or more different sensors without depending only on one of the camera sensor and the laser sensor, It is possible to overcome the disadvantages of the sensor.

In addition, according to the present invention, when the cleaner detects the current position or determines the traveling direction, the camera sensor and the laser sensor are used together to reduce the influence of errors generated in the camera sensor and the laser sensor, respectively .

According to the present invention, in the case of a vacuum cleaner traveling along a closed curve, the position or path information of the vacuum cleaner is adjusted each time the closed curve is circulated one time, thereby causing a slip between the vacuum cleaner and the floor, It is possible to accurately detect the position and path information of the cleaner.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

Claims (17)

main body;
A driving unit for providing driving force for moving the main body;
A camera sensor for generating an image associated with the external environment of the body;
A laser sensor for scanning an object existing around the main body to generate laser scanning information;
Extracting information related to an image pattern included in the external environment using an image generated by the camera sensor,
Extracting information related to a topographic form formed by an object existing around the main body using laser scanning information generated by the laser sensor,
A controller for modifying information related to the position and the route of the main body using information related to the image pattern and information related to the topographic type,
And a storage unit configured to store a database formed of the image, the information related to the image pattern, the laser scan information, the information related to the topographic shape, and the position information on which the image and the laser scan information are generated,
Wherein,
The camera sensor and the laser sensor are operated at predetermined intervals,
Information relating to the image pattern extracted from the image, the laser scan information generated by the laser sensor, and the geometric shape extracted from the laser scan information And information related to a position where the camera sensor and the laser sensor are operated is updated in the database. delete delete The method according to claim 1,
Further comprising a communication unit for receiving a signal related to the operation of the vacuum cleaner,
Wherein,
Wherein the control unit controls the driving unit to return the cleaner to the driving start point when the communication unit receives a signal for returning the cleaner to the driving start point. delete The method according to claim 1,
Wherein,
Wherein the information relating to the extracted image pattern and the geographical shape is returned to the driving start point by comparing the database with information related to the extracted image pattern and the geographical shape to set the information related to the route for returning to the driving start point. The method according to claim 6,
Wherein,
If the information related to the image pattern extracted at one point during the return to the driving start point and the information related to the image pattern corresponding to the one point among the information related to the image pattern stored in the database do not match, Wherein the path for returning to the self-traveling mode is modified. The method according to claim 6,
Wherein,
If the information related to the geographical shape extracted at one point during the return to the driving start point and the information related to the geographical shape corresponding to the one point among the information related to the geographical shape stored in the database do not match, Wherein the path for returning to the self-traveling mode is modified. The method according to claim 6,
Wherein,
If one of the image pattern extracted from the one point and the image pattern stored in the database coincide with each other in the course of returning to the travel start point, the topographic pattern corresponding to the one of the image patterns, Comparing topographical forms,
And corrects a path for returning to the starting point of travel based on the comparison result. The method according to claim 6,
Wherein,
Wherein when one of the laser scan information generated at the one point and the laser scan information stored in the database coincide with each other during the return to the travel start point, the image pattern corresponding to any one of the laser scan information, The extracted image patterns are compared,
And corrects a path for returning to the starting point of travel based on the comparison result. The method according to claim 1,
Wherein the control unit includes a local path correction module and a global path correction module,
Wherein the local path correction module comprises:
Wherein a plurality of frames related to the position of the robot are generated using the laser scanning information obtained at a plurality of time points. 12. The method of claim 11,
Wherein the local path correction module comprises:
Selects one of the plurality of generated frames, and acquires an image corresponding to the selected frame. 13. The method of claim 12,
Wherein the selected frame is a frame generated at a current position of the main body. 13. The method of claim 12,
Wherein the local path correction module comprises:
And performs a position recognition on a position corresponding to the selected frame based on the image corresponding to the selected frame. 15. The method of claim 14,
Wherein the local path correction module comprises:
Extracting feature points from the image,
Generating a descriptor corresponding to the image based on the extracted feature points,
And determines whether to return to a position corresponding to the selected frame based on the generated descriptor. 15. The method of claim 14,
Wherein the global path correction module comprises:
When it is determined that the position corresponding to the selected frame is the same as the position corresponding to the other frame, the laser scanning information of the selected frame and the laser scanning information of the other frame are matched with each other Wherein the self-propelled vehicle is a self-propelled vehicle. 17. The method of claim 16,
Wherein the global path correction module comprises:
And corrects the path of the cleaner based on a result of matching the laser scan information of the selected frame with the laser scan information of the other frame.

Images