KR100429976B1 - Behavior learning heighten method of robot - Google Patents

Abstract

The present invention relates to a behavior learning method of a robot, which accepts behavior patterns of other robots or rules specified by a user through a network and learns the behavior patterns of the robot. To this end, the present invention includes a first process of receiving a behavior pattern of another robot through a network; A second process of temporarily increasing a behavior selection probability for the actions associated with the behavior pattern of the first process to express the corresponding behavior; A third step of determining whether the user's response is favorable by judging the user's response to the presently expressed behavior by the user's praise or scolding; As a result of the determination of the third process, if the user's response is favorable, the fourth process increases the probability of selecting a behavior for the related actions.

Description

BEHAVIOR LEARNING HEIGHTEN METHOD OF ROBOT}

The present invention relates to a behavior learning method of a robot, and more particularly, to a behavior learning method of a robot to further learn the behavior of another robot through a network.

In general, service robots and pet robots perform simple reinforcement learning on specific behaviors by awaking user feedback.

That is, the robot recognizes the user's praise when stroking the touch sensor on the head or neck, or when a voice word corresponding to a positive meaning is recognized, and scolds when a strong hitting the touch sensor or a voice word corresponding to a negative meaning is recognized. To be recognized.

As described above, when a feedback corresponding to a compliment is received, the behavior is strengthened by increasing a subsequent expression rate of only the behavior that was just performed, and when a feedback corresponding to a scolding is received, only the behavior that was just performed Subsequently, this behavior is attenuated by lowering the expression rate.

In other words, the behavior learning method of the conventional robot has taken a method of reinforcing only the behavior corresponding to the user's feedback information about a specific behavior, and this method simply obeys the robot's words. Create a passive robot that only does fractional learning,

In addition, since the learning is performed only by the user, the amount of learning information is insufficient, which causes the user to easily feel bored with the robot.

The present invention has been made to solve the above problems, to provide a behavior learning method of the robot to learn the behavior patterns of other robots through the network or the rules specified by the user to learn their behavior patterns. There is a purpose.

1 is a block diagram showing the configuration of a device to which the behavior learning method of the present invention is applied.

Figure 2 is a flow chart for the behavior learning method of the present invention robot.

Figure 3 is a schematic diagram showing the operation of the behavior learning method of the robot in Figure 2;

***** Description of the symbols for the main parts of the drawings *****

110: motion control unit 111: digital signal processing unit

112: PWM generator 113-1 to 113-N: Motor

1 to N: Potentiometer 120: Motion control unit

121: microprocessor 122: flash memory

The present invention for achieving the above object, the first process of accepting the behavior pattern of the other robot through the network; A second process of temporarily increasing a behavior selection probability for the actions associated with the behavior pattern of the first process to express the corresponding behavior; A third step of determining whether the user's response is favorable by judging the user's response to the presently expressed behavior by the user's praise or scolding; As a result of the determination of the third process, if the user's response is favorable, the fourth process may increase the action selection probability for the actions to be related.

Hereinafter, with reference to the accompanying drawings, the action and effect of the behavior learning method of the robot according to the present invention will be described in detail.

Figure 1 is a block diagram showing the configuration of the device to which the behavior learning method of the present invention is applied, as shown in the movement pattern of other robots or user's learning rules through a wireless LAN as shown in the movement through the physical link A motor controller 110 which transmits to the controller and controls the driving of the motors 113-1 to 113-N according to the command signal of the motion controller 120; After expressing the behavior pattern of the other robot or the learning rules of the user inputted from the motor control unit 110, this is determined by the movement pattern of the robot by an external stimulus input through the touch panel and the microphone and the command signal corresponding thereto. It consists of a motion control unit 120 to transmit to the motor control unit 110.

The motor controller 110 may include a wireless LAN (not shown) for receiving behavior patterns of other robots or user instructions; A sensor 114 for sensing an external stimulus; An image capture 116 for capturing an external image through the CD 115; The sensor 114 receives an image of the stimulus and the image gap 116 and transmits the image to the motion controller 120 through the physical link, and sends a command signal for a specific operation transmitted from the motion transmitter 120. A digital signal processor 111 for digital signal processing and outputting a motor driving control signal accordingly; Receives the motor drive control signal of the digital signal processor 111 and outputs a pulse width modulated signal for generating an operation speed corresponding to the motor drive control signal, and the position of a specific operation corresponding to the motor drive control signal. PWM generator 112 for outputting a position signal for indicating a; A plurality of motors 113-1 to 113-N for operating the respective links according to the pulse width modulation signal and the position signal of the PWM generator 112; The position of the current joint through the link is detected, the error of the motion according to the command is detected and the potentiometer for readjusting the detected error.

The motion control unit 120 includes: an audio codec unit 124 that receives a voice signal through a microphone 125 and codes the same; A flash memory 122 for storing an operating program and an application program; A touch panel 123 formed integrally with the LCD panel and receiving an external magnetic pole; A RAM 127 for storing voice signals and external stimuli input through the audio codec unit 124 and the touch panel 123; After processing the external stimulus and voice signal input through the audio codec unit 124 and the touch panel 123 to determine the behavior of the robot according to the predetermined signal, the command signal for the behavior of the robot through the physical link motor It consists of a microprocessor 121 for transmitting to the control unit 110.

The physical link consists of an I / O bus, USB, or RS232-C cable.

FIG. 2 is a flow chart illustrating a behavior learning method of a robot, and a first process of receiving a behavior pattern of another robot through a network as shown in the drawing; A second process of temporarily increasing a behavior selection probability for the actions associated with the behavior pattern of the first process to express the corresponding behavior; A third step of determining whether the user's response is favorable; As a result of the determination of the third step, if the user's response is favorable, the fourth step of increasing the action selection probability for the actions to be related is described.

First, the robot has a behavior selection process (Behavior Selection) that can select the action according to the internal state, the internal state is the emotion (emotion) such as anger, pleasure, surprise, and the desire (Motivation) such as appetite, sexual desire Is done.

The action selection process is performed aperiodically by a microprocessor, and the motion information of the selected robot is approximated by representing the stationary attitude of the robot at a specific time interval. Same as the principle of (Animation).

However, as the movie is represented by the image information, the robot's stationary position is represented by the current indication value of all joints of the robot. In the case of the rotation joint, the current indication value is an angle, and in the linear joint, the current indication value is displaced. do.

At this time, the instruction values of all the joints representing the robot's stationary position are defined as a frame, and a frame created in a time series is defined as motion information.

Here, when the operation of the present invention is described, first, after analyzing the actions of the robot executable by the current mechanical device, and implements the Commom-Sense Stereotype DB by setting the connection ratio of each action, the Commom-Sense Stereotype The database modifies or adds specific items to adjust the connection ratio between arbitrary actions.

Here, the Commom-Sense Stereotype DB is connected to the PC by a wireless LAN or an RS232C cable, and the user replaces contents to adjust the connection ratio between actions.

For example, barking or digging in a robot's actions can be tied to dog-like behavior, and salute or floating posture can be tied to soldier-like behavior.

At this time, the robot accepts the behavior patterns of other robots through the network and increases the probability of selecting a behavior for the behaviors associated with the behavior patterns to express the corresponding behavior.

In other words, by accepting the behavior pattern of the other robot through the wireless LAN by referring to this to adjust the behavior selection probability of the robot, to mimic the behavior pattern of the other robot.

Then, it is determined whether the user accepts the behavioral pattern of the other robot in a favorable manner and decides to determine it as his own behavioral pattern.

In other words, if the user inputs a feedback external input, the user's favorability is determined by determining whether the external input is praised or made by the user.

At this time, the user's input through the touch sensor is used to determine the compliment or scolding, which part of the touch sensor received feedback from the user is complimented or scolded, or complimented according to the duration of the touch screen pressed by the user. Distinguish between cognition and scolding

As another method of determining the praise or scolding, the scrutiny or compliment is judged by comparing the user's voice input with the word DB storing the words corresponding to the compliment and scolding.

If the inputted external feedback is praised or scolded by the user, the corresponding action selection probability is increased or decreased.

In other words, if the feedback input is praised by the user, the current robot behavior is recognized using the time series data of the currently stored joint indication value, and then the probability of expression of the recognized robot behavior is increased by a specific value so that other robots can recognize it. Learn the behavior pattern of the user's own behavior pattern, and if the feedback input is scolded by the user, it recognizes the current robot behavior by using time series data of the currently stored joint indication value, and then Decrease the expression probability by a certain value.

If the rules set by the user are accepted through the WLAN, the behavior selection probability value is adjusted in the above-described manner to show the behavior according to the rules of the user. Decide whether to accept it as a behavioral pattern.

Here, if there are behaviors associated with the behavior patterns of the other robots, the probability of selection for the associated behaviors is increased or decreased, in the case of behaviors associated with praise, the probability of occurrence increases, and in the case of behaviors associated with scolding. Decreases.

Referring to FIG. 3 in detail, the action selection process sets an expression probability of each operation according to the state of the Emotion Modeling unit, and determines an operation to be performed according to the set expression probability, wherein the Emotion Modeling unit inputs an external input. It is continuously updated by synthesizing the performance status of robots and robots.

As such, when the robot is operating normally, when an external stimulus is input through a touch sensor or a microphone, the external stimulus is stored in the RAM, and the feedback processing unit judges whether the input is a person's praise or adoration. If it is judged not to be scolded, the external input stored in the RAM is not used and the action selection process continues.

On the contrary, if it is judged to be praised or scolded, the time series data of the pre-stored joint indication value is used to find out what the current robot's behavior is and increase or decrease the expression probability of the behavior by a specific value according to the praise or scolding. Then, the probability of expression of the action is stored.

The microprocessor then uses Commom-Sense Stereotype DB to determine if there are behaviors associated with the behavior of the robot that received compliment or scolding feedback. After increasing or decreasing as appropriate, it returns to the autonomous mode which continues the action selection process continuously.

On the other hand, if the behavior pattern of another robot is accepted through the WLAN, temporarily adjust the expression probability of the behavior.

For example, it is assumed that the current behavior pattern of the robot is to avoid the left side unconditionally when encountering an obstacle, and the newly adopted behavior pattern through the WLAN is to look around the left and right side and to avoid the obstacle-free side. In addition, the robot randomly increases the probability of a series of behavioral expressions by looking around the left and right, which is a method entered through the WLAN, and determining the avoidance direction.

When the new method is selected and executed by the increased probability, the user praises it, and when the obstacle is found in the future, the probability of looking around the left and right and selecting the direction is decidedly increased.

In addition, when the user makes a rule to rotate 360 degrees when there is an obstacle using the PC to find the place without the obstacle and move forward in that direction, the user can arbitrarily adjust the behavior expression probability for the rule as described above. The response is then determined and the subsequent probability of the action is determined.

The specific embodiments or examples made in the detailed description of the present invention are intended to clarify the technical contents of the present invention to the extent that they should not be construed as limited to these specific embodiments and should not be construed in consultation. Various changes can be made within the scope of.

As described in detail above, the present invention has an effect of easily acquiring various behaviors by accepting behavior patterns of other robots or rules specified by a user and learning them with their own behavior patterns through a network.

Claims (9)

A first process of receiving a behavior pattern of another robot through a network; A second process of temporarily increasing a behavior selection probability for the actions associated with the behavior pattern of the first process to express the corresponding behavior; A third step of determining whether the user's response is favorable by judging the user's response to the presently expressed behavior by the user's praise or scolding; And as a result of the determination of the third process, if the user's response is favorable, the behavior learning method of the robot according to claim 4, wherein the behavior is performed as a fourth process of increasing the probability of selecting a behavior. The method of claim 1, wherein the first process comprises: Behavioral learning method of the robot further comprising the step of accepting the user's rules over the network. delete The method of claim 1, wherein praise and scolding Behavior learning method of the robot, characterized in that determined by the user input through the touch sensor. The method of claim 4, wherein the robot is distinguished by praise or congratulation of which part of the touch sensor is fed back. [5] The method of claim 4, wherein the robot distinguishes between praise and condolence according to the duration of the pressing of the touch screen by the user. The method of claim 1, wherein praise and scolding The behavior learning method of the robot, characterized by comparing the word DB storing the words corresponding to the praise and scolding with the user's voice input to determine the scolding or praise. The method of claim 1, wherein the third process comprises: Recognizing the current robot behavior using the time series data of the joint indication value stored in the present case, and increasing the expression probability of the recognized robot behavior by a specific value. Behavior Learning Method. The method of claim 1, wherein the third process comprises: Recognizing the current robot behavior using time series data of the currently stored joint indication value, and then reducing the expression probability of the recognized robot behavior by a specific value. Behavior Learning Method.