CN103991492A - Intelligent trolley based on Kinect technology - Google Patents

Abstract

The object of the present invention discloses a kind of smart car based on Kinect technology, including chassis, rotating shaft assembly, wheel assembly, Kinect sensor, computer and lower computer. After the Kinect sensor collects the operator's body movements or voice instructions from the operator, it is transmitted to the computer. After the computer analyzes the body movements of the operator or the voice uttered by the operator, it sends instructions to the lower computer. The lower computer controls the forward rotation, reverse rotation or stop rotation of each motor according to the instruction, so as to control the motion state of the smart car.

Description

A smart car based on Kinect technology

technical field

The invention relates to an intelligent sports car. the

Background technique

Smart cars are commonly used devices in modern industrial production. Existing smart cars can operate automatically in an environment according to a preset mode, without human management, and can be applied to material transportation, assembly, welding parts, scientific exploration, etc. The smart car can display time, speed, and mileage in real time. It has functions such as automatic tracking, light finding, and obstacle avoidance. It can be programmed to control driving speed, accurately locate parking, and remotely transmit images. the

Kinect is a somatosensory peripheral developed by Microsoft Corporation, which is actually a 3D somatosensory camera. The device has 3 cameras in total, the RGB color camera in the middle, the infrared projector (infrared emitter) and the infrared camera (CMOS infrared sensing device) on the left and right respectively for depth detection. Kinect is equipped with focus tracking technology, and the base motor will rotate as the focused object moves. At the same time, Kinect also has a built-in microphone array for speech recognition. the

The design of the moving part of the existing smart car is relatively simple, and it needs complex programming control to realize the movement in all directions, and it is difficult to realize the in-situ steering. This seriously affects the performance of the smart car. the

Contents of the invention

The purpose of the invention is to solve the problem that the existing smart car is difficult to control the movement. the

The technical scheme adopted for realizing the object of the present invention is such that a kind of smart car based on Kinect technology includes a chassis, a rotating shaft assembly, a wheel assembly, a Kinect sensor, a computer and a lower computer. the

The lower surface of the chassis is a horizontal plane, and the four rotating shaft assemblies are installed on the lower surface of the chassis. The shaft assembly includes a motor, an output shaft and a bearing. The motor and bearings are fixed on the lower surface of the chassis. The output shaft is parallel to the lower surface of the chassis. One end of the output shaft is connected to the rotating shaft of the motor, and the other end of the output shaft passes through the bearing to install the wheel assembly. Among the four rotating shaft assemblies installed on the lower surface of the chassis, the output shafts of two adjacent rotating shaft assemblies are perpendicular to each other. the

After the Kinect sensor collects the operator's body movements or voice instructions from the operator, it is transmitted to the computer. the

After the computer analyzes the body movements of the operator or the voice sent by the operator, it sends instructions to the lower computer. the

The lower computer controls the forward rotation, reverse rotation or stop rotation of each motor according to the instruction, so as to control the motion state of the smart car. the

Further, an emergency stop switch is also included. The emergency stop switch is used to control the on-off of the power supply of each motor. When the smart car is moving, if the Kinect sensor detects that the distance between the smart car and the obstacle is lower than a threshold, an emergency stop command is sent to the lower computer. After receiving the emergency stop command, the lower computer sends a signal to the emergency stop switch to cut off the power supply of each motor. the

The technical effect of the present invention is undoubted, and the kinematic system of the trolley has a design of multiple rotating shafts, which can realize flexible steering. In addition, the present invention integrates the Kinect system, so that the trolley understands the operator's instructions, accurately realizes movements in all directions, and exerts excellent performance. the

Description of drawings

Fig. 1 is the structural representation of the present invention;

Fig. 2 is the bottom view of the present invention;

Fig. 3 is the structural representation of wheel part;

Fig. 4 is computer control flowchart;

Figure 5 is a flow chart of the lower computer control. the

In the figure: chassis 1, rotating shaft assembly 2, motor 201, output shaft 202, bearing 203, wheel assembly 3, hub 301, mounting hole 302, spoke 303, regular polygon frame 304, roller 305, platform 4, Kinect sensor 5 . the

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and embodiments, but it should not be understood that the scope of the subject matter of the present invention is limited to the following embodiments. Without departing from the above-mentioned technical ideas of the present invention, various replacements and changes made according to common technical knowledge and conventional means in this field shall be included in the protection scope of the present invention. the

Example 1:

A smart car based on Kinect technology includes a chassis 1, a shaft assembly 2, a wheel assembly 3, a Kinect sensor 5, a computer and a lower computer. the

The lower surface of the chassis 1 is a horizontal plane, and the four rotating shaft assemblies 2 are installed on the lower surface of the chassis 1 . The shaft assembly 2 includes a motor 201 , an output shaft 202 and a bearing 203 . The motor 201 and the bearing 203 are fixed on the lower surface of the chassis 1 . The output shaft 202 is parallel to the lower surface of the chassis 1. One end of the output shaft 202 is connected to the rotating shaft of the motor 201 , and the other end of the output shaft 202 passes through the bearing 203 to install the wheel assembly 3 . Among the four rotating shaft assemblies 2 installed on the lower surface of the chassis 1 , the output shafts 202 of two adjacent rotating shaft assemblies 2 are perpendicular to each other. the

The Kinect sensor 5 collects the operator's body movements or voice commands from the operator, and transmits them to the computer. In the embodiment, the six voice commands issued by the operator "forward, right turn, left, right, left turn and backward" will be collected by the microphone of the Kinect sensor 5 and sent to the computer. the

After the computer analyzes the body movements of the operator or the voice uttered by the operator, it sends instructions to the lower computer. the

The lower computer controls each motor 201 (four motors 201 ) to rotate forward, reverse or stop rotating according to the instructions, so as to control the motion state of the smart car. the

In the embodiment, the motors 201 on the four shaft assemblies 2 are respectively denoted as a first motor, a second motor, a third motor and a fourth motor. the

The computer analyzes that when the operator issues the voice command "forward", the lower computer controls the second motor and the fourth motor to rotate forward, and the first motor and the third motor do not move. Make the smart car move forward. the

The computer analyzes that when the operator issues the voice command "backward", the lower computer controls the second motor and the fourth motor to reverse, and the first motor and the third motor do not move. Make the smart car move backward. the

The computer analyzes that when the operator issues the voice command "turn left", the lower computer controls the first motor and the third motor to rotate forward, and the second motor and the fourth motor do not move. Make the smart car move to the left. the

The computer parses out that when the operator sends out the voice command "turn to the right", the lower computer controls the first motor and the third motor to reverse, and the second motor and the fourth motor do not move. Make the smart car move to the right. the

The computer analyzes that when the operator issues the voice command "turn left", the lower computer controls the first motor to rotate forward, and the third motor, the second motor and the fourth motor do not move. Make the smart car turn left. the

The computer analyzes that when the operator issues the voice command "turn right", the lower computer controls the third motor to rotate forward, and the first motor, the second motor and the fourth motor do not move. Make the smart car turn right. the

Example 2:

The main part of this embodiment is the same as that of Embodiment 1. More specifically, in this embodiment, the lower computer unit is arranged on the upper surface of the aluminum plate, which mainly includes an MCU circuit board, a minimum system board of the MCU, a speed measuring module, a lead storage battery, Speed measurement module circuit board; two speed measurement module circuit boards are symmetrically distributed at the two corners of the square aluminum plate in the same direction, and the lead battery is located in the center of the aluminum plate; it is used to provide power, the battery and the DC motor are connected by wire, and the motor and the speed measurement module are fixed with screws , and connected with the speed measurement module circuit board in a wired manner, used to measure the motor speed in real time and transmit data to the MCU circuit board, the MCU circuit board and its minimum system board are connected by screws, and the MCU circuit board and its minimum system board are connected through the RS232 serial port It is connected with the host computer (computer and KINECT sensor), and the rotation direction of the motor is controlled by the MCU circuit board. The top control unit, namely the upper computer, is composed of KINECT sensor and computer. The KINECT sensor is fixed on the small plexiglass plate through the belt to receive voice and body movement information. The computer is located in the center of the middle plexiglass platform and can process the information data transmitted by KINECT. And issue commands to the drive system, the computer and the KINECT sensor are connected through the USB data line, and the computer is connected with the MCU circuit board through the RS232 serial port. the

The total control includes a power switch and an emergency stop switch, both of which are symmetrically distributed at the two corners of the back side of the middle plexiglass platform. The power switch is used to control the power on and off, and the emergency stop switch is used to quickly disconnect the power supply. the

The following 1 to 9 specific implementation processes are expanded and refined on the basis of Example 1:

First, the user appears in the camera within the recognition range of KINECT, makes body movements or sends out voice commands, KINECT receives the image and voice information through the infrared camera and microphone and sends the data information to the computer, and the SDK program toolkit on the computer passes The LABVIEW language is used for processing, and the motion information is sent to the MCU circuit board of the lower computer through the RS232 serial port, and then the steering (forward or reverse) and speed of the motor is controlled through the wire. the

1) The user appears in the camera within the KINECT recognition range, sends out the voice command "forward" or "backward", the KINECT microphone receives the voice information and sends the data information to the computer, and the KINECT body voice control platform on the computer is controlled by LABVIEW The language processes the received information (forward/backward distance can be set), and sends the motion information to the MCU circuit board of the lower computer through the RS232 serial port, and then controls the wires, the front and rear motors stop, and the left and right motors rotate clockwise/counterclockwise . (The motor turns to the same side as the reference frame)

2) The user appears in the camera within the recognition range of KINECT, sends out the voice command "turn right", the KINECT microphone receives the voice information and sends the data information to the computer, and the KINECT body voice control platform on the computer receives it through the LABVIEW language pair. Process the information (rotation angle can be set), send the motion information to the MCU circuit board of the lower computer through the RS232 serial port, and then control the motor through the wire, the right motor rotates counterclockwise, the left motor rotates clockwise, and the front motor clockwise Rotate, the rear motor rotates counterclockwise. (The motor turns to the same side as the reference frame)

3) The user appears in the camera within the recognition range of KINECT, sends out the voice command "turn left", the KINECT microphone receives the voice information and sends the data information to the computer, and the KINECT body voice control platform on the computer receives it through the LABVIEW language pair. The information is processed (the rotation angle can be set), and the motion information is sent to the MCU circuit board of the lower computer through the RS232 serial port, and then the motor is controlled through the wire, the right motor rotates clockwise, the left motor rotates counterclockwise, and the front motor counterclockwise Rotate, the rear motor rotates clockwise. (The motor turns to the same side as the reference frame). the

4) The user appears in the camera within the recognition range of KINECT, sends out the voice command "face me", the KINECT microphone receives the voice information and sends the data information to the computer, and the KINECT body voice control platform on the computer receives it through the LABVIEW language pair. The information is processed, and the motion information is sent to the MCU circuit board of the lower computer through the RS232 serial port, and then the motor is controlled through the wire. When the camera recognizes some body features, the platform will "turn left" or "turn right" Go to the direction of the sound. When there is no limb bone signal in the camera, the platform rotates 180 degrees in the way of "turning left" or "turning right". (The motor turns to the same side as the reference frame)

5) The user appears in the camera within the KINECT recognition range, sends out the voice command "pause" or "end", the KINECT microphone receives the voice information and sends the data information to the computer, and the KINECT body voice control platform on the computer uses the LABVIEW language to The received information is processed, and the motion information is sent to the MCU circuit board of the lower computer through the RS232 serial port, and then the motor is controlled through the wire, and all the motors stop. the

6) The user appears in the camera within the recognition range of KINECT, and sends out the limb bone command "right hand to the left", the KINECT microphone receives the voice information and sends the data information to the computer, and the KINECT body voice control platform on the computer communicates with the user through the LABVIEW language. The received information is processed (the rotation angle can be set), and the motion information is sent to the MCU circuit board of the lower computer through the RS232 serial port, and then the motor is controlled through the wire. The right motor rotates clockwise, the left motor rotates counterclockwise, and the front motor Rotate counterclockwise, the rear motor rotates clockwise. (The motor turns to the same side as the reference frame). the

7) The user appears in the camera within the recognition range of KINECT, and sends out the limb bone command "point to the right with the left hand", and the KINECT microphone receives the voice information and sends the data information to the computer. The received information is processed (the rotation angle can be set), and the motion information is sent to the MCU circuit board of the lower computer through the RS232 serial port, and then the motor is controlled through the wire. The right motor rotates counterclockwise, the left motor rotates clockwise, and the front motor Rotate clockwise, the rear motor rotates counterclockwise. (The motor turns to the same side as the reference frame). the

8) The user appears in the camera within the recognition range of KINECT, and sends out the limb bone command "raise the right hand" or "raise the left hand", the KINECT microphone receives the voice information and sends the data information to the computer, and the KINECT body voice control on the computer The platform processes the received information through the LABVIEW language (forward/backward distance can be set), and sends the motion information to the MCU circuit board of the lower computer through the RS232 serial port, and then controls the motor through the wire, the front and rear motors stop, and the left and right motors clockwise /Anticlockwise rotation. (The motor turns to the same side as the reference frame). the

9) The user appears in the camera within the recognition range of KINECT, and sends out the limb skeleton instruction "hands crossed in front of the abdomen", and the KINECT microphone receives the voice information and sends the data information to the computer. Process the received information (the rotation angle can be set), send the motion information to the MCU circuit board of the lower computer through the RS232 serial port, and then control the motors through the wires, and all the motors stop. the

Example 3:

This embodiment further describes the control system of Embodiment 1 or 2. the

See Figure 4 and Figure 5:

1. Hardware initialization stage. This stage is the first stage when the hardware of the lower computer is powered on. In this stage, firstly initialize the hardware connected to the system platform, such as kinect sensor, lower computer hardware, etc. the

2. Judge whether the initialization is successful; if the judgment is "No", enter the error processing stage; if the judgment is "Yes", set the system parameters on the host computer (kinect sensor and computer), including speed, distance and system platform the steering angle. the

3. Judge whether the host computer system receives audio, video, and depth data streams from the sensor. If the judgment is "No", then re-initialize the hardware; if the judgment is "Yes", the host computer directly collects and processes sensor information. the

4. Parallel processing of relevant data, simultaneous activation of speech recognition, bone tracking and gesture recognition, and color image display. the

5. Perform speech and gesture recognition command judgment; in this stage, judge whether the speech recognition result conforms to the established speech command, if the judgment result is "No", continue to perform language recognition, if the judgment result is "Yes", then match and recognize As a result, a specific instruction is generated; at the same time, in this stage, it is judged whether the sensor has tracked the human skeleton, if the judgment result is "No", continue to perform gesture recognition; if the judgment result is "Yes", continue to judge whether the recognized gesture conforms to the established Gesture algorithm: judge whether it conforms to the established gesture algorithm, if the judgment result is "no", then continue to judge whether the gesture conforms to the established algorithm, if the judgment result is "yes", then generate a specific instruction. the

6. After generating a specific command, the upper computer sends the command to the serial port and sends the serial port command. the

7. The lower computer and the upper computer are connected for data through the serial port data line, and judge whether the lower computer receives the command or not. If the judgment result is "No", then resend the serial port command; if the judgment result is "Yes", the lower computer executes the command. the

8. During the process of executing the command, the lower computer needs to judge the new command. If the judgment result is "No", the lower computer continues to execute, and then waits for a new command; if the judgment result is "Yes", then transfer to the lower computer to execute the command, that is, overwrite the previous command and execute the new command. the

9. After the command is executed, the system re-sets the system parameters. At this stage, the platform speed, driving distance and steering angle can be changed, or not, and the system performs a new round of voice gesture recognition. the

Example 4:

This embodiment further discloses the structure of a smart car on the basis of embodiment 1. Referring to FIGS. 1-3 , the car includes a chassis 1 , a rotating shaft assembly 2 and a wheel assembly 3 . The lower surface of the chassis 1 is a horizontal plane, and the four rotating shaft assemblies 2 are installed on the lower surface of the chassis 1 . the

Referring to FIG. 2 , the rotating shaft assembly 2 includes a motor 201 , an output shaft 202 and a bearing 203 . The motor 201 and the bearing 203 are fixed on the lower surface of the chassis 1 . The output shaft 202 is parallel to the lower surface of the chassis 1 . the

The upper surface of described chassis 1 is equipped with support bar, and described Kinect sensor 5 is installed on the upper end of described support bar. the

One end of the output shaft 202 is connected to the rotating shaft of the motor 201 , and the other end of the output shaft 202 passes through the bearing 203 to install the wheel assembly 3 . In an embodiment, the outer ring of the bearing 203 is fixed on the lower surface of the chassis 1 . The output shaft 202 passes through and is fixed on the inner ring of the bearing 203 . One end of the output shaft 202 passing through the inner ring of the bearing 203 is suspended around the chassis 1 to facilitate the installation of the wheel assembly 3 . the

In the embodiment, the chassis 1 is square. Among the four rotating shaft assemblies 2 installed on the lower surface of the chassis 1 , the output shafts 202 of two adjacent rotating shaft assemblies 2 are perpendicular to each other. That is, a "ten" character is drawn at the center of the square chassis 1, and the four rotating shaft assemblies 2 are respectively installed on the four branches of the "ten" character. That is, the four output shafts 202 are radially installed on the lower surface of the square chassis 1 . the

The wheel assembly 3 includes a hub 301 , spokes 303 , a regular polygonal frame 304 and a roller 305 . There is a mounting hole 302 at the center of the hub 301 . The mounting hole 302 is sleeved on the output shaft 202 so that the wheel assembly 3 is mounted on the output shaft 202 . Several spokes 303 are connected to the outer circumference of the hub 301 . The hub 301 is located in the regular polygonal frame 304 , and the regular polygonal frame 304 is concentric with the hub 301 . One end of the spokes 303 is connected to the outer circumference of the hub 301 , and the other end is connected to the vertices of the regular polygonal frame 304 . Several said rollers 305 are installed on each side of said regular polygonal frame 304 . The roller 305 can roll freely. the

Embodiment 5:

The main structure of this embodiment is the same as that of Embodiment 4. In this embodiment, a platform 4 is also included, and the platform 4 is installed on the upper surface of the chassis 1 . The regular polygonal frame 304 is a regular dodecagonal frame, the number of the spokes 303 is twelve, and the number of the rollers 305 is twelve. That is, the regular polygonal frame 304 in the wheel assembly 3 is a regular dodecagonal metal frame bent from a metal strip with a circular cross section. The regular dodecagonal metal frame is sleeved on the outside of the hub 301 . The regular dodecagonal metal frame is fixed with the hub 301 through twelve spokes 303, and the center of the hub 301 coincides with the center of the regular dodecagon. In this embodiment, the roller 305 is a hollow cylinder with both ends open, and its material can be rubber. Twelve of the rollers 305 are sleeved on each side of the regular dodecagonal metal frame, and can roll freely. Preferably, bearings are installed on each side of the regular dodecagonal metal frame, and the rollers 305 are sleeved on these bearings. the

Embodiment 6:

The main part of this embodiment is the same as Embodiment 1, but also includes an emergency stop switch. When the smart car is moving, if the Kinect sensor 5 detects that the distance between the smart car and the obstacle is lower than the threshold, it will send an emergency stop command to the lower computer. After the lower computer receives the emergency stop command, it sends a signal to the emergency stop switch to cut off the power of each motor 201 . the

Claims (2)

1. the intelligent carriage based on Kinect technology, is characterized in that: comprise chassis (1), rotating shaft assembly (2), wheel set (3), Kinect sensor (5), computing machine and lower computer;

The lower surface on described chassis (1) is horizontal surface, and four described rotating shaft assemblies (2) are arranged on the lower surface of described chassis (1); Described rotating shaft assembly (2) comprises motor (201), output shaft (202) and bearing (203); Described motor (201) and bearing (203) are fixed on the lower surface of described chassis (1); Described output shaft (202) is parallel to the lower surface of described chassis (1); One end of described output shaft (202) connects the rotating shaft of described motor (201), and the other end of described output shaft (202), through after described bearing (203), is installed described wheel set (3); Be arranged in four described rotating shaft assemblies (2) of lower surface on described chassis (1), the output shaft (202) of adjacent two rotating shaft assemblies (2) is mutually vertical;

Described Kinect sensor (5) collects after the limb action of operating personal or phonetic order that operating personal sends, passes to described computing machine;

The voice that described computing machine sends the limb action of operating personal or operating personal send instruction to described lower computer after resolving;

Described lower computer according to instruction control that each motor (201) is rotated in the forward, contrarotation or stop the rotation, to control the state of kinematic motion of intelligent carriage.

2. a kind of intelligent carriage based on Kinect technology according to claim 1, is characterized in that: also comprise an emergency stop switch; Described emergency stop switch is used for controlling the break-make of the power supply of each motor (201);

When intelligent carriage moves, if when described Kinect sensor (5) detects distance between intelligent carriage and obstacle lower than threshold value, send stop command to described lower computer; Described lower computer receives after stop command, to described emergency stop switch, sends signal to cut off the power supply of each motor (201).