WO2017219319A1 - Automatic vehicle driving method and automatic vehicle driving system - Google Patents

Abstract

Provided are an automatic vehicle driving method and an automatic vehicle driving system. The automatic vehicle driving system comprises: a sensor (110) for sensing a person in a vehicle; a profile information acquisition unit (120) for acquiring profile information about the person in the vehicle; a classification unit (130) for classifying the person in the vehicle; and a self-adaptive control unit (140) for determining a corresponding automatic driving and travelling mode according to the classification and performing self-adaptive control on the vehicle. Provided are a method and system for performing self-adaptive control by means of in-vehicle sensing, these being able to provide a safer and more efficient automatic driving method and system.

Description

Vehicle automatic driving method and vehicle automatic driving system Technical field

The present invention generally relates to a vehicle automatic driving technique, and more particularly to an in-vehicle sensing based vehicle automatic driving method and a vehicle automatic driving system.

Background technique

In the automatic driving system, the plan mainly simulates the various possible automatic driving schemes in the next time period based on the results of the environment perception, including speed adjustment and steering adjustment, etc.; then passes the best solution to the decision-making layer to realize the vehicle. Physical control. Environmental perception is mainly the external physical road perception and cognition, as well as the perception and cognition of moving targets around the vehicle.

In the existing automatic driving system, the sensing result of the external environment and the speed sensor from the vehicle itself are mainly used for motion planning and control, and the perception of the interior environment is less, and the existing perception of the interior of the vehicle is Generally limited to the perception of the driver's state, such as fatigue driving.

Summary of the invention

According to an aspect of the present invention, a method for automatically driving a vehicle includes: sensing a person inside the vehicle, obtaining profile information of a person inside the vehicle; classifying the person inside the vehicle; and determining a corresponding one according to the classification Auto-driving driving mode and adaptive control of the vehicle.

Further, the vehicle automatic driving method may further include: continuously receiving feedback of the in-vehicle personnel during driving; and adaptively adjusting parameters of the automatic driving according to feedback of the in-vehicle personnel.

Further, in the vehicle automatic driving method, the outline information of the in-vehicle personnel may include a traveling mode preferred by each person in the vehicle.

Further, in the vehicle automatic driving method, each person's preferred driving mode may be represented by a corresponding driving parameter.

Further, in the vehicle automatic driving method, classifying the in-vehicle personnel may include: determining whether each in-vehicle person is a patient; determining whether each in-vehicle person is an elderly person, a child, or a disabled person; And to determine if everyone has set the preferred driving mode.

Further, in the vehicle automatic driving method, the automatic driving mode may include: a comfort mode, a normal mode, and a sport mode, and the determining, according to the classification, determining the corresponding autonomous driving mode may include determining whether the in-vehicle personnel have The patient, if there is a patient, determines that the automatic driving mode is the comfort mode; if the person in the vehicle has no patient, it is determined whether each person in the vehicle has a preferred driving mode, and if each person in the vehicle has a preferred driving mode, the use Among the preferred driving modes, the most comfortable driving mode; and if not everyone in the car has a preferred driving mode, determine whether there is an elderly person, a child, or a disabled person among those who do not have a preferred driving mode; If it exists, use the comfort mode.

Further, in the vehicle automatic driving method, sensing the in-vehicle personnel may include using at least one of the following: a camera, a pressure sensor and a microphone, a fingerprint recognizer, and an infrared sensor.

Further, in the vehicle automatic driving method, obtaining the profile information of the in-vehicle personnel may include: identifying the identity of the in-vehicle personnel, and identifying the identity of the in-vehicle personnel includes: based on the identity information database, by one of the following identification technologies or The combination is used to identify the identity of the person in the vehicle: face recognition, voiceprint recognition, fingerprint recognition, iris recognition, infrared recognition; and based on the identity of the identified person in the vehicle, the contour information of the person in the vehicle is retrieved from the profile information database. The contour information is stored in association with the identity of the person in the vehicle in the profile information database.

Further, in the automatic vehicle driving method, in the case where the outline information of the in-vehicle personnel is not stored in the contour database, the contour information input of the in-vehicle personnel can be received and stored in the database.

Further, in the automatic vehicle driving method, the outline information may include one or more of age, gender, mood, health status information, and preferred driving mode.

Further, in the vehicle automatic driving method, different modes may correspond to associated planning and control parameters, which may be selected from one or more of the following items: normal acceleration, abnormal acceleration, normal deceleration, abnormality Deceleration, maximum value of front wheel yaw and vehicle speed ratio, safety distance, minimum frequency of lane change, and safe distance of lane change.

Further, in the automatic driving method of the vehicle, adaptively adjusting the parameter of the automatic driving according to the feedback of the person in the vehicle may include: responding to the feedback command of the current driving state by the person in the vehicle, using the preset corresponding to the feedback Adjustments are made by parameters and adjustments that indicate a determined adjustment to the driving state.

Further, in the automatic driving method of the vehicle, when the user no longer gives a feedback command, The current parameter is recorded as the driving mode parameter of the user's preference.

Further, in the vehicle automatic driving method, adaptively adjusting the parameters of the automatic driving according to the feedback of the in-vehicle personnel may include: adjusting the learning method in response to the feedback command of the in-vehicle personnel to the current driving state.

Further, in the vehicle automatic driving method, one or more of the identification technologies may be run in a local in-vehicle computer system, in a cloud server, or in a manner in which the local in-vehicle computer system and the cloud server cooperate with each other.

Further, in the vehicle automatic driving method, one or both of the identity information database and the profile information database of the in-vehicle personnel are stored in the cloud storage.

Further, the vehicle automatic driving method may further include: sensing whether a specific animal and/or a specific item exists in the vehicle; determining a corresponding automatic driving driving mode based on sensing that a specific animal and/or a specific item exists in the vehicle.

According to another aspect of the present invention, a vehicle automatic driving system is provided, which may include: a sensor for sensing a person inside the vehicle; a contour information obtaining unit for obtaining contour information of a person inside the vehicle; and a classification unit for the interior of the vehicle The personnel are classified; and the adaptive control unit determines the corresponding automatic driving mode according to the classification and adaptively controls the vehicle.

Further, the vehicle automatic driving system may further include: a feedback receiving unit that continuously receives feedback of the in-vehicle personnel during running; and the adaptive control unit adaptively adjusts parameters of the automatic driving according to feedback of the in-vehicle personnel.

In one example, the profile information of the in-vehicle personnel may include a preferred travel mode for each person in the vehicle.

In one example, in the vehicle automatic driving system, categorizing the in-vehicle personnel by the categorizing unit may include: determining whether each in-vehicle person is a patient; determining whether each in-vehicle person is an elderly person, a child, or a disabled person; And to determine if everyone has set the preferred driving mode.

In one example, in the vehicle automatic driving system, the automatic driving mode may include: a comfort mode, a normal mode, and a sport mode, and the determining, according to the classification, determining the corresponding autonomous driving mode includes: determining whether the in-vehicle personnel There is a patient, if there is a patient, it is determined that the automatic driving mode is a comfortable mode; if the person inside the vehicle has no patient, it is determined whether each person in the vehicle has a preferred driving mode, and if everyone in the car has a preferred driving mode, then Using the most comfortable driving mode among the preferred driving modes; and if not everyone in the car has a preferred driving mode, determining whether there is an elderly person, a child, or a disabled person among those who do not have a preferred driving mode; And if it exists, use comfort mode.

In one example, adaptively adjusting the parameters of the automated driving based on feedback from the in-vehicle personnel may include adjusting using an enhanced learning method in response to a feedback command from the in-vehicle personnel to the current driving state.

In one example, in a vehicle autopilot system, the sensor may also sense the presence or absence of a particular animal and/or particular item within the vehicle; and the adaptive control unit determines based on sensing that a particular animal and/or particular item is present within the vehicle. The corresponding autonomous driving mode.

According to another aspect of the present invention, a vehicle automatic driving method is provided, which may include: sensing a person in a vehicle, obtaining contour information of a person in the vehicle; and adapting the vehicle based on the obtained contour information of the person inside the vehicle control.

The vehicle automatic driving system and method of the embodiments of the present invention pay attention to the perception of the inside of the vehicle, determine the corresponding automatic driving mode, and continuously perform the in-vehicle situation sensing, and adaptively control the vehicle to provide safer and more efficient. Automated driving methods and systems.

DRAWINGS

These and/or other aspects and advantages of the present invention will become more apparent from the following detailed description of the embodiments of the invention.

FIG. 1 shows a block diagram showing the structure of a vehicle automatic driving system 100 according to an embodiment of the present invention.

FIG. 2 shows a functional block diagram of a profile information acquisition unit module 120 in accordance with one embodiment of the present invention.

FIG. 3 is a block diagram showing the structure of a vehicle automatic driving system 200 according to another embodiment of the present invention.

FIG. 4 shows a schematic structural view of a vehicle automatic driving system 300 according to another embodiment of the present invention.

FIG. 5 illustrates a general flow diagram of a method of automated vehicle driving 400 in accordance with one embodiment of the present invention.

FIG. 6 depicts a more detailed process of an exemplary vehicle autopilot method 500 in accordance with one embodiment of the present invention.

FIG. 7 shows a flow diagram of an exemplary adaptive control method 600 in accordance with an embodiment of the present invention.

FIG. 8 illustrates a flow diagram of an exemplary adaptive control method 700 using an enhanced learning method in accordance with another embodiment of the present invention.

detailed description

The present invention will be further described in detail below in conjunction with the drawings and specific embodiments.

The profile information, which is the information of the person in the car, can be registered by the person in the car, preferably including the driving mode preferred by the person in the car, and may also include name, age, gender, health condition, mood, and the like.

Adaptive control and/or adjustment refers to automatically adjusting the parameters of the automatic driving according to the feedback of the personnel in the vehicle until the requirements of the personnel in the vehicle are met.

The invention pays attention to the perception of the personnel in the vehicle, classifies and analyzes the personnel inside the vehicle, determines the corresponding automatic driving mode, and continuously carries out the situation sensing in the vehicle, and adaptively controls the vehicle, thereby providing safer and more efficient. Automated driving methods and systems.

FIG. 1 shows a block diagram showing the structure of a vehicle automatic driving system 100 according to an embodiment of the present invention.

The vehicle automatic driving system 100 includes a sensor 110, a contour information obtaining unit 120, a categorizing unit 130, and an adaptive control unit 140. The sensor 110 senses the in-vehicle personnel, the contour information obtaining unit 120 is configured to obtain the contour information of the in-vehicle personnel, the classification unit 130 classifies the in-vehicle personnel, and the adaptive control unit 140 determines the corresponding according to the classification. Auto-driving driving mode and adaptive control of the vehicle.

The contour information obtaining unit 120, the categorizing unit 130, and the adaptive control unit 140 herein may be implemented by software of an onboard computer, hardware, or a combination of both, or may be implemented by an onboard computer in conjunction with cloud storage and/or application services. ,

Install high-performance embedded computer systems in autonomous vehicles to process sensor data and perform tasks such as sensing, analysis, planning, decision making, and control. The in-vehicle computer system includes components such as CPU, memory, and permanent storage media, as well as optional FPGA, ASI C, DSP, GPU and other computational acceleration components. The in-vehicle computer system can be a stand-alone system or a distributed system. In the case of a distributed system, some kind of network connection (such as Ethernet, etc.) is used between multiple computing nodes. Multiple nodes of a distributed system can be isomorphic or heterogeneous, such as using different architectures and operating systems.

The in-vehicle computer system can operate independently or be connected to the cloud 20. Cloud-based systems can store a large amount of driver and passenger information from individual vehicle uploads and can provide face recognition and Identification services such as voiceprint recognition.

The sensor here is a generalized concept and does not specifically refer to a certain sensor device, but may be a collection of sensor devices.

Sense of in-vehicle personnel includes sensing the image, sound, quality, etc. of the person inside the vehicle.

For example, a person in the sensing vehicle can use at least one of the following means: a camera, a pressure sensor and a microphone, a fingerprint reader, an infrared sensor.

For the in-vehicle sensor device, consider the following settings.

(1) Camera inside the car. One or more cameras can be installed, and each camera can be a monocular, binocular, or depth camera. For the mounting method, for example, one of the following two can be considered.

Installation method 1: Install a camera facing the face in front of each seat to clearly capture the front of the face.

Installation method 2: You can choose other camera installation methods, such as using a camera to take multiple seats, provided that you can capture the front of all drivers and passengers' faces.

(2) Pressure (mass) sensor on each seat. The number of passengers in the car can be provided in real time, and the quality information can be used to identify whether there is a rough age information such as a child in the passenger.

(3) Microphone. The in-car microphone can capture the passenger's voice commands and feedback, and understand the commands and feedback through natural language processing. And can identify different people according to the voiceprint.

Microphone installation method: one can be installed in the middle position in the car, or one in the middle position of the front row and the middle position in the rear row.

The above sensor devices are merely examples, sensor devices may be added or subtracted as desired, and any existing or future occurring sensor devices may be used in the present invention.

Regarding the outline information obtaining unit 120, the outline information herein may include, for example but not limited to, identity, age, gender, face characteristics, voiceprint characteristics, health status, emotional state, age, and the like.

In a preferred example, the profile information of the in-vehicle personnel includes its preferred travel mode. In one example, according to the degree of comfort, the exercise mode is divided into a comfort mode, a normal mode, and a sport mode, and the comfort level is sequentially reduced. Different exercise modes correspond to different planning and control parameters, such as: normal acceleration, extraordinary acceleration, Normal deceleration, super deceleration, maximum lateral control, safety distance, minimum frequency of lane change, safe distance of lane change, etc. It should be noted that even if it is the same comfort mode, the corresponding parameters can be different for different people. For example, regarding the comfort mode, it is generally desirable for the elderly to drive the vehicle as smoothly as possible, so it is desirable to have a lower acceleration and change the frequency of the lane. The rate is lower; and for young people, there is no such high requirement for stability. Therefore, preferably, the preferred automatic driving mode can be customized for each person.

FIG. 2 shows a functional block diagram of a profile information acquisition unit module 120 in accordance with one embodiment of the present invention.

The contour information acquisition unit module 120 may include an identity recognition module 121, a contour information collection module 124, other attributes and status detection modules 123 of the in-vehicle personnel, and a contour information integration module 122.

In one example, the profile information can be obtained as follows: First, the identity of the person in the vehicle is identified by the identity recognition module 121, for example, based on the identity information database, the facial features, voiceprint features, and iris features sensed based on the in-vehicle sensor And infrared characteristics, etc., identify the identity of the person in the vehicle by one or a combination of the following identification technologies: face recognition, voiceprint recognition, fingerprint recognition, iris recognition, infrared recognition; and recognition based vehicle The identity of the insider, the profile information of the in-vehicle personnel is retrieved from the profile information database, and the profile information is stored in association with the identity of the in-vehicle personnel in the profile information database. The profile information database and the identity information database herein may be separate or shared together, and the identity information here is, for example, the ID of the person in the vehicle (mobile phone number, ID card information) together with the facial features of the person inside the vehicle. , voiceprint features, handprint features, iris features, infrared features, vascular features, etc. can uniquely identify a person's identity, based on sensory information (images, sounds, fingerprints, etc.) obtained through sensors and a database of identity information, Identification technology is used to identify the identity of people in the car. In the profile information database, profile information such as name, age, gender, health status, preferred exercise mode, and the like are stored in association with the in-vehicle person identification.

In the case that a certain in-vehicle personnel information is missing in the identity information database and/or the profile information database, the profile information collection module 124 collects profile information of the new person, and can input some basic materials such as name and age by, for example, voice interaction. , gender, preferred driving mode, etc.; the camera in the car can take multiple photos, as well as record voice voiceprints, fingerprints, etc., and store the information into identity information records and/or contour information records in the identity information database and/or contours. In the information database.

In addition to obtaining the profile information already in the identity information database and/or the profile information database, the current in-vehicle personnel information can also be obtained in real time by other properties and status detection modules 123 of the in-vehicle personnel, for example by images taken by the camera. Perform image processing, measure the blood pressure of the person in the car, heartbeat, etc., or obtain the current mood of the person in the car by interacting with the voice of the person inside the car. Status and current health status.

The contour information synthesis module 122 integrates the information from the contour information database, the information collected by the contour information collection module 134, and other attributes of the vehicle personnel and the information of the state detection module 123 to obtain contour information of each vehicle personnel for subsequent return. Class units and adaptive control units are used.

The module and profile information database and the identity information database of the identity recognition module 121 (including the face recognition module, the voiceprint recognition module, the iris recognition module, the fingerprint recognition module, etc.) shown in FIG. 2 can be run and stored in the onboard computer and/or Or the cloud.

Returning to Figure 1, the categorizing unit 130 classifies the personnel in the vehicle. For example, based on the profile information of the in-vehicle personnel obtained by the profile information obtaining unit 120, determining the category to which each in-vehicle person belongs, for example, classifying the in-vehicle personnel includes: determining whether each in-vehicle person is a patient; determining each car Whether the person is an elderly person, a child or a disabled person; and whether each person has set a preferred driving mode.

The adaptive control unit 140 determines a corresponding autonomous driving mode according to the categorization and adaptively controls the vehicle.

The adaptive control unit 140 determines, according to the classification, that the corresponding automatic driving mode comprises:

Determining whether there is a patient in the vehicle, and if there is a patient, determining that the automatic driving mode is a comfortable mode;

If there is no patient in the vehicle, it is determined whether each person in the car has a preferred driving mode, and if each person in the car has a preferred driving mode, the driving mode with the highest comfort among the preferred driving modes is used;

If not everyone in the car has a preferred driving mode, it is determined whether there is an elderly person, a child, or a disabled person among those who do not have the preferred driving mode; and if present, the comfort mode is used.

The above classification of the in-vehicle personnel is only a preferred example, and different classifications can be performed as needed.

The process of adaptive control will be described in detail later in conjunction with the method flow chart.

FIG. 3 is a block diagram showing the structure of a vehicle automatic driving system 200 according to another embodiment of the present invention.

Compared to FIG. 1, the vehicle automatic driving system 200 further includes a feedback receiving unit 250. The structure and working process of the sensor 210, the contour information obtaining unit 220, and the categorizing unit 230 in FIG. 3 may be similar to the corresponding portions shown in FIG. 1, and details are not described herein again.

The feedback receiving unit 250 continuously receives the feedback of the in-vehicle personnel during the running, and the adaptive control unit 240 adaptively adjusts the parameters of the automatic driving according to the feedback of the in-vehicle personnel.

The way to receive feedback from the people in the car can be voice mode, through the voice recognition module to identify the feedback of the people inside the car, such as, for example, "open fast, open slow, open, don't be too meat", etc.; Other means, such as providing a predetermined physical control button for pressing on the vehicle, etc.; or feedback by a vehicle driving control application on the in-vehicle's mobile phone; or gestures can be used to give feedback, Deaf people are particularly suitable;

The adaptive adjustment of the parameters of the automatic driving according to the feedback of the personnel in the vehicle may include: one is to preset the parameters and the adjustment amount that each command needs to be adjusted according to the prior knowledge; the other is to adopt the method of enhanced learning in a large number of Learn to determine these parameter values while driving.

FIG. 4 shows a schematic structural view of a vehicle automatic driving system 300 according to another embodiment of the present invention.

Compared with FIG. 3, the vehicle automatic driving system 300 of FIG. 4 further includes a driving mode command receiving module 360. The other modules 310, 320, 330, and 350 may be the same as the corresponding modules shown in FIG. 3, and details are not described herein again.

The driving mode command receiving module 360 is configured to receive an explicitly specified vehicle driving mode of a person in the vehicle, and the system provides a plurality of predefined driving modes: such as a normal mode, a comfort mode, a sports mode, and the like. The manner of receiving the driving mode command of the person in the vehicle may be a voice mode, and the driving mode command is recognized by the voice recognition module, for example, “comfort mode”, “sport mode”, etc.; For example, a predetermined physical control button is provided on the car for pressing, for example, two buttons, one for "better" and one for "worse", of course, more buttons can be designed, and other forms can be designed. Feedback hardware, etc.; or gestures can be used to give feedback, which is especially useful for deaf people; or feedback through vehicle driving control applications on the phone of a person in the car.

In one example, the sensor also senses whether a particular animal and/or a particular item is present in the vehicle, and the adaptive control unit determines the corresponding autonomous driving mode based on sensing that a particular animal and/or particular item is present within the vehicle. . For example, to detect the presence of fragile items in the car, such as porcelain, glass products, etc., whether there are some small animals in the car, such as kittens or puppies, thereby adjusting the automatic driving mode, such as when the presence of porcelain is sensed. Adjust the automatic driving exercise mode to the comfort mode to drive more smoothly.

A vehicle automatic driving method according to an embodiment of the present invention will be described below with reference to the accompanying drawings. The vehicle automatic driving method is implemented in combination with the aforementioned vehicle automatic driving system.

FIG. 5 illustrates an overall flow of a vehicle automatic driving method 400 in accordance with one embodiment of the present invention. Figure.

As shown in FIG. 5, in step S410, the in-vehicle personnel are sensed by the sensor to obtain contour information of the in-vehicle personnel.

In step S420, the persons in the vehicle are classified.

In step S430, according to the classification, the corresponding automatic driving mode is determined, and the vehicle is adaptively controlled.

In one example, the vehicle autopilot method 400 further includes: continuously receiving feedback from the in-vehicle personnel during driving; and adaptively adjusting parameters of the autonomous driving based on feedback from the in-vehicle personnel.

Adjusting the parameters of the automatic driving adaptively according to the feedback of the in-vehicle personnel may include: adjusting, in response to a feedback command of the in-vehicle personnel to the current driving state, using a preset parameter and an adjustment amount corresponding to the feedback, the adjusting The feedback command indicates a certain adjustment to the driving state.

In one example, when the user no longer gives a feedback command, the current parameter can be recorded as the preferred driving mode parameter for the user, and the user's profile information is updated to the profile information database.

In one example, adaptively adjusting the parameters of the automated driving based on feedback from the in-vehicle personnel includes adjusting in an enhanced learning method in response to feedback commands from the in-vehicle personnel to the current driving state.

In one example, the profile information profile information of the in-vehicle personnel may include, for example, without limitation, identity, age, gender, facial features, voiceprint features, health status, emotional state, age, and the like.

The preferred driving mode for each person can be represented by the corresponding driving parameters, that is, the preferred exercising mode can be customized for each person.

In one example, categorizing the in-vehicle personnel includes determining whether each in-vehicle person is a patient, determining whether each in-vehicle person is an elderly person, a child, or a disabled person, and determining whether each person has set a preference Driving mode.

In one example, the automatic driving mode may include: a comfort mode, a normal mode, and a sport mode, and determining, according to the categorization, determining a corresponding autonomous driving mode includes: determining whether a person in the vehicle has a patient, and if there is a patient, Then, the automatic driving mode is determined to be a comfortable mode; if the person in the vehicle has no patient, it is determined whether each person in the vehicle has a preferred driving mode, and if each person in the vehicle has a preferred driving mode, the preferred driving mode is used. The most comfortable driving mode; and if not everyone in the car has a preferred driving mode, determine whether there is an elderly person, a child or a disabled person among those who do not have a preferred driving mode; and if present, use the comfort mode .

Different driving modes correspond to different planning and control parameters, such as: normal acceleration, abnormal acceleration, normal deceleration, super deceleration, maximum value of front wheel declination and vehicle speed ratio, safety distance, minimum frequency of lane change. , lane change safety distance and so on.

In one example, sensing the in-vehicle personnel includes using at least one of the following: a camera, a pressure sensor and a microphone, a fingerprint reader, an infrared sensor.

In one example, obtaining the profile information of the in-vehicle personnel includes: identifying the identity of the in-vehicle personnel, identifying the identity of the in-vehicle personnel, including identifying the in-vehicle by one or a combination of the following identification technologies based on the identity information database The identity of the person: face recognition, voiceprint recognition, fingerprint recognition, iris recognition, infrared recognition; and based on the identity of the identified person in the vehicle, retrieve the contour information of the person in the vehicle from the profile information database, in the profile information database Profile information is stored in association with the identity of the person in the vehicle.

In one example, in the case where the outline information of the in-vehicle person is not stored in the outline database, the outline information input of the in-vehicle person is received and stored in the database.

In one example, one or more of the identification technologies operate in a cloud server or in a manner that the local onboard computer system and the cloud server cooperate with each other. Similarly, other application services can also run on a local onboard computer system, or run in the cloud, or in a way that the local onboard computer system and the cloud server work together.

Similarly, one or both of the identity information database and the profile information database of the in-vehicle personnel may be stored in the local computer system and/or stored in the cloud storage, or stored in a cooperative manner in both.

In one example, the vehicle autopilot method may also sense the presence or absence of a particular animal and/or particular item within the vehicle; based on sensing the presence of a particular animal and/or particular item within the vehicle, determining a corresponding autonomous driving mode of travel.

As a specific example, a more detailed process of an exemplary vehicle autopilot method 500 in accordance with one embodiment of the present invention is described below in conjunction with FIG.

As shown in Figure 6, after each start of the trip, the onboard autopilot computer system reads the sensor input and processes it by each sensor module (such as face recognition, voiceprint recognition, etc.), and then by the recognition module (such as Identification), and finally controlled according to a certain adaptive algorithm until the end of the trip. Adjustments can be made in the adaptive algorithm taking into account driver and passenger feedback.

As shown in FIG. 6, each of the identification modules performs processing based on input of each sensor device such as a camera, a microphone, a weight sensor, etc., specifically, by the face recognition module in step S510. Face recognition, voiceprint recognition is performed by the voiceprint recognition module in step S520, and other attributes and states of the person in the vehicle are identified by other attributes and state recognition modules of the in-vehicle personnel in step S530.

The details are detailed below.

(1) Face recognition

The specific address, the face image in the car is obtained in real time by the on-board camera, the number of passengers and the number of passengers in the car is recognized by image processing according to the face feature, and the driver, the identity, age, gender, mood and the like of the passenger are recognized.

The face recognition module can run on a vehicle-mounted local computer system, or in a server running in the cloud, or run in two systems in a collaborative manner for better results.

Information such as identity, age, gender, and facial features of the in-vehicle personnel is stored in a permanent storage of the in-vehicle local computer system, and optionally in cloud storage, such as in an identity information database and/or a profile information database, as before The identity information database and the profile information database may be combined into one, or may be separately stored and organized.

(2) voiceprint recognition

The specific address, voiceprint recognition through the microphone, requires the car voiceprint recognition module to initiate and the passenger's voice interaction to achieve.

The voiceprint recognition module can run on a vehicle-mounted local computer system, or in a server running in the cloud, or run in two systems in a collaborative manner for better results.

Information such as the identity, age, gender, voiceprint characteristics of the vehicle personnel can be stored in the permanent storage of the onboard local computer system, and optionally in cloud storage, such as in an identity information database and/or profile information database, such as As mentioned above, both the identity information database and the profile information database can be combined into one, or can be stored and organized separately.

(3) Other attributes and status detection of the personnel in the vehicle

Other attributes and status detection modules of the vehicle personnel can detect more information, such as the number of personnel and the health status of each person (health, fatigue, etc.) by processing information sensed by sensor devices such as cameras, microphones, and pressure sensors. Illness, disability, etc.), emotional state (happy, sad, angry, nervous, surprised, etc.), age (child, youth, middle-aged, elderly).

Information such as other attributes and status of the in-vehicle personnel can be stored in the permanent storage of the onboard local computer system, as well as in optional cloud storage, or in a collaborative manner to achieve better results in both systems. For example, it is stored in the identity information database and/or the profile information database. As described above, both the identity information database and the profile information database may be combined into one, or may be separately stored and grouped. Weaving.

Returning to FIG. 6, in step S540, based on the results of face recognition S510, voiceprint recognition S520, and other attributes of the in-vehicle personnel and status recognition S530, reference is made to user attributes stored in the local/cloud (eg, identity information database and profile information). The user attribute stored in the database) identifies the identity information of the person in the vehicle and obtains the outline information of the person inside the vehicle. The contour information here may be retrieved from the contour information database, or input as a subsequent source based on 1 (in the case where there is no record of the person in the profile information database), or other persons in the vehicle The attributes and status identification are collected, or they can be obtained from the combination of the above.

Further, there is a voice command recognition and processing path in the vehicle automatic driving method 500.

In step S570, a voice command is recognized. It should be noted that the monitoring, identification and processing of voice commands exist in the whole process of automatic driving of the vehicle.

After the voice command is recognized, in step S580, it is determined whether the voice command is a manual designated travel mode. If the result of the determination is "Yes", the flow proceeds to the adaptive control step of step S550; if the result of the determination is "NO", the process proceeds to step S590.

In step S590, it is determined whether the voice command is feedback on the driving situation. If the result of the determination is "Yes", the flow proceeds to the adaptive control step of step S550; if the result of the determination is "NO", the process proceeds to step S591.

In step S591, it is determined whether the voice command is for personnel information entry. If the result of the determination is "Y (Y)", the flow proceeds to step S540 (shown as 1 in the figure) to perform in-vehicle personnel information entry, wherein in step S540, some basic data can be entered through voice interaction. Such as age, gender, preferred driving mode; and the camera in the car can take multiple photos, as well as recording voice voiceprints, fingerprints, etc., and store this information as identification information and/or contour information on the vehicle computer and / or cloud On the storage; if the judgment result is "No (N)", it is determined to be an invalid command, and it is not processed.

In this example, the speech recognition module recognizes the above three commands (manually specified driving mode, feedback on driving conditions, personnel information entry), and ignores other unrecognizable speech. Other commands can also be designed, identified, and processed as needed.

The speech recognition module can run on a local computer system in the vehicle, or in a server running in the cloud, or run in two systems in a collaborative manner for better results.

In step S550, adaptive control is performed. A detailed description of an example of the adaptive control method will be given later with reference to FIG.

In step S560, it is determined whether the vehicle automatic driving stroke ends, and if the result is "Y", the process results, otherwise returns to the beginning of the process, and continues to monitor the sensor input.

There are two kinds of adaptive control adjustment methods. One is to preset the parameters and adjustments that each command needs to adjust according to the prior knowledge. The other is to use the method of reinforcement learning to learn to determine these parameter values in a large number of driving. An example of the above two adaptive control adjustment methods will be described below with reference to FIGS. 7 and 8, respectively.

A detailed description of an example of an adaptive control method based on prior knowledge is given below in conjunction with FIG.

FIG. 7 shows a flow diagram of an exemplary adaptive control method 600 that may be applied to step S550 shown in FIG. 6, in accordance with an embodiment of the present invention.

As shown in FIG. 7, in step S610, based on the contour information of the in-vehicle personnel, it is determined whether each in-vehicle person is classified as a patient. When it is determined that a certain in-vehicle person is classified as a patient, the flow proceeds to step S611, and the system proceeds to step S611. Comfort mode is used by default. If it is determined that there is no patient among the persons in the vehicle based on the outline information of the in-vehicle personnel, the flow advances to step S620.

In step S620, it is determined whether each person in the vehicle has a preference mode, a specific address, and the outline information of each person in the vehicle is checked to see if all of the preferred driving modes are set. If step S620 determines that each of the in-vehicle personnel has their own preferred driving mode, then go to step S621, in which the driving mode of the vehicle is set to be the highest among the preferred driving modes of all in-vehicle personnel. Driving mode.

If it is determined in step S620 that not every in-vehicle person has his own preferred driving mode, then step S630 is reached.

In step S630, it is determined whether there is an elderly person, a child, and a disabled person among the persons in the driving mode in which the preference is not set. If the determination result is YES, the process goes to step S631, in which the comfort mode is used by default.

In step S640, the adaptive control unit collects sensor data and receives voice feedback from the person in the vehicle.

In step S650, it is determined whether or not there is voice feedback. If the result of the determination is YES, the process proceeds to step S660, otherwise, the process returns to step S640.

In step S660, the control is adaptively adjusted according to the voice feedback of each person, and these features are used to update the traveling mode of the corresponding individual. For example, the automatic driving system provides default control parameters for the comfort mode, and a passenger requires that the control during driving is smoother and safer. At this time, he will issue the command "re-stable", and the automatic driving system will receive the feedback command. These parameters are fine-tuned. And that The passenger may repeatedly issue commands to adjust until the final no longer adjusts, the system records the passenger's personalized comfort mode parameters as the passenger's preferred driving mode, and updates the passenger's profile information to the identity information database and/or profile. Information database.

In step S670, it is determined whether the vehicle has finished traveling, for example, has arrived at the destination, or received a parking order of the passenger, and the like. In the case where it is judged at step S670 that the running of the vehicle is over, the process ends, otherwise the process returns to step S640.

FIG. 8 illustrates a flow diagram of an exemplary adaptive control method 700 using an enhanced learning method, which may be applied to step S550 shown in FIG. 6, in accordance with another embodiment of the present invention.

Compared with FIG. 7, the difference of FIG. 8 is that the step S750 of adjusting the parameter control amount using the enhanced learning method is replaced with the step S660, and there is no step corresponding to the step S650 of determining whether or not there is voice feedback of FIG. The remaining steps in FIG. 8 are similar to the steps in FIG. 7 and will not be described again.

The speech feedback judging step is removed in FIG. 8 because, in the enhanced learning method, the speech feedback is used as a reward function value of the enhanced learning method, and when there is no speech in the vehicle, it is considered that there is a certain return function value, so It is not necessary to specifically judge whether or not there is voice feedback as a condition of the flow jump as shown in FIG.

A description is given below of an exemplary implementation of adjusting the automatic driving plan and control parameter control amount using the enhanced learning method, which may be used for step S750 of FIG.

Enhanced learning algorithms have different classifications, such as Monte Carlo methods and time difference methods, but the commonalities and cores of these methods include state sets, action sets, and reward functions. In one example, the state set, action set, and reward function are defined as follows.

State set S:

S1: longitudinal speed

S2: longitudinal acceleration

S3: lateral speed

S4: lateral acceleration

S5: the distance between the car and the car in front of the driveway

S6: the distance between the car and the car behind the driveway

S7: distance from the car in front of the left lane

S8: the distance between the car and the left lane

S9: distance from the car in front of the right lane

S10: distance from the car behind the right lane

S11: expected longitudinal speed

S12: expected lateral speed

S13: the orientation of the desired lane change

All of the above states are discretized interval values.

Action Set A:

A1: The throttle is 0, the brake is 0, and the front wheel is 0.

A2: Throttle is 0, brake is 0, front wheel yaw is +0.5 degrees

A3: The throttle is 0, the brake is 0, and the front wheel is +1 degree.

...

A51: throttle is 0, brake is 0, front wheel yaw is +25 degrees

A52: Throttle is 0, brake is 0, front wheel yaw angle is -0.5 degrees

A53: The throttle is 0, the brake is 0, and the front wheel is -1.0 degrees.

...

A100: Throttle is 0, brake is 0, front wheel yaw angle is -25 degrees

A101: throttle is 1, brake is 0, front wheel yaw is 0

A102: throttle is 2, brake is 0, front wheel yaw is 0

A103: throttle is 3, brake is 0, front wheel yaw is 0

A104: throttle is 4, brake is 0, front wheel yaw is 0

A105: throttle is 0, brake is 1, front wheel yaw is 0

A106: throttle is 0, brake is 2, front wheel yaw is 0

A107: The throttle is 0, the brake is 3, and the front wheel is 0.

A108: throttle is 0, brake is 4, front wheel yaw is 0

All of the above actions (throttle, brake, and front wheel yaw) are discretized values.

Return function R:

The feedback from the people in the car, the feedback information at this time is measured by the satisfaction of the people in the car, divided into five discrete values as shown in the following table, as shown in the fourth row of the table, for the case without feedback, The return function value r is set to 0, which is considered to be the same as the case where the person in the vehicle responds (for example, by voice) to "normal" or "just". The corresponding setting between the feedback information of this table and the reward function value r is merely an example, and those skilled in the art can design according to the situation.

Feedback Return r Very bad -2 Worse -1 General, just or no feedback 0 Better +1 very good +2

As an example of the type of time difference method in the enhanced learning method, taking the Q-Learning algorithm as an example, the exemplary process is as follows:

1. Initialize Q(s, a) to any value

2. Repeat the following loop until the change in policy is less than the preset threshold.

· Initialize s to any value

Repeat the following steps until s is terminated

Applying policyπ(s,a) in Q (for example, using the ε-greedy method), select an action a for the current state s

Perform action a, observe the immediate return r, and the next state s’

·Q(s,a)+=α[r+γmax _a ′Q(s',a′)-Q(s,a)]

·S=S’

In the above algorithm, a is a coefficient that controls the learning speed.

The formula for the ε-greedy method is as follows:

Here, preferably, the adaptive planning and control module operates on a vehicle-mounted local computer system.

In the foregoing exemplary description, the automatic driving mode is described as including a comfort mode, a normal mode, and a sport mode. This is just an example, allowing for more granular classification or more coarse-grained classification as needed.

It should be noted that in FIG. 7 and FIG. 8 , voice feedback is taken as an example of the feedback method, but this is only an example, and other feedback methods may be used depending on the situation, such as a physical button set on the car, an app on the passenger's mobile phone. Programs, passenger gestures, etc.

The vehicles in this article should be interpreted broadly. In addition to various large, medium and small vehicles traveling on land, they can also include ships on the surface of the water, and even aircraft that fly in the air.

It should be noted that the methods, systems, and devices discussed above are intended only as examples. It is emphasized that various embodiments may omit, replace, or add various processes or components as appropriate. For example, it is to be understood that in alternative embodiments, the methods may be performed in a different order than described, and the various steps may be added, omitted or combined. Moreover, features described with respect to certain embodiments may be combined in various other embodiments. Different aspects and elements of the embodiments can be combined in a similar manner. In addition, it should be emphasized that the technology continues to evolve and thus many of the elements illustrated herein are merely examples and are not to be construed as limiting the scope of the invention.

Moreover, the embodiments may be implemented in hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When the tasks are implemented in software, firmware, middleware or microcode, the program code or code segments needed to perform the tasks can be stored in a computer readable medium, such as a storage medium, and the processor can perform the required tasks.

The embodiments of the present invention have been described above, and the foregoing description is illustrative, not limiting, and not limited to the disclosed embodiments. Numerous modifications and changes will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims (25)

A vehicle automatic driving method includes: Sense the insider of the car and obtain the outline information of the person inside the car; Classify the personnel in the car; and According to the classification, the corresponding automatic driving mode is determined and the vehicle is adaptively controlled. The vehicle automatic driving method according to claim 1, further comprising: Continuously receiving feedback from people in the car while driving; and The parameters of the automatic driving are adaptively adjusted according to the feedback of the insiders. The vehicle automatic driving method according to claim 1, wherein the profile information of the in-vehicle personnel includes a traveling mode preferred by each person in the vehicle. The vehicle automatic driving method according to claim 3, wherein each of the preferred driving modes of the person is represented by a corresponding driving parameter. The vehicle automatic driving method according to claim 1, wherein said classifying the in-vehicle personnel comprises: Determine if each person in the car is a patient; Determine if each person in the car is an elderly person, a child or a disabled person; Determine if everyone has set the preferred driving mode. The vehicle automatic driving method according to claim 1, wherein said automatic driving mode comprises: a comfort mode, a normal mode, and a sport mode, and said determining the corresponding autonomous driving mode according to said categorization comprises: Determining whether there is a patient in the vehicle, and if there is a patient, determining that the automatic driving mode is a comfortable mode; If there is no patient in the vehicle, it is determined whether each person in the car has a preferred driving mode, and if each person in the car has a preferred driving mode, the driving mode with the highest comfort among the preferred driving modes is used; If not everyone in the car has a preferred driving mode, it is determined whether there is an elderly person, a child, or a disabled person among those who do not have the preferred driving mode; and if present, the comfort mode is used. The vehicle automatic driving method according to claim 1, said sensing in-vehicle personnel comprising using at least one of the following means: Camera, pressure sensor and microphone, fingerprint reader, infrared sensor. The vehicle automatic driving method according to claim 1, wherein the obtaining the contour information of the in-vehicle personnel comprises: Identifying the identity of the person in the vehicle and identifying the identity of the person in the vehicle includes identifying the identity of the person in the vehicle by one or a combination of the following identification technologies based on the identity information database: face recognition, voiceprint recognition, fingerprint recognition , iris recognition, infrared recognition; Based on the identified identity of the in-vehicle personnel, the profile information of the in-vehicle personnel is retrieved from the profile information database, and the profile information is stored in association with the identity of the in-vehicle personnel in the profile information database. According to the automatic driving method of claim 8, in the case where the outline information of the in-vehicle person is not stored in the outline database, the outline information input of the in-vehicle person is received and stored in the database. The vehicle automatic driving method according to claim 8, wherein said profile information comprises: One or more of age, gender, mood, health status information, and preferred driving patterns. The vehicle automatic driving method of claim 1, the different modes corresponding to associated planning and control parameters selected from one or more of the following: Normal acceleration, abnormal acceleration, normal deceleration, super deceleration, maximum value of front wheel declination and vehicle speed ratio, safety distance, minimum frequency of lane change, and safe distance of lane change. The vehicle automatic driving method according to claim 2, wherein the adaptively adjusting parameters of the automatic driving according to feedback of a person in the vehicle includes: In response to a feedback command from the person in the vehicle to the current driving state, the adjustment is made using a preset parameter and an adjustment amount corresponding to the feedback, the feedback command indicating a determination adjustment of the driving state. The vehicle automatic driving method according to claim 12, wherein when the user no longer gives a feedback command, the current parameter is recorded as the preferred driving mode parameter of the user. The vehicle automatic driving method according to claim 2, wherein the adaptively adjusting parameters of the automatic driving according to feedback of a person in the vehicle includes: The enhanced learning method is used to adjust in response to the feedback command of the person in the vehicle to the current driving state. The vehicle automatic driving method according to claim 8, wherein one or more of said identification technologies operate in a local in-vehicle computer system, in a cloud server, or in a local in-vehicle calculation The machine system and the cloud server operate in a cooperative manner. The vehicle automatic driving method according to claim 8, wherein one or both of the identity information database and the profile information database of the in-vehicle personnel are stored in a local in-vehicle computer system or in cloud storage. The vehicle automatic driving method according to any one of claims 1 to 16, further comprising: Sensing the presence of specific animals and/or specific items in the vehicle; Based on sensing the presence of a particular animal and/or particular item within the vehicle, a corresponding autonomous driving mode is determined. A vehicle automatic driving system comprising: a sensor that senses the person inside the vehicle; a contour information obtaining unit, configured to obtain contour information of a person in the vehicle; a classification unit that classifies the personnel in the vehicle; The adaptive control unit determines the corresponding automatic driving mode according to the classification, and adaptively controls the vehicle. The automatic vehicle driving system of claim 18, further comprising: a feedback receiving unit that continuously receives feedback from the inside of the vehicle during driving; The adaptive control unit adaptively adjusts the parameters of the automatic driving according to the feedback of the person inside the vehicle. The vehicle automatic driving system according to claim 18, wherein the profile information of the in-vehicle personnel includes a traveling mode preferred by each person in the vehicle. The vehicle automatic driving system according to claim 18, wherein said classifying the in-vehicle personnel comprises: Determine if each person in the car is a patient; Determine if each person in the car is an elderly person, a child or a disabled person; Determine if everyone has set the preferred driving mode. A vehicle automatic driving system according to any one of claims 18 to 21, wherein said automatic driving mode comprises: a comfort mode, a normal mode, and a sport mode, and said determining the corresponding autonomous driving mode according to said categorization comprises: Determining whether there is a patient in the vehicle, and if there is a patient, determining that the automatic driving mode is a comfortable mode; If there are no patients in the car, determine if everyone in the car has a preferred driving mode. If everyone in the car has a preferred driving mode, use these preferred driving modes to be comfortable. The highest driving mode; If not everyone in the car has a preferred driving mode, it is determined whether there is an elderly person, a child, or a disabled person among those who do not have the preferred driving mode; and if present, the comfort mode is used. The vehicle automatic driving system according to claim 19, wherein said adaptively adjusting parameters of the automatic driving according to feedback from a person in the vehicle includes: The enhanced learning method is used to adjust in response to the feedback command of the person in the vehicle to the current driving state. A vehicle automatic driving system according to any one of claims 18 to 21, wherein The sensor also senses whether a particular animal and/or a particular item is present in the vehicle; The adaptive control unit determines the corresponding autonomous driving mode based on sensing that a particular animal and/or particular item is present within the vehicle. A vehicle automatic driving method includes: Sense the insider of the car and obtain the outline information of the person inside the car; The vehicle is adaptively controlled based on the obtained contour information of the in-vehicle personnel.

Images