CN116409238A

CN116409238A - Vehicle-mounted interaction system, method, device and storage medium

Info

Publication number: CN116409238A
Application number: CN202111666708.7A
Authority: CN
Inventors: 赵正伟; 李晓东; 沈旭东
Original assignee: Shanghai Jidu Automobile Co Ltd
Current assignee: Shanghai Jidu Automobile Co Ltd
Priority date: 2021-12-31
Filing date: 2021-12-31
Publication date: 2023-07-11

Abstract

The embodiment of the application provides a vehicle-mounted interaction system, a method, equipment and a storage medium, wherein the system comprises the following components: the display device is used for displaying an environment image, and the environment image comprises a target interaction object; the controller is coupled with the display device and the plurality of external interaction devices and is used for receiving a selection instruction of a user in the vehicle for the target interaction object, determining the position information of the target interaction object relative to the vehicle and determining the target external interaction device corresponding to the position information according to the setting positions corresponding to the plurality of external interaction devices; the external interaction devices are arranged on the outer sides of the vehicle bodies corresponding to different directions; and the target external interaction device is used for outputting interaction information to the target interaction object under the control of the controller.

Description

Vehicle-mounted interaction system, method, device and storage medium

Technical Field

The application relates to the technical field of intelligent interaction, in particular to a vehicle-mounted interaction system, a vehicle-mounted interaction method, vehicle-mounted interaction equipment and a vehicle-mounted interaction storage medium.

Background

Scenes that require interaction with an off-board person are often encountered during driving of a car. In an actual scene, interaction of warning sounds can be carried out only through whistling, or communication can be directly carried out through opening vehicle windows and vehicle doors.

However, due to noise interference and distance limitations, the interaction efficiency inside and outside the automobile is low.

Disclosure of Invention

The embodiment of the application provides a vehicle-mounted interaction system, a vehicle-mounted interaction method, vehicle-mounted interaction equipment and a vehicle-mounted storage medium, which are used for improving interaction efficiency inside and outside an automobile.

In a first aspect, an embodiment of the present application provides a vehicle-mounted interaction system, the system including:

a display device for displaying an environmental image, the environmental image comprising a target interactive object;

the controller is coupled with the display device and the plurality of external interaction devices and is used for receiving an instruction for selecting a target interaction object, determining the position information of the target interaction object relative to the vehicle, and determining the target external interaction device corresponding to the position information according to the setting positions corresponding to the plurality of external interaction devices;

the external interaction devices are arranged at different positions outside the vehicle body of the vehicle;

the external target interaction device is configured to interact with the target interaction object under the control of the controller, where the external interaction devices include external speakers according to an embodiment of the present application.

According to an embodiment of the present application, the plurality of external interaction devices further includes a plurality of external microphones corresponding to the setting positions of the plurality of speakers.

According to an embodiment of the present application, the system further includes a rotation mechanism for mounting the external speaker, wherein the rotation mechanism includes:

the first bracket is fixed on the vehicle body and is used for fixing the first motor or the second motor;

the first motor is coupled with the controller and is used for adjusting the course angle of the external loudspeaker under the control of the controller so as to be matched with the position information;

the second motor is coupled with the controller and is used for adjusting the pitch angle of the external loudspeaker under the control of the controller so as to be matched with the position information;

the second bracket is fixed on the output shaft of the second motor or the output shaft of the first motor and is used for fixing the external loudspeaker, and the external loudspeaker is fixed on the second bracket;

the output shaft of the first motor is perpendicular to the output shaft of the second motor, and the arrangement modes of the first motor and the second motor comprise any one of the following modes: the first motor is fixed on the output shaft of the second motor, and the second motor is fixed on the output shaft of the first motor.

According to the embodiment of the application, the external interaction devices are respectively arranged on the cross beam at the front side of the vehicle body, the longitudinal beams at the left side and the right side of the vehicle body and the anti-collision beam at the rear side of the vehicle body.

According to an embodiment of the application, the external speaker comprises a directional speaker and the external microphone comprises a directional microphone.

According to the embodiment of the application, the sound producing direction of the external speaker and the sound receiving direction of the external microphone are determined by the position information.

According to an embodiment of the present application, the system further comprises: a built-in interaction device comprising a built-in speaker and a built-in microphone;

the built-in speaker is coupled with the controller and is used for outputting first audio information which is acquired by the external microphone and corresponds to the target interaction object;

the built-in microphone is coupled with the controller and used for collecting second audio information of a user in the vehicle and sending the second audio information to the target external loudspeaker corresponding to the target interaction object through the controller.

According to an embodiment of the present application, further comprising: an image generating device for generating the environment image outside the vehicle.

According to an embodiment of the present application, the image generating apparatus includes at least one of: vehicle-mounted camera and imaging radar.

In a second aspect, an embodiment of the present application provides a vehicle-mounted interaction method, where the method includes:

Displaying the collected environment image outside the vehicle on a map interface;

determining a selected target interactive object in the environment image;

determining position information of the target interactive object relative to the vehicle;

determining target external interaction equipment corresponding to the position information according to the setting positions corresponding to the external interaction equipment, wherein the external interaction equipment is arranged at different positions outside the vehicle body of the vehicle;

and outputting interaction information corresponding to the target interaction object through the target external interaction device.

According to the embodiment of the application, the plurality of external interaction devices comprise a plurality of external speakers, and the target external interaction device is a target external speaker; after the external target interaction device corresponding to the position information is determined, the method further comprises the following steps:

and controlling the target external speaker to rotate according to the position information so as to enable the sounding direction of the target external speaker to be matched with the position information.

According to an embodiment of the present application, the target external interaction device is a target external speaker, and the outputting, by the target external interaction device, interaction information corresponding to the target interaction object includes:

And responding to the selection operation of the user on the displayed first interaction mode, outputting preset audio information corresponding to the first interaction mode through the target external loudspeaker, wherein the first interaction mode corresponds to a mode of playing the preset audio information outside the vehicle.

According to the embodiment of the application, the target external interaction device comprises a target external speaker and a target external microphone corresponding to the setting position of the target external speaker, and an internal speaker and an internal microphone are arranged in a vehicle;

the outputting, by the external target interaction device, interaction information corresponding to the target interaction object includes:

responding to the selection operation of a user on a displayed second interaction mode, starting the built-in loudspeaker and the built-in microphone, wherein the second interaction mode corresponds to an interaction mode of bidirectional interaction inside and outside the vehicle;

outputting first audio information corresponding to the target interaction object, which is acquired by the target external microphone, through the built-in loudspeaker;

and outputting the second audio information of the in-vehicle user acquired through the built-in microphone to the target external loudspeaker.

In a third aspect, an embodiment of the present application provides a vehicle-mounted interaction device, where the device includes:

The display module is used for displaying the collected environment image outside the vehicle on the map interface;

a first determining module for determining a target interactive object selected in the environment image;

the second determining module is used for determining the position information of the target interactive object relative to the vehicle;

the third determining module is used for determining target external interaction equipment corresponding to the position information according to the setting positions corresponding to the external interaction equipment, and the external interaction equipment is arranged at different positions on the outer side of the vehicle body of the vehicle;

and the interaction module is used for outputting interaction information corresponding to the target interaction object through the target external interaction device.

In a fourth aspect, an embodiment of the present application provides a vehicle-mounted interaction device, where the vehicle-mounted interaction device includes: a memory, a processor; wherein the memory has executable code stored thereon which, when executed by the processor, causes the processor to perform the vehicle-mounted interaction method as described in the second aspect.

In a fifth aspect, embodiments of the present application provide a vehicle comprising an interaction device according to the third aspect or an interaction apparatus according to the fourth aspect.

In a sixth aspect, embodiments of the present application provide a non-transitory machine-readable storage medium having executable code stored thereon, which when executed by a processor of an electronic device, causes the processor to perform the in-vehicle interaction method according to the second aspect.

In the scheme of the embodiment of the application, the interaction efficiency and effect inside and outside the automobile are improved. By capturing an image of the environment outside the vehicle, the user can then select a target interactive object in the display device; determining the position information of the target interaction object relative to the vehicle based on the controller, and determining target external interaction equipment corresponding to the position information according to the set positions corresponding to the external interaction equipment; finally, the sounding direction of the target external interaction device is adjusted, so that the sounding direction of the target external interaction device corresponds to the target interaction object, and directional interaction with the target interaction object is realized. Therefore, through the directional interaction between the external target interaction equipment and the target interaction object, the propagation effect of the interaction information is improved, and the interaction efficiency inside and outside the automobile is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic structural diagram of a vehicle-mounted interaction system provided in an embodiment of the present application;

fig. 2A is a schematic diagram of an installation position of an external interaction device according to an embodiment of the present application;

fig. 2B is a schematic diagram of an installation position of an external interaction device of a front cross beam, a left longitudinal beam and a right longitudinal beam according to an embodiment of the present application;

fig. 2C is a schematic diagram of an installation position of an external interaction device of a rear side impact beam according to an embodiment of the present application;

fig. 2D is a schematic diagram of a sound field range of an external interaction device according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of a rotating mechanism according to an embodiment of the present disclosure;

fig. 4 is a schematic structural view of a rotating mechanism according to another embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of a built-in interaction device according to an embodiment of the present application;

fig. 6 is a schematic diagram of a vehicle-mounted interaction method provided in an embodiment of the present application;

FIG. 7A is a schematic view of pitch angle provided by an embodiment of the present application;

FIG. 7B is a schematic view of a heading angle provided by an embodiment of the present application;

fig. 8 is a schematic diagram of another vehicle-mounted interaction method provided in an embodiment of the present application;

FIG. 9 is a schematic diagram of interactions between vehicles provided in an embodiment of the present application;

fig. 10 is a schematic diagram of an in-vehicle apparatus provided in an embodiment of the present application;

fig. 11 is a schematic structural diagram of a vehicle-mounted interaction device provided in an embodiment of the present application;

fig. 12 is a schematic structural diagram of a vehicle according to an embodiment of the present application.

Detailed Description

For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

In addition, in some of the flows described in the specification and claims of this application and the foregoing figures, a plurality of operations occurring in a particular order are included, but it should be understood that the operations may be performed out of order or performed in parallel, with the order of operations such as 101, 102, etc., being merely used to distinguish between the various operations, the order of the operations itself not representing any order of execution. In addition, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first" and "second" herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, and are not limited to the "first" and the "second" being different types.

In the running or starting process of the automobile, due to interference of external noise and limitation of distance, a driver cannot communicate or effectively prompt with personnel outside the automobile, so that the communication efficiency between the inside and the outside of the automobile is low. In some cases, even safety problems may occur because in-car personnel may not be able to adequately prompt pedestrians on the road. In an actual scene, interaction of warning sounds can be carried out only through whistling, or communication can be directly carried out through opening vehicle windows and vehicle doors. However, the above method also has poor communication efficiency and effect due to noise and distance.

In order to solve the above technical problems, the core idea of the vehicle-mounted interaction system provided in the embodiment of the present application is that: by capturing an image of the environment outside the vehicle, the user can then select a target interactive object in the display device; determining the position information of the target interaction object relative to the vehicle based on the controller, and determining target external interaction equipment corresponding to the position information according to the set positions corresponding to the external interaction equipment; finally, the sounding direction of the target external interaction device is adjusted, so that the sounding direction of the target external interaction device corresponds to the target interaction object, and directional interaction with the target interaction object is realized. Therefore, through the directional interaction between the external target interaction equipment and the target interaction object, the propagation effect of the interaction information is improved, and the interaction efficiency inside and outside the automobile is improved.

The vehicle-mounted interaction system provided by the embodiment of the application can be borne on any kind of vehicle. The vehicle-mounted interaction method provided by the embodiment of the application can be executed by an electronic device, and the electronic device can be a terminal device such as a vehicle machine, a PC (personal computer), a notebook computer and the like, or can be a server. The server may be a physical server or may be a virtual server. The server may be a physical or virtual server on the user side, or may be a cloud server.

The vehicle-mounted interactive system provided by the application is described in detail below in connection with the following embodiments.

Fig. 1 is a schematic structural diagram of a vehicle-mounted interaction system provided in an embodiment of the present application. As shown in fig. 1, the in-vehicle interactive system 100 may include an image generating apparatus 101, a display device 102, a controller 103, and a plurality of external interactive apparatuses 104.

An image generating device 101 for acquiring an environmental image outside the vehicle. In the present embodiment, the image generating apparatus 101 may include a plurality of in-vehicle cameras that may be disposed outside the vehicle body in different directions. For example, the vehicle-mounted camera may be mounted on a rear view mirror on two sides of the vehicle body, a front anti-collision beam and a rear anti-collision beam of the vehicle body, or a front windshield and a rear windshield of the vehicle body, and according to specific implementation requirements, the vehicle-mounted camera may be disposed at any position of the vehicle body.

Specifically, the vehicle-mounted camera may include a monocular camera, a binocular camera, a trinocular camera, a wide-angle camera, a night vision camera, a fisheye camera, or the like. Furthermore, the vehicle-mounted camera can measure information such as the distance between the object and the vehicle body based on a pure vision method.

In the present embodiment, the image generating apparatus may further include a plurality of imaging radars, and the environment image outside the vehicle may also be generated by a radar imaging method.

A display device 102, coupled to the image generation apparatus 101, for displaying the ambient image. In this embodiment, the display device 102 may include a display terminal such as a car screen, a mobile phone, a computer, and the like, and may also include intelligent wearable display devices such as AR glasses and VR glasses. The display device 102 may be communicatively connected to the image generation apparatus 101 to acquire and display the environmental image acquired by the image generation apparatus 101. Specifically, the environment image may include a plurality of interactive objects, and the interactive objects may include automobiles, pedestrians, and the like. Further, the display device 102 may obtain an interactive object selection operation of the in-vehicle user to determine the target interactive object. The selection operation may include clicking, sliding, etc. In addition, the target interaction object may be determined by capturing the line of sight of the user by the driver monitoring system (driver monitoring system, DMS), or may be determined by gestures, eye, head rotation, etc. of the user.

The controller 103 is coupled to the display device 102 and the plurality of external interaction devices 104, and is configured to receive an instruction for selecting a target interaction object, determine position information of the target interaction object relative to the vehicle, and determine a target external interaction device 104 corresponding to the position information according to setting positions corresponding to the plurality of external interaction devices 104. In this embodiment, the instruction for selecting the target interactive object may be issued by a user in the vehicle by operating the display device 102, and the controller 103 may determine the position information of the target interactive object with respect to the vehicle according to the environment image, for example: the target interactive object is located in front of, behind or to the right of the vehicle, etc. The controller 103 may then determine the target external interactive device 104 according to the installation position of each external interactive device 104 on the vehicle, for example, the external interactive device 104 closest to the target interactive object may be determined as the target external interactive device 104, or the external interactive device 104 on the same side as the target interactive object may be determined as the target external interactive device 104. The external target interaction device 104 is configured to interact with a target interaction object under the control of the controller, where the interaction manner may include sending information or receiving information. It should be noted that the number of the external target interaction devices 104 may be one or more, and the number of the external target interaction devices 104 is not specifically limited in the embodiment of the present application.

Further, the controller 103 may determine the sounding angle of the external target interactive device 104 according to the position of the external target interactive object relative to the vehicle, so that the sound field of the external target interactive device 104 covers the external target interactive object. Through the directional interaction between the target external interaction device 104 and the target interaction object, the propagation effect of the interaction information is improved, and therefore the interaction efficiency inside and outside the automobile is improved.

In this embodiment, the external interaction devices 104 may be disposed at different positions outside the vehicle body, for example, the external interaction devices 104 may be disposed at positions in different directions of the vehicle body. The target external interactive device 104 may be used to output interactive information to a target interactive object under the control of the controller 103.

According to an embodiment of the present application, the plurality of external interaction devices 104 includes a plurality of external speakers, and in particular, the external speakers may include sound, speakers, and the like. The external interaction devices 104 further include external microphones corresponding to the locations of the speakers.

In this embodiment, the external speakers and the external microphones may be respectively mounted in four directions of front, rear, left and right of the vehicle body, and at least one external speaker and one external microphone are mounted on the vehicle body in each direction.

In practice, the external speaker comprises a directional speaker, and the external microphone comprises a directional microphone. The sound emission direction of the external speaker and the sound emission direction of the external microphone can be determined by the position information of the target interactive object relative to the vehicle.

The following describes an exemplary installation position of the external interaction device according to the embodiment of the present application with reference to fig. 2A to 2D. As shown in fig. 2A, taking 4 external interaction devices 201 as an example, four external interaction devices 201 may be respectively disposed at a front end, a rear end, a left end and a right end of a vehicle body. Specifically, as shown in fig. 2B, the external interactive device 201 located at the front end of the vehicle body may be mounted on the front cross member 202 on the front side of the vehicle body, and the external interactive devices 201 located at the left and right ends of the vehicle body may be mounted on the left side member 203 of the vehicle body and the right side member 204 of the vehicle body, respectively. Also, as shown in fig. 2C, the external interaction device 201 at the rear end of the vehicle body may be mounted to the rear side impact beam 205 at the rear end of the vehicle body. It should be noted that, the above embodiments are merely exemplary embodiments, and the external interaction device 201 may be installed at any position of the vehicle body according to specific implementation requirements, and the installation position of the external interaction device 201 is not specifically limited in this embodiment of the present application. Further, the external microphone may be mounted at the same position as the external speaker, or may be mounted at a position different from the external speaker.

According to the embodiment of the application, each external interaction device 201 may correspond to different interaction angles, as shown in fig. 2D, the interaction angle corresponding to the external interaction device 201 located at the front end of the vehicle body is a range of 90 degrees at the front end of the vehicle body, the interaction angle corresponding to the external interaction device 201 located at the rear end of the vehicle body is a range of 90 degrees at the rear end of the vehicle body, the interaction angle corresponding to the external interaction device 201 located at the left end of the vehicle body is a range of 90 degrees at the left end of the vehicle body, and the interaction angle corresponding to the external interaction device 201 located at the right end of the vehicle body is a range of 90 degrees at the right end of the vehicle body.

According to an embodiment of the application, the vehicle-mounted interaction system further comprises a rotating mechanism for installing the external loudspeaker. As shown in fig. 3, the rotation mechanism 300 includes a first bracket 301, a first motor 302, a second motor 303, and a second bracket 304.

Specifically, the first bracket 301 is mounted on the vehicle body, the first motor 302 is fixed at the bottom of the first bracket 301, the second motor 303 is fixed on the output shaft of the first motor 302, the second bracket 304 is fixed on the output shaft of the second motor 303, the external speaker 305 may be mounted on the second bracket 304, and the output shaft of the first motor 302 is perpendicular to the output shaft of the second motor 303. The first stand-alone motor 303 and the second motor 303 are both coupled to the controller, the first motor 302 can adjust the course angle of the external speaker 305 under the control of the controller, and the second motor 303 can adjust the pitch angle of the external speaker 305 under the control of the controller, so that the sound field of the external speaker 305 is matched with the position information of the target interactive object relative to the vehicle through the adjustment.

In another embodiment of the present application, the first bracket 301 is mounted on the vehicle body, the second motor 303 may be fixed to the bottom of the first bracket 301, the first motor 302 is fixed to the output shaft of the second motor 303, the second bracket 304 is fixed to the output shaft of the first motor 302, the external speaker 305 may be mounted on the second bracket 304, and the output shaft of the first motor 302 is perpendicular to the output shaft of the second motor 303. Correspondingly, the first stand-alone motor 303 and the second motor 303 are both coupled with the controller, the first motor 302 can still adjust the course angle of the external speaker 305 under the control of the controller, the second motor 303 can still adjust the pitch angle of the external speaker 305 under the control of the controller, and through the adjustment, the sound field of the external speaker 305 can also be matched with the position information of the target interaction object relative to the vehicle. Furthermore, the rotating mechanism can be provided with a singlechip to realize information interaction between the controller and the first motor and the second motor. As shown in fig. 4, the controller may be connected to the single-chip microcomputer through the first step-down circuit, or may be connected to the single-chip microcomputer through the wireless receiving module. The singlechip is electrically connected with the first motor and the second motor. When the external loudspeaker is in operation, the singlechip is powered by the second voltage reduction circuit and the power supply, the controller sends the position information to the singlechip, and the singlechip respectively adjusts the rotation angles of the first motor and the second motor according to the position information so as to enable the sound field of the external loudspeaker to be matched with the position information of the target interaction object relative to the vehicle.

To further facilitate communication between the interior and the exterior of the vehicle cabin, the in-vehicle interactive system may further include a built-in interactive device. As shown in fig. 5, the built-in interaction device includes a built-in speaker 501 and a built-in microphone 502.

Both the built-in speaker 501 and the built-in microphone 502 may be installed in the cab of the vehicle. Both the built-in speaker 501 and the built-in microphone 502 are coupled to the controller 503. The internal speaker 501 may be used to output first audio information corresponding to a target interactive object collected by the external microphone 504; the built-in microphone 502 may be used to collect second audio information of the user in the vehicle and send it to the target external speaker 505 corresponding to the target interactive object through the controller 503.

Through the built-in interaction device, the in-vehicle user can receive information sent by the target interaction object in real time and can send the information to the target interaction object in real time. Based on the built-in interaction device, the communication efficiency and effect between the interior and the exterior of the vehicle cab can be improved.

After the vehicle-mounted interaction system provided by the embodiment of the present application is introduced, the vehicle-mounted interaction method provided by the embodiment of the present application is described in detail below with reference to fig. 6.

Fig. 6 is a schematic diagram of a vehicle-mounted interaction method provided in an embodiment of the present application. As shown in fig. 6, the in-vehicle interaction method includes S601 to S605.

At S601, the acquired environment image outside the vehicle is displayed on the map interface. In this embodiment, an environment image outside the vehicle may be acquired by the image generating apparatus, and the map interface may be presented on the display device.

At S602, a target interactive object selected in the environment image is determined. In this embodiment, the selection operation of the user on the interactive object in the vehicle may be acquired through the display device, so as to determine the target interactive object in the environment image.

At S603, position information of the target interactive object with respect to the vehicle is determined. In this embodiment, the method for determining the position information of the target interactive object with respect to the vehicle may be performed by the controller.

In S604, a target external interaction device corresponding to the position information is determined according to the setting positions corresponding to the external interaction devices, where the external interaction devices are located at different positions outside the body of the vehicle. In this embodiment, the method for determining the external interaction device may also be executed by the controller. The external interaction devices can be respectively arranged at the positions of the vehicle body in different directions.

In S605, the interaction information corresponding to the target interaction object is output through the target external interaction device.

According to the embodiment of the application, the external interaction devices comprise external speakers and external microphones, and the target external interaction device is a target external speaker. And after the external target interaction equipment corresponding to the position information is determined, controlling the external target loudspeaker to rotate according to the position information so as to enable the sounding direction of the external target loudspeaker to be matched with the position information.

In practical application, preferably, the external speaker and the external microphone can be operated in a directional mode, and the sounding angle of the external speaker and the radio angle of the external microphone can be adjusted through rotation of a mechanical structure or through beam forming of the array.

In this embodiment, according to the position information, the heading angle and pitch angle of the target interactive object with respect to the vehicle may be determined.

As shown in fig. 7A, taking the target interactive object as an example of a pedestrian Z, where the height of the pedestrian Z is X meters and the distance from the vehicle in the vehicle body direction is L meters, the pitch angle α of the target interactive object with respect to the vehicle can be determined according to the following formula (1).

α＝arctan(X/L) (1)

As shown in fig. 7B, taking the above pedestrian Z as an example, the distance between the pedestrian Z and the vehicle in the direction perpendicular to the vehicle body is M meters, the heading angle β of the target interactive object with respect to the vehicle can be determined according to the following formula (2).

β＝arctan(N/L) (2)

According to an embodiment of the present application, the target external interaction device is a target external speaker, and outputting, by the target external interaction device, interaction information corresponding to a target interaction object, including: and responding to the selection operation of the user on the displayed first interaction mode, and outputting preset audio information corresponding to the first interaction mode through the target external loudspeaker.

In this embodiment, the first interaction mode may include a mode of playing preset audio information outside the vehicle, that is, a one-way interaction mode. The preset audio information may include preset alert tones, etc. Specifically, the user may select the interaction mode at the display device, and after selecting the first interaction mode, may select the preset audio information, and finally, output the preset audio information through the external interaction device.

In practical application, for example, if a user in a vehicle wants to send a preset prompt tone to a pedestrian in front of the vehicle, the user can select the pedestrian in the display device and continue to select the first interaction mode and the preset prompt tone, and then the external interaction device can play the preset prompt tone based on the position of the pedestrian.

According to the embodiment of the application, the target external interaction device comprises a target external speaker and a target external microphone corresponding to the setting position of the target external speaker, and an internal speaker and an internal microphone are arranged in the vehicle.

Outputting, by the target external interaction device, interaction information corresponding to the target interaction object, including: and responding to the selection operation of the user on the displayed second interaction mode, starting the built-in loudspeaker and the built-in microphone, wherein the second interaction mode corresponds to the interaction mode of the bidirectional interaction between the inside and the outside of the vehicle. And outputting first audio information corresponding to the target interaction object, which is acquired by the target external microphone, through the built-in loudspeaker. And outputting the second audio information of the in-vehicle user acquired through the built-in microphone to the target external loudspeaker.

In this embodiment, the second interaction mode may include a bidirectional interaction mode, that is, a mode in which the user in the vehicle performs bidirectional information interaction with the target interaction object. Specifically, the external microphone can acquire the voice information of the target interaction object in real time and play the voice information in the in-vehicle speaker; the built-in microphone can acquire the voice information of the user in the vehicle in real time and play the voice information in the external loudspeaker.

In practical applications, for example, a user in a vehicle wants to communicate with a pedestrian in front of the vehicle. The second interaction mode may be selected in the display device to communicate with the internal interaction device via the external interaction device.

Fig. 8 schematically illustrates a schematic diagram of another vehicle-mounted interaction method according to an embodiment of the application.

As shown in fig. 8, when the user in the vehicle needs to interact with the pedestrian outside the vehicle, the map interface may be displayed by the display device, and the pedestrian, the vehicle, or the like may be selected as the target interaction object on the map interface. The user may then select an interaction pattern on the display device. After the interaction mode is determined, the external interaction device adjusts the external loudspeaker to the position of the target interaction object. In the case of the first interaction mode, the user may select a preset audio to be played by the external speaker. And under the condition of the second interaction mode, starting the built-in interaction equipment, and enabling the user to interact with the target interaction object through the built-in interaction equipment and the external interaction equipment.

In practical applications, there are also scenes in which vehicles interact with each other. For example, in the driving process, a user of a certain vehicle needs to perform information interaction with a user of another adjacent vehicle, and at this time, information interaction between vehicles can also be performed by using the vehicle-mounted interaction method provided by the embodiment of the present application.

Specifically, as shown in fig. 9, information interaction between the vehicle a and the vehicle B is taken as an example. During traveling, the display device of the vehicle a displays the environment image generated by the image generating device, and the user of the vehicle a can select the vehicle B as a target interaction object at the map interface while selecting the first interaction mode or the second interaction mode. Then, the vehicle A generates an interaction request according to a selection result of a user, and sends the interaction request to the vehicle B, and the vehicle B displays the position information and the interaction mode information of the vehicle A in a display device after acquiring the interaction request. The user of vehicle B may choose to approve or reject the interaction at the map interface.

In the case where the user of the vehicle B agrees to the interaction, the vehicle B may interact with the user of the vehicle a based on the interaction pattern information summarized by the interaction information. In the case of the first interaction mode (i.e., the vehicle a sends audio information to the vehicle B), the user of the vehicle a turns on the external speaker and adjusts the sound field of the external speaker to the direction of the vehicle B; the vehicle B starts the external microphone, and adjusts the sound receiving angle of the external microphone to the direction of the vehicle A, so that the user A sends preset audio information to the vehicle B through the external loudspeaker, and the vehicle B can acquire the preset audio information sent by the vehicle A through the external microphone. In the second interaction mode (i.e., the two-way interaction between the vehicle a and the vehicle B), both the vehicle a and the vehicle B turn on the external speaker and the external microphone, and simultaneously adjust the external speaker and the external microphone to the direction of the opposite vehicle, so that the vehicle a can perform the two-way interaction between the vehicle B.

In the case where the user of the vehicle B refuses the interaction, the vehicle B may return information refusing the interaction to the vehicle a so as to prompt the user of the vehicle a with the information.

Vehicle-mounted interaction devices of one or more embodiments of the present application are described in detail below.

Fig. 10 is a schematic structural diagram of a vehicle-mounted interaction device provided in the embodiment of the present application, and as shown in fig. 10, the device 1000 includes a display module 1001, a first determining module 1002, a second determining module 1003, a third determining module 1004, and an interaction module 1005.

And a display module 1001 for displaying the collected environment image outside the vehicle on the map interface.

A first determining module 1002 is configured to determine a target interactive object selected in the environment image.

A second determining module 1003 is configured to determine positional information of the target interactive object relative to the vehicle.

The third determining module 1004 is configured to determine, according to the set positions corresponding to the plurality of external interaction devices, a target external interaction device corresponding to the position information, where the plurality of external interaction devices are located at different positions outside the vehicle body.

The interaction module 1005 is configured to output, through the external target interaction device, interaction information corresponding to the target interaction object.

According to an embodiment of the present application, the vehicle-mounted interaction device 1000 further includes a rotation module.

And the rotating module is used for controlling the target external loudspeaker to rotate according to the position information so as to enable the sounding direction of the target external loudspeaker to be matched with the position information.

According to an embodiment of the present application, the interaction module 1005 comprises a response unit.

The response unit is used for responding to the selection operation of the user on the displayed first interaction mode, outputting preset audio information corresponding to the first interaction mode through the target external loudspeaker, wherein the first interaction mode corresponds to a mode of playing the preset audio information outside the automobile.

According to an embodiment of the present application, the interaction module 1005 further comprises a first response unit, a first output unit and a second output unit.

The first response unit is used for responding to the selection operation of a user on a displayed second interaction mode, starting the built-in loudspeaker and the built-in microphone, and the second interaction mode corresponds to an interaction mode of bidirectional interaction inside and outside the vehicle;

the first output unit is used for outputting first audio information which is acquired by the target external microphone and corresponds to the target interaction object through the built-in loudspeaker;

and the second output unit is used for outputting second audio information of the in-vehicle user acquired through the built-in microphone to the target external loudspeaker.

In one possible design, the structure of the interaction device shown in fig. 10 may be implemented as an interaction device. As shown in fig. 11, the interactive apparatus 1100 may include: processor 1101, memory 1102. Wherein the memory 1102 has stored thereon executable code that, when executed by the processor 1101, at least causes the processor 1101 to implement the in-vehicle interaction method as provided in the embodiment of fig. 6 described above.

The interactive device may further include a communication interface 1103 in the structure for communicating with other devices.

The embodiment of the application also provides a vehicle. The vehicle comprises the vehicle-mounted interaction device or the vehicle-mounted interaction equipment provided by any embodiment of the application, and has corresponding functional modules and beneficial effects.

Fig. 12 is a schematic structural diagram of another vehicle provided in this embodiment, and as shown in fig. 12, the vehicle 1200 may include one or more of the following components: a processing component 1202, a memory 1204, a power component 1206, a multimedia component 1208, an audio component 1210, an input/output (I/O) interface 1212, a sensor component 1214, and a communications component 1216.

The processing component 1202 generally controls overall operation of the electronic device 1200, such as operations associated with display, telephone call, data communication, camera operation, and recording operation. The processing component 1202 may include one or more processors 1220 to execute instructions to perform all or part of the steps of methods S601-S605 described above. Further, the processing component 1202 may include one or more modules that facilitate interactions between the processing component 1202 and other components. For example, the processing component 1202 may include a multimedia module to facilitate interaction between the multimedia component 1208 and the processing component 1202.

The memory 1204 is configured to store various types of data to support operations at the electronic device 1200. Examples of such data include instructions for any application or method operating on the electronic device 1200, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 1204 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The power supply assembly 1206 provides power to the various components of the electronic device 1200. The power supply components 1206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 1200.

The multimedia component 1208 includes a screen between the electronic device 1200 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or sliding action, but also the duration and pressure associated with the touch or sliding operation. In some embodiments, the multimedia component 1208 includes a front camera and/or a rear camera. When the electronic device 1200 is in an operational mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 1210 is configured to output and/or input audio signals. For example, the audio component 1210 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 1200 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 1204 or transmitted via the communications component 1216. In some embodiments, audio assembly 1210 further includes a speaker for outputting audio signals.

The input/output interface 1212 provides an interface between the processing component 1202 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.

The sensor assembly 1214 includes one or more sensors for providing status assessment of various aspects of the electronic device 1200. For example, the sensor assembly 1214 may detect an on/off state of the electronic device 1200, a relative positioning of components such as a display and keypad of the electronic device 1200, a change in position of the electronic device 1200 or a component of the electronic device 1200, the presence or absence of a user's contact with the electronic device 1200, an orientation or acceleration/deceleration of the electronic device 1200, and a change in temperature of the electronic device 1200. The sensor assembly 1214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1214 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communications component 1216 is configured to facilitate communication between the electronic device 1200 and other devices, either wired or wireless. The electronic device 1200 may access a wireless network based on a communication standard, such as WiFi,2G or 3G or 4G, or a combination thereof. In one exemplary embodiment, the communication component 1216 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communications component 1216 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 1200 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.

In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, such as memory 1204, comprising instructions executable by processor 1220 of electronic device 1200 to perform the above-described method. For example, the non-transitory computer readable storage medium may be implemented by any type of volatile or non-volatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

Additionally, embodiments of the present application provide a non-transitory machine-readable storage medium having executable code stored thereon that, when executed by a processor of an electronic device, causes the processor to perform the in-vehicle interaction method provided in the embodiment of fig. 6 described above.

The apparatus embodiments described above are merely illustrative, wherein the various modules illustrated as separate components may or may not be physically separate. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by adding necessary general purpose hardware platforms, or may be implemented by a combination of hardware and software. Based on such understanding, the foregoing aspects, in essence, or portions thereof, may be embodied in the form of a computer program product, which may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting thereof; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.

Claims

1. A vehicle-mounted interactive system, comprising:

the controller is coupled with the display device and the plurality of external interaction devices and is used for receiving an instruction for selecting a target interaction object, determining the position information of the target interaction object relative to the vehicle, and determining target external interaction devices corresponding to the position information according to the setting positions corresponding to the plurality of external interaction devices;

and the target external interaction device is used for interacting with the target interaction object under the control of the controller.

2. The system of claim 1, wherein the plurality of external interaction devices comprises a plurality of external speakers.

3. The system of claim 2, wherein the plurality of external interaction devices further comprises a plurality of external microphones corresponding to the plurality of speaker placement locations.

4. The system of claim 2, further comprising a swivel mechanism for mounting the external speaker, the swivel mechanism comprising:

5. The system of claim 1, wherein the plurality of external interactive devices are respectively disposed on a cross member of the front side of the vehicle body, on side rails of the left and right sides of the vehicle body, and on an impact beam of the rear side of the vehicle body.

6. The system of claim 3, wherein the external speaker comprises a directional speaker and the external microphone comprises a directional microphone.

7. The system of claim 6, wherein the sound emission direction of the external speaker and the sound pickup direction of the external microphone are determined from the location information.

8. A system according to claim 3, wherein the system further comprises: a built-in interaction device comprising a built-in speaker and a built-in microphone;

9. The system of claim 1, further comprising:

an image generating device for generating the environment image outside the vehicle.

10. The system of claim 9, wherein the image generation device comprises at least one of: vehicle-mounted camera and imaging radar.

11. A vehicle-mounted interaction method, comprising:

determining a selected target interactive object in the environment image;

determining target external interaction equipment corresponding to the position information according to the setting positions corresponding to the external interaction equipment, wherein the external interaction equipment is arranged at different positions outside the vehicle body;

12. The method of claim 11, wherein the plurality of external interaction devices comprises a plurality of external speakers, the target external interaction device being a target external speaker; after the external target interaction device corresponding to the position information is determined, the method further comprises the following steps:

13. The method of claim 11, wherein the target external interaction device is a target external speaker, and the outputting, by the target external interaction device, interaction information corresponding to the target interaction object comprises:

14. The method of claim 11, wherein the target external interaction device comprises a target external speaker and a target external microphone corresponding to a setting position of the target external speaker, and wherein an internal speaker and an internal microphone are provided in the vehicle;

15. A vehicle-mounted interactive apparatus, comprising:

the third determining module is used for determining target external interaction equipment corresponding to the position information according to the setting positions corresponding to the external interaction equipment, and the external interaction equipment is arranged at different positions outside the vehicle body;

16. A vehicle-mounted interactive apparatus, characterized by comprising: a memory, a processor; wherein the memory has stored thereon executable code which, when executed by the processor, causes the processor to perform the in-vehicle interaction method of any of claims 7 to 10.

17. A vehicle comprising the interaction device of claim 15 or the interaction apparatus of claim 16.

18. A non-transitory machine-readable storage medium having stored thereon executable code, which when executed by a processor of an electronic device, causes the processor to perform the in-vehicle interaction method of any of claims 11-14.