CN104102007A - Head-mounted display and control method thereof - Google Patents

Abstract

The invention provides a head-mounted display and a control method thereof. The head mounted display includes an image acquisition unit, a storage unit, a projection display unit, and a processor. The image acquisition unit is used for shooting a viewed picture viewed by a user. The storage unit is used for storing the viewed pictures. The processor judges whether the user is in the driving environment or the parking environment in the vehicle according to the viewed picture. When the processor detects that the user is in the driving environment in the vehicle, the head-mounted display is switched to be in the driving mode, and whether a moving object exists in the viewed picture is judged. If yes, the processor controls the projection display unit to project the reminding signal. When the processor detects that the user is in the parking environment, the head-mounted display is switched to be in the parking mode, and the image acquisition unit is controlled to record parking information.

Description

Head-mounted display and control method thereof

technical field

The present invention relates to a head-mounted display, and in particular to a head-mounted display applicable to driving and parking environments and a control method thereof.

Background technique

As the size of the head-mounted display (HMD) becomes smaller and smaller and the display viewing angle becomes larger and larger, the head-mounted display also adds many derivative application functions. But the application of the existing head-mounted display is only provided for the user to use when standing still or walking. For example, the head-mounted display "Google glasses" proposed by the existing Google Company has its product orientation preset as a replacement for a smart phone, providing users with the ability to display emails, incoming call information or time information when walking normally, but At present, there is no application of head-mounted display in the driving environment.

Contents of the invention

The invention provides a head-mounted display and a control method thereof. The head-mounted display is applied to a driving environment and can be used as a driving assistance device and a parking assistance tool.

The head-mounted display of the present invention is suitable to be worn on the user's head, and it includes an image acquisition unit, a storage unit, a projection display unit and a processor. Wherein, the image acquisition unit is used for photographing the viewed picture watched by the user. The storage unit is used for storing the viewed pictures. The processor is coupled to the image acquisition unit, the storage unit and the projection display unit. The processor judges that the user is in the driving environment or the parking environment in the car according to the viewed picture. Wherein, when the processor detects that the user is in the driving environment in the vehicle, the head-mounted display is switched to the driving mode. Moreover, the processor judges whether there is a moving object in the viewed picture, and if so, the processor controls the projection display unit to project a reminder signal. Wherein, when the processor detects that the user is in a parking environment, the head-mounted display is switched to a parking mode. and. The processor controls the image acquisition unit to record at least one piece of parking information.

In an embodiment of the present invention, the above-mentioned processor performs image recognition processing on the viewed image to determine whether there is at least one preset object in the viewed image, and when the at least one preset object exists in the viewed image, the processor It is determined that the user is in the driving environment inside the vehicle.

In an embodiment of the present invention, the aforementioned head-mounted display further includes a positioning unit coupled to the processor. Wherein when the head-mounted display is in the driving mode, the processor performs image recognition processing on the viewed picture to judge the current road conditions, and automatically update one or more navigation parameters of the positioning unit accordingly.

In an embodiment of the present invention, the aforementioned head-mounted display further includes a touch input unit coupled to the processor. When the head-mounted display is in the parking mode and the touch input unit receives the user's touch signal, the processor activates the image acquisition unit and the positioning unit to record the user's walking route.

In an embodiment of the present invention, the aforementioned head-mounted display further includes a communication unit coupled to the storage unit and the processor. Wherein when the head-mounted display is in the driving mode, the communication unit is connected to the vehicle positioning unit through a wired or wireless method to receive instant navigation information. And, the processor controls the projection display unit to project and display the real-time navigation information.

In an embodiment of the present invention, the above-mentioned communication unit is connected to one or more vehicle image sensors in a wired or wireless manner, so as to receive images captured by the vehicle image sensors and store them in the storage unit. The processor controls the projection display unit to project and display a picture of one of the vehicle image sensors.

In an embodiment of the present invention, the aforementioned head-mounted display further includes an acceleration sensor coupled to the processor. The acceleration sensor is used to detect a plurality of acceleration changes to generate a plurality of acceleration change signals, so that the processor can determine whether the user performs a parking action, a braking action or a turning action according to the acceleration change signals.

In an embodiment of the present invention, when the processor determines that at least one of the acceleration change signals is greater than a speed change threshold, the processor controls the projection display unit to reduce the frequency of projecting the reminder signal.

In an embodiment of the present invention, when the head-mounted display is in the parking mode, the processor controls the image acquisition unit to perform horizontal panoramic shooting to generate a panoramic image, and the processor judges that the parking environment in which the user is located is an indoor scene. or outdoor scenes, and record at least one of parking space numbers, adjacent license plate information, or markers.

In an embodiment of the present invention, the above-mentioned head-mounted display further includes a voice recorder, which is coupled to the processor and the storage unit, and is used for recording or playing the user's voice description data for the parking information.

A head-mounted display control method of the present invention is suitable for a head-mounted display worn on a user's head, and includes the following steps. At least one viewed image viewed by the user is captured by the head-mounted display to determine whether the user is in a driving environment or a parking environment in the vehicle. When the user is in the driving environment in the car, the head-mounted display is switched to the driving mode. And, it is judged whether there is a moving object in the viewed picture, and if so, the head-mounted display is controlled to project a reminder signal. When the user is in a parking environment, switch the head-mounted display to a parking mode. And, the head-mounted display is controlled to record at least one parking information.

Based on the above, the head-mounted display and its control method provided by the present invention can be used as a driving assistance device for the user during driving to improve driving safety; and can assist the user to record various parking information to help the user quickly Find where the car is parked.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail with reference to the accompanying drawings.

Description of drawings

FIG. 1 is a block diagram of a head-mounted display according to an embodiment of the present invention;

FIG. 2 is a flow chart of a control method for a head-mounted display according to an embodiment of the present invention;

Fig. 3 is a schematic diagram of a viewed picture according to an embodiment of the present invention;

Fig. 4 is a schematic diagram of an application scenario of a parking mode according to an embodiment of the present invention;

Fig. 5 is a block diagram of a head-mounted display according to another embodiment of the present invention;

Fig. 6 is a schematic diagram of an application scenario of another parking mode according to another embodiment of the present invention.

Explanation of reference signs:

01~14: parking space number;

100, 500: Head-mounted display;

110: image acquisition unit;

120: storage unit;

130: projection display unit;

140: processor;

301: steering wheel;

302: Dashboard;

303: front windshield;

304, 401: arrows;

550: positioning unit;

560: communication unit;

570: acceleration sensor;

580: voice recorder;

590: touch input unit;

601: route;

CP: current parking position;

U: user;

S201-S211: each step of the control method of the head-mounted display.

Detailed ways

FIG. 1 is a block diagram of a head-mounted display according to an embodiment of the present invention. The head-mounted display 100 of this embodiment is suitable for wearing on the user's head, so that the user's eyes can receive the image information provided by the head-mounted display 100 or view the image information that the user sees through the head-mounted display 100. The received images are captured and recorded. Referring to FIG. 1 , the head mounted display 100 includes an image acquisition unit 110 , a storage unit 120 , a projection display unit 130 and a processor 140 . Its functions are described as follows:

The image acquisition unit 110 includes a lens, a charge-coupled device (Charge-coupled Device, CCD) or a complementary metal-oxide-semiconductor (Complementary Metal Oxide Semiconductor, CMOS) photosensitive element, etc., and can be used to capture images viewed by the user through the head-mounted display 100 The image scene received by the user is hereinafter referred to as the "viewed picture" of the user.

The storage unit 120 is, for example, any form of fixed or removable random access memory (Random Access Memory, RAM), read-only memory (Read-Only Memory, ROM), flash memory (Flash memory), hard disk, or other Similar devices or a combination of these devices can be used to store viewed images or other data.

The projection display unit 130 may include lenses, reflection elements, refraction elements, etc., and may be used for adjusting the focus. By wearing the head-mounted display 100 , the user can watch the real image of the surrounding environment and the virtual image displayed by the projection display unit 110 at the same time.

The processor 140 is coupled to the image acquisition unit 110, the storage unit 120 and the projection display unit 130, and it is, for example, a central processing unit (Central Processing Unit, CPU), or other programmable general purpose or special purpose microprocessor (Microprocessor ), application specific integrated circuit (Application Specific Integrated Circuit, ASIC) and other hardware devices with computing functions, or a combination of the above hardware devices.

Fig. 2 is a flow chart of a control method for a head-mounted display according to an embodiment of the present invention. The method of this embodiment is applicable to the head-mounted display 100 of FIG. 1 , and the detailed steps of the method of this embodiment are described below with each component in the head-mounted display 100:

Firstly, in step S201 , the image capture unit 110 of the head mounted display 100 captures one or more viewed frames viewed by the user, and stores the viewed frames in the storage unit 120 . Wherein, the image acquisition unit 110 can be set to shoot continuously, or fixedly capture the images viewed by the user according to a preset frequency, which is not limited here.

Next, in step S203 , the processor 140 determines that the user is in a driving environment or a parking environment in a car according to the above-mentioned viewed images. In detail, the processor 140 first performs image recognition processing on the viewed frame to determine whether there is at least one preset object in the viewed frame. In one embodiment, the preset object is, for example, a steering wheel or an instrument panel, etc., and the preset object can be set according to actual application conditions, which is not limited here. When there are preset objects such as a steering wheel or an instrument panel in the viewed image, the processor 140 can determine that the user is in a driving environment inside the vehicle. For example, FIG. 3 is a schematic diagram of a displayed frame according to an embodiment of the present invention. Referring to FIG. 3 , the viewed image 300 includes preset objects such as a steering wheel 301 , an instrument panel 302 , and a front windshield 303 , so it can be known that the user is in a driving environment inside the car.

When the processor 140 determines that the user is in the driving environment in the car according to the viewed image, then the processor 140 switches the head-mounted display 100 to the driving mode in step S205 . In step S207, in the driving mode, the processor 140 further determines whether there is a moving object in the viewed frame. For example, the traveling direction of the moving object is a moving object that is different from the traveling direction of the car or a horizontal moving object (shown by arrow 304 in FIG. 3 ). It should be noted that the present invention does not limit the traveling direction or source direction of the moving object. Moreover, the moving object is, for example, a car, a person, or other objects, and is not limited to the above. In a broad sense, any object that can change the viewed picture can be regarded as a moving object in the present invention. If the determination in step S207 is yes, the processor 140 controls the projection display unit 130 to project and display the reminder signal. In this way, the head-mounted display 100 can assist in reminding the user to identify the vehicle situation or emergency ahead in the driving mode, thereby improving driving safety.

Going back to step S203 of FIG. 2 , if the processor 140 performs image recognition processing according to the viewed image and the preset object does not appear, it can be determined that the user is in a parking environment. Following step S209, the processor 140 switches the head-mounted display 100 to the parking mode. In step S211, in the parking mode, the processor 140 controls the image acquisition unit 110 to record at least one piece of parking information. In detail, the processor 140 controls the image acquisition unit 110 to perform horizontal panoramic shooting to generate a panoramic image, based on which the processor determines whether the parking environment of the user is an indoor scene or an outdoor scene, and records the parking space number and adjacent license plate information or at least one of markers, etc., as parking information. For example, FIG. 4 is a schematic diagram of an application scenario of a parking mode according to an embodiment of the present invention. For example, when the user is facing the current parking position CP, the processor 140 can control the image acquisition unit 110 to take horizontal panoramic shots from left to right according to the arrow direction 401 to generate a panoramic image. In this way, after the processor 140 performs image recognition on the panoramic image, it can obtain the parking space number of the current parking position CP (the number “04” shown in FIG. 4 ) and other adjacent parking space number information, adjacent License plate information or markers, etc. The sign is, for example, a section number (not shown in FIG. 4 ) on a pole of the parking lot. In this way, when the user returns to the parking lot again, the head-mounted display 100 can use the projection display unit 130 to provide the previously recorded parking information to the user in the form of a projection display, so that the user can quickly find the parking place of the car. Location.

In order to make the content of the present invention more clear, another embodiment is given below as an example in which the present invention can indeed be implemented. FIG. 5 is a block diagram of a head-mounted display according to another embodiment of the present invention. Please refer to FIG. 5 , in addition to the image acquisition unit 110 , the storage unit 120 , the projection display unit 130 and the processor 140 , the head-mounted display 500 of this embodiment may also include a positioning unit 550 coupled to the processor 140 , a communication Unit 560 , acceleration sensor 570 , voice recorder 580 and touch input unit 590 .

The positioning unit 550 is, for example, a Global Positioning System (Global Positioning System, GPS) or an Assisted Global Positioning System (Assisted Global Positioning System, AGPS), so that it can receive satellite positioning signals at any time for positioning, and cooperate with a geographic information database (such as an electronic map) ) or a geographic information system (Geography Information System, GIS), can provide detailed geographic information to the user wearing the head-mounted display 500.

The communication unit 560 can connect the head-mounted display 500 with the vehicle positioning unit or the vehicle image sensor of the car via wired and/or wireless ways, and can perform bidirectional transmission. Wherein the wired mode is for example Universal Serial Bus (Universal Serial Bus, USB) transmission interface etc.;

The acceleration sensor 570 is, for example, a gravity sensor (G-sensor) or an angular velocity sensor, and can be used to detect the acceleration variation of the head-mounted display 500 to generate an acceleration variation signal. The voice recorder 580 can be used to record the user's voice description data for the parking information, and can write the voice description data into the storage unit 120 or read the voice description data from the storage unit 120 for playback. The touch input unit 590 receives a user's touch operation signal, and the processor 140 can activate or deactivate the operation of components such as the image acquisition unit 110 , the positioning unit 550 , or the voice recorder 580 according to the touch operation signal.

In the following, various application situations of the head-mounted display 500 in the driving mode and the parking mode will be described in detail by enumerating embodiments according to each component of the head-mounted display 500 .

First of all, when the head-mounted display 500 is set in the driving mode, the processor 140 can not only determine whether there is a moving object in the viewed image according to the image captured by the image acquisition unit 110 and remind the user to pay attention, but also A reminder signal is generated according to a change in GPS data (calculated by the positioning unit 550 ) or a change in acceleration data (calculated by the acceleration sensor 570 ). In another embodiment, the processor 140 can also use pedestrian detection technology to determine whether the relative distance between the pedestrian and the user (that is, the driver) is sufficient, detect sign lights, street signs, etc.; There are lane departures. The processor 140 can present the above-mentioned various driving safety judgment results through the projection display unit 130 one by one. However, it should be noted that the processor 140 can omit non-emergency information (such as time reminder information, email information, etc.) during the period when the driver needs to concentrate on driving (such as when the sign light is green).

In an embodiment, the processor 140 can also use the positioning unit 550 to obtain the current driving speed for calculation, so as to determine whether the safety distance from the preceding vehicle is sufficient. The processor 140 can also identify the current road condition with the help of the image recognition result of the viewed frame, so as to automatically update one or more navigation parameters of the positioning unit 550 . For example, the picture captured by the image acquisition unit 110 can be used to determine whether the current road condition is blocked, whether the road is repaired, whether it is a one-way street, etc., and can allow the positioning unit 550 to update its navigation parameters (avoid the blockage or maintain the road). , one-way street, etc.) to plan a new route in real time, and perform real-time navigation through the projection display unit 130 .

In another embodiment, if the head-mounted display 500 itself does not have a built-in positioning unit 550, it can also be connected to a vehicle positioning unit (such as a vehicle navigation system) through the communication unit 560 to receive real-time navigation information. In addition, the head-mounted display 500 can also use the communication unit 560 to connect to one or more vehicle image sensors by wire or wirelessly, so as to receive images captured by the vehicle image sensors and store them in the storage unit 120 . The processor 140 can control the projection display unit 130 to project an image of one of the vehicle image sensors. For example, when the user needs to reverse the car or confirm the blind spots behind the car, the processor 140 can directly control the projection display unit 130 to switch to the image captured by the image sensor behind the car. In this way, the user does not need to go back and check with his eyes, which can improve driving safety.

On the other hand, the head-mounted display 500 can use the acceleration sensor 570 to detect multiple acceleration variations to generate multiple acceleration variation signals. The processor 140 can be used to determine whether the user performs a parking action, a braking action, or a turning action, etc. according to these acceleration change signals. When the processor 140 determines that the acceleration change signal is greater than the speed change threshold, the processor 140 may control the projection display unit 130 to reduce the frequency of projecting the reminder signal, so as to reduce the driving fatigue of the user. Wherein, the speed change threshold can be set in advance by those skilled in the art, and there is no limitation here.

In other words, the head-mounted display 500 of this embodiment can use the acceleration sensor 570 to collect the user's driving habits, such as whether the user often stops and brakes, starts unstable, or turns too fast, etc., and then automatically generates information adjustments. Mechanism (whether to display the reminder signal, the display frequency of the reminder signal, etc.) to match the driving habits of individual users.

When the processor 140 of the head-mounted display 500 determines through the image acquisition unit 110 and the acceleration sensor 570 that the user parks or gets off the vehicle, the head-mounted display 500 is set in the parking mode.

In the parking mode, the processor 140 controls the image acquisition unit 110 to perform horizontal panoramic shooting to generate a panoramic image. The processor 140 comprehensively determines whether the current parking environment is an outdoor scene or an indoor scene by judging the brightness or color temperature of the panoramic image. For example, if the brightness of the panoramic image is dark and the color temperature is low, it is an indoor scene; otherwise, it is an outdoor scene. If it is an outdoor scene, the positioning unit 550 can be used to record the latitude and longitude information, or the image acquisition unit 110 can be used to photograph street signs, intersection locations or nearby building markers. If it is an indoor scene, the image acquisition unit 110 can also be used to capture at least one of the parking space number, adjacent license plate information, or markers. In addition, the user can also activate the image acquisition unit 110 and the positioning unit 550 through the touch input unit 590 to record the user's walking route from the parking place to the entrance and exit of the parking lot. In another embodiment, the user can also use the voice recorder 580 to record the user's voice description data for the parking information.

In this way, when the user returns to the parking lot again, he can display the previously recorded various parking information through the head-mounted display 500, or play the voice description data of the parking information through the voice recorder 580, so as to quickly assist the use to find a parking spot. Fig. 6 is a schematic diagram of an application scenario of another parking mode according to another embodiment of the present invention. Please refer to FIG. 4 and FIG. 6 in conjunction with FIG. 6 . Since the user U has taken a horizontal panoramic shot when he parked previously, he can obtain the number information of his own and other adjacent parking spaces, the adjacent license plate information, and the like. Therefore, when the user U recognizes the number "07" using the head-mounted display 500, the head-mounted display 500 can generate a prompt signal to inform the user that his car is parked in the row of number "07" instead of the number "07". This row numbered "14". In another embodiment, the head-mounted display 500 can also use the previously recorded walking route of the positioning unit 550 to project and display the route for the user to find the parking location (such as the route 601 in FIG. 6 ). Finally, it should be noted that the implementation of the present invention is not limited to the above description, and the content of the above embodiments can be appropriately changed according to actual needs.

To sum up, the head-mounted display and its control method provided by the present invention can set the head-mounted display in the driving mode or the parking mode by performing image recognition on the viewed image. In the driving mode, the head-mounted display can help the user identify the vehicle situation ahead, remind emergency events, or perform real-time navigation, etc., thereby improving driving safety. In the parking mode, the head-mounted display can use images, positioning or voice to record various parking information, so that when the user returns to the parking lot again, he can easily and quickly find the parking position of the car. Furthermore, the head-mounted display can automatically generate an information adjustment mechanism to match the driving habits of individual users.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.

Claims (11)

1. A head-mounted display suitable for being worn on a user's head, characterized in that it comprises: an image acquisition unit, configured to capture at least one viewed image viewed by the user; a storage unit for storing the viewed pictures; a projection display unit; and A processor, coupled to the image acquisition unit, the storage unit and the projection display unit, judges that the user is in a driving environment or a parking environment according to the viewed images, Wherein when the processor detects that the user is in the driving environment in the car, switch the head-mounted display to a driving mode, and the processor judges whether there is a moving object in the viewed images, if so, the processor controls The projection display unit projects a reminder signal, Wherein when the processor detects that the user is in the parking environment, the head-mounted display is switched to a parking mode, and the processor controls the image acquisition unit to record at least one parking information. 2. The head-mounted display according to claim 1, characterized in that: The processor performs an image recognition process on the viewed frames to determine whether there is at least one preset object in the viewed frames, and when the at least one preset object exists in the viewed frames, the processor determines the The user is in the driving environment in the vehicle. 3. The head-mounted display according to claim 1, wherein the head-mounted display further comprises: A positioning unit, coupled to the processor, when the head-mounted display is in the driving mode, the processor performs an image recognition process on the viewed images to judge a current road condition, so as to automatically update the position of the positioning unit At least one navigation parameter. 4. The head-mounted display according to claim 3, wherein the head-mounted display further comprises: a touch input unit coupled to the processor, when the head-mounted display is in the parking mode and the touch input unit receives a touch signal from the user, the processor activates the image acquisition unit and the The positioning unit is used to record a walking route of the user. 5. The head-mounted display according to claim 1, wherein the head-mounted display further comprises: A communication unit, coupled between the storage unit and the processor, when the head-mounted display is in the driving mode, the communication unit is connected to a vehicle positioning unit through a wired method or a wireless method to receive A real-time navigation information, and the processor controls the projection display unit to project and display the real-time navigation information. 6. The head-mounted display according to claim 5, characterized in that: The communication unit is connected to at least one vehicle image sensor through the wired method or the wireless method, so as to receive the images captured by the vehicle image sensors and store them in the storage unit, and the processor controls the projection display unit to project and display the images A picture of one of the image sensors used in vehicles. 7. The head-mounted display according to claim 1, further comprising: An acceleration sensor, coupled to the processor, is used to detect a plurality of acceleration changes to generate a plurality of acceleration change signals, so that the processor can judge whether the user performs a parking action, a braking action or a turn according to the acceleration change signals action. 8. The head-mounted display according to claim 7, characterized in that: When the processor determines that at least one of the acceleration change signals is greater than a speed change threshold, the processor controls the projection display unit to reduce the frequency of projecting the reminder signal. 9. The head-mounted display according to claim 1, wherein when the head-mounted display is in the parking mode, the processor controls the image acquisition unit to perform horizontal panoramic shooting to generate a panoramic image for judging The parking environment where the user is located is an indoor scene or an outdoor scene, and at least one of parking space number, adjacent license plate information or marker is recorded. 10. The head-mounted display according to claim 1, further comprising: A voice recorder, coupled between the processor and the storage unit, is used to record or play the user's voice description data for the at least one parking information. 11. A method for controlling a head-mounted display, suitable for a head-mounted display worn on a user's head, characterized in that it comprises: Shooting at least one image viewed by the user through the head-mounted display to determine whether the user is in a driving environment or a parking environment; When the user is in the driving environment in the car, switch the head-mounted display to a driving mode, and determine whether there are moving objects in the viewed images, and if so, control the head-mounted display to project a reminder signal; and When the user is in the parking environment, switch the head-mounted display to a parking mode, and control the head-mounted display to record at least one parking information.