US20160140840A1

US20160140840A1 - Pedestrian warning system

Info

Publication number: US20160140840A1
Application number: US15/007,301
Authority: US
Inventors: William D. Duncan; Roderick A. Hyde; Elizabeth E. Nugent; Robert C. Petroski; Yaroslav A. Urzhumov
Original assignee: Elwha LLC
Current assignee: Elwha LLC
Priority date: 2013-10-18
Filing date: 2016-01-27
Publication date: 2016-05-19
Also published as: US20150109149A1; US9286794B2

Abstract

A pedestrian warning system monitors an attention level of a pedestrian and alerts the pedestrian of traffic areas or detected vehicles.

Description

If an Application Data Sheet (ADS) has been filed on the filing date of this application, it is incorporated by reference herein. Any applications claimed on the ADS for priority under 35 U.S.C. §§ 119, 120, 121, or 365(c), and any and all parent, grandparent, great-grandparent, etc. applications of such applications, are also incorporated by reference, including any priority claims made in those applications and any material incorporated by reference, to the extent such subject matter is not inconsistent herewith.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of the earliest available effective filing date(s) from the following listed application(s) (the “Priority Applications”), if any, listed below (e.g., claims earliest available priority dates for other than provisional patent applications or claims benefits under 35 USC § 119(e) for provisional patent applications, for any and all parent, grandparent, great-grandparent, etc. applications of the Priority Application(s)).

Priority Applications:

The present application constitutes a continuation-in-part of U.S. patent application Ser. No. 14/057,732, entitled PEDESTRIAN WARNING SYSTEM, naming Jesse R. Cheatham, III, William D. Duncan, Roderick A. Hyde, Nathan P. Myhrvold, and Lowell L. Wood, Jr. as inventors, filed Oct. 18, 2013 with attorney docket no. 0713-035-001-000000.
If the listings of applications provided above are inconsistent with the listings provided via an ADS, it is the intent of the Applicant to claim priority to each application that appears in the Domestic Benefit/National Stage Information section of the ADS and to each application that appears in the Priority Applications section of this application.
All subject matter of the Priority Applications and of any and all applications related to the Priority Applications by priority claims (directly or indirectly), including any priority claims made and subject matter incorporated by reference therein as of the filing date of the instant application, is incorporated herein by reference to the extent such subject matter is not inconsistent herewith.

SUMMARY

In one aspect, a pedestrian warning device includes a computing device (e.g., a mobile telephone or wearable device) configured to be worn or carried by a pedestrian and to monitor an attention level of the pedestrian, and an alarm (e.g., an audible, visible, or haptic alarm) configured to warn the pedestrian of a vehicle detected by the computing device. The pedestrian device may include a camera or a microphone configured to detect the vehicle. The alarm may be configured to warn the pedestrian only if a predicted collision probability is above a threshold, or the attention level of the pedestrian is below a threshold. The alarm may be in response to the attention level of the pedestrian, or in a mode selected in response to the attention level of the pedestrian. The alarm may be in a mode selected in response to an activity type of the pedestrian (e.g., typing, talking, taking a picture, taking a video, having a video conference, playing a timed game, playing an untimed game, reading, watching a display, or using a display-based application). The alarm may be configured to warn the pedestrian in response to attention history, eye position, eye position history, head orientation, head orientation history, position or motion of the pedestrian, position or motion of the computing device, or alarm response history of the pedestrian. The pedestrian warning device may further include a GPS configured to identify a location of the pedestrian (e.g., to determine if the pedestrian is crossing a street at an intersection), and may monitor the gaze or head position of the pedestrian and warn the pedestrian if he attempts to cross without looking in a direction of oncoming traffic. The pedestrian warning device may further include a sensor configured to monitor a traffic signal, in which case the alarm may be configured to alert the pedestrian if he attempts to cross the street in contravention of the traffic signal. The pedestrian warning device may further include a memory, which may be configured to store a record of the pedestrian's actions, a record of vehicle detections, or a record of alarms issued.
In another aspect, a method of warning a pedestrian of a vehicle includes monitoring an attention level of the pedestrian by monitoring an electronic device carried or worn by the pedestrian (e.g., a mobile telephone or wearable device), detecting a vehicle (e.g., with a camera or microphone), and, in response to the detected vehicle and the monitored attention level, alerting the pedestrian to the presence of the vehicle (e.g., with a visual, audible, or haptic alarm). Alerting the pedestrian may include alerting the pedestrian in response to an attention level, attention history, eye position, eye position history, head orientation, head orientation history, alarm reponse history, position, or motion of the pedestrian, or in response to a position or motion of the device. Monitoring the attention level may include monitoring a device activity (e.g., typing, talking, taking a picture, taking a video, having a video conference, playing a timed game, playing an untimed game, reading, watching a display, or using a display-based application), in which case an alert mode may be selected in response to the device activity. The method may further include storing data in a memory, such as a record of the pedestrian's actions, a record of vehicle detections, or a record of pedestrian alerts.
In another aspect, a system for monitoring the behavior of a pedestrian includes a computing device configured to be worn or carried by a pedestrian, to monitor an attention level of the pedestrian, and to detect a vehicle (e.g., with a camera or microphone), and a memory configured to store information about the actions of the pedestrian. The system may further include a transmitter configured to transmit information about the actions of the pedestrian to a remote location, or a display configured to display information stored in the memory. The device may be configured to store a record of pedestrian activity (e.g., typing, talking, taking a picture, taking a video, having a video conference, playing a timed game, playing an untimed game, reading, watching a display, or using a display-based application). The system may furethr include a GPS configured to identify a location of the pedestrian, in which case the memory may be configured to store the identified location. The memory may be configured to store gaze direction, head position, device orientation, or a record of vehicle detections.
In another aspect, a method for monitoring the behavior of a pedestrian includes monitoring an attention level, a position, and a motion of the pedestrian, and alerting the pedestrian (e.g., with a visible, audible, or haptic alarm) if the pedestrian is determined to be in danger from a vehicle in response to the monitored data. Monitoring the position of the pedestrian may include determining the proximity of the pedestrian to a street (e.g., with a GPS, a camera, or a microphone). A GPS may monitor the position or motion of the pedestrian. Monitoring motion may include determining that the pedestrian is moving toward a street. Alerting the pedestrian may be in response to an attention level, attention history, eye position, eye position history, head orientation, head orientation history, device orientation, device orientation history, or alarm response history. Monitoring the attention level may include monitoring a device activity of the pedestrian (e.g., typing, talking, taking a picture, taking a video, having a video conference, playing a timed game, playing an untimed game, reading, watching a display, or using a display-based application), in which case it may further include selecting an alert mode in response to the device activity. The method may further include storing a record of vehicle detections or pedestrian alerts in a memory.
The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 shows a cellular telephone running an application for warning a pedestrian.

FIG. 2 shows a wearable device running an application that detects a vehicle and warns a pedestrian.

FIG. 3 is a flowchart illustrating the use of the applications shown in FIG. 1 and FIG. 2.

FIG. 4 is a flowchart illustrating another use of the applications shown in FIG. 1 and FIG. 2.

DETAILED DESCRIPTION

In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented here.
“Pedestrian,” as that term is used herein, includes a person that is not currently in a vehicle, or is using a “pedestrian vehicle” typically permitted on a sidewalk, such as a bicycle, wagon, skateboard, scooter, wheelchair (including motorized wheelchairs), or personal transport such as a SEGWAY™. Pedestrians may include adults or children. “Demographics” of a pedestrian may include age, sex, physical capabilities (or capabilities of a pedestrian vehicle in use), or other relevant information about the pedestrian. “Identifying information” of a pedestrian may include information such as name, address, guardian, next-of-kin, or the like.
“Garment,” as that term is used herein, includes clothing, apparel, or jewelry such as but not limited to shirts, coats, pants, shoes, hats, collars, bracelets, earrings, belts, or backpacks, and also objects designed to be carried by pedestrians, such as but not limited to briefcases, purses, suitcases, keyfobs, cellphones, or tablets.
This application makes reference to technologies described more fully in U.S. patent application Ser. Nos. 13/950,719, filed Jul. 25, 2013, and 14/011,275, filed Aug. 27, 2013, both entitled “Wearable Radar Retroreflectors.” Both of these applications are incorporated by reference herein.
Pedestrians must maintain awareness of their surroundings to avoid being struck by cars or other vehicles. This awareness is sometimes compromised by pedestrians' use of electronics. Mobile phones and texting can be particularly absorbing, as can various games available on “smart” phones and other devices. Furthermore, children, pets, and even adults caught by surprise may panic at the sight of an oncoming vehicle and may use counterproductive avoidance strategies like trying to outrun a car. A system that uses ingrained reflexes, especially when issued before the oncoming vehicle is seen (e.g., in a parent's voice), may produce more productive behavior.
FIG. 1 illustrates an implementation of a pedestrian warning system designed to alert a pedestrian who is distracted by texting. Telephone 302 includes a global positioning system (GPS) 304, that monitors the location of the pedestrian relative to the street. (In some embodiments, the phone may also use its camera 308 or its microphone 310 to detect a vehicle, and may optionally modify its warnings in response to such detection.) In the embodiment shown in FIG. 1, while the phone was being used for texting, a warning box 306 shows that the pedestrian is approaching an intersection. The illustrated phone 302 also includes a traffic light sensor 312, and the display indicates the light is red (“NO CROSSING”). (Traffic light sensors are described in U.S. Pat. No. 6,985,073, which is incorporated by reference herein to the extent not inconsistent herewith.) While element 312 is shown in FIG. 1 as the traditional symbol for a photodiode, which may be configured to directly “look at” the traffic signal and respond to its color, it may also be implemented as an antenna or other device configured to receive a broadcast signal from the traffic signal, or a microphone or similar device configured to interpret an audible signal from the traffic signal. The words “NO CROSSING” may change to “GREEN LIGHT” or a pedestrian walking symbol or any other appropriate symbol or message when the pedestrian is permitted to cross. In this or other embodiments, telephone 302 may also (or instead) emit an audible or haptic alarm. In some embodiments, the first alarm may be audible, for example, with warning box 306 appearing over the texter's screen only when it appears that he may be about to step into the intersection. In some embodiments, an audible alarm may be recorded in the voice of the pedestrian's parent or guardian, or another authority figure that the pedestrian is expected to reflexively obey.
In another embodiment, FIG. 2 shows an a wearable device 402 that is configured to detect a vehicle 404 and to warn the pedestrian of it. In the illustrated embodiment, the vehicle is detected via microphone 406 of device 402, which monitors for a characteristic engine noise. In other embodiments, the device may use a camera to watch for cars or other vehicles. These embodiments are expected to perform better at detecting electric vehicles, which may be relatively silent compared to conventional internal-combustion engines. However, makers of electric vehicles sometimes include audio recordings of engine noise to prevent pedestrians from being taken by surprise. In some embodiments, the device may listen specifically for such recordings. Apps may include sound libraries or other methods of recognizing such sounds, especially if, in the future, recorded or digitally-generated sounds become standard for such cars.
In the illustrated embodiment, microphone 406 is a stereo microphone that allows device 402 to identify the approximate direction of vehicle 404. In some embodiments, a monaural microphone may be used. Even though this does not typically permit the direction of the vehicle to be determined without additional hardware, device 402 is still able to warn the pedestrian of the vehicle. In some embodiments, the device's camera may supplement (e.g., looking for a car once it has been detected by the microphone) or replace the microphone for vehicle detection. The wearable device illustrated in FIG. 2 may also monitor a head position (or eye direction) of the pedestrian as further described below.
In some embodiments, device 402 (or phone 302) may include a memory configured to store information about the actions of the pedestrian. In such embodiments, the device may optionally also warn the pedestrian of approaching vehicles, but the memory may provide a record of the pedestrian's behavior in response to vehicles that may be reviewed by the pedestrian or his parent or guardian. This review may be used to facilitate behavior management systems to improve the safety of the pedestrian in the presence of traffic.
FIG. 3 is a flowchart 500 showing how the pedestrian warning system of FIG. 1 or FIG. 2 may be used. The system continuously (or intermittently) monitors 502 an attention level of a pedestrian, for example by monitoring the activity of the pedestrian on the device or by monitoring his head position or eye position (gaze direction) to determine whether he is looking for traffic. The system also detects 504 a vehicle, for example using a camera, microphone, or both. In some embodiments, the system may proceed directly to alerting the pedestrian, while in others, it first determines 508 whether to alert (or how to alert, as described below). For example, if the device has been monitoring head position, it may decide to alert only if the pedestrian has not turned his head in the direction of the vehicle recently. In response to the detected vehicle and the monitored attention level, the device alerts 506 the pedestrian to the vehicle. The form of the alert may be determined by the attention level or device activity. For example, the device may use a visual alert 510 when the pedestrian is looking at a handheld device (e.g., texting or watching a video), or an audio alert 512 or haptic alert 514 when the pedestrian is not looking at a handheld device (e.g., talking on the phone or wearing the device shown in FIG. 2). Multiple alerts may be given sequentially or simultaneously depending on the response to the first alert. The details of the alarms used and the conditions under which the device will alert the user may be user-defined, either by the pedestrian or by another user (e.g., a parent or guardian).
In some embodiments, step 506 may be omitted from the method illustrated in FIG. 3, and the device may be used simply to create a record of pedestrian (and optionally vehicle) behavior, as shown in FIG. 4. The device monitors 602 an attention level of the pedestrian and detects 604 a vehicle as described above. Optionally, it may issue an alert as described in connection with FIG. 3 (optional step not depicted in FIG. 4). It stores 606 a record of the behavior of the pedestrian around the vehicle in a memory (e.g., did the pedestrian look at the vehicle, did the pedestrian slow down or stop at an intersection, did the pedestrian pause in using the device as the vehicle approached). It may also store 608 a record of the behavior of the vehicle (e.g., did it stop, did it slow down, did it approach the pedestrian too closely). Such a record may be used, for example, to instruct the pedestrian about proper behavior in future encounters, or to provide a record in the event of a collision or near-miss between the vehicle and the pedestrian. In some embodiments, detection 604 of the vehicle may be omitted and instead (or additiontally) the device may use a GPS or other device to identify the position of the pedestrian and record his behavior in the vicinity of expected traffic positions such as streets.
Various embodiments of pedestrian warning devices and methods have been described herein. In general, features that have been described in connection with one particular embodiment may be used in other embodiments, unless context dictates otherwise. For example, the eye position monitoring described in connection with FIG. 2 may be employed in the phones described in connection with FIG. 1, or with any of the embodiments described herein. For the sake of brevity, descriptions of such features have not been repeated, but will be understood to be included in the different aspects and embodiments described herein.
It will be understood that, in general, terms used herein, and especially in the appended claims, are generally intended as “open” terms (e.g., the term “including” should be interpreted as “including but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “includes but is not limited to,” etc.). It will be further understood that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of introductory phrases such as “at least one” or “one or more” to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles “a” or “an” limits any particular claim containing such introduced claim recitation to inventions containing only one such recitation, even when the same claim includes the introductory phrases “one or more” or “at least one” and indefinite articles such as “a” or “an” (e.g., “a transmitter” should typically be interpreted to mean “at least one transmitter”); the same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, it will be recognized that such recitation should typically be interpreted to mean at least the recited number (e.g., the bare recitation of “two alarms,” or “a plurality of alarms,” without other modifiers, typically means at least two alarms). Furthermore, in those instances where a phrase such as “at least one of A, B, and C,” “at least one of A, B, or C,” or “an [item] selected from the group consisting of A, B, and C,” is used, in general such a construction is intended to be disjunctive (e.g., any of these phrases would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together, and may further include more than one of A, B, or C, such as A₁, A₂, and C together, A, B₁, B₂, C₁, and C₂together, or B₁and B₂together). It will be further understood that virtually any disjunctive word or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase “A or B” will be understood to include the possibilities of “A” or “B” or “A and B.” While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Claims

What is claimed is:

1. A pedestrian warning device, comprising:

a computing device configured to be worn or carried by a pedestrian and to monitor an attention level of the pedestrian; and

an alarm configured to warn the pedestrian of a vehicle detected by the computing device, wherein the alarm is configured to warn the pedestrian only if the attention level is determined to be below a threshold.

2. The pedestrian warning device of claim 1, wherein the pedestrian warning device includes a camera configured to detect the vehicle.

3. The pedestrian warning device of claim 1, wherein the pedestrian warning device includes a microphone configured to detect the vehicle

4. The pedestrian warning device of claim 1, wherein the alarm is configured to warn the pedestrian only if a predicted collision probability is determined to be above a threshold.

5. (canceled)

6. (canceled)

7. The pedestrian warning device of claim 1, wherein the alarm is configured to warn the pedestrian in a mode selected in response to the attention level of the pedestrian.

8. The pedestrian warning device of claim 1, wherein the alarm is configured to warn the pedestrian in a mode selected in response to an activity type of the pedestrian.

9. The pedestrian warning device of claim 8, wherein the activity type is selected from the group consisting of typing, talking, taking a picture, taking a video, having a video conference, playing a timed game, playing an untimed game, reading, watching a display, and using a display-based application.

10. The pedestrian warning device of claim 1, wherein the alarm is configured to warn the pedestrian in response to an attention history of the pedestrian.

11. The pedestrian warning device of claim 1, wherein the alarm is configured to warn the pedestrian in response to an eye position of the pedestrian.

12. The pedestrian warning device of claim 1, wherein the alarm is configured to warn the pedestrian in response to an eye position history of the pedestrian.

13. The pedestrian warning device of claim 1, wherein the alarm is configured to warn the pedestrian in response to a head orientation of the pedestrian.

14. The pedestrian warning device of claim 1, wherein the alarm is configured to warn the pedestrian in response to a head orientation history of the pedestrian.

15. The pedestrian warning device of claim 1, wherein the alarm is configured to warn the pedestrian in response to a position of the pedestrian.

16. The pedestrian warning device of claim 1, wherein the alarm is configured to warn the pedestrian in response to a motion of the pedestrian.

17. The pedestrian warning device of claim 1, wherein the alarm is configured to warn the pedestrian in response to a position of the computing device.

18. The pedestrian warning device of claim 1, wherein the alarm is configured to warn the pedestrian in response to an orientation of the computing device.

19. The pedestrian warning device of claim 1, wherein the alarm is configured to warn the pedestrian in response to an alarm response history of the pedestrian.

20. The pedestrian warning device of claim 1, wherein the computing device is a mobile telephone.

21. The pedestrian warning device of claim 1, wherein the computing device is a wearable device.

22. The pedestrian warning device of claim 1, further comprising a GPS configured to identify a location of the pedestrian.

23. The pedestrian warning device of claim 22, wherein the GPS is configured to identify whether the pedestrian is crossing a street at an intersection.

24. The pedestrian warning device of claim 22, wherein the pedestrian warning device is configured to monitor the gaze of the pedestrian, and the alarm is configured to warn the pedestrian in response to an attempt to cross a street without looking in a direction of oncoming traffic.

25. The pedestrian warning device of claim 22, wherein the pedestrian warning device is configured to monitor the head position of the pedestrian, and the alarm is configured to warn the pedestrian in response to an attempt to cross a street without looking in a direction of oncoming traffic.

26. The pedestrian warning device of claim 1, wherein the pedestrian warning device is configured to monitor the gaze of the pedestrian, and the alarm is configured to warn the pedestrian in response to an attempt to cross a street without looking in a direction of oncoming traffic.

27. The pedestrian warning device of claim 1, wherein the pedestrian warning device is configured to monitor the head position of the pedestrian, and the alarm is configured to warn the pedestrian in response to an attempt to cross a street without looking in a direction of oncoming traffic.

28. The pedestrian warning device of claim 1, further comprising a sensor configured to monitor a traffic signal.

29. The pedestrian warning device of claim 28, wherein the alarm is configured to warn the pedestrian in response to an attempt to cross a street in contravention of the traffic signal.

30. The pedestrian warning device of claim 1, wherein the alarm is audible.

31. The pedestrian warning device of claim 1, wherein the alarm is visible.

32. The pedestrian warning device of claim 1, wherein the alarm is haptic.

33. The pedestrian warning device of claim 1, further comprising a memory.

34. The pedestrian warning device of claim 33, wherein the memory is configured to store a record of the pedestrian's actions.

35. The pedestrian warning device of claim 33, wherein the memory is configured to store a record of vehicle detections.

36. The pedestrian warning device of claim 33, wherein the memory is configured to store a record of alarms issued.

37. A method of warning a pedestrian of a vehicle, comprising:

monitoring an attention level of the pedestrian by monitoring an electronic device carried or worn by the pedestrian;

detecting a vehicle; and

in response to the detected vehicle and the monitored attention level, alerting the pedestrian to the presence of the vehicle if the monitored attention level is below a threshold.

38-49. (canceled)

50. The method of claim 37, wherein monitoring the attention level includes monitoring a device activity of the pedestrian.

51. The method of claim 50, wherein alerting the pedestrian includes selecting an alert mode in response to the device activity of the pedestrian.

52. The method of claim 51, wherein the device activity is selected from the group consisting of typing, talking, taking a picture, taking a video, having a video conference, playing a timed game, playing an untimed game, reading, watching a display, and using a display-based application.

53-62. (canceled)

63. A system for monitoring the behavior of a pedestrian, comprising:

a computing device configured:

to be worn or carried by a pedestrian;

to monitor an attention level of the pedestrian; and

to detect a vehicle; and

a memory configured to store information about the actions of the pedestrian when a vehicle is detected.

64-100. (canceled)