EP3105696A1

EP3105696A1 - Entryway based authentication system

Info

Publication number: EP3105696A1
Application number: EP15746074.2A
Authority: EP
Inventors: Steve COUSINS; Jonathan BINNEY; Chad ROCKEY; Adrian CANOSO
Original assignee: Savioke Inc
Current assignee: Savioke Inc
Priority date: 2014-02-10
Filing date: 2015-02-10
Publication date: 2016-12-21
Also published as: EP3105696A4

Abstract

A system can include at least one robot configured to autonomously navigate from a first location to a destination zone, the at least one robot having a generally cylindrical shape and including a storage container for storing at least one item for delivery, at least one image based depth sensor mounted proximate a top end of the robot, and a controller configured to navigate to the destination zone and to authenticate a delivery target for the item.

Description

Entryway Based Authentication System

TECHN ICAL FIELD

The present disclosure relates generally to authentication of persons and/or objects at predetermined locations for a mobile delivery robot, including authentication of persons at particular locales, such as doorways and/or entryways.

BACKGROUND

Facilitating delivery or receipt of packages using mobile autonomous robots can improve service to residents and staff in hospitality spaces including hotels, nursing homes, hospitals, residential apartment buildings, or office buildings. For example, a hotel guest can order room service, and an autonomous robotic delivery service could deliver the guest's order directly to their door. After authentication of the delivery recipient, the package can be made available to the hotel guest.

Authentication typically requires that a guest take deliberate actions to identify themselves. For example, a recipient can sign for their delivery, use a provided RFID dongle or key card, or enter a passcode to confirm identity and permit package delivery. In a hotel-like environment, a recipient could swipe their room key through a card reader in order to authenticate. Alternatively, a prospective hotel room resident can provide biometric data such as face image, fingerprint, or voice samples during check-in a biometric. All of these mechanisms require the recipient to take a deliberate action in order to authenticate themselves, may be considered by many to be an unwarranted invasion of privacy, or may require unconventional user actions that are difficult to learn or perform.

BRIEF DESCRIPTION OF THE DRAWINGS FIGS. 1A to 1 D are diagrams showing the authentication and delivering of an item to a person according to various embodiments;

FIGS. 2A to 2F various views of robots according to embodiments.

FIG. 3 is a cartoon illustrating a depth sensor mounted on a robot maneuvering in an environment.

FIG. 4 is a diagram showing one example of a robot having multiple depth sensors mounted on a rotatable, generally cylindrical robot, according to an embodiment.

FIG. 5A and 5B are diagrams showing fields of view for sensors of a robot according to embodiments. FIG. 6 is a perspective view of a robot according to one particular embodiment.

FIG. 7 is a flow diagram of a robot operation according to an embodiment.

FIG. 8 is a flow diagram of a robot operation according to another embodiment.

FIG. 9 is a flow diagram of a robot operation according to a further embodiment.

FIGS. 10A to 10C are a series of views showing robot operations according to an embodiment.

FIG. 1 1 is a flow diagram of a robot operation according to a particular embodiment.

DETAILED DESCRIPTION

Embodiments can include systems, methods and devices for the authentication of designated persons at particular locations. In some embodiments, systems include robots with low cost sensor systems and can interact with entryways or geographic locales. In certain embodiments, sensor systems can be configured to assist in authenticating people or areas such as an open doorway or entryway threshold prior to delivering items.

FIG. 1 A shows a series of panels illustrating methods and systems according to particular embodiments. In the embodiment shown, the system can execute a method for authenticating and delivering an item to a person.

A method 101 can include a person, located in a predetermined area, ordering one or more items. As seen in a first panel 103, in particular embodiments, this can include a hotel guest ordering an item such as food, drinks, snacks, toiletries, magazines, books, or other miscellaneous items. Ordering can be accomplished using any suitable method, including but not limited to using a mobile applet for a tablet or phone, a web page interaction, a television cable interface, or by simply calling a concierge or other hotel employee.

A method 101 can further include an order being received and the particular item(s) placed in a securable storage container of a robot. Such actions can be executed with a purely automated system, or in part with the assistance of people. In the embodiment shown, as seen in a second panel 105, an order can be received by a person (e.g., concierge) at a location within a hotel, the desired item(s) 107 can be found (e.g., in a hotel store or stockroom), and then placed into a top accessible, securable container 116 of a mobile robot 100. Once item(s) are placed in a securable storage container of the robot (e.g., 1 16), the container can be secured (e.g., closed and locked). Such a closing/locking can be an automated operation, one executed by a person, or combinations thereof.

A method 101 can further include a robot, which securely contains delivery items, being directed to a delivery location. In some embodiments, such an action can include a robot 100 receiving destination information by any suitable means and being directed to a room corresponding to the order (the designated room). In some embodiments, this can also be an automated step, with the designated room information being sent to the robot 100 via a wired or wireless connection. In a particular embodiment, a person (e.g., concierge) can direct the robot 100 to proceed to a designated room by manually entering a room number in an interface mounted on the robot 100, including but not limited to a tablet-type interface or keyboard.

A method 101 can also include a robot autonomously navigating from the location at which it received its delivery item(s) to a designated delivery location. Such an action can include avoiding stationary and moving objects along the way. In some embodiments, this can include navigating a building, a campus of multiple buildings, or an ever larger geographical area. In the embodiment shown, as seen in a third panel 109, a robot 100 can autonomously navigate through hallways of an environment 122 (e.g., hotel), avoiding other guests or objects potentially blocking a route to the designated room.

A method 101 can further include a robot arriving at the delivery location and stopping at a particular zone associated with the location. In some embodiments, a notification can be generated to signal the arrival of the item(s). Following an authentication action, the securable storage container can enable the item(s) to be removed. This can include, but is not limited to, the storage container opening or becoming unlocked. In the embodiment shown, as seen in a fourth panel 1 11 , a robot 100 can arrive at a destination zone 132, which can be the front of the ordering guest's door. A notification can be given to the guest to indicate the arrival of the item(s). In some embodiments, this can include a person (e.g., concierge) contacting the guest to inform them of the robot's presence at the door (e.g., phone, electronic message, etc.) or an automated message can be sent to the guest (e.g., mobile applet or other electronic interface can signal the guest of the robot's arrival). In certain embodiments, in addition or alternatively, the robot 100 can provide a suitable notification, including visual or audio signals such as those generated by on-board speakers of the robot, a tablet with speakers mounted to the robot, or the like. In still other embodiments, the robot 100 can be maneuvered to physically knock the door to provide a notification. When the door opens, the robot 100 can complete authentication and allow pickup of the item(s) 107 by the guest (e.g., preset item(s), unlock and open a lid, etc.). Authentication can include, but is not limited to, identifying that the door has opened, alone or in conjunction with additional electronic or biometric identification techniques.

FIGS. 1 B, 1 C and 1 D are a series of top-down looking cartoon views showing movement of a robot through an area containing obstacles to a delivery zone, as well as the opening of a container to complete delivery according to an embodiment.

In some embodiments, a robot containing item(s) for delivery can evade stationary and moving obstacles to reach a delivery zone. In the embodiment of FIG. 1 B, a robot 100 can navigate through a hallway 122 (i.e., an environment) with various obstacles, to a delivery zone 134 positioned adjacent to a door 113. Such an operation can include the robot 100 maneuvering through a hallway 150, navigating to evade obstacles such as a cart 1 15 and a person 126, in order to position itself in a destination zone 134 in front of a doorway 1 13. FIG. 1 B shows a destination zone suitable for an inwardly opening door 113, as indicated by door open angle 1 17.

In some embodiments, features at a delivery location can be used to identify and/or confirm a delivery zone. In the embodiment of FIG. 1 C, a robot 100 can maneuver to a delivery zone 134 adjacent to a door 1 13. As seen in FIG. 1 C, a hallway 122 (i.e., environment) can be bounded on one side by a wall 170 having a door 1 14 into a room. A robot 100 having a secured container 116 can autonomously navigate in a direction indicate by arrow 132 to a delivery zone 134 outlined by dotted lines, with the delivery zone 134 being in front and slightly to the side of the door 113.

In some embodiments, a robot can arrive at a delivery zone and enable delivered item(s) to be retrieved. In particular embodiments, a robot can orient itself in a predetermined fashion to provide easy access to the item(s). In the embodiment of FIG. 1 D, once it has arrived at delivery zone 134 adjacent to the door 113 corresponding to the ordering guest's room, a robot 170 can complete delivery upon door 1 13 opening. In some embodiments this can include unlocking container 1 16. In a particular embodiment, a robot 100 can navigate to the delivery zone 134 and rotate in place (indicated by arrow 133) to present the container 116 in a position easily retrievable by the guest. When the door 1 13 opens, the robot 100 can complete authentication (using the opening of the pre-selected door as a portion of the authentication process) and allow pickup of the item(s) resting in the container 1 16 by the guest. In some embodiments, this can include unlock and opening a lid of the container 1 16.

While embodiments show a delivery zone that is relative to a door in a hotel, any suitable location can serve as a delivery zone. For example, an entryway or threshold area, a defined delivery zone, designated restaurant tables, guest occupied reception or meeting room chairs, poolside lounges, or even a biometrically identified guest can be serve as, or be used to derive, a delivery zone.

While embodiments show a robot delivering items to a guest, delivery can be for any other suitable task. For example, a robot can be used to deliver cleaning supplies or materials to carts of cleaning staff, while in other embodiments robots can deliver items to other robots for later pickup. To identify a delivery zone or assist in the identification of a delivery zone, a robot can use image sensors, depth sensors, position sensors, or the like. The sensors can be used to identify room numbers and determine if opening of a door has occurred. Active or passive identification systems can be used, including but not limited to RFID tags, Bluetooth beacons, QR coded markings, ultrasonic emitters, or other suitable area or target identification mechanism or marking system. While such sensors can be mounted on the robot itself, in some embodiments, all or a portion of the sensors can be separate from the robot, but can transmit sensor data to the robot, or have such data be retrieved by the robot.

In one operational embodiment for determining the designated delivery zone and assisting in

authentication, a robot can use a precomputed (if door sizes are standardized) or locally determined three dimensional (3D) door opening model. For example, once a robot 100 is localized in front of a door, it can detect the state of the door (open or closed) by using depth or other suitable 3D sensors to measure door dimensions and position. Typically, a door is positioned on a plane that is perpendicular to the floor, and rotates on hinges. As the door opens and closes, it sweeps out an arc along the floor. The 3D sensor data is fit to this model of the door and its surrounding walls. By comparing the orientation of the sensed door plane to the detected walls and the map, the robot estimates the angle of the door, determines whether it is in an open or closed state, and can determine whether or not the door opening will contact the robot. The robot can use the model to position itself in a position that allows for ease of delivery, while preventing contact with the opening door. In certain embodiments the robot can position itself in a non-blocking position to allow entry or exit of guests through the doorway even during delivery.

In some building types, such as hotels, doors are normally locked and often open inward. A person present in the room, or a person who can open the door has been authenticated to a certain extent by the hotel. For hotel delivery of inexpensive items, this level of authentication can be sufficient for many applications. In practice, the robot may be programmed to unlock a bin or cargo carrier so that a person can remove its load once the robot detects that the door is open. Individual hotels or institutions can augment this authentication technique with others if needed, such as asking the person receiving the delivery to sign on a tablet, by use of a tablet mediated video interface with a hotel employee, detection of a guest key card with RFID or magnetic strip, personal identification number (PIN) generated for a guest at check-in, or other suitable means. More advanced biometric techniques including but not limited to fingerprint, voice analysis, or facial image identification can also be used.

In some embodiments, removal of a delivered item can be presumed, and the lid automatically closed and relocked. In other embodiments, active measures can be utilized to confirm the item(s) have been removed, including but not limited to weight or pressure sensors, RFID tags, imaging, ultrasonic sensors can be removal of item attached is detected.

As will be appreciated, various procedures can be used for deliveries. For example, in some embodiments, one or more types of delivery may not require a locked container, with the robot simply being loaded with an item and autonomously maneuvering to a delivery zone, authenticating (i.e., detecting when the door opens), and the item(s) can be presented for delivery.

In another delivery procedure, an item(s) can be placed in lockable container inside or to one side (e.g., rear) of a robot, where the robot can maneuver to a delivery zone (e.g., door), authenticate (e.g., interact with a guest), and rotate to present a side or rear container holding item(s) for delivery. FIG. 2A is side view and FIG. 2B is a top view of a robot 200 that can be included in embodiments. A robot 200 can include one or more depth sensors 202, a body 204 and a movement system 206. A depth sensor 202 can be mounted on and/or within a robot body 204. A depth sensor 202 can have a relatively low field of view (FOV). A relatively low FOV can be less than 180° in one direction in some embodiments, less than 90° in one direction in particular embodiments, and less than 65° in one direction in very particular embodiments. As shown in FIG. 2A, a depth sensor 202 can be angled downward for a FOV 208 that encompasses the region immediately in front of the robot 200. In particular embodiments, a FOV 208 can include a portion of the robot 200 itself. In embodiments with multiple depth sensors 202, such depth sensors can have different FOV angles or the same FOV angles.

An addition or alternatively, a depth sensor 202 can have a relatively low FOV and be angled forward (with respect to robot forward movement) for a FOV 208' that encompasses the region forward of the robot 200. In some embodiments, a robot 200 can include one depth sensor 202 having one of the FOVs shown (208 or 208').

Alternatively, a robot 200 includes a depth sensor 202 capable of moving between multiple FOV orientations (e.g., between 208 and 208'). In other embodiments, a robot 200 can include multiple depth sensors 202, one of which provides a different FOV (e.g., one provides 208 and another provides FOV 208').

In some embodiments, a depth sensor 202 can operate by combining image capture with a beam emitter. In a particular embodiment, a depth sensor 202 can include an image sensor and an emitter that emits some spectra of light (e.g., any of infrared, visible or ultraviolet). The image sensor can detect objects by the emitted light reflected off the objects.

One or more depth sensors 202 can be fixedly mounted to a body 200. In such embodiments, to scan a region greater than a field of view, a robot 200 is capable of rotational movement to enable the fixed sensors to scan the environment. In other embodiments, one or more depth sensors 202 can be movably mounted to a body 200. In particular embodiments, such movable mountings can provide only limited movement for a depth sensor 202. In very particular embodiments, limited movement of depth sensor mountings can add no more than 45° to the depth sensor's existing FOV.

According to embodiments, a robot body 204 can have a generally cylindrical or elongated shape. A robot body 204 can have a height 212 that is greater than its width 120. In some embodiments, a height 212 can be no less than 1.5 times the width 210. Further, a robot body 204 can have a vertical size conducive to interaction with people. Accordingly, according to some embodiments, a robot height 212 can be between 0.8 to 2 meters, in particular embodiments, between 1.2 and 1.5 meters. In addition or alternatively, a robot 200 can have a width sufficient to store deliverable items, while at the same time being small enough to enable ease of movement in an environment. Accordingly, according to some embodiments, a robot diameter or maximum width can be less than a meter, in some embodiments between 30 and 60 cm, and in particular embodiments, between 40 and 50 cm.

A generally cylindrical/elongated body 204 can have a low profile surface when the robot 200 is in motion. That is, as a robot 200 moves, there can be no structures significantly projecting outward in a lateral direction. In some embodiments, a low profile body surface will have no structures extending away from the main body surface by more than 1/3 a width of the body, and in particular embodiments, not more than ¼ a width of the body. Such a generally cylindrical/elongated body can provide for more efficient movement in an environment, as a space occupied by a robot 200 can be essentially uniform in all lateral directions.

It is noted that in some embodiments, a robot 200 can maintain a low profile shape whether moving or stationary. However, in other embodiments, when a robot 200 is not moving, structures may extend outward from a body 204. As but one example, a robot 200 can include doors that swing away from a body 204 to enable access to a storage container and/or other locations interior to the body (e.g., maintenance access). Other embodiments, can have other deployable structure when the robot is not in motion.

According to embodiments, depth sensor(s) 202 can be mounted in a top portion of a body 204. In some embodiments, a top portion can be the upper 1/3 of the robot height 212. In particular embodiments, depth sensor(s) 202 can be mounted in a top 20% of the robot height 212.

A movement system 206 can include any suitable movement system that enables a robot 200 to move in its operating environment, including but not limited to wheeled systems, tracked systems, roller systems, or combinations thereof. In a particular embodiment, a movement system 206 can enable a robot 200 to have both linear and rotational movement. In one particular embodiment, a movement system 206 can include at least two wheels positioned apart from one another, each capable of independent rotation in either direction.

Referring still to FIGS. 2A and 2B, in some embodiments, a robot 200 can further include a user interface

(l/F) 214. A user l/F 214 can enable a robot 200 to be directed or programmed to perform various tasks and/or to interact with other people. In a particular embodiment, a user l/F 214 can be a touch screen l/F for a low profile. According to particular embodiments, a robot 200 can be a delivery robot and a user l/F 214 can be used to authenticate delivery to an indicated destination and/or person.

Along these lines, a robot 200 can also include a container 216. A container 216 can be formed within a body 204, to maintain a low profile shape. In some embodiments, a container 216 can be securable, having some structure to limit access to stored contents. In a particular embodiment, a robot 200 can include a door/lid 218 for securing the container 216. A door/lid 218 may or may not be lockable.

According to embodiments, a robot can have generally cylindrical or elongated body. In some embodiments, such a shape can be one that maintains a generally closed curved shape in lateral cross section. Such a shape may vary according to vertical position, however. A generally cylindrical body does not require a circular or ellipsoid cross section. As shown in top view of FIG. 2B, a generally cylindrical body 204 has rounded features, but is not circular or ellipsoid. In some embodiments, a generally cylindrical body, when viewed in lateral cross section, can occupy a majority of the area of an inscribing circle or ellipse. Thus, as shown in FIG. 2B, a body 204 occupies a majority of an inscribing circle 220.

FIGS. 2C to 2F show various other cross sectional shapes that can be included in a generally cylindrical body. Each of FIGS. 2C to 2F shows a body 204-C to 204-F that occupies the majority of the area of an inscribing circle 220. The various body cross section shapes of FIGS. 2B to 2F are provide by way of example only, and should not be construed as limiting.

Having described a sensor configuration and corresponding robot according to various embodiments, methods of operation for a robot according to embodiments will now be described.

As seen in the cartoon illustration of FIG. 3, a robot 300 can move in an environment 322 such as a hallway or room, even in the presence of potentially blocking objects or people moving through the area. In particular embodiments, a robot 300 can be any of those described herein, or equivalents. A robot 300 can be autonomously movable in an environment 322 that can include multiple fixed objects 324-0 and 324-1 , as well as one or more movable objects 326, such as person moving toward a door 328 in a direction indicated by arrow 330.

In addition to translational movement along a floor of the environment 322, robot 300 can rotate through 360°, permitting environment scanning with one or more sensors 302 fixedly mounted or having a limited movement. A sensor 302 can include at least one image based depth sensor. A robot 300 can move in a direction indicated by arrow 332 to a target destination zone 334 in front of the door 328. Upon reaching the target destination zone 328, deliveries held in a securable container 316, which, in the particular embodiment shown, can be built into a top of the robot 300. Deliveries within securable container 316 can be removed by a room occupant (not shown).

Sensor(s) 302 can be fixed or movably mounted near or at a top of the robot 300. In some embodiments, a key area to sense during obstacle avoidance can be the area directly in a movement path (e.g., 322) of the robot 300, particularly the area directly in front of the robot 300. Accordingly, in some embodiments, sensors 302 (including one or more depth sensors) can include one or more sensors that are directed generally downward or outward, with a field of view typically maintained to include an area into which the robot 300 is moving.

In some embodiments, sensor(s) 302 can include a depth camera that is mounted such that it points directly downward, with about half of its field of view filled with a body of robot 300 while the remaining half can be used for obstacle detection. In some embodiments, a depth sensor within sensors 302 can be mounted out and down at an angle of up to FV/2 from vertical to provide greater viewable area for obstacle detection.

In very particular embodiments, depth sensors can include components similar to, or derived from, video gaming technology, enabling three dimensional sensing. Such depth sensors can be more cost effective than wide FOV laser-based sensors employed in conventional systems. Very particular examples of possible sensors of this type can include, but are not limited to, the Kinect manufactured by Microsoft Corporation, Carmine by Primsense (now owned by Apple Computer), or DepthSense 325 by SoftKinetic. Such depth sensors can be more cost effective, and typically direct infrared light to bounce off objects and be captured by an image sensor in order to determine how far those objects are from the sensor; while further incorporating an video camera (such as an RGB video camera) to allow the depth image to be combined with the video image.

Compared to commonly available laser sensors, depth sensors included in a robot according to embodiments can have a much narrower field of view (typically less than 90°), a much shorter effective range of depth detection (around 1 -3 meters), and often have a "dead zone" with limited or absent depth ranging within a half meter or so of the depth sensor.

According to some embodiments, mounting one or more depth sensors as described herein can overcome limitations associated with a typically narrow field of view and other limitations of such depth sensors. In certain embodiments, a depth sensor can be movable, with hinged, rail, hydraulic piston, or other suitable actuating mechanisms used to rotate, elevate, depress, oscillate, or laterally scan the depth sensor. In other embodiments, multiple depth sensors can be used and generally directed so that forward, backward, upward and downward regions are monitored. In certain embodiments, conventional RGB CMOS or CCD sensors can be used, alone or in combination with narrowband, wideband, polarization or other spectral filters. Embodiments can also include infrared, ultraviolet, or other imaging focal plane array devices to allow for hyperspectral image processing. This can allow, for example, monitoring and tracking of guides, markers, or pathways that are not visible, or not easily visible to people.

In some embodiments, ambient light such as sunlight, incandescent, halogen, LED, fluorescent or other commonly available artificial source may illuminate the environment in which a robot (e.g., 100, 200, 300) moves, and depth sensors of the robot can use such light to detect objects/obstacles. In addition or alternatively, a robot can have one or more attached (movable or fixed) light sources to augment or serve as a light source for object/obstacle detection. Such light sources can augment ambient light intensity and/or provide wavelengths not available in the ambient light source and/or substitute for ambient light in dark environments.

If a robot (e.g., 100, 200, 300) includes such light sources, the light sources may be mounted along with, or separately from, the depth sensors, and can include monochromatic or near monochromatic light sources such as lasers, light emitting diodes (LEDs), or organic light emitting diodes (OLEDs). In some embodiments, broadband light sources may be provided by multiple LEDs of varying wavelength (including infrared or ultraviolet LEDs), halogen lamps or other suitable conventional light source. Various light shields, lenses, mirrors, reflective surfaces, or other optics can provide wide light beams for area illumination or tightly focused beams for improved local illumination intensity.

Interaction with a robot (e.g., 100, 200, 300) can be provided by local input or network interface. As but a few examples, local input can be through a touchpad, by voice or gesture control, or by dedicated remote controllers. Local display of status, functionality, and error messages or the like may be afforded by a touchpad display. The display can be a conventional LCD display, a bistable displays (such electronic paper or similar), an OLED display, or other suitable display. Local user input can include a robot mounted pad, hard or soft keyboard, touch sensitive element (which may be integrated as part of the optional display), or similar, to provide for user input, voice control, or camera mediated user gestural control.

In certain embodiments, a wired or wireless connect subsystem can be used to connect to another user interaction device such as a laptop, tablet, or smart phone (not shown). Optionally, data and control signals can be received, generated, or transported between varieties of external data sources, including wireless networks, personal area networks, cellular networks, the Internet, or cloud mediated data sources. In addition, a robot (e.g., 100, 200, 300) may include a source of local data (e.g. a hard drive, solid state drive, flash memory, or any other suitable memory, including dynamic memory, such as SRAM or DRAM) that can allow for local data storage of user- specified preferences or protocols.

In one particular embodiment, multiple communication systems can be provided. For example, a robot (e.g.,

100, 200, 300) can be provided with a direct Wi-Fi connection (802.11 b/g/n), as well as a separate 4G cell connection provided as a back-up communication channel (e.g., such as that included on an interface tablet computer). Similarly, tablet or robot mounted Bluetooth or other local communication systems can be used to identify pre-positioned radio beacons, or to form a part of a user interface via a user smartphone or tablet.

According to embodiments, when a robot (e.g., 100, 200, 300) autonomously moves to conduct a task, it can rely on localization for tracking its current position. A typical example of localization technologies is a simultaneous localization and mapping (SLAM) technique. Thus, a mobile robot (e.g., 100, 200, 300) can use SLAM to detect information of surroundings of a work space where the robot conducts a task and process the detected information to construct a map corresponding to the work space while at the same time estimating its absolute position.

In certain embodiments, Bluetooth beacons, radio beacons, light emitting devices, and/or visible patterns can be placed at particular sites or objects to assist robot navigation.

In some embodiments, a robot (e.g., 100, 200, 300) can carry a wide range of amenities and supplies in various optional lockers, shipping containers, or shelving units, including food and beverages. Some of these supplies (especially beverages) may spill or be damaged if the robot does not move smoothly and gently. Such a problem can be especially acute when the robot starts and stops, particularly during emergency stops (e.g., when someone jumps into its path). In one embodiment the robot (e.g., 100, 200, 300) can be controlled to gently accelerate and decelerate, minimizing the forces felt by the payload. To enable such a response, a robot (e.g., 100, 200, 300) can have a motor control system of sufficient fidelity for smoothly decelerate multiple motors (wheels) simultaneously. In a particular embodiments, a robot (e.g., 100, 200, 300 can include a high-frequency (e.g. 1000 Hz) motor control loop system.

Referring still to FIG. 3, in operation, a robot (e.g., 100, 200, 300) can detect a movable object 326, and plan a new path to avoid the object, or slow down or halt until the object 326 is no longer in the desired movement path 332 of the robot (e.g., 100, 200, 300). Stationary objects 324-0/1 can be avoided according to a local movement plan. Stationary objects 324-0/1 can already be known, and verified by location detection, or newly detected by the robot.

FIG. 4 shows a robot 400 according to one particular embodiment, as well as selected components of the robot 400 in an exploded view. In some embodiments, robot 400 can be one very particular implementation of robot 100 shown in FIGS. 1A to 3.

A robot 400 can have a generally cylindrical shape about a vertical midline 434. Advantageously, this shape simplifies movement calculations and simplifies rotation in place, since position and potential interactions of objects with extending arms or the like do not have to be determined. A touch tablet computing device (tablet) 414 can be included for user input and/or messaging, and can be mounted at the top of the robot at an angle convenient for viewing and user input. In addition to a visible display, tablet 414 can be used for speech input/output, and/or for processing and controlling the robot 400.

In some embodiments, a speaker 436 separate from the tablet 414 can also be included for providing audible instructions or notices.

In some embodiments, a storage container 416 can be included within a body 404 of the robot, positioned behind the tablet 414. In some embodiments, storage container 416 is securable. In particular embodiments, storage container 416 is lockable, and can be controlled to unlock for delivery to a recipient only when a destination has been reached an authorization to unlock is received.

In the embodiment of FIG. 4, robot 400 can support multiple fixed depth sensors, including a forward looking depth sensor 402-0 and a downward looking depth sensor 402-1 mounted adjacent to each other. In the disclosed embodiment, the depth sensors (402-0/1 ) can be fixedly mounted in a manner that does not require a turret or movable actuators. In particular embodiments, each depth sensor 402-0/1 can include a beam emitting device and an image sensor that detects the beam as it reflects off of objects. In a very particular embodiment, depth sensors 402-0/1 can include an IR emitter and IR image sensor, such as an IR video camera. In some embodiments, a robot 400 can include a video camera 402-2 to provide additional imaging capabilities. In very particular embodiments, a video camera 402-2 can be an RGB CMOS type video camera.

In addition to depth sensors, a robot 400 can include one or more other sensors. Referring still to FIG. 4, a robot 400 can further include a base mounted sonar array 438 and a wide angle sonar 440 mounted near a top of the robot 400.

A robot 400 can be controlled by one or more processors executing stored instructions that can be responsive to sensor inputs and/or transmitted inputs. In a particular embodiment, an x86 or similar central processing unit 442 can be used in conjunctions with one or more microcontrollers 444 and motor controllers 446 for local control of movement of the robot 400.

In the embodiment shown, differential drive motors 448 powered by batteries 450 can provide movement by driving wheels (not shown) that support the robot 400. In particular embodiments, batteries 450 can be lithium ion or some other battery type, rechargeable battery systems being preferred. A drive mechanism includes separate drive motors 448 each attached to its own wheel, in a differential drive configuration. In some embodiments such a drive mechanism can allow for a robot velocity of 1.5 meters/second, and the ability to move up and down ramps, as well as on level ground. In a particular embodiments, a robot 400 can include two drive wheels between 4-8 inches in diameter, preferably about six inches in diameter.

According to embodiments, a robot 400 can be sized to have a height of between 0.8 to 2 meters, preferably between 1.2 to 1.5 meters, and a diameter of between 30- 60 centimeters, preferably between 40-50 centimeters. Such physical dimensions can enable robot 400 to easily move through hallways and doorways.

FIGS. 5A and 5B are diagrams showing depth sensing field of view configurations according to embodiments. Such field of view configurations can be included in any of the robots described herein.

FIG. 5A is a perspective view showing a robot 500 and two corresponding fields of view 508 and 508'. FIG. 5B is a side view showing a top portion of the robot 500. In the embodiment shown, a robot 500 can have two depth sensors 502-0/1 fixedly mounted near a top of the robot 500. Depth sensors 502-0/1 can be any suitable depth sensor, but in particular embodiments, can include an emitting device and image sensor as described herein, or an equivalent. In some embodiments, one or both depth sensors 502-0/1 can include an RGB CMOS type video camera. In the embodiment shown, depth sensors 502-0/1 can be positioned adjacent to each other. In some embodiments, depth sensors 502-0/1 can be within a few centimeters of one another. In some embodiments, both depth sensors 502-0/1 can produce color images with corresponding depth information. That is, both depth sensors 502-0/1 can include video cameras.

Depth sensors 502-0/1 can be mounted at the top of the robot 500 facing the forward traveling direction of the robot (i.e., the front). In one particular embodiment, depth sensors 502-0/1 can be mounted 80cm to 85cm above the floor. One depth sensor 502-1 can be pointed directly ahead, while the other depth sensor 502-0 can be angled downward to image the floor directly ahead of the robot. Such an angle is shown as 536 in FIG. 5B. In a very particular embodiment, such an angle can be between 50° and 80° in the vertical direction, preferably between 60° and 70°, and in one very particular implementation, about 65°. A vertical field of view angle for each depth sensor 502-0/1 is shown as 538 and 538' in FIG. 5A. In some embodiments, such field of view angles can be less than 180°, in other embodiments such angles can be less than 90°, and in further embodiments less than 60°. In one very particular embodiment, a field of view for each depth sensor 502-0/1 can be 57° in the horizontal direction and 45° in the vertical direction.

Referring still to FIG. 5A, in some embodiments, a near edge of the field of view 508 for a downward pointed depth sensor 502-0 can intersect a lower portion of the robot body 504 (i.e., the base). This is represented in FIG. 5A by the pear-shaped dashed line 508" on the front of the robot 500 that shows the edge of the field of view 508 as it intersects the curved body 504 of the robot 500. The trapezoid shape 508 on the floor in front of the robot shows the area visible to the downward facing depth sensor 502-0. The vertical trapezoid 508' in front of the robot 500 shows the area visible to the forward facing depth sensor 502-1.

The robot 500 of FIGS. 5A and 5B can also include a movement system 506, user l/F 516, or storage container 516 according to any of the embodiments described herein, or equivalents.

Referring to FIG. 6, a robot 600 according to a further embodiment is shown in a perspective view. A robot 600 can have a body 604 with a generally cylindrical or elongated shape. One or more depth sensors 602 can be mounted on an upper portion of the body 604 to enable downward and/or forward facing depth sensors. Depth sensors 602 can have any of the configurations and/or components described for embodiments herein, or equivalents. A robot 600 can include a user l/F 614 also mounted on an upper portion of body 614. In one embodiments, user l/F 614 can be a tablet computing device with a touchscreen. A robot 600 can include a storage container 616, which in the embodiment shown, can extend into a top surface of the robot body 604. In the embodiment shown, a storage container 616 can be securable, including a door/lid 618 which can be closed. In particular embodiments, door/lid 618 is lockable and can be unlocked and/or opened upon authentication in a delivery operation.

A robot 600 can include additional body mounted items 640, which can include, but are not limited to, lighting structures to provide notification lighting, lighting for use by sensors 602, or one or more additional sensors.

While various robots, robot sensor mounting arrangements, and related methods have been described above, additional methods will now be described with reference to a number of diagrams.

FIG. 7 is a flow diagram of a method 750 according to an embodiment. A method 750 can include image sensor based depth sensors scanning for the presence of local objects/obstacles 752. Such actions can include one or more image based depth sensors mounted at or toward a top of a robot body that detect a beam emitted from the depth sensor, such as a light of a non-visible or visible spectrum. In very particular embodiments, such actions can include an infrared video camera operating in conjunction with an infrared emitter to determine the presence and distance of a local object/obstacle 752.

A method can also include additional sensors scanning for local objects and/or determining a local position of a robot (754). Based on data generated by blocks 752 and 754, a local occupancy grid can be derived 756. As but one example, upon detecting a new object/obstacle, a local occupancy grid can be updated to include the presence of the object/obstacle, as well as whether such an object/obstacle is in motion or stationary. A method 750 can also include a robot using map information 758 in conjunction with a local occupancy grid to create local navigation controls 758. In some embodiments, map information can also be used to create a global plan of navigation 760, prior to creating the local plan. FIG. 8 shows another method 850 according to an embodiment. A method 850 can include a robot navigating according to a local plan 860. A robot can use a sensor having an image sensor and emitted signal to detect local object(s)/obstacle(s) 862. If no local object/obstacle is detected (N from 862), a method 850 can return to navigating according to the local plan (860). If a local object/obstacle is detected (Y from 862), a method 850 can revise a local occupancy grid stored in the robot 864 that accounts for the detected object(s)/obstacle(s), and revise its local plan 866 accordingly. A method 800 can then return to navigating according to the local plan (860).

FIG. 9 shows another method 950 according to an embodiment. A method 950 can include actions like those of FIG. 8. Such like actions are referred to by the same reference character but with the leading digit being a "9" instead of an "8".

Method 950 differs from that of FIG. 8 in that action 962 can include a robot using forward and downward facing low field of view (FOV) depth sensors to detect local object(s)/obstacle(s) 962. A low FOV depth sensor can have a field of view less than 180° in some embodiments, less than 90° in other embodiments, and less than 60° in a particular embodiment.

FIGS. 10A to 10C are a sequence of diagrams showing robot navigation according to a particular embodiment. FIGS. 10A to 10C show an environment 1022 (a hallway) that can include target destination zones (1034-A to 1034-D) (areas in front of doors). Environment limits or known objects are shown as 1070-A (walls) and 1070-B (known furniture). These features can be derived from map data, global occupancy grid, or local occupancy grid. A robot 1000 can follow a local plan 1074 derived from a local occupancy grid, map data, and global plan. Environment 1022 also includes a new object 1072. New object 1072 is not included or known from the map data, global plan or local plan.

In FIGS. 10A to 10C it is assumed that the robot 1000 is navigating to target destination 1034-C according to local plan 1074. Further a field of view for a sensor (or group of sensors) is shown as 1008. A robot 1000 and its corresponding sensor(s) can include any of those described herein, or equivalents.

Referring to FIG. 10A, a robot 1000 can move according to a local plan 1074, which takes into account the presence of known objects, which includes avoiding obstacle 1070-B.

Referring to FIG. 10B, as robot 1000 continues along local plan 1074 new object 1072 appears in its sensor(s) field of view 1008, and so is detected. Robot 1000 can update its local occupancy grid to include new object 1072.

Referring to FIG. 10C, based on the new local occupancy grid, robot 1000 can generate a new local plan 1074' that enables it to arrive at the target destination 1034-C, while at the same time avoiding the new object 1072.

FIG. 11 is a flow chart illustrating process steps for autonomous robot navigation according to an embodiment. In FIG. 1 1 , a robot can navigate through an environment with variously positioned movable and non- movable items, as well as a target destination.

Referring to FIG. 11 , a sensor suite of a robot can include one or more sonars 1180, one or more bump sensors 1 182, and any of odometers, inertial measurement units (IMU), or other sensors configured to work with beacons, including beacons that emit light or radio signals (represented by 1184). A sensor suite can also include one or more camera systems 1 152 capable providing video or multiple still images.

Sensor data from the camera images (1 152) and/or odometers and position beacons (1184) can be used to localize a position of the robot 1186. In some embodiments, map data 1 158 can also be used in localization (1186). Localization 1186 can be used to arrive at a robot pose 1188, which can include the robot's position and orientation in a local environment. In the embodiment shown, sensor data from the camera images (1152), sonar(s) 1 180 and bump sensor(s) can provide local knowledge of object position in the environment around the robot (local occupancy grid) 1 156.

Data from the sensor suite (e.g., 1152) in combination with map data 1 158 can be used to arrive at a global occupancy grid 1 190.

In the embodiment shown, a user interface 1 1 14 can be used to enter/receive data that indicates a destination for the robot. Such data can be used to arrive at a goal pose 1 192, which can include the position of a target destination. In some embodiments, such a user interface data can be used in conjunction with map data 1 158 to arrive at a goal pose 1192.

A given robot pose 1188, goal pose 1192 and global occupancy grid 1 190 can be used by a global planner 1194 to generate a global plan (distance map) 1 160. In some embodiments, map data can be used with a global occupancy grid that integrates known positions of objects in the mapped area, and in conjunction with robot pose input and the goal pose, a robot global plan 1 160 for navigation can be generated. The global plan 1 160 can be reduced to a distance map.

Referring still to FIG. 1 1 , a local planner 1196 can combine a global plan 1160 with integrated data of local object positions from the local occupancy grid 1 156 and the robot pose 1 188. With such data, a local planner 1 196 can adjust the local plan to avoid or go around blockages or obstructions in the robot path. Local sensing and re- orientation of the robot (twists) 1 158 can verify local paths, and provide control input to the wheels (1198) for partial or full rotation of the robot, or backward or forward movement that avoids objects or people while traveling to a desired destination.

It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the spirit or scope of the invention. Thus, it is intended that the disclosed embodiments cover modifications and variations that come within the scope of the claims that eventually issue in a patent(s) originating from this application and their equivalents. In particular, it is explicitly contemplated that any part or whole of any two or more of the embodiments and their modifications described above can be combined in whole or in part. It is also understood that other embodiments of this invention may be practiced in the absence of an element/step not specifically disclosed herein.

Claims

IN THE CLAIMS

What is claimed is:

1. A system, comprising:

at least one robot configured to autonomously navigate from a first location to a destination zone, the at least one robot having a generally cylindrical shape and including

a storage container for storing at least one item for delivery,

at least one image based depth sensor mounted proximate a top end of the robot, and

a controller configured to navigate to the destination zone and to authenticate a delivery target for the item.

2. The system of claim 1 , wherein:

the controller is configured to authenticate the delivery target in response to at least an opening of a door at the destination zone.

The system of claim 2, wherein:

the delivery target is a person; and

the controller is configured to further authenticate the delivery target by a human interface. The system of claim 2, wherein:

the controller is configured to further authenticate the delivery target via a video interface included with the

The system of claim 2, wherein:

the delivery target is a person that is given an identification item; and

controller is configured to further authenticate the delivery target via the identification item.

The system of claim 1 , wherein:

the storage container is lockable; and

the controller is further configured to unlock the container once authentication is complete.

7. The system of claim 1 , wherein:

the at least one image based depth sensor is mounted on one side of the robot;

the storage container is located on a different side of the robot than the at least one image based depth sensor; and

the controller is further configured to rotate to present the storage container in a particular orientation with respect to the destination zone.

The system of claim 1 , further including: the robot is configured to operate in a work space; and

a plurality of position indicating objects located in the work space; wherein

the controller is configured to determine a location of the robot based, at least in part, upon detecting the position indicating objects.

9. The system of claim 1 , further including:

the depth sensor has a field of view (FOV) of no more than 180°; and

a drive mechanism configured to provide linear and rotational movement for the robot. 10. The system of claim 1 , wherein:

the at least one depth sensor comprises an image sensor in combination with an emitter device configured to emit a signal that bounces off objects and is captured by the image sensor to determine a distance of such objects from the robot.

11. The mobile robot of claim 1 , wherein:

the at least one depth sensor has a field of view (FOV) and is mounted with a downward angle up to one half its FOV from vertical.

12. The mobile robot of claim 1 , wherein:

the at least one depth sensor includes a plurality of depth sensors, including

a first depth sensor, and

a second depth sensor mounted proximate the first depth sensor having a FOV orientation different from that of the first depth sensor. 13. A method, comprising:

storing at least one item in a storage container of a robot having a generally cylindrical shape;

delivering the at least one item to a destination zone with the robot via autonomous navigation, the autonomous navigation including determining the presence of obstacles in the robot's path with at least one forward facing image based depth sensor;

attempting to authenticate a delivery target for the item; and

if the delivery target is authenticated, presenting the item to the delivery target.

14. The method of claim 13, wherein:

attempting to authenticate the delivery target includes detecting an opening of a door at the destination zone.

15. The method of claim 14, wherein:

attempting to authenticate the delivery target includes actions selected from the group of: receiving a predetermined input from a person at a human interface on the robot; a video interface transaction; input of an identification given to a person at an operating location of the robot

16. The method of claim 14, wherein:

the storage container is lockable; and

17, A robot for delivering items to a destination zone, comprising:

a generally cylindrical shape;

lockable storage container;

at least one image based depth sensor mounted proximate a top end of the robot;

a controller configured to determine if a door in front of the robot has been opened; and

an authentication system to authenticate a delivery target for the item in response to detecting an opening of a door at a destination zone. 18. The robot of claim 17, further including:

a human interface device for receiving inputs from a person; and

the authentication system authenticates the delivery target further in response to inputs from the human interface device. 19. The robot of claim 17, further including:

the authentication system authenticates the delivery target further in response to personal identification items given to persons by an establishment operating the robot.

20. The mobile robot of claim 17, wherein:

the at least one depth sensor includes a plurality of depth sensors, including

a first depth sensor, and

a second depth sensor mounted proximate the first depth sensor having a FOV orientation different from that of the first depth sensor.