CN110612431B - Cross-field of view for autonomous vehicle systems - Google Patents

Abstract

An imaging system for a vehicle is provided. In an embodiment, an imaging system includes an imaging module, a first camera coupled to the imaging module, a second camera coupled to the imaging module, and a mounting assembly configured to attach the imaging module to a vehicle such that the first and second cameras face outward relative to the vehicle. The first camera has a first field of view and a first optical axis, and the second camera has a second field of view and a second optical axis. The first optical axis intersects the second optical axis at least one intersection point of the intersecting planes. The first camera is focused at a first horizontal distance beyond the intersection of the intersecting planes and the second camera is focused at a second horizontal distance beyond the intersection of the intersecting planes.

Description

Cross Field of View for Autonomous Vehicle Systems

Cross References to Related Applications

This application claims priority to U.S. Provisional Patent Application No. 62/504504, filed May 10, 2017, which is hereby incorporated by reference in its entirety.

technical field

The present disclosure relates generally to camera systems for autonomous vehicles. In another aspect, the present disclosure generally relates to camera systems with crossed fields of view.

Background technique

An autonomous vehicle may need to consider multiple factors and make appropriate decisions based on these factors in order to reach the intended destination safely and accurately. For example, in order to navigate to a destination, an autonomous vehicle may also need to recognize its position within a road (e.g., a particular lane within a multi-lane road), navigate alongside other vehicles, avoid obstacles and pedestrians, observe traffic signals and signs, and traveling from one road to another at appropriate intersections or junctions. Utilizing and interpreting the vast amount of information collected by an autonomous vehicle as the vehicle travels to its destination presents many design challenges. The vast amount of data that autonomous vehicles may need to analyze, access, and/or store (eg, captured image data, map data, GPS data, sensor data, etc.) presents challenges that may actually limit or even adversely affect autonomous navigation. For example, an autonomous vehicle may need to process and interpret visual information (eg, information captured from multiple cameras located at discrete locations on the vehicle) as part of the collected data. Each camera will have a specific field of view. Where multiple cameras are used together, the fields of view of the cameras may overlap and/or be redundant in some cases.

Contents of the invention

Embodiments consistent with the present disclosure provide systems and methods for autonomous vehicle navigation. The disclosed embodiments may use cameras to provide autonomous vehicle navigation features. For example, consistent with the disclosed embodiments, the disclosed system may include one, two, three or more cameras monitoring the vehicle environment. Each camera's field of view may overlap another camera or even multiple cameras. The disclosed system may provide navigational responses based, for example, on analysis of images captured by one or more cameras. Navigation responses may also take into account other data including, for example, global positioning system (GPS) data, sensor data (eg, from accelerometers, speed sensors, suspension sensors, etc.), and/or other map data.

In one embodiment, an imaging system for a vehicle is provided. The imaging system can include an imaging module and a first camera coupled to the imaging module. The first camera can have a first field of view and a first optical axis. The imaging system may also include a second camera coupled to the imaging module. The second camera can have a second field of view and a second optical axis. The imaging system may further include a mount assembly configured to attach the imaging module to the vehicle such that the first camera and the second camera face outward relative to the vehicle. Additionally, the first optical axis may intersect the second optical axis at at least one intersection point of the intersecting plane. Additionally, the first camera may be focused at a first horizontal distance beyond the intersection of the intersecting planes and the second camera may be focused at a second horizontal distance beyond the intersection of the intersecting planes; and, the first field of view and the second Fields of view can form combined fields of view.

In one embodiment, an imaging system for a vehicle is provided. The imaging system can include an imaging module and a first camera coupled to the imaging module. The first camera can have a first field of view and a first optical axis. The imaging system may also include a second camera coupled to the imaging module. The second camera can have a second field of view and a second optical axis. The imaging system may also include a third camera coupled to the imaging module. The third camera may have a third field of view and a third optical axis. The imaging system may also include a mount assembly configured to attach the imaging module to an interior window of the vehicle such that the first camera, the second camera, and the third camera face outward relative to the vehicle. In addition, the first optical axis may intersect the second optical axis at at least one first intersection point of the first intersection plane, the first optical axis may intersect the third optical axis at at least one second intersection point of the second intersection plane, And the second optical axis may intersect the third optical axis at at least one third intersection point of the third intersection plane. In addition, the first field of view, the second field of view and the third field of view may form a combined field of view.

In one embodiment, an imaging system for a vehicle is provided. The imaging system may include an imaging module configured to arrange a plurality of cameras along a semi-circular arc. Multiple cameras may be oriented toward the radius of the semicircle. The imaging system may also include a mount assembly configured to attach the imaging module to an interior window of the vehicle such that the plurality of cameras faces outward relative to the vehicle. Additionally, each respective camera of the plurality of cameras may have a respective field of view and a respective optical axis projecting outwardly from a single relatively small and transparent opening. In addition, each respective field of view at least partially overlaps the radius of the semicircle, and the radius of the semicircle is centered within a single relatively small transparent opening. Furthermore, each respective optical axis intersects each other respective optical axis at at least one respective intersection point of a respective intersection plane, and all respective fields of view form a combined field of view.

The foregoing general description and the following detailed description are exemplary and explanatory only and do not limit the claims.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate various disclosed embodiments. In the picture:

FIG. 1 is a schematic representation of an exemplary system consistent with disclosed embodiments.

FIG. 2A is a schematic side view representation of an exemplary vehicle including a system consistent with disclosed embodiments.

FIG. 2B is a schematic top view representation of the vehicle and system shown in FIG. 2A consistent with disclosed embodiments.

2C is a schematic top view representation of another embodiment of a vehicle including a system consistent with disclosed embodiments.

2D is a schematic top view representation of yet another embodiment of a vehicle including a system consistent with disclosed embodiments.

2E is a schematic top view representation of yet another embodiment of a vehicle including a system consistent with disclosed embodiments.

FIG. 2F is a schematic representation of an exemplary vehicle control system consistent with disclosed embodiments.

3A is a schematic representation of the interior of a vehicle including a rearview mirror and a user interface for a vehicle imaging system, consistent with disclosed embodiments.

3B is an illustration of an example of a camera mount configured to be positioned behind a rearview mirror and against a vehicle windshield, consistent with disclosed embodiments.

3C is an illustration of the camera mount shown in FIG. 3B viewed from a different angle, consistent with disclosed embodiments.

3D is an illustration of an example of a camera mount configured to be positioned behind a rearview mirror and against a vehicle windshield, consistent with disclosed embodiments.

4 is an exemplary block diagram of a memory configured to store instructions for performing one or more operations, consistent with disclosed embodiments.

5A is a flowchart illustrating an example process for eliciting one or more navigational responses based on monocular image analysis, consistent with disclosed embodiments.

5B is a flowchart illustrating an example process for detecting one or more vehicles and/or pedestrians in a set of images, consistent with the disclosed embodiments.

5C is a flowchart illustrating an exemplary process for detecting road markings and/or lane geometry information in a set of images, consistent with the disclosed embodiments.

5D is a flowchart illustrating an exemplary process for detecting traffic lights in a set of images consistent with disclosed embodiments.

FIG. 5E is a flowchart illustrating an example process for eliciting one or more navigational responses based on a vehicle path, consistent with disclosed embodiments.

FIG. 5F is a flowchart illustrating an example process for determining whether a lead vehicle is changing lanes, consistent with disclosed embodiments.

6 is a flowchart illustrating an example process for eliciting one or more navigational responses based on stereoscopic image analysis, consistent with disclosed embodiments.

7 is a flowchart illustrating an exemplary process for eliciting one or more navigational responses based on analysis of three sets of images, consistent with disclosed embodiments.

Figure 8 is a schematic representation of an embodiment of an imaging system consistent with the disclosed embodiments.

9 is a schematic diagram of a single camera wide field of view imaging system.

Figure 10 is a schematic representation of another embodiment of an imaging system consistent with the disclosed embodiments.

11A is a schematic plan view representation of an exemplary imaging system having a combined field of view consistent with disclosed embodiments.

11B is a schematic plan view representation of an exemplary imaging system having a combined field of view consistent with disclosed embodiments.

12A is a schematic plan view representation of another exemplary imaging system having a combined field of view consistent with disclosed embodiments.

12B is a schematic plan view representation of another exemplary imaging system having a combined field of view consistent with disclosed embodiments.

13 is a perspective view representation of another exemplary imaging system having a combined field of view consistent with disclosed embodiments.

14 is a perspective view representation of another exemplary imaging system having a combined field of view consistent with disclosed embodiments.

15 is a perspective view representation of another exemplary imaging system having a combined field of view consistent with disclosed embodiments.

16 is a perspective view representation of the imaging system of FIG. 15 consistent with disclosed embodiments.

17 is a perspective view representation of another exemplary imaging system having a combined field of view consistent with disclosed embodiments.

18 is a perspective view representation of the imaging system of FIG. 17 consistent with disclosed embodiments.

19 is a schematic plan view representation of another exemplary imaging system having a combined field of view consistent with disclosed embodiments.

20 is a schematic plan view representation of another exemplary imaging system having a combined field of view consistent with disclosed embodiments.

21 is a front view representation of another exemplary imaging system consistent with the disclosed embodiments.

22 is a perspective view of an exemplary vehicle including an imaging system consistent with disclosed embodiments.

23 is a side view of an exemplary vehicle including an imaging system consistent with disclosed embodiments.

FIG. 24 is a schematic plan view representation of FIG. 23 consistent with disclosed embodiments.

25 is a schematic plan view representation of an imaging system consistent with disclosed embodiments.

Detailed ways

The following detailed description refers to the accompanying drawings. Wherever possible, the same reference numbers will be used in the drawings and the following description to refer to the same or like parts. While a number of illustrative embodiments have been described herein, modifications, changes, and other implementations are possible. For example, substitutions, additions, or modifications may be made to components shown in the figures, and the illustrative methods described herein may be modified by substituting, reordering, removing, or adding steps of the disclosed methods. Therefore, the following detailed description is not limited to the disclosed embodiments and examples. Rather, the proper scope is defined by the appended claims.

Autonomous Vehicle Overview

As used throughout this disclosure, the term "autonomous vehicle" refers to a vehicle capable of implementing at least one navigational change without driver input. "Navigation change" refers to a change in one or more of steering, braking, or acceleration of a vehicle. To be autonomous, a vehicle does not need to be fully autonomous (eg, operate fully without a driver or driver input). In contrast, autonomous vehicles include those vehicles that can operate under driver control during certain periods of time and without driver control during other periods of time. Autonomous vehicles may also include vehicles that only control certain aspects of vehicle navigation, such as steering (eg, to maintain the vehicle's path between vehicle lane constraints), but may leave other aspects to the driver (eg, braking). In some cases, an autonomous vehicle may handle some or all aspects of the vehicle's braking, speed control, and/or steering.

Because human drivers typically rely on visual cues and observation commands to control vehicles, transportation infrastructure has been built accordingly, with lane markings, traffic signs, and traffic lights all designed to provide visual information to the driver. Given these design features of transportation infrastructure, autonomous vehicles may include cameras and a processing unit that analyzes visual information captured from the vehicle's environment. Visual information may include, for example, components of the transportation infrastructure (eg, lane markings, traffic signs, traffic lights, etc.) and other obstacles (eg, other vehicles, pedestrians, trash, etc.) observable to the driver. In addition, autonomous vehicles can also use stored information, such as information that provides a model of the vehicle's environment when navigating. For example, a vehicle may use GPS data, sensor data (e.g., from accelerometers, speed sensors, suspension sensors, etc.), and/or other map data to provide information about its environment while the vehicle is in motion, and the vehicle (and other vehicles) may Use this information to position yourself on the model.

In some embodiments of the present disclosure, an autonomous vehicle may use information obtained while navigating (eg, from cameras, GPS devices, accelerometers, speed sensors, suspension sensors, etc.). In other embodiments, the autonomous vehicle may use information obtained from past navigations of the vehicle (or other vehicles) when navigating. In other embodiments, the autonomous vehicle may use a combination of information obtained while navigating and information obtained from past navigations. The following sections provide an overview of a system consistent with the disclosed embodiments, followed by an overview of a forward imaging system and a method consistent with the system. The following sections disclose systems and methods for constructing, using, and updating sparse maps for autonomous vehicle navigation.

As used throughout this disclosure, the term "field of view" refers to the total area that a camera is able to view in three dimensions. When this disclosure describes a field of view with reference to a single angle, that single angle refers to a two-dimensional horizontal field of view. As used throughout this disclosure, the term "optical axis" refers to the central axis of a camera's field of view. In other words, the "optical axis" is the vector at the projected center of the camera's visible area. In other words, the "optical axis" is the axis about which the camera's field of view is symmetrically oriented.

System Overview

FIG. 1 is a block diagram representation of a system 100 consistent with an exemplary disclosed embodiment. System 100 may include various components depending on the requirements of a particular implementation. In some embodiments, system 100 may include a processing unit 110 , an image acquisition unit 120 , a position sensor 130 , one or more storage units 140 , 150 , a map database 160 , a user interface 170 and a wireless transceiver 172 . Processing unit 110 may include one or more processing devices. In some embodiments, the processing unit 110 may include an application processor 180, an image processor 190, or any other suitable processing device. Similarly, image acquisition unit 120 may include any number of image acquisition devices and components depending on the requirements of a particular application. In some embodiments, image acquisition unit 120 may include one or more image capture devices (eg, cameras), such as image capture device 122 , image capture device 124 , and image capture device 126 . System 100 may also include a data interface 128 communicatively connecting processing device 110 to image acquisition device 120 . For example, data interface 128 may include any one or more wired and/or wireless links for transferring image data acquired by image acquisition device 120 to processing unit 110 .

Wireless transceiver 172 may include one or more devices configured to switch transmissions over the air to one or more networks (eg, cellular, Internet, etc.) through the use of radio frequency, infrared frequencies, magnetic or electric fields. Wireless transceiver 172 may transmit and/or receive data using any known standard (e.g., WiFi, Bluetooth Smart, 802.15.4, ZigBee, etc.). Such transmissions may include communications from the host vehicle to one or more remote servers. Such transfers may also include communications (one-way or two-way) between the host vehicle and one or more target vehicles in the environment of the host vehicle (e.g. navigation coordination of vehicles), even including broadcast transmissions to unspecified recipients in the vicinity of the transmitting vehicle.

Both the application processor 180 and the image processor 190 may include various types of processing devices. For example, one or both of applications processor 180 and image processor 190 may include a microprocessor, a pre-processor (such as an image pre-processor), a graphics processing unit (GPU), a central processing unit (CPU), support circuitry , digital signal processor, integrated circuit, memory, or any other type of device suitable for running applications and image processing and analysis. In some embodiments, application processor 180 and/or image processor 190 may include any type of single or multi-core processor, mobile device microcontroller, central processing unit, or the like. Various processing devices can be used, including, for example, processors available from manufacturers such as etc.) or a GPU available from the manufacturer (e.g. /> etc.), and can include various architectures (e.g. x86 processors, /> wait).

In some embodiments, application processor 180 and/or image processor 190 may include Get any EyeQ series processor chip. These processor designs each include multiple processing units with local memory and instruction sets. Such processors may include video inputs for receiving image data from multiple image sensors, and may also include video output capabilities. In one example, /> Using 90nm Micron technology running at 332Mhz. Architecture consists of two floating-point, hyper-threaded 32-bit RISC CPUs(/> cores), five Visual Computing Engines (VCEs), three vector microcode processors/> Denali 64-bit mobile DDR controller, 128-bit internal supersonic interconnect, dual 16-bit video input and 18-bit video output controllers, 16-channel DMA, and multiple peripherals. The MIPS34K CPU manages five VCEs, three VMP ^TMs and DMAs, a second MIPS34K CPU and multi-channel DMA, and other peripherals. Five VCEs, three /> and MIPS34K CPUs can perform the intensive visual computations required for versatile bundled applications. In another example, as a 3rd generation processor and than /> six times more powerful /> can be used with the disclosed embodiments. In other examples, /> and/or /> can be used with the disclosed embodiments. Of course, any newer or future EyeQ processing devices may also be used with the disclosed embodiments.

Any processing device disclosed herein can be configured to perform certain functions. Configuring a processing device, such as any of the described EyeQ processors or other controllers or microprocessors, to perform certain functions may include programming computer-executable instructions and making these instructions available to the processing device to operate within the processing device. Executed during operation. In some embodiments, configuring the processing device may include programming the processing device directly with architectural instructions. For example, a processing device, such as a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like, may be configured using, for example, one or more hardware description languages (HDL).

In other embodiments, configuring the processing device may include storing executable instructions on memory accessible to the processing device during operation. For example, a processing device may, during operation, access memory to obtain and execute stored instructions. In either case, the processing device configured to perform the sensing, image analysis, and/or navigation functions disclosed herein represents a dedicated hardware-based system that controls multiple hardware-based components of the host vehicle.

Although FIG. 1 depicts two separate processing devices included in processing unit 110, more or fewer processing devices may be used. For example, in some embodiments, a single processing device may be used to accomplish the tasks of applications processor 180 and image processor 190 . In other embodiments, these tasks may be performed by more than two processing devices. Furthermore, in some embodiments, the system 100 may include one or more processing units 110 without including other components, such as the image acquisition unit 120 .

The processing unit 110 may include various types of devices. For example, the processing unit 110 may include various devices such as a controller, an image pre-processor, a central processing unit (CPU), a graphics processing unit (GPU), support circuits, a digital signal processor, an integrated circuit, memory, or Any other type of equipment for processing and analysis. Image pre-processors may include video processors for capturing, digitizing and processing images from the image sensor. A CPU may include any number of microcontrollers or microprocessors. A GPU may also include any number of microcontrollers or microprocessors. Support circuitry may be any number of circuits known in the art, including cache, power, clock, and input-output circuitry. The memory may store software that controls the operation of the system when executed by the processor. The memory may include databases and image processing software. The memory may include any number of random access memory, read only memory, flash memory, disk drives, optical memory, tape memory, removable memory, and other types of memory. In an example, the memory may be separate from the processing unit 110 . In another example, the memory may be integrated into the processing unit 110 .

Each memory 140 , 150 may include software instructions that, when executed by a processor (eg, applications processor 180 and/or graphics processor 190 ), may control the operation of various aspects of system 100 . These memory units can include various databases and image processing software as well as trained systems such as neural networks or deep neural networks. The memory unit may include random access memory (RAM), read only memory (ROM), flash memory, disk drives, optical storage, magnetic tape storage, removable memory, and/or any other type of memory. In some embodiments, storage units 140 , 150 may be separate from application processor 180 and/or image processor 190 . In other embodiments, these storage units may be integrated into the application processor 180 and/or the image processor 190 .

Position sensor 130 may comprise any type of device suitable for determining a position relative to at least one component of system 100 . In some embodiments, location sensor 130 may include a GPS receiver. Such receivers can determine a user's location and velocity by processing signals broadcast by Global Positioning System satellites. Location information from location sensor 130 may be provided to application processor 180 and/or image processor 190 .

In some embodiments, the system 100 may include, for example, a speed sensor (e.g., tachometer, speedometer) for measuring the speed of the vehicle 200 and/or an accelerometer (single-axis or multi-axis) for measuring the acceleration of the vehicle 200 class components.

User interface 170 may include any device suitable for providing information to or receiving input from one or more users of system 100 . In some embodiments, user interface 170 may include user input devices including, for example, touch screens, microphones, keyboards, pointing devices, scroll wheels, cameras, knobs, buttons, and the like. With such an input device, a user may be able to select on-screen menu options by typing instructions or information, providing voice commands, using buttons, pointer or eye-tracking functionality, or through any other suitable technique for communicating information to system 100 to provide information input or commands to the system 100.

User interface 170 may be equipped with one or more processing devices configured to provide information to or receive information from a user and to process the information for use by applications processor 180, for example. In some embodiments, such a processing device may execute instructions for recognizing and tracking eye movements, receiving and interpreting voice commands, recognizing and interpreting touches and/or gestures made on a touch screen, responding to keyboard input or menu selections, etc. . In some embodiments, user interface 170 may include a display, speaker, haptic device, and/or any other device that provides output information to a user.

Map database 160 may include any type of database for storing map data useful to system 100 . In some embodiments, the map database 160 may include data related to the location of various items, including roads, water features, geographic features, businesses, attractions, restaurants, gas stations, etc., in a reference coordinate system. Map database 160 may store not only the locations of the items, but also descriptors related to the items, including, for example, names associated with any stored features. In some embodiments, map database 160 may be physically located with other components of system 100 . Alternatively or additionally, map database 160 or a portion thereof may be located remotely relative to other components of system 100 (eg, processing unit 110 ). In such an embodiment, information from map database 160 may be downloaded to a network (eg, via a cellular network and/or the Internet, etc.) via a wired or wireless data connection. In some cases, map database 160 may store a sparse data model that includes polynomial representations for certain road features (eg, lane markings) or object trajectories for the host vehicle. Systems and methods for generating such maps are discussed below with reference to FIGS. 8-19.

Image capture devices 122, 124, and 126 may each comprise any type of device suitable for capturing at least one image from the environment. Also, any number of image capture devices may be used to acquire images for input to the image processor. Some embodiments may include only a single image capture device, while other embodiments may include two, three, or even four or more image capture devices. Image capture devices 122, 124, and 126 are further described below with reference to FIGS. 2B-2E.

System 100 or its individual components may be incorporated into a variety of different platforms. In some embodiments, the system 100 may be included on a vehicle 200 as shown in FIG. 2A . For example, vehicle 200 may be equipped with processing unit 110 and any other components of system 100 as described above with respect to FIG. 1 . While in some embodiments vehicle 200 may be equipped with only a single image capture device (eg, camera), in other embodiments, such as those discussed in connection with FIGS. 2B-2E , multiple image capture devices may be used. For example, as shown in FIG. 2A , either of the image capture devices 122 and 124 of the vehicle 200 may be part of an ADAS (Advanced Driver Assistance System) imaging device.

The image capture device included on the vehicle 200 as part of the image capture unit 120 may be located in any suitable location. In some embodiments, as shown in Figures 2A-2E and 3A-3C, the image capture device 122 may be located near the rearview mirror. This location may provide a line of sight similar to that of the driver of the vehicle 200, which may help determine what is and is not visible to the driver. The image capture device 122 may be located anywhere near the rearview mirror, but placing the image capture device 122 on the driver's side of the mirror may further aid in obtaining an image representative of the driver's field of view and/or line of sight.

Other locations of the image capture device of the image acquisition unit 120 may also be used. For example, image capture device 124 may be located on or in a bumper of vehicle 200 . Such a location may be particularly suitable for image capture devices with a wide field of view. The line of sight of the image capture device located on the bumper may not be the same as that of the driver, therefore, the bumper image capture device and the driver may not always see the same objects. Image capture devices such as image capture devices 122, 124, and 126 may also be located at other locations. For example, the image capture device may be located on or in one or both of the side mirrors of the vehicle 200, on the roof of the vehicle 200, on the hood of the vehicle 200, on the trunk of the vehicle 200, on the side of the vehicle 200 , mounted on any window of the vehicle 200 , positioned behind or in front of a window, and mounted in or near a light pattern on the front and/or rear of the vehicle 200 .

In addition to the image capture device, vehicle 200 may include various other components of system 100 . For example, processing unit 110 may be included on vehicle 200, integrated with or separate from an engine control unit (ECU) of the vehicle. Vehicle 200 may also be equipped with a position sensor 130 , such as a GPS receiver, and may also include a map database 160 and storage units 140 and 150 .

As previously mentioned, the wireless transceiver 172 can receive data and/or over one or more networks (eg, cellular networks, the Internet, etc.). For example, wireless transceiver 172 may upload data collected by system 100 to one or more servers, and may download data from one or more servers. Via wireless transceiver 172 , system 100 may, for example, receive periodic or on-demand updates to data stored in map database 160 , memory 140 , and/or memory 150 . Similarly, wireless transceiver 172 may receive any data from system 100 (eg, images captured by image acquisition unit 120 , data received by position sensor 130 or other sensors, vehicle control system, etc.) and/or be processed by processing unit 110 Any data uploaded to one or more servers.

System 100 may upload data to a server (eg, to the cloud) based on privacy level settings. For example, the system 100 may implement privacy level settings to regulate or limit the type of data (including metadata) sent to the server that can uniquely identify the vehicle and/or the driver/owner of the vehicle. Such settings may be set by the user via, for example, wireless transceiver 172 , initialized by factory default settings, or by data received by wireless transceiver 172 .

In some embodiments, the system 100 can upload data according to a "high" privacy level, and with settings set, the system 100 can transmit without any details about the specific vehicle and/or driver/owner Data (such as location information related to routes, captured images, etc.). For example, when uploading data under a "high" privacy setting, the system 100 may not include a vehicle identification number (VIN) or the name of the driver or owner of the vehicle, and may instead transmit data, such as captured images and/or related routes Relevant limited location information.

Other privacy levels are expected. For example, the system 100 may transmit data to the server according to a "medium" privacy level and include additional information not included at a "high" privacy level, such as the make and/or model of the vehicle and/or the type of vehicle (e.g., passenger car , off-road vehicles, trucks, etc.). In some embodiments, system 100 may upload data according to a "low" privacy level. At a "low" privacy level setting, the system 100 may upload data and include sufficient information to uniquely identify a particular vehicle, owner/driver, and/or part or all of a route traveled by the vehicle. Such "low" privacy level data may include, for example, one or more of the following: VIN, driver/owner name, point of origin of the vehicle prior to departure, intended destination of the vehicle, make of the vehicle, and/or Or model, vehicle type, etc.

2A is a schematic side view representation of an exemplary vehicle imaging system consistent with disclosed embodiments. Figure 2B is a schematic top view of the embodiment shown in Figure 2A. As shown in FIG. 2B , the disclosed embodiments may include a vehicle 200 including the system 100 in its body with the first image positioned near the rearview mirror of the vehicle 200 and/or proximate to the driver. Capture device 122 , second image capture device 124 positioned on or in a bumper area of vehicle 200 , such as one of bumper areas 210 , and processing unit 110 .

As shown in FIG. 2C , both image capture devices 122 and 124 may be positioned near a rearview mirror of vehicle 200 and/or in close proximity to the driver. Additionally, although two image capture devices 122 and 124 are shown in Figures 2B and 2C, it should be understood that other embodiments may include more than two image capture devices. For example, in the embodiment shown in FIGS. 2D and 2E , the first, second, and third image capture devices 122 , 124 , 126 are included in the system 100 of the vehicle 200 .

As shown in FIG. 2D , image capture device 122 may be positioned near a rearview mirror of vehicle 200 and/or near the driver, and image capture devices 124 and 126 may be positioned in a bumper area of vehicle 200 (eg, between bumper area 210 ). 1) On or in. And as shown in FIG. 2E , the image capture devices 122 , 124 , and 126 may be positioned near the rearview mirror of the vehicle 200 and/or near the driver's seat. The disclosed embodiments are not limited to any particular number and configuration of image capture devices, and the image capture devices may be positioned in any suitable location within and/or on the vehicle 200 .

It should be understood that the disclosed embodiments are not limited to vehicles and may find application in other contexts. It should also be understood that the disclosed embodiments are not limited to a particular type of vehicle 200, and are applicable to all types of vehicles, including automobiles, trucks, trailers, and other types of vehicles.

First image capture device 122 may comprise any suitable type of image capture device. Image capture device 122 may include an optical axis. In an example, image capture device 122 may include an Aptina M9V024 WVGA sensor with a global shutter. In other embodiments, image capture device 122 may provide a resolution of 1280x960 pixels and may include a rolling shutter. Image capture device 122 may include various optical elements. In some embodiments, for example, one or more lenses may be included to provide a desired focal length and field of view for the image capture device. In some embodiments, image capture device 122 may be associated with a 6mm lens or a 12mm lens. In some embodiments, image capture device 122 may be configured to capture an image having a desired field of view (FOV) 202 , as shown in FIG. 2D . For example, image capture device 122 may be configured to have a regular FOV, such as in the range of 40 degrees to 56 degrees, including 46 degree FOV, 50 degree FOV, 52 degree FOV, or greater. Alternatively, the image capture device 122 may be configured with a narrow FOV in the range of 23 degrees to 40 degrees, such as a 28 degree FOV or a 36 degree FOV. Additionally, image capture device 122 may be configured to have a wide FOV in the range of 100 degrees to 180 degrees. In some embodiments, image capture device 122 may include a wide-angle bumper camera or a camera with an FOV of up to 180 degrees. In some embodiments, image capture device 122 may be a 7.2M pixel image capture device with an aspect ratio of approximately 2:1 (eg, HxV = 3800x1900 pixels) and with a horizontal FOV of approximately 100 degrees. Such an image capture device can be used instead of a three image capture device configuration. In embodiments where an image capture device uses radially symmetric lenses, the vertical FOV of such an image capture device may be significantly less than 50 degrees due to significant lens distortion. For example, such a lens may not be radially symmetric, which would allow a vertical FOV greater than 50 degrees and a horizontal FOV of 100 degrees.

The first image capture device 122 may acquire a plurality of first images of a scene related to the vehicle 200 . Each of the plurality of first images may be acquired as a series of image scanlines, which may be captured using a rolling shutter. Each scan line may include multiple pixels.

The first image capture device 122 may have a scan rate associated with the acquisition of each of the first series of image scanlines. A scan rate may refer to a rate at which an image sensor may acquire image data related to each pixel included in a specific scan line.

Image capture devices 122, 124, and 126 may include any suitable type and number of image sensors, including, for example, CCD sensors or CMOS sensors. In an embodiment, a CMOS image sensor may be used with a rolling shutter such that each pixel in a row is read at a time, and the scanning of the rows is done row by row until the entire image frame has been captured. In some embodiments, rows may be captured sequentially from top to bottom with respect to the frame.

In some embodiments, one or more of the image capture devices disclosed herein (eg, image capture devices 122, 124, and 126) may constitute a high-resolution imager and may have a resolution greater than 5M pixels, 7M pixels, 10M pixels, or larger resolution.

The use of a rolling shutter can cause pixels in different rows to be exposed and captured at different times, which can lead to skew and other image artifacts in the captured image frame. On the other hand, when the image capture device 122 is configured to operate with a global or synchronized shutter, all pixels may be exposed for the same amount of time and during a common exposure time period. As a result, image data in frames collected from a system employing a global shutter represents a snapshot of the entire FOV (such as FOV 202 ) at a particular time. In contrast, in rolling shutter applications, each line in a frame is exposed and data is captured at different times. Therefore, in an image capture device with a rolling shutter, moving objects may appear distorted. This phenomenon will be described in more detail below.

Second image capture device 124 and third image capture device 126 may be any type of image capture device. As with first image capture device 122, each of image capture devices 124 and 126 may include an optical axis. In an embodiment, each of image capture devices 124 and 126 may include an Aptina M9V024 WVGA sensor with a global shutter. Alternatively, each of image capture devices 124 and 126 may include a rolling shutter. Like image capture device 122, image capture devices 124 and 126 may be configured to include various lenses and optical elements. In some embodiments, the lenses associated with image capture devices 124 and 126 may provide a FOV (such as FOVs 204 and 206 ) that is the same as or narrower than the FOV (such as FOV 202 ) associated with image capture device 122 . For example, image capture devices 124 and 126 may have a FOV of 40 degrees, 30 degrees, 26 degrees, 23 degrees, 20 degrees, or less.

Image capture devices 124 and 126 may acquire a plurality of second and third images of a scene related to vehicle 200 . Each of the plurality of second and third images may be acquired as a second and third series of image scanlines, which may be captured using a rolling shutter. Each scan line or row can have multiple pixels. Image capture devices 124 and 126 may have second and third scan rates associated with the acquisition of each image scan line included in the second and third series.

Each image capture device 122 , 124 , and 126 may be positioned in any suitable position and orientation relative to the vehicle 200 . The relative positioning of image capture devices 122, 124, and 126 may be selected to facilitate fusing together the information obtained from the image capture devices. For example, in some embodiments, the FOV associated with image capture device 124 (such as FOV 204 ) may be partially or in part with the FOV associated with image capture device 122 (such as FOV 202 ) and the FOV associated with image capture device 126 (such as FOV 206 ). overlap completely.

Image capture devices 122 , 124 , and 126 may be located on vehicle 200 at any suitable relative height. In one example, there may be a height difference between image capture devices 122, 124, and 126, which may provide sufficient disparity information to enable stereoscopic analysis. For example, as shown in Figure 2A, the two image capture devices 122 and 124 are at different heights. For example, there may also be lateral displacement differences between image capture devices 122, 124, and 126, giving additional disparity information for stereoscopic analysis by processing unit 110. As shown in Figures 2C and 2D, the lateral displacement difference can be represented by dx. In some embodiments, there may be a front-to-back displacement (eg, range displacement) between image capture devices 122 , 124 , and 126 . For example, image capture device 122 may be located 0.5 to 2 meters or more behind image capture device 124 and/or image capture device 126 . This type of displacement can enable one of the image capture devices to cover the potential blind spot of the other image capture device.

Image capture device 122 may have any suitable resolution capability (e.g., the number of pixels associated with image sensor), and the resolution of the image sensor associated with image capture device 122 may be higher, lower, or comparable to that of image capture device 124 and The resolution of the 126 related image sensors is the same. In some embodiments, image sensors associated with image capture device 122 and/or image capture devices 124 and 126 may have a resolution of 640x480, 1024x768, 1280x960, or any other suitable resolution.

The frame rate (eg, the rate at which an image capture device acquires a set of pixel data for one image frame before continuing to capture pixel data associated with the next image frame) is controllable. The frame rate associated with image capture device 122 may be higher, lower, or the same as the frame rate associated with image capture devices 124 and 126 . Frame rates associated with image capture devices 122, 124, and 126 may depend on various factors that may affect frame rate timing. For example, one or more of image capture devices 122, 124, and 126 may include a Optional pixel delay period. Typically, image data corresponding to each pixel may be acquired according to the clock rate of the device (eg, one pixel per clock cycle). Additionally, in embodiments that include rolling shutters, one or more of image capture devices 122, 124, and 126 may include a mechanism for acquiring a pixel associated with a row of image sensors in image capture devices 122, 124, and/or 126. An optional horizontal blanking period to apply before or after the image data. Additionally, one or more of image capture devices 122, 124, and/or 126 may include an optional vertical blanking period applied before or after acquiring image data associated with image frames of image capture devices 122, 124, and 126 .

These timing controls allow the synchronization of the frame rates associated with image capture devices 122, 124, and 126, even though each has a different line scan rate. Additionally, as will be discussed in more detail below, even though the field of view of image capture device 122 is different than the FOVs of image capture devices 124 and 126, these optional timing controls, along with other factors such as image sensor resolution, maximum line scan rate, etc.) may also synchronize image capture from areas where the FOV of image capture device 122 overlaps with one or more FOVs of image capture devices 124 and 126 .

Frame rate timing in image capture devices 122, 124, and 126 may depend on the resolution of the associated image sensor. For example, assuming two devices have similar line scan rates, if one device includes an image sensor with a resolution of 640x480 and the other device includes an image sensor with a resolution of 1280x960, it will take more The sensor acquires frames of image data.

Another factor that may affect the timing of image data acquisition in image capture devices 122, 124, and 126 is the maximum line scan rate. For example, acquiring a line of image data from the image sensors included in image capture devices 122, 124, and 126 will require some minimum amount of time. Assuming no increase in the pixel delay period, the minimum amount of time to acquire a line of image data will be related to the maximum line scan rate for a particular device. A device that offers a higher maximum line scan rate has the potential to provide a higher frame rate than a device with a lower maximum line scan rate. In some embodiments, one or more of image capture devices 124 and 126 may have a higher maximum line scan rate than the maximum line scan rate associated with image capture device 122 . In some embodiments, the maximum line scan rate of image capture device 124 and/or 126 may be 1.25, 1.5, 1.75, or 2 or more times the maximum line scan rate of image capture device 122 .

In another embodiment, image capture devices 122, 124, and 126 may have the same maximum line scan rate, but image capture device 122 may operate at a scan rate less than or equal to its maximum scan rate. The system may be configured such that one or more of image capture devices 124 and 126 operates at a line scan rate equal to that of image capture device 122 . In other examples, the system may be configured such that the line scan rate of image capture device 124 and/or image capture device 126 may be 1.25, 1.5, 1.75, or 2 or more times the line scan rate of image capture device 122 .

In some embodiments, image capture devices 122, 124, and 126 may be asymmetrical. That is, they can include cameras with different fields of view (FOV) and focal lengths. For example, the fields of view of image capture devices 122 , 124 , and 126 may include any desired area relative to the environment of vehicle 200 . In some embodiments, one or more of image capture devices 122 , 124 , and 126 may be configured to acquire image data from the environment in front of vehicle 200 , behind vehicle 200 , to the sides of vehicle 200 , or combinations thereof.

Additionally, the focal length associated with each image capture device 122, 124, and/or 126 may be selectable (eg, by including appropriate lenses, etc.) such that each device captures objects within a desired range of distances relative to the vehicle 200 Image. For example, in some embodiments, image capture devices 122, 124, and 126 may capture images of close-up objects within a few meters of the vehicle. The image capture devices 122, 124 and 126 may also be configured to acquire images of objects at greater distances from the vehicle (eg 25m, 50m, 100m, 150m or more). In addition, the focal lengths of image capture devices 122, 124, and 126 can be selected such that one image capture device (eg, image capture device 122) can acquire images of objects that are relatively close to the vehicle (eg, within 10 m or 20 m), while the other image capture devices Images of objects at greater distances (eg, greater than 20m, 50m, 100m, 150m, etc.) from the vehicle 200 may be acquired by image capture devices (eg, image capture devices 124 and 126).

According to some embodiments, the FOV of one or more image capture devices 122, 124, and 126 may have a wide angle. For example, having a FOV of 140 degrees may be advantageous, particularly for image capture devices 122 , 124 , and 126 that may be used to capture images of the area near vehicle 200 . For example, image capture device 122 may be used to capture images of areas to the right or left of vehicle 200, and in such embodiments, it may be desirable for image capture device 122 to have a wide FOV (eg, at least 140 degrees).

The fields of view associated with each of image capture devices 122, 124, and 126 may depend on the respective focal lengths. For example, as the focal length increases, the corresponding field of view decreases.

Image capture devices 122, 124, and 126 may be configured to have any suitable field of view. In a particular example, image capture device 122 may have a horizontal FOV of 46 degrees, image capture device 124 may have a horizontal FOV of 23 degrees, and image capture device 126 may have a horizontal FOV of between 23 degrees and 46 degrees. In another example, image capture device 122 may have a horizontal FOV of 52 degrees, image capture device 124 may have a horizontal FOV of 26 degrees, and image capture device 126 may have a horizontal FOV of between 26 degrees and 52 degrees. In some embodiments, the ratio of the FOV of image capture device 122 to the FOV of image capture device 124 and/or image capture device 126 may vary from 1.5 to 2.0. In other embodiments, this ratio may vary between 1.25 and 2.25.

System 100 may be configured such that the field of view of image capture device 122 at least partially or completely overlaps the field of view of image capture device 124 and/or image capture device 126 . In some embodiments, system 100 may be configured such that the fields of view of image capture devices 124 and 126 fall within (eg, narrower than) the field of view of image capture device 122 and share a common center therewith, for example. In other embodiments, image capture devices 122, 124, and 126 may capture adjacent FOVs or may have partial overlap in their FOVs. In some embodiments, the fields of view of image capture devices 122 , 124 , and 126 may be aligned such that the center of narrower FOV image capture devices 124 and/or 126 may be located in the lower half of the field of view of wider FOV device 122 .

FIG. 2F is a schematic representation of an exemplary vehicle control system consistent with disclosed embodiments. As shown in FIG. 2F , vehicle 200 may include a throttle system 220 , a braking system 230 and a steering system 240 . System 100 may communicate with one or more of throttle system 220, braking system 230, and steering system 240 via one or more data links (eg, any one or more wired and/or wireless links for transmitting data) One provides input (such as a control signal). For example, based on analysis of images acquired by image capture devices 122, 124, and/or 126, system 100 may provide control signals to one or more of throttle system 220, braking system 230, and steering system 240 to navigate the vehicle 200 (eg, by causing acceleration, turning, lane departure, etc.). Additionally, system 100 may receive input from one or more of throttle system 220, braking system 230, and steering system 24 indicative of the operating conditions of vehicle 200 (eg, speed, whether vehicle 200 is braking and/or turning, etc.) . Further details are provided below in connection with Figures 4-7.

As shown in FIG. 3A , the vehicle 200 may also include a user interface 170 for interacting with a driver or passenger of the vehicle 200 . For example, user interface 170 in a vehicle application may include touch screen 320 , knob 330 , buttons 340 and microphone 350 . A driver or passenger of the vehicle 200 may also use handles (eg, located on or near the steering column of the vehicle 200 including, for example, a turn signal handle), buttons (eg, located on the steering wheel of the vehicle 200 ), etc., to interact with the system 100 . In some embodiments, microphone 350 may be positioned adjacent rearview mirror 310 . Similarly, image capture device 122 may be located near rearview mirror 310 in some embodiments. In some embodiments, the user interface 170 may also include one or more speakers 360 (eg, speakers of a vehicle audio system). For example, system 100 may provide various notifications (eg, alarms) via speaker 360 .

3B-3D are illustrations of an exemplary camera mount 370 configured to be positioned behind a rearview mirror (eg, rearview mirror 310 ) and in close proximity to a vehicle windshield, consistent with disclosed embodiments . As shown in FIG. 3B , camera mount 370 may include image capture devices 122 , 124 , and 126 . Image capture devices 124 and 126 may be located behind an anti-glare cover 380, which may be flush with the vehicle windshield and include a composition of film and/or anti-reflective material. For example, anti-glare cover 380 may be positioned such that the cover is aligned against a vehicle windshield having a matching slope. In some embodiments, for example, each of image capture devices 122, 124, and 126 may be positioned behind anti-glare shield 380, as shown in FIG. 3D. The disclosed embodiments are not limited to any particular configuration of image capture devices 122 , 124 , and 126 , camera mount 370 , and anti-glare shield 380 . FIG. 3C is an illustration of the camera mount 370 shown in FIG. 3B viewed from the front.

Various changes and/or modifications may be made to the foregoing disclosed embodiments, as will be appreciated by those skilled in the art having the benefit of this disclosure. For example, not all components are essential to the operation of system 100 . Furthermore, any components may be located in any suitable portion of system 100, and components may be rearranged in various configurations while providing the functionality of the disclosed embodiments. Accordingly, the foregoing configurations are examples, and regardless of the configurations discussed above, the system 100 may provide a wide range of functionality to analyze the surrounding environment of the vehicle 200 and navigate the vehicle 200 in response to the analysis.

As discussed in further detail below and consistent with the various disclosed embodiments, the system 100 may provide various features related to autonomous driving and/or driver assistance technologies. For example, system 100 may analyze image data, location data (eg, GPS location information), map data, speed data, and/or data from sensors included in vehicle 200 . System 100 may collect data for analysis from, for example, image acquisition unit 120, position sensor 130, and other sensors. Additionally, the system 100 may analyze the collected data to determine whether the vehicle 200 should take a certain action, and then automatically take the determined action without human intervention. For example, system 100 may automatically control the braking, acceleration, and/or steering of vehicle 200 while vehicle 200 is navigating without human intervention (e.g., one or more transmit control signals). Additionally, the system 100 may analyze the collected data and issue warnings and/or alerts to vehicle occupants based on the analysis of the collected data. Additional details regarding various embodiments provided by system 100 are provided below.

forward multiple imaging system

As described above, system 100 may provide driver assistance functionality using a multi-camera system. The multi-camera system may use one or more cameras facing the forward direction of the vehicle. In other embodiments, a multi-camera system may include one or more cameras facing the side of the vehicle or the rear of the vehicle. In one embodiment, for example, system 100 may utilize a dual camera imaging system, where first and second cameras (eg, image capture devices 122 and 124 ) may be located on the front and/or sides of a vehicle (eg, vehicle 200 ). The field of view of the first camera may be larger, smaller, or partially overlap the field of view of the second camera. In addition, the first camera can be connected to the first image processor to perform monocular image analysis on the image provided by the first camera, and the second camera can be connected to the second image processor to perform monocular image analysis on the image provided by the second camera. image analysis. The output (eg processed information) of the first and second image processors may be combined. In some embodiments, the second image processor may receive images from both the first camera and the second camera to perform stereoscopic analysis. In another embodiment, system 100 may use a three-camera imaging system, where each camera has a different field of view. Thus, such a system can make decisions based on information derived from objects located at different distances in front of and to the sides of the vehicle. Reference to monocular image analysis may refer to instances where image analysis is performed based on images captured from a single viewpoint (eg, from a single camera). Stereoscopic image analysis may refer to an instance where image analysis is performed based on two or more images captured with one or more variations of image capture parameters. For example, captured images suitable for performing stereoscopic image analysis may include images captured from two or more different positions, from different fields of view, using different focal lengths, disparity information, and the like.

For example, in one embodiment, system 100 may implement a three-camera configuration using image capture devices 122, 124, and 126. In such a configuration, image capture device 122 may provide a narrow field of view (e.g., 34 degrees or other value selected from a range of approximately 20 to 45 degrees, etc.), and image capture device 124 may provide a wide field of view (e.g., 150 degrees or other values selected from the range of about 100 to about 180 degrees), and image capture device 126 may provide an intermediate field of view (eg, 46 degrees or other values selected from the range of about 35 to about 60 degrees). In some embodiments, image capture device 126 may act as a primary or primary camera. Image capture devices 122, 124, and 126 may be positioned behind rearview mirror 310 and positioned substantially side-by-side (eg, 6 cm apart). Additionally, in some embodiments, one or more of image capture devices 122 , 124 , and 126 may be mounted behind a glare shield 380 flush with the windshield of vehicle 200 , as described above. Such an enclosure can minimize the effect of any reflections in the vehicle on the image capture devices 122, 124, and 126.

In another embodiment, as described above in connection with FIGS. 3B and 3C , a wide field of view camera (such as image capture device 124 in the example above) can be mounted between a narrow field of view and a main field of view camera (such as image capture device 124 in the example above). devices 122 and 126) below. This configuration can provide a free line of sight from the wide field camera. To reduce reflections, the camera may be mounted close to the windshield of the vehicle 200 and a polarizer may be included on the camera to attenuate reflected light.

Three-camera systems can offer certain performance characteristics. For example, some embodiments may include the ability to verify detection of objects by one camera based on detection results from another camera. In the three-camera configuration described above, the processing unit 110 may include, for example, three processing devices (eg, three EyeQ series processor chips as described above), each dedicated to processing images generated by the image capture devices 122, 124 and One or more of the 126 captured images.

In a three-camera system, the first processing device can receive images from the main camera and the narrow field of view camera and perform vision processing on the narrow FOV camera, for example to detect other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other roadways object. Furthermore, the first processing device may calculate the disparity of pixels between the images from the main camera and the narrow camera and create a 3D reconstruction of the environment of the vehicle 200 . The first processing device may then combine the 3D reconstruction with 3D map data or 3D information calculated based on information from another camera.

A second processing device may receive images from the main camera and perform vision processing to detect other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Additionally, the second processing device may calculate camera displacement and based on this displacement, calculate the disparity of pixels between successive images and create a 3D reconstruction of the scene (eg structures from motion). The second processing device may send the structure from the motion-based 3D reconstruction to the first processing device for combination with the stereoscopic 3D image.

A third processing device may receive images from the wide FOV camera and process the images to detect vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. The third processing device may further execute additional processing instructions to analyze the images to identify objects moving in the images, such as vehicles changing lanes, pedestrians, etc.

In some embodiments, having image-based information streams that are captured and processed independently may provide an opportunity to provide redundancy in the system. Such redundancy may include, for example, using a first image capture device and images processed from that device to verify and/or supplement information obtained by capturing and processing image information from at least a second image capture device.

In some embodiments, system 100 may use two image capture devices (eg, image capture devices 122 and 124 ) in providing navigation assistance to vehicle 200 and use a third image capture device (eg, image capture device 126 ) to provide redundancy. and verify the analysis of data received from the other two image capture devices. For example, in such a configuration, image capture devices 122 and 124 may provide images for stereoscopic analysis by system 100 to navigate vehicle 200, while image capture device 126 may provide images for monocular analysis by system 100 to provide redundancy. and verify information obtained based on images captured from image capture device 122 and/or image capture device 124 . That is, image capture device 126 (and corresponding processing device) may be considered as providing a redundant subsystem to provide checking of analyzes derived from image capture devices 122 and 124 (e.g., to provide automatic emergency braking (AEB) system). Additionally, in some embodiments, the received data may be supplemented based on information received from one or more sensors (eg, radar, lidar, acoustic wave sensors, information received from one or more transceivers external to the vehicle, etc.) redundancy and validation.

Those skilled in the art will recognize that the above camera configurations, camera placements, number of cameras, camera locations, etc. are examples only. These and other components described with respect to the overall system can be assembled and used in various different configurations without departing from the scope of the disclosed embodiments. Below are more details on the use of multi-camera systems to provide driver assistance and/or autonomous functions.

4 is an example functional block diagram of memory 140 and/or 150, which may be stored/programmed with instructions for performing one or more operations consistent with the disclosed embodiments. Although the following refers to memory 140 , those skilled in the art will recognize that instructions may be stored in memory 140 and/or 150 .

As shown in FIG. 4 , the memory 140 may store a monocular image analysis module 402 , a stereoscopic image analysis module 404 , a velocity and acceleration module 406 , and a navigation response module 408 . The disclosed embodiments are not limited to any particular configuration of memory 140 . Also, the application processor 180 and/or the image processor 190 may execute instructions stored in any one of the modules 402 , 404 , 406 , and 408 included in the memory 140 . Those skilled in the art will appreciate that references to processing unit 110 in the following discussion may refer to application processor 180 and image processor 190 individually or collectively. Accordingly, steps of any of the following processes may be performed by one or more processing devices.

In an embodiment, monocular image analysis module 402 may store instructions (such as computer vision software) that, when executed by processing unit 110 , perform monocular analysis on a set of images acquired by one of image capture devices 122 , 124 , and 126 . image analysis. In some embodiments, processing unit 110 may combine information from a set of images with additional sensory information (eg, information from radar, lidar, etc.) to perform monocular image analysis. As described below in connection with FIGS. 5A-5D , monocular image analysis module 402 may include instructions for detecting a set of features within the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic Signal lights, hazardous objects, and any other features relevant to the vehicle's environment. Based on this analysis, system 100 (eg, via processing unit 110 ) may induce one or more navigational responses in vehicle 200 , such as turns, lane departures, changes in acceleration, etc., as described below in connection with navigational response module 408 .

In one embodiment, the stereoscopic image analysis module 404 may store instructions (such as computer vision software) that when executed by the processing unit 110 analyze the combined image capture devices selected from any of the image capture devices 122 , 124 , and 126 . The acquired first and second sets of images are subjected to stereoscopic image analysis. In some embodiments, the processing unit 110 may combine information from the first and second sets of images with additional sensory information (eg, information from radar) to perform stereoscopic image analysis. For example, stereoscopic image analysis module 404 may include instructions for performing stereoscopic image analysis based on the first set of images acquired by image capture device 124 and the second set of images acquired by image capture device 126 . As described below in connection with FIG. 6, stereo image analysis module 404 may include instructions for detecting a set of features within the first and second sets of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps , traffic lights, dangerous objects, etc. Based on this analysis, the processing unit 110 may induce one or more navigational responses in the vehicle 200 , such as turns, lane departures, changes in acceleration, etc., as described below in connection with the navigational response module 408 . Additionally, in some embodiments, the stereoscopic image analysis module 404 may be implemented with a trained system (such as a neural network or a deep neural network) or an untrained system (such as may be configured to use computer vision algorithms to detect and/or label systems from which sensory information is captured and processed) related to objects in the environment). In an embodiment, the stereoscopic image analysis module 404 and/or other image processing modules may be configured to use a combination of trained and untrained systems.

In an embodiment, the velocity and acceleration module 406 may store software configured to analyze data received from one or more computing and electromechanical devices in the vehicle 200 configured to cause the velocity and acceleration of the vehicle 200 to and/or changes in acceleration. For example, processing unit 110 may execute instructions associated with velocity and acceleration module 406 to calculate a target velocity of vehicle 200 based on data derived from execution of monocular image analysis module 402 and/or stereo image analysis module 404 . Such data may include, for example, target position, velocity and/or acceleration, position and/or velocity of vehicle 200 relative to nearby vehicles, pedestrians or road objects, position information of vehicle 200 relative to lane markings of the road, and the like. Additionally, the processing unit 110 may calculate the vehicle 200 based on sensory input (eg, information from radar) and input from other systems of the vehicle 200 (eg, the throttle system 220 , the braking system 230 and/or the steering system 240 of the vehicle 200 ). target speed. Based on the calculated target speed, the processing unit 110 may transmit electronic signals to the throttle system 220, the braking system 230 and/or the steering system 240 of the vehicle 200 to respond by, for example, physically applying the brakes or releasing the accelerator of the vehicle 200. Trigger changes in velocity and/or acceleration.

In an embodiment, navigation response module 408 may store software executable by processing unit 110 to determine a desired navigation response based on data derived from execution of monocular image analysis module 402 and/or stereo image analysis module 404 . Such data may include position and velocity information related to nearby vehicles, pedestrians and road objects, target position information of the vehicle 200, and the like. Additionally, in some embodiments, the navigation response may be based (in part or in whole) on map data, a predetermined location of the vehicle 200, and/or detected from the execution of the monocular image analysis module 402 and/or the stereo image analysis module 404 The relative velocity or relative acceleration between the vehicle 200 and one or more objects. The navigation response module 408 may also determine desired navigational responses based on sensory input (eg, information from radar) and input from other systems of the vehicle 200 (such as the throttle system 220, the braking system 230, and/or the steering system 240 of the vehicle 200). Navigation responsive. Based on the desired navigational response, the processing unit 110 may transmit electronic signals to the throttle system 220, the braking system 230 and the steering system 240 of the vehicle 200 to trigger the desired navigational response, for example by turning the steering wheel of the vehicle 200 to achieve a predetermined angle rotation. In some embodiments, the processing unit 110 may use the output of the navigation response module 408 (eg, the desired navigation response) as input to the execution of the speed and acceleration module 406 to calculate the speed change of the vehicle 200 .

Furthermore, any of the modules disclosed herein (eg, modules 402, 404, and 406) can implement techniques related to a trained system (such as a neural network or deep neural network) or an untrained system.

FIG. 5A is a flowchart illustrating an example process 500A for eliciting one or more navigational responses based on monocular image analysis, consistent with disclosed embodiments. At step 510 , the processing unit 110 may receive a plurality of images via the data interface 128 between the processing unit 110 and the image acquisition unit 120 . For example, a camera included in image acquisition unit 120 (such as image capture device 122 having field of view 202 ) can capture multiple images of an area in front of vehicle 200 (eg, to the sides or rear of the vehicle) and transmit them via a data connection (eg, digital, wired, USB, wireless, bluetooth, etc.) to the processing unit 110. At step 520, the processing unit 110 may execute the monocular image analysis module 402 to analyze the plurality of images, as described in more detail below in connection with FIGS. 5B-5D. By performing this analysis, processing unit 110 may detect a set of features within the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, and the like.

In step 520 , the processing unit 110 may also execute the monocular image analysis module 402 to detect various road hazards, such as parts of truck tires, fallen road signs, loose goods, small animals, and the like. Road hazards can vary in structure, shape, size, and color, which can make detecting such hazards more challenging. In some embodiments, the processing unit 110 may execute the monocular image analysis module 402 to perform multi-frame analysis on multiple images to detect road hazards. For example, the processing unit 110 may estimate camera motion between consecutive image frames and calculate the disparity of pixels between frames to construct a 3D map of the road. The processing unit 110 may then use the 3D map to detect the road surface and hazards present on the road surface.

At step 530 , the processing unit 110 may execute the navigation response module 408 to induce one or more navigation responses in the vehicle 200 based on the analysis performed at step 520 and the techniques described above in connection with FIG. 4 . Navigation responses may include, for example, turns, lane departures, changes in acceleration, and the like. In some embodiments, processing unit 110 may use data derived from execution of velocity and acceleration module 406 to induce one or more navigational responses. Additionally, multiple navigation responses can occur simultaneously, sequentially, or any combination thereof. For example, the processing unit 110 may cause the vehicle 200 to move across a lane and then be accelerated by, for example, sequentially transmitting control signals to the steering system 240 and the throttle system 220 of the vehicle 200 . Alternatively, the processing unit 110 may brake the vehicle 200 while changing lanes by, for example, simultaneously transmitting control signals to the braking system 230 and the steering system 240 of the vehicle 200 .

FIG. 5B is a flowchart illustrating an example process 500B for detecting one or more vehicles and/or pedestrians in a set of images, consistent with the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to implement process 500B. At step 540, the processing unit 110 may determine a set of candidate objects representing possible vehicles and/or pedestrians. For example, processing unit 110 may scan one or more images, compare the images to one or more predetermined patterns, and identify within each image a possible location that may contain an object of interest such as a vehicle, pedestrian, or part thereof. The predetermined pattern can be designed to achieve a high "false hit" rate and a low "miss" rate. For example, the processing unit 110 may use a low threshold similar to a predetermined pattern to identify candidate objects as possible vehicles or pedestrians. Doing so may allow the processing unit 110 to reduce the likelihood of missing (eg, not recognizing) candidates representing vehicles or pedestrians.

At step 542, the processing unit 110 may filter the set of candidate objects based on the classification criteria to exclude certain candidates (eg, irrelevant or less relevant objects). Such criteria may be derived from various attributes associated with object types stored in a database (eg, the database stored in memory 140). Attributes may include the object's shape, size, texture, location (eg, relative to vehicle 200 ), and the like. Accordingly, processing unit 110 may use one or more sets of criteria to reject false candidates from the set of candidates.

In step 544, the processing unit 110 may analyze the plurality of frames of the image to determine whether the objects in the set of candidate objects represent vehicles and/or pedestrians. For example, the processing unit 110 may track a detected candidate object over successive frames and accumulate frame-by-frame data (eg, size, position relative to the vehicle 200 , etc.) related to the detected object. Additionally, the processing unit 110 may estimate parameters of a detected object and compare the object's frame-by-frame position data with the predicted position.

In step 546, the processing unit 110 may construct a set of measurements for the detected object. For example, such measurements may include position, velocity, and acceleration values (relative to vehicle 200 ) associated with detected objects. In some embodiments, the processing unit 110 may be based on estimation techniques such as Kalman filtering or linear quadratic estimation (LQE) using a series of time-based observations and/or based on the available modeling data for bicycles, road signs, etc.) to construct measurements. Kalman filtering may be based on a measure of object scale, where the scale measure is proportional to the time to collision (eg, the amount of time it took for the vehicle 200 to reach the object). Thus, by performing steps 540-546, the processing unit 110 can identify vehicles and pedestrians appearing within the set of captured images, and derive information (eg, position, velocity, size) related to the vehicles and pedestrians. Based on the identification and derived information, processing unit 110 may induce one or more navigational responses in vehicle 200, as described above in connection with FIG. 5A.

At step 548, the processing unit 110 may perform an optical flow analysis of one or more images to reduce the likelihood of detecting a "false hit" and missing a candidate object representing a vehicle or pedestrian. For example, optical flow analysis may refer to analyzing motion patterns relative to the vehicle 200 in one or more images related to other vehicles and pedestrians, which are distinct from road surface motion. The processing unit 110 may calculate the motion of the candidate object by observing different positions of the object on a plurality of image frames captured at different times. The processing unit 110 may use the position and time values as input to a mathematical model to calculate the motion of the candidate object. Therefore, optical flow analysis may provide another method of detecting vehicles and pedestrians near the vehicle 200 . The processing unit 110 may perform optical flow analysis in conjunction with steps 540 - 546 to provide redundancy for detecting vehicles and pedestrians and increase the reliability of the system 100 .

FIG. 5C is a flowchart illustrating an exemplary process 500C for detecting road markings and/or lane geometry information in a set of images, consistent with the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to implement process 500C. At step 550, the processing unit 110 may detect a set of objects by scanning the one or more images. To detect segments of lane markings, lane geometry information, and other relevant road markings, processing unit 110 may filter the set of objects to exclude those determined to be irrelevant (eg, small potholes, small rocks, etc.). In step 552, the processing unit 110 may group together the segments detected in step 550 belonging to the same road marking or lane marking. Based on this grouping, the processing unit 110 may develop a model, such as a mathematical model, representing the detected segments.

At step 554, the processing unit 110 may construct a set of measurements related to the detected segments. In some embodiments, the processing unit 110 may create a projection of the detected segment from the image plane onto the real world plane. The projection can be characterized using a cubic polynomial with coefficients corresponding to physical properties such as the detected road's position, slope, curvature and curvature derivative. When generating the projections, the processing unit 110 may take into account changes in the road surface as well as pitch and roll rates associated with the vehicle 200 . Furthermore, the processing unit 110 may model the road elevation by analyzing the positional and motion cues present on the road surface. Additionally, the processing unit 110 may estimate pitch and roll rates associated with the vehicle 200 by tracking a set of feature points in one or more images.

In step 556, the processing unit 110 may perform multi-frame analysis by, for example, tracking the detected segments over successive image frames and accumulating frame-by-frame data related to the detected segments. As the processing unit 110 performs multi-frame analysis, the set of measurements constructed at step 554 may become more reliable and correlated with higher and higher confidence levels. Thus, by performing steps 550, 552, 554 and 556, the processing unit 110 may identify road markings appearing within the set of captured images and derive lane geometry information. Based on the identification and derived information, processing unit 110 may induce one or more navigational responses in vehicle 200, as described above in connection with FIG. 5A.

In step 558, the processing unit 110 may consider additional sources of information to further develop a safety model of the vehicle 200 in its surroundings. The processing unit 110 may use the safety model to define situations in which the system 100 may perform autonomous control of the vehicle 200 in a safe manner. To develop the safety model, in some embodiments, the processing unit 110 may take into account the position and motion of other vehicles, detected road edges and obstacles, and/or the general road shape extracted from map data, such as data from the map database 160 describe. By taking into account additional sources of information, the processing unit 110 may provide redundancy for detecting road markings and lane geometry and increase the reliability of the system 100 .

FIG. 5D is a flowchart illustrating an exemplary process 500D for detecting traffic lights in a set of images, consistent with the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to implement process 500D. At step 560, the processing unit 110 may scan the set of images and identify objects that appear in the images at locations that may contain traffic lights. For example, the processing unit 110 may filter the identified objects to construct a set of candidate objects, excluding those objects that are less likely to correspond to traffic lights. Filtering may be based on various attributes related to traffic lights, such as shape, size, texture, location (eg, relative to vehicle 200 ), and the like. Such attributes may be based on multiple instances of traffic lights and traffic control signals and stored in a database. In some embodiments, the processing unit 110 may perform a multi-frame analysis on the set of candidate objects reflecting possible traffic lights. For example, the processing unit 110 may track candidate objects over successive image frames, estimate the real world positions of the candidate objects, and filter out those objects that are moving (which are unlikely to be traffic lights). In some embodiments, the processing unit 110 may perform a color analysis on the candidate objects and identify the relative position of the detected color that occurs inside the likely traffic signal.

At step 562, the processing unit 110 may analyze the geometry of the intersection. The analysis can be based on any combination of: (i) the number of lanes detected on either side of the vehicle 200, (ii) markers detected on the road (such as arrow markers), and (iii) data from map data (such as from The intersection description extracted from the data of the map database 160). The processing unit 110 may use information derived from the execution of the monocular analysis module 402 for the analysis. Furthermore, the processing unit 110 may determine a correspondence between traffic lights detected at step 560 and lanes present near the vehicle 200 .

At step 564, the processing unit 110 may update the confidence scores associated with the analyzed intersection geometry and detected traffic lights as the vehicle 200 approaches the intersection. For example, the estimated number of traffic lights present at an intersection may affect the confidence level compared to the actual number present at the intersection. Thus, based on the degree of confidence, the processing unit 110 may delegate control to the driver of the vehicle 200 in order to improve the safety situation. By performing steps 560, 562, and 564, processing unit 110 may identify traffic lights appearing within the set of captured images and analyze intersection geometry information. Based on the identification and analysis, the processing unit 110 may cause one or more navigational responses in the vehicle 200, as described above in connection with FIG. 5A.

FIG. 5E is a flowchart illustrating an example process 500E for eliciting one or more navigational responses in a vehicle 200 based on a vehicle path, consistent with disclosed embodiments. At step 570 , the processing unit 110 may construct an initial vehicle route associated with the vehicle 200 . The vehicle path may be represented using a set of points expressed in coordinates (x,z), and the distance di between two points in the set may fall in the range of 1 to 5 meters. In an embodiment, the processing unit 110 may use two polynomials, such as left and right road polynomials, to construct the initial vehicle path. The processing unit 110 may calculate the geometric midpoint between the two polynomials and offset each point included in the resultant vehicle path by a predetermined offset (e.g., smart lane offset), if any (zero offset The amount of displacement may correspond to driving in the middle of the lane). The offset can be in a direction perpendicular to the segment between any two points in the vehicle's path. In another embodiment, the processing unit 110 may use a polynomial and the estimated lane width to offset each point of the vehicle's path by the estimated lane width plus half a predetermined offset (e.g., smart lane offset) .

At step 572 , the processing unit 110 may update the vehicle route constructed at step 570 . The processing unit 110 may use a higher resolution to reconstruct the vehicle path constructed at step 570 such that the distance dk between two points in the set of points representing the vehicle path is smaller than the aforementioned distance di. For example, the distance dk may fall within the range of 0.1 to 0.3 meters. The processing unit 110 may reconstruct the vehicle path using a parabolic spline algorithm, which may generate a cumulative distance vector S corresponding to the total length of the vehicle path (ie based on the set of points representing the vehicle path).

At step 574 , the processing unit 110 may determine a forward-sight point (expressed as (xl,zl) in coordinates) based on the updated vehicle path constructed at step 572 . The processing unit 110 may extract the foresight point from the cumulative distance vector S, and the foresight point may be related to the foresight distance and the foresight time. The look-ahead distance, the lower limit of which may range between 10 and 20 meters, may be calculated as the product of the vehicle 200 speed and the look-ahead time. For example, as the speed of the vehicle 200 decreases, the look-ahead distance may also decrease (eg, until it reaches a lower limit). The look-ahead time, which may be in the range of 0.5 to 1.5 seconds, may be inversely proportional to the gain of one or more control loops related to causing a navigation response in the vehicle 200 , such as a heading error tracking control loop. For example, the gain of the heading error tracking control loop may depend on the bandwidth of the yaw rate loop, steering actuator loop, vehicle lateral dynamics, etc. Therefore, the higher the gain of the heading error tracking control loop, the shorter the look-ahead time.

At step 576 , the processing unit 110 may determine a heading error and a yaw rate command based on the foresight point determined at step 574 . The processing unit 110 may determine the heading error by calculating the arc tangent of the foresight point, eg arctan(x _l /z _l ). The processing unit 110 may determine the yaw rate command as the product of the heading error and the advanced control gain. If the look-ahead distance is not at the lower limit, the advanced control gain can be equal to: (2/look-ahead time). Otherwise, the advanced control gain may be equal to: (2*vehicle speed 200/look ahead distance).

FIG. 5F is a flowchart illustrating an example process 500F for determining whether a lead vehicle is changing lanes, consistent with the disclosed embodiments. At step 580 , the processing unit 110 may determine navigation information related to a lead vehicle (eg, a vehicle traveling ahead of the vehicle 200 ). For example, processing unit 110 may determine the position, velocity (eg, direction and velocity), and/or acceleration of the lead vehicle using the techniques described above in connection with FIGS. 5A and 5B . Processing unit 110 may also use the techniques described above in connection with FIG. 5E to determine one or more road polynomials, forward sight points (associated with vehicle 200), and/or snail trails (e.g., a set of points describing the path traveled by the leading vehicle) .

At step 582 , the processing unit 110 may analyze the navigation information determined at step 580 . In an embodiment, the processing unit 110 may calculate the distance between the snail trail and the road polynomial (eg along the trail). If this distance along the trail varies by more than a predetermined threshold (e.g. 0.1 to 0.2 meters on straight roads, 0.3 to 0.4 meters on moderately curved roads, 0.5 to 0.6 meters on roads with sharp turns), then The processing unit 110 may determine that the lead vehicle may be changing lanes. In the event that multiple vehicles are detected traveling in front of the vehicle 200, the processing unit 110 may compare the snail trails associated with each vehicle. Based on this comparison, the processing unit 110 may determine that a vehicle whose snail trail does not match those of other vehicles may be changing lanes. The processing unit 110 may additionally compare the curvature of the snail trail (associated with the lead vehicle) with the expected curvature of the road segment in which the lead vehicle is traveling. The expected curvature may be extracted from map data (eg, from map database 160 ), road polynomials, snail trails of other vehicles, prior knowledge about the road, and the like. If the difference between the curvature of the snail trail and the expected curvature of the road segment exceeds a predetermined threshold, the processing unit 110 may determine that the leading vehicle may be changing lanes.

In another embodiment, the processing unit 110 may compare the instantaneous position of the leading vehicle with the forward sight point (related to the vehicle 200 ) within a certain period of time (eg, 0.5 to 1.5 seconds). If the distance between the instantaneous position of the leading vehicle and the forward-sight point changes within a certain period of time, and the sum of the cumulative changes exceeds a predetermined threshold (for example, 0.3 to 0.4 meters on a straight road, 0.7 to 0.7 meters on a moderately curved road 0.8 meters, 1.3 to 1.7 meters on a road with a sharp turn), the processing unit 110 may determine that the leading vehicle may be changing lanes. In another embodiment, the processing unit 110 may analyze the geometry of the snail trail by comparing the lateral distance traveled along the trail with the expected curvature of the snail trail. The expected radius of curvature can be determined according to the following calculation: (δ _z ² +δ _x ² )/2/(δ _x ), where δ _x represents the lateral distance traveled and δ _z represents the longitudinal distance traveled. If the difference between the lateral distance traveled and the expected curvature exceeds a predetermined threshold (eg, 500 to 700 meters), the processing unit 110 may determine that the lead vehicle may be changing lanes. In another embodiment, the processing unit 110 may analyze the position of the leading vehicle. If the position of the lead vehicle obscures the road polynomial (eg, the lead vehicle is overlaid on top of the road polynomial), the processing unit 110 may determine that the lead vehicle may be changing lanes. Where the lead vehicle is positioned such that another vehicle is detected in front of the lead vehicle and the snail trails of the two vehicles are not parallel, the processing unit 110 may determine that the (closer) lead vehicle may be changing lanes.

At step 584 , the processing unit 110 may determine whether the lead vehicle 200 is changing lanes based on the analysis performed at step 582 . For example, processing unit 110 may make the determination based on a weighted average of the individual analyzes performed at step 582 . For example, under such an approach, a decision made by the processing unit 110 based on a particular type of analysis that the lead vehicle may be changing lanes may be given a value of "1" (and a "0" to indicate a determination that the lead vehicle is unlikely to be changing lanes ). Different analyzes performed at step 582 may be assigned different weights, and the disclosed embodiments are not limited to any particular combination of analyzes and weights.

FIG. 6 is a flowchart illustrating an example process 600 for eliciting one or more navigational responses based on stereoscopic image analysis, consistent with disclosed embodiments. At step 610 , the processing unit 110 may receive the first and second plurality of images via the data interface 128 . For example, a camera included in image acquisition unit 120 , such as image capture devices 122 and 124 having fields of view 202 and 204 , can capture first and second plurality of images of the area in front of vehicle 200 and transmit them via a digital connection (eg, USB , wireless, Bluetooth, etc.) transmit them to the processing unit 110. In some embodiments, processing unit 110 may receive the first and second plurality of images via two or more data interfaces. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

At step 620, the processing unit 110 may execute the stereo image analysis module 404 to perform stereo image analysis on the first and second plurality of images to create a 3D map of the road ahead of the vehicle and detect features within the images, such as lane markings, vehicles, Pedestrians, road signs, highway exit ramps, traffic lights, road hazards, etc. Stereoscopic image analysis can be performed in a manner similar to the steps described above in connection with Figures 5A-5D. For example, processing unit 110 may execute stereoscopic image analysis module 404 to detect candidate objects (e.g., vehicles, pedestrians, road markings, traffic lights, road hazards, etc.) within the first and second plurality of images, filtering out subgroups according to respective criteria candidate objects, and perform multi-frame analysis, construct measurements, and determine confidence for the remaining candidates. In performing the steps described above, the processing unit 110 may take into account information from the first and second plurality of images, rather than information from only one set of images. For example, processing unit 110 may analyze differences in pixel-level data (or other subsets of data in the two streams of captured images) of candidate objects appearing in the first and second plurality of images. As another example, the processing unit 110 may estimate candidate objects by observing that an object appears in one of the multiple images but not the other or relative to other differences that may exist relative to objects appearing in the two image streams (e.g., relative position and/or velocity of the vehicle 200). For example, position, velocity and/or acceleration relative to vehicle 200 may be determined based on the trajectory, position, motion characteristics, etc. of features associated with objects appearing in one or both of the image streams.

At step 630 , the processing unit 110 may execute the navigation response module 408 to induce one or more navigation responses in the vehicle 200 based on the analysis performed at step 620 and the techniques described above in connection with FIG. 4 . Navigational responses may include, for example, turns, lane departures, changes in acceleration, changes in speed, braking, and the like. In some embodiments, processing unit 110 may use data derived from execution of velocity and acceleration module 406 to induce one or more navigational responses. Additionally, multiple navigation responses can occur simultaneously, sequentially, or any combination thereof.

FIG. 7 is a flowchart illustrating an exemplary process 700 for eliciting one or more navigational responses based on the analysis of three sets of images, consistent with the disclosed embodiments. At step 710 , the processing unit 110 may receive the first, second and third plurality of images via the data interface 128 . For example, a camera included in image acquisition unit 120 (such as image capture devices 122 , 124 , and 126 having fields of view 202 , 204 , and 206 ) may capture first, second, and a third plurality of images and transmit them to the processing unit 110 via a digital connection (eg, USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive the first, second and third plurality of images via three or more data interfaces. For example, each of the image capture devices 122 , 124 , 126 may have an associated data interface for transferring data to the processing unit 110 . The disclosed embodiments are not limited to any particular data interface configuration or protocol.

At step 720, the processing unit 110 may analyze the first, second, and third plurality of images to detect features within the images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, road hazards, etc. . Analysis can be performed in a manner similar to the steps described above in connection with FIGS. 5A-5D and 6 . For example, processing unit 110 may perform monocular image analysis (eg, as performed by monocular image analysis module 402 and based on the steps described above in connection with FIGS. 5A-5D ) on each of the first, second, and third plurality of images. Alternatively, processing unit 110 may perform stereoscopic image analysis on the first and second plurality of images, the second and third plurality of images, and/or the first and third plurality of images (e.g., via the performed and based on the steps described above in connection with FIG. 6). Processed information corresponding to the analysis of the first, second and/or third plurality of images may be combined. In some embodiments, processing unit 110 may perform a combination of monocular and stereoscopic image analysis. For example, processing unit 110 may perform monocular image analysis on the first plurality of images (e.g., by performance of monocular image analysis module 402) and stereoscopic image analysis on the second and third plurality of images (e.g., by performance of stereo image analysis module 404). implement). The configuration of image capture devices 122, 124, and 126, including their respective positions and fields of view 202, 204, and 206, may affect the type of analysis performed on the first, second, and third plurality of images. The disclosed embodiments are not limited to a particular configuration of image capture devices 122, 124, and 126 or the type of analysis performed on the first, second, and third plurality of images.

In some embodiments, processing unit 110 may perform tests on system 100 based on the images acquired and analyzed at steps 710 and 720 . Such testing may provide an indication of the overall performance of system 100 for certain configurations of image capture devices 122 , 124 , and 126 . For example, the processing unit 110 may determine a "false hit" (eg, a situation where the system 100 incorrectly determines the presence of a vehicle or pedestrian) and a "miss" ratio.

At step 730 , the processing unit 110 may induce one or more navigational responses in the vehicle 200 based on information derived from two of the first, second and third plurality of images. The selection of two of the first, second and third plurality of images may depend on various factors such as the number, type and size of objects detected in each of the plurality of images. The processing unit 110 may also be based on the image quality and resolution, the effective field of view reflected in the image, the number of frames captured, the extent to which one or more objects of interest are actually present in the frame (e.g., the number of frames in which the object appears) percentage, proportion of objects appearing in each such frame, etc.), etc.)

In some embodiments, processing unit 110 may select to derive information from two of the first, second, and third plurality of images by determining the degree to which information derived from one image source agrees with information derived from the other image source. Information. For example, processing unit 110 may combine the processed information derived from each of image capture devices 122, 124, and 126 (whether by monocular analysis, stereoscopic analysis, or any combination of the two) and determine Consistent visual indicators (eg, lane markings, detected vehicles and their positions and/or paths, detected traffic lights, etc.) on images captured by capture devices 122 , 124 , and 126 . The processing unit 110 may also exclude inconsistent information on the captured image (eg, vehicles changing lanes, lane models indicating vehicles are too close to the vehicle 200 , etc.). Accordingly, the processing unit 110 may select information derived from two of the first, second and third plurality of images based on the determination of consistent and inconsistent information.

Navigation responses may include, for example, turns, lane departures, changes in acceleration, and the like. Processing unit 110 may cause one or more navigational responses based on the analysis performed at step 720 and the techniques described above in connection with FIG. 4 . Processing unit 110 may also use data derived from execution of velocity and acceleration module 406 to induce one or more navigational responses. In some embodiments, the processing unit 110 may, based on the relative position, relative velocity and/or relative acceleration between the vehicle 200 and an object detected within any of the first, second and third plurality of images, cause One or more navigation responses. Multiple navigation responses can occur simultaneously, sequentially, or any combination thereof.

Imaging System with Crossed Field of View

In some embodiments, vehicle 200 may include an imaging system. The imaging system may include an imaging module configured to be mounted on or within a host vehicle, eg, an autonomous vehicle such as vehicle 200 . As used herein, the term "imaging module" refers to a structure that houses and orients at least one camera. The imaging module may include additional hardware and wiring depending on the type of camera employed. The imaging module may in turn be coupled to a mounting assembly configured to attach the imaging module to the vehicle such that at least one camera housed within the imaging module faces outward relative to the vehicle, such as vehicle 200 . As used herein, the term “mount assembly” refers to any hardware or device configured to couple a mount module to a vehicle, such as vehicle 200 . The mounting assembly may include mounting brackets, mounting suction couplings, mounting frames, mounting adhesives, quick/connect couplings, etc. as understood by those of ordinary skill in the art.

Throughout this disclosure, the term "intersection" is sometimes used to have a specific geometric meaning as understood by those of ordinary skill in the art of cameras and camera optics. For example, the term "intersection" does not necessarily denote an actual physical intersection, or even an intersection of projections, but rather an overlap of at least two projections in at least one plane. This is especially true for vector-based optical axes that are parallel to the ground and originate from alternative light sources at different heights (measured from the ground). For example, consider a first vector-based optical axis projecting outward and parallel to the ground at a height of 1 meter and a second vector-based optical axis projecting outward and parallel to the ground at a height of 1.2 meters. According to this example, a first vector-based optical axis may "cross" a second vector-based optical axis when projected onto a plan view, even though the vectors do not necessarily intersect.

In other words, if the optical axis based on the first 3D vector is projected as the first 2D optical axis (or first line) and the optical axis based on the second 3D vector is projected as the second 2D optical axis (or second line ) and viewing them from a planar perspective (or another two-dimensional perspective), the first and second optical axes appear to intersect each other at a "point of intersection". In addition, it should be understood that the "intersection point" is derived from geometric principles, not necessarily the actual intersection of two optical axes in three-dimensional space.

Another way to understand "intersection point" is that it is the position of a point on a plane that intersects a first vector-based optical axis and a second vector-based optical axis derived from geometrical principles. Similarly, "intersection plane" is to be understood as a geometric plane intersecting at least two vector-based optical axes. Thus, an "intersection plane" comprises a respective "intersection point" corresponding to the intersection of the projections of at least two optical axes.

In some embodiments, an imaging system may include at least two cameras having respective fields of view that overlap each other and form a combined field of view. In some embodiments, an imaging system may include three or more cameras having respective fields of view that intersect at least one other camera and form a combined field of view.

In some embodiments, the mounting assembly may be configured to attach the imaging module to the vehicle such that the camera faces outward relative to the vehicle. In some embodiments, the mounting assembly may also orient the cameras so that they face outward relative to the vehicle and parallel to the ground. The mounting assembly may be configured to attach the imaging module to an interior window of the vehicle or another component of the vehicle, such as a bumper, pillar, headlight, taillight, or the like. In some embodiments, the mounting assembly may also be configured to compensate for window tilt.

In some embodiments, an imaging system may include a wiper assembly having a wiper blade configured to remove obstructions from the fields of view of the respective cameras. The wiper assembly may also include timers, sensors and motors configured to control the wiper blade as desired.

In some embodiments, the imaging system may include a glare screen or filter. Glare screens or filters can improve the performance of imaging systems by reducing glare from incident light caused by the inclination (eg, slope) of the window. In addition, the anti-glare shield can be configured to provide an aperture to the camera, thereby increasing the depth of field, resulting in multiple different objects at a wide range of distances remaining in focus.

Figure 8 is a schematic representation of an embodiment of an imaging system with two cameras. This exemplary embodiment may include at least a first camera 805 and a second camera 809 . Although two cameras are shown in FIG. 8, in some embodiments, the imaging system may include more than two cameras (eg, three cameras, four cameras, five cameras, etc.). In some embodiments, cameras (eg, first camera 805 and second camera 809 ) may share one or more features of image capture devices 122 , 124 , and 126 as described above.

As shown in Figure 8, the first camera 805 has a first field of view with an optical axis 805a, and the second camera 809 has a second field of view with an optical axis 809a. According to some embodiments, the first camera 805 and the second camera 809 are arranged (e.g., oriented) such that the first optical axis 805a of the first camera 805 is in at least one plane (e.g., a horizontal plane, a vertical plane, or a horizontal plane and a vertical plane). both) intersects the second optical axis 809a of the second camera 809. In some embodiments, the first camera 805 and the second camera 809 may be secured with an imaging module that is secured or coupled to a mounting assembly that is in turn secured or coupled to a mounting bracket. In some embodiments, the imaging module is configured such that the first camera 805 and the second camera 809 are arranged along a semicircular arc.

It should be appreciated that FIG. 8 shows the view projections of cameras 805 and 809 in two dimensions (X, Y) or "2D." Optical axes 805a and 809a are vector-based optical axes, although they are shown as two-dimensional projections for illustration purposes only. As shown, optical axis 805a intersects optical axis 809a at intersection point 888 of an intersection plane (not shown), although optical axes 805a and 809a appear to intersect in the 2D representation shown, they may in 3D doesn't actually intersect. As shown, intersection point 888 coincides with the central area of blank transparent area 803 . Thus, the optical axis 805a intersects the optical axis 809a in the horizontal plane. In other embodiments, optical axis 805a intersects optical axis 809a in a vertical plane. In still other embodiments, the optical axis 805a intersects the optical axis 809a in the horizontal and vertical planes. Although intersection point 888 is shown at a location coinciding with a central area of relatively small and transparent region 803, intersection point 888 may be positioned differently. For example, the intersection point 888 of the intersecting planes may be located farther from the first camera 805 and the second camera 809 such that it is located outside the relatively small and transparent area 803 . Alternatively, the intersection point 888 of the intersection planes may be located closer to the first camera 805 and the second camera 809 such that it is located outside the relatively small and transparent area 803 . As such, the intersection point 888 of the intersecting planes may be located at a predetermined distance from the relatively small and transparent region 803, for example in the range of about 0.2 meters to 2.0 meters or 0.5 meters to 1.0 meters. In at least one embodiment, the first camera 805 and the second camera 809 are configured to be mounted behind a window of the vehicle, and the intersection point 888 of the intersecting plane is located between the window and the first camera 805 and the second camera 809 . In at least one embodiment, the first camera 805 and the second camera 809 are configured to be mounted behind a window of the vehicle, and the intersection point 888 of the intersecting planes is located at a predetermined distance from the outer surface of the window.

In an exemplary embodiment, the first camera 805 is focused at a focal point P1 and the second camera 809 is focused at a focal point P2. Thus, the focal point P1 is located at a first horizontal distance beyond the intersection point 888 of the intersecting planes and the focal point P2 is located at a second horizontal distance beyond the intersection point 888 of the intersecting planes. As shown, the first horizontal distance and the second horizontal distance are substantially equal distances, although they may be different distances in alternative embodiments. For example, P1 may be about 1.5 times the horizontal distance of P2. In other embodiments, P1 may be about 1.25, 1.75, 2.0, 2.5, or 3.0 times the horizontal distance of P2. Furthermore, it should be noted that P1 and P2 are not necessarily singular points in three-dimensional space, ie, the focal points P1 and P2 may each contain a respective focal area as understood by those of ordinary skill in the art of cameras and camera optics. Furthermore, the focal area corresponding to the focal point P1 and the focal point area corresponding to the focal point P2 may at least partially overlap.

In an exemplary embodiment, the intersection point 888 of the intersecting planes is spaced from the first camera 805 and the second camera 809 by a separation distance Dy approximately equal to the shortest distance between the lens of the first camera 805 and the lens of the second camera 807 Dx. As shown, the shortest distance between the lens of the first camera 805 and the lens of the second camera 807 is represented by Dx, and the separation distance between the intersection point 888 of the intersecting plane and the first camera 805 and the second camera 809 is represented by Dy express. It should be understood that the separation distance Dy can be measured from the lens of the camera 805 or the camera 809 and thus the separation distance Dy can be different. In some embodiments, the separation distance Dy may fall within a range of one to four times the shortest distance Dx. In other embodiments, the separation distance Dy may fall within a range of one to two times, two to three times, three to four times, two to three times, or two to four times the shortest distance Dx. In some embodiments, Dx and Dy may be expressed as a ratio defining a distance at which an intersection point (eg, intersection point 888 ) of an intersection plane may be located. For example, Dy≤N×Dx, where 2≤N≥4.

As shown, the first field of view of camera 805 and the second field of view of camera 809 overlap and form a combined field of view of approximately 90 degrees. In some embodiments, the first field of view of camera 805 and the second field of view of camera 809 may only partially overlap, but still form a combined field of view. A blank transparent area 803 may be delineated by component 801 as a border. Component 801 may be a solid feature such as a vehicle component, such as a pillar, bumper, door panel, headlight, side window, front window, or the like. In at least one embodiment, component 801 may include a blank transparent region 803 that allows light to pass therethrough such that blank transparent region 803 is at least partially surrounded by a non-transparent region corresponding to the shaded region of FIG. 8 (ie, component 801 ). . As such, the relatively small and transparent area 803 is smaller than the relatively transparent area required for a wide-angle camera having a wide-angle field of view equal to the combined field of view of the first and second cameras. In other embodiments, component 801 may be the perimeter of an imaging mount or module adhered to the exterior of the host vehicle.

Figure 9 is a schematic representation of a single camera wide field of view system. In some embodiments, cameras (eg, single camera 905 ) may share one or more features of image capture devices 122 , 124 , and 126 as described above. A single camera 905 has a field of view with an optical axis 905 a protruding from a relatively large empty transparent area 903 . As shown, the optical axis 905a projects outwardly and divides the field of view of a single camera 905 into two symmetrical regions. The field of view of the single camera 905 is substantially equal to the combined field of view of the first camera 801 and the second camera 809 of FIG. 8 . As shown, a single camera 905 has a field of view of approximately 90 degrees. As can be seen by comparing Figures 9 and 8, the blank transparent area 903 (which may be delineated by part 901 as a boundary) is larger than the transparent area 803 of Figure 8, but the combined field of view of the imaging system of Figure 8 is substantially equal to the single The field of view of the camera. Blank transparent area 903 must be larger than blank transparent area 803 to accommodate fields of view with similar coverage areas since a single camera widefield system uses a single camera 905 . Accordingly, the footprint associated with a single wide field of view system is larger than that of the embodiment of FIG. 8 . It is an object of the present disclosure to minimize the footprint of the imaging system while accommodating a wide field of view (combined field of view).

Figure 10 is a schematic representation of an embodiment of an imaging system with three cameras. This exemplary embodiment may be similar to the embodiment of FIG. 8 . An exemplary embodiment may include a first camera 1005 , a second camera 1007 and a third camera 1009 . In some embodiments, cameras (eg, first camera 1005, second camera 1007, and third camera 1009) may share one or more characteristics of image capture devices 122, 124, and 126 as described above.

As shown in FIG. 10, the first camera 1005 has a first field of view with an optical axis 1005a, the second camera 1009 has a second field of view with an optical axis 1009a, and the third camera 1007 has a first field of view with an optical axis 1007a. Three fields of view. It should be appreciated that FIG. 10 shows the view projections of cameras 1005, 1007, and 1009 in two dimensions (X, Y) or "2D." Optical axes 1005a, 1007a, and 1009a are vector-based optical axes, although they are shown as two-dimensional projections for illustration purposes only. As shown, optical axis 1005a, optical axis 1009a, and optical axis 1007a intersect each other at at least one intersection point 1010 in an intersecting plane (eg, a horizontal plane, a vertical plane, or both). In this exemplary embodiment, the third camera 1007 is positioned substantially in the center of the first camera 1005 and the second camera 1009 and is equidistant from them. However, in other embodiments, the third camera 1007 may be positioned alternately, for example not in the center and/or closer to the first camera 1005 or the second camera 1009 . Still in other embodiments, the third camera 1007 may be positioned in front of the first camera 1005 and the second camera 1009 , eg closer to the relatively small and blank transparent area 1003 . As shown, an empty transparent area 1003 may be delineated by component 1001 as a border. Part 1001 may be a solid feature such as a vehicle part, such as a pillar, bumper, door panel, headlight, side window, front window, or the like. In some embodiments, the first camera 1005, the second camera 1009, and the third camera 1007 may be secured with an imaging module that is secured or coupled to a mounting assembly that is in turn secured or coupled to a mounting bracket. In some embodiments, the imaging module can be configured to arrange the first camera 1005 , the second camera 1009 and the third camera 1007 along a semicircular arc. For example, the imaging module may be shaped as a semicircle.

As shown, the optical axes 1005a, 1009a, and 1007a intersect each other at an intersection point 1010 of an intersection plane (not shown). As shown, the intersection point 1010 of the intersection plane coincides with the central area of the blank transparent area 1003 . It should be noted, however, that optical axes 1005a, 1009a, and 1007a may independently intersect in any manner. That is, there may be additional intersection points and/or intersection planes (not shown). Additionally, in some embodiments, intersection 1010 of intersecting planes may represent multiple independent intersections of coincident independent intersecting planes. For example, optical axes 1005a and 1009a intersect to form a first intersection point of a first intersection plane, optical axes 1005a and 1007a intersect to form a second intersection point of a second intersection plane, and optical axes 1009a and 1007a intersect to form a third intersection plane the third intersection point. As shown, the first intersection point of the first intersection plane, the second intersection point of the second intersection plane, and the third intersection point of the third intersection plane coincide with each other and form a coincidence of coincident intersection planes denoted intersection point 1010 intersection. Therefore, it should be understood that the term "intersection point" refers to a position where at least two optical axes (such as optical axes 1005a, 1009a and 1007a) intersect each other in at least one plane, such as a horizontal plane and/or a vertical plane.

Although the intersection point 1010 of the intersecting planes is shown at a location coinciding with the central area of the relatively small and transparent region 1003, the intersection point 1010 of the intersecting planes may be positioned differently. For example, the intersection point 1010 of the intersecting planes may be located further from the first camera 1005 , the second camera 1009 and/or the third camera 1007 such that it is located outside the relatively small and transparent area 1003 . Alternatively, the intersection point 1010 of the intersection planes may be located closer to the first camera 1005 , the second camera 1009 and/or the third camera 1007 such that it is located outside the relatively small and transparent area 803 . As such, the intersection point 1010 of the intersecting planes may be located a predetermined distance from the relatively small and transparent region 1003, for example in the range of about 0.01 to 2.0 meters, 0.1 to 0.5 meters, or 0.5 to 1.0 meters.

In an exemplary embodiment, the first camera 1005 is focused at a focal point P1, the second camera 1009 is focused at a focal point P2, and the third camera 1007 is focused at a focal point P3. Thus, the focal point P1 is located a first horizontal distance beyond the intersection 1010 , the focal point P2 is located a second horizontal distance beyond the intersection 1010 , and the focal point P3 is located a third horizontal distance beyond the intersection 1010 . As shown, the first, second and third horizontal distances are substantially equal distances, although they may be different distances in alternative embodiments. For example, P1 may be about 1.5 times the horizontal distance of P2 and/or P3, and vice versa. In other embodiments, P1 may be about 1.25, 1.75, 2.0, 2.5, or 3.0 times the horizontal distance of P2. Furthermore, it should be noted that P1 , P2 and P3 are not necessarily singular points in three-dimensional space, ie they may each contain a respective focal region, as understood by those of ordinary skill in the art of cameras and camera optics. Still further, the focal areas corresponding to P1, P2 and P3 may at least partially overlap.

In an exemplary embodiment, the intersection point of intersection point 1010 is spaced apart from first camera 1005, second camera 1009, and third camera 1007 by a separation distance Dy approximately equal to the lens of first camera 1005 and the lens of second camera 1007. The shortest distance Dx between. As shown, the shortest distance between the lens of the first camera 1005 and the lens of the second camera 1007 is represented by Dx, and the separation distance between the intersection point 1010 of the intersecting plane and the first camera 1005 and the second camera 1009 is represented by Dy express. It should be understood that the separation distance Dy may be measured from the lens of the camera 1005, the camera 1009 or the camera 1007 and thus may be different. It should also be understood that there may be a unique shortest distance Dx between the various cameras. In this way, each camera 1005, 1009 and 1007 may have a respective shortest separation distance Dy and a respective shortest distance Dx. In some embodiments, the separation distance Dy may fall within a range of one to four times the shortest distance Dx. In other embodiments, the separation distance Dy may fall within a range of one to two times, two to three times, three to four times, two to three times, or two to four times the shortest distance Dx. In some embodiments, Dx may refer to the separation between the two cameras that are farthest from each other, such as the first camera 1005 and the second camera 1009 . In some embodiments, Dx and Dy may be expressed as a ratio defining the distance over which the intersection point 1010 of the intersection plane may be located. For example, Dy≤N×Dx, where 2≤N≥4.

As shown, the field of view of the first camera 1005 overlaps with the second field of view of the second camera 1009 . Both the fields of view of the first camera 1005 and the second camera 1009 overlap with the field of view of the third camera 1007 . In this exemplary embodiment, optical axis 1005a intersects optical axis 1009a and optical axis 1007a in the central region of relatively small and empty transparent region 1003 . As such, the relatively small and transparent area 1003 is smaller than the substantially transparent area required for a wide-angle camera having a wide-angle field of view equal to the combined fields of view of the first, second, and third cameras. As shown, the first field of view of camera 1005, the second field of view of camera 1009, and the third field of view of camera 1007 form a combined field of view of approximately 150 degrees. In other embodiments, the combined field of view may range from 45 degrees to 180 degrees. For example, the combined field of view can be 55 degrees, 65 degrees, 75 degrees, 85 degrees, 95 degrees, 105 degrees, 115 degrees, 125 degrees, 135 degrees, 145 degrees, 155 degrees, 165 degrees, or 175 degrees.

11A is a schematic plan view representation of an exemplary imaging system consistent with the three-camera embodiment of FIG. 10 . As shown, the example imaging system of FIG. 11A may be mounted on a rear side window of a vehicle 99 or on a component such as a pillar portion of the body of the vehicle 99 . In some embodiments, vehicle 99 may be an autonomous vehicle. For ease of understanding, the vehicle 99 is shown to have a central longitudinal axis Cx that divides the vehicle 99 longitudinally into two substantially symmetrical halves. The exemplary imaging system includes three cameras, each with a respective field of view. For example, the first field of view F1 corresponds to the first camera with a small cap lens of 66 degrees, the second field of view F2 corresponds to the second camera with a small cap lens of 66 degrees, and the third field of view F3 corresponds to the camera with a small cap lens of 66 degrees. Lens for the third camera. In an exemplary embodiment, each camera has a field of view of 66 degrees. The first field of view F1 overlaps with the third field of view F3. Likewise, the second field of view overlaps with the third field of view F3.

In FIG. 11A , the first field of view F1 may be offset by 5 degrees from the central longitudinal axis Cx of the vehicle 99 . In other embodiments, F1 may be offset from the central longitudinal axis Cx of the vehicle 99 in the range of 5 degrees to 30 degrees, such as 10 degrees, 15 degrees, 20 degrees or 25 degrees. The second field of view F2 may be offset by 13 degrees from the central longitudinal axis Cx of the vehicle 99 . In other embodiments, F2 may be offset from the central longitudinal axis Cx of the vehicle 99 by a range of 5 degrees to 30 degrees, such as 10 degrees, 15 degrees, 20 degrees or 25 degrees. In an exemplary embodiment, the third field of view F3 overlaps the first field of view F1 and the second field of view F2 by an equal amount. In other embodiments, the third field of view F3 is substantially perpendicular to the central longitudinal axis Cx of the vehicle 99 . In other embodiments, the third field of view F3 may be offset from the central longitudinal axis Cx of the vehicle 99 , ie its optical axis is not perpendicular to the central longitudinal axis Cx.

In an exemplary embodiment, the combined field of view is 162 degrees. In other embodiments, the combined field of view may be larger or smaller. For example, in other exemplary embodiments, the combined field of view may range from 100 degrees to 175 degrees, such as 110 degrees, 120 degrees, 130 degrees, 140 degrees, 150 degrees, 165 degrees, and the like.

11B is a schematic plan view representation of an exemplary imaging system consistent with the two-camera embodiment of FIG. 8 . An exemplary imaging system is similar to the embodiment of FIG. 11A. Therefore, similar features will not be described in detail. In an exemplary embodiment, the first field of view F _1B corresponds to the first camera with a small 66-degree cap lens, and the third field of view F _3B corresponds to the second camera with a small 66-degree cap lens. In an exemplary embodiment, each camera has a field of view of 66 degrees. The first field of view F _1B overlaps with the third field of view F _3B .

In FIG. 11B , the first field of view F _1B may be offset by 5 degrees from the central longitudinal axis Cx of the vehicle 99 . In other embodiments, F _1B may be offset from the central longitudinal axis Cx of the vehicle 99 by a range of 5 degrees to 30 degrees, such as 10 degrees, 15 degrees, 20 degrees or 25 degrees. The second field of view F _3B may be offset by 60 degrees from the central longitudinal axis Cx of the vehicle 99 . In other embodiments, the F _3B may be offset from the central longitudinal axis Cx of the vehicle 99 by a range of 30 degrees to 90 degrees, such as 10 degrees, 20 degrees, 30 degrees, 40 degrees, 50 degrees, 60 degrees, 70 degrees, 80 degrees or 90 degrees.

In an exemplary embodiment, the combined field of view is 115 degrees. In other embodiments, the combined field of view may be larger or smaller. For example, in other exemplary embodiments, the combined field of view may range from 90 degrees to 175 degrees, such as 95 degrees, 100 degrees, 110 degrees, 120 degrees, 130 degrees, 140 degrees, 150 degrees, 165 degrees, and the like.

12A is a schematic plan view representation of an exemplary imaging system consistent with the three-camera embodiment of FIG. 10 . In FIG. 12A, the exemplary imaging system includes three cameras, each with a respective field of view. For example, the first field of view F4 corresponds to the first camera with a 52-degree small cap lens, the second field of view F5 corresponds to the second camera with a 52-degree small cap lens, and the third field of view F6 corresponds to the camera with a 100-degree lens. third camera. In an exemplary embodiment, the first field of view F4 overlaps with the third field of view F6. Likewise, the second field of view F5 overlaps with the third field of view F6. In other embodiments, the first camera may include a small 52 degree cap lens and the second camera may include a small 52 degree cap lens.

In FIG. 12A , the first field of view F4 is offset by 5 degrees from the central longitudinal axis Cx of the vehicle 99 . In other embodiments, F4 may be offset from the central longitudinal axis Cx of the vehicle 99 by a range of 5 degrees to 30 degrees, such as 10 degrees, 15 degrees, 20 degrees or 25 degrees. The second field of view F5 may be offset from the central longitudinal axis Cx of the vehicle 99 by 13.1 degrees. In other embodiments, F5 may be offset from the central longitudinal axis Cx of the vehicle 99 by a range of 5 degrees to 30 degrees, such as 10 degrees, 15 degrees, 20 degrees or 25 degrees. In an exemplary embodiment, the third field of view F6 overlaps the first field of view F4 and the second field of view F5 by an equal amount. In other embodiments, the third field of view F6 is substantially perpendicular to the central longitudinal axis Cx of the vehicle 99 . In other embodiments, the third field of view F6 may be offset from the central longitudinal axis Cx of the vehicle 99 , ie its optical axis is not perpendicular to the central longitudinal axis Cx.

In some embodiments, the third field of view is oriented such that it overlaps the first field of view F4 and the second field of view F6 by an equal amount. In an exemplary embodiment, the combined field of view is 161.9 degrees. In other embodiments, the combined field of view may be larger or smaller. For example, in other exemplary embodiments, the combined field of view may range from 100 degrees to 175 degrees, such as 110 degrees, 120 degrees, 130 degrees, 140 degrees, 150 degrees, 165 degrees, and the like.

12B is a schematic plan view representation of an exemplary imaging system consistent with the three-camera embodiment of FIG. 8 . This exemplary imaging system is similar to the embodiment of Figure 12B. Therefore, similar features will not be described in detail. In FIG. 12B, the exemplary imaging system includes two cameras, each with a respective field of view. For example, the first field of view F _4B corresponds to the first camera with a 52 degree small cover lens, and the second field of view F _5B corresponds to the second camera with a 100 degree lens. In the exemplary embodiment, the first field of view _F4B overlaps the second field of view F6B.

In FIG. 12B , the first field of view F _4B is offset by 5 degrees from the central longitudinal axis Cx of the vehicle 99 . In other embodiments, F _4B may be offset from the central longitudinal axis Cx of the vehicle 99 by a range of 5 degrees to 30 degrees, such as 10 degrees, 15 degrees, 20 degrees or 25 degrees. In the exemplary embodiment, the second field of view F6B is offset by 43 degrees from the central longitudinal axis Cx of the vehicle 99 . In other embodiments, the second field of view F6B is substantially perpendicular to the central longitudinal axis Cx of the vehicle 99 . In other embodiments, the second field of view F6B may be offset from the central longitudinal axis Cx of the vehicle 99 , ie its optical axis is not perpendicular to the central longitudinal axis Cx.

In an exemplary embodiment, the combined field of view is 132 degrees. In other embodiments, the combined field of view may be larger or smaller. For example, in other exemplary embodiments, the combined field of view may range from 100 degrees to 175 degrees, such as 110 degrees, 120 degrees, 130 degrees, 140 degrees, 150 degrees, 165 degrees, and the like.

13 is a schematic plan view representation of an exemplary imaging system consistent with the three-camera embodiment of FIG. 10 . In FIG. 13, the exemplary imaging system includes three cameras, each with a respective field of view. For example, the first field of view F7 corresponds to the first camera with a 52-degree small cap lens, the second field of view F8 corresponds to the second camera with a 52-degree small cap lens, and the third field of view F9 corresponds to the camera with a 100-degree lens. third camera. In an exemplary embodiment, the first field of view F7 overlaps the third field of view F9 by 17 degrees. Likewise, the second field of view F8 overlaps the third field of view F9 by 17 degrees. In other embodiments, the first field of view F7 and the second field of view F9 may respectively overlap with the third field of view F9 within a range of 10 degrees to 35 degrees. For example, about 15 degrees, 20 degrees, 25 degrees, 30 degrees or 35 degrees, etc. In FIG. 13 , the first field of view F7 is offset by 5 degrees from the central longitudinal axis Cx of the vehicle 99 and the second field of view F8 is offset by 5 degrees from the central longitudinal axis Cx of the vehicle 99 . In this exemplary embodiment, the third field of view F9 is substantially perpendicular to the central longitudinal axis Cx of the vehicle 99 , ie its optical axis (not shown) is substantially perpendicular to the central longitudinal axis Cx of the vehicle 99 . In other exemplary embodiments, the third field of view F9 may be offset from the central longitudinal axis Cx of the vehicle 99 , ie its optical axis is not perpendicular to the central longitudinal axis Cx.

14 is a schematic plan view surface of an exemplary imaging system consistent with the three-camera embodiment of FIG. 10 . In FIG. 14, the exemplary imaging system includes three cameras, each with a respective field of view. For example, the first field of view F10 corresponds to the first camera with a small cap lens of 52 degrees, the second field of view F11 corresponds to the second camera with a small cap lens of 52 degrees, and the third field of view F12 corresponds to the camera with a small cap lens of 100 degrees. third camera. In the exemplary embodiment, the first field of view F10 is offset by 7 degrees from the central longitudinal axis Cx of the vehicle 99 and the second field of view F11 is offset by 13 degrees from the central longitudinal axis Cx of the vehicle 99 . In the exemplary embodiment, the third field of view F12 is not completely perpendicular to the central longitudinal axis Cx of the vehicle 99 , but rather can be said to be substantially perpendicular.

15 and 16 are perspective representations of another exemplary imaging system having a combined field of view consistent with the disclosed embodiments. Imaging system 1400 includes a first camera 1402 and a second camera 1404 . Imaging system 1400 may further include a mount assembly 1408 that houses first camera 1402 and second camera 1404 . Mounting assembly 1408 may be configured to attach imaging module 1400 to a vehicle such that first camera 1402 and second camera 1404 face outward relative to the vehicle, as shown in FIG. 15 . Although shown in FIG. 15 as being located on the side of the vehicle 99, the imaging system 1400 may be located behind any window of the vehicle 99 (e.g., front windows, side windows, rear windows), or included in or attached to any component of the vehicle 99. (such as pillars, bumpers, door panels, headlights, trunk lids, fenders, luggage racks, cross bars, etc.). For example, when included in a vehicle component, imaging system 1400 may be positioned behind a transparent surface (eg, glass, plexiglass, etc.) provided at an opening of the component.

Mount assembly 1408 may be configured to orient first camera 1402 and second camera 1404 parallel to the ground. Alternatively, mount assembly 1408 may be configured to orient first camera 1402 and second camera 1404 at an off-angle relative to the ground, such as 5 degrees, 10 degrees, or 15 degrees.

Imaging system 1400 can further include a wiper assembly 1406 that includes at least one wiper blade. The wiper assembly 1406 may be configured to clear obstructions from the respective fields of view of the first camera 1402 and the second camera 1404 . In some embodiments, wiper assembly 1406 may be mounted on the exterior of vehicle 99 . Wiper assembly 1406 may include sensing features, timer features, electric actuators, articulating actuators, rotary actuators, and the like.

17 and 18 are perspective representations of another exemplary imaging system having a combined field of view consistent with disclosed embodiments. Imaging system 1800 includes a first camera 1802 (partially hidden for illustration purposes), a second camera 1804 , and a third camera 1806 . The mounting assembly (see 1408 of FIG. 16 ) is configured to attach the imaging module 1800 to the vehicle 99 such that the first camera 1802 , the second camera 1804 and the third camera 1806 face outward relative to the side rear windows of the vehicle 99 . In other embodiments, the mounting assembly (see 1408 of FIG. 16 ) may be configured to attach the imaging module 1800 to the front windshield of the vehicle 99 , as discussed in further detail below in connection with FIGS. 19 and 20 . In still other embodiments, the mounting assembly (see 1408 of FIG. 16 ) can be configured to attach the imaging module 1800 to a component of the vehicle 99, such as a bumper, pillar, door panel, headlight, deck lid, fender , Luggage racks, cross bars, etc.

According to a windshield embodiment, imaging system 1800 may be attached to a windshield of a vehicle. According to this exemplary embodiment, the third camera module may be mounted directly forward (perpendicular to the axis passing through the rear wheels of the vehicle 99 ), so that the optical axis of the third camera is also directly forward. The first camera may be mounted to the left of the third camera and oriented such that the optical axis of the first camera crosses the optical axis of the third camera (e.g., from left to right), and the second camera is mounted to the right of the third camera And oriented such that the optical axis of the second camera crosses the optical axis of the third camera (eg from right to left). It should also be understood that the optical axes of the first and second cameras may also intersect each other, although they need not necessarily intersect.

19 and 20 illustrate various embodiments of an exemplary imaging system for use on the front windshield of a vehicle 99 . Figures 19 and 20 are similar to the previously described embodiments, and thus like features and similarities will not be described in detail again. It should be understood that all the aforementioned exemplary ranges apply equally to the embodiment according to FIGS. 19 and 20 .

19 is a schematic plan view representation of an exemplary imaging system consistent with the three-camera embodiment of FIGS. 10 and 11-14. In FIG. 19, the exemplary imaging system includes three cameras, each with a respective field of view. For example, the first field of view F13 corresponds to the first camera with a small cap lens of 66 degrees, the second field of view F14 corresponds to the second camera with a small cap lens of 66 degrees, and the third field of view F15 corresponds to the camera with a small cap lens of 66 degrees. third camera. In an exemplary embodiment, the first field of view F13 overlaps with the third field of view F15. Likewise, the second field of view F14 overlaps with the third field of view F15.

20 is a schematic plan view representation of an exemplary imaging system consistent with the three-camera embodiment of FIG. 19 . In FIG. 20, the exemplary imaging system includes three cameras, each with a respective field of view. For example, the first field of view F16 corresponds to the first camera with a small cap lens of 52 degrees, the second field of view F17 corresponds to the second camera with a small cap lens of 52 degrees, and the third field of view F18 corresponds to the camera with a small cap lens of 100 degrees. third camera. In an exemplary embodiment, the first field of view F16 overlaps with the third field of view F18. Likewise, the second field of view F17 overlaps with the third field of view F18. In some embodiments, the combined field of view may include 180 degrees in front of the vehicle.

FIG. 21 is a side view representation of an exemplary imaging system 2200 . FIG. 22 is a perspective view of an imaging system 2200 consistent with the three-camera embodiment of FIG. 10 . As shown, imaging system 2200 is mounted on the interior rear window of vehicle 99 . In other embodiments, imaging system 220 may be mounted in the front windshield of vehicle 99 in accordance with the principles of the present disclosure. The imaging system 2200 may include an anti-glare shield 2202 surrounding a relatively small and transparent area 2204 that allows light to pass through the windows to the cameras 2205 , 2207 , and 2009 . The anti-glare cover 2202 may include dark adhesives, blackout paints, colorants, polarizers, printed or painted areas for exposure, or any combination thereof. At least one advantage of anti-glare shield 2202 is that glare from incident light caused by the slope (slope) of the windshield can be reduced. An anti-glare shield 2202 may surround a relatively small and transparent area 2204 configured to provide an aperture to the cameras 1005, 1007, and 1009, thereby increasing the depth of field of the cameras 2205, 2207, and 2209 and desirably enabling multiple different images over a wide range of distances. The subject is able to stay in focus. In some embodiments, relatively small and transparent regions 2204 may be colored or polarized.

FIG. 23 is a side view of an exemplary imaging system 2200 positioned on a front windshield 2300 . As shown, the front windshield 2300 has an inclination of 40 degrees relative to a straight horizontal surface such as the ground. While the example of FIG. 23 shows a windshield 2300 having a slope of 40 degrees, windshields having other slopes (eg, 35 degrees, 38 degrees, 42 degrees, 45 degrees, etc.) are compatible with the disclosed embodiments. unanimous. In the exemplary embodiment, imaging system 2200 includes three cameras 2205, 2207, and 2209 (see FIG. 22 ), but only first camera 2205 and third camera 2207 are shown because camera 2209 would be viewed by camera 2209 in side view. 2205 covered. As shown, cameras 2205 and 2207 are at different heights. Accordingly, the vectors associated with each respective optical axis (shown by the respective dashed lines) may not intersect in the vertical plane. However, the vectors associated with each respective optical axis (shown by respective dashed lines) may intersect in the horizontal plane. Additionally, in some embodiments, imaging system 220 may include two cameras, as previously described. In an exemplary embodiment, imaging module 2350 is shaped to have an approximate slope that is substantially equal to windshield 2300 , eg, about 40 degrees relative to a straight horizontal surface. In other embodiments, imaging module 2350 may be generally rectangular and rely solely on mounting components (not labeled for ease of understanding) to account for windshield 2300 slope. Additionally, a mounting assembly (not labeled for ease of understanding) is configured to hold imaging module 2350 fixedly relative to the window such that cameras 2005 , 2007 and 2009 protrude outwardly relative to windshield 2300 . In addition, the optical axes of the cameras 2205, 2207 and 2209 (indicated by dashed arrows) may protrude outward parallel to the ground. In this way, the imaging module and mounting assembly can account for the slope (slope) of the windshield. It should be understood that in the same manner, the imaging module and mounting assembly can address other levels and types of tilt, such as horizontal, vertical, combinations of horizontal and vertical.

FIG. 24 is a schematic plan view representation of FIG. 23 . In Fig. 24, three cameras are shown, namely a first camera 2205, a second camera 2209 and a third camera 2207. As shown, three cameras 2205 , 2209 and 2207 protrude outward through a relatively small and blank transparent area surrounded by anti-glare shield 2202 . The anti-glare shield is configured to provide an aperture to the cameras 1005, 1007, and 1009, thereby increasing the depth of field of the cameras 2205, 2207, and 2209, and enabling multiple different objects at a wide range of distances to remain in focus. In some embodiments, relatively small and transparent regions 2204 may be colored or polarized.

In one embodiment, the first, second, and third cameras may use the same opening in anti-glare cover 2202 (eg, FIG. 22 ). In other embodiments, each camera may have its own anti-glare shield and corresponding blank transparent area (not shown). In some examples, the opening in the printed or painted area used by the camera may be equal to or smaller than that required for a camera with a field of view having similar characteristics to the combined FOV of the three (or more) cameras in the camera module. area.

In an embodiment, a third camera is optionally located between the first camera and the second camera. Therefore, it should be appreciated that since the side cameras (first and second cameras 2205, 2207) can be combined to form a combined field of view that is larger than that of the third camera 2207 alone, a larger field of view can be used in the third camera by utilizing the combined FOV. Thin lenses without affecting the performance of the camera module. This may be particularly advantageous because the third camera 2207 may be attached closer to the windshield (or window) of the vehicle than the first and second cameras 2205, 2207. It will further be appreciated that by having the central camera 2207 fit more closely on the windshield, greater field of view coverage can be achieved or supported with smaller openings in the printing or painting in the window.

25 is a schematic plan view representation of another embodiment consistent with the present disclosure. In an exemplary embodiment, the imaging module 2512 includes cameras arranged in the shape of a semicircle whose radius coincides with the relatively small and blank transparent area 2511 . That is, in an exemplary embodiment, the imaging module is configured to arrange a plurality of cameras 2501, 2502, 2503, 2504, 2505, 2506, and 2507 along a semicircular arc. As shown, the camera is oriented towards the radius of the semicircle. Consistent with other disclosed embodiments, a mounting assembly (not shown) may be configured to attach imaging module 2511 to an interior window (or any other component) of a vehicle such that the camera faces outward relative to the vehicle. In other embodiments where the imaging module 2512 has an arcuate shape, the corresponding radius may not be positioned to coincide with the relatively small and blank transparent area 2511, eg it may be on either side. In the exemplary embodiment, there are seven symmetrically oriented cameras 2501, 2502, 2503, 2504, 2505, 2506, and 2507, although in other embodiments, there may be more or fewer cameras. In some embodiments, the cameras may not be symmetrically spaced along a semicircular arc.

In the exemplary embodiment, each respective camera 2501 , 2502 , 2503 , 2504 , 2505 , 2506 , and 2507 has a respective field of view (represented by a corresponding triangular area) and a view outward from a single relatively small and transparent opening 2511 . The corresponding optical axis protrudes (indicated by the corresponding dashed line). In the exemplary embodiment, the combined field of view Fc is 170 degrees, although in other embodiments it may be more or less, such as in the range of about 100 to 180 degrees. As shown, each camera 2501, 2502, 2503, 2504, 2505, 2506, and 2507 has a relatively narrow field of view when compared to the combined field of view Fc.

As shown, each respective field of view at least partially overlaps the radius of the semicircle, and the radius of the semicircle is centered on a single relatively small and transparent opening 2511 . Additionally, at least one respective intersection point of each respective optical axis (indicated by a corresponding dashed line) in a respective intersecting plane (not shown for ease of understanding) intersects each other respective optical axis, consistent with the above disclosure (See Figure 10). In this way, each intersection of two optical axes corresponds to a respective intersection point of a corresponding intersection plane. In an exemplary embodiment, each respective optical axis intersects every other respective optical axis in at least one horizontal plane. In other embodiments, each respective optical axis intersects every other respective optical axis in at least a vertical plane. In still other embodiments, each respective optical axis intersects every other respective optical axis in both the horizontal and vertical planes.

In an exemplary embodiment, relatively small and transparent area 2511 is configured to provide an aperture to each camera 2501, 2502, 2503, 2504, 2505, 2506, and 2507, thereby increasing the , 2506, and 2507 depth of field, and enables multiple different objects at a wide range of distances to remain in focus. Furthermore, the relatively small and transparent area 2511 is smaller than the relatively transparent area required for a wide-angle camera having a wide-angle field of view approximately equal to the combined field of view Fc. In some embodiments, the combined field of view may be on the order of at least 180 degrees in front of the vehicle 99 . As shown, an empty transparent area 2511 may be delineated by component 2510 as a border. Part 2510 may be a solid feature such as a vehicle part, such as a pillar, bumper, door panel, headlight, side window, front window, or the like.

In an exemplary embodiment, each camera 2501, 2502, 2503, 2504, 2505, 2506, and 2507 is equally spaced from at least one immediately adjacent camera 2501, 2502, 2503, 2504, 2505, 2506, or 2507 of the plurality of cameras. distance. In other embodiments, 2501, 2502, 2503, 2504, 2505, 2506, and 2507 are not spaced an equal distance apart. In an exemplary embodiment, each respective intersection point of each respective intersection plane is at least an equal distance from the nearest camera between immediately adjacent cameras. As shown, each respective intersection point of each respective intersection plane is at most four times the equal distance from the nearest camera between immediately adjacent cameras. In other embodiments, each respective intersection point of each respective intersection plane is at most six times the equal distance between immediately adjacent cameras from the nearest camera.

In an exemplary embodiment, the camera with the largest angle of repose between its corresponding optical axis and the relatively small and empty transparent region 2511 may have a polarizing filter 2515 . Polarizing filter 2515 may help filter out or avoid reflections of incident light refracted by relatively small and transparent opening 2511 . Also, by using multiple cameras, the polarizing filter 2515 can be attached to only those cameras that are most affected. Additionally, since the exemplary embodiment has multiple cameras 2501, 2502, 2503, 2504, 2505, 2506, and 2507, each camera's exposure level can be optimized for that particular camera. Also, glare from sunlight hitting the lens of one particular camera may not affect images from other cameras. In this way, multiple cameras 2501, 2502, 2503, 2504, 2505, 2506, and 2507 may be configured to provide redundancy in image quality and avoid the possibility of poor exposure.

In at least one embodiment, seven VGA resolution camera cubes of about 2 mm are spaced from each other along a semi-circular arc, as shown in FIG. 25 . According to this embodiment, the semicircle corresponds to a circle with a diameter of about 10 mm. At least one advantage of this embodiment is that it results in a very compact high resolution combined field of view.

In a similar but alternative embodiment, multiple cameras may be mounted in the hemisphere, with a first ring of 12 cameras positioned adjacent a relatively small and transparent hemispherical opening. In this embodiment, each of the 12 cameras of the first ring may have a polarizing filter. Additionally, a second ring of 8 cameras, a third ring of 4 cameras, and a central camera may be included.

Those of ordinary skill in the art will appreciate that image processing techniques may be used to combine the fields of view of two or more cameras to provide a combined field of view, as may be consistent with the disclosure herein. For example, at least one processing device (eg, processing unit 110 ) may execute program instructions to provide a combined field of view and/or analyze one or more images captured by two or more cameras. If the respective fields of view of two or more cameras partially overlap, any composite combined field of view may include the overlapping regions of the fields of view of the two or more cameras and any of the two or more cameras non-overlapping regions. Cropping can be used to reduce or prevent redundancy in imaging techniques consistent with the present disclosure.

Those of ordinary skill in the art will also appreciate that the disclosed embodiments may be positioned anywhere within or on the host vehicle and are not necessarily limited to the ego vehicle. Additionally, in some embodiments, multiple imaging systems may be installed on the vehicle 99 . For example, the first imaging system can be installed on the right rear window, the second imaging system can be installed on the left rear window, the third imaging system can be installed on the rear rear window, and the fourth imaging system can be installed on the left rear window. On the windshield. According to this embodiment, multiple imaging systems may combine to form a fully panoramic combined field of view around the vehicle 99 .

The foregoing description has been presented for purposes of illustration. It is not exhaustive and is not limited to the precise forms or embodiments disclosed. Modifications and alterations will be apparent to those skilled in the art from consideration of the specification and practice of the disclosed embodiments.

Furthermore, while illustrative embodiments have been described herein, those skilled in the art would appreciate, based on this disclosure, that any and all embodiments have equivalent elements, modifications, omissions, combinations (eg, each of the various aspects), changes and/or substitutions. The limitations in the claims are to be interpreted broadly based on the language employed in the claims, and not limited to the examples described in this specification or during the prosecution of this application. These examples should be construed as non-exclusive. Furthermore, the steps of the disclosed methods may be modified in any way, including by reordering steps and/or inserting or deleting steps. Accordingly, the specification and examples are to be considered as illustrative only, with a true scope and spirit being indicated by the appended claims, along with their full scope of equivalents.

Claims (25)

1. An imaging system for a vehicle, comprising:

An imaging module;

a first camera coupled to the imaging module, the first camera including a first lens having a first field of view and a first optical axis;

a second camera coupled to the imaging module, the second camera including a second lens having a second field of view and a second optical axis, the second field of view being different from the first field of view, the angle of the first lens being greater than the angle of the second lens; and

a mounting assembly configured to attach the imaging module to an interior rear window of the vehicle such that the first and second cameras are positioned at the same elevation and the first and second cameras face outwardly relative to the vehicle,

wherein the first optical axis intersects the second optical axis at least one intersection point of an intersection plane,

wherein the first camera is focused at a first horizontal distance beyond the intersection of the intersection planes and the second camera is focused at a second horizontal distance beyond the intersection of the intersection planes, and

wherein the first and second fields of view at least partially overlap and form a combined field of view, and the first field of view is perpendicular to a central longitudinal axis of the vehicle;

wherein the intersection of the intersecting planes is spaced from the first and second cameras by a separation distance that is one to four times the shortest distance between the lens of the first camera and the lens of the second camera;

Wherein the internal rear window includes an antiglare shield surrounding a transparent area that allows light to pass through the window to the imaging module, the transparent area being smaller than a required transparent area of a wide-angle camera having a wide-angle field of view equal to a combined field of view of the first and second cameras.

2. The imaging system of claim 1, wherein the first optical axis intersects the second optical axis in a horizontal plane.

3. The imaging system of claim 1, wherein the first optical axis intersects the second optical axis in a vertical plane.

4. The imaging system of claim 1, wherein the first optical axis intersects the second optical axis in both a horizontal plane and a vertical plane.

5. The imaging system of claim 1, wherein the first horizontal distance and the second horizontal distance are equal distances.

6. The imaging system of claim 1, wherein the first horizontal distance is two or three times the second horizontal distance.

7. The imaging system of claim 1, wherein an intersection of the intersecting planes is located between the inner rear window and the first and second cameras.

8. The imaging system of claim 1, wherein an intersection of the intersecting planes is located a predetermined distance from an outer surface of the inner rear window.

9. The imaging system of claim 1, wherein the combined field of view comprises at least 180 degrees.

10. The imaging system of claim 1, further comprising:

a wiper assembly including at least one wiper blade configured to clear a barrier from respective fields of view of the first and second cameras.

11. The imaging system of claim 1, wherein the imaging module is configured to arrange the first and second cameras side-by-side.

12. The imaging system of claim 1, wherein the transparent region is configured to provide apertures to the first and second cameras to increase a depth of field of the first camera and a depth of field of the second camera and enable a plurality of different objects at a wide range of distances to remain in focus.

13. The imaging system of claim 1, wherein an intersection of the intersecting planes is at a center of the transparent region.

14. An imaging system for a vehicle, comprising:

an imaging module;

a first camera coupled to the imaging module, the first camera including a first lens having a first field of view and a first optical axis;

a second camera coupled to the imaging module, the second camera including a second lens having a second field of view and a second optical axis, the second field of view being different from the first field of view, the angle of the first lens being greater than the angle of the second lens;

A third camera coupled to the imaging module, the third camera including a third lens having a third field of view and a third optical axis, the third field of view being different from the first field of view;

a mounting assembly configured to attach the imaging module to an interior rear window of the vehicle such that the first, second, and third cameras are positioned at a same height and the first, second, and third cameras face outwardly relative to the vehicle, wherein:

the first optical axis intersects the second optical axis at least one first intersection point of a first intersection plane;

the first optical axis intersects the third optical axis at least one second intersection point of a second intersection plane;

the second optical axis intersects the third optical axis at least one third point of a third intersection plane; and

the first and second fields of view at least partially overlap with a third field of view, and the first, second, and third fields of view form a combined field of view, and the third field of view is perpendicular to a central longitudinal axis of the vehicle;

the first intersection of the first intersecting plane, the second intersection of the second intersecting plane, and the third intersection of the intersecting planes coincide with one another and form a coincident intersection of coincident intersecting planes;

The coincidence intersection point of the coincidence intersection plane is spaced apart from the first, second, and third cameras by a separation distance that is one to four times the shortest distance between the first, second, and third cameras;

the internal rear window includes an antiglare shield surrounding a transparent area that allows light to pass through the window to the imaging module, and the transparent area is less than a transparent area required for a wide-angle camera having a wide-angle field of view equal to a combined field of view of the first, second, and third cameras.

15. The imaging system of claim 14, wherein the first optical axis intersects the second optical axis in a horizontal plane, the first optical axis intersects the third optical axis in a horizontal plane, and the second optical axis intersects the third optical axis in a horizontal plane.

16. The imaging system of claim 14, wherein the first optical axis intersects the second optical axis in a vertical plane, the first optical axis intersects the third optical axis in a vertical plane, and the second optical axis intersects the third optical axis in a vertical plane.

17. The imaging system of claim 14, wherein the first optical axis intersects the second optical axis on both a horizontal axis and a vertical axis, the first optical axis intersects the third optical axis on both a horizontal axis and a vertical axis, and the second optical axis intersects the third optical axis on both a horizontal axis and a vertical axis.

18. The imaging system of claim 14, wherein the third camera is positioned centrally between the first and second cameras and is equidistantly spaced between the first and second cameras.

19. The imaging system of claim 14, wherein the mounting assembly is configured to attach the imaging module to a rear interior window.

20. The imaging system of claim 14, further comprising:

a wiper assembly including at least one wiper blade configured to clear a barrier from respective fields of view of the first, second, and third cameras.

21. The imaging system of claim 14, wherein the imaging module is configured to arrange the first, second, and third cameras side-by-side.

22. The imaging system of claim 19, wherein the rear interior window comprises an antiglare shield comprising a compound film and an antireflective material.

23. The imaging system of claim 22, wherein the rear interior window includes a transparent region surrounded by a non-transparent region.

24. The imaging system of claim 14, wherein the transparent region is configured to provide apertures for the first, second, and third cameras, thereby increasing a depth of field of the first camera, a depth of field of the second camera, and a depth of field of the third camera, and enabling a plurality of different objects at a wide range of distances to remain in focus.

25. The imaging system of claim 24, wherein the coincidence intersection point of the coincidence intersection plane coincides with a central region of the transparent region.