HK1148377A

HK1148377A - Method of and apparatus for producing lane information

Info

Publication number: HK1148377A
Application number: HK11102323.1A
Authority: HK
Inventors: 马尔钦‧米夏尔‧克米奇克; 卢卡什‧彼得‧塔博罗维斯基
Original assignee: 电子地图有限公司
Filing date: 2007-11-16
Publication date: 2011-09-02

Abstract

The invention relates to a method of producing lane information for use in a map database. The method comprises: - acquiring one or more source images of a road surface and associated position and orientation data, the road having a direction and lane markings parallel to the direction of the road; - acquiring road information representative of the direction of said road; - transforming the one or more source images to obtain a transformed image in dependence of the road information, wherein each column of pixels of the transformed image corresponds to a surface parallel to the direction of said road; - applying a filter with asymmetrical mask on the transformed image to obtain a filtered image; and, - producing lane information from the filtered image in dependence of the position and orientation data associated with the one or more source images.

Description

Method and apparatus for generating lane information

Technical Field

The present invention relates to a method for generating lane information for use in a map database. The invention further relates to an apparatus for generating lane information, a computer program product and a processor-readable medium carrying the computer program product.

Background

There is a need to collect large amounts of horizontal road information, such as lane dividers, road centerlines, road widths, etc., for digital map databases used in navigation systems and similar systems. The geographical location of the road information may be stored as absolute or relative location information. For example, the center line may be stored with absolute geographical position information, and the road width may be stored with relative position information, which is relative information with respect to the absolute geographical position of the center line. The road information may be obtained by interpreting a high resolution aerial corrected (orthographically) image. The orthorectified image is a "scaled" image depicting the ground features as seen from above their rectified ground position, where the distortion caused by camera and flight characteristics and heave displacements have been removed using photogrammetric techniques. An orthorectified image is a type of aerial photograph that has been geometrically rectified ("orthorectified") so that the scale of the photograph is uniform, meaning that the photograph can be considered equivalent to a map. The orthorectified image can be used to measure true distance because it is an accurate representation of the earth's surface, adjusted for terrain relief, lens distortion, and camera pitch. Orthorectified views differ from perspective views in that orthorectified views are projected at right angles to a reference plane, whereas perspective views are projected from the surface to the reference plane from a single fixed position or viewpoint. The orthorectified image may be obtained by any suitable map projection. The map projection may be a projection through a surface, such as a cylindrical projection, a pseudo-cylindrical projection, a hybrid projection, a conical projection, a pseudo-conical projection, or an azimuthal projection. The projection may also be a projection by saving a metrology property. Common for the map projections is that they are all orthogonal projections, which means that each pixel represents a point on a surface of a reference plane (an approximately spherical ellipsoid) seen along a line perpendicular to the surface. Thus, each pixel of the orthorectified image of the earth's surface roughly corresponds to a view of the earth's surface as seen along a line perpendicular to the approximately spherical ellipsoid.

In addition to the above projection constraints, the orthorectified image also includes metadata that enables an algorithm to reference any pixel of the orthorectified image to a point in a geographic coordinate reference system. The exact location of each pixel on an approximately spherical ellipsoid is known. Thus, the location and size of the ground features (e.g., horizontal road information) may be retrieved from the orthorectified image, and highly accurate distances and earth coordinates may be calculated. Metadata of the geocoded ortho-corrected image defines a projected coordinate reference system to determine for each pixel a corresponding location in the geographic coordinate reference system.

The geocoded ortho-corrected image shows the road surface with road markings, if any. Image processing algorithms enable us to detect road markings and determine corresponding pixels. The metadata enables us to accurately determine the location of the road marker in the geographic coordinate reference system.

Such high resolution aerial orthographic corrected images should have pixel sizes below 25 cm. Obtaining such images is extremely expensive and does not guarantee that all road level information is captured.

An orthorectified image may be obtained from the aerial image. However, errors are often introduced, which can lead to inaccurate mapping of the geographic location data. The main problem is that aerial images are usually taken not exactly perpendicular to the earth's surface. Even when a picture is taken close to the earth's surface, only the center of the picture is exactly perpendicular to the earth's surface. In order to orthorectified such an image, height information of the terrain must additionally be obtained. Accuracy can be improved by taking overlapping images and comparing the same surface obtained from successive images from the same aerial camera. There are limits to the accuracy achieved to additional cost. It should be noted that the aerial orthographic corrected image may be a mosaic of the top view image.

Furthermore, to obtain "horizontal" road information from an aerial, ortho-corrected image, the image must be analyzed. In the image, the road surface must be detected. Since the geographical position associated with the aerial ortho-corrected image and the geographical position in the map database have been obtained from different geographical determination devices, it is not always possible to use the geographical position of the road in the map database directly to determine exactly where the road surface is located in the ortho-corrected image.

Today, "vertical" road information, such as speed limits, direction signposts, etc., for digital map databases used in navigation systems and similar systems, can be obtained by analyzing and interpreting horizontal picture images and other data collected by means of mobile collection devices that are only on earth. The term "vertical" indicates that the information plane of the road information is substantially parallel to the gravity vector. Mobile mapping vehicles (which are land-based vehicles, such as automobiles or vans) are used to collect movement data for the purpose of enhancing digital map databases. Examples of enhancements are traffic signs, route signs, traffic lights, locations of street signs displaying street names, etc.

Mobile mapping vehicles have several cameras, some of which are stereographic cameras and all of which are precisely geo-located by vehicles having onboard positioning systems (e.g., precision GPS receivers) and other onboard position determination equipment (e.g., inertial navigation systems-INS). While traveling on a road network, a sequence of geocoded images is captured. These images may be video or still picture images. Geocoding means attaching a location associated with an image to metadata of the image. In the present invention, the position is derived from a position determining system of a vehicle comprising a GPS receiver and possibly an INS and possibly distance and orientation measuring means.

A mobile mapping vehicle records more than one image in a sequence of images of an object (e.g., a building or road surface), and for each image of the sequence of images, the geographic position and orientation relative to a coordinate reference system is accurately determined. A sequence of images with corresponding geographical position and orientation information will be referred to as a sequence of geocoded images. Since the sequence of images obtained by the camera represents a visual perspective of "horizontal" road information, the image processing algorithm may provide a solution to extract road information from the sequence of images. The geographical position of the camera is known accurately by means of an onboard positioning system (e.g. GPS receiver) and other additional position and orientation determining equipment (e.g. inertial navigation system-INS).

Lane information is present in both aerial images and sequences of images captured by mobile mapping vehicles. With complex image processing algorithms, lane information may be detected and corresponding position information may be determined.

Disclosure of Invention

The present invention seeks to provide an improved method of generating lane information for use in a map database.

According to the invention, the method comprises:

-acquiring one or more source images and associated position and orientation data of a road surface, the road having a direction and lane markings parallel to the direction of the road;

-obtaining road information representative of the direction of the road;

-transforming the one or more source images in dependence on the road information to obtain a transformed image, wherein each pixel column of the transformed image corresponds to a surface parallel to the direction of the road;

-applying a filter with an asymmetric mask to the transformed image to obtain a filtered image; and a process for the preparation of a coating,

-generating lane information from the filtered image in dependence of the position and orientation data associated with the one or more source images.

The invention is based on the following recognition: information on the location and direction of roads can be easily obtained from a digital map database. Furthermore, the location and orientation of the road can be easily obtained from a mobile mapping vehicle traveling on the road. The direction of a vehicle travelling on a road is more or less parallel to the direction of said road. Further, a lane mark (e.g., a lane mark that is a linear mark sprayed on the surface of a road) is a mark in a direction parallel to the road. Today, geocoded images (which may be top-view or orthorectified view images) are publicly available, where for each pixel, the geographic location in a predefined coordinate system is known. The geocoded image can be an aerial or satellite image. The one or more source images may also be source images that have been obtained by: one or more sequences of images and associated position and orientation data obtained with a land-based camera mounted on a moving vehicle traveling on the road are retrieved and a normalization process is performed on the one or more sequences of images to obtain the one or more images and associated position and orientation data. The normalization process may include an orthorectification process.

And filtering the source image to enhance the extraction of the lane information. The enhancement improves the detection rate of the object to be detected. Commonly known filters, such as noise filters, morphological filters and edge filters, may be used to enhance the extraction. However, since the orientation of the linear objects is not known in the source image, the filter must be an orientation-invariant filter. These filters are symmetric filters with two dimensions that perform a function in two directions simultaneously. However, when the orientation of the linear object to be detected is known, a filter that operates in only one dimension may be used. These filters have a reduced number of parameters. Therefore, less computing power is required to perform such filtering. Furthermore, if the orientation and form of the object are known, a simpler shape detection algorithm may be used, since the shape filter only has to detect objects in the image with a known orientation. Thus, according to the invention, before extracting the lane information from the source image, an image is obtained using a road direction, wherein each pixel column is parallel to the road direction. Thus, the linear lane markers will have a vertical orientation in the image. This allows us to use functions with only one dimension on the transformed image to enhance the image, for example by emphasizing features of lane information. These filters are called asymmetric filters because they perform filtering on an image in only one direction. Examples of emphasizing features in an image by filtering are reduction of image noise, amplification of the width of linear road markings, and removal or suppression of unwanted image information. After filtering the image, the linear lane paint may be detected using standard feature extraction techniques. Furthermore, by knowing the orientation of the linear object to be detected, a more efficient emphasis filter can be designed.

In an embodiment of the invention, the tracking information and the associated position and orientation information of the one or more image sequences have been captured simultaneously from output signals generated by positioning determining means mounted in the moving vehicle. This feature improves the relative accuracy of the position information of the generated lane information. The position and orientation of the vehicle is determined with a position determination device, which may include a GPS receiver and inertial measurement devices, such as one or more gyroscopes and/or accelerometers. Since the distance between the terrestrial-based camera and the recorded earth's surface is limited and the geographic position of the camera is precisely known with onboard positioning systems (e.g., GPS receivers) and other additional position and orientation determining equipment (e.g., inertial navigation systems-INS), the absolute geographic position of each pixel (assuming the pixel is a representation of the earth's surface) can be precisely determined. This enables the algorithm as disclosed in the unpublished patent application PCT/NL2006/050252 to produce an orthorectified image and associated position and orientation data with great precision. A typical Mobile Mapping System (MMS) generates an orthorectified mosaic or image with an 8cm resolution with a relative accuracy of 50cm and an absolute accuracy of 200cm over 100 m. The precise position and orientation data enables us to transform the orthorectified image such that each pixel column of the transformed image corresponds to a surface parallel to the direction of the road. It should be noted that the transformation may be optional if the orthorectification process produces an image in which each column is already parallel to the direction of travel of the vehicle.

In an embodiment of the invention, the transformation comprises a rotation operation. Image rotation is a simple function for aligning an orthorectified view image or top-view image such that a column of pixels corresponds to a line on the earth's surface parallel to the derived direction of the road.

In an embodiment of the invention, applying the filter with the asymmetric mask comprises:

-applying first a first filter having structured elements enlarging the width of a line in a direction perpendicular to the driving direction of the vehicle and secondly a second filter having structured elements reducing the length of a line in a direction parallel to the driving direction of the vehicle to the transformed image.

In a further embodiment of the present invention, the first filter is a maximum filter and the second filter is a minimum filter. These filters are very simple filters to enlarge the width of the object and to reduce the length of the object.

In a further embodiment of the invention, applying the filter with the asymmetric mask further comprises:

-applying again a third filter having a structuring element enlarging the length of a line in a direction parallel to the direction of travel to its original size. In an advantageous embodiment, the third filter is a maximum filter. These features allow us to restore the length of the object to its original size, which enables us to accurately determine the length of each segment of the dashed line.

In an embodiment of the present invention, generating the lane information includes:

-searching for a solid line in the filtered image; and a process for the preparation of a coating,

-calculating the position of the solid line from the position and orientation data associated with the one or more source images. In an exemplary embodiment, generating the lane information includes:

-searching for a rectangle in the filtered image; and a process for the preparation of a coating,

-calculating a position of a rectangle from the position and orientation data associated with the one or more source images.

The present invention may be implemented using software, hardware or a combination of software and hardware. When all or part of the present invention is implemented in software, the software may reside on a processor-readable storage medium. Examples of suitable processor-readable storage media include floppy disks, hard disks, CD ROMs, DVDs, memory ICs, and the like. When a system includes hardware, the hardware may include: an output device (e.g., a monitor, speaker, or printer); input devices (e.g., a keyboard, a pointing device, and/or a microphone); and a processor in communication with the output device; and a processor-readable storage medium in communication with the processor. The processor-readable storage medium stores code that is capable of programming the processor to perform acts that implement the present invention. The processes of the present invention may also be implemented on a server accessible via a telephone line or other network or internet connection.

Drawings

The present invention will be discussed in more detail below using a number of exemplary embodiments with reference to the attached drawings, which are intended to illustrate the invention and not to limit its scope, which is defined by the appended claims and equivalents thereof, in which

FIG. 1 shows an MMS system with a camera;

FIG. 2 shows a graphical representation of position and orientation parameters;

FIG. 3 shows a block diagram of a computer arrangement by means of which the present invention may be implemented;

FIG. 4 is a flow chart of an example embodiment of a process for generating lane information according to the present invention;

FIG. 5 shows a side view of the general principle of converting a source image into an orthorectified image block;

FIG. 6 shows a top view of the general principle of converting a source image into an orthorectified image block;

FIG. 7 shows an orthorectified image of a road segment;

FIG. 8 shows a flow diagram of an example embodiment of asymmetric filtering;

FIG. 9 shows an exemplary embodiment of a filter mask;

FIG. 10 shows a filtered image of a road segment; and is

FIG. 11 shows features found in a filtered image like that shown in FIG. 10.

Detailed Description

Fig. 1 shows an MMS system in the form of a car 1. The automobile 1 is provided with one or more cameras 9(I), I ═ 1, 2, 3, … I. The car 1 may be driven by a driver along a road of interest.

The automobile 1 includes a plurality of wheels 2. Further, the automobile 1 is provided with a high-precision position determining device. As shown in fig. 1, the position determination apparatus includes the following components:

● GPS (global positioning system) unit connected to the antenna 8 and arranged to communicate with a plurality of satellites SLi (i ═ 1, 2, 3, …) and to calculate position signals from signals received from the satellites SLi. The GPS unit is connected to the microprocessor mup. Based on the signal received from the GPS unit, the microprocessor μ P can determine a suitable display signal to be displayed on the monitor 4 in the car 1, informing the driver of the location of the car and in which direction the car may be travelling. A differential GPS unit may be used instead of a GPS unit. Differential Global Positioning System (DGPS) is an enhancement to the Global Positioning System (GPS) that uses a network of fixed ground-based reference stations to broadcast the difference between the position indicated by the satellite system and a known fixed position. These stations broadcast the difference between the measured satellite pseudoranges and the actual (internally computed) pseudoranges, and the receiver station may correct its pseudoranges by the same amount.

● DMI (distance measuring instrument). This instrument is an odometer that measures the distance traveled by the car 1 by sensing the number of revolutions of one or more of the wheels 2. The DMI is also connected to a microprocessor μ P to allow the microprocessor μ P to calculate a display signal from the output signal from the GPS unit while taking into account the distance measured by the DMI.

● IMU (inertial measurement unit). Such an IMU may be implemented as 3 gyroscopic units arranged to measure rotational and translational accelerations in 3 orthogonal directions. The IMU is also connected to the microprocessor μ P to allow the microprocessor μ P to account for the DMI measurements while calculating a display signal from the output signal from the GPS unit. The IMU may also include a dead reckoning sensor.

The system as shown in fig. 1 is a so-called "mobile mapping system" which collects geographical data, for example by taking pictures with one or more cameras 9(i) mounted on the car 1. The camera is connected to the microprocessor μ P. The camera 9(i) in front of the car may be a stereo camera. The camera may be arranged to generate a sequence of images in which images have been captured at a predefined frame rate. In an exemplary embodiment, one or more of the cameras are still picture cameras arranged to capture pictures at each predefined displacement or each time interval of the car 1. The predefined displacement is selected such that two consecutive pictures comprise similar portions of the road surface. For example, a picture may be captured after every 8 meters of travel.

It is generally desirable to provide position and orientation measurements as accurately as possible from the 3 measurement units GPS, IMU and DMI. These position and orientation data are measured while the camera 9(i) takes pictures. These pictures are stored for later use in a suitable memory of said μ P in connection with corresponding position and orientation data of the car 1 collected while taking these pictures. The picture includes information about lane information, such as the center of the road, road surface edges, and road width.

Fig. 2 shows which location signals can be obtained from the three measurement units GPS, DMI and IMU shown in fig. 1. Figure 2 shows that the microprocessor mup is arranged to calculate 6 different parameters, namely 3 distance parameters x, y, z relative to an origin in a predetermined coordinate system and respectively ω_x、ω_y、ω_zAnd 3 angular parameters representing rotation about the x-axis, y-axis, and z-axis, respectively. The z direction coincides with the direction of the gravity vector. The global UTM coordinate system may be used as the predetermined coordinate system.

The microprocessor mup and memory in the car 1 may be implemented as a computer arrangement. An example of such a computer arrangement is shown in fig. 3.

In fig. 3, a diagrammatic view of a computer arrangement 300 is given, which comprises a processor 311 for performing arithmetic operations. In the embodiment shown in fig. 1, the processor will be a microprocessor μ P.

The processor 311 is connected to a plurality of memory components, including a magnetic hard disk 312, Read Only Memory (ROM)313, Electrically Erasable Programmable Read Only Memory (EEPROM)314, and Random Access Memory (RAM) 315. Not all of these memory types need necessarily be provided. Moreover, these memory components need not be physically located close to the processor 311, but may be located remote from the processor 311.

The processor 311 is also connected to means for inputting instructions, data, etc., such as a keyboard 316 and a mouse 317, for example, by a user. Other input means known to those skilled in the art may also be provided, such as a touch screen, a trackball and/or a voice converter.

A read unit 319 is provided which is connected to the processor 311. The reading unit 319 is arranged to read data from and possibly write data to a removable data carrier or a removable storage medium, such as a floppy disk 320 or a CDROM 321. Other removable data carriers may be magnetic tapes, DVDs, CD-R, DVD-R, memory sticks, etc., as known to those skilled in the art.

The processor 311 may be connected to a printer 323 to print out data on paper and to a display 318, such as a monitor or LCD (liquid crystal display) screen or any other type of display known to those skilled in the art.

The processor 311 may be connected to a microphone 329.

Further, the processor 311 may be connected to a communication network 327, such as a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), the Internet, etc., by way of the I/O component 325. The processor 311 may be arranged to communicate with other communication arrangements over the network 327.

The data carrier 320, 321 may comprise a computer program product in the form of data and instructions arranged to provide the processor with the ability to perform a method according to the invention. Alternatively, however, such a computer program product may be downloaded via the telecommunications network 327.

The processor 311 may be implemented as a stand-alone system, or as a plurality of parallel-operating processors each arranged to implement sub-tasks of a larger computer program, or as one or more main processors having a number of sub-processors. Portions of the functionality of the present invention may even be implemented by remote processors in communication with the processor 311 through a telecommunications network 327.

The components contained in the computer system of fig. 3 are those typically found in general purpose computer systems, and are intended to represent a broad category of such computer components well known in the art.

Thus, the computer system of FIG. 3 may be a personal computer, a workstation, a minicomputer, a mainframe computer, and the like. The computers may also include different bus configurations, networked platforms, multi-processor platforms, and the like. Various operating systems may be used, including UNIX, Solaris, Linux, Windows, Macintosh OS, and other suitable operating systems.

For post-processing of the images and scans taken by the camera 9(i), an arrangement similar to that in fig. 3 will be used, but this arrangement will not be located in the car 1 but may conveniently be located in a building for off-line post-processing. The images and scans taken by camera 9(i) are stored in one or more memories 312-315. This storage may be accomplished by first storing the image and scan on a DVD, memory stick, or the like, or transmitting it from memory 9, possibly wirelessly.

Fig. 4 shows a flow chart of an exemplary embodiment of a process of generating lane information according to the present invention. The process begins with an MMS (mobile mapping system) time period 41 by capturing a sequence of source images with associated position and orientation data in a coordinate reference system with the mobile mapping vehicle shown in fig. 1 and storing the captured data on a storage medium. In process block 42, the captured data is processed to generate an orthorectified image block for each source image, the orthorectified image block having metadata corresponding to associated position and orientation data in the coordinate reference system. The orthorectification process removes image distortion introduced by the set geometry and terrain, and resamples the image to a uniform ground sample distance and user-specified map projection. The associated position and orientation data includes position signals available from the GPS, DMI, and IMU, as well as the position and orientation of the respective camera relative to the position and orientation of the automobile. The generation of an orthorectified image block from a source image will be described in more detail below. The position and orientation data enables us to superimpose the orthorectified images (including similar parts of the road surface) to obtain an orthorectified mosaic. The orthorectified image 43 may be an orthorectified image generated from only one source image or may be a mosaic of orthorectified images representing straight road segments varying from 10 meters to 100 meters, for example.

Furthermore, from the captured data, a direction of travel of the mobile mapping vehicle on the road may be derived. When the vehicle is traveling on a road surface, the direction of travel is substantially parallel to the direction of the road. In this case, the driving direction is an estimation of the road direction. The road direction is indicated by block 44.

It should be noted that the method according to the invention can be applied to any georeferenced orthorectified image, for example, an aerial or satellite orthorectified image (provided that the resolution of the image is sufficient to detect lane information). Further, the direction of the road may be retrieved from a commercially available map database. However, since the mobile mapping vehicle records both image data and position and orientation data of the vehicle at the same time, this data enables us to select image portions comprising road surfaces more accurately in the image and calculate positions in the coordinate reference system very accurately than when done with aerial or satellite images.

A method of generating an orthorectified image and associated metadata defining position and orientation data for the orthorectified image is disclosed in the unpublished patent application PCT/NL 2006/050252. This approach enables us to generate very accurate geocoded orthorectified images from only the mobile mapping system data. The geocoded image has a pixel resolution of 8cm (relative positional accuracy within the image) and the metadata defining the position and orientation of the image on the surface of the earth has absolute geolocation accuracies of 1 meter and 0.1 degrees, respectively.

In block 45, the orthorectified image 43 is transformed into a transformed image 46 using the road direction. In an embodiment, the road direction corresponds to a driving direction of the mobile mapping vehicle. In another embodiment, the road direction is derived from aerial or satellite images. And in yet another embodiment the road direction is derived from data in a digital map database. The orthorectified image is normalized using the transform at block 45. In the transformed image 46, the direction and resolution of the road are normalized. Normalization means that in the transformed image 46, the roads have a predefined orientation and each pixel of the transformed image represents a fixed predefined physical area of the earth's surface. This enables us to apply only a simple set of asymmetric filters to the transformed image. The pixel size is adjusted in a normalization process (which may be part of the orthorectification process) to make the pixel scale uniform. The road direction is normalized by rotating the orthorectified image. Preferably, the road direction is parallel to a column of pixels in the transformed image. The transformed image includes associated position and orientation data. The position and orientation data enables a process to determine, for each pixel in the transformed image, a corresponding geographic location in a predefined coordinate system.

In block 47, asymmetric filtering is performed on the transformed image to obtain a filtered image 48. Exemplary embodiments of the filter will be described below. It has been found that a set of asymmetric filters can be used for each country because in each country the road markings have standardized predefined dimensions. By knowing the size of the road markings in the image, the filter can be optimized for that size. The dimensions of the road marking may be the width of the lane marking and the length of the dashed lane separator, where the length and width are defined in the transformed image in the number of pixels. In one embodiment, a pixel corresponds to a physical region of 8x8cm of the earth's surface. A dashed line, typically 3m long and 20cm wide, will have a length of 37 pixels and a width of 2 to 3 pixels in the transformed image.

The goal of asymmetric filtering, which in the example given above is a combination of the first, second and third filters, is to emphasize the information searched in the transformed image and reduce the noise in the transformed image. Some examples of reinforcement are: size enlargement and brightness increase/decrease of pixel values. Both enhancement and noise reduction improve feature recognition in the filtered image.

In block 50, feature recognition is performed. The feature recognition algorithm used is not an essential feature of the present invention. The features to be detected are substantially vertical lines or rectangles in the filtered image. The line may be a dashed line, a solid line, a double solid line, or any other straight line in the filtered image. Those skilled in the art know which algorithms are suitable for detecting the feature. The algorithm must find and detect the solid line or rectangle in the filtered image. After detecting a feature in the filtered image, a program determines the x, y position of the feature in the filtered image. Since the filtered image corresponds to a transformed image that includes associated position and orientation data, the geographic location of the feature may be derived from the position and orientation data associated with the transformed image. The width of roads and lanes may be calculated by detecting the first two or more parallel solid or dashed lines in the filtered image. The distance between parallel lines can be translated into a distance between lines by using the position and orientation data. The recognized features and associated location data are stored in a database for use in a digital map database.

Fig. 5 shows a side view of the general principle of converting a source image into an orthorectified image block, performed in block 42. The image sensor 101 in the camera or CCD camera 202 (shown in fig. 6) records a sequence of source images. The source images represent more or less vertical images recorded by a land based camera 9(i) mounted on a car as shown in fig. 1. The source image may be a sequence of still pictures recorded by means of a still picture camera, the camera being triggered every shift of, for example, 8 meters. The camera comprising the image sensor has a viewing angle a. The angle of view a is determined by the focal length 102 of the lens of the camera. The viewing angle alpha may be in the range 45 deg. < alpha < 180 deg.. Furthermore, the camera has a viewing axis 103, which is located in the center of the viewing angle. In fig. 1, the viewing axis 103 is parallel to the horizontal plane 104. The image sensor 101 is mounted perpendicular to the viewing axis 103. In this case, the image sensor 101 records a "pure" vertical source image. If the height of the image sensor relative to a horizontal plane (e.g., the earth's surface) is further known, the image recorded by the image sensor 101 may be transformed into an orthorectified image block that represents a scaled version of the orthorectified view of the horizontal plane. To obtain a horizontal image with a suitable resolution in the horizontal direction, a limited area of the image sensor is used. Fig. 5 shows a portion 106 of the image sensor 101 corresponding to a portion 108 in the horizontal plane. The minimum acceptable resolution of the orthorectified image block determines the maximum distance between the image sensor and the farthest point in the horizontal plane. With the aid of the geometry, source images retrieved from terrestrial based cameras can be converted into any virtual plane. An orthorectified image block can be obtained from a source image even if the viewing axis is at a known angle relative to the horizontal plane.

Fig. 6 shows an orthorectified view of the general principle of converting a source image into an orthorectified image block 200. The angle of view α and orientation of the visual axes 103, 218 of the camera 202 determine the portion of the horizontal plane that is recorded by the image sensor 101. The boundaries of the orthorectified image block 200 are indicated by reference 224. In fig. 6, the boresight 218 of the camera 202 coincides with the directional center axis of the vehicle, which in turn corresponds to the lane markings of the road. The set of attributes necessary for navigation systems and similar systems, as well as their positional accuracy, require a predefined minimum resolution of the orthorectified image block. These requirements limit the portion of the horizontal plane that is available from the source image. The maximum distance 206 between the position of the camera focus 208 relative to the horizontal plane and the boundary of the area of the horizontal plane determines the minimum resolution. Further, in practice, maximum distance 206 may be limited by the minimum distance between two vehicles while traveling on a particular road. By thus defining the maximum distance, it has the following advantages: in most cases, the road surface in the orthorectified image block does not include the back of a car driving in front of the mobile mapping vehicle. Further, the difference between the maximum distance 206 and the minimum distance 204 determines the maximum allowable distance between successive recordings of images by the camera. This may define a capture frame rate and may limit the maximum travel speed of the vehicle. The rectangle of the horizontal plane corresponds to an area in the source image which has approximately the form of a trapezoid. As can be seen in fig. 6, the minimum distance and view angle a determine whether the orthorectified image block 200 comprises a small area 210 having no corresponding area in the source image. Orthorectified image block 200 is a dashed square and small area 210 is a small triangle with the closed corners of the dashed square indicated by 200 cut away.

In an embodiment, the orthorectified image block 200 corresponds to an area of 16m width 220 and 16m length 222. In the case of every 8 meters of captured image, 99% of the road surface can be seen in two successive images. For roads wider than 16m, a front-looking camera and a side-looking camera must be used to generate an orthorectified tile of the road width. A road portion that cannot be retrieved from an image captured by the front-view camera is retrieved from an image captured by the side-view camera. The side view camera may be any camera having an oblique or vertical viewing axis relative to the direction of the vehicle. The orthorectified image block 200 is now a mosaic of orthorectified images obtained from a front-view camera and a side-view camera. For further processing of the orthorectified picture, it is advantageous to have orthorectified image blocks in the form of rectangles. Pixels of the orthorectified image block that do not have an associated pixel in the source image will be assigned a predefined color value. Examples of predefined color values are colors corresponding to non-existing road surface colors or values that would normally not exist or hardly exist in the source image. This reduces the likelihood of errors in the further processing of the orthorectified image block.

In an embodiment of transforming a source image to obtain orthorectified image blocks for each pixel 216 (having a distance 214 from the visual axis and a distance 204 from the focal point 208), the corresponding location in the source image is determined by means of a geometry described in more detail in the unpublished patent application PCT/NL2006/050252, which is incorporated herein by reference. It should be noted that the resolution (the physical size represented by each pixel) is changed (made larger) when converting the source image into an orthorectified image. Size increase is accomplished by averaging the color values of the associated pixels in the source image to obtain the color values of the pixels of the orthorectified image. The averaging has the effect of clustering the road surface color samples and reducing noise within the process.

Each ortho-corrected image is stored with associated position and orientation data. With the position and orientation in the coordinate reference system, the geographic location of each pixel of the orthorectified image is defined. The position and orientation data enables the processor to superimpose the orthorectified images to obtain larger orthorectified images or orthorectified mosaics. Fig. 7 shows an orthorectified image of a road segment, which has been obtained by superimposing 5 orthorectified images 702 … 710. The superposition may be based on metadata of the respective orthorectified image block. The metadata for each ortho-corrected image block is derived from a position determination function, including from the geographic position of the moving vehicle, the direction of travel or orientation of the moving vehicle, the position of the camera on the moving vehicle, and the orientation of the camera on the moving vehicle. The parameters used to derive the geographic position of the orthorectified image block are stored as position and orientation data associated with the source image. The pixel values in the overlap region 712 may be obtained by averaging the pixel values of the overlapping pixels or by selecting the value of one of the overlapping pixels. In one embodiment, pixel values are selected from an image in which the distance between the geographic locations corresponding to a pixel is closest to the geographic location of the camera that captured the image. In this way, the best resolution in the combined ortho-corrected image of road segments is preserved.

Fig. 7 further shows lines 714, 716 corresponding to the left and right sides of the road surface. These sides can be detected by the methods disclosed in unpublished International patent application PCT/NL2007/050159, which is incorporated herein by reference. Detection of the left and right sides of the road surface is used to reduce the number of pixels processed for finding horizontal lane information. Fig. 7 shows a two-lane roadway and side spaces. Fig. 7 may be obtained by processing images obtained from a front-view camera. To obtain a five lane roadway, more than one camera is required. An orthorectified mosaic of the roadways of five lanes may be obtained by processing a sequence of images produced by one or two front-view cameras and two side-view cameras.

Fig. 7 shows a small region having a color value similar to that of a dotted line in the middle of a road segment. The selected size of the filter to be used will determine whether the small region will be removed or emphasized in the filtered image.

Fig. 8 shows a flow diagram of an example embodiment of a process of asymmetric filtering. The size of the filter mask in fig. 8 and 9 is chosen for explanatory reasons only. One skilled in the art will select the appropriate size depending on the situation and requirements. The inputs to the process are an orthorectified image 902 of a road segment and associated position and orientation data and data 904 defining the direction of the road segment. The orthorectified image 902 represents a substantially straight portion of the road. The road may have any arbitrary orientation and location in the orthorectified image 902. The data 904 defining the direction of the road segment describes the geographic location and orientation of the road in a predefined coordinate system. By combining the position and orientation data associated with the ortho-corrected image with data 904, the ortho-corrected image may be transformed into a transformed image 908, where the direction of the road is parallel to the columns of pixels. The transformation process is indicated by process block 906. The transformation process performs at least a rotation function on the orthorectified image. In the case where the resolution of the orthorectified image varies from image to image, the transformation process further comprises a scaling function that enables the transformation function to obtain transformed images that all have the same resolution. Further, the transformation process calculates associated position and orientation data. The associated position and orientation data defines a relationship between the position of a pixel in the image and a corresponding geographic location. This enables us to determine the true size and location of the lane information for recognized lane information in the transformed image. With the transformation process, a normalized orthorectified image and corresponding position and orientation data are obtained.

The transformed image has data that is typically noisy, not only due to the noise of the camera but also due to scratches and speckles appearing on the road surface. As such, extracting lane information from an image is relatively difficult, especially when the direction of the road in the image is not known.

Typically, such images are filtered by means of a filter having a symmetric filter mask or a two-dimensional window. The filter transforms one image into another based on a particular function. This function outputs, for each pixel of the input image, the value of the corresponding pixel in a new image computed as a combination of neighboring pixel values. The considered neighboring pixels form a so-called filter mask. The symmetric filter mask has the following advantages: the filtering functions performed on the image are equivalent in the horizontal and vertical directions. Filters with symmetric filter masks are extremely useful in image processing for improving images where the orientation of the object is not known. Examples of improvements are modifying features of the object to be recognized (e.g. edges of the object in the image), smoothing noise, etc. Symmetric morphological filters are generally known to those skilled in the art. Morphological filters provide a wide range of operators to image processing, all based on several simple mathematical concepts from set theory. The operators are particularly useful for analysis of binary images and common uses include edge detection, noise removal, image enhancement, and image segmentation. Some types of morphological filters are dilation, erosion, open, close.

Since the orientation of the object to be recognized is known, i.e., parallel to the pixel columns, the image may be filtered using an asymmetric filter before the desired information is recognized from the image. Asymmetric filtering is a data preparation or data adjustment step applied prior to the actual feature detection or recognition step. For implementations of lane information identification, a filter is used to amplify the required information and reduce noise in the same time. Lines in the filtered image according to lane information may be recognized by, for example, a hough transform.

The transformed image 908 is supplied to a horizontal filter 910 to obtain a horizontal filtered image. A horizontal filter is a filter with an asymmetric filter mask or one-dimensional window, where the mask or window encompasses only three or more adjacent pixels in a pixel row. The horizontal filter may have a filter mask 912 as shown in fig. 9. The filter mask 912 has a window corresponding to 5 adjacent pixels in a row of pixels. When the output value is calculated by means of the filter mask 912, the value will be assigned to the pixel 952 in the associated horizontal filtered image. The location of pixel 952 in the horizontally filtered image is similar to the location in the transformed image. Thus, the output value will be assigned to the pixel in the middle of the window defined by the horizontal filter mask 912. In one embodiment, the horizontal filter is a maximum filter. The maximum filter is a filter in which the output value corresponds to the value of the pixel having the maximum value according to the mask. In an embodiment, the maximum value corresponds to a pixel value having an RGB value, the RGB value corresponding to a highest luminance.

The algorithm to perform maximum filtering for pixels of the input image may be described as follows:

-calculating a luminance from the RGB values for each pixel in the mask,

-finding the pixel with the highest luminance in the mask,

-retrieving the RGB value corresponding to the highest pixel, an

-assigning the RGB values to the output of the filter.

This filter is applied to all pixels of the entire image.

The horizontal filter 910 corresponds to a filter having a structured element that expands the width of a line in a direction perpendicular to the vehicle traveling direction. The structured elements are small grids representing pixels and are structures that are applied to an image to change the image content.

Next, the horizontally filtered image is supplied to a first vertical filter 914 to obtain a vertically filtered image. A vertical filter is a filter with an asymmetric filter mask or one-dimensional window, where the mask or window encompasses only three or more adjacent pixels in a pixel column. The first vertical filter 914 may have a filter mask 916 as shown in fig. 9. Filter mask 916 has windows corresponding to 5 adjacent pixels in a column of pixels. With the filter mask 916, a value will be generated for the pixel 956 corresponding to the pixel in the window covered by the vertical filter mask 916. In an embodiment, the first vertical filter is a minimum filter. A minimum filter is a filter in which the output value corresponds to the value of the pixel having the minimum value according to the mask. In one embodiment, the minimum value corresponds to a pixel value having an RGB value corresponding to the lowest luminance or darkest pixel.

The algorithm to perform minimum filtering for the pixels of the input image may be described as follows:

-calculating a luminance from the RGB values for each pixel in the mask,

-finding the pixel with the lowest luminance (darkest pixel) in the mask,

-retrieving the RGB values corresponding to the darkest pixels, and

-assigning the RGB values to the output of the filter.

This filter is applied to all pixels of the entire image.

The first vertical filter 914 corresponds to a filter having structured elements that reduce the length of a line in a direction parallel to the vehicle traveling direction.

Optionally, the vertically filtered image obtained from the first vertical filter 914 is supplied to a second vertical filter 918 to obtain a filtered image 922. The second vertical filter 918 has a filter mask 920 as shown in fig. 9. The filter mask 920 has windows corresponding to 5 adjacent pixels in a column of pixels. In one embodiment, the second vertical filter 918 is a max filter.

The second vertical filter 918 corresponds to a filter having a structured element that expands the length of a line in a direction parallel to the vehicle traveling direction.

An asymmetric filter may be applied to the transformed image because the orientation of the lane information in the transformed image is known. The horizontal filter enlarges the width of the lane information in the horizontal direction of the image, and thus along a pixel row. In the RBG space, the lane markings have a lighter color than the road surface material. The maximum filter will stretch the width of the lane information by m-1 pixels, where m is the number of adjacent pixels of the filter mask. For example, a vertical line having a width of one pixel will become a vertical line having m pixels. In addition, the maximum filter reduces noise along the rows of pixels in the image.

The first vertical filter, which is the minimum filter, reduces noise along the row and removes one or more pixels that have relatively bright color values but cannot represent lane information. The first vertical filter will reduce the vertical size of the lane information by n-1 pixels, where n is the number of adjacent pixels of the filter mask. The value of n should be less than the length in the pixels of the smallest line segment to be detected. It has been found that n values in the range of 10 to 25 pixels are well suited for filtering a dashed line having a length of 3m in an image having a pixel size of 8x8 cm.

If n has a value greater than the maximum vertical length in the pixels of the small region in the middle of the road segment shown in FIG. 7, that small region will be removed. The line width of the broken line in fig. 7 is 2 pixels. The small region has a length of 8 to 9 pixels. Therefore, in the case where n is 10, the small region will be removed.

Optionally, a second vertical filter, which is a maximum filter, further reduces noise and also restores the vertical size of lane information if the size of the filter mask is similar to the size of the first vertical filter. This option is necessary if the length of the lane information must be determined accurately.

With the horizontal filter 910 and the first vertical filter 912, a filtered image is obtained in which the position of the lane information parallel to the road direction, e.g., defined by the trajectory or centerline of the vehicle, can be accurately determined. The filtered image obtained by filtering with the aid of the asymmetric filter mask is well suited for further recognition of the lane information. The width of the lane information is enlarged while the position of the center line of the lane information remains unchanged. Furthermore, the lane information and the road surface are noise reduced without losing important information.

To accurately determine the length of the lane information (e.g., the individual lines of the dashed line), the size of the filter mask of the first vertical filter should be similar to the size of the second vertical filter. However, by knowing the size of the first vertical filter 914, the length of the lane information derived from the output image of the first vertical filter can be corrected accordingly. Similarly, the width of the lane information may be found by correcting the width found in the filtered image independent of the size of the horizontal filter.

The number m of neighboring pixels of the mask of the first vertical filter should be smaller than the line segment in the transformed image that we are looking for. Otherwise, the lane information will be erroneously removed.

The number n of adjacent pixels of the mask of the horizontal filter depends on the size of the first vertical filter and the deviation of the road direction from the direction of the lane information to be detected. The trajectory of a mobile mapping vehicle may be used as an estimate of road direction. However, if the vehicle changes lanes, the direction of the vehicle will deviate from the true direction of the road. The deviation will result in an improper transformation of the orthorectified image, which means an improper rotation. The lane information will not be parallel to the pixel columns and will have an angle corresponding to the deviation. The following equation defines the relationship between n and m:

wherein:

m is the size of the number of pixels of the mask of the horizontal filter,

w is the minimum width in pixels in the transformed image of the lane information to be detected,

n is the size of the number of pixels of the mask of the first vertical filter, and

angle _ of _ vehicle is the maximum angular difference between the heading direction of the car and the actual direction of the road in degrees.

In one embodiment, the following values apply: n-10, w-3 and m-4. These parameter values allow for an angle of _ vehicle of up to 30.9 degrees. The angle of _ vehicle is typically within 5 degrees for a straight-driving vehicle. The angle of _ vehicle of a sudden lane change may be up to 9 degrees. It can be clear that the values n, w and m of angle of _ horizontal depend on the horizontal and vertical resolution of the transformed image.

The above equation describes the maximum allowable angle _ of _ vehicle. The angle of _ vehicle will correspond to the angle of the lane information in the transform relative to the pixel column. When the vehicle is driving on the road curve, the first and last pixels of the lane marker in the transformed image will also have an angle relative to the pixel column. Thus, the above equation may also be used to determine the optimal values for m and n for a predetermined maximum amount of curvature. The values m and n may be selected by those skilled in the art to optimize the filtering process for the conditions encountered with respect to vehicle angle and amount of curvature.

If the above equation is not met, the line may be divided into more than one segment or the length of the line may be erroneously reduced. It is further obvious to the person skilled in the art that m should be smaller than the minimum number of pixels between two parallel lines. Otherwise, the two lines will be joined and treated as one line.

Fig. 10 shows a filtered image of a road segment. The road segments have a resolution similar to the road segments shown in fig. 7. FIG. 10 shows a dashed lane divider 1002 and left and right curb solid lines 1006, 1004, respectively. It can be seen that the asymmetric horizontal and vertical filters magnify the width of the lane information. Furthermore, pixel noise in the filtered image is reduced.

The filtered image shown in fig. 10 is applied to a line recognition method. To reduce the number of false detections, the area in the filtered image used to recognize lane information may be defined at the location of the left and right sides 1010, 1008 of the road surface. Since the lane information is only on the road surface, image areas outside the road surface may be discarded. The left and right sides of the road surface may be determined by the method disclosed in the unpublished international patent application PCT/NL 2007/050159. A simple pattern matching algorithm to detect lane information may be used as a line having a strict shape. In the present application, it is sufficient for the pattern recognition algorithm to search only two types of patterns: solid lines and rectangles as portions of dashed lines. Further, the type of line may be determined in consideration of the color of the line. Fig. 11 shows the regions of features present in the filtered image like that shown in fig. 10. Since the positions of the left and right sides of the road can be determined prior to analyzing the filtered image, the roadside can be closed from analysis. Therefore, the white spot at the right side of fig. 10 is not recognized as the lane information.

The embodiments described above all use an orthorectified image on which the transformation and asymmetric filtering is performed. The orthorectified images can be easily combined to obtain an orthorectified mosaic of road segments or roads with more than five lanes, for example. It should be noted that instead of generating an orthorectified image of 16mx16m from the source image, a top view image may be generated instead. The top-view image is an image in which each pixel is seen from above in one point. A camera with a visual axis perpendicular to the earth's surface will provide an overhead view image. The processing of the overhead view image by means of the method according to the invention will provide a filtered image which is well suited for recognizing lane information. By knowing the projections used, the location of the geographical reference of the lane information can be accurately derived from the asymmetrically filtered overhead view image.

As described above, lane information may be generated using an aerial or satellite ortho-corrected image. The road for which lane information must be determined is now sampled. Each sample has a position and orientation in a coordinate reference system. For each sample, pixels of the aerial or satellite image corresponding to the area around the pixel (e.g., 20mx20m) are first transformed into an image in which each pixel column of the transformed image corresponds to a surface parallel to the direction of the road. The transformed image is then asymmetrically filtered.

It should further be noted that the images as collected in a mobile mapping vehicle need not be georeferenced for the purpose of applying asymmetric filters or even for a lane recognition step. The camera is mounted on the vehicle. Thus, the orientation of the camera with respect to the direction of travel is known. This allows us to generate filtered images and detect features in the images without geo-referenced position and orientation data. Inertial Navigation Systems (INS) enable us to derive relative position information in a coordinate reference system for each detected feature. Some form of geographic reference requires sufficient accuracy to match detected lane information to the appropriate road in the map database. Such geographic references and map matching are well known in the art.

It should also be further noted that a geographical reference of sufficient accuracy is required for the aerial and satellite images so that the road position and orientation from the appropriate road can be used to determine where and in what orientation on the image the method according to the invention is applied and then correlate the generated lane information back to the appropriate road in the map database.

The present invention studies image blocks of a preselected length corresponding to a road. From the images taken by the mobile mapping vehicle, an image with a length of 16m of the road can be generated. A very curved road is a road with a radius of up to 100 m. The arc of this curved lane marker will have a 10 degree deviation between the start and end points for 16 meters in front of the displayed car. However, the applied asymmetric filter should be large enough to eliminate the assumed noise level but should always be small enough to eliminate the lane markers we are looking for. The smallest lane we are looking for is marked as a dashed line. The dashed line is typically 3m long. Therefore, the size of the filter should be less than 3m in real world coordinates. In a suitable embodiment, the size of the first vertical filter is 25 pixels. For a pixel size of 8x8cm, this corresponds to a filter having a length of 2 m. The deviation of the arc within 2m will only vary by 1 to 2 pixels. This deviation does not introduce problems when using the method according to the invention for curved roads.

Similarly, lane changes do not introduce problems. When driving at 40km/h and changing lanes within 2 seconds, which is extremely fast, the vehicle moves forward 25m and sideways 4 m. This corresponds to a 9 degree deviation from the road direction.

As described above, an orthorectified image of a road segment (for example) can be processed by the method according to the invention. The recognized road information, along with its type and location and size (if necessary), is stored in a database for use in a digital map database. The geographic location of the lane information may be derived from the position and orientation data associated with the transformed image. The position may be in the form of a geographical position in a predefined coordinate system as an absolute position or as a relative position with respect to the trajectory of the vehicle or the position of the road obtained from a database. Determining a direction of a road using the location information of the road from a database. Lane information may be used to generate a more realistic view of the road surface in a navigation system. For example, narrowing of a road from, for example, three lanes to two lanes may be visualized. Furthermore, the lane information may be highly suitable for determining, for example, non-parking/parking areas and road segments in which exceeding of other cars is prohibited. Furthermore, as navigation becomes more accurate, the position of the vehicle within the lane may become part of the system for safety applications as well as finer direction cues (moving to the left lane to facilitate your upcoming turns).

Needless to say, this accurate detection of lane markings and successive lane positions also enables the system to automatically determine the number of lanes and the width of the lanes. This may be done by searching for two or more parallel solid or dashed lines in the filtered image. Subsequently, the number of lanes can be easily derived. Furthermore, the width of the corresponding lane may be calculated from position and orientation data associated with the one or more source images.

The above described method may be performed automatically. It may happen that the quality of the image is such that some correction is required to perform the image processing tool and the object recognition tool of the present invention. For example, lane information for a continuous road segment may be analyzed for discontinuities along a road. For example, a pair of consecutive road segments may have different location and/or type information along the road. In that case, one may review the corresponding transformed image or orthorectified image. Now, the method includes some verification and manual adaptation actions to achieve the possibility of confirming or adapting the intermediate results. These actions may also be adapted to accept intermediate or final results of the lane information generation. By these actions, the quality of the database including the lane information can be improved.

The lane information generated by the present invention generates lane information for each orthorectified image and stores it in a database. The lane information may be further processed to reduce the amount of information. For example, road information corresponding to an image associated with a road segment may be reduced to one parameter of the road width of the segment. Furthermore, if the road section is sufficiently smooth, the lane separator may be described by a set of parameters including at least the end point and shape point of the section. The line representing the lane separator may be stored by a coefficient of a polynomial. Alternatively, for a set of regularly positioned lanes, lane information may be embodied in width and deviation from the centerline.

The foregoing detailed description of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. For example, by changing the horizontal and vertical maximum filters and the vertical and horizontal minimum filters, road information corresponding to road markings of a line perpendicular to the road direction may be generated.

Furthermore, the minimum and maximum filters may be replaced by any other kind of asymmetric filter, such as edge detection filters, image clustering filters, statistical filters. The transformation based on the estimated road direction provides a transformed image in which the features to be detected have a known orientation, i.e. the vertical orientation of the lane separator. This enables us to use a filter that is only effective in filtering an image in one direction, i.e. along a row of pixels of a column of pixels. This is in contrast to a symmetric filter that is capable of detecting features in an image having either orientation. Thus, any filter or combination of filters having a filter mask based on the orientation of features to be detected in the transformed image 46 may be used in a method according to the present invention. The known orientation of the feature in the image enables us to use a less complex filter than would have to be used if the orientation of the feature was not known in the image.

The described embodiments were chosen in order to best explain the principles of the invention and its practical application, to thereby enable others skilled in the art to best utilize the invention in various embodiments and with various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims appended hereto.

Claims

1. A method of generating lane information for use in a map database, comprising:

acquiring one or more source images and associated position and orientation data of a road surface, the road having a direction and lane markings parallel to the direction of the road;

acquiring road information representing the direction of the road;

transforming the one or more source images in dependence on the road information to obtain a transformed image, wherein each pixel column of the transformed image corresponds to a surface parallel to the direction of the road;

applying a filter having an asymmetric mask to the transformed image to obtain a filtered image; and a process for the preparation of a coating,

generating lane information from the filtered image as a function of the position and orientation data associated with the one or more source images.

2. The method of claim 1, wherein the road information is obtained from a map database.

3. The method of claim 1, wherein the road information is obtained from tracking information generated by a position determination means mounted in a moving vehicle traveling on the road.

4. The method of any one of claims 1, 2, and 3, wherein the one or more source images are aerial or satellite images.

5. The method of claim 4, wherein the image is an orthorectified image.

6. The method of any one of claims 1, 2 and 3, wherein the one or more source images have been obtained by:

retrieving one or more image sequences and associated position and orientation data obtained with one or more land-based cameras mounted on a moving vehicle traveling on the road; and a process for the preparation of a coating,

a normalization process is performed on the one or more sequences of images to obtain the one or more images and position and orientation data associated with the one or more images.

7. The method of claim 6, wherein the one or more cameras include at least one camera having a boresight along a direction of travel of the moving vehicle.

8. The method of claim 6 or 7, wherein the one or more cameras include at least one camera having an oblique boresight relative to the direction of travel of the moving vehicle.

9. The method of claim 6, 7, or 8, wherein the one or more cameras include at least one camera having a visual axis perpendicular to the direction of travel of the moving vehicle.

10. The method according to one of claims 6-9, wherein the normalization process produces an orthorectified image.

11. A method according to any of claims 6 to 10 as dependent on claim 3, wherein the tracking information and associated position and orientation information for the one or more image sequences have been captured simultaneously from output signals generated by position determining means mounted in the moving vehicle.

12. The method of any one of claims 1-11, wherein transforming includes a rotation operation.

13. The method of any of claims 1-12, wherein applying a filter with an asymmetric mask comprises:

applying first a first filter having structured elements that expand a width of a line in a direction perpendicular to the driving direction of the vehicle and second a second filter having structured elements that reduce a length of a line in a direction parallel to the driving direction of the vehicle to the transformed image.

14. The method of claim 13, wherein the first filter is a maximum filter and the second filter is a minimum filter.

15. The method of claim 13 or 14, wherein applying a filter with an asymmetric mask further comprises:

a third filter is again applied, which has a structuring element that enlarges the length of the line in a direction parallel to the direction of travel to its original size.

16. The method of claim 15, wherein the third filter is a max filter.

17. The method of any of claims 1-16, wherein generating lane information comprises:

searching for a solid line in the filtered image; and a process for the preparation of a coating,

calculating the position of the solid line from the position and orientation data associated with the one or more source images.

18. The method of any of claims 1-16, wherein generating lane information comprises:

searching for a rectangle in the filtered image; and a process for the preparation of a coating,

calculating a position of a rectangle from the position and orientation data associated with the one or more source images.

19. The method of any of claims 1-16, wherein generating lane information comprises:

searching for two or more parallel solid or dashed lines in the filtered image; and a process for the preparation of a coating,

calculating a width of a lane from the positions of the two or more solid or dashed lines from the position and orientation data associated with the one or more source images.

20. The method of any of claims 1-17, wherein generating lane information comprises:

and calculating the number of lanes according to the two or more solid lines or the broken lines.

21. An apparatus for performing the method of any one of claims 1-20, the apparatus comprising:

an input device;

a processor-readable storage medium; and

a processor in communication with the input device and the processor-readable storage medium;

an output device for enabling connection with a display unit;

the processor-readable storage medium stores code to program the processor to perform a method comprising:

acquiring one or more source images and associated position and orientation data;

obtaining tracking information of a vehicle traveling on a road having lane markings parallel to a direction of the road;

transforming the one or more source images in dependence on the tracking information to obtain a transformed image, wherein the transformed image represents a road surface in front of the vehicle and each column of pixels corresponds to a surface parallel to a direction of travel of the vehicle;

22. A computer program product comprising instructions which, when loaded on a computer arrangement, allow said computer arrangement to perform any of the methods according to claims 1-20.

23. A processor readable medium carrying a computer program product, which when loaded on a computer arrangement, allows said computer arrangement to perform any of the methods of claims 1-20.