WO2000011862A1

WO2000011862A1 - A method and device for the alignment of digital images

Info

Publication number: WO2000011862A1
Application number: PCT/US1999/018923
Authority: WO
Inventors: Michael P. Keyes
Original assignee: Applied Science Fiction, Inc.
Priority date: 1998-08-21
Filing date: 1999-08-17
Publication date: 2000-03-02
Also published as: AU5492599A; TW456145B

Abstract

A method of processing a plurality of digital images, comprising forming a first digital image of an image medium, wherein said first digital image is a first raster image having a first plurality of pixels; determining a plurality of first reference locations of a plurality of reference markers of said image medium relative to said first raster image, said plurality of reference markers being substantially fixed with respect to said image medium; forming a second digital image of said image medium, wherein said second digital image is a second raster image having a second plurality of pixels; determining a plurality of second reference locations of said plurality of reference markers of said image medium relative to said second raster image; determining a mapping rule that transforms said first reference locations to said second reference locations for each of said plurality of reference markers; and mapping said first plurality of pixels into said second plurality of pixels based on said mapping rule.

Description

A METHOD AND DEVICE FOR THE ALIGNMENT OF DIGITAL IMAGES

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention pertains to a digital image processing device and method, and more particularly to a digital image device and method that aligns and/or combines a plurality of digital images.

2. Description of the Related Art

Numerous scanning devices are known for capturing- digital images of objects. For example, there are numerous flat-bed and sheet-fed scanners currently available on the market for converting photographs, pages of text, and transparencies to digital images. There are also conventional film scanners available which scan photographic film to produce digital images. Most of the current film scanners use one set of three linear charged coupled devices (CCD) to scan photographic film. Each of the three CCD's scans in one region of the visible spectrum: typically red, green and blue channels. The three CCD's are fixed in a rigid structure with high precision at the time of manufacturing the scanner. In such conventional scanners, image data is captured in each color channel (e.g., red, green and blue) at substantially the same time. The three CCD's pass over the film once, thus providing three separate color channel scans at substantially the same time. The data from the three separate color channels are aligned based on the precise physical dimensions known at the time of manufacturing the scanner.

Another type of film scanner is described by Edgar in U.S. Patent Nos. 5,519,510 and 5,519,550, the entire contents of which are incorporated herein by reference. Edgar teaches a device and method for chemically processing and scanning photographic films. Scanning the photographic film in separate color channels with different sets of CCD's positioned on either side of the photographic film produces images of superior quality according to the methods and device of Edgar. Edgar also teaches that it is advantageous to perform multiple scans of the film with each of the separate color channels using different sets of CCD's. For example, it may be advantageous to capture data for the image highlights at an early stage of development of the photographic film, to capture the mid-tones at an intermediate stage of development, and to capture the shadow image data at a later stage of photographic film development. Previous methods of aligning the scanned images by precision construction at the time of manufacture are inadequate for devices which have image channels located at different places and/or taken at different times. Due to the multiple scans for separate color channels with multiple sets of CCD's, the previous methods of aligning images result in fuzzy and unappealing images with color fringes around the edges when they are used to align images from spatially separated image channels.

Most prior art references which address the problems of aligning photographic film are directed more towards the field of motion picture film to correct undesirable motions of the film at the time of viewing. The patent to Witte, U.S. Patent No. 5,734,171 is an example of such a prior art reference. Witte is directed to automatically correcting vertical and horizontal picture unsteadiness for scanning motion picture film through a telecine film scanner. The device of Witte is constructed such that photosensors are positioned specifically over portions where the perforation holes will be observed. However, these sensors produce an analog signal which merely signals when pixel lines should be read from a frame store. The device of Witte is specialized for telecine film scanners in which the film is in continuous motion through the scanner. Furthermore, it corrects only lateral deviations of the film. The prior art devices, such as that of Witte, are not readily adaptable to scanners other than telecine film scanners and do not permit more general corrections in both lateral and longitudinal directions, and for rotational and magnification errors.

SUMMARY OF THE INVENTION

Accordingly, it is an object of the present invention to provide a digital image processing device and method to align two or more image channel scans.

It is another object of the present invention to provide a digital image processing device and method to align two or more separate image channel scans before combining them into a final digital image.

It is another object of the present invention to provide a digital image processing device and method to align each of separate image channels in such a way that, when objects in an image are represented by digital signals in at least two image channels, the signals from those objects are in direct alignment in the final image. It is another object of the present invention to provide a digital image processing device and method that corrects translational misalignments in separate image channels.

It is another object of the present invention to provide a digital image processing device and method that corrects for rotational errors between two separate image channels.

It is a further object of the present invention to provide a digital image processing device and method which corrects magnification errors between at least two separate image channels.

It is yet another object of the present invention to provide a digital image processing device and method which corrects for any combination of translational, rotational and magnification errors between at least two image channels.

The above and related objects of the present invention are realized by providing a digital image processing device and method in which two or more digital images are formed from separate scans. The separate scans may be from different detectors at different locations, or the same detectors at different times. Preferably, at least two separate scans of a photographic film in different color channels are made in different locations from one another. The separate color channel scans, according to the present invention, are combined to form the final color digital image.

Although the following description refers to the case in which there is one image channel at one scanning station and a second image channel at a second scanning station, this is presented as a simple case which is useful for describing the broader concepts of the invention. The invention includes both single image channels at each of a plurality of scanning stations, as well as a plurality of image channels at one, some, or all of the scanning stations. In practice, three or four image channels at each of two or more scanning stations has been found to be a useful configuration of the invention.

According to the present invention, an image in the first image channel is processed to determine a location of at least one reference mark on the image medium which is fixed relative to the image medium. The location of the same reference mark is then determined relative to a second image in the second image channel. Since the reference mark is fixed relative to the image medium, one can then determine a relationship to map the first image into the second image based on the requirement that the reference mark, as expressed in terms of the two separate images, must coincide in the final image. The determination of locations of two or more reference marks provides a more precise mapping and a greater range of different types of mappings. For example, two or more reference points are required in order to correct focusing errors.

In the preferred embodiment, the scanning device is a photographic film scanner, and the image medium is a photographic film. The sprocket holes on conventional photographic film provide suitable reference marks that are fixed in the image medium since they are currently manufactured according to industry standards. Preferably, at least the regions around the particular sprocket holes that are to be used as reference marks are preprocessed by a high-pass spatial filter. The pre-processing of the region of the images around the sprocket holes by the high-pass spatial filtering provides an image of the edges of the sprocket holes as the output from the filter.

Although sprocket holes in photographic film provide a preferred embodiment for the reference marks, the scope of the invention includes other types of reference marks that are fixed relative to the medium. For example, for film that has no reference holes, one can punch notches into the film at known fixed locations on the film.

In the preferred embodiment, each scanned image of the photographic film has uniquely numbered pixel columns extending across and substantially orthogonal to the short length of the photographic film. Preferably, there are pixel rows extending along the entire length of the photographic film which are orthogonal to the pixel columns. The rows and columns are each numbered sequentially such that a unique combination of row and column number specifies a unique pixel in the given image. The reference mark detector detects a reference mark and assigns a unique combination of row and column number to identify the location of the reference mark with respect to the subject image. The reference mark detector is not limited to assigning only integer values of row and column number to the reference mark.

In the preferred embodiment, the reference mark detector assigns a weighted average row number and a weighted average column number to each detected reference mark. In general, a detected reference mark will be assigned a fractional row number and a fractional column number. Generally, an edge of the sprocket hole which is substantially parallel to the pixel columns will actually extend over a plurality of pixel columns. In the preferred embodiment, a weighted average pixel column is assigned to that edge by weighting each column number along that edge by the number of edge pixels in that column. A similar procedure is performed to detect an edge of the sprocket hole which is substantially parallel to the pixel rows. The intersection of the line corresponding to the weighted average column number and the line corresponding to the weighted average row number provides a fiducial point which serves as a reference mark fixed in the image medium, but expressed relative to the subject image.

The above-noted procedure is repeated for at least a second image to determine the location of the same reference mark expressed relative to the second image. An image combiner determines a mapping from the requirement that the reference mark in the first image coincide with the same reference mark as expressed in terms of the second image. This thus provides a mapping relation to map the entire first image such that it coincides with the entire second image.

This procedure is not limited to only two images. For a third image, the fiducial point for the same reference mark that is fixed relative to the image medium is determined in terms of the third image. In this case, the fiducial point expressed in terms of each of the three images must coincide. This provides a mapping relation to map all three images into coincidence. Typically, a plurality of images at one scanning station are aligned with each other and with at least another plurality of images from at least another plurality of images from at least another scanning station. Preferably, one determines a plurality of fiducial points in each image. In this case, the fiducial points determined with respect to each image are required to coincide with the fiducial points corresponding to the same physical reference mark. Once the separate images are aligned, one may combine the images by conventional methods.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects and advantages of the present invention will become more apparent and more readily appreciated from the following detailed description of the presently preferred exemplary embodiment of the invention taken in conjunction with the accompanying drawings, of which: FIGURE 1 is a schematic illustration of the digital image processing device according to the preferred embodiment of the invention;

FIGURE 2 is a schematic illustration showing a more detailed view of the digital image processing device according to the preferred embodiment of the invention;

FIGURE 3 shows views of a photographic film which help illustrate the filtering and detection devices and methods according to the preferred embodiment of the invention;

FIGURE 4 is a flow-chart illustrating the digital image processing method according to the preferred embodiment of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

The image processing device according to the present invention is designated generally by the reference numeral 10 in Figure 1. In the preferred embodiment, the image processing device 10, generally has an image scanning device 12, a data processor 14, an output display device 16 and an input device 18. In addition, the image processing device 10 may include one or more generally available peripheral devices 20. In the preferred embodiment, the image scanner 12 is an electronic film developing apparatus as described in U.S. Patent Application No. 08/955,853, the data processor 14 is a personal computer or a workstation, the output device 16 is a video monitor, and the input device 18 is a keyboard. Although the image scanner 12 is an electronic film developer in the preferred embodiment, the image scanner is not limited to being only an electronic film developer. Other scanning devices, including devices which scan media other than photographic film, are encompassed within the scope and spirit of the invention.

Figure 2 is a schematic illustration which illustrates some components interior to the electronic film developer 12, and a schematic representation of the data processing elements of the data processor 14. The scanning device 12 has at least two scanning stations 24 and 26. In another preferred embodiment, the scanning device 12 has a third scanning station (not shown). A greater or lesser number of scanning stations than that of the preferred embodiments may be used. In the preferred embodiment, exposed photographic film 22 is directed to move through the scanning stations in the longitudinal direction 28. The photographic film 22 has reference markers 30 at one transverse side of the photographic film 22. Preferably, the reference markers 30 are sprocket holes, such as the sprocket hole 32, in the photographic film 22. The photographic film 22 has additional reference markers 36 in the transverse direction 34 opposite to the reference markers 30. A portion of the photographic film 38 is scanned over a time interval at scanning station 24. At a later time, the same section of film 38 scanned at scanning station 24 is displaced to reach scanning station 26 (represented by reference numeral 38') so that it is scanned at scanning station 26. Preferably, each scanning station 24 and 26 includes at least one illumination source, a detector and imaging optics. Scanning stations for electronic film development are described in more detail in U.S. Patent No. 5,519,510 and U.S. Patent No. 5,155,596.

In electronic film development, the photographic film 22 is typically subjected to film development treatment prior to the scanning station 24 and another film development treatment between scanning stations 24 and 26. Consequently, the film will be at one stage of development at scanning station 24, and at another stage of film development at station 26. However, there are many different embodiments of digital film development, as noted in the above-referenced patent application and in U.S. Patent No. 5,519,510. This invention is not limited to a specific type of digital film processing, and is generally applicable to film processing in which it is necessary to establish a location and/or orientation of a scanned image relative to the scanned medium.

Scanned image data is transferred from each scanning station 24 and 26 to the data processor 14. The data processor 14 has a digital image data processor 40 that is in communication with scanning station 24 and scanning station 26. The digital image data processor 40 is also in communication with a data storage unit 42 that stores processed image data, preferably in a conventional raster image format. The data storage unit 42 is in communication with a high-pass spatial filter 44 such that it receives stored raster images from the data storage unit 42. A reference mark detector 46 is in communication with the high-pass spatial filter 44 such that it receives filtered images from the high-pass spatial filter 44. The reference mark detector 46 is also in communication with the data storage unit 42. The image combiner 48 is in communication with the reference mark detector 46 and the data storage unit

42. In the preferred embodiment, the digital image data processor 40, the high-pass spatial filter 44, the reference mark detector 46 and the image combiner 48 are implemented in practice by programming a personal computer or a workstation. However, the invention includes other embodiments in which the components are implemented as dedicated hardware components.

Preferably, the digital image data processor 40 is a conventional digital image data processor that processes scanned data from scanning stations 24 and 26 and outputs a digital raster image in a conventional format to be stored in data storage unit 42. The data storage unit 42 may be either a conventional hard drive or random access memory (RAM) chip, or a combination of both. Preferably, the high-pass spatial filter uses a conventional spatial mask such as that described in R.C. Gonzalez and R.E. Woods, "Digital Image Processing", pages 189-249, the entire contents of which are incorporated herein by reference. In such a high-pass spatial filter, a three-pixel-by-three-pixel mask is usually sufficient, although one may select larger masks. For a three-pixel-by-three-pixel mask, the center mask element is given a weight with a value of 8 and each neighboring pixel is given a weight of -1. The mask is then applied to each pixel of the raster image. If the subject pixel is in a fairly uniform region of the image, the sum of all the neighboring pixel image values multiplied by the mask value will cancel with the central value, thus leading to essentially a zero output value. However, in the region of an edge in the image, the output will be non-zero. Consequently, such a filter will provide an output image which represents edges of the original image.

FIGURE 3 is a schematic illustration of the photographic film 22 with labels that help illustrate the device and method according to the preferred embodiment of the invention. Regions of photographic images 50, 52 and 54 are represented schematically in FIGURE 3. A region of sprocket holes 56 is also shown in an enlarged view in FIGURE 3. An image scan at one of the scan stations 24 or 26 can be either a transmission scan in which illumination light is transmitted through the medium to be detected on the opposite side of the photographic film 22, or may be a reflection scan in which reflected light is detected. In either case, the sprocket holes provide edges in which there is an abrupt change in either the transmission or reflection characteristics of the photographic film 22. A high-pass spatial filtering of the region of the photographic film 56 that includes some of the sprocket holes 30 will produce an image of the sprocket hole edges.

In the preferred embodiment of the invention, each scanning station 24 and 26 has a plurality of image channels. In a preferred embodiment, two transmission and two reflection channels are used in each scanning station 24 and 26. In other embodiments, it was found to be sufficient to use two reflection channels and one transmission channel in each scanning station 24 and 26, or two transmission channels and one reflection channel.

The sprocket hole edge 58 is a schematic illustration of an image produced by the high- pass spatial filtering. The schematic illustration of the sprocket hole edge 58 is actually the negative of what one would normally obtain as output from the high-pass spatial filter 44. In other words, the region around the sprocket hole edge 58 would have output values of approximately zero, and thus would normally be assigned a zero intensity, or black, value. The sprocket hole edge 58 would then be bright against the dark, black, background. The reference mark detector 46 correlates the pixel rows and pixel columns with the edges of the reference mark. A single pixel column 60 and a single pixel row 62 is illustrated in FIGURE 3. As is well known in the art, a raster image has a series of pixel columns and rows extending across the image to form a rectangular grid of pixels.

The reference mark detector 46 assigns a column number to each pixel column along the entire scanned photographic film 22. Preferably, the pixel columns are numbered such that they increase linearly along the longitudinal direction 28 of the photographic film 22. For example, a pixel column number will increase by one unit going from one pixel column to the next pixel column along the longitudinal direction 28 of the photographic film 22. Similarly, the reference mark detector 46 assigns a row number to each pixel row along the transverse direction 34 such that the pixel row number increases linearly in the transverse direction 34 of the photographic film 22. The reference mark detector 46 then correlates the location of the raster image relative to the reference markers, such as sprocket hole 58. FIGURE 3 shows an example in which the sprocket hole edge image 58 is correlated with column 60 and row 62 of the raster image. The point of intersection 64 between the column 60 and row 62 of the raster image is referred to as the fiducial point. In general, each sprocket hole will be slightly different in size and shape due to manufacturing tolerances. Furthermore, the corners of the sprocket holes are not perfectly square. Finally, the sides of the sprocket holes may be slightly skewed with respect to the pixel rows and columns.

In the preferred embodiment, the fiducial point 64 is assigned weighted average row and column numbers. The weighted average column number for the side 66 of the sprocket hole 58 is determined by summing the total number of pixels in column 60 that correspond to the pixel edge 66. This procedure is repeated for all pixel columns in the region around pixel column 60. Each pixel column number, such as pixel column 60, and the pixel columns in the region around pixel column 60, are multiplied by the corresponding number of edge pixels 66 within that column. Each of the products is summed and the resulting sum is divided by a normalization factor such as the total number of pixels in all pixel columns corresponding to the edge 66. This provides a weighted average column number that corresponds to the fiducial point 64. In general, the weighted average column number will be a fractional value intermediate between two adjacent pixel columns of the raster image, although a precise coincidence with a pixel column is possible.

In the preferred embodiment, the reference mark detector 46 similarly assigns a weighted average row number to the fiducial point 64. However, in this case, the weighted average is performed with respect to the edge 68 of the sprocket hole edge image 58. Although the reference mark detector 46 assigns raster image row and column numbers according to a weighted average, the invention includes other less preferred embodiments in which row and column numbers are assigned in a different way. For example, one could assign the rows and columns according to whichever row and column has the maximum number of edge image pixels. There are numerous other currently less preferred reference mark detectors that are within the scope and spirit of the invention.

Although the reference mark detector 46 was described in regard to fiducial point 64, one can repeat the procedure for another fiducial point, for example, fiducial point 70 of the sprocket hole edge image 58. Furthermore, one could repeat the procedure for three or even four fiducial points for a given sprocket hole and/or reproduce the procedure for each sprocket hole along the photographic film 22. The more reference marks, such as fiducial points 64 and 70, that are determined for a given photographic film 22, the greater the accuracy. However, determining a greater number of reference marks increases the complexity and imposes greater computational demands. Suitable results were obtained by determining fiducial points for sprocket holes for each of the four corners of the scanned image. This permitted magnification errors to be corrected. The high-pass spatial filter 44 and reference mark detector 46 were described in reference to sprocket holes in a photographic film. However, the invention is more generally applicable to reference marks fixed in some form relative to the image medium as long as the reference marks have image edges which show up as output signals from the high-pass spatial filter. For example, one may deliberately impose indentations or apply a substance to serve as standard reference marks which are fixed relative to the image medium. FIGURE 3 also shows an example of notches 71 which one may punch into the film 22 to serve as reference marks instead of the sprocket holes. The example of FIGURE 3 shows an only two notches, but one would typically punch notches at substantially evenly spaced intervals along the entire length of each transverse edge of the photographic film.

The image combiner 48 receives data from the reference mark detector 46 and correlates the reference marks, such as fiducial point 64, with a first raster image obtained from scanning station 24. At some time later than the scan at scanning station 24, the sprocket hole 72 reaches the scanning station 26. The image combiner 48 also receives data from the reference mark detector 46 in which the fiducial point 64' of the same sprocket hole 72 is expressed in terms of column number 60' and row number 62' in a second raster image obtained from the scanning station 26. Since the sprocket hole 72 remains substantially unchanged between scanning stations 24 and 26, establishing a mapping rule between the fiducial point 64 in terms of column 60 and row 62 and the fiducial point 64' in terms of column 60' and row 62' provides a mapping rule which can be used for all other pixels in going from the first image to the second image. Such a mapping is not limited to only one reference marker. One may conduct the mapping for two or more fiducial points per sprocket hole and for a plurality of sprocket holes.

In operation, a photographic film 22 is fed through the scanning device 12 by a conventional drive mechanism (not shown). Conventional drive mechanisms usually have a sprocket which engages the sprocket holes to move the photographic film 22 through the scanning device 12. A region of the photographic film, for example, the region 38, traverses the scanning station 24 illustrated in FIGURE 2. As the region of the photographic film 38 is scanned, an image of the photographic film 22 is captured and processed by the digital image data processor 40 in at least one, but typically three or four image channels. The captured image includes at least a portion of a photographic film 22 which includes either sprocket holes 30 or sprocket holes 36, or both. In this way, a first digital image of substantially the entire photographic film 22 is generated as the entire length of the photographic film 22 is drawn across the scanning station 24 in the longitudinal direction 28. After captured image data from the scanning station 24 is processed by the digital image data processor 40, it is stored in data storage unit 42. The first raster image of the photographic film 22, captured at scanning station 24 and processed by the digital image data processor 40, can be stored in data storage unit 42 as either a single file or a plurality of files.

The region of the photographic film 38 reaches scanning station 26 at a later time. The region of the photographic film 38 at the later time is indicated schematically as reference numeral 38'. As the entire length of the photographic film 22 traverses the scanning station 26 by being moved in the longitudinal direction 28, a second digital image of the photographic film 22 is formed by processing the captured signals from scanning station 26 in the digital image data processor 40. The second digital image of the photographic film 22 is stored as conventional raster image data as either a single file, or plurality of files, in data storage unit 42 after being processed by digital image data processor 40.

The preferred embodiment is described in terms of two scanning stations 24 and 26.

However, the invention is not limited to only two scanning stations. It is often desirable to have at least a third scanning station displaced a distance along the longitudinal direction 28 relative to the scanning station 26. The invention anticipates generally a plurality of scanning stations in which scanning stations 24 and 26 in FIGURE 2 are shown as an example.

Each scanning station, such as scanning stations 24 and 26, may have a single illumination source, and a single detector. In other embodiments, at least one, or a plurality, of the scanning stations has a plurality of illumination sources and/or a plurality of detectors. The scanning stations may comprise illumination sources and detectors in visible regions of the electromagnetic spectrum, or other regions, such as in an infra-red region of the spectrum. For example, a scanning station may have conventional red, green, and blue detectors. Furthermore, the detectors may detect signals after illumination light has been either reflected or transmitted from the photographic film 22. In other embodiments, the detectors may detect reflected light in selected wavelength bands, and transmitted light in other wavelength bands. The first raster image stored in data storage unit 42 is filtered by high-pass spatial filter 44. Since the first raster image data includes at least a portion of the photographic film 22 which includes reference markers such as sprocket holes 30, the high-pass spatial filter 44 will generate an edge image of the portions of the sprocket holes included within the first raster image. The first raster image may, in general, include the entire surface of the photographic film 22; however, it has been found to be adequate to include only a portion of the sprocket holes. In addition, the high-pass spatial filter 44 can generally filter the entire first raster image. It has been found adequate to only filter a portion of the first raster image around a region where the sprocket holes 30 are expected. Furthermore, a portion of a sprocket hole, such as sprocket hole edge image 58 in FIGURE 3, which includes only approximately 30 pixels of an edge, such as edge 66 extending downward in FIGURE 3 from the fiducial point 64, to be sufficient. Such a limited high-pass spatial filtering of the first raster image was found to be adequate and more efficient than filtering the entire image. The edge images of the sprocket holes generated by the high-pass spatial filter 44 are transferred to the reference mark detector 46. The reference mark detector 46 assigns a column number and row number to a fiducial point, such as fiducial point 64, corresponding to a corner of the sprocket hole edge image 58.

In the preferred embodiment, a weighted average row number and a weighted average column number is assigned to the fiducial points. In the preferred embodiment, the weighted average is determined as described above. The scope and spirit of the invention includes other methods, including currently less-preferred detection methods.

The reference mark detector 46 is in communication with the data storage unit 42 so that all detected reference marks, such as the fiducial point 64 relative to the first raster image, are stored in the data storage unit 42.

The processing of the first raster image generated from scanning station 24 is described above. The procedure is repeated for a second raster image generated from scanning station 26. Similarly, this procedure can be repeated for a plurality of raster images obtained from a plurality of scanning stations. After the second raster image has been filtered by high-pass spatial filter 44 and the reference markers have been detected by reference mark detector 46, the data is stored in the data storage unit 42. The data includes the first raster image data and its corresponding relationship to at least one, but preferably a plurality, of reference markers. Similarly, the data storage unit 42 also holds data for the second raster image and the corresponding at least one reference marker in relation to the second raster image. The image combiner 48 determines a relationship between a reference marker of the first raster image to the same physical reference marker, but expressed in terms of the second raster image. Since the reference marker is the same physical marker, such as a corner of a sprocket hole, a mapping relation can be determined by equating the location of the reference marker in terms of the first raster image to the reference marker expressed in terms of the second raster image. Of course, it is equivalent to express the mapping going from the second raster image to the first raster image rather than vice versa. Additional reference markers can permit one to obtain a more accurate mapping, or one can produce a plurality of local mapping rules at different sections along the photographic film 22.

Once a mapping rule is known for mapping the reference markers expressed in terms of the first raster image to the corresponding reference markers expressed in terms of the second raster image, the image combiner 48 maps pixels of the first raster image into the second raster image. The image combiner 48 is not limited to mapping only one raster image into another raster image. The procedure can be repeated for mapping a plurality of raster images into one raster image.

Typically, each pixel in the first raster image will be mapped to a corresponding image in the second raster image. This corresponds to a translation along the longitudinal direction 28 and/or along the transverse direction 34. However, if there is a difference in the digitized resolution between the first and second raster images, or if there is a difference in magnification between the scanning station 24 and scanning station 26, a many-pixel-to-one or one-to-many-pixel mapping may be desirable. For example, if the distance between fiducial points 64 and 70 differs from that of 64' and 70', then such a one-to-many or many-to-one mapping may be required.

After the image combiner 48 maps the first raster image into the second raster image, it is generally desirable to form a combined image using conventional digital image processing techniques. In the preferred embodiment, the image combiner 48 provides a combined output digital image which can be directed to conventional display devices and/or storage devices. In some cases, however, a combined image is not formed when it is desirable to maintain aligned image data in separate image channels.

The preferred embodiment envisions at least two scanning stations, such as scanning stations 24 and 26 illustrated in FIGURE 2, which are displaced relative to each other. In the preferred embodiment, a photographic film is drawn through the scanning device 12. However, one skilled in the art would recognize from the teachings of this disclosure many possible variations of the preferred embodiment which are also encompassed within the scope and spirit of the invention. For example, one may have a fixed image medium and either a single, or a plurality, of scanning heads may move across the fixed image medium, much like a conventional flat-bed scanner. One may then move the single or multiple head scanner across the image medium in a plurality of passes. This would thus allow one to generate a plurality of raster images in which it may be desirable to accurately combine them into a single raster image.

Claims

WHAT IS CLAIMED IS:

1. A method of processing a plurality of digital images, comprising: forming a first digital image of an image medium, wherein said first digital image is a first raster image having a first plurality of pixels; determining a plurality of first reference locations of a plurality of reference markers of said image medium relative to said first raster image, said plurality of reference markers being substantially fixed with respect to said image medium; forming a second digital image of said image medium, wherein said second digital image is a second raster image having a second plurality of pixels; determining a plurality of second reference locations of said plurality of reference markers of said image medium relative to said second raster image; determining a mapping rule that transforms said first reference locations to said second reference locations for each of said plurality of reference markers; and mapping said first plurality of pixels into said second plurality of pixels based on said mapping rule.

2. A method of processing a plurality of digital images according to claim 1, wherein said image medium is a photographic film having a longitudinal direction and a transverse direction.

3. A method of processing a plurality of digital images according to claim 2, wherein said plurality of reference markers are sprocket holes in said photographic film in which substantially one half of said sprocket holes are spaced along one transverse edge of said photographic film and the remaining sprocket holes are spaced along the opposing transverse edge of said photographic film.

4. A method of processing a plurality of digital images according to claim 2, wherein said plurality of reference markers are notches in said photographic film in which substantially one half of said notches are spaced along one transverse edge of said photographic film and the remaining notches are spaced along the opposing transverse edge of said photographic film.

5. A method of processing a plurality of digital images according to claim 3, wherein said first plurality of pixels forms a first grid that is substantially rectangular, said first grid having columns that extend substantially in said transverse direction and rows that extend substantially in said longitudinal direction of said photographic film, each of said columns being assigned a unique column number that increases in said longitudinal direction and each of said rows being assigned a unique row number that increases in said transverse direction, said determining a plurality of first reference locations determines a transverse and a longitudinal edge of each of said sprocket holes relative to said first raster image by performing a high-pass spatial image filtering of said first raster image and assigning a unique column number, or a fraction thereof, to each said transverse edge and a unique row number, or a fraction thereof, to each said longitudinal edge of said sprocket holes, said second plurality of pixels forms a second grid that is substantially rectangular, said second grid having columns that extend substantially in said transverse direction and rows that extend substantially in said longitudinal direction of said photographic film, each of said columns being assigned a unique column number that increases in said longitudinal direction and each of said rows being assigned a unique row number that increases in said transverse direction, said determining a plurality of second reference locations determines a transverse and a longitudinal edge of each of said sprocket holes relative to said second raster image by performing a high-pass spatial image filtering of said second raster image and assigning a unique column number, or a fraction thereof, to each said transverse edge and a unique row number, or a fraction thereof, to each said longitudinal edge of said sprocket holes.

6. A method of processing a plurality of digital images according to claim 5, wherein said assigning a unique column number, or a fraction thereof, to each said transverse edge of said sprocket holes assigns a weighted average column number obtained by weighting each column number by a number of pixels within that column having any nonzero pixel value corresponding to said transverse edge of said sprocket hole, and said assigning a unique row number, or a fraction thereof, to each said longitudinal edge of said sprocket holes assigns a weighted average row number obtained by weighting each row number by a number of pixels within that row having any nonzero pixel value corresponding to said longitudinal edge of said sprocket hole.

7. A method of processing a plurality of digital images according to claim 5, further comprising determining a first fiducial point for each said sprocket hole relative to said first raster image; and determining a second fiducial point for each said sprocket hole relative to said second raster image, wherein said first fiducial point is a point of intersection between each longitudinal line through said assigned column number and a corresponding one of said vertical lines through said assigned row number of one of said sprocket holes relative to said first raster image, and said second fiducial point is a point of intersection between each longitudinal line through said assigned column number and a corresponding one of said vertical lines through said assigned row number of one of said sprocket holes relative to said second raster image.

8. A method of processing a plurality of digital images according to claim 1, wherein said mapping rule is a one-to-one mapping of said first plurality of pixels into said second plurality of pixels corresponding to a non-rescaled mapping.

9. A method of processing a plurality of digital images according to claim 1, wherein said mapping rule maps a sub-plurality of said first plurality of pixels into a sub-plurality of said second plurality of pixels such that said sub-plurality of said second plurality of pixels is less than said sub-plurality of said first plurality of pixels, thereby corresponding to a reduction in scale of at least a portion of said first raster image after being mapped into said second raster image.

10. A method of processing a plurality of digital images according to claim 1, wherein said mapping rule maps a sub-plurality of said first plurality of pixels into a sub-plurality of said second plurality of pixels such that said sub-plurality of said second plurality of pixels is greater than said sub-plurality of said first plurality of pixels, thereby corresponding to a magnification of at least a portion of said first raster image after being mapped into said second raster image.

11. A method of processing a plurality of digital images according to claim 1, wherein said first digital image of said image medium is an image of light having a first spectral distribution concentrated primarily in a first portion of the visible region of the electromagnetic spectrum, and said second digital image of said image medium is an image of light having a second spectral distribution concentrated primarily in a second portion of the visible region of the electromagnetic spectrum.

12. A method of processing a plurality of digital images according to claim 1, wherein said first digital image of said image medium is an image of light having a first spectral distribution concentrated primarily in a first portion of the infrared region of the electromagnetic spectrum, and said second digital image of said image medium is an image of light having a second spectral distribution concentrated primarily in a second portion of the infrared region of the electromagnetic spectrum.

13. A method of processing a plurality of digital images according to claim 1, wherein said first digital image of said image medium is reflection image of radiation reflected from said image medium, and said second digital image of said image medium is a transmission image of radiation transmitted through said image medium.

14. A method of processing a plurality of digital images according to claim 1, further comprising: forming a third digital image of an image medium, wherein said third digital image is a third raster image having a third plurality of pixels; determining a plurality of third reference locations of said plurality of reference markers of said image medium relative to said third raster image; determining a second mapping rule that transforms said third reference locations to at least one of said first or said second reference location; and mapping said third plurality of pixels into said at least one of said first or said second plurality of pixels based on said second mapping rule.

15. A method of processing a plurality of digital images, comprising: forming a first digital image of an image medium, wherein said first digital image is a first raster image having a first plurality of pixels; performing a high-pass spatial filtering of said first digital image; determining a first reference location of a reference marker of said image medium relative to a first high-pass spatial-filtered image of said first raster image, said reference marker being substantially fixed with respect to said image medium; forming a second digital image of said image medium, wherein said second digital image is a second raster image having a second plurality of pixels; performing a high-pass spatial filtering of said second digital image; determining a second reference location of said reference marker of said image medium relative to a second high-pass spatial-filtered image of said second raster image; determining a mapping rule that transforms said first reference location to said second reference location for said reference marker; and mapping said first plurality of pixels into said second plurality of pixels based on said mapping rule.

16. A method of processing a plurality of digital images according to claim 15, wherein said image medium is a photographic film having a longitudinal direction and a transverse direction.

17. A method of processing a plurality of digital images according to claim 16, wherein said first and second digital images of said image medium are formed at spatially separated locations relative to each other.

18. A method of processing a plurality of digital images according to claim 17, wherein said reference marker is a sprocket hole in said photographic film.

19. A method of processing a plurality of digital images according to claim 18, wherein said first plurality of pixels forms a first grid that is substantially rectangular, said first grid having first columns that extend substantially in said transverse direction and first rows that extend substantially in said longitudinal direction of said photographic film, each of said first columns being assigned a first column number such that the first column numbers increase from one to the next first column number along said longitudinal direction, and each of said first rows being assigned a first row number such that the first row numbers increase from one to the next first row number in said transverse direction, said determining a first reference location determines a transverse and a longitudinal edge of said sprocket hole relative to said first high-pass spatial-filtered image of said first raster image and assigns a first column number, or a fraction thereof, to said transverse edge and a first row number, or a fraction thereof, to said longitudinal edge of said sprocket hole, said second plurality of pixels forms a second grid that is substantially rectangular, said second grid having second columns that extend substantially in said transverse direction and second rows that extend substantially in said longitudinal direction of said photographic film, each of said second columns being assigned a second column number such that the second column numbers increase from one to the next second column number along said longitudinal direction, and each of said second rows being assigned a second row number such that the second row numbers increase from one to the next second row number in said transverse direction, and said determining a second reference location determines a transverse and a longitudinal edge of said sprocket hole relative to said second high-pass spatial-filtered image of said second raster image and assigns a second column number, or a fraction thereof, to said transverse edge and a second row number, or a fraction thereof, to said longitudinal edge of said sprocket hole.

20. A method of processing a plurality of digital images according to claim 19, wherein said assigning first and second column numbers, or fractions thereof, to said transverse edge of said sprocket hole assigns weighted average column numbers obtained by weighting each first and second column number by a number of pixels within that column having any nonzero pixel value corresponding to said transverse edge of said sprocket hole, and said assigning first and second row numbers, or fractions thereof, to said longitudinal edge of said sprocket hole assigns weighted average row numbers obtained by weighting each first and second row numbers by a number of pixels within that row having any nonzero pixel value corresponding to said longitudinal edge of said sprocket hole.

21. A method of processing a plurality of digital images according to claim 20, further comprising determining a first fiducial point for said sprocket hole relative to said first raster image; and determining a second fiducial point for said sprocket hole relative to said second raster image, wherein said first fiducial point is a point of intersection between a first longitudinal line through said assigned first column number and a first vertical line through said assigned first row number of said sprocket hole relative to said first raster image, and said second fiducial point is a point of intersection between a second longitudinal line through said assigned second column number and a second vertical lines through said assigned second row number of said sprocket hole relative to said second raster image.

22. A method of processing a plurality of digital images according to claim 15, wherein said mapping rule is a one-to-one mapping of said first plurality of pixels into said second plurality of pixels corresponding to a non-rescaled mapping.

23. A method of processing a plurality of digital images according to claim 15, wherein said mapping rule maps a sub-plurality of said first plurality of pixels into a sub-plurality of said second plurality of pixels such that said sub-plurality of said second plurality of pixels is less than said sub-plurality of said first plurality of pixels, thereby corresponding to a reduction in scale of at least a portion of said first raster image after being mapped into said second raster image.

24. A method of processing a plurality of digital images according to claim 15, wherein said mapping rule maps a sub-plurality of said first plurality of pixels into a sub-plurality of said second plurality of pixels such that said sub-plurality of said second plurality of pixels is greater than said sub-plurality of said first plurality of pixels, thereby corresponding to a magnification of at least a portion of said first raster image after being mapped into said second raster image.

25. A method of processing a plurality of digital images according to claim 15, wherein said first digital image of said image medium is an image of light having a first spectral distribution concentrated primarily in a first portion of the visible region of the electromagnetic spectrum, and said second digital image of said image medium is an image of light having a second spectral distribution concentrated primarily in a second portion of the visible region of the electromagnetic spectrum.

26. A method of processing a plurality of digital images according to claim 15, wherein said first digital image of said image medium is reflection image of radiation reflected from said image medium, and said second digital image of said image medium is a transmission image of radiation transmitted through said image medium.

27. A method of processing a plurality of digital images according to claim 15, further comprising: forming a third digital image of an image medium, wherein said third digital image is a third raster image having a third plurality of pixels; determining a plurality of third reference locations of a plurality of reference markers of said image medium relative to said third raster image, said plurality of reference markers being substantially fixed with respect to said image medium; determining a second mapping rule that transforms said third reference locations to at least one of said first or said second reference location; and mapping said third plurality of pixels into said at least one of said first or said second plurality of pixels based on said second mapping rule.

28. A method of processing a plurality of digital images according to claim 15, further comprising: determining a first reference location of a second reference marker of said image medium relative to a first high-pass spatial-filtered image of said first raster image, said second reference marker being substantially fixed with respect to said image medium; determining a second reference location of said second reference marker of said image medium relative to a second high-pass spatial-filtered image of said second raster image; determining a mapping rule that transforms said first reference locations to said second reference locations of said first-mentioned and said second reference markers; and mapping said first plurality of pixels into said second plurality of pixels based on said mapping rule.

29. A digital image processing device, comprising: a digital image scanner having at least two digital image channels; a digital image processor in communication with said digital image scanner; a data storage device in communication with said digital image processor; a high-pass spatial filter in communication with said data storage device; a reference mark detector in communication with said high-pass spatial filter and said data storage device; and an image combiner in communication with said reference mark detector and said data storage device, wherein said at least two image channels are at separate locations, or at different times, said at least two image channels produce at least two separate images of an image medium, and said image combiner substantially aligns said two separate images to correct at least one of a translational, rotational, or magnification error.

30. A digital image processing device according to claim 29, wherein said digital image scanner is a photographic film scanner and said image medium is a photographic film.

31. A digital image processing device according to claim 29, wherein said image combiner forms a combined output image from said substantially aligned at least two images.

32. A digital image processing device according to claim 29, wherein at least one of said at least two digital image channels is in a visible region of the spectrum.

33. A digital image processing device according to claim 29, wherein at least one of said at least two digital image channels is in an infrared region of the spectrum.

34. A digital image processing device according to claim 29, wherein at least one of said at least two digital image channels is a reflection channel in which light reflected from an image medium is detected.

35. A digital image processing device according to claim 29, wherein at least one of said at least two digital image channels is a transmission channel in which light transmitted through an image medium is detected.

36. A digital image processing device according to claim 29, wherein said digital image scanner has at least two scanning stations, and each of said at least two scanning stations has a plurality of image channels.