JP2000148991A - Image processing method and image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an image processing method and an image pickup device which can accurately detect an artificial product fast even in a state wherein there are many disturbing images like a water surface. SOLUTION: The properties of the same subject are divided into areas for each specific range according to mutual differences among the values of image signals of mutually different wavelength bands outputted by an image pickup means for images of the photographed subject and it is decided which area each of the pixels constituting one frame belongs to. When the value of the 1st difference obtained by subtracting the value of the image signal corresponding to a 2nd wavelength band of about 3 to 4 (μm) from the value of the image signal corresponding to a 1st wavelength band of about 4 to 5 (μm) is plus and the value of the 2nd difference obtained by subtracting the value of the image signal corresponding to a 3rd wavelength band of about 1 to 1.5 (μm) from the value of the image signal corresponding to the 2nd wavelength band of about 3 to 4 (μm) is minus, it is decided that the pixel belongs to the area showing the properties of a water surface reflected image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing method and an image pickup apparatus suitable for video monitoring using an image in the infrared region on the water surface.

[0002]

2. Description of the Related Art In recent years, examples of remote monitoring using a video camera and automatic monitoring by extracting a predetermined object from a picture taken by the video camera by a computer have been increasing. Traffic monitoring is one example, but it is difficult to monitor a wide area outdoors for 24 hours with visible light.

[0003]

Therefore, an infrared image is used for monitoring an object in an environment where sunlight and illumination light are scarce. However, there are many disturbance images such as waves on water (especially on the sea), and it is difficult to recognize a specific object.

On the other hand, for example, by averaging a plurality of temporally continuous frames or averaging screens shot by a plurality of video cameras having slightly different viewpoints, irregularities (such as redundancy and Although there is a method of reducing a disturbance signal (with low correlation), there are many problems in terms of speed and accuracy.

[0005] That is, in the conventional imaging method using one infrared camera or sea monitoring of a harbor or the like using an infrared spectroscope, it was not possible to accurately and quickly detect sea surface waves and artifacts (such as ships).

The present invention has been made under such a background, and an image processing method and an image pickup apparatus capable of accurately and quickly detecting an artifact or the like even in a situation where there are many disturbed images such as a water surface. It is intended to provide.

[0007]

In order to solve the above-mentioned problems, according to the first aspect of the present invention, a plurality of wavelength bands different from each other, which are output by an image pickup means for the same photographed subject image. The characteristics of the subject are divided into a plurality of regions for each predetermined range based on the difference between the values of the image signals, and it is determined to which of the plurality of regions each of the plurality of pixels constituting one frame belongs. It is characterized by the following. According to the invention described in claim 2, in the image processing method described in claim 1, the value of the image signal corresponding to the first wavelength band and the image signal corresponding to the second wavelength band And a second difference calculated from the value of the image signal corresponding to the second wavelength band and the value of the image signal corresponding to the third wavelength band. The characteristics of the subject are divided into a plurality of regions for each predetermined range based on the combination of the above, and each of a plurality of pixels constituting the one frame belongs to a region showing the characteristics of the water surface reflection image, or It is characterized in that it is determined whether it belongs to an area exhibiting properties other than the reflection image. According to the third aspect of the present invention, in the image processing method according to the second aspect, the value of the image signal corresponding to the first wavelength band, which is approximately 4 to 5 μm, is approximately determined. The second difference, in which the value of the first difference obtained by subtracting the value of the image signal corresponding to the second wavelength band of 3 to 4 μm is positive, and is approximately 3 to 4 μm. The value of the second difference obtained by subtracting the value of the image signal corresponding to the third wavelength band, which is approximately 1 to 1.5 [μm], from the value of the image signal corresponding to the wavelength band is negative. In some cases, it is determined that the pixel belongs to an area indicating the properties of the water surface reflection image. According to a fourth aspect of the present invention, in the image processing method according to any one of the second and third aspects, the subject image is condensed by a single condensing means, and each of the image is vertically and horizontally predetermined. The method is characterized in that the one frame is formed for each number, and for all of the plurality of pixels constituting the one frame, it is sequentially determined whether or not the pixel exhibits a property other than the water surface reflection image. According to a fifth aspect of the present invention, in the image processing method according to any one of the first to fourth aspects, the one frame represented by the values of the plurality of image signals is configured. The information possessed by each of the plurality of pixels is changed corresponding to each of the plurality of regions. According to the invention described in claim 6, an image pickup means for outputting a plurality of image signals of different wavelength bands to the same photographed subject image and a plurality of pixels constituting one frame Image processing means for recognizing a property of a subject indicated by the pixel based on a mutual difference between values of a plurality of image signals output by the imaging means. Also, in the invention according to claim 7, in the imaging device according to claim 6, the image processing unit is configured to perform the image processing based on a mutual difference between a plurality of image signals output by the imaging unit. In addition to dividing the property of the subject into a plurality of areas for each predetermined range, discrimination data serving as a reference for determining to which of the plurality of areas the difference between the image signal values belongs is set in advance in the area setting. Means for determining the area to which each of the plurality of pixels constituting the one frame belongs based on the discrimination data. Also, in the invention according to claim 8, in the imaging device according to claim 7, the image processing unit includes a value of the image signal corresponding to the first wavelength band and a value of the image signal corresponding to the second wavelength band. A first difference calculated from a value of the corresponding image signal, and a first difference calculated from a value of the image signal corresponding to the second wavelength band and a value of the image signal corresponding to a third wavelength band. And dividing the property of the subject into a plurality of regions for each predetermined range based on a combination with the second difference, and determining each of the plurality of pixels constituting the one frame by using a water surface reflection image. It is characterized in that it is determined whether it belongs to a region showing a property or belongs to a region showing a property other than the water surface reflection image. In the imaging device according to the ninth aspect, in the imaging device according to the eighth aspect, the determination unit may determine the image signal corresponding to the first wavelength band of approximately 4 to 5 μm. Is positive, and the value of the first difference obtained by subtracting the value of the image signal corresponding to the second wavelength band, which is approximately 3 to 4 μm, is positive, and approximately 3 to 4 μm.
The value obtained by subtracting the value of the image signal corresponding to the third wavelength band of approximately 1 to 1.5 [μm] from the value of the image signal corresponding to the second wavelength band of [μm]. When the value of the difference between the two is negative, it is determined that the pixel belongs to an area exhibiting the properties of the water surface reflection image. Also, in the invention according to claim 10, in the imaging device according to any one of claims 6 to 9, the imaging unit may be configured to apply different wavelength bands to the same subject image. A plurality of cameras that output corresponding image signals,
It is characterized by comprising a sole condensing means for condensing the subject image and a spectral means for dispersing the condensed subject image to each of the plurality of cameras. In the imaging device according to the eleventh aspect, in the imaging device according to the tenth aspect, the spectroscopic unit transmits the collected subject image to each of the plurality of cameras for each of different wavelength bands. It is characterized by spectral separation.

In the present invention, in the range of 1 to 1.5 [μm], 3 to 4
Simultaneous acquisition of images in the same field of view with three infrared cameras having sensitivity in the [μm] band and 4 to 5 [μm] band to acquire three types of images having different wavelength characteristics, Ships can be separated and extracted. In the extraction processing, attention is paid to the signal intensity characteristics with respect to the wavelength of the wave and the wavelength of the ship, and the signal intensity ratio between the respective wavelength bands is calculated for each pixel, patterned, and discriminated.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of an imaging device according to an embodiment of the present invention. A focusing optical system (lens group) 1 included in the imaging unit 4 illustrated in FIG. 1 collects light in an infrared band capable of forming an image according to a distance to a target, an angle of view, or resolution.

The imaging light that has passed through the condensing optical system 1 is divided into a plurality of lights having different wavelength bands by a spectral optical system (dichroic mirror group) 2, and each light is divided into IR (Infrared R).
ays: Infrared) cameras _{3-1, 3-2} are supplied to the _3-2. The IR camera 3 _-1 composed of a detector sensitive to filter the same band of wavelengths 1 to 1.5 [[mu] m] band. The IR camera _3-2 is composed of a filter and a detector in the 3-4 [μm] band, and the IR camera _3-3 is composed of a filter in the wavelength 4-5 [μm] band and a detector having sensitivity in the same band. You.

The video signal output from each IR camera 3 _-1 , 3 _-2 , 3 _-3 is converted into an A / D (A
nalog / Digital: analog-digital) converter 7 _-1 ,
The data is converted into digital data by 7 _-2 and 7 _-3 and stored in the corresponding frame memories 10 _-1 , 10 _-2 and 10 _-3 .

An area setting circuit 13 in the signal processing device 6
Sets a discrimination area according to the imaging environment and targets, etc., to create a discriminating region data D _14. The determination circuit 15 performs a comparison process based on the data of the frame memories 10 _-1 , 10 _-2 , and 10 _-3 and the discrimination area data D ₁₄ (described in detail later),
Image data of a ship or the like is extracted.

The result extracted in this manner is output I / F
(Interface) The signals are output from the signal processing device 6 via the interfaces 16 and 17, displayed on the display device 18, and recorded on the recording device 19.

FIGS. 2 and 3 are explanatory diagrams for explaining the principle of target extraction. FIG. 2 is a diagram showing reflection characteristics of the sea surface in various wave states, and FIG. FIG. 6 is a diagram modeling a wavelength-dependent state of signal intensity for each state.

In FIG. 2, A represents the sea surface where white waves are standing, B represents the rough sea surface, and C represents the signal intensity for each wavelength when the calm sea surface is imaged. As shown in FIG. 2, in the band around the wavelength [1 μm], there is a characteristic that the signal intensity becomes higher as the white wave becomes greater than in other wavelength bands.

Therefore, in the present embodiment, the wavelength of the imaging light is approximately 1 to 1.5 [μ] as information for determining the state of the wave.
m], about 3 to 4 μm, and about 4 to 5 μm. FIG. 3 illustrates the above-described signals and the wavelength dependence of the image signal (D) of the artificial object such as a ship.

According to the model shown in FIG. 3, the signal intensity ratio between the three wavelengths in the image signal of the sea surface in the state where the white wave stands is from wavelength 1 to 1.5 [μm] to wavelength 3 to 4 [μm].
m] slope S ₃₁ to the downward sloping (this is defined as "negative"), the slope S ₅₃ from the wavelength 3 to 4 [[mu] m] to a wavelength of 4 to 5 [[mu] m] right up (this "positive" Is defined).

FIG. 4 shows the characteristics of the reflection characteristics in FIG.
This is a pattern expressed in a quadrant graph.
To 1.5 [[mu] m] The horizontal axis of inclination S ₃₁ of the signal strength up 3 to 4 [[mu] m] band from band (the right side is positive), and 3 to 4 [[mu] m]
The slope S ₅₃ of the signal strength from the strip to the 4 to 5 [[mu] m] range are those taken in the vertical axis (upward positive).

In FIG. 4, the case where all the inclinations are small is the area 21, and the case where the inclination S ₅₃ is positive and the inclination S ₃₁ is negative is the area 2.
2, the slope S ₅₃ is inclined in the positive S ₃₁ even in the case of positive region 23, region 24 where S ₃₁ tilt slope S ₅₃ is negative even negative, and when the inclination S ₅₃ is S ₃₁ tilt in the negative is positive Is an area 25. Discriminating region data D ₁₄ of the above, the regions 21-2
5 is threshold value data for each of No. 5.

[0020] According to the pattern shown in FIG. 4, event A related sea image signal, B and C are negative S ₃₁ tilt slope S ₅₃ is positive, it can be seen that belongs to the area 22. Also from the model of FIG. 3, the event D is an image signal of the artifact because even S ₃₁ tilt slope S ₅₃ is negative is negative, it can be seen that belongs to the area 24.

That is, the judgment circuit 15 is provided in the frame memory 10
Each event included in the image signal is patterned into a graph of four quadrants as shown in FIG. 4 based on the data of ₋₁ , 10 ₋₂ , and 10 ₋₃ and the discrimination region data D _14, and distributed in the region 22. The event is determined to be an image due to sea surface reflection, and the other events distributed in the region 21 and the regions 23 to 25 may be determined to be reflections from the artifact.

FIG. 5 is a flowchart showing an example of a procedure for processing an image signal according to the present embodiment. An artificial object such as a ship is obtained from a sea photographed image of a port or the like composed of horizontal x dots and vertical y dots. The image processing to recognize is shown.

First, the signal processing device 6 captures image signals of each wavelength band from each IR camera 3 _-1 , 3 _-2 , 3 _-3 ,
The data is converted into digital data by A / Ds 7 _-1 , 7 _-2 , and 7 _-3 (Step St1). The digital data is temporarily stored in the frame memories 10 _-1 , 10 _-2 ,
10 _-3 .

The decision circuit 15 has a frame memory 10 _-1 , 1
Image data is read from 0 _-2 and 10 _-3 , and the signal intensity value is normalized by reference data created in advance according to the sensitivity characteristics of each IR camera 3 _-1 , 3 _-2 and 3 _-3 and the amount of incident light. (Step St2). In this embodiment, since the processing is sequentially performed for each pixel constituting one screen, the pointer i of the x coordinate and the pointer j of the y coordinate are initialized to 0 (Step St3).

Next, the determination circuit 15
_-1 , 10 _-2 and 10 _-3 correspond to the same point on the screen.
4 [[mu] m] range of data and 4 to 5 [[mu] m] reads the band data comparison, calculates an inclination S ₅₃ of this point (step St4). Then, the slope S ₅₃ of this point it is checked whether shows the inclination characteristic in sea surface image (step S
t5).

As described with reference to FIG. 4, the events of the sea surface image are distributed in the region where the slope _S53 is a positive value. Therefore, the slope S
_{If 53} is a positive value, it is determined that it indicates a tilt peculiar to the sea surface image.

[0027] In step St5, when the inclination S ₅₃ of this point is determined to be a sea surface image specific slope reads the data of 1 to 1.5 [[mu] m] band of the same points 3 to 4 [mu
m] compared to the band of the data to calculate the slope S ₃₁ (Step St6).

[0028] Then, the inclination S ₃₁ checks whether shows the inclination characteristic in sea surface image (step St 7). In this case, are as described also in FIG. 4, the slope S ₃₁ events in sea image are distributed in the negative region. Therefore, when the inclination S ₃₁ is a negative value, it is determined that indicates the inclination of the specific to the sea surface image.

[0029] In step St 7, and when the inclination S ₃₁ is determined to be a sea surface image specific slope, the pixel in this respect exists in a region 22 shown in FIG. 4 is an image obtained by photographing the sea decision (Step St8). On the other hand, when S ₅₃ tilt in the above step St5 is it is determined that if it is determined not to be the sea surface image specific slope, and the slope S ₃₁ is not the sea image characteristic slope in step St7, the pixels of the point 4 Therefore, it is determined that the image is obtained by photographing an area other than the sea surface (that is, an artificial object) (Step St9).

In the processing of step St8 or St9, after determining whether or not the event at this point is a sea surface, the luminance (brightness) of the pixel at this point is determined according to, for example, whether or not the event is a sea surface image. And changing the color tone.

For example, if the image is a sea surface image, the brightness of this pixel is reduced, and if the image is not a sea surface image, the brightness of the image is increased or the color tone is changed. By doing so, an image of an artifact or the like is emphasized as compared with an image of the sea surface having many disturbance images. Alternatively, a configuration may be provided for sequentially recognizing the position of the artificial object extracted in this way to obtain a course, or for detecting movement or approach of the artificial object.

After such processing, one screen (frame)
It is confirmed whether or not the processing has been completed for all pixels (Step St10), and if not completed, predetermined numerical values are added to the pointers i and j (Step St11).
Then, the process returns to step St4, and proceeds to processing of the next pixel.

When the processing of all the pixels on one screen is completed, the image data for one screen is displayed on the display device 18 or recorded on the recording device 19 as necessary (Step St12).

In the above-described embodiment, an image was taken with an infrared camera having a maximum sensitivity in a band of approximately 1 to 1.5 [μm], approximately 3 to 4 [μm], and approximately 4 to 5 [μm]. Although an example of processing an image has been described above, an infrared camera having a sensitivity in a wavelength band of 8 to 12 [μm], a visible light camera, or the like may be further provided. As a result, it is possible to more accurately or quickly extract a predetermined object, extract an object having other characteristics, and eliminate a disturbance image.

[0035]

As described above, according to the present invention, an object can be obtained based on the difference between the values of a plurality of image signals of different wavelength bands output by the imaging means for the same image of the object. Is divided into a plurality of regions for each predetermined range, and it is determined to which of the plurality of regions each of the plurality of pixels constituting one frame belongs. Further, a first difference calculated from a value of the image signal corresponding to the first wavelength band and a value of the image signal corresponding to the second wavelength band, and a value of the image signal corresponding to the second wavelength band And dividing the property of the subject into a plurality of regions for each predetermined range based on a combination of the second difference calculated from the value of the image signal corresponding to the third wavelength band and a plurality of regions constituting one frame. It is determined whether each of the pixels belongs to an area exhibiting properties of the water surface reflection image or belongs to an area exhibiting properties other than the water surface reflection image. A first value obtained by subtracting a value of an image signal corresponding to a second wavelength band of approximately 3 to 4 [μm] from a value of an image signal corresponding to a first wavelength band of approximately 4 to 5 [μm].
Is approximately 1 to 1.5 from the value of the image signal corresponding to the second wavelength band in which the value of is positive and is approximately 3 to 4 [μm].
When the value of the second difference obtained by subtracting the value of the image signal corresponding to the third wavelength band of [μm] is negative, it is determined that the pixel belongs to the region exhibiting the properties of the water surface reflection image. In addition, the subject image is condensed by a single condensing means to form a predetermined number of frames each in the vertical and horizontal directions, and the pixels are sequentially arranged for all of the plurality of pixels constituting one frame except for the water surface reflection image. Is determined. In addition, since information possessed by each of a plurality of pixels constituting one frame represented by the values of a plurality of image signals is changed corresponding to each of a plurality of regions, in a situation where there are many disturbance images such as a water surface. In this case, an effect is obtained that an image processing method and an imaging device capable of accurately and quickly detecting an artificial object or the like can be realized.

That is, in the present invention, using image data from an infrared camera having sensitivity to a plurality of (three) wavelength bands,
Since the reflection characteristics of the sea surface and the reflection characteristics of the ship are characterized by a patterned discrimination area and the separation determination process is performed,
Sea surface waves can be separated from artificial objects such as ships.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an imaging device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing examples of sea surface reflection characteristics in various wave states.

FIG. 3 is an explanatory diagram modeling a wavelength-dependent state of signal intensity for each state of each wave shown in FIG. 2;

FIG. 4 is an explanatory diagram in which the characteristics of the reflection characteristics in FIG. 3 are represented by patterning into a graph of four quadrants.

FIG. 5 is a flowchart illustrating an example of a processing procedure of an image signal according to the embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Condensing optical system (condensing means) 2 Spectral optical system (spectroscopic means) _3-1 , _3-2 , _3-3 IR camera (camera) 4 Imaging part (imaging means) 6 Signal processing part (image processing means) 7 _-1 , 7 _-2 , 7 _-3 A / D 10 _-1 , 10 _-2 , 10 _-3 Frame memory 13 Area setting circuit (area setting means) 15 Judgment circuit (judging means) 16, 17 Output I / F 18 display device 19 recording device 21-25 region D ₁₄ discriminating region data (discrimination data) S ₃₁ slope (second difference) S ₅₃ slope (first difference)

Claims (11)

[Claims] 1. A method for determining a property of a subject for each predetermined range based on a mutual difference between values of a plurality of image signals in different wavelength bands output from an imaging unit (4) for the same photographed subject image. An image processing method comprising: determining which of a plurality of pixels constituting one frame belongs to which of the plurality of regions (21, 22, 23, 24, 25). 2. A first difference (S ₅₃ ) calculated from a value of the image signal corresponding to a first wavelength band and a value of the image signal corresponding to a second wavelength band, and the second difference (S ₅₃ ). The property of the subject is determined based on a combination of the value of the image signal corresponding to the wavelength band of the second and the second difference (S ₃₁ ) calculated from the value of the image signal corresponding to the third wavelength band. And whether each of the plurality of pixels constituting the one frame belongs to a region exhibiting properties of a water-surface reflection image or belongs to a region exhibiting properties other than a water-surface reflection image The image processing method according to claim 1, wherein any one of them is determined. 3. A value of the image signal corresponding to the first wavelength band, which is approximately 4 to 5 μm, is approximately 3 to 4 μm.
m], wherein the value of the first difference obtained by subtracting the value of the image signal corresponding to the second wavelength band is positive, and is approximately 3 to 4 μm. The third is approximately 1 to 1.5 [μm] from the value of the corresponding image signal;
When the value of the second difference obtained by subtracting the value of the image signal corresponding to the wavelength band is negative, it is determined that the pixel belongs to an area exhibiting the properties of the water surface reflection image. Item 3. The image processing method according to Item 2. 4. A method for converging said subject image by a single condensing means (1) to form a predetermined number of said one frame in each of the vertical and horizontal directions, and for all of a plurality of pixels constituting said one frame 4. The image processing method according to claim 2, wherein it is sequentially determined whether or not the pixel has a property other than the water surface reflection image. 5. The method according to claim 1, wherein information included in each of the plurality of pixels constituting the one frame represented by the values of the plurality of image signals is changed corresponding to each of the plurality of regions. The image processing method according to any one of claims 1 to 4. 6. An image pickup means for outputting a plurality of image signals of different wavelength bands to the same photographed subject image, and a plurality of images outputted by the image pickup means for each of a plurality of pixels constituting one frame. An image pickup apparatus comprising: an image processing unit (6) for recognizing a property of a subject indicated by a pixel based on a difference between signal values. 7. The image processing unit divides a property of the subject into a plurality of regions in a predetermined range based on a mutual difference between values of a plurality of image signals output by the imaging unit. and the value of the difference between each other as a reference determines belong to any of the plurality of regions discriminated data (D ₁₄₎ is preset region setting means (13), wherein one frame on the basis of the discrimination data The imaging device according to claim 6, further comprising: a determination unit (15) configured to determine the area to which each of the plurality of pixels configuring the region belongs. 8. The image processing means includes: a first difference calculated from a value of the image signal corresponding to a first wavelength band and a value of the image signal corresponding to a second wavelength band; The property of the subject is set in a predetermined range based on a combination of a value of the image signal corresponding to the second wavelength band and a second difference calculated from a value of the image signal corresponding to the third wavelength band. Each of the plurality of pixels constituting the one frame belongs to a region indicating a property of a water surface reflection image or a region indicating a property other than a water surface reflection image. The imaging device according to claim 7, wherein any one of the members belongs to the group. 9. The method according to claim 1, wherein the determining unit determines that the value of the image signal corresponding to the first wavelength band of about 4 to 5 [μm] corresponds to the second wavelength band of about 3 to 4 [μm]. The value of the first difference obtained by subtracting the value of the corresponding image signal is positive, and the value of the image signal corresponding to the second wavelength band, which is approximately 3 to 4 μm, is approximately 1 to 1. A region where the pixel exhibits the property of the water surface reflection image when the value of the second difference obtained by subtracting the value of the image signal corresponding to the third wavelength band of 1.5 [μm] is negative. The imaging device according to claim 8, wherein the image pickup device is determined to belong to 10. The plurality of cameras (3 _-1 , 3 _-2 , 3 _-2 , _3-3) outputting image signals corresponding to mutually different wavelength bands for the same subject image.
_3-3 ), a sole condensing means for condensing the subject image, and a spectroscopic means (2) for dispersing the condensed subject image to each of the plurality of cameras. The imaging device according to any one of claims 6 to 9, wherein 11. The imaging apparatus according to claim 10, wherein the spectroscopy unit disperses the collected subject image to each of the plurality of cameras for each of different wavelength bands.