JP2020123867A - Image projection system and projection area detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect an image projection system and a projection area capable of more accurately detecting a projection area based on a set value of a projection device without using a special projection pattern such as a test chart when a disturbance is present. To provide a method. An image display means including an image display element that displays image information according to an image signal input from an input unit, and a projection means that projects the image information displayed by the image display means onto a projected surface. It is characterized by having an imaging means for capturing a projected image projected by the projection means, and a control means for detecting a projection region using a part of the image information obtained by the imaging means. [Selection diagram] Fig. 1

Description

The present invention relates to an image projection system used in a liquid crystal projector or the like and a projection area detection method, and more particularly to an image projection system capable of detecting a projection area more accurately based on a setting value of a projection device.

When an image is projected using an image projection device such as a liquid crystal projector, various automatic corrections such as autofocus, keystone correction, and deterioration over time are performed. In the automatic correction, in many cases, a projected image is photographed by a camera (imaging device), and the automatic correction is performed based on information such as a pixel value obtained from the captured image. Conventionally, a method of detecting a region of a projection image from an obtained captured image has been proposed.

Patent Document 1 discloses an image projection system in which a plurality of test charts are arranged on a matrix on a projection screen and the projection area of each projection screen is accurately determined by photographing the test chart with an imaging device.

Further, in Patent Document 2, the projection area projected on the whiteboard is determined by contour extraction. After that, the moment feature amounts of the two contours are calculated from the obtained plurality of contour information, and the presence or absence of a shield is determined. An image projection system has been proposed that warns the user of the presence of a shielding object.

JP2012-142669A JP, 2005-171124, A

In the image projection system of Patent Document 1, a test chart must be projected in order to detect the projection area, and when there is a pattern on the screen or wall that is the projection surface, or when the projection light is reflected and the projection area is present. If there is a disturbance such as the case where light is radiated to the outside, the detection accuracy of the disturbance greatly decreases. Further, in the case of a projection device equipped with a camera, it is generally necessary to set an angle of view that covers the projection area after the lens shift or zoom of the projection device, which may make it difficult to determine the test chart. There is.

In the image projection system of Patent Document 2, it is necessary to shoot a plurality of times to determine the projection area, and it is impossible to extract the projection area when the shield is always present at the same position. Further, as in the case of Patent Document 1, if there is a disturbance, the detection accuracy of the disturbance greatly decreases.

The present invention provides an image projection system and a projection area that can detect a projection area more accurately based on a set value of a projection device without using a special projection pattern such as a test chart when a disturbance exists. It is intended to provide a detection method.

In order to achieve the above object, the image projection system according to the present invention,
An image display unit including an image display element that displays image information according to an image signal input from the input unit,
Projection means for projecting image information displayed by the image display means onto a projection surface,
Image pickup means for picking up a projection image projected by the projection means;
Control means for detecting a projection area by using a part of the image information obtained by the image pickup means;
It is characterized by having.

Advantageous Effects of Invention According to the present invention, an image capable of more accurately detecting a projection area based on a set value of a projection device without using a special projection pattern such as a test chart when a disturbance exists is provided. The projection system and the detection method of the projection area can be provided.

Arrangement diagram of the image projection system of the embodiment of the present invention Block diagram showing a schematic configuration of an image projection system of an embodiment of the present invention Flowchart showing the operation of detecting the projection area according to the embodiment of the present invention Flowchart showing operation of projection area estimation Flowchart showing operation of projection area detection Illustration of calibration data Illustration of guess area Explanatory diagram showing the effect of the guess region

Hereinafter, modes for carrying out the present invention will be described in detail with reference to the drawings.

The present invention is applicable to image projection devices such as liquid crystal projectors and DLP (Digital Light Processing) projectors.

The image projection system of the present invention has an image display unit including an image display element that displays image information according to an image signal input from the input unit. It has a projection means (projection lens) for projecting the image information displayed by the image display means onto the projection surface, and an imaging means for picking up the projection image projected by the projection means. Then, based on the image information obtained by the image pickup means, a position setting value (shift setting value) of the projection image projected on the projection surface, a setting value of the size of the projection image (zoom setting value), and projection It has a control unit that estimates a projection area by using at least one of the lens models (lens model information) and controls so as to detect the projection area from the obtained estimated projection area (hereinafter, estimated area). Further, the projection area detecting method of the present invention has the following steps.

An image projection step of projecting the image information displayed on the image display element on the projection surface by the projection means in accordance with the image signal input by the user from the input section, and the projection image projected on the projection surface by the imaging means. The projection image capturing step of capturing an image is included. Then, using at least one of the set value of the position of the projected image (shift set value), the set value of the size of the projected image (zoom set value), and the model of the lens (lens model information), the imaging unit The projection area estimation step of estimating the position at which the projection image is captured is obtained from the image information obtained in (1). Furthermore, it has a projection area detection step of detecting a projection area from the estimation area obtained in the projection area estimation step.

Embodiments of an image projection system and a projection area detecting method according to the present invention will be described below with reference to the drawings.

First Embodiment FIG. 1 is a layout explanatory diagram of an image projection system according to a first embodiment of the present invention.

The image projection system 100 shown in FIG. 1 has a projection means 1 for projecting an image on a projection surface 2 such as a screen, and an imaging means 3 for photographing a projection image projected on the projection surface 2. Further, it has a control unit 4 (microcomputer) for controlling so as to detect the projection area from the captured image.

The embodiment shown in FIG. 1 shows a state in which the disturbance 21 exists on the projection surface 2.

Here, the disturbance refers to a pattern or some object on the screen, and a state where there is a significant change in luminance or chromaticity outside the projection area due to reflection of projection light or incidence of outside light. Represent

FIG. 2 is a block diagram showing a schematic configuration of the image projection system 100 according to this embodiment.

The configuration of the image projection system 100 will be described.

The projection unit 1 has an image display system 110 including a liquid crystal panel (image display element) 102 and a projection optical system 111 including a projection lens 103. In addition, the projection means 1 has a light source 101. The image display system 110 includes a liquid crystal panel 102, an image signal input unit 104, an image processing unit 105, and a control unit 106. Further, it has an operation unit 109 as an operation means for operating various operations.

The image display element forming the liquid crystal panel 102 modulates the light emitted from the light source 101 according to the input image signal. The light modulated by the image display element including the liquid crystal panel 102 enters the projection lens 103 as a projection image. The projection image is projected onto the projection surface 2 such as a screen by the projection lens 103.

The image signal input unit 104 has terminals for inputting various image signals such as VGA signals, DVI signals, and HDMI (registered trademark) signals, and a receiver IC for receiving the image signals input through these terminals. Etc. are provided.

The image processing unit 105 generates an image signal obtained by performing image processing such as color unevenness correction, gamma correction, contrast correction, color conversion, and aspect conversion on the image signal input from the image signal input unit 104. Further, according to an instruction from the control unit 106, an OSD (On Screen Display) image is superimposed on the generated image signal. The OSD image includes not only image data such as a bitmap image prepared in advance, but also various images generated based on a drawing instruction in units of straight lines, rectangles, characters, or pixels.

The projection optical system 111 has a projection lens 103, a lens drive unit 107, and a lens exchange unit 108. The lens driving unit 107 has a shift unit 107a that changes the position of the projection image projected on the projection surface 2 and a zoom unit 107b that changes the size of the projection image.

The control unit 106 is connected to the image processing unit 105, the lens driving unit 107, and the operation unit 109. The control unit 106 is composed of a microcomputer that controls the power supply and state of each unit of the image projection system 100. The control unit 106 issues a command for shift adjustment or zoom adjustment to the lens driving unit 107 in response to a control signal from the outside via the operation unit 109, or the image processing unit 105 in response to an operation from the operation unit 109, for example. The command to superimpose the menu screen is issued to.

The lens driving unit 107 drives the positions of some of the lenses that form the projection lens 103 to adjust the size of the projection image and the shift unit 107a that moves the projection image on the screen in the vertical and horizontal directions. The zoom unit 107b is controlled. In addition, the lens driving unit has a storage area for holding the set values, and holds the shift set values of the shift unit 107a in the vertical and horizontal directions, the zoom set values of the zoom unit 107b, and the lens model information of the lens replacement unit 108. Therefore, the control unit 106 can refer to this value.

The lens exchange unit 108 is a mechanism when the user attaches and detaches the lens, and can be replaced with another lens such as a wide-angle lens or a telephoto lens. The lens replacement unit 108 sets the currently used lens model information in the storage area of the lens driving unit 107.

The operation unit 109 includes a button for a user to input an operation, an infrared light receiving unit for receiving infrared rays from a remote controller, and the like, and notifies the control unit 106 of the input operation. The types of operations to be input include, for example, turning the power on/off, deciding and canceling, calling up a menu for various settings and information confirmation, and pointing up/down/left/right.

The image pickup means 3 shown in FIG. 1 has an image pickup optical system and a sensor (image pickup element) such as a CCD (Charge Coupled Device) or a CMOS (Complementary MOS). The image pickup means 3 picks up a projection image on the projection surface 2 projected by the projection means 1.

The control unit 4 uses the shift setting value of the shift unit 107a, the zoom setting value of the zoom unit 107b, and the lens model information of the lens exchanging unit 108, which are obtained by the projection unit 1, to cause the image capturing unit 3 to perform the operation. Control is performed so that the projection area projected by the projection unit 1 on the projection surface 2 is detected from the obtained image information.

In the present embodiment, the projection means 1 and the image pickup means 3 are connected to the control means 4, but one image projection apparatus in which the projection means 1, the image pickup means 3, the control means 4, etc. are built-in. You may comprise. At that time, the imaging unit 3 is connected to the control unit 106, and the function of the control unit 4 is built in the control unit 106.

3 to 5 are flowcharts for explaining the operation of detecting the projection area, which is controlled by the control means 4 of the present embodiment.

The flowcharts shown in FIG. 3 to FIG. 5 show the setting values of the shift unit 107a and the zoom setting value of the zoom unit 107b when the power is turned on or when the projection area detection item of the menu called by the operation unit 109 is selected. The process is started when the setting value of at least one of the lens model information of the lens exchanging unit 108 is changed.

When the process of detecting the projection area is started in FIG. 3, the inspection image is projected from the projection lens 103 onto the projection surface 2 in step S100 (image projection step). At this time, the inspection image is a white image with the maximum overall brightness.

In step S101, the projection image projected in step S100 is photographed using the image pickup means 3 (projection image pickup step).

In step S102, as in step S100, the inspection image is projected from the projection lens 103 onto the projection surface 2 (image projection step). At this time, the inspection image is a black image with the minimum overall brightness.

In step S103, the projection image projected in step S102 is photographed using the imaging unit 3 as in step S101 (projection image capturing step).

In step S104, a background difference is performed using the captured image of the white image obtained in step S101 and the captured image of the black image obtained in step S103. The background difference means that the pixel value of the captured image of the white image and the pixel value of the captured image of the black image are differentiated at the same location. This makes it possible to remove noise that is not caused by the projected image. The processes of steps S102 to S104 can be omitted.

In step S105, the lens driving unit 107 refers to at least one of the shift setting value of the shift unit 107a, the zoom setting value of the zoom unit 107b, and the lens model information of the lens exchanging unit 108. In this embodiment, the projection area is detected by using all of these three pieces of information.

Subsequently, in step S110, the projection area is estimated using at least one of the shift setting value, the zoom setting value, and the lens model information (projection area estimating step). FIG. 4 shows a detailed internal flowchart in step S110.

In FIG. 4, in step S111, calibration data based on lens model information calculated in advance is read. FIG. 6 shows an explanatory diagram of the calibration data. These figures show the captured image when the projected image is captured, where the solid rectangle represents the projected image, the dotted rectangle represents the captured image, and the circle represents the center coordinates of the projected image.

FIG. 6A is an explanatory diagram of calibration data regarding shift.

A rectangle 500 is a projected image when the shift is at the center of the left, right, up and down, a rectangle 501 is a projected image when the shift is at the top of the left and right center, and a rectangle 502 is a projected image when the shift is at the bottom of the left and right center. A rectangle 503 represents a projected image when the shift is the center of the up and down at the left end, and a rectangle 504 represents a projected image when the shift is the center of the up and down at the right end. Further, the center coordinates of the rectangles 500, 501, 502, 503, 504 are circles 510, 511, 512, 513, 514, respectively. At this time, the shift calibration data represents the center coordinates of the projected image represented by circles 510, 511, 512, 513, and 514.

FIG. 6B is an explanatory diagram of calibration data regarding zoom.

A rectangle 520 represents a projection image when the zoom is at the tele end, a rectangle 521 represents a projection image when the zoom is at the center, and a rectangle 522 represents a projection image when the zoom is at the wide end. The center coordinates of the rectangles 520, 521 and 522 are the circle 530. At this time, the zoom calibration data represents the distance on the captured image from the circle 530 to the four corners of the rectangles 520 and 522.

Here, it should be noted that the shift and zoom calibration data change according to the lens model information. That is, if the model of the lens changes, different calibration data must be used. Note that the calibration data may be held for each lens model, or one calibration data may be converted according to each lens model.

Further, since the relative positional relationship cannot be maintained unless the image pickup means 3 is built in the projection means 1, these calibration data must be calculated each time. Therefore, it is preferable that the image pickup means 3 is built in the projection means 1 or the relative positional relationship is maintained. Otherwise, the calibration data is recalculated before starting the step of detecting the projected area.

Then, in step S112, the center coordinates of the projection area are estimated based on the shift setting value. The center position of the projection image is estimated using the calibration data read in step S111 and the referenced shift setting value. Here, the horizontal shift setting value

, The left shift setting value

, Right end shift setting value

, The calibration data for the leftmost shift

, The calibration data for the rightmost shift

The center coordinates of the guess region

Can be calculated as follows.

Similarly, the vertical shift setting value

, The lower shift setting value

, The upper shift setting value

, The calibration data of the bottom shift

, The calibration data for the top shift

The center coordinates of the guess region

Can be calculated as follows.

Further horizontal shift setting value

And vertical shift setting

In the case of the combination of

The center coordinates of the guess region

Can be calculated as follows.

In step S113, the size of the projection area is estimated based on the zoom setting value. The size of the projection image is estimated by using the calibration data read in step S111 and the referred zoom setting value. Here, set the zoom setting value

, The zoom setting value of the tele end

, Wide end zoom setting

, Tele end zoom calibration data

, Wide end zoom calibration data

, Constant

The size of the guess region

Can be calculated as follows.

The position/size of the projection area slightly changes in the captured image according to the distance between the projection unit 1 and the projection surface 2. This increases as the optical center of the projection device and the optical center of the imaging device are separated from each other. Therefore, the constant

To introduce. That is, by increasing the size of the estimation area, the estimation area can be calculated so as to cover the projection area. This constant

May be set in advance according to the distance between the optical center of the projection device and the imaging device, or may be reset when a proper projection region is not obtained in step S124.

In step S114, the projection area is estimated from the center coordinates and the size of the projection area obtained in steps S112 and S113. FIG. 7 shows an actual example of the estimation area. In these figures, a captured image when a projected image is captured is represented. A solid line rectangle 540 represents the projected image, a dotted line rectangle represents the captured image, and double lines 541 and 542 represent the estimated region. The estimated region may be rectangular as shown in FIG. 7A or circular as shown in FIG. 7B. Any other shape may be used as long as the projection area can be included in the estimation area.

By the step S110 described above, it is possible to estimate the projection area.

Subsequently, in step S120, a projection area is detected from the estimated area (projection area detection step). FIG. 5 shows a detailed internal flowchart in step S120.

In FIG. 5, in step S121, feature points are detected. Laplace operators, Harris operators, etc. are generally known for the detection of feature points, but by using these, points where the brightness and chromaticity change drastically are detected on the image.

Then, in step S122, noise removal is performed from the obtained plurality of feature points. In noise removal, points such as white noise that occur irregularly and feature points other than corner points of the projected image are excluded. This uses the nature of the corner points of the projected image. Generally, in the case of a projected image, looking at the area around the corner point to be detected, about 1/4 is a bright point and about 3/4 is a dark point. Therefore, it is possible to prevent the detection error of the projection area by excluding those that do not satisfy this property.

In step S123, four corner points in the projection area are determined from the plurality of feature points from which noise has been removed in step S122. In determining a corner point, the positive and negative differential values in the vertical and horizontal directions are used to divide into four areas, and the representative value of each area is used as the corner point. Where the image coordinates

, The differential value in the horizontal direction

, The vertical derivative

, The threshold

The value divided into areas

Can be calculated as follows.

If is not a corner point,

In the case of, the area is divided into points that are either corner points. In this example,

At

The maximum value of is the representative value, that is, the corner point.

In step S124, the validity of the projection area is verified from the four corner points obtained in step S123. For example, the absolute value of the length of each side, the error amount of the length of the facing side, the angle with the adjacent side, the aspect ratio of the projection area, and the like can be considered. As a result of verifying these, if it is determined that the projection area is appropriate, the process proceeds to step S125, and if not, the process proceeds to step S126.

In step S125, the area surrounded by the four corner points obtained in step S123 is set as the projection area. The obtained projection area is saved in a storage area with a flag or the like so that it can be used in other processing (for example, various automatic corrections such as autofocus, keystone correction, and deterioration over time).

In step S126, error processing is performed when the projection area is not obtained. At this time, a warning may be displayed so that the user can confirm it, and the procedure returns to step S113.

May be reset and the calculation performed again.

The projection area can be determined by step S120 described above.

FIG. 8 shows an explanatory diagram that the disturbance can be eliminated by using the estimation region. These figures show a captured image when a projected image is captured. A solid line rectangle 550 is a projected image, a dotted line rectangle is a captured image, a black rectangle 551 is a disturbance, a double line 554 is an estimated region, and a thick line. Reference numerals 552, 553, and 555 represent the detected projection areas.

FIG. 8A is an example in which the lower left corner point is erroneously detected due to disturbance and an error occurs in the validity verification in step S124. Further, FIG. 8B is an example in which all the corner points are erroneously detected by the disturbance and the disturbance portion is set as the projection area. On the other hand, in (c) of FIG. 8 in which the estimated area is used, the projection area can be accurately detected. This is because the factors of disturbance could be eliminated by the estimation region.

According to the present embodiment, when there is a disturbance, the projection area can be detected more accurately based on the set value of the projection device without using a special projection pattern such as a test chart.

100 image projection system, 1 projection means, 2 projection surface, 21,551 disturbance,
3 image pickup means, 4 control means, 101 light source, 102 liquid crystal panel,
103 projection lens, 104 image signal input unit, 105 image processing unit,
106 control unit, 107 lens drive unit, 107a shift unit,
107b zoom unit, 108 lens replacement unit, 109 operation unit,
110 image display system, 111 projection optical system,
500, 501, 502, 503, 504, 520, 521, 522, 540, 550 Projection image,
510, 511, 512, 513, 514, 530 central coordinates of the projected image,
541, 542, 554 guess area, 552, 553, 555 projection area

Claims (4)

An image display unit including an image display element that displays image information according to an image signal input from the input unit,
Projection means for projecting image information displayed by the image display means onto a projection surface,
Image pickup means for picking up a projection image projected by the projection means;
Control means for detecting a projection area by using a part of the image information obtained by the image pickup means;
An image projection system comprising: The projection unit, a shift unit for changing the position of the projection image projected on the projection surface,
The image projection system according to claim 1, further comprising at least one of a zoom unit that changes a size of a projected image and a lens replacement unit that attaches and detaches a lens. The control unit uses at least one of the shift setting value of the shift unit of the projection unit, the zoom setting value of the zoom unit, and the lens model information of the lens exchanging unit, from the image information obtained by the imaging unit. The projection area estimation step of estimating a position at which a projection image is captured, and the projection area detection step of detecting a projection area from the estimation area obtained by the projection area estimation step. Item 2. The image projection system according to item 2. The image projection system has an operation unit that detects an operation by a user, and when the image projection system is activated or when an execution operation is selected by the operation unit, or a shift set value of a shift unit and a zoom unit. 2. The projection area estimation step and the projection area detection step are started when at least one of the zoom setting value and the lens model information of the lens replacement unit is changed, or both of them. The image projection system according to claim 3.