JP2019169913A - Projection control device, projection device, correction image projection method, and program - Google Patents

Abstract

[PROBLEMS] To provide a projection control device, a photographing device, a correction image projection method, and a program that can execute initial settings as easily as possible even when an image is projected on a projection target other than a dedicated screen. To do. A projection control device (projection device) includes a first correction image including a marker image for position detection and a non-marker image, and a first image including a measurement image and a marker image for image correction. An acquisition unit (processing unit 12) for acquiring a second correction image, and a projection control unit for controlling a projection unit (projection unit 13) capable of projecting the first correction image and the second correction image onto a projection target. (Processing unit 12), detection means capable of detecting a marker image from the captured image of the first correction image and the second correction image projected by the projection means, and the first correction detected by the detection means Detection condition setting means for setting a detection condition of the marker image in the second correction image based on a detection result of the marker image of the image for correction. [Selection] Figure 2

Description

  The present invention relates to a projection control device, a projection device, a correction image projection method, and a program.

  In a projection apparatus that projects an image, a technique for displaying an image projected by the projection apparatus in a correct color or shape when a projection surface such as a wall has color, pattern, distortion, or the like has been proposed (for example, Patent Document 1).

JP 2007-259472 A

  In order to correct the projection image by accurately grasping the color reproduction state in the projection range of the projection apparatus, including the technique described in the above-mentioned patent document, the color information of the image projected on the projection plane by the projection apparatus is used. It is necessary to provide means for obtaining (a color sensor in the case of Patent Document 1). In this case, if the position of the projection device is different from the position of the viewer of the projection image, it is difficult for the projection device having the photographing unit to correct the image reflecting the position of the viewer even if the projection image is captured. There are cases.

  Therefore, as a means for accurately obtaining the color information of the image, a separate photographing device such as a digital camera used by the user of the projection device can be considered. If the color correction of the projection apparatus can be executed by taking a projection image with a general digital camera, the initial setting of the projection apparatus can be realized easily and accurately.

  However, in order to correct an image more accurately, it is necessary to continuously project a plurality of correction image patterns by a projection device and to photograph the plurality of image patterns with a digital camera. If these projected images are photographed with a digital camera in a handheld state, the photographing position of the digital camera itself is not stable, and there is a concern about the occurrence of camera shake.

  Therefore, it becomes difficult to accurately detect a range projected by the projection device in continuously obtained captured images. There is a method of fixing the digital camera with a tripod or the like in order to accurately capture the projected image, but the labor and time required for the initial setting becomes much more complicated.

  The present invention has been made in view of the above circumstances, and the object of the present invention is to perform initial setting as easily as possible even when an image is projected onto a projection target other than a dedicated screen. It is an object of the present invention to provide a projection control device, a projection device, a projection control method, and a program that can be used.

  The projection control device of the present invention includes a first correction image including a marker image for position detection and a non-marker image, and a second correction image including a measurement image for image correction and the marker image. Acquisition means for acquiring the first correction image, projection control means for controlling the projection means capable of projecting the second correction image and the second correction image onto the projection target, and the first image projected by the projection means Based on a detection unit capable of detecting the marker image from a captured image of the correction image and the second correction image, and based on the detection result of the marker image of the first correction image detected by the detection unit And a detection condition setting means for setting a detection condition for the marker image in the second correction image.

  A projection apparatus according to the present invention includes the above projection control apparatus and the projection unit.

  The correction image projection method of the present invention includes a marker image for position detection, a first correction image including a non-marker image, and a second correction image including a measurement image for image correction and the marker image. An acquisition step of acquiring an image by an acquisition unit; a projection control step of projecting the first correction image and the second correction image onto a projection target by a projection unit; and the first image projected by the projection unit A detection step of detecting the marker image from the correction image and the second correction image, and a detection result of the marker image of the first correction image detected in the detection step. And a detection condition setting step of setting a detection condition for the marker image in the second correction image.

  The program of the present invention is a program executed by a computer, and the computer includes a first correction image including a marker image for position detection and a non-marker image, a measurement image for image correction, and the measurement image. An acquisition unit that acquires a second correction image including a marker image by an acquisition unit, a projection control unit that projects the first correction image and the second correction image onto a projection target by a projection unit, and the projection Detection means for detecting the marker image from the first correction image and the second correction image projected by the means, and the marker image of the first correction image detected by the detection means. Based on the detection result, the detection function setting means for setting the detection condition of the marker image in the second correction image is used.

  According to the present invention, it is possible to provide a projection control device, a correction image projection method, and a program capable of executing initial settings as easily as possible even when an image is projected onto a projection target other than a dedicated screen. can do.

It is a figure which shows the outline | summary of the projection system which concerns on embodiment of this invention. It is a figure which shows the structure of the projection system which concerns on embodiment of this invention. It is a figure which shows the image for correction | amendment which concerns on embodiment of this invention, (a)-(c) is a correction image for color correction of a to-be-measured object, (d)-(f) is distortion of to-be-measured object. It is a correction image for correction. It is a figure which shows the structure of the marker which concerns on embodiment of this invention. It is a figure which shows the structure of the outline information table which concerns on embodiment of this invention. It is a flowchart figure of the projection apparatus which concerns on embodiment of this invention. It is a flowchart figure of the imaging device which concerns on embodiment of this invention. It is a flowchart figure of the marker detection program which concerns on embodiment of this invention. The image which the imaging device which concerns on embodiment of this invention processes is shown, (a) shows the 1st image for correction | amendment image | photographed with the imaging device, (b) extracted the outline of the image for 1st correction | amendment. The latter image is shown, and (c) is an enlarged view of the contour of the extracted marker portion.

  Hereinafter, modes for carrying out the present invention will be described. FIG. 1 is a diagram showing an outline of the projection system 1. The projection system 1 includes a projection device 10 (projection control device) that projects an image on a projection surface 31 of a projection target 30 such as a curtain or a wall, a digital camera that captures an image projected on the projection surface 31, and a camera-equipped mobile phone. And a photographing device 20 such as a telephone. The photographing device 20 may be any digital device with a portable image input function, and may be a video camera, a smartphone (high-function mobile phone), a tablet terminal, or the like. In the present embodiment, a curtain is shown as the projection object 30. The projection object 30 is suspended by a rail or the like (not shown), and has a wavy undulation mainly on the surface along the lateral direction. Further, colors and patterns are formed on the projection surface 31.

  The projection device 10 is arranged at a different position from the imaging device 20. The projection apparatus 10 in FIG. 1 is disposed on the left side with respect to the projection surface 31 and projects an image from an oblique direction. The imaging device 20 is arranged at the position of the user who views the image projected on the projection surface 31 such as substantially in front of the projection surface 31. Therefore, the image captured by the image capturing device 20 and the image visually recognized by the user can be configured so that the angle of view substantially matches. The imaging device 20 may be held by a user.

  An outline of the operation of the projection system 1 will be described. When the projection image projected by the projection device 10 remains projected, the projection image 31 is affected by the shape and pattern of the projection surface 31 and the difference between the projection direction of the projection device 10 and the user's line-of-sight direction. From this viewpoint, it is observed in a color and shape different from the original image data. Therefore, in the present embodiment, the processing unit 12 corrects the image data so that the user can suitably view the projected image from the user's viewpoint. When the projection device 10 projects a correction image 111 (to be described later) onto the projection surface 31, the imaging device 20 captures the projected correction image. The imaging device 20 obtains image correction information based on the difference between the correction image 111 projected by the projection device 10 and the correction image 111 in the captured image 60 captured by the imaging device 20, and sends the image correction information to the projection device 10. The data is transmitted via wired or wireless communication. Through the processing described above, the projection apparatus 10 can project an image based on the image correction information so that the basic image data is visually recognized in a manner intended for the user who is an observer.

  FIG. 2 is a diagram showing a configuration of the projection system 1. The projection apparatus 10 includes a storage unit 11, a processing unit 12 (acquisition unit, projection control unit), a projection unit 13 (projection unit), an operation unit 14, a communication unit 15, and an audio processing unit 16, each connected by an internal bus. ing. The storage unit 11 includes, for example, an SSD (Solid State Drive) or an SRAM (Static Random Access Memory). The storage unit 11 stores image data for viewing (not shown), a correction image 111 for correcting the image data, a control program 112 for the projection apparatus 10, and the like.

  The processing unit 12 reads the control program 112 stored in the storage unit 11 and controls the projection apparatus 10 in an integrated manner. The processing unit 12 includes an image correction processing unit 121. The image correction processing unit 121 acquires the correction image 111 from the storage unit 11 and transmits it to the projection unit 13. Further, the image correction processing unit 121 corrects the image data for viewing stored in the storage unit 11 based on the image correction information 25 a received from the imaging device 20, and transmits the converted image after correction to the projection unit 13. To do. Further, the image correction processing unit 121 compares the correction image 111 in the captured image 60 acquired from the imaging device 20 as described above with the correction image 111 stored in the storage unit 11, thereby obtaining the image correction information 25 a. You may make it calculate.

  The projection unit 13 projects the image of the correction image 111 or the image data for viewing sent from the processing unit 12 including the image correction processing unit 121 at a frame rate according to a preset image format. The projection unit 13 forms an image by emitting light of different colors for each pixel within one frame. The color for each pixel can be expressed by emitting light in a plurality of different wavelength bands in a time division manner. In the projection apparatus 10 of the present embodiment, the projection unit 13 is configured as a projection unit of a DLP (registered trademark) (Digital Light Processing) projector, but may be other types such as a liquid crystal projector. The image emitted from the projection unit 13 is projected onto the projection plane 31 in FIG.

  The operation unit 14 receives an operation signal from an operation key or the like provided in the housing of the projection device 10 and transmits the operation signal to the processing unit 12 via the bus. The processing unit 12 executes various functions such as projection processing in accordance with operation signals from the operation unit 14.

  The communication unit 15 receives an operation signal such as an infrared modulation signal from a remote controller (not shown) and transmits the operation signal to the processing unit 12. Further, the communication unit 15 can transmit and receive data to and from the communication unit 25 of the photographing apparatus 20. In addition, the communication unit 15 can include an external input terminal, and can receive image data for viewing from an external device. Communication with the communication unit 25 by the communication unit 15 can be performed by wire or wireless.

  The sound processing unit 16 includes a sound source circuit such as a PCM sound source, and drives the speaker 17 to diffuse and emit sound. When the image data to be projected includes an audio signal, the audio processing unit 16 converts the audio signal into an analog signal and outputs a sound through the speaker 17 during the projection operation.

  The imaging device 20 includes an imaging unit 21, a processing unit 22 (detection unit, detection condition setting unit), an input / output control unit 23, a storage unit 24, and a communication unit 25, which are connected by an internal bus. The imaging unit 21 captures an image projected on the projection surface 31 of FIG.

  The processing unit 22 has a function of controlling the imaging device 20. The processing unit 22 includes a measurement processing unit 221. The measurement processing unit 221 analyzes the captured image 60 including the correction image 111 captured by the capturing unit 21. The measurement processing unit 221 transmits the analysis result of the correction image 111 as the image correction information 25a to the projection device 10 via the communication unit 25. In addition, the measurement processing unit 221 transmits a projection instruction 25 b to the projection device 10 via the communication unit 25 according to the analysis result of the captured image 60.

  The input / output control unit 23 transmits information corresponding to the operation instruction input from the operation button 231 by the user to the processing unit 22. The display device 232 is a screen or an indicator lamp that displays an image output from the input / output control unit 23.

  The storage unit 24 stores a contour information table 241 and a control program for the photographing apparatus 20. The control program includes a marker detection program 242 (detection means, detection condition setting means). Further, the storage unit 24 stores the detected position information 243 of the image of the marker 41 (described later in FIG. 3 and the like) detected from the captured image 60 captured by the capturing unit 21 and the projection plane image in the captured image 60.

  Next, the correction image 111 stored in the storage unit 11 of the projection apparatus 10 will be described with reference to FIGS. 3 (a) to 3 (f). The correction images 111a to 111f can be images having an aspect ratio corresponding to the projection mode, and the entire image has a horizontally long rectangular shape in FIG. Each of the correction images 111a to 111f has image position identification markers 41 at the four corners of the entire image, and most other regions are used as measurement images 42 for image correction. This image correction includes color correction or image shape correction.

  FIG. 4 is a diagram showing the configuration of the marker 41 for position detection of the correction images 111a to 111f. The marker 41 is configured by a first polygon 43 and a second polygon 44 arranged inside the first polygon 43. The first polygon 43 is formed in a regular quadrangle in which the opposite sides are parallel and have the same length, and each interior angle is 90 degrees. Each side of the first polygon 43 has a predetermined width colored black by the outer edges 431a to 431d and the inner edges 432a to 432d, and is surrounded by the edges 432a to 432d. The inside of 43 is colored white. The outside of the first polygon 43 divided by the outer edges 431a to 431d is colored with a color corresponding to the measurement image 42 (see FIG. 3) of the correction image 111. The second polygon 44 is formed in a region surrounded by the inner edges 432 a to 432 d of the first polygon 43. The second polygon 44 is formed in an equilateral triangle with each edge 441a to 441c having the same length. The interior surrounded by the respective edges 441a to 441c of the second polygon 44 is colored black with the same color as the first polygon. In the present embodiment, the imaging device 20 detects the edges 432a to 432d of the first polygon 43 and the edges 441a to 441c of the second polygon 44 to identify the marker 41.

  Upper and lower edges 431 a, 431 b, 432 a, 432 b of the first polygon 43 are arranged in parallel with the lower edge 441 c of the second polygon 44. Further, the respective edges 431 a to 431 d and 432 a to 432 d of the first polygon 43 are formed so as to be non-parallel to the left and right edges 441 a and 441 b of the second polygon 44. As described above, in the marker 41, the first polygon 43 and the second polygon 44 have different numbers of vertices, but at least one side of the first polygon 43 has the second polygon 44. The other side different from the one side of the first polygon 43 is formed in parallel with any side of the second polygon 44.

  3A to 3C are correction images 111a to 111c for color correction of the projection plane 31. FIG. In the correction image 111a, the correction image 111b, and the correction image 111c, the measurement image 42 on substantially the entire surface is black, gray, and white, respectively. The projection device 10 projects the correction images 111 a to 111 c onto the projection surface 31 to cause the photographing device 20 to photograph, and changes in color due to the pattern formed on the projection surface 31 and distortion of the projection surface 31. Is detected.

  3D to 3F are correction images 111d to 111f for correcting distortion of the image projected on the projection target 30. FIG. The correction image 111d, the correction image 111e, and the correction image 111f have a checkered pattern in which substantially square white squares and black squares are alternately arranged in the vertical direction and the horizontal direction on the measurement image 42 on the substantially entire surface. The correction images 111d to 111f can detect the distortion of the projection surface 31 with different resolutions depending on the checkered pattern size. The projection device 10 projects the correction images 111d to 111f on the projection surface 31 and causes the photographing device 20 to photograph. The imaging device 20 can detect distortion of the entire projection surface 31 by combining a plurality of correction images 111d to 111f.

  FIG. 5 is a diagram showing the configuration of the contour information table 241. The contour information table 241 stores a contour number 51, a lower layer contour number 52, and a lower layer contour number 53 in association with each other. The method for acquiring each data will be described later with reference to the flowchart of FIG. The contour number 51 is an identification number set for each contour extracted in the contour extraction process (step S302 in FIG. 8). The lower-layer contour number 52 indicates how many contours are included in a layer inside one contour when the contour corresponding to the contour number 51 is a closed region. The lower layer contour number 53 is an identification number set for each contour counted by the number 52 of lower layer contours. Thus, the contour information table 241 can store the hierarchical structure of each contour.

  Next, a method for projecting the correction image 111 using the correction images 111a to 111f by the projection device 10 and the photographing device 20 will be described. In brief, in the present embodiment, the projection apparatus 10 includes a total of six correction images 111, a black correction image 111a, a gray correction image 111b, a white correction image 111c, and a large checkered image. The pattern correction image 111d, the medium checkered pattern correction image 111e, and the small checkered pattern correction image 111f are projected onto the projection surface 31 in this order. In this case, the square corners of the checkered images 111d, 111e, and 111f having different sizes are positioned so as not to overlap each other during projection. The projected correction image 111 (111a to 111f) is captured by the imaging device 20. The projection apparatus 10 receives the projection instruction 25b for the next correction image 111 in accordance with the detection result of the position of the marker 41 in the captured image 60 (60a to 60f) from the imaging apparatus 20, and sequentially projects the correction image 111. . Specific processing will be described below.

  FIG. 6 is a flowchart of the projection apparatus 10 including an acquisition process, a projection control process, and the like. In step S <b> 101, the image correction processing unit 121 (processing unit 12) acquires the first correction image 111 a from the storage unit 11 as the first correction image, and causes the projection unit 13 to project the image on the projection surface 31. The first correction image is an image that first causes the photographing apparatus 20 to detect the marker 41 by projecting a series of correction images 111. In the present embodiment, the measurement image 42 of the correction image 111a that is the first correction image is a dummy image that does not meet some or all of the marker determination conditions in steps S303, S306, S308, S311, and S313 of FIG. Functions as (non-marker image). Therefore, it is possible to reduce an increase in processing load caused by the imaging device 20 extracting an image different from the marker 41 as a marker candidate. The marker determination condition may include that contour extraction is performed in the contour extraction process in step 302.

  In step S <b> 102, the processing unit 12 determines whether the projection instruction 25 b for the next correction image 111 has been received from the imaging device 20. When the processing unit 12 receives the projection instruction 25b for the next correction image 111 (S102, YES), the processing unit 12 proceeds to the process of step S103, and does not receive the projection instruction 25b for the next correction image 111 (S102, NO). The determination process in step S102 is repeated.

  In step S103, the processing unit 12 determines whether the projection instruction 25b is a reprojection instruction for the correction image 111 projected last time. When the projection instruction 25b is a reprojection instruction (S103, YES), the processing unit 12 returns to the process of step S101. When the projection instruction 25b is not a reprojection instruction (S103, NO), the processing unit 12 proceeds to the process of step S104.

  In step S <b> 104, the image correction processing unit 121 (processing unit 12) stores the next correction image 111 b (second correction image) of the correction image 111 a previously projected as the next correction image 111. And projected onto the projection plane 31 by the projection unit 13.

  In step ST1, the image correction processing unit 121 (processing unit 12) stores, as the next correction image 111, the next correction image 111c (second correction image) of the correction image 111b projected last time. And projected onto the projection plane 31 by the projection unit 13.

  Thereafter, the processing unit 12 repeats the process of step ST1, sequentially acquires the correction images 111d to 111f as second correction images from the storage unit 11 in a predetermined order, and the projection unit 13 causes the projection surface 31 to acquire the correction images. Project. When the processing unit 12 projects the predetermined correction images 111a to 111f, the processing ends.

  FIG. 7 is a flowchart of the photographing apparatus 20 including a detection process, a detection condition setting process, and the like. In step S <b> 201, the processing unit 22 causes the photographing unit 21 to photograph the correction image 111 a (first correction image), acquires the correction image 111 a as the photographed image 60 a, and stores it in the storage unit 24. A captured image 60a illustrated in FIG. 9A is an image obtained when the correction image 111a is captured, and includes the correction image 111a and an external region 60a1 around the correction image 111a. In the example of the photographed image 60a, since the surrounding environment is relatively dark, the external region 60a1 is acquired as a black region substantially equivalent to the measurement image 42. In addition, the pattern 31 a formed on the projection surface 31 is photographed together with the marker 41 in the region indicated by the two-dot chain line on which the correction image 111 a is projected.

  In step ST21 (ST2), the measurement processing unit 221 (processing unit 22) performs a marker detection process described later with reference to FIG. 8, and detects the marker 41 of the correction image 111a acquired last time. In the present embodiment, binarization is performed on the captured image 60 to detect the position of the marker 41. Therefore, when the measurement image 42 is projected as a dummy image having a substantially black surface and the correction image 111a is projected as the first correction image, the number of contours extracted in the marker detection process of step ST21 is reduced. And the processing load when the marker detection program 242 is executed can be reduced. In step ST21 (ST2), when the marker 41 is detected by the marker detection process, the position of the marker 41 is stored in the storage unit 24 as the detection position information 243.

  In step S202, the processing unit 22 determines whether the marker 41 has been detected in step ST21. When the marker 41 can be detected (S202, YES), the processing unit 22 proceeds to the process of step S203, and when the marker 41 cannot be detected (S202, NO), the process proceeds to the process of step S207.

  In step S <b> 207, the processing unit 22 transmits a projection instruction 25 b for reprojecting the correction image 111 a to the projection device 10 via the communication unit 25.

  In step S203, the processing unit 22 instructs the projection apparatus 10 through the communication unit 25 to provide a projection instruction 25b for projecting the next correction image 111b.

  In step S204, the processing unit 22 causes the photographing unit 21 to photograph the correction image 111b (second correction image), which is the next correction image 111, and obtains a photographed image 60b obtained by photographing the correction image 111b. And stored in the storage unit 24.

  In step S <b> 205, the measurement processing unit 221 (processing unit 22) acquires the detection position information 243 of the marker 41 of the correction image 111 a stored in the storage unit 24 from the storage unit 24. Then, the measurement processing unit 221 determines, based on the detection position information 243, a region around which the marker 41 is detected in the previous marker detection process as a target for the marker detection process of the next correction image 111b.

  In step ST22 (ST2), the measurement processing unit 221 (processing unit 22) performs marker detection processing, which will be described later with reference to FIG. 8, on the region determined in step S205 for the correction image 111b acquired last time, and the correction image 111b. The marker 41 is detected.

  In step S206, the processing unit 22 determines whether the marker 41 has been detected in step ST22. When the marker 41 can be detected (S206, YES), the processing unit 22 proceeds to the process of step ST3, and when the marker 41 cannot be detected (S206, NO), the process proceeds to the process of step S208.

  In step S208, the measurement processing unit 221 (processing unit 22) changes the area in the captured image 60 (60b) on which the previous marker detection processing has been performed. For example, the measurement processing unit 221 can enlarge the target area in the captured image 60b on which the previous marker detection process has been performed.

  In step ST <b> 3, the processing unit 22 repeats the processes of steps S <b> 203 to S <b> 206 and S <b> 208 to detect each marker 41 of the captured images 60 c to 60 f obtained by sequentially capturing the correction images 111 c to 111 f.

  In step S209, the measurement processing unit 221 specifies the positions of the correction images 111a to 111f in the captured images 60a to 60f from the positions of the markers 41 detected from the captured images 60a to 60f. Specifically, in the present embodiment, the correction images 111a to 111f shown in FIG. 3 are all provided with the markers 41 at the same positions at the four corners of the image. Therefore, the measurement processing unit 221 uses the correction images 111a. By detecting the position shift of the marker 41 between .about.111f, the position shift of the parallel component and the rotation component of the correction images 111a to 111f is detected. The photographing apparatus 20 detects a positional shift between the plurality of correction images 111a to 111f and corrects the offset. As a result, it is possible to correct the misalignment of the correction images 111a to 111f between the captured images 60a to 60f caused by camera shake or the like. After the measurement processing unit 221 detects the position of the marker 41 of each of the captured images 60a to 60f, the measurement processing unit 221 detects the coordinates of the intersection (corner) of the checkered pattern from the captured images 60d to 60f, and the correction that the projection device 10 has projected. The degree of distortion of the work images 111a to 111f can be obtained. Moreover, the imaging device 20 can obtain the color change for each position of the projection surface 31 by combining the captured images 60 d to 60 f with the captured images 60 a to 60 c. The measurement processing unit 221 generates these pieces of information as image correction information 25a.

  In step S210, the measurement processing unit 221 transmits the image correction information 25a to the projection device 10 via the communication unit 25. Thereby, the projection apparatus 10 can correct the image data for viewing by the image correction processing unit 121 based on the image correction information 25a, and project the image so that the intended image can be visually recognized by the user. Next, the detection process of the marker 41 will be described.

  FIG. 8 is a flowchart of a marker detection program executed by the measurement processing unit 221 (processing unit 22). In this figure, an example in which the captured image 60a in FIG. 9A is processed will be described. However, the captured images 60b to 60f obtained by capturing the correction images 111b to 111f are similar to the region determined in step S205 in FIG. Can be processed. First, the measurement processing unit 221 reads the captured image 60 a stored in the storage unit 24. In step S301, the measurement processing unit 221 performs a captured image conversion process that binarizes the captured image 60a by an appropriate method.

  In step S302, the measurement processing unit 221 performs contour extraction processing having hierarchical information from the binarized image converted in step S201. The contour extraction process can be performed using, for example, the findContours function of the OpenCV library. As a specific example, the measurement processing unit 221 detects a boundary between a black pixel and a white pixel of a binarized image as an outline. FIG. 9B shows a contour extraction image 61a obtained by visualizing the result of the contour extraction processing. In the example shown in the figure, eight contours 711 to 718 corresponding to the marker 41 and five contours 721 to 725 corresponding to the pattern 31a are extracted. The measurement processing unit 221 sets contour numbers as identification information in order from the outside of the contour extraction image 61a, for example, to the detected contour. In this figure, contour numbers a1 to a4 are set in order for the contours 711 to 714 in the imaging range 710. Contour numbers a6 to a9 are set in order for the contours 715 to 718 extracted inside the contours 711 to 714, respectively. The contour number a5 is set for the contour 721, and the contour numbers a10 to a13 are set for the contours 722 to 725.

  In addition, the measurement processing unit 221 uses the number of lower layer contours and the lower layer contour number as the information of the layers to which the respective contours 711 to 718 and 721 to 725 belong, the lower layer contour number 52 of the contour information table 241, and the lower layer contour numbers. Correspondingly stored as hierarchical contour number 53. As a result, it is possible to grasp the internal / external positional relationship when a certain contour is nested in a closed region of another contour.

  In step S303, the measurement processing unit 221 determines whether there is an unprocessed contour that has not been processed for marker detection. When there is an unprocessed contour (S303, YES), the measurement processing unit 221 proceeds to the process of step S304, and when there is no unprocessed contour (S303, NO), the process ends. The measurement processing unit 221 proceeds to the process of step S304 because the processes of the contours 711 to 718 and 721 to 725 are not performed in the initial state (S303, YES).

  In step S304, the measurement processing unit 221 determines the contour C0 to be processed. For example, the measurement processing unit 221 determines the contour of the contour number having the smallest numerical value from the contour information table 241 as the contour C0 to be processed. In the example of the contour extraction image 61a, in the initial state, the measurement processing unit 221 extracts the contour 711 having the smallest contour number 51 “a1” from the contour numbers 51 as the contour C0 to be processed.

  In step S <b> 305, the measurement processing unit 221 performs linear approximation processing that can set a closed region on the extracted contour C <b> 0. The straight line approximation process can be performed by using, for example, an approPolyDP function of the OpenCV library. FIG. 9C is an enlarged view of the contour 711 and the contour 715 extracted in steps S301 to S304. When the contour C0 to be processed is a contour 711, it is linearly approximated to a quadrangle closed by four line segments 711a to 711d.

  In step S306, the measurement processing unit 221 determines whether the contour C0 is a closed line segment having the number of vertices “4”. When the measurement processing unit 221 determines that the contour C0 has the number of vertices “4” (S306, YES), the process proceeds to the process of step S307, and when the contour C0 does not determine that the number of vertices is “4” (S306, NO). ) Return to step S303. If the contour C0 to be processed is the contour 711, the number of vertices is “4” (S306, YES), and the process proceeds to step S70.

  In step S307, the measurement processing unit 221 extracts the contour of the next lower hierarchy inside the contour C0 to be processed. For example, when the contour C0 to be processed is the contour 711, the measurement processing unit 221 refers to the contour information table 241 in FIG. 5 and the number of lower layer contours 52 corresponding to the contour number 51 “a1” is “1” or more. If it is “1” or more, the contour 715 of the lower layer contour number 53 “a6” corresponding to the contour number 51 “a1” is extracted. When a plurality of lower layer contour numbers 53 are stored as in the contour number 51 “a5”, a plurality of lower layer contour numbers are extracted. When the lower layer contour number 52 is “0”, the contour number of the lower layer is not extracted.

  In step S308, the measurement processing unit 221 determines whether there is an unprocessed contour in the contour extracted in step S307. If the measurement processing unit 221 determines that there is an unprocessed contour (S308, YES), the process proceeds to step S309, and if it is not determined that there is an unprocessed contour (S308, NO), the process returns to step S303.

  In step S309, the measurement processing unit 221 determines a processing target contour C1 from unprocessed contours among the contours extracted in step S307 inside the contour C0. As the contour C1 to be processed, the contour with the smallest contour number can be selected from the unprocessed contours. For example, when the contour 715 of the lower layer contour number 53 “a6” inside the contour 711 is extracted in step S307 and no processing is performed, the contour 715 is determined as the processing target contour C1.

  In step S310, the measurement processing unit 221 performs a linear approximation process that can set a closed region on the contour C1 determined in step S309. The straight line approximation process can be performed using the approPolyDP function of the OpenCV library, as in step S305. If the contour C1 to be processed is the contour 715, it is linearly approximated to a triangle closed by three line segments 715a to 715c shown in FIG.

  In step S311, the measurement processing unit 221 determines whether the contour C1 is a closed line segment having the number of vertices “3”. When the measurement processing unit 221 determines that the contour C1 has the number of vertices “3” (S311, YES), the process proceeds to the process of step S312 and when the contour C1 does not determine that the number of vertices is “3” (S311, NO) ) Return to the process of step S308. When the contour C1 to be processed is the contour 715, the number of vertices is “3” (S311, YES), and the process proceeds to step S312.

  In step S312, the measurement processing unit 221 calculates the area S0 of the contour C0 and the area S1 of the contour C1, respectively. In an example in which the contour C0 is the contour 711 and the contour C1 is the contour 715, in step S312, the area S0 of the contour 711 (contour C0) with the number of vertices “4” and the contour 715 (contour C1) with the number of vertices “3”. ) Is calculated.

  In step S313, the measurement processing unit 221 determines whether the value obtained by dividing the area S0 by the area S1 (the ratio between the area S0 and the area S1) is larger than a predetermined minimum area ratio Rmin and smaller than the maximum area ratio Rmax. . If the value obtained by dividing the area S0 by the area S1 is larger than the minimum area ratio Rmin and smaller than the maximum area ratio Rmax (S313, YES), the measurement processing unit 221 proceeds to the process of step S314. On the other hand, if the value obtained by dividing the area S0 by the area S1 is equal to or less than the minimum area ratio Rmin or the maximum area ratio Rmax (S313, NO), the measurement processing unit 221 returns to the process of step S308. The range of the minimum area ratio Rmin and the maximum area ratio Rmax can be determined based on the square area on the marker design and the area of the triangle in consideration of the imaging capability and measurement error of the imaging apparatus 20.

  In step S <b> 314, the measurement processing unit 221 extracts the currently selected combination of the contour C <b> 0 and the contour C <b> 1 as a marker candidate and stores it in the storage unit 24.

  Note that if the contour C0 to be processed is the contour 721 (contour number “a5”) unrelated to the marker 41 shown in FIG. 9B in steps S304 to S306, the number of vertices is “4”. The process proceeds to step S307. In steps S307 to S309, the contour 722 or the contour 723 corresponding to the contour number “a10” or “a11” is extracted as the contour C1 from the lower layer contour number 53 corresponding to the contour number 51 “a5” of the contour information table 241. Since neither the contour 722 nor the contour 723 has the number of vertices “3” (S 311, NO), the combination of the contour 721 and the contour 722 or the contour 723 is excluded from the marker candidates.

  Similarly, even if the contour C0 to be processed in steps S304 to S306 is the contour 722 or the contour 723, the contour 724 and the contour 725 arranged inside them are not the number of vertices “3” (NO in S311). The combination of the contour 722 and the contour 724 and the combination of the contour 723 and the contour 725 are excluded from the marker candidates.

  Through the above steps S303 to S314, from the contour extracted image 61a of FIG. 9B, as a marker candidate, a combination of a contour 711 and a contour 715, a combination of a contour 712 and a contour 716, a combination of a contour 713 and a contour 717, and A combination of the contour 714 and the contour 718 is extracted as a candidate for the marker 41. The measurement processing unit 221 can determine the extracted four marker candidates as the marker 41 projected by the projection apparatus 10.

  The measurement processing unit 221 similarly performs the marker detection processing in steps S301 to S314 for the remaining captured images 60b to 60f obtained by capturing the correction images 111b to 111f.

  In addition, after the process of FIG. 8, it is good also as a structure which further narrows down the marker candidate extracted by step S301-S314 by the separate determination means, and determines with the marker 41 for image position identification. For example, the measurement processing unit 221 finally uses the combination of the contour C0 and the contour C1 (for example, the contour 711 and the contour 715) extracted as the marker candidates in step S314 for image position adjustment between the correction images 111a to 111f. Can be determined. As a determination method, the following (1) determination by line segment length and (2) determination by inclination of a line segment can be used.

(1) Determination by Line Segment Length In determination by line segment length, the measurement processing unit 221 first calculates the lengths of the line segments 711a to 711d and 715a to 715c of the contour 711 and the contour 715. The measurement processing unit 221 can determine as a marker when the lengths of the line segments 711a to 711d and 715a to 715c are within a predetermined threshold range. The threshold ranges applied to the line segments 711a to 711d and the line segments 715a to 715c can be set to different ranges. The threshold ranges applied to the line segments 711a to 711d and 715a to 715c may be different for the contour 711 and the contour 715, respectively. In the determination by the line segment length, the threshold range may be determined based on the relative length of each of the line segments 711a to 711d and 715a to 715c.

(2) Determination by Line Segment Inclination In determination by line segment inclination, the measurement processing unit 221 determines the marker 41 when the inclination of each of the line segments 711a to 711d and 715a to 715c is within a predetermined threshold range. be able to. The threshold ranges applied to the line segments 711a to 711d and 715a to 715c may be different from each other. The threshold range can be determined by a relative inclination with respect to a specific other line segment. For example, if one line segment of the first polygon 43 is not parallel to at least each line segment of the second polygon 44, it can be determined that it is a marker. In the example of FIG. 9C, one line segment 711a, 711b of the outline 711 of the first polygon 43 and the line segment 715c of the outline 715 of the second polygon 44 are substantially parallel. Since one line segment 711c, 711d of the outline 711 of one polygon 43 and the line segments 715a to 715c of the outline 715 of the second polygon 44 are not in a parallel relationship, the combination of the outline 711 and the outline 715 is It can be determined that it is a marker.

  Further, it may be determined whether to use as a marker for image position adjustment by combining the determination based on the line segment length and the determination based on the inclination of the line segment.

  As described above, if a marker is extracted, a number of marker candidates different from the number arranged in the correction image 111 are extracted, or if the marker 41 of the captured image is detected in an unclear position or shape, Marker candidates can be removed. Thus, the detection accuracy of the marker 41 can be increased and the accuracy of position correction between the correction images 111 can be improved.

  Note that the projection instruction 25b in step S207 may be an instruction for re-projecting the previously-corrected correction image 111a, or a different image for projecting the marker 41 is projected as the first correction image. It may be an instruction. For example, the processing unit 22 may transmit, as the projection instruction 25b, an instruction to project the correction image 111c having the measurement image 42 whose substantially entire surface is white to the projection apparatus 10. Even if the marker detection process of step ST2 is performed on the captured image 60 obtained by capturing the correction image 111c, the number of extracted contours can be reduced, and the processing load when the marker detection program 242 is executed is reduced. be able to.

  In the present embodiment, the method of continuously projecting the correction images 111b to 111f after the projection device 10 projects the first correction image 111a (see step ST1 in FIG. 6) has been described. The correction images 111b to 111c may be projected one by one every time the projection apparatus 10 receives the projection instruction 25b. In addition to the plain single color image such as the black image of the correction image 111a and the white image of the correction image 111c, the dummy image that does not conform to the marker detection condition arranged on the substantially entire surface of the first correction image. A blue noise image or a noise image having a pattern greatly different from the shape and size of the detected marker may be arranged.

  In the present embodiment, the black image of the correction image 111a is used as the dummy image that does not conform to the marker detection condition that is arranged on the substantially entire surface of the first correction image. You may comprise so that it may project on. For example, it may be selected according to a user input operation detected via the operation unit 14 of the projection apparatus 10 or may be selected by an input operation from an external device connected via a communication unit. In addition, when the projection apparatus 10 includes an illuminance sensor and it is determined that the surrounding environment is dark from the detection result of the illuminance sensor, the black image and the marker image of the correction image 111a are projected, and the surrounding environment is determined to be bright. In such a case, the noise image and the marker image may be projected.

  With this configuration, when the surroundings are sufficiently dark, the effect of the pattern on the projection surface can be suppressed by projecting a black image, and when the surroundings are bright, the noise image is projected to project the pattern on the projection surface. It can be difficult to detect.

  The conditions for narrowing down the extracted first polygon 43 and second polygon 44 as candidates for the marker 41 include determination based on the ratio between the area S0 and the area S1, determination based on the line segment length, and line segment Any one or more of the determinations based on the inclination can be performed in combination.

  Further, in the present embodiment, the example in which the imaging device 20 includes the marker detection program 242 and executes the marker detection processing of FIG. 6 has been described, but the marker detection program 242 is stored in the projection device 10 and the imaging device. The projection apparatus 10 may execute the marker detection process of FIG. 6 by transmitting the captured image 60 captured by the 20 to the projection apparatus 10. In this case, the projection apparatus 10 can determine whether or not to project the next correction image 111.

  Further, in the present embodiment, the first polygon 43 is formed in a black frame shape and the inside is formed in white and the second polygon 44 is formed in black for the marker 41. One polygon 43 may be formed in a white frame shape with a black interior, and the second polygon 44 may be formed in white.

  Moreover, the projection apparatus 10 can also be configured to include a projection unit 13 as a projection unit and a projection control apparatus. Thereby, the projection system 1 can be comprised in various aspects.

  As described above, the projection control apparatus (projection apparatus 10) of the present embodiment includes the first correction image including the marker image for position detection and the non-marker image, and the measurement image for image correction. An acquisition unit that acquires a second correction image including a marker image, a projection control unit that controls a projection unit that can project the first correction image and the second correction image onto a projection target, and a projection unit A detection unit capable of detecting a marker image from the first correction image and the second correction image projected by the detection unit, and a detection result of the marker image of the first correction image detected by the detection unit. And a detection condition setting means for setting a detection condition for the marker image in the second correction image. This makes it possible to reliably detect the marker 41 of each correction image 111 while reducing the processing load when the marker 41 is detected for the first time in the first correction image, and to image the projection target other than the dedicated screen. Even when projecting, the initial setting can be easily executed.

  Further, the projection device 10 in which the non-marker image is a black image for color correction can perform color correction while facilitating the detection of the first marker 41 by capturing the first correction image. . Therefore, the processing load of the program can be reduced.

  In addition, the projection device 10 in which the image correction is color correction or shape correction can accurately correct an image for viewing using a plurality of correction images 111.

  Further, the marker image of the first correction image and the marker image of the second correction image are arranged at the same position, and the detection means detects the marker image of the first correction image. The projection apparatus 10 that performs the detection process of the marker image of the second correction image only needs to execute the marker detection process by narrowing down to a region where the marker 41 is likely to exist in advance, thereby reducing the processing load of the program. be able to.

  Further, when the detection unit does not detect the marker image of the second correction image, the projection device 10 that performs the marker image detection processing by enlarging the target region of the marker detection processing of the second correction image is the projection device The marker 10 can be detected from the second correction image without projecting the second correction image again.

  In addition, when the projection control unit does not detect the marker image of the first correction image by the detection unit, the projection control device controls the projection unit to project the correction image obtained by reversing the first correction image in black and white. Makes it possible to detect the marker 41 by the same determination process for the correction image 111 configured with different color schemes while facilitating the detection of the first marker 41.

  Further, the projection device 10 characterized in that the non-marker image is an image that does not match a part or all of the marker determination conditions, and can arrange various forms of images that are not detected as the marker 41 as dummy images, The possibility of erroneous detection during marker detection can be reduced.

  The embodiment described above is presented as an example, and is not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

The invention described in the first claim of the present application will be appended below.
[1] Acquisition means for acquiring a first correction image including a marker image for position detection and a non-marker image, and a second correction image including a measurement image for image correction and the marker image When,
A projection control means for controlling a projection means capable of projecting the first correction image and the second correction image onto a projection target;
Detection means capable of detecting the marker image from the captured images of the first correction image and the second correction image projected by the projection means;
Detection condition setting means for setting a detection condition for the marker image in the second correction image based on a detection result of the marker image of the first correction image detected by the detection means;
A projection control apparatus comprising:
[2] The above [1], wherein the non-marker image includes at least one of a blue noise image, a noise image having a pattern greatly different from the shape and size of the detected marker, and a plain single color image. The projection control apparatus described in 1.
[3] The projection control apparatus according to [1] or [2], wherein the non-marker image is a black image for color correction.
[4] The projection control apparatus according to any one of [1] to [3], wherein the image correction is color correction or shape correction.
[5] The marker image of the first correction image and the marker image of the second correction image are arranged at the same position,
The detection means performs a process of detecting the marker image of the second correction image for a region where the marker image of the first correction image is detected.
The projection control apparatus according to any one of [1] to [4] above, wherein
[6] When the marker image of the second correction image is not detected, the detection unit enlarges a target area of the marker detection process of the second correction image and performs the marker image detection process. The projection control apparatus according to any one of [1] to [5] above, wherein
[7] When the detection unit does not detect the marker image of the first correction image, the projection control unit projects the correction image obtained by reversing the first correction image in black and white. The projection control apparatus according to any one of [1] to [6], wherein the projection unit is controlled.
[8] The projection control apparatus according to any one of [1] to [7], wherein the non-marker image is an image that does not match a part or all of the marker determination conditions.
[9] The projection control device according to any one of [1] to [8],
The projection means;
A projection apparatus comprising:
[10] The acquisition unit acquires a marker image for position detection, a first correction image including a non-marker image, and a second correction image including a measurement image for image correction and the marker image. Acquisition process;
A projection control step of projecting the first correction image and the second correction image onto a projection target by a projection unit;
A detection step of detecting the marker image from the first correction image and the second correction image projected by the projection means;
A detection condition setting step of setting a detection condition of the marker image in the second correction image based on the detection result of the marker image of the first correction image detected in the detection step;
An image projection method for correction, comprising:
[11] A program executed by a computer, wherein the computer is
Acquisition means for acquiring a first correction image including a marker image for position detection and a non-marker image, and a second correction image including a measurement image for image correction and the marker image. ,
A projection control means for projecting the first correction image and the second correction image onto a projection target by a projection means;
Detection means for detecting the marker image from the first correction image and the second correction image projected by the projection means, and the marker of the first correction image detected by the detection means Detection condition setting means for setting a detection condition for the marker image in the second correction image based on the detection result of the image;
A program characterized by functioning as

DESCRIPTION OF SYMBOLS 1 Projection system 10 Projection apparatus 11 Memory | storage part 12 Processing part 13 Projection part 14 Operation part 15 Communication part 16 Audio | voice processing part 17 Speaker 20 Imaging device 21 Imaging part 22 Processing part 23 Input / output control part 24 Storage part 25 Communication part 25a Image correction Information 25b Projection instruction 30 Projection object 31 Projection surface 41 Marker 42 Measurement image 43 First polygon 44 Second polygon 51 Contour number 52 Number of lower layer contours 53 Lower layer contour number 60 (60a to 60f) Captured image 60a1 External area 61a Contour extraction image 111 Correction image 111a to 111f Correction image 112 Control program 121 Image correction processing unit 221 Measurement processing unit 231 Operation buttons 232 Display device 241 Contour information table 242 Marker detection program 243 Detection position information 431a to 431d 432a-4 2d edge 441a~441c edge 710 shooting range 711-715 contour 711a~711d segment 715a~715c segment 716 contours 717 outline 718 contour C0 contour C1 contour Rmax maximum area ratio Rmin minimum area ratio S0 area S1 area

Claims (11)

An acquisition means for acquiring a first correction image including a marker image for position detection and a non-marker image, and a second correction image including a measurement image for image correction and the marker image;
A projection control means for controlling a projection means capable of projecting the first correction image and the second correction image onto a projection target;
Detection means capable of detecting the marker image from the captured images of the first correction image and the second correction image projected by the projection means;
Detection condition setting means for setting a detection condition for the marker image in the second correction image based on a detection result of the marker image of the first correction image detected by the detection means;
A projection control apparatus comprising:   2. The projection according to claim 1, wherein the non-marker image includes at least one of a blue noise image, a noise image having a pattern greatly different from the shape and size of the detected marker, and a plain single color image. Control device.   The projection control apparatus according to claim 1, wherein the non-marker image is a black image for color correction.   The projection control apparatus according to claim 1, wherein the image correction is color correction or shape correction. The marker image of the first correction image and the marker image of the second correction image are arranged at the same position,
The detection means performs a process of detecting the marker image of the second correction image for a region where the marker image of the first correction image is detected.
The projection control apparatus according to claim 1, wherein the projection control apparatus is a projection control apparatus.   When the marker image of the second correction image is not detected, the detection unit performs the marker image detection process by enlarging a target area of the marker detection process of the second correction image. The projection control apparatus according to claim 1.   The projection control unit causes the projection unit to project a correction image obtained by reversing the first correction image in black and white when the detection unit does not detect the marker image of the first correction image. The projection control apparatus according to claim 1, wherein the projection control apparatus controls the projection control apparatus.   The projection control apparatus according to claim 1, wherein the non-marker image is an image that does not meet some or all of the marker determination conditions. A projection control device according to any one of claims 1 to 8,
The projection means;
A projection apparatus comprising: An acquisition step of acquiring, by an acquisition means, a marker image for position detection, a first correction image including a non-marker image, and a second correction image including a measurement image for image correction and the marker image; ,
A projection control step of projecting the first correction image and the second correction image onto a projection target by a projection unit;
A detection step of detecting the marker image from the first correction image and the second correction image projected by the projection means;
A detection condition setting step of setting a detection condition of the marker image in the second correction image based on the detection result of the marker image of the first correction image detected in the detection step;
An image projection method for correction, comprising: A program executed by a computer, wherein the computer is
Acquisition means for acquiring a first correction image including a marker image for position detection and a non-marker image, and a second correction image including a measurement image for image correction and the marker image. ,
A projection control means for projecting the first correction image and the second correction image onto a projection target by a projection means;
Detection means for detecting the marker image from the first correction image and the second correction image projected by the projection means, and the marker of the first correction image detected by the detection means Detection condition setting means for setting a detection condition for the marker image in the second correction image based on the detection result of the image;
A program characterized by functioning as

Images