JP2019204034A - Projection control device, control method thereof, projection system, program and storage medium - Google Patents

Abstract

An object of the present invention is to provide a projection control device that enables a user to easily adjust the projection positions of a plurality of projection devices. A projection control device 200 of the present invention controls an individual adjustment mode for controlling a projection area of one of a plurality of projectors 100 and a projection area of two or more projectors 100 among the plurality of projectors 100. The collective adjustment mode is set, and any one of a plurality of adjustment modes including the adjustment mode is set. Further, when the control of the projection area of one projector is executed in the individual adjustment mode before the collective adjustment mode is set, the CPU 201 uses the projective transformation matrix used to control the projection areas of two or more projectors. It is characterized by correcting. [Selection diagram]

Description

  The present invention relates to a projection control apparatus that controls the projection positions of a plurality of projection apparatuses, a control method therefor, a projection system including the projection apparatus and the projection control apparatus, a program for realizing projection control, and a memory that stores the program It relates to the medium.

  There is a projection method in which projection images projected by a plurality of projectors (projectors) are combined on a projection plane to display an image. When projecting an image by such a projection method, it is necessary to adjust the projection position of each projection apparatus in advance.

  Patent Document 1 discloses a display system that performs tiling projection using a plurality of projectors connected to a network. In this display system, an image in which group information and identification information of each of a plurality of selectable projectors connected to a network are associated is displayed. This makes it easy for the user to distinguish a projector for which group information has been set in advance for use in tiling projection from other projectors connected to the network.

  Among the methods of projecting images with a plurality of projection devices, in the stack projection in which the projection images of the projection devices are displayed in an overlapping manner, the projection positions of the projection devices are adjusted in advance so that the projection positions of the plurality of projection devices overlap. There is a need. In order to adjust the projection position of each projection apparatus, there is a control application for capturing a projection image of each projection apparatus and automatically adjusting the projection position. When the projection position is controlled by such an automatic control application, a slight shift may occur in the projection position due to an external factor during or after the adjustment.

JP 2017-26832 A

  In the conventional control application, after the projection position is controlled by the automatic control application, the adjustment of the projection position may be manually performed individually. In this case, it is complicated to select and operate each of the projection apparatuses whose projection positions have already been adjusted. Further, if the projection position is moved with respect to a combination of projection apparatuses that do not require adjustment, the deviation of the projection position may be further increased.

  In view of the above-described problems, an object of the projection control apparatus of the present invention is to provide a projection control apparatus that allows a user to easily adjust the projection positions of a plurality of projection apparatuses.

  One aspect of the projection control apparatus of the present invention is a projection control apparatus that controls the position or shape of the projection areas of a plurality of projection apparatuses, and controls the projection area of one of the plurality of projection apparatuses. A setting unit for setting an adjustment mode including a first adjustment mode and a second adjustment mode for controlling a projection area of two or more projection devices among the plurality of projection devices; and an adjustment mode set by the setting unit. And adjusting means for adjusting the position or shape of the projection area of the projection apparatus, wherein the setting means sets the second adjustment mode, and the setting means sets the second adjustment mode. When the control of the projection area of the one projection apparatus has been executed in the first adjustment mode before, the adjustment means corrects the parameter used for the control of the projection area of the two or more projection apparatuses. And butterflies.

  According to the projection control apparatus of the present invention, when the position adjustment of the projection positions of a plurality of projection apparatuses is performed, the subsequent adjustment can be correctly performed even when a deviation occurs. Therefore, the user can easily adjust the projection positions of the plurality of projection apparatuses.

It is a figure which shows an example of a projection system structure. It is a block diagram which shows the functional block of a projector and a projection control apparatus. It is a flowchart which shows a projection area adjustment flow. It is a 1st schematic diagram which shows GUI displayed on a display part. It is a 2nd schematic diagram which shows GUI displayed on a display part. It is a 3rd schematic diagram which shows GUI displayed on a display part. It is a 4th schematic diagram which shows GUI displayed on a display part. It is a flowchart which shows an automatic alignment process. It is a schematic diagram which shows the coordinate of a captured image, and the panel coordinate of a projector. It is a flowchart which shows the projection position correction process after a position alignment process. It is a schematic diagram which shows the marker projected on a projection surface. It is a 5th schematic diagram which shows GUI displayed on a display part. It is a schematic diagram which shows the projective transformation in keystone correction.

  Hereinafter, examples of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiments. The embodiments of the present invention show preferred modes of the invention and do not limit the scope of the invention.

  FIG. 1 is a diagram illustrating an example of a configuration of a projection system according to the present embodiment. The projection system includes projectors 100a, 100b, 100c, and 100d, a projection control device 200, a video distributor 300, and an imaging device 400. The projection system is a multi-projection system that projects one image by combining the projection images projected from the projectors 100a, 100b, 100c, and 100d.

  The projector 100 a is a projection device that projects the projection image A onto the projection plane 500. The projector 100 b is a projection device that projects the projection image B onto the projection plane 500. The projector 100 c is a projection device that projects the projection image C onto the projection plane 500. The projector 100d is a projection device that projects the projection image D onto the projection plane 500. Hereinafter, when individually referring to each projector, the reference numerals of the projector 100a, the projector 100b, the projector 100c, and the projector 100d are given respectively. Further, when a configuration or process common to each projector is mentioned, the projector 100 is shown.

  Assume that the projectors 100a to 100d are installed in advance to perform stack projection in which the projection images A to D are projected on the projection surface 500 in an overlapping manner. However, it is difficult to adjust the position of the projection area of each projector in units of pixels by checking the position of the projection area visually by the user and adjusting the installation position manually by the user. In this embodiment, the position control of the projection area for aligning the projection area of each projector in units of pixels is executed by the alignment process by the projection control apparatus 200.

  The projection control apparatus 200 is assumed to be a control computer that is connected to the projectors 100a to 100d via a network and controls the projectors 100a to 100d. The projection control apparatus 200 is assumed to be a notebook personal computer (PC) including a display unit and an operation unit. The projector included in the projection system is connected to the projection control apparatus 200 so that they can communicate with each other. In this embodiment, it is assumed that communication between the projector 100 and the projection control apparatus 200 is performed in a local area network (LAN) using TCP / IP as a communication protocol.

  Note that the projection control apparatus 200 may use another information processing apparatus such as a smartphone or a tablet. Communication between the projector 100 and the projection control apparatus 200 may be wired communication or wireless communication, and the communication protocol is not particularly limited. Projection control apparatus 200 can control operations of projectors 100a to 100d by transmitting predetermined commands to projectors 100a to 100d. The projectors 100 a to 100 d perform an operation according to the command received from the projection control apparatus 200 and transmit the operation result to the projection control apparatus 200.

  The video distributor 300 duplicates the video signal received from the projection control apparatus 200 and supplies it to the projectors 100a to 100d.

  The imaging apparatus 400 is a camera that captures an area of the projection plane 500 including the projection images A to D projected by the projectors 100 a to 100 d and outputs the captured image to the projection control apparatus 200. The imaging device 400 is, for example, a digital camera, a web camera, or a network camera. Alternatively, the imaging apparatus 400 may be built in the projection control apparatus 200. When the imaging device 400 is a device different from the projection control device 200 (an imaging device external to the projection control device 200), the imaging device 400 is connected to the projection control device 200 so as to be communicable directly or via a network. . The projection control apparatus 200 can control the operation of the imaging apparatus 400 by transmitting a predetermined command to the imaging apparatus 400. For example, the imaging apparatus 400 can capture an image of the projection plane 500 in response to a request from the projection control apparatus 200 and transmit the obtained image data (captured image) to the projection control apparatus 200.

  FIG. 2 is a block diagram showing functional blocks of the projector 100 and the projection control apparatus 200 included in the projection system. The projector 100 includes a CPU 101, a RAM 102, a ROM 103, a projection unit 104, a projection control unit 105, a VRAM 106, an operation unit 107, a network IF 108, an image processing unit 109, and a video input unit 110. These functional blocks are communicably connected via an internal bus 111.

  A CPU (Central Processing Unit) 101 is a control processor that controls each functional block of the projector 100. The CPU 101 is assumed to be a programmable processor that realizes the operation of the projector 100 by reading a program stored in the ROM 103 into the RAM 102 and executing the program.

  The RAM 102 is a storage medium used as a work memory when the CPU 101 executes a program. The RAM 102 stores a program and variables used for executing the program. The RAM 102 may be used for other purposes (for example, as a data buffer).

  The ROM 103 is a storage medium in which various setting values such as a program executed by the CPU 101 and GUI data used for displaying a menu screen are stored. The ROM 103 may be rewritable.

  The projection unit 104 includes a light source, a projection optical system, and the like, and projects an optical image based on the projection image supplied from the projection control unit 105. In this embodiment, a liquid crystal panel is used as an optical modulation element, and an optical image based on the projection image is generated by controlling the reflectance or transmittance of light from the light source according to the projection image. Project to.

  The projection control unit 105 supplies the projection image data supplied from the image processing unit 109 to the projection unit 104.

  The VRAM 106 is a video memory that stores projection image data received from the outside (for example, a PC or a media player).

  The operation unit 107 has input devices such as key buttons, switches, and a touch panel, and receives instructions from the user to the projector 100. The CPU 101 monitors the operation of the operation unit 107. When the operation of the operation unit 107 is detected, the CPU 101 executes processing according to the detected operation. When the projector 100 has a remote controller, the operation unit 107 notifies the CPU 101 of an operation signal received from the remote controller.

  A network IF 108 is an interface for connecting the projector 100 to a communication network, and has a configuration conforming to the standard of the supported communication network. In the present embodiment, the projector 100 is connected to a common local network with the projection control apparatus 200 through the network IF 108. Therefore, communication between the projector 100 and the projection control apparatus 200 is executed through the network IF 108.

  The image processing unit 109 applies various image processing to the video signal supplied to the video input unit 110 and stored in the VRAM 106 as necessary, and supplies the image signal to the projection control unit 105. The image processing unit 109 may be a microprocessor for image processing, for example. Alternatively, a function corresponding to the image processing unit 109 may be realized by the CPU 101 executing a program stored in the ROM 103.

  Image processing that can be applied by the image processing unit 109 includes frame thinning processing, frame interpolation processing, resolution conversion processing, OSD overlap processing such as a menu screen, keystone correction processing, and edge blending processing. It is not limited to.

  The video input unit 110 is an interface that directly or indirectly receives a video signal output from an external device (in this embodiment, the projection control device 200), and has a configuration corresponding to the video signal to be supported. The video input unit 110 includes, for example, one or more of a composite terminal, an S video terminal, a D terminal, a component terminal, an analog RGB terminal, a DVI-I terminal, a DVI-D terminal, and an HDMI (registered trademark) terminal. When receiving an analog video signal, the video input unit 110 converts the analog video signal into a digital video signal and stores the digital video signal in the VRAM 106.

  Next, the functional configuration of the projection control apparatus 200 will be described. The projection control apparatus 200 may be a general-purpose computer to which an external display can be connected, and thus has a functional configuration according to the general-purpose computer. The projection control apparatus 200 includes a CPU 201, a RAM 202, a ROM 203, an operation unit 204, a display unit 205, a network IF 206, a video output unit 207, and a communication unit 208. These functional blocks are communicably connected via an internal bus 209.

  The CPU 201 is an example of a programmable processor, and implements the operation of the projection control apparatus 200 by, for example, reading a program (OS or application program) stored in the ROM 203 into the RAM 202 and executing it.

  The RAM 202 is used as a work memory when the CPU 201 executes a program. The RAM 202 stores a program and variables used for executing the program. The RAM 202 may be used for other purposes (for example, as a data buffer).

  The ROM 203 may be rewritable. The ROM 203 stores programs executed by the CPU 201, GUI data used for displaying menu screens, various setting values, and the like. Note that the projection control apparatus 200 may have a storage device (HDD or SSD) having a larger capacity than the ROM 203, and in this case, a program having a large capacity such as an OS or an application program may be stored in the storage device.

  The operation unit 204 includes input devices such as a keyboard, a pointing device (such as a mouse), a touch panel, and a switch, and receives instructions from the user to the projection control apparatus 200. The keyboard may be a software keyboard. The CPU 201 monitors the operation of the operation unit 204. When the operation of the operation unit 204 is detected, processing according to the detected operation is executed.

  The display unit 205 is, for example, a liquid crystal panel or an organic EL panel. The display unit 205 displays a screen provided by the OS or application program. The display unit 205 may be an external device. The display unit 205 may be a touch display.

  A network IF 206 is an interface for connecting the projection control apparatus 200 to a communication network, and has a configuration conforming to the standard of the supported communication network. In the present embodiment, the projection control apparatus 200 is connected to a local network common to the projector 100 through the network IF 206. Therefore, communication between the projection control apparatus 200 and the projector 100 is executed through the network IF 206.

  The video output unit 207 is an interface that transmits a video signal to an external device (in this embodiment, the projector 100 or the video distributor 300), and has a configuration corresponding to the video signal to be supported. The video output unit 207 includes, for example, one or more of a composite terminal, an S video terminal, a D terminal, a component terminal, an analog RGB terminal, a DVI-I terminal, a DVI-D terminal, and an HDMI (registered trademark) terminal.

  In this embodiment, the UI screen of the projection control application program having the projection area adjustment function of the projector 100 is displayed on the display unit 205. However, the UI screen is displayed on an external device connected to the video output unit 207. It may be displayed.

  The communication unit 208 is a communication interface for performing serial communication with an external device, for example, and is typically a USB interface, but may have a configuration conforming to another standard such as RS-232C. In this embodiment, it is assumed that the imaging apparatus 400 is connected to the communication unit 208. However, there is no particular limitation on the communication method between the imaging apparatus 400 and the projection control apparatus 200, which conforms to any standard supported by both. Communication can be performed.

  Next, keystone correction will be described. Keystone correction is a geometric transformation of the original image so as to cancel the trapezoidal distortion that occurs in the projected image according to the deviation between the normal direction of the projection plane and the projection direction (generally the optical axis of the projection optical system). This is correction (geometric correction) to be (deformed). The keystone correction is executed by the CPU 101 of the projector 100 in accordance with the input instruction. Since geometric transformation of an image can be realized by projective transformation, performing keystone correction is equivalent to determining a projection transformation parameter (control parameter) that is a correction amount of geometric correction. For example, the CPU 101 can determine projection transformation parameters based on the movement amount and movement direction of each vertex of the original image on the rectangle, and can provide the parameters to the image processing unit 109.

  FIG. 13 is a schematic diagram showing projective transformation in keystone correction. For example, assuming that the coordinates of the original image are (xs, ys), the coordinates (xd, yd) of the image after deformation by projective transformation are expressed by the following Equation 1.

  Here, M is a 3 × 3 matrix, which is a projective transformation matrix from the original image to the transformed image. Further, xso and yso are the coordinates of the upper left vertex of the original image indicated by a solid line in FIG. ) Is the coordinate value of the vertex corresponding to.

The CPU 101 supplies the matrix M of Expression 1 and its inverse matrix M ⁻¹ to the image processing unit 109 as offset parameters (xso, yso) and (xdo, ydo) as parameters for keystone correction. The image processing unit 109 can obtain the coordinates (xs, ys) of the original image corresponding to the coordinate values (xd, yd) after the keystone correction according to the following Expression 2.

  If the coordinates xs and ys of the original image obtained by Expression 2 are both integers, the image processing unit 109 uses the pixel values of the original image coordinates (xs, ys) as they are as the coordinates (xd, yd) of the image after keystone correction. ) Pixel value. On the other hand, when the coordinates of the original image obtained by Expression 2 are not integers, the image processing unit 109 calculates the pixel value corresponding to the original image coordinates (xs, ys) by interpolation using the values of a plurality of surrounding pixels. Can be sought. The interpolation calculation can be performed using any of known interpolation calculations such as bilinear and bicubic. When the coordinates of the original image obtained by Expression 2 are the coordinates of the external area of the original image, the image processing unit 109 sets the pixel value of the coordinates (xd, yd) of the image after keystone correction to black (0 ) Or the background color set by the user. In this way, the image processing unit 109 can obtain pixel values for all coordinates of the image after the keystone correction, and create a converted image.

Here, it is assumed that both the matrix M and its inverse matrix M ⁻¹ are supplied from the CPU 101 of the projector 100 to the image processing unit 109, but only one of the matrices is supplied, and the other matrix is image processing. The part 109 may obtain it.

  The coordinates of the vertices of the image after the keystone correction are obtained, for example, by causing the user to input a movement amount through the operation unit 107 so that the vertices are projected at a desired position for each vertex of the projection image. Can do. At this time, in order to support the input of the movement amount, the CPU 201 may cause the projector 100 to project a test pattern using the function of the projection control application program.

  Therefore, the projector 100 can move or deform the shape of the projection region to a desired position by obtaining the keystone deformation amount.

  Next, a projection region adjustment flow executed by the projection control apparatus 200 will be described. When the projection control apparatus 200 executes the projection control application program, a flow for adjusting the position of the projection area of the projection apparatus is executed. FIG. 3 is a flowchart showing a projection area adjustment flow.

  In step S <b> 301, the CPU 201 of the projection control apparatus 200 executes target projector selection processing for selecting a plurality of projectors that are targets of projection area adjustment from the projectors 100 with which the projection control apparatus 200 can communicate. The target projector selection process includes a process of detecting a connected projector connected to the projection control apparatus 200 via a network, a process of displaying the connected projector, and a process of determining the target projector.

  FIG. 4 is a schematic diagram illustrating an example of an operation screen (Graphical User Interface, GUI) 610 displayed on the display unit 205 by the CPU 201 during the execution of the processes of S301 to S303. The user can control the execution of the above-described processing by operating the GUI 610 displayed on the display unit 205 via the operation unit 204. The GUI 610 includes buttons 611a to 611d, a list 612, and check boxes 613a to 613d.

  First, the CPU 201 executes processing for detecting a connected projector connected to the projection control apparatus 200 via a network. The button 611a is a GUI for instructing the start of execution of processing for detecting a connected projector connected to the projection control apparatus 200 via a network. When the CPU 201 detects that the button 611 a has been pressed via the operation unit 204, the CPU 201 requests identification information for identifying each projector from each projector via the network IF 206. Specifically, a command for requesting identification information is broadcast on the network using a protocol such as UDP. In addition, you may use protocols other than UDP for acquisition of identification information. For example, the identification information is information indicating at least one of a projector name and an IP address. The identification information may include information indicating the keystone deformation amount.

  When the CPU 101 of the projector 100 connected to the network receives a command via the network IF 108, the CPU 101 transmits data including identification information for identifying the projector itself to the projection control apparatus 200. The CPU 201 of the projection control apparatus 200 receives the identification information transmitted from the projector 100 in response to the command.

  Next, the CPU 201 executes processing for displaying the connected projector. Based on the acquired identification information, the CPU 201 displays information indicating the projector of the projector 100 connected to the projection control apparatus 200 via the network on the display unit 205. A list 612 is a list view that shows at least one of information indicating a projector name and an IP address of the projector 100 connected to the projection control apparatus 200 via a network in a list format. It should be noted that the order of projectors displayed in list view 402 may be the order in which they are detected, or may be sorted based on specific rules.

  Next, the CPU 201 executes processing for determining a target projector from the projectors displayed in the list.

  List 612 includes a check box 613 for selecting each projector. As shown in FIG. 4, if four projectors 100a to 100d are connected to the projection control apparatus 200 via a network, check boxes 613a to 613d are displayed next to the identification information of each projector in the list 612. The The user selects a projector (target projector) that is a target for adjusting the projection area by checking the check boxes 613a to 613d. The target projector may be a projector that superimposes projection areas when performing stack projection.

  For example, it is assumed that the target projector is selected as the projector 100a (Projector 1, 192.168.254.251) and the projector 100b (Projector 2, 192.168.254.252). In this case, the user only has to check the check box 613b and the check box 613d of the GUI 610. Information (for example, projector name, IP address, etc.) regarding the projector whose check box is checked is stored in the RAM 202 of the projection control apparatus 200. Note that a projector whose projection area can be easily adjusted may be selected as the target projector by projecting a test pattern described later. In this embodiment, it is assumed that the projector 100a and the projector 100b are selected as target projectors. The process proceeds to S302.

  In step S302, control is performed so that each target projector projects a test pattern. The button 611b is a GUI for instructing the target projector to start projecting a test pattern. The button 611c is a GUI for instructing the target projector to stop the projection of the test pattern.

  When the CPU 201 detects that the button 611b has been pressed via the operation unit 204, the CPU 201 transmits a command for instructing display of the test pattern to each of the target projectors via the network IF 206. The test pattern is a test pattern for easily confirming the size and position of the display area of each projector 100. For example, it is assumed that the test pattern is a grid image. The test pattern may be transmitted from the projection control apparatus 200 in association with a command for instructing display of a predetermined test pattern to the target projector, or a plurality of commands for causing the projector to draw an arbitrary straight line, figure, character string, or the like. You may transmit in combination.

  When the CPU 201 detects that the button 611c is pressed, the CPU 201 transmits a command for instructing to stop the projection of the test pattern to each of the target projectors through the network IF 206.

  The button 611d is a GUI for inputting an instruction to shift the process from S302 to S303. When the CPU 201 detects that the button 611d has been pressed via the operation unit 204, the process proceeds to S303. At this time, it is desirable that the process proceeds while the target projector is projecting the test pattern.

  In step S <b> 303, the CPU 201 executes processing for selecting the imaging device 400 used for the alignment processing. In step S <b> 303, the CPU 201 selects the imaging device 400 used for the alignment process in accordance with a user input. FIG. 5 is a schematic diagram showing a GUI 620 that the CPU 201 displays on the display unit 205 during the execution of the process of S303. The GUI 620 includes buttons 621 a to 621 c, a drop-down list 622, a display area 623, and a check box 624.

  When the CPU 201 detects that the button 621 a has been pressed via the operation unit 204, the CPU 201 of the projection control device 200 displays information indicating the imaging device connected via the communication unit 208 or the network IF 206 (for example, the product name of the camera). ) To get. Then, the CPU 201 displays the acquired information in the drop-down list 622. The user selects one of the plurality of cameras displayed in the drop-down list 622 as the imaging device 400 used for the alignment process. Note that a plurality of imaging devices 400 may be selected.

  The display area 623 is an area for displaying an image captured by the camera (imaging device 400) selected from the drop-down list 622. The CPU 201 of the projection control apparatus 200 transmits a command for instructing shooting to the imaging apparatus 400 selected, and displays an image in the display area 623 based on the acquired captured image. Assume that the image displayed in the display area 623 is a live view image of the selected imaging device 400. A still image based on a captured image captured by the image capturing apparatus 400 at a predetermined timing may be used. The user can confirm whether or not all the projection images (test patterns) projected by the target projector are contained in the captured image shown in the display area 623. The user performs optical adjustments such as the position and angle of the imaging apparatus 400 and zoom so that the projected image projected by the target projector is included in the captured image.

  A check box 624 is a GUI for setting whether or not to execute an imaging parameter automatic setting process in which the projection control apparatus 200 automatically calculates imaging parameters (aperture value, shutter speed, etc.) of the imaging apparatus 400. By checking the check box 624, it is set to execute the automatic imaging parameter setting processing.

  The button 621b is a GUI for inputting an instruction for returning the process from S303 to S301. When the CPU 201 detects that the button 621b has been pressed via the operation unit 204, the process returns to S301, and the GUI 610 shown in FIG.

  The button 621c is a GUI for inputting an instruction to complete S303. When the CPU 201 detects that the button 621b has been pressed via the operation unit 204, the process proceeds to S304.

  In S304, it is determined whether or not there is an instruction to execute the automatic setting process. The CPU 201 determines whether or not there is an instruction to execute the automatic setting process according to the instruction input in the check box 624. If there is no instruction to execute the automatic setting process, the process proceeds to S305. Otherwise, the process proceeds to S310.

  In step S <b> 305, the CPU 201 executes imaging parameter setting processing of the imaging apparatus 400. The imaging parameters are, for example, shutter speed, ISO sensitivity, and aperture value. FIG. 6 is a schematic diagram showing a GUI 630 that the CPU 201 displays on the display unit 205 during the execution of the process of S305. The GUI 630 includes an imaging parameter setting area 631, drop-down lists 632a to 632c, a display area 633, and buttons 634a to 634c.

  The imaging parameter setting area 631 is an area for displaying a GUI for setting imaging parameters. The imaging parameter setting area 631 includes drop-down lists 632a to 632c for setting the shutter speed, ISO sensitivity, and aperture value of the imaging apparatus 400, respectively. When the user selects a setting value of each imaging parameter from the drop-down lists 632a to 632c, the imaging parameter of the imaging apparatus 400 is set. Note that the camera parameters that can be set are not limited to this, and for example, white balance, photometry method, and the like may be set.

  The button 634 a is a GUI for inputting an instruction for outputting an imaging instruction to the imaging apparatus 400. When the CPU 201 detects that the button 634 a has been pressed via the operation unit 204, an imaging instruction is output to the imaging device 400. The imaging apparatus 400 captures the projection plane 500 according to the imaging instruction and outputs the acquired image data (captured image) to the projection control apparatus 200.

  The display area 633 is a display area for displaying an image based on the captured image acquired from the imaging apparatus 400. The captured image acquired at this time and the image to be displayed are for confirming whether or not the imaging parameters such as the shutter speed have been set correctly, and are preferably still images, but may be live view images. Absent.

  The button 634b is a GUI for inputting an instruction for returning the processing from S305 to S303. When the CPU 201 detects that the button 621b has been pressed via the operation unit 204, the process returns to S301, and the GUI 610 shown in FIG.

  The button 634c is a GUI for inputting an instruction to complete S305. When the CPU 201 detects that the button 634c has been pressed via the operation unit 204, the process proceeds to S306.

  S310 is an automatic setting process in which the imaging parameter execution process executed by the user's GUI operation in S304 is automatically executed under the control of the CPU 201. In step S310, the CPU 201 transmits an instruction to control the target projectors one by one so as to project a test pattern. In addition, the CPU 201 causes the imaging device 400 to capture an image in a state where one of the target projectors is projecting a test pattern. The CPU 201 analyzes the obtained captured image, and determines whether a sufficient amount of light is obtained and the focus position is present. The CPU 201 controls the imaging parameter according to the determination result, and repeats the above-described imaging and determination to set the optimal imaging parameter. In step S <b> 310, the process proceeds to step S <b> 306 in response to the completion of the above-described imaging parameter setting for all target projectors.

  S306 is a process of selecting a method (adjustment mode) of the alignment process. FIG. 7 is a schematic diagram showing a GUI 640 that the CPU 201 displays on the display unit 205 during the execution of the process of S306. The GUI 640 includes a radio button 641, a 4-point designation GUI 642, a radio button 643, a drop-down list 644, a button 645, and buttons 646a and 646b.

  In S306, the user performs a four-point designated adjustment mode in which the four corners of the projection area are aligned with the designated four points, and a reference PJ matching mode in which the projection area of another target projector is aligned with the projection area of the set reference projector. Select one of them. The user selects the adjustment mode by selecting one of the radio button 641 and the radio button 643 of the GUI 640. Information regarding the adjustment mode selected by the user is stored in its own RAM 202 by the CPU 201 of the projection control apparatus 200.

  When the user checks the radio button 641, the 4-point designated adjustment mode is selected. In the 4-point designation adjustment mode, control parameters such as keystone correction are automatically determined for the projection areas of the target projectors so that the vertices of the individual projection areas are aligned with four predetermined points. The four-point designation adjustment is useful when the projection target position is clear, for example, when the projection surface is a screen with a frame. Note that the number of points whose coordinates can be adjusted may be less than 4, or may be 5 points or more including coordinates other than the vertexes.

  The four-point designation GUI 642 is a GUI for the user to designate four points that match the vertices of the projection area in the four-point start point adjustment mode. For example, the user designates four points by moving the cursor displayed in the four-point designation GUI 642.

  When the user checks the radio button 643, the reference PJ conforming mode is selected. In the reference PJ conformity mode, control parameters such as keystone correction for the projection area of the other target projector are automatically set so that the projection area of the other target projector matches the projection area of the reference projector set among the target projectors. This mode is determined by. In this mode, the automatic alignment is executed when the position of the projection area of the reference projector is adjusted to the designated position. A keystone correction amount for automatically matching the projection area of a projector other than the reference projector with the projection area of the reference projector is determined. This is an effective function when the projection target position is not clear (for example, projection onto a wall surface, etc.) unlike the four-point designation adjustment mode.

  The drop-down list 644 is for selecting a reference projector. The projector that can be selected here is the target projector selected in S301. The user selects one desired projector from the drop-down list, and the CPU 201 of the projection control apparatus 200 stores the selected projector in its own RAM 202 as the reference projector.

  The button 645 is a GUI for inputting an instruction to project a specific test pattern to the selected reference projector. When the CPU 201 detects that the button 645 has been pressed via the operation unit 204, the CPU 201 transmits a command for displaying a specific test pattern to each of the target projectors via the network IF 206. At this time, the user can easily confirm which projector on the projection plane is the reference projector by displaying a test pattern that is different in color, brightness, or shape from the other target projectors only in the reference projector. .

  The button 646a is a GUI for inputting an instruction for returning the processing from S305 to S304. When the CPU 201 detects that the button 646a has been pressed via the operation unit 204, the process returns to S304, and the GUI 610 shown in FIG.

  The button 646b is a GUI for inputting an instruction to complete S306 and determine the adjustment mode. When the CPU 201 detects that the button 646b has been pressed via the operation unit 204, the process proceeds to S307.

  In step S <b> 307, the CPU 201 executes the alignment process of the projection area of the target projector according to the selected adjustment mode. FIG. 8 is a flowchart showing the automatic alignment process.

  In the alignment processing, the coordinate system of the liquid crystal panel of each target projector and the projective transformation matrix of the imaging device 400 camera are calculated by the processing of S801 to S804, respectively, and stored in its own RAM 202.

  In step S <b> 801, the CPU 201 determines whether a projection transformation matrix to be described later has been calculated for all target projectors. If projective transformation matrices for all target projectors have not been calculated, the process proceeds to S802.

  In step S802, the CPU 201 outputs an instruction via the network IF 206 to display a test pattern on any of the target projectors (for example, the projector 100a). In step S <b> 803, the CPU 201 outputs a shooting instruction to the imaging apparatus 400 via the communication unit 208 and acquires a captured image from the imaging apparatus 400.

  In step S804, the CPU 201 detects the coordinates of the four corner points (feature points) of the test pattern projected by the target projector 100a from the captured image obtained in step S803, and the projector 100a and the imaging device 400 determine the coordinates from the obtained coordinates. A projective transformation matrix indicating the correspondence is calculated. As a method for detecting a feature point of a captured image, a known technique such as feature point detection in an image can be used.

  FIG. 9 is a schematic diagram showing the coordinates of the captured image and the panel coordinates of the projector. The projective transformation matrix calculation process in S804 will be described with reference to FIG. FIG. 9A is a schematic diagram illustrating a coordinate system (camera coordinate plane) of a captured image acquired from the imaging apparatus 400. FIG. 9B is a schematic diagram showing the panel coordinate system (projector coordinate plane) of the target projector displaying the test pattern in S802. Assuming that the coordinates on the projector coordinate plane are (xi, yi) and the coordinates on the camera coordinate plane are (Xi, Yi), the projective transformation equations are expressed by equations 3 and 4 (i is a natural number). At this time, the variables having the same i correspond to each other.

  Here, a to h are predetermined constants.

  Expression 5 is obtained by modifying Expression 3 and Expression 4 above and expressing them in a matrix.

  M at this time is a projective transformation matrix. This projective transformation matrix has four corresponding points (x1, y1, X1, Y1), (x2, y2, X2, Y2), (x3, y3, X3, Y3) with respect to Equations 3 and 4. , (X4, y4, X4, Y4) can be respectively calculated. That is, if a correspondence relationship between at least four coordinates of planes is known, a projective transformation matrix can be calculated.

  For example, in S802, the selected target projector projects a square test pattern (shaded portion in FIG. 9B). At this time, a projective transformation matrix is calculated based on the correspondence between the coordinates of the four vertices of the projected image (rectangle) in the captured image and the coordinates of the four vertices of the quadrant of the test pattern on the panel of the target projector. It becomes possible. P_a, P_b, P_c, and P_d on the camera coordinate plane in FIG. 9A correspond to p_a, p_b, p_c, and p_d on the projector coordinate plane in FIG. 9B, respectively. A projective transformation matrix is calculated from the correspondence of these four known points.

  By using the obtained projective transformation matrix, it is possible to perform projective transformation of unknown coordinate points. For example, the point p_e on the projector coordinate plane can be calculated by multiplying the point P_e on the camera coordinate plane by the calculated projective transformation matrix. Further, by multiplying p_e on the projector coordinate plane by the inverse matrix of the calculated projective transformation matrix, the corresponding point P_e on the camera coordinate plane can be calculated.

  Although the projective transformation matrix can be calculated by the method described above, the image projected by the projector when used for the calculation need not be a quadrangle. Any image may be used as long as it can obtain the correspondence of at least four coordinates between the camera coordinate plane and the projector coordinate plane.

  If the projection transformation matrices of all target projectors have been calculated in S801, the process proceeds to S805.

  In step S805, the CPU 201 determines whether or not the adjustment mode designated by the user is the four-point designated adjustment mode. If it is determined in S805 that the adjustment mode is the 4-point designated adjustment mode, the process proceeds to S806.

  In step S806, the CPU 201 executes a projection shape designation process for determining four points (designated points) for designating the post-deformation shape of the projection region. Details of the processing will be described later. The process proceeds to S807.

  In step S <b> 807, the CPU 201 determines control parameters for the projection areas of the target projectors so that the projection areas of the target projectors overlap on the projection plane. For example, it is assumed that the control parameter is a keystone deformation amount. The CPU 201 uses the projective transformation matrix of each target projector stored in the RAM 202 to calculate the coordinates (corresponding designated points) on the projector coordinate plane of each target projector corresponding to the coordinates of the markers (designated points) in the captured image. To do. Specifically, the CPU 201 determines the keystone deformation amount of each target projector so that the four corners of the projection area of each target projector are located at the corresponding designated points. The process proceeds to S808.

  If it is determined in S805 that the adjustment mode is not the 4-point designated adjustment mode, it means that the adjustment mode is the reference PJ adjustment mode. In this case, the process proceeds to S809.

  In step S809, the CPU 201 determines a control parameter for the projection area of another target projector based on the position of the projection area of the reference projector set in step S306. Specifically, the keystone deformation amount of each target projector is determined based on the positions of the four corners on the camera coordinate plane of the reference projector photographed in S803, the positions of the target projectors other than the reference projector, and the projective transformation matrix. To do. The process proceeds to S808.

  In step S808, the CPU 201 transmits the keystone deformation amount calculated in step S807 to each target projector via the network IF 206. The image processing unit 109 of each target projector executes input image shape correction based on the acquired keystone deformation amount. That is, S808 is a process of controlling the position or shape of the projection area of each target projector based on the determined control parameter.

  Through the above process, the alignment process for superimposing the projection area of the selected target projector is autonomously executed by the CPU 201 of the projection control apparatus 200.

<Characteristic operation of the present invention>
The projection shape designation process in S806 will be described. FIG. 10 is a flowchart of the projection shape designation process.

  In step S1101, the CPU 201 executes projection control for generating a marker for designating the deformed shape and projecting the marker on the projector. FIG. 11 is a schematic diagram showing a marker to be projected onto the projection plane. The marker is an L-shaped marker 1202 to 1205 in which the L-shaped marker indicates the apex of the deformed shape, and a line segment 1201 connecting the L-shaped markers 1202 to 1205. The shape and color of the marker may be any as long as it can specify at least four coordinates on the projection plane.

  As a marker for designating the deformed shape, an image generated by the CPU 201 may be transmitted via the network IF 206 or the video output unit 207. Alternatively, the CPU 201 may transmit a command that causes each target projector 100 (projector 100a and projector 100b) to draw an arbitrary line segment or polygon via the network IF 206. The CPU 101 of each target projector 100 may draw a marker image based on the received command. The markers displayed on the projectors 100a and 100b may be configured to draw in different colors. For example, when a red marker is drawn by the projector 100a and a green marker is drawn by the projector 100b, a region where the markers overlap is displayed in yellow. By drawing with different colors for each projector, it is possible to easily determine whether the markers overlap or are shifted. In the example of FIG. 11, the markers of the projectors 100a and 100b are shown to be completely overlapped to make the drawing easier to see, but this is different when the value of the projective transformation matrix in S904 is shifted due to shaking of the screen. A marker is drawn at the position. When the marker display is completed in S1101, the process proceeds to S1102.

  In step S1102, the CPU 201 controls the display unit 205 to display a GUI screen for designating the deformed shape. FIG. 12 is a schematic diagram illustrating a GUI 710 that is displayed on the display unit 205 by the CPU 201 during the execution of the process of S1102. The GUI 710 includes operation buttons 711 a to 711 d, a display area 712, a list 713, a check box 714, and an execution button 715.

  The operation buttons 711a to 711d are cross-shaped buttons for inputting instructions for moving the corresponding marker vertically and horizontally. The operation button 711a is a button for operating the post-deformation shape designation marker at the upper left vertex. The operation button 711b is a button for operating the post-deformation shape designation marker at the upper right vertex. The operation button 711c is a button for operating the post-deformation shape designation marker at the lower right vertex. The operation button 711d is a button for operating the post-deformation shape designation marker at the lower left vertex.

  The display area 712 is an area for displaying a live view image of the projection plane 500 acquired from the imaging device 400. The user operates the post-deformation shape designation marker by directly observing the projection plane or operating the operation button 711 while viewing the live view image.

  A list 713 is a list showing a list of target projectors. The check box 714 is an instruction icon for selecting a projector to be used for adjustment among target projectors shown in the list. The target projector corresponding to the check box in which the user has checked is set as the adjustment target. As described above, in this embodiment, it is assumed that the projector 100a (Projector 1) and the projector 100b (Projector 2) are selected as target projectors. The adjustment can be executed simultaneously for a plurality of units or for each unit.

  The check box 714 is displayed in a state where all target projectors are checked in the initial state. When there is no disturbance such as shaking of the screen, the display positions of all the target projectors are not displaced at the stage where the GUI 710 is displayed. Therefore, in the subsequent adjustment, it is desirable to adjust the projection positions of all the target projectors simultaneously. On the other hand, when there is shaking of the screen or the like, there may be a deviation in the projection position of the target projector when the GUI 710 is displayed. In such a case, with the check box 714, the projector to be adjusted is selected and the projection position is individually adjusted to eliminate the deviation of the projection position between the target projectors. After that, it is desirable to select all target projectors and adjust the entire display position.

  In step S1103, the CPU 201 determines whether the user operation is completed. When the operation button 711 is operated, the CPU 201 determines that the user operation has not been completed, and the process proceeds to S1104.

  In step S1104, the CPU 201 determines whether a plurality of target projectors to be adjusted are selected. When the check box 714 corresponding to two or more target projectors is input, the CPU 201 determines that a plurality of target projectors to be adjusted have been selected. If it is determined that a plurality of target projectors to be adjusted are selected, the process proceeds to S1105. Otherwise, the process proceeds to S1107.

  In step S1105, before entering the adjustment of the projection positions of the plurality of target projectors selected as adjustment targets, it is determined whether there is a projector for which the adjustment of the projection position has been individually performed among the plurality of selected target projectors. If there is no individually adjusted projector, the process proceeds to S1107.

  If there is a projector that has been individually adjusted, in step S1106, the CPU 201 performs a projective transformation matrix update process on the projector that has been individually adjusted. Specifically, the current GUI in the camera coordinate system is first calculated from the projection transformation matrix of the selected projector other than the individually adjusted projector and the coordinates of the four points used for the keystone transformation using Equation 5. The adjustment position is calculated. After that, the CPU 201 assumes that the coordinates of the four points used for the current keystone deformation of the projector that has been individually adjusted are equal to the adjustment position of the current GUI in the camera coordinate system that has been obtained earlier, and the projection transformation matrix. Is recalculated and stored in the RAM 202. The process proceeds to S1107.

  Even if the projection position of each projector is shifted due to a disturbance such as a screen, it is possible to select the projector whose projection position has shifted from the list 713 and make individual adjustments, so that the overall adjustment can be performed without deviation. It becomes.

  That is, when the position adjustment of the projection area of any of the target projectors has been executed before simultaneously controlling the positions or shapes of the projection areas of the plurality of target projectors, the projection transformation matrix of the target projector is obtained by the position adjustment. Change. If the adjustment of the projection areas of a plurality of target projectors is executed without performing the process of S1106, the projection positions of the target projectors gradually shift with respect to others. This is because the projection conversion matrix calculated in S904 does not eliminate the shift, and therefore, when a plurality of projectors are selected and operated in the list view 713, the projection position gradually shifts.

  In step S <b> 1107, the CPU 201 updates the coordinates of the deformed shape on the camera coordinate plane stored in the RAM 202. In step S <b> 1108, the CPU 201 regenerates the marker image and projects it on the selected target projector.

  Until the execution button 715 of FIG. 12 is operated by the user, the operation button 711 and the check box 714 can be operated. When the execution button 715 in FIG. 12 is operated by the user, it is determined in S1103 that the user's operation has been completed, and the projection shape designation processing ends. When the projection shape designation process is completed, the process proceeds to S807 in the flow of FIG.

  As described above, according to the present embodiment, when the stack projection position is adjusted according to the present invention, the subsequent adjustment can be correctly performed even if a deviation occurs.

  In the above-described projection shape designation processing, it has been described that the projective transformation matrix is updated at the timing when a plurality of units are adjusted after performing individual adjustments. You may comprise so that it may update. In that case, after the processing of S1108, it is determined whether or not the adjustment is individual adjustment, and if it is individual adjustment, the projective transformation matrix is updated. For example, four projection positions in the camera coordinate system are calculated using the projection transformation matrix of the projector that is closest in position to the projector that has been individually adjusted and the four coordinates of the keystone correction. Thereafter, if it is assumed that the four points and the four points at the current projection position of the projector performing the individual adjustment are equal, it is possible to calculate the projective transformation matrix of the projector performing the individual adjustment.

  In the present embodiment, the example in which the projection transformation matrix is automatically updated inside the projection shape designation process has been described. However, the projection transformation matrix may be explicitly updated by a user instruction. good. For example, by pressing the adjustment button 716 of the GUI 710 after selecting the projector to be adjusted, the CPU 201 may receive an instruction to update the projection transformation matrix from the user, and the update process may be performed. In response to pressing of the individual adjustment button 716, individual adjustment was performed from the four keystone deformation points of the adjusted projector, the four keystone deformation points of the unadjusted projector, and the projection transformation matrix. The projection transformation matrix of the projector can be updated.

  In the present embodiment, it is described that the projection transformation matrix is updated by assuming that the positions of the projectors are equal to each other. May be requested. When using the imaging apparatus 400, for example, after completion of individual adjustment, the projection plane 500 is photographed once by the imaging apparatus 400, the deviation between the projectors is recognized, and if the deviation is within a predetermined range, the projection is performed again. It is also possible to adopt a configuration in which the transformation matrix is updated.

[Other Examples]
It goes without saying that the object of the present invention can also be achieved by supplying a storage medium storing software program codes for realizing the functions of the above-described embodiments to the apparatus. At this time, the computer (or CPU or MPU) including the control unit of the supplied apparatus reads and executes the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, the OS (basic system or operating system) running on the apparatus performs part or all of the processing based on the instruction of the program code described above, and the functions of the above-described embodiments are realized by the processing. Needless to say, cases are also included.

  Further, the program code read from the storage medium may be written to a memory provided in a function expansion board inserted in the apparatus or a function expansion unit connected to the computer, and the functions of the above-described embodiments may be realized. Needless to say, it is included. At this time, based on the instruction of the program code, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing.

DESCRIPTION OF SYMBOLS 100 Projector 200 Projection control apparatus 201 CPU
202 RAM
203 ROM
204 HDD
205 Display unit 206 Network IF
207 Video output unit 208 Communication unit 209 Operation unit 210 Internal bus 400 Imaging device

Claims (13)

A projection control device for controlling the position or shape of the projection region of a plurality of projection devices,
Adjustment including a first adjustment mode for controlling a projection area of one of the plurality of projection apparatuses and a second adjustment mode for controlling a projection area of two or more of the plurality of projection apparatuses. Setting means for setting the mode;
Adjusting means for adjusting the position or shape of the projection area of the projection device according to the adjustment mode set by the setting means;
With
The control of the projection area of the one projection device is executed in the first adjustment mode before the setting means sets the second adjustment mode and the setting means sets the second adjustment mode. In the projection control apparatus, the adjustment unit corrects a parameter used for controlling a projection area of the two or more projection apparatuses.   The projection control apparatus according to claim 1, wherein the adjustment unit performs the correction of the parameter every time the projection area is controlled in the first adjustment mode.   The projection control apparatus according to claim 1, wherein when the setting unit switches from the first adjustment mode to the second adjustment mode, the adjustment unit performs the correction of the parameter. Obtaining means for obtaining an image obtained by imaging a region of a projection plane including a projection image projected by the plurality of projection devices;
The adjustment unit determines whether or not the deviation of the projection positions of the plurality of projection apparatuses is within a predetermined range in the captured image, and the deviation of the projection positions of the plurality of projection apparatuses is within a predetermined range. The projection control apparatus according to claim 1, wherein the correction of the parameter is executed. Receiving means for receiving an instruction to execute correction of the parameter by the user;
The projection control apparatus according to claim 1, wherein the adjustment unit executes the correction of the parameter in response to the reception unit receiving the execution instruction. A projection control device according to any one of claims 1 to 5,
A plurality of projection devices capable of controlling the position or shape of the projection region by the projection control device;
Including projection system. A control method of a projection control device for controlling the positions or shapes of projection regions of a plurality of projection devices,
Adjustment including a first adjustment mode for controlling a projection area of one of the plurality of projection apparatuses and a second adjustment mode for controlling a projection area of two or more of the plurality of projection apparatuses. A setting process for setting the mode;
An adjustment step of adjusting the position or shape of the projection area of the projection device according to the adjustment mode set by the setting means;
With
The control of the projection area of the one projection device is executed in the first adjustment mode before the setting step sets the second adjustment mode and the setting means sets the second adjustment mode. And the adjusting step corrects parameters used for controlling the projection areas of the two or more projection apparatuses.   The method of controlling a projection control apparatus according to claim 7, wherein the adjustment step executes the correction of the parameter every time the control of the projection area is executed in the first adjustment mode.   The control of the projection control apparatus according to claim 7, wherein when the setting step switches from the first adjustment mode to the second adjustment mode, the adjustment step performs correction of the parameter. Method. An acquisition step of acquiring an image obtained by imaging a region of a projection plane including a projection image projected by the plurality of projection devices;
The adjustment step determines whether or not the deviation of the projection positions of the plurality of projection apparatuses is within a predetermined range in the captured image, and the deviation of the projection positions of the plurality of projection apparatuses is within a predetermined range. The projection control apparatus control method according to claim 7, wherein the correction of the parameter is executed in a case. A reception step of receiving an instruction to execute correction of the parameter by the user;
The method of controlling a projection control apparatus according to claim 7, wherein the adjustment step executes the correction of the parameter in response to the reception step receiving the execution instruction.   The program for a processor to perform the control method of the projection control apparatus of any one of Claim 7 thru | or 11.   A storage medium storing a program for a processor to execute the control method of the projection control apparatus according to claim 12, wherein the processor can read the program.

Images