JP2016161626A - Control device, program, and projection system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a control device, a program, and a projection system capable of restricting projection light from a projection device such as a plurality of projectors in accordance with the visual field of a person.SOLUTION: A reproduction control device 6 acquires site information including the positions of a plurality of sites of a person who is present in a three-dimensional space, and generates the three-dimensional model of the head of the person on the basis of the acquired site information, and specifies the visual line direction of the person on the basis of the acquired site information, and sets a projection restriction area on the surface of the three-dimensional model of the head on the basis of the specified visual line direction. Then, the reproduction control device 6 performs projection transformation to the projection restriction area of the three-dimensional model for each two-dimensional plane to which each of a plurality of projectors 5a to 5d projects a video, and determines the projection condition of the video projected by each of the projectors 5a to 5d to the two-dimensional plane on which the projection restriction area is subjected to projection transformation.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a technical field of a projection system that projects an image on projection surfaces in a plurality of directions.

  Conventionally, as disclosed in, for example, Patent Document 1, a sensor is provided at a position facing a projection-type image display device and a screen, and a screen of the position of a person who enters between the projection-type image display device and the screen is provided. A technique is known in which the brightness corresponding to the screen is determined and the brightness of the screen surface is changed according to the position of the person who has entered. According to this technique, it is possible to prevent strong light from entering the human eye.

JP 2004-348078 A

  In the conventional technology described above, the position of the person is determined by determining a reaction indicating the presence or absence of the person by a sensor for detecting the position of the person. However, it is not possible to accurately determine whether or not projection light affects a person's vision only by determination based on the presence or absence of the person. For example, when using a plurality of projectors, depending on the orientation of the person relative to the projection direction of the projector, even if the projection light hits the person, there is a case where the human vision is not affected. In such a case, if the projection light from the projector is suppressed based on the presence / absence of a person, the projection light is suppressed to an unnecessary area, which is unnecessary for the projected image more than the effect on the human vision. There are restrictions.

  The present invention has been made in view of the above points, and provides a control device, a program, and a projection system capable of limiting projection light from projection devices such as a plurality of projectors according to the field of view of a person. To do.

  In order to solve the above-described problem, the invention described in claim 1 includes a plurality of projection apparatuses that project images from a plurality of arranged different positions onto a projection plane in each direction, and at least one position detection apparatus. A control device used in a projection system comprising: a part including positions of a plurality of parts of a person in a three-dimensional space in a range from the plurality of projection apparatuses to a projection plane based on a detection result of the position detection apparatus From the acquisition means for acquiring information, the generation means for generating a three-dimensional model of the person's head based on the part information acquired by the acquisition means, and the person information from the part information acquired by the acquisition means And a projection restriction region including a region of the line-of-sight direction specified by the specifying unit is provided on a surface of the three-dimensional model of the head generated by the generating unit. And a two-dimensional plane on which each of the plurality of projection devices projects an image with reference to the position of each of the plurality of projection devices. Conversion means for projecting and converting, and determining means for determining a projection condition of an image projected by the projection device that projects the projection restricted region onto the two-dimensional plane that has been projected and transformed by the converting means. Features.

  According to a second aspect of the present invention, in the control device according to the first aspect, the setting means uses the range of the field of view of the person relative to the line-of-sight direction as a projection restricted region on the surface of the three-dimensional model. It is characterized by setting.

  According to a third aspect of the present invention, in the control device according to the first or second aspect, the acquisition means includes the position of the head of the person, the position of the right shoulder of the person, and the left shoulder of the person. The part information including the position of the part, in the part information, the part information corresponding to the head of the person, the part information corresponding to the right shoulder of the person, The image processing apparatus further includes a calculation unit that calculates a normal vector based on the part information corresponding to the left shoulder of the person, and the specifying unit specifies the direction of the normal vector calculated by the calculation unit as the line-of-sight direction. It is characterized by doing.

  According to a fourth aspect of the present invention, in the control device according to the first or second aspect, the part information is a rotation angle of the part, and a rotation angle based on another part connected to the part. And calculating means for calculating a rotation angle of the head based on the rotation angle included in each of the part information, wherein the specifying means is configured to calculate the line of sight from the rotation angle calculated by the calculation means. It is characterized by specifying a direction.

  The invention according to claim 5 is used in a projection system including a plurality of projection apparatuses that project images onto a projection plane in each direction from a plurality of different positions arranged, and at least one position detection apparatus. A program to be executed by a computer used in a control device, including a position including a plurality of positions of a person in a three-dimensional space in a range from the plurality of projection devices to a projection plane based on a detection result of the position detection device From the acquisition step of acquiring information, the generation step of generating a three-dimensional model of the head of the person based on the part information acquired by the acquisition step, and the part information acquired by the acquisition step, the person A specifying step of specifying the line-of-sight direction and a surface of the three-dimensional model of the head generated by the generating step according to the specifying step. A setting step of setting a projection restriction region including a region of the identified line-of-sight direction; and the projection restriction region of the three-dimensional model set by the setting step with reference to the position of each of the plurality of projection devices. A conversion step of performing projection conversion for each two-dimensional plane on which an image is projected by each of the plurality of projection devices, and an image projected by the projection device that projects the projection-restricted region on the two-dimensional plane that has been transformed by the conversion step And a determination step of determining the projection conditions of the computer.

  The invention according to claim 6 is a projection system comprising a plurality of projection devices that project images onto a projection surface in each direction from a plurality of different positions arranged, and at least one position detection device. The control device included in the projection system acquires part information including the positions of a plurality of parts of a person in a three-dimensional space in a range from the plurality of projection apparatuses to the projection plane from the detection result of the position detection apparatus. Based on the acquisition unit, the generation unit that generates the three-dimensional model of the person's head from the part information acquired by the acquisition unit, and the part information acquired by the acquisition unit, the line-of-sight direction of the person is determined. A specifying unit for specifying, and a setting for setting a projection restriction region including a region in the line-of-sight direction specified by the specifying unit on the surface of the three-dimensional model of the head generated by the generating unit And the projection-restricted region of the three-dimensional model set by the setting unit is projected for each two-dimensional plane on which each of the plurality of projection devices projects an image with reference to the position of each of the plurality of projection devices. Conversion means for converting, and determination means for determining a projection condition of an image projected by the projection device that projects the two-dimensional plane on which the projection restricted region is projected and converted by the conversion means. To do.

  According to the first, fifth, and sixth aspects of the invention, it is possible to limit the projection light from the plurality of projection devices according to the line-of-sight direction of the person.

  According to the second aspect of the present invention, it is possible to limit the projection light from the plurality of projection devices according to the field of view of the person.

  According to the third aspect of the present invention, the line-of-sight direction can be quickly identified from the position of the head, the position of the left shoulder, and the position of the right shoulder.

  According to the fourth aspect of the present invention, it is possible to specify a more accurate line-of-sight direction.

It is a figure which shows the example of a schematic structure of the projection system S of this embodiment. (A) is a figure showing an example of the positional relationship of the apparatus arrange | positioned in the room seen from the upper direction of the room where the projection system S is used, (B) is the room of the room where the projection system S is used. It is a figure showing an example of the positional relationship of the apparatus arrange | positioned in the room seen from the side which the arrow D shows. (A) is a figure which shows an example of the some site | part which forms a person's skeleton, (B) is a normal line determined by the position a of the head, the position b of the left shoulder, and the position c of the right shoulder It is a figure which shows an example of a vector. It is a flowchart which shows an example of the projection control process of the reproduction | regeneration control apparatus 6 of this embodiment. (A) is a figure which shows an example of the visual field of the person in a room, (B) is the figure which looked at the three-dimensional model of the person's head from the head part direction, (C) is a person It is the figure which looked at the three-dimensional model of the head from the side of the face. It is a conceptual diagram which shows an example of a mode that the projection restriction | limiting area | region in a three-dimensional coordinate system is projected on the two-dimensional plane of projector 5a-5d. It is a figure explaining one of the effects of the projection control process of this embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[1. Outline of Projection System Configuration and Operation]
First, with reference to FIG. 1 etc., the structure and operation | movement outline | summary of the projection system S of this embodiment are demonstrated. As shown in FIG. 1, the projection system S includes a display / operation device 1, a music playback device 2, a camera 3, video playback devices 4a to 4e, projectors 5a to 5e, a playback control device 6, and the like. . Here, the projectors 5a to 5e are examples of a projection device. The reproduction control device 6 is an example of a control device. The projection system S is used in a room in an amusement facility such as a karaoke store. The display / operation device 1, the music playback device 2, and the camera 3 are connected to the playback control device 6 via a wired or wireless connection, respectively. The video reproduction apparatuses 4a to 4e and the projectors 5a to 5e are connected to the reproduction control apparatus 6 via a wired LAN (Local Area Network) or a wireless LAN, a hub, and the like, respectively. Further, the projectors 5a to 5e are connected to the video reproduction devices 4a to 4e, respectively, via wired or wireless.

  As shown in FIG. 2A, the projectors 5a to 5d are arranged to project images on any two wall surfaces of the wall surfaces a to d, respectively. In other words, the projectors 5a to 5d project images from a plurality of positions arranged at different positions onto the projection surfaces in the projection ranges that fall within the respective projection angles θa to θd in the respective projection directions. At this time, each of the projectors 5a to 5d projects an image on a projection plane including two wall surfaces. In the example of FIG. 2 (A), four wall surfaces a to d on the front, rear, left and right of the person are projection surfaces. When the projector 5c is arranged at a corner between the wall surface c and the wall surface b of the rectangular room, an image is projected onto a partial region of the wall surface a and a partial region of the wall surface d including the diagonal of the room. When the projector 5d is arranged at the corner between the wall surface c and the wall surface d of the rectangular room, an image is projected onto a partial area of the wall surface a and a partial area of the wall surface b including the diagonal of the room. The projectors 5a to 5d project images on any two wall surfaces by overlapping a part of each projection area. In the example of FIG. 2A, a part of the projection surface onto which the image is projected overlaps between the plurality of projectors 5a to 5d. As a result, it is possible to project an image on the entire circumference of the side wall surface of the rectangular room by the projectors 5a to 5d. However, the projection direction or the projection angles θa to θd may be set so that the projection plane onto which the video is projected does not overlap between the plurality of projectors 5a to 5d. In addition, each of the projectors 5a to 5d may project an image on one specific projection plane. The projection plane may be on the ceiling surface and floor surface of the room.

  As shown in FIG. 2A, the projector 5e is arranged so as to project an image on the screen Sc installed on the wall surface d (for example, arranged on the ceiling as shown in FIG. 2B). ing. That is, the projector 5e projects an image on the projection surface on the screen Sc in the direction of the range within the projection angle θe from the arranged position. In the example of FIG. 2, the images from the projectors 5b and 5c are also projected on the projection surface on the screen Sc. However, the video projected from the projectors 5b and 5c is set to a video that does not adversely affect the video projected from the projector 5e. The bad influence means that, for example, the projectors 5b and 5c project a projected image on the screen Sc projected from the projector 5e by projecting a low transparency image such as black on a large area area. In the example of FIG. 2, the person is facing the direction in which the screen Sc is present (the direction of the line of sight shown in FIG. 2A), and is viewing the image projected on the screen Sc. As shown in FIGS. 2A and 2B, the camera 3 is installed on the wall surface d and images a range that falls within the angle of view θ of the camera 3. For example, an image captured by the camera 3 is projected onto the screen Sc by the projector 5e. When music is being played by the music playback device 2, the lyrics telop of the music is superimposed on the video imaged by the camera 3 and projected onto the screen Sc by the projector 5e. Thereby, in the example of FIG. 2, the person who sees the screen Sc is imaged by the camera 3 from the front of the wall d on which the camera 3 is installed.

  In the example of FIG. 2, five video playback devices 4 a to 4 e and five projectors 5 a to 5 e are shown. However, the video playback devices and the projectors are one or more, four or less, or six or more, respectively. It may be. In the example of FIG. 1, the video playback device and the projector correspond one-to-one, but may correspond to one-to-N (N is a natural number of 2 or more). For example, a plurality of projectors may be connected to one video reproduction device. When there is one video playback device, the video playback device may be incorporated in the playback control device 6 as video playback means (configured by hardware or software).

  As the display / operation device 1, for example, a touch panel display including a display screen is used. The display / operation device 1 may be, for example, a remote controller of the music playback device 2. The display / operation device 1 may be incorporated in the music playback device 2 or the playback control device 6. In this case, for example, the display / operation device 1 is a display unit formed integrally with the music playback device 2 or the playback control device 6. The display / operation device 1 displays, on the display screen, titles of videos that are candidates for selection of videos projected by the projectors 5a to 5d, titles of songs that are candidates for selection of music to be played back by the music playback device 2, and the like. The display / operation device 1 selects a video or a song according to a selection operation by the user. For example, the display / operation device 1 transmits a video ID for identifying the selected video to the reproduction control device 6. Further, the display / operation device 1 transmits, for example, a music ID for identifying the selected music to the reproduction control device 6. The music ID transmitted to the reproduction control device 6 is transmitted from the reproduction control device 6 to the music reproduction device 2, for example. The display / operation device 1 may transmit a music ID for identifying the selected music to the music playback device 2, for example.

  The music playback device 2 is a device for playing back music. The music reproducing device 2 stores a plurality of music files in the storage unit. The music file stores music data of music such as karaoke music. Each music file is associated with, for example, a music ID for identifying the music. Upon receiving the music ID transmitted from the display / operation device 1 or the playback control device 6, the music playback device 2 plays the music data indicated by the received music ID and outputs the music signal to a speaker (not shown). To do. When the music playback device 2 is a karaoke commander, the music playback device 2 includes the music signal and a singing voice signal of a user (for example, a person shown in FIG. 2) collected by a microphone (not shown). Are mixed and output to a speaker (not shown).

  As the camera 3, for example, an RGB camera and a depth camera are used. The depth camera is an example of a position detection device. The depth camera is a depth sensor such as an infrared laser. A depth image indicating position information including the depth to each position of the imaging range in the room is acquired by the depth camera. In the case of the example in FIG. 2, this position information includes part information indicating the positions of a plurality of parts of a person in a three-dimensional space in the range from the projectors 5 a to 5 d to the projection plane. The part information indicating the positions of a plurality of parts of a person is, for example, information in which position coordinates are associated with each part, and can be specified from the depth to each position of the imaging range indicated in the depth image. is there. Specifically, by comparing the depth to each position of the imaging range acquired by the depth camera with the person position pattern data for determining the position of each part of the person, each part of the person and its position are determined. Identified. As the person position pattern data to be compared with the depth of the depth image, for example, predetermined pattern data corresponding to the depth of each position of the depth image is read out from many types of person position patterns stored in advance in a predetermined database or the like. Used. Note that the positions of a plurality of human parts to be obtained may be specified by the reproduction control device 6 based on the depth to each position. In the example of FIG. 3A, the head skeleton, the left shoulder b, the right shoulder c, the left elbow d, the right elbow e, the left hand f, the right hand g, and the neck are formed as the parts forming the human skeleton. h, torso i, waist j, left hip k, right hip l, left knee m, right knee n, left foot o, and right foot p are shown. Such part information may include the rotation angle Θ of the part as shown in FIG. The rotation angle Θ of a part is a rotation angle based on another part connected to the part. Alternatively, the rotation angle may be based on the position of a specific part indirectly connected to the part. For example, the rotation angle Θ of the head a is a rotation angle with respect to the neck h connected to the head a. The rotation angle Θ of the neck portion h is a rotation angle with respect to the body portion i connected to the neck portion h. That is, the rotation angle Θ of the head a can also be a rotation angle with the trunk portion i as a reference. The part information including the rotation angle Θ is expressed as (X, Y, Z, Θ). Note that the position information may include part information indicating the position of the part in addition to the part forming the skeleton illustrated in FIG. For example, the position of the head a shown in FIG. 3A indicates the center (three-dimensional coordinates) when the head of the person is a sphere, but the positions of a plurality of parts on the surface of the person's head are shown. The site | part information to show may be contained. The camera 3 continuously captures an imaging range that falls within the angle of view θ of the camera 3, for example, and transmits the imaging result to the reproduction control device 6. This imaging result includes an RGB image and a depth image.

  The video playback devices 4a to 4e are devices that play back video. The video includes a still image and a moving image. For example, a moving image is displayed by switching image frames of a plurality of still images in time series. Each of the video reproduction devices 4a to 4e reproduces video data instructed from the reproduction control device 6, for example. The video reproduction devices 4a to 4e reproduce the video data instructed from the reproduction control device 6 and output the video signals to the projectors 5a to 5e, respectively. The video data to be reproduced is stored in a storage unit (not shown) of each of the video reproduction devices 4a to 4e. The video playback devices 4a to 4d, for example, cut out the areas of the respective projection ranges for the corresponding projectors 5a to 5d from the original video to be projected stored in the storage unit 62 of the playback control device 6 to be described later. Video data to be played back by the video playback devices 4a to 4d corresponding to the projectors 5a to 5d is generated and stored in the storage unit. Alternatively, the video reproduction devices 4 a to 4 e may acquire the video data to be reproduced from the storage unit of the reproduction control device 6. When reproducing the video data, the video reproduction devices 4a to 4d acquire the range of the field of view of the person imaged by the camera 3 from the reproduction control device 6 described later as mask data. The video playback devices 4a to 4d play back video in which the acquired mask data is combined with the video data. The projectors 5a to 5e output images by emitting light corresponding to the image signals reproduced and output by the image reproducing devices 4a to 4e, respectively. Thereby, the projectors 5a to 5e project images on the projection planes in the direction of the range within the respective projection angles θa to θe. Further, the projectors 5a to 5e, for example, turn on or off the power, or change the projection direction or the projection angles θa to θe, for example, according to the control signal from the reproduction control device 6.

  The playback control device 6 is a device that performs music playback control, video projection control, and the like. As shown in FIG. 1, the reproduction control device 6 includes IFs 61 a to 61 d, a storage unit 62, and a control unit 63. The IFs 61 a to 61 d, the storage unit 62, and the control unit 63 are connected to the bus 64. The IF 61 a is an interface between the music playback device 2 and the playback control device 6. The IF 61 b is an interface between the display / operation device 1 and the reproduction control device 6. The IF 61 c is an interface between the camera 3 and the playback control device 6. The IF 61d is an interface with the LAN.

  The storage unit 62 is configured by, for example, a hard disk drive. The storage unit 62 stores an OS (Operating System), the program of the present invention, and the like. The program of the present invention is a program for causing a CPU to execute a control process described later. Further, the storage unit 62 stores a video file of an original video that is original data of a video projected by the projectors 5a to 5d. Examples of original videos include artist promotional videos, company or product advertising videos, and the like. The control unit 63 includes a CPU (Center Processing Unit) as a computer, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The control unit 63 is an example of an acquisition unit, a generation unit, a specification unit, a setting unit, a conversion unit, a determination unit, and a calculation unit according to the present invention. The control unit 63 executes projection control processing according to the program of the present invention.

[2. Projection Control Processing of Reproduction Control Device 6]
Next, with reference to FIG. 4 etc., the projection control process of the reproduction | regeneration control apparatus 6 of this embodiment is demonstrated. For example, when a user (person) entering the room operates the display / operation device 1 to select a desired video (original video), the video ID of the selected video is transmitted to the playback control device 6. When receiving the video ID from the display / operation device 1, the playback control device 6 instructs the video playback devices 4a to 4e to play back video files corresponding to the received video ID as data to be played back, and the projection shown in FIG. Control processing is started. In this way, the video selected by the user is the projection target. Note that a video to be projected may be a default setting. In this case, for example, when the user who enters the room receives the operation signal output from the display / operation device 1 by operating the display / operation device 1, the reproduction control device 6 performs the projection control process shown in FIG. 4. Start.

  When the projection control process shown in FIG. 4 is started, the control unit 63 receives the imaging result captured by the camera 3 from the camera 3 at predetermined time intervals, and from the received imaging result (depth image), Part information is acquired (step S1). Specifically, for each of a plurality of parts of a person, position coordinates associated with the part are acquired from the camera 3 as part information. Alternatively, in step S <b> 1, the control unit 63 receives an imaging result (depth image) from the camera 3, and compares the result of imaging (depth information) with the above-described person position pattern data, thereby providing a plurality of parts of the person. , Position coordinates are specified for each part. As the part information, for example, the head a, the left shoulder b, the right shoulder c, the left elbow d, the right elbow e, the left hand f, the right hand g, which are the human skeleton shown in FIG. Position coordinates of the neck h, the torso i, the waist j, the left hip k, the right hip l, the left knee m, the right knee n, the left foot o, and the right foot p in association with each part. Is determined. Note that the part information may be an estimated value of a part of a plurality of parts. Further, the part information may be acquired only for a part of the plurality of parts including the head or the right shoulder and the left shoulder. Next, the control unit 63 acquires part information corresponding to the head of the person from the part information acquired in step S1 (step S2). Next, the control unit 63 generates a three-dimensional model of the person's head from the part information acquired in step S2 (step S3). For example, the control unit 63 generates a sphere (or ellipsoid) of a predetermined size centered on the position of the head (three-dimensional coordinates of the head) as a three-dimensional model of the person's head. This three-dimensional model is composed of, for example, a set of coordinates of a plurality of parts on the surface of a person's head. The control unit 63 acquires part information indicating the position of each of a plurality of parts on the surface of the person's head from the part information acquired in step S1, and the position (three-dimensional coordinate) indicated by each acquired part information. ) To generate a three-dimensional model of a person's head. At this time, the control unit 63 may generate the three-dimensional model based on each piece of part information acquired in time series, with each piece of part information on the surface of the person's head being continuous in time series. At this time, the control unit 63 may determine the front of the person from the time-series movement transition of each part information by analyzing a plurality of images captured in time series by the camera 3.

  Next, the control unit 63 determines whether or not the normal vector of the person's face can be acquired from the part information acquired in step S1 (step S4). For example, when the part information acquired in step S1 includes part information of a predetermined number or more of parts forming the human skeleton, it is determined that the normal vector of the person's face can be acquired. For example, the normal vector of the human face can be calculated from the rotation angle Θ of the head. Here, the rotation angle Θ of the head is calculated with reference to the rotation angle Θ of the neck connected to the head. The rotation angle Θ of the neck is a rotation angle based on the rotation angle Θ of the body i. That is, the rotation angle Θ of the head is calculated by a predetermined calculation based on the rotation angle Θ of other parts. For this reason, if more part information (for example, from the left foot part and the right foot part to the head part) can be obtained, a more accurate rotation angle Θ can be obtained, and as a result, the normal of the human face is obtained. The accuracy of the vector is also improved. Alternatively, in step S4, based on the depth of the imaging result (depth image) received from the camera 3 and the head part information acquired in step S2, from the depth corresponding to the face of the head, for example, the nose It may be determined that the normal vector of the face of the person can be acquired when the position of a predetermined part of the face such as can be identified. Therefore, when it is determined from the part information acquired in step S1 that the normal vector of the person's face can be acquired (step S4: YES), the process proceeds to step S5. On the other hand, when it is determined that the normal vector of the person's face cannot be acquired (step S4: NO), the process proceeds to step S7.

  In step S5, the control unit 63 calculates the rotation angle Θ of the person's head based on the rotation angle Θ included in the part information of each part forming the skeleton of the person. Next, the control unit 63 calculates a normal vector of the human face from the calculated rotation angle Θ. Alternatively, when it is determined in step S4 that the position of the predetermined part of the face can be specified, in step S5, the direction from the predetermined part of the face toward the front with respect to the face is the human face method. Determined as a line vector. And the control part 63 specifies the direction which the calculated normal vector shows as a person's gaze direction (step S6), and progresses to step S9. With this configuration, it is possible to specify a more accurate line-of-sight direction (normal vector).

  In step S7, the control unit 63 acquires part information corresponding to the right shoulder of the person and part information corresponding to the left shoulder of the person from the part information acquired in step S1. Next, the control unit 63 calculates the legal information based on the part information corresponding to the head acquired in step S2 and the part information corresponding to the right shoulder and the part information corresponding to the left shoulder acquired in step S7. Calculate the line vector.

  For example, as shown in FIGS. 3A and 3B, the control unit 63 generates a vector ab between the head position a and the left shoulder position b, and the head position a and the right shoulder. A normal vector is calculated by obtaining an outer product (vector ab × vector ac) with the vector ac between the position c. And the control part 63 specifies the direction which the calculated normal vector shows as a person's gaze direction (step S8), and progresses to step S9. With this configuration, the line-of-sight direction (normal vector) can be quickly identified from the position a of the head, the position b of the left shoulder, and the position c of the right shoulder.

  In step S9, the control unit 63 sets a projection restricted region on the surface of the three-dimensional model of the human head generated in step S3 based on the line-of-sight direction specified in step S6 or step S8 (that is, A projection restriction region including the region in the line-of-sight direction is set on the surface of the three-dimensional model). This projection restriction area is an area in which projection from the projectors 5a to 5e should be restricted (for example, prohibited). For example, the control unit 63 sets the range of the field of view of the person with reference to the direction of the line of sight of the person as a projection restriction region on the surface of the three-dimensional model of the head of the person. The control unit 63 sets 1 to the coordinates in the projection restricted area of the three-dimensional model of the head, and sets the other coordinates to 0. Or the control part 63 sets black to the coordinate in the projection restriction | limiting area | region of the three-dimensional model of a head, and sets white to other coordinates.

  In the example of FIG. 5A, a field of view determined by a horizontal viewing angle with respect to a person's line-of-sight direction is shown. However, the field of view of a person is determined by a horizontal viewing angle (left and right viewing angle) and a vertical viewing angle (up and down viewing angle) based on the direction of the person's line of sight. The horizontal viewing angle is preset in the range of, for example, 190 degrees to 210 degrees, and the vertical viewing angle is preset in the range of, for example, 120 to 130 degrees. The control unit 63 obtains the range of the field of view of the person from the horizontal viewing angle and the vertical viewing angle, and sets this as a projection limited region on the surface of the three-dimensional model of the person's head. FIGS. 5B and 5C are diagrams illustrating an example of the projection restriction area LA0 set on the surface of the three-dimensional model of the human head. FIG. 5B is a view of a three-dimensional model of the person's head viewed from the top, and FIG. 5C is a view of the three-dimensional model of the person's head viewed from the side of the face. It is. In the example of FIGS. 5B and 5C, the projection limited area LA0 is set by adding the movable range of the rotation angle of the neck to the range determined by the horizontal viewing angle and the vertical viewing angle. Specifically, as the horizontal viewing angle (left and right viewing angle), an angle that becomes a visual field when the neck is in a movable range in which the neck rotates in the left direction and the right direction is determined from the reference of the gaze direction of the person. As the vertical viewing angle (vertical viewing angle), an angle that becomes a visual field when the neck is in a movable range in which the neck moves in the upward and downward directions is determined from the reference of the gaze direction of the person.

  Next, the control unit 63 sets the projection restriction area of the three-dimensional model set in step S9 for each two-dimensional plane on which each of the projectors 5a to 5d projects an image with reference to the positions of the plurality of projectors 5a to 5d. A projection conversion process is performed (step S10). A three-dimensional model of a person's head is represented by a three-dimensional coordinate system with the camera 3 as a reference. For this reason, the control unit 63 projects and converts the projection restricted area in the three-dimensional coordinate system into the two-dimensional planar coordinate system of each projector 5a to 5d. For example, the control unit 63 converts the coordinates of the projection restricted area in the three-dimensional coordinate system into the coordinates of the projection restricted area in the two-dimensional plane of the projectors 5a to 5d using a matrix for projective transformation.

  For example, the projection limited area LA1 projected and converted onto the two-dimensional plane of the projector 5a shown in FIG. 6 is a projection obtained by viewing the three-dimensional model of the human head from the position of the projector 5a (for example, the origin on the two-dimensional plane). This is the restricted area LA0. Further, for example, the projection limited area LA2 projected and converted to the two-dimensional plane of the projector 5b shown in FIG. 6 is the projection limited area LA0 in which the three-dimensional model of the person's head is viewed from the position of the projector 5b. Further, for example, the projection limited area LA3 projected and converted to the two-dimensional plane of the projector 5d shown in FIG. 6 is the projection limited area LA0 in which the three-dimensional model of the person's head is viewed from the position of the projector 5d. When the three-dimensional model of the person's head is viewed from the position of the projector 5c, the projection restriction area LA0 is hidden behind the area behind the person's head, so that the projection restriction is applied to the two-dimensional plane of the projector 5c. The area is not projected. Note that the range of the two-dimensional plane of each of the projectors 5a to 5d is equal to the range (projection range) that falls within the projection angles θa to θd of the projectors 5a to 5d, for example. Further, the two-dimensional planes of the projectors 5a to 5d may be on a common coordinate system. In this case, the position of the two-dimensional plane is used as the origin, and the positions of the projectors 5a to 5d are defined on the two-dimensional plane.

  In step S11, the control unit 63 determines the projection conditions for the images projected by the projectors 5a to 5d, and executes a mask data generation process according to the determined projection conditions. The mask data generation process is a process in which, for example, the control unit 63 determines mask data to be combined with a video as a video projection condition. In the process in this case, the control unit 63 generates, for example, mask data corresponding to the projection restricted area projected and converted to the two-dimensional plane for each two-dimensional plane obtained by projecting the projection restricted area. Each two-dimensional plane corresponds to each projection plane projected by the projectors 5a to 5d on which the video data is reproduced by the video reproduction devices 4a to 4d. That is, the coordinates of the mask data in the two-dimensional plane coincide with, for example, the coordinates of the projection restricted area. In the mask data generation process, the alpha value indicating the transparency of the coordinates in the two-dimensional plane corresponding to the projection restricted area where the video data to be reproduced is projected and converted is indicated as opaque for each of the video reproducing devices 4a to 4d as the mask data. A process of determining 1 is performed. At this time, a process of determining the alpha value of coordinates other than the area corresponding to the projection restricted area as 0 may be further performed. Alternatively, the mask data generation process is a process of setting the expression color of coordinates in the two-dimensional plane corresponding to the projection restricted area to be projected and converted as black as mask data. Next, the control unit 63 transmits the generated mask data to each of the video reproduction devices 4a to 4d that project onto each two-dimensional plane. That is, the control unit 63 transmits the generated mask data to each of the video reproduction devices 4a to 4d in order to synthesize the generated mask data with the generated video (step S12). Thereby, in the video reproduction devices 4a to 4d, a video in which the mask data is combined with the video data is generated as a projected video. Specifically, synthesizing the mask data with the video data means that 0 is set to the alpha value at the coordinates determined corresponding to the projection restricted area in the mask data for the video data. Also, for example, the expression color in the projection restriction area shown in the mask data in the video data is painted in black, or the coordinate expression color corresponding to the projection restriction area of the video data is based on the alpha value defined in the mask data. A process of superimposing the semi-transparent black mask is performed.

  In step S11, the control unit 63 may determine to change the projection angle or the projection direction of the projector as a video projection condition. In the process in this case, the control unit 63 transmits, for example, a control command for changing the projection angle or the projection direction by a predetermined angle to a projector corresponding to a two-dimensional plane including the projection restricted area. This is an angle at which this projection restricted area deviates from the projection range. The projector that has received this control command changes the projection angle or the projection direction by a predetermined angle. Or the control part 63 may determine the illumination intensity fall of the light emitted from a projector as a projection condition of an image | video. In the process in this case, the control unit 63 transmits a control command for reducing the illuminance to, for example, a projector corresponding to a two-dimensional plane including the projection restricted area. The projector that has received this control command reduces the illuminance (brightness) of the light emitted from the light emitting unit. Alternatively, the control unit 63 may determine to change the original video to be projected as the video projection condition. In the process in this case, for example, the control unit 63 generates a black image as an image to be projected by a projector corresponding to a two-dimensional plane including the projection restricted area. Note that an image to be projected by a projector corresponding to a two-dimensional plane that does not include the projection restriction region is generated from the original image to be projected, as described above.

  As shown in FIG. 7A, when an image is not projected from the projector 5e, a shadow of a person is made on the screen Sc by projection light from the projector 5c, for example. On the other hand, as shown in FIG. 7B, when an image is projected from the projector 5e, for example, the shadow caused by the projection light from the projector 5c is erased by the image from the projector 5e. By the way, as in the conventional example shown in FIG. 7C, when performing mask processing for synthesizing a black image in consideration of only the planar head contour, the mask processing is performed in a range slightly wider than the head contour. It is desirable. However, in this case, the image of the masked area outside the head outline is a black image, unlike the above-described shadow, and thus cannot be erased by the image from the projector 5e. Since an unnatural image due to the resolution is formed, the person who is looking at the screen Sc is uncomfortable. In particular, when projected from a long distance, as shown by “enlargement” in FIG. 7C, dots that are masked outside the head contour can be visually recognized around the head contour due to the resolution. End up. On the other hand, as in this embodiment shown in FIG. 7D, a projection restricted area is set in the three-dimensional model of the human head in consideration of the field of view, and mask processing is performed through projection conversion to a two-dimensional plane. In the case of the configuration to be performed, the range in which the mask process is performed can be limited. For this reason, since the image subjected to the mask process is not projected onto the screen Sc, for example, it is possible to prevent the person looking at the screen Sc from feeling uncomfortable. In particular, the image projected on the screen Sc in front of the person enters the field of view (highly perceptual recognition), but according to the present embodiment, it is possible to prevent an unnatural image from being projected.

  As described above, the reproduction control device 6 acquires part information including the positions of a plurality of parts of a person in a three-dimensional space, and generates a three-dimensional model of the person's head based on the acquired part information. Then, the gaze direction of the person is specified based on the acquired part information, and a projection restriction region is set on the surface of the three-dimensional model of the head based on the specified gaze direction. Then, the reproduction control device 6 projects and converts the projection restricted area of the three-dimensional model for each two-dimensional plane on which the plurality of projectors 5a to 5d project images, and converts the projection restricted area to the two-dimensional plane obtained by the projection conversion. Projection conditions for images projected by the projectors 5a to 5d to be projected are determined. For this reason, according to the present embodiment, it is possible to limit the projection light from the plurality of projectors 5a to 5d according to the field of view of the person. For example, it is possible to reduce glare by appropriately limiting the projection light in a range that enters a human field of view. On the other hand, even if the projection light from the projector hits the person, the projection light is prevented from being limited to a range that does not affect the human vision (the range in which the projection light does not need to be limited), so that the person does not feel uncomfortable. Can be.

DESCRIPTION OF SYMBOLS 1 Display / operation device 2 Music playback device 3 Cameras 4a-4e Video playback devices 5a-5e Projector 6 Playback control device S Projection system

Claims (6)

A control device used in a projection system including a plurality of projection devices that project images from a plurality of arranged different positions onto a projection plane in each direction, and at least one position detection device,
Obtaining means for obtaining part information including positions of a plurality of parts of a person in a three-dimensional space in a range from the plurality of projection apparatuses to a projection plane, from the detection result of the position detection apparatus;
Generating means for generating a three-dimensional model of the head of the person based on the part information acquired by the acquiring means;
From the part information acquired by the acquisition means, a specifying means for specifying the gaze direction of the person,
Setting means for setting a projection restriction region including a region in the line-of-sight direction specified by the specifying unit on the surface of the three-dimensional model of the head generated by the generating unit;
Conversion for projecting and transforming the projection restricted region of the three-dimensional model set by the setting unit for each two-dimensional plane on which each of the plurality of projection devices projects an image with reference to the position of each of the plurality of projection devices. Means,
Determining means for determining a projection condition of an image projected by the projection device that projects the projection restricted region onto the two-dimensional plane obtained by projecting and transforming the projection restricted region by the transforming means;
A control device comprising:   The control device according to claim 1, wherein the setting unit sets a range of the field of view of the person based on the line-of-sight direction as a projection restriction region on a surface of the three-dimensional model. The acquisition means acquires the part information including the position of the head of the person, the position of the right shoulder of the person, and the position of the left shoulder of the person,
Among the part information, based on the part information corresponding to the head of the person, the part information corresponding to the right shoulder of the person, and the part information corresponding to the left shoulder of the person, A calculation means for calculating a normal vector;
The control device according to claim 1, wherein the specifying unit specifies the direction of the normal vector calculated by the calculating unit as the line-of-sight direction. The part information includes a rotation angle of the part and includes a rotation angle based on another part connected to the part,
A calculation means for calculating a rotation angle of the head based on the rotation angle included in each of the part information;
The control device according to claim 1, wherein the specifying unit specifies the line-of-sight direction from the rotation angle calculated by the calculating unit. A computer used in a control device used in a projection system including a plurality of projection devices that project images onto a projection plane in each direction from a plurality of different positions and at least one position detection device. A program,
An acquisition step of acquiring part information including positions of a plurality of parts of a person in a three-dimensional space in a range from the plurality of projection apparatuses to a projection plane, from the detection result of the position detection apparatus;
A generating step for generating a three-dimensional model of the person's head based on the part information acquired in the acquiring step;
From the part information acquired by the acquisition step, a specifying step of specifying the gaze direction of the person,
A setting step of setting a projection restriction region including a region of the line-of-sight direction specified by the specifying step on the surface of the three-dimensional model of the head generated by the generating step;
Conversion for projecting and transforming the projection restricted area of the three-dimensional model set by the setting step for each two-dimensional plane on which each of the plurality of projection devices projects an image with reference to the position of each of the plurality of projection devices. Steps,
A determination step of determining a projection condition of an image projected by the projection device that projects the projection-restricted region onto the two-dimensional plane obtained by the projection conversion by the conversion step;
A program that causes a computer to execute. A projection system comprising: a plurality of projection devices that project an image onto a projection plane in each direction from a plurality of different positions arranged; and at least one position detection device,
The control device provided in the projection system includes:
Obtaining means for obtaining part information including positions of a plurality of parts of a person in a three-dimensional space in a range from the plurality of projection apparatuses to a projection plane, from the detection result of the position detection apparatus;
Generating means for generating a three-dimensional model of the head of the person based on the part information acquired by the acquiring means;
From the part information acquired by the acquisition means, a specifying means for specifying the gaze direction of the person,
Setting means for setting a projection restriction region including a region in the line-of-sight direction specified by the specifying unit on the surface of the three-dimensional model of the head generated by the generating unit;
Conversion for projecting and transforming the projection restricted region of the three-dimensional model set by the setting unit for each two-dimensional plane on which each of the plurality of projection devices projects an image with reference to the position of each of the plurality of projection devices. Means,
Determining means for determining a projection condition of an image projected by the projection device that projects the projection restricted region onto the two-dimensional plane obtained by projecting and transforming the projection restricted region by the transforming means;
A projection system comprising: