JP2006178948A - Prop input device and method for mapping object from two-dimensional camera image to three-dimensional space for controlling action in game program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a prop input device which does not require the calibration of a system dependent on lighting conditions. <P>SOLUTION: A hand-manipulated prop is picked up via a single video camera, and the part of the image pertaining to the object is isolated for mapping the position and orientation of the object into a three-dimensional space. The three-dimensional information of the object is stored in a memory and used for controlling action in a game program corresponding to a virtual object in a scene on a video screen. Algorithms for deriving the three-dimensional information for various props employ geometry processing including an area information computation, edge detection and/or color transition localization to find the position and orientation of the prop from two-dimensional pixel data. Colors of stripes on the props are determined so that separation in a two-dimentional chrominance color space is maximized, aiming at detecting significant color transitions. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a computer vision system, and more specifically, an object is captured by a video camera, an image portion related to the object is separated by analysis of a camera image, and the position and direction of the object are mapped to a three-dimensional space. Related to the system. The three-dimensional information of the object is stored in a memory and used for controlling actions in the game program such as rendering of a virtual object in a scene on a video screen.

  It is a well-known technique to capture a moving object using a digital video camera and to process various video images captured by the video camera to display various screens. As an example, Patent Document 1 (hereinafter referred to as the Segen patent) in which Segen is an inventor discloses a method for animating a sports event. The present invention is incorporated herein by reference to this Segen patent. According to this method, the position of the tennis ball in the game is captured by a plurality of video cameras, and a three-dimensional point on the tennis court and a two-dimensional point (pixel) in the digital image in the camera field of view. A set of mathematical formulas showing the relationship is used. The position of each pixel that makes up the ball shown in a particular digital image is related to the specific three-dimensional position of the ball being used in the game, and by impacting each video image by triangulation, A series of image frames are analyzed using a least squares method to align the ball with a trajectory that represents the motion segment of the ball in the unloaded state.

  As described to some extent in the Segen patent, if three-dimensional information about the position and motion of an object is required, the motion of the object can be reproduced as an animation using various methods known in the art. Can do. In such a method, a program for appropriately creating an animation of an object in a video game is used.

  That is, the Segen patent seeks to determine the three-dimensional position of a moving object based on a plurality of two-dimensional video images captured within a certain period. If the three-dimensional position of the “real” object can be obtained, this information can be used to control the game program by various methods generally known to game programmers.

  However, the system of the Segen patent requires multiple video cameras because it obtains position information about the object based on analysis using triangulation. Furthermore, since the object detected by the Segen patent is a simple sphere, it does not include position information (for example, tilt) in the direction of the object in space. Therefore, in the Segen patent system, the position and orientation of an object can be determined based on a two-dimensional video image using a single video camera, whether the object is moving or stationary. It cannot be reproduced.

  Game programs typically use virtual objects that combine three-dimensional geometric shapes. During execution of the game program, three-dimensional information (position and direction) between the objects is determined by control input parameters input using an input device such as a joystick or a game controller. The three-dimensional position and information of a virtual object are projected on a two-dimensional screen after performing background setting, lighting setting, shading, etc., and three-dimensional by the rendering processing function of the game machine. A scene with a sense of perspective is created.

  As an example, a “virtual object” that operates according to the operation of a “real” object as part of a moving image displayed on the screen is assumed. In order to display the virtual object, the calculated three-dimensional information is used to determine the position and direction of the “virtual object” in the memory space of the game machine, and the image is rendered by a known projection process. Three-dimensional information is displayed on the screen as a realistic image with perspective.

  However, in the above-described technique, the capture of the object may not be successful due to various problems. That is, there is a particularly serious problem in accurately extracting only the pixels of the video image that clearly correspond to the object of interest. For example, if the object is a single color and uses a single color for the background that is significantly different from the color of the object, it is relatively easy to capture the movement of the object. However, even if a bright color is used for the object, it is not easy to capture the object if a plurality of colors are used for the background or the background is moving. Also, if the lighting conditions are changed, the apparent color of the object captured by the video camera is greatly affected. Therefore, in an object detection method that detects an object of a specific color, a large error is likely to occur, and recalibration must always be performed as the illumination condition changes. In order to use a video game program in a general home, it is necessary to dramatically increase the flexibility and reliability of a computer vision system for capturing conventional objects.

US Pat. No. 6,072,504

  An object of the present invention is to provide an object capturing system suitable for a video game program, solve the above-described disadvantages in the prior art, and capture the position and direction of an object, ie, a columnar object that functions as an input device. An object is to provide an object capturing system capable of executing an action in a game program.

  Another object of the present invention is to provide an interface for a game that replaces a conventional joystick. A user can generate an action in a game by standing in front of a video camera connected to a game machine and physically moving (manipulating) an object within the field of view of the camera.

  Another object of the present invention is to map a two-dimensional information of a pixel group identified as corresponding to an manipulated object into a three-dimensional space, so that a three-dimensional object can be obtained from a single video camera image. 3D information, that is, position and direction information.

  Furthermore, another object of the present invention is to provide three-dimensional information of an object including a rotation component of the manipulated object.

  Furthermore, another object of the present invention is to clearly recognize a pixel group corresponding to the manipulated object from the video image in order to obtain information necessary for obtaining information on the three-dimensional position and orientation of the object. Thus, it is an object of the present invention to provide a technique for determining the color of an object based on the color transition so that the pixel group is most easily identified.

  A method for controlling a game program according to the present invention includes a step of manipulating a physical object in a field of view of one video camera connected to a processing unit having a memory and a display unit, and the video program captured by the video camera. Supplying the two-dimensional pixel information to the processing unit, extracting at least one pixel group corresponding to the object from the two-dimensional pixel information, and based on the at least one pixel group Obtaining a set of parameters for defining a two-dimensional geometric shape, and executing a predetermined algorithm based on the set of parameters to determine the two-dimensional geometric shape By mapping to three-dimensionally defined data with various numerical values, the position and direction of the manipulated object Obtaining the geometric numerical information in the corresponding three dimensions, storing the data defined in the three dimensions in the memory, and using the geometric numerical information on the display unit. And a step of controlling an action.

In this case, the object is a solid cylinder, the geometric shape defined by the extracted pixel group is a quadrangle, and the predetermined algorithm obtains the length l of the pixel group; Determining the widths w ₁ and w ₂ of the pixel groups of at least two different lengths along the pixel group and between the Y axis and the direction defined by the length l in the XY plane of the image A step of obtaining an inclination (angle) θ of the object, a step of obtaining an inclination φ of the object in the YZ plane based on the ratio w ₁ : w ₂ of the two widths, and the pixel group weighted by the inclination φ A depth value z of the object based on a total length Nφ of pixels constituting the pixel group weighted by the inclination φ or the length lφ of Unishi may be.

The object is a solid cylinder, the geometric shape defined by the extracted pixel group is a quadrangle, and the predetermined algorithm obtains a length l of the pixel group; Determining the widths w ₁ and w ₂ of the group of pixels of at least two different lengths along the group and between the Y axis and the direction defined by the length l in the XY plane of the image A step of obtaining an inclination (angle) θ, a step of _obtaining a depth value z of the object based on an average width w _avg of the pixel group, and a ratio l of the length l of the pixel group and the average width w _avg The step of determining the inclination φ of the object in the YZ plane based on w _avg may be included.

Further, the object includes a plain cylinder and a plain sphere attached to the cylinder, the color of the cylinder is different from the color of the sphere, and the object corresponds to the cylinder in the extracted pixel group. The geometric shape defined by the portion to be square is a square, the geometric shape defined by the portion corresponding to the sphere is a circle, and the predetermined algorithm is the sphere of the extracted pixel group. Determining the center point (point P _S ) of the circle defined by the portion corresponding to, and the pixel area of the entire circle, and defining the portion of the extracted pixel group corresponding to the cylinder. and obtaining and determining the another point P at the center line of the rectangle, the length l between the points P _S and the point P, the image In the XY plane, defined by the step of obtaining the inclination θ of the object between the Y axis and the direction defined by the length l, and the portion of the extracted pixel group corresponding to the sphere. determining a depth value z of the object, based on the total number N of pixels constituting the circle that, based on said depth value z the length l between the points P _S and the point P , Obtaining a tilt φ of the object in the YZ plane.

The object includes a plain cylinder with a plurality of plain stripes in the circumferential direction, and the color of the stripe is different from the color of the cylinder, and the cylinder and the stripe are extracted in the pixel group extraction step. A plurality of color transitions between, and the predetermined algorithm setting a threshold D _t that defines a maximum degree of separation in a two-dimensional chrominance color space; and the color of the stripe and the cylinder detecting color transitions based on a value D obtained by the synthesis chrominance signals Cr and Cb corresponding to the color, when said value D exceeds said predetermined threshold value D _t, the color transition of sufficient degree occurs and determining that the respective lengths l ₁ between the stripes by the color transition of the sufficient degree and A step of defining _2, respective lengths l ₁ and the ratio of l ₂ l ₁ between the stripes: based on l _2, determining a tilt φ of the object in the Y-Z plane, the length determining the depth value z of the object based on the sum of l ₁ and l ₂ and the inclination φ of the object in the YZ plane.

  Further, the object is formed of a plain cylinder with a spiral plain stripe, and the color of the stripe is different from the color of the cylinder, and at least one corresponding to the stripe in the pixel group extraction step. One pixel group may be extracted, and may further include a step of executing a predetermined algorithm for obtaining the degree of rotation of the object.

  In this case, the step of executing a predetermined algorithm for obtaining the degree of rotation of the object includes a step of extracting a pixel group corresponding to the cylinder and a step of defining a helix H based on the pixel group corresponding to the stripe. Determining a center line of a pixel group corresponding to the cylinder; an end of the extracted pixel group corresponding to the cylinder; and a point on the helix H at which the center line intersects the helix H. A step of obtaining a height h therebetween, and the degree of rotation of the cylinder may be obtained based on a ratio of the height h to a total length l of a pixel group corresponding to the cylinder.

Next, a color selection method according to the present invention includes an object captured using a video camera and an image processing system that detect colors using chrominance coordinates (Cr, Cb) defined in a two-dimensional color space. In the color space, an object having at least two colors defined by coordinates (Cr ₁ , Cb ₁ ) and (Cr ₂ , Cb ₂ ) is captured by the video camera in the color space. Selecting a diameter having a predetermined slope θ on a hue circle defined by the color space, and each point defined by the coordinates (Cr ₁ , Cb ₁ ) and (Cr ₂ , Cb ₂ ) Projecting a pair of lines in a direction normal to the diameter, and a distance D between each point at which the line is projected onto the diameter Calculating the calculated distance D is characterized by comprising the steps of: selecting a color which is suitable for detecting only a color exceeding a predetermined threshold.

  In this case, the distance D may be calculated by the following equation.

D = [Cr ₁ · cos θ + Cb ₁ · sin θ] − [Cr ₂ · cos θ + Cb ₂ · sin θ]
Next, an input device according to the present invention is an input device comprising a cylinder body with a stripe and a handle, and the color of the stripe is different from the color of the cylinder body, and at least the stripe It is possible to input three-dimensional data by inputting two-dimensional image data consisting of a pixel group corresponding to the pixel group and a pixel group corresponding to the cylinder body to the image processing unit via the camera. And

  In this case, the stripe may be spiral, and rotation data about the cylinder body axis may be input.

  Next, a method of inputting a motion of a 3D object according to the present invention includes a step of operating a 3D object comprising a cylinder body with a stripe and a handle, and capturing the motion of the 3D object with a camera. Obtaining two-dimensional image data comprising at least a pixel group corresponding to the cylinder body and a pixel group corresponding to the stripe, and analyzing the two-dimensional image data to obtain a three-dimensional image of the object. And a step of calculating various data.

  In this case, the stripe is spiral, and the analysis step of the two-dimensional image data analyzes the relationship between the pixel group corresponding to the cylinder body and the pixel group corresponding to the stripe to analyze the cylinder body. You may make it include the step which calculates the data regarding the rotation centering on an axis | shaft.

  Next, a system for inputting a motion of a three-dimensional object according to the present invention comprises an input device comprising a cylinder body with a stripe, a camera for capturing the motion of the input device, and an image processing unit, A camera captures the operation of the input device and supplies two-dimensional image data consisting of a pixel group corresponding to the cylinder body and a pixel group corresponding to the stripe to the image processing unit, and the image processing unit The two-dimensional image data is analyzed to calculate three-dimensional data based on the operation of the input device.

  In this case, the stripe is spiral, and the image processing unit analyzes the relationship between the pixel group corresponding to the cylinder body and the pixel group corresponding to the stripe, and relates to rotation about the axis of the cylinder body. Data may be calculated.

  The above-described objects, features, and advantages of the present invention, as well as other objects, features, and advantages will be better understood in view of the accompanying drawings in which preferred embodiments of the present invention are illustrated and the following description. I will.

  As described above, according to the present invention, the following effects can be obtained.

(1) An object capturing system suitable for a video game program, for example, an object capturing system capable of capturing an action in a game program by capturing the position and direction of an object, that is, a columnar object that functions as an input device. be able to.

(2) It is possible to provide an interface for a game that replaces a conventional joystick.

(3) By mapping the two-dimensional information of the pixel group identified as corresponding to the manipulated object to the three-dimensional space, the three-dimensional information of the object in three dimensions from the image of one video camera, That is, a location and method can be provided.

(4) It is possible to provide three-dimensional information of the object including the rotation component of the operated object.

(5) In order to obtain information necessary for obtaining information on the three-dimensional position and direction of the object, the pixel group is set so that the pixel group corresponding to the manipulated object is clearly recognized from the video image. A technique can be provided for determining the color of an object that is most easily identifiable based on the color transition.

  A preferred embodiment of a columnar input device according to the present invention will be described below and described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram illustrating a configuration of a main part of the video game machine 60. The video game machine 60 is suitable for using an operable object (columnar object) instead of a conventional input device.

  The video game machine 60 constitutes a part of the entertainment system 110 according to the present invention shown in FIG. The video game machine 60 executes a microprocessor (MPU) 112 that controls the entire entertainment system 110, executes various programs, stores data, and performs floating-point vector operations necessary for geometry processing. A vector arithmetic unit 116, an image processor 120 that generates data based on control by the MPU 112, a monitor 80 that outputs a video signal such as a CRT, and a transfer bus between the MPU 112, the vector arithmetic unit 116, and the image processor 120. In order to control a graphic interface (GIF) 122 that performs data intermediation, an input / output port 124 that enables transmission / reception of data to / from peripheral devices, a kernel, and the like, an OSD function including, for example, a flash memory is provided. Built-in Includes a ROM (OSDROM) 126, a real time clock 128 having calendar and timer functions and (RTC).

  The main memory 114, the vector operation unit 116, the GIF 122, the OSDROM 126, the RTC 128, and the input / output port 124 are connected to the MPU 112 via the data bus 130.

  Connected to the data bus 130 is a compressed image decoder (IPU) 138 that expands a moving image and a texture image compressed to acquire image data. For example, the IPU 138 decodes a bitstream according to the standard MPEG2 format to acquire data, a function to decode a macroblock, a function to perform inverse discrete cosine transform, a function to perform color space conversion, It has a function of performing vector quantization.

  The sound system includes a sound processing unit (SPU) 171 that generates musical effects or other sound effects according to instructions of the MPU 112, a sound buffer 173 in which waveform data is recorded by the SPU 171, and a musical sound generated by the SPU 171. Speaker 175 for outputting various effects or other sound effects. The speaker 175 may be incorporated in the monitor 80 or may be an external speaker connected to the audio output terminal.

  The data bus 130 is connected to a communication interface 140 having a digital data input / output function for enabling input of digital contents according to the present invention. The communication interface 140 can be connected to a modem 50, a network card, or the like. For example, user input data can be transmitted to the server terminal on the network via the communication interface 140, and status data can be received from the server terminal. The input / output port 124 records an input device 132 (also referred to as a controller) for inputting data such as key input data and coordinate data to the entertainment system 110, and various programs and data (object data, texture data, etc.). An optical disc drive 136 for reproducing the contents of the optical disc 70 such as a CD-ROM is connected.

  In the present invention, in addition to the input device 132 or as an alternative to the input device 132, a digital video camera 190 is connected to the input / output port 124. The input / output port 124 may be a single input interface or a plurality of input interfaces. The input / output port 124 may be a serial interface or a USB interface. The digital video camera 190 can be used by using this convenient USB input interface or other conventional interface.

  The image processor 120 described above includes a rendering engine 170, a memory interface 172, an image memory 174, and a display controller 176 such as a programmable CRT controller.

  The rendering engine 170 renders predetermined image data in the image memory 174 through the memory interface 172 in accordance with the rendering command given by the MPU 112.

  The first bus 178 connects between the memory interface 172 and the rendering engine 170, and the second bus 180 connects between the memory interface 172 and the image memory 174. Each of the first bus 178 and the second bus 180 has a bit width of 128 bits, for example, and the rendering engine 170 can perform rendering on the image memory 174 at high speed.

  The rendering engine 170 converts image data of 320 × 240 pixels or 640 × 480 pixels or image data compliant with, for example, a standard NTSC system or PAL system in real time, that is, every 1/60 seconds to 1/30 seconds. It is possible to render 10 to several tens of times every time.

  The image memory 174 employs a unified memory structure, and the texture rendering area and the display rendering area can be set to the same area.

  The display controller 176 writes the texture data read from the optical disk 70 by the optical disk drive 136 or the texture data created on the main memory 114 into the texture rendering area of the image memory 174 via the memory interface 172, and the image memory 174 The image data rendered in the display rendering area is read out via the memory interface 172, and the image data is output to the monitor 80 and can be displayed on the screen.

  Next, referring to FIG. 2, a configuration of the entire system in which a user having a columnar object operates the object in front of the digital video camera to generate an action in the video game will be described.

  As shown in FIG. 2, a bar-shaped object including a black handle 303 and a cylinder 301 having a bright color (for example, a saturated color) can be used as the columnar object. The user can use a digital video camera 190 such as a USB web camera (Webcam) or a digital camcorder (camcorder) connected to an input / output port 124 of a game machine such as “PlayStation 2 (trade name)” manufactured by Sony Computer Entertainment. Stand in front of. When the user moves the object in front of the digital video camera 190, the characteristics of the cylinder 301 of the object are detected by the digital video camera 190, and a process described later is performed in order to separate and identify the pixel group corresponding to only the cylinder 301. Done. First, three-dimensional data related to the cylinder (object) 301 such as the position and direction in the three-dimensional space is calculated and stored in the main memory 114 of the video game machine 60. Next, an action in the game program displayed on the screen of the monitor 80 is executed based on the three-dimensional data regarding the object by a known rendering technique. For example, a virtual object such as a torch can be moved in the game scene in response to the action of the real object by the user. When the user moves the object, the position and direction of the object are changed, and the object data stored in the main memory 114 and the rendering of the object in the rendering area of the image memory 174 are continuously updated. The virtual object, that is, the position and direction of the torch are also changed.

  As described above, the necessary information is the three-dimensional data of the object (cylinder 301 in the case of FIG. 2). However, the image captured by the digital video camera 190 is only two-dimensional pixel information about the object. In addition, before calculating the data of a three-dimensional object, it is necessary to identify pixels relating to the object itself.

  FIG. 3 is a block diagram illustrating a function used to capture and identify a pixel group corresponding to a columnar object when the columnar object is operated by a user. It will be understood that the function indicated by each block is performed by software executed by the MPU 112 of the video game machine 60. Also, not all functions shown by the blocks in FIG. 3 are used. In particular, the specification of the position by color transition is used only in the embodiments related to FIGS. 6A and 6B described later.

  First, pixel data input from the digital video camera 190 is supplied to the video game machine 60 through the input / output interface (input / output port) 124, and the following processing can be executed. As shown in FIG. 3, first, for example, when each pixel of an image is sampled for each raster, color segmentation processing (step S201) is performed. In this color subdivision process, the color of each pixel is determined, and the image is divided into various two-dimensional segments of different colors. Next, depending on the embodiment, the position is specified by color transition (step S203). In specifying the position by this color transition, a region where segments of different colors are adjacent is specified more specifically, and a position where a clear color transition has occurred in the image is defined. Next, depending on the embodiment, geometry processing (step S205) is performed. This geometry processing is performed by edge detection processing or calculation processing for obtaining region information, and contour lines, curves, and / or polygons corresponding to the edges of the target object are algebraically or geometrically. Defined. For example, in the case of the cylinder 301 illustrated in FIG. 2, the pixel region has a substantially rectangular shape corresponding to the front view of the cylinder 301. With the algebraic or geometric information of the rectangular pixel region, it is possible to define the center, width, length, and two-dimensional direction of the pixel group corresponding to only the object.

  In step S207, the three-dimensional position and orientation of the object are calculated according to the algorithm described below in connection with the preferred embodiment of the present invention.

  Finally, the three-dimensional position and direction data is processed by the Kalman filter to improve performance (step S209). This process is performed to estimate the point at which an object is scheduled to move within a certain period of time and to remove false measurements that deviate from the true data set that is presumed to be impossible. Another reason for performing Kalman filtering is that the digital video camera 190 generates an image at 30 Hz, whereas a normal display operates at 60 Hz, thus compensating for this frequency difference, ie, the gap between data. This is to control the action in the game program. Since smoothing of discrete data by Kalman filter processing is known in the field of computer vision, no further detailed description will be given.

  As shown in FIG. 4A, the columnar object is a solid, monochrome cylinder 301, preferably attached to a black handle 303. In order to completely define the position and direction of an object in a three-dimensional space, the position and depth value z (that is, on the Z axis of the point p) of a given point p (usually the center of the object) in the XY plane. And the angle information of the object in at least two different planes, eg, the tilt θ in the XY plane and the tilt φ in the YZ plane must be determined. Information about the actual physical length and diameter of the cylinder 301 and the focal length of the camera can be used for scaling, but are not necessary for programming actions in the game program. This is because the virtual object on the screen does not need to have the same length and diameter as the columnar object (cylinder 301) and may have a different shape.

  Next, a description will be given with reference to FIG. 4B. FIG. 4B shows a two-dimensional image 305 of the object generated by the digital video camera 190. The cylindrical object 301 is captured in the video image 305 as a generally rectangular pixel group 307. However, the width along the direction defined by the length l of the pixel group 307 may change because the object is affected by the inclination in the φ direction and the distance between the digital video camera 190 and the columnar object. It will be understood that the tilt of the object in the φ direction is not directly visible in the video image 305.

  In order to determine the length, center point, and the like of the pixel group 307 according to the above-described geometry processing (step S205), a known area information calculation method is used. The area information includes the area, the center of gravity, the moment about the X axis, the moment about the Y axis, other important moments, the angle of each important moment, etc. Used to calculate the moment of inertia of an object about a specific axis. For example, when obtaining a moment about the X axis and the Y axis, if each pixel constituting the pixel group corresponds to a particle of a certain uniform mass m constituting a thin homogeneous sheet or thin plate, the coordinates A moment Mx about the X axis and the Y axis in the coordinate system of n particles (pixels) existing on a plane is defined by the following equation.

  The center of mass in the coordinate system is located at the point (x, y) defined by the following equation.

  Further, assuming that the thin plate has a geometric figure center such as a rectangle shown in FIG. 4B or a circle shown in FIG. 5B described later, the center of mass of the thin plate is a geometric figure. Corresponds to the center. That is, for example, if the area information of the pixel area is known such that the two-dimensional shape of the pixel area is a rectangle, the width, height, and direction can be directly calculated. For a circular object, the same calculation can be performed when determining the center point and radius. Typical calculation methods for rectangles and circles are described in standard university-level calculus and physics texts.

  Since the video image 305 has already been expressed in the XY plane, the x coordinate and the y coordinate of the center point p can be obtained directly from the video image 305. If the information on the line L indicating the direction defined by the longitudinal axis of the pixel group 307 corresponding to the cylinder 301 is known, the value of the angle θ can be obtained using the above-described geometry processing (step S205). It can be obtained directly from the image 305. Usually, a line passing through the center point p is used as the line L in the longitudinal direction.

In order to determine the value of the slope φ, some other information is needed, i.e. information about the widths W ₁ and W ₂ of the pixel group at at least two different positions. The slope φ can be obtained by using the ratio w ₁ : w ₂ of the values of the widths w ₁ and w ₂ of the pixel group. More specifically, when the cylinder 301 is inclined such that the upper end side is closer to the digital video camera 190 than the lower end side, the lower end side of the cylinder 301 is away from the digital video camera 190, and therefore the pixel group of the image The value of the width w ₂ is larger than the value of the width w ₁ . Conversely, when the cylinder 301 is tilted so that the lower end side is closer to the digital video camera 190 than the upper end side, the upper end side of the cylinder 301 is away from the digital video camera 190, and thus the width w ₁ of the pixel group of the image. of the value is greater than the value of the width w _2. The ratio w ₂ / w ₁ between the widths w ₁ and w ₂ of the pixel group is proportional to the inclination φ of the cylinder 301 on the YZ plane. Therefore, the magnitude of the value of the slope φ can be obtained using this ratio w ₂ / w ₁ . Usually, in order to obtain a more accurate value, a plurality of equidistant measurements are performed between the edges of the pixel group 307, and the ratio w ₂ / w ₁ is obtained using the average value.

  The depth value z can be obtained by various methods. However, the size and number of pixels constituting the two-dimensional pixel group 307 are affected by both the inclination of the object in the φ direction and the actual distance from the digital video camera 190 to the object. More specifically, when the object is tilted in the φ direction, the apparent length of the object seen from the digital video camera 190 is shortened, so that the length l of the pixel group 307 is also shortened. Further, when the object moves away from the digital video camera 190 along the Z axis, the apparent size of the entire object including the length l decreases. Therefore, the length l alone cannot be regarded as an index indicating the distance between the digital video camera 190 and the object. That is, the depth value z needs to be obtained by a function of the length l and the slope φ.

  However, if the value of the slope φ is known, a value obtained by weighting l with φ can be obtained. If it is assumed that l is a value obtained by weighting l by φ and φ is constant, the pixel length lφ in the image is determined by moving the object closer to the digital video camera 190 or moving away from the digital video camera 190. Change. Since lφ is proportional to the depth value z, the depth value z can be obtained from lφ.

  Another method for obtaining the depth value z is performed by counting the total number of pixels corresponding to the object in the pixel group 307. When the object approaches the digital video camera 190 or moves away from the digital video camera 190, the number of pixels constituting the pixel group 307 increases or decreases in proportion to the depth value z. However, since the number of pixels in the pixel group 307 is affected by the inclination in the φ direction, it is necessary to first obtain a weighted value Nφ by weighting the number N of pixels with φ. Since Nφ is proportional to the depth value z, the depth value z can be obtained from Nφ.

Another effective way to determine the depth value z is to use the average width w _avg of the rectangle (pixel group). The average width w _avg of the rectangle can be obtained by measuring the width of the rectangle a specific number of times and then dividing the total measurement value by the number of measurements. It will be apparent that the average width of the pixel group is influenced not only by the depth value z but also by the cylinder tilt φ. Also, φ can be obtained from the ratio (l: _wavg ) between the total length and the average width of the pixel group. Furthermore, the slope φ can be obtained by comparing the values of the pixel widths w ₁ and w ₂ .

  FIG. 5A shows a columnar object used in another embodiment (second embodiment). FIG. 5B shows a method for obtaining the three-dimensional information of the columnar object of FIG. 5A, that is, the position and direction of the columnar object from the two-dimensional video image 305.

  The columnar object according to the second embodiment is a cylindrical (bar-shaped) object, similar to the columnar object according to the first embodiment shown in FIG. A spherical object 309 having a color different from that of the cylinder 301 is fixed. Although not shown, the end of the cylinder 301 may be slightly projected so that it can be viewed from the upper end of the spherical object 309. As will be described later, by providing the spherical object 309, the depth value z and the inclination of the object in the φ direction can be easily obtained. In order to obtain the depth value z, it is not necessary to measure the relative width of the cylinder 301, and it is not necessary to weight the inclination φ.

As shown in FIG. 5B, the pixel group 311 corresponding to the spherical object 309 is represented on the image as a two-dimensional circle. According to the present embodiment, the radius R and the center point P _S of the circle can be obtained by calculating the area information described above. Further, in this case, the total number of pixels constituting the circular pixel group 311 can be calculated from the circular pixel region. It will be appreciated that the circular pixel area increases as the spherical object 309 approaches the digital video camera 190 and decreases as the distance from the digital video camera 190 increases. Since the total number of pixels of the pixel group 311 constituting the circle is proportional to the depth value z, the depth value z can be obtained.

  It will be understood that according to the second embodiment, unlike the cylinder in the first embodiment, the shape and size of the circular pixel group are not affected by the angle of the inclination φ. That is, even when the entire object is tilted in the φ direction, the spherical object 309 and the pixel group 311 maintain substantially the same shape, and unlike the case of the cylinder 301 (length), the spherical object 309 and the pixel group 311 are shortened by the tilt. There is nothing. Accordingly, the total number of pixels of the pixel group constituting the circle in the image is always proportional to the depth value z, and weighting with the inclination φ is performed as in the first embodiment in order to obtain the depth value z. There is an advantage that becomes unnecessary.

  Further, as in the first embodiment, the inclination of the object in the θ direction can be obtained directly from the image. That is, the inclination of the object in the θ direction can be obtained from the angle θ between the longitudinal center line of the pixel group 307 corresponding to the cylinder 301 and the Y axis.

The inclination in the φ direction is obtained by a method different from the method of the first embodiment. In other words, the depth value z obtained as described above and the length l between the center point P _S of the circle 311 and the center point P of the pixel group 307 corresponding to the cylinder 301 are obtained. If the depth value z is any value, the length l viewed from the digital video camera 190 becomes shorter as the object tilts in the φ direction. Therefore, if the depth value z is known, the degree of inclination in the φ direction can be easily obtained from the length l, and the relative width as in the first embodiment shown in FIGS. 4A and 4B. It is not necessary to calculate the value or width ratio.

  FIG. 6A shows a columnar object according to still another embodiment of the present invention.

As shown in FIG. 6A, the columnar object includes a substantially cylindrical main body 301. Further, the cylinder 301 is provided with three stripes S ₁ , S ₂ and S ₃ having different colors from the cylinder 301. Preferably, the stripes S ₁ , S ₂ and S ₃ have the same width and are equally spaced at both ends and the center of the cylinder 301.

  According to this embodiment, a two-dimensional line is formed in the pixel group constituting the cylinder 301 extracted from the image, and color transition can be confirmed by this two-dimensional line. . In order to obtain the values of the depth value z, the angle θ, and the inclination φ, the position of the color transition that occurs along the line L defined by the longitudinal direction of the cylinder 301 is obtained.

That is, as shown in FIG. 6B, in order to identify the position where a clear color transition occurs in the line L, the line L along the longitudinal direction of the cylinder 301 when viewed from the digital video camera 190 is displayed. It is necessary to sample the corresponding pixel group. In particular, in order to detect such a color transition, chrominance values Cr and Cb in the YCrCb signal output from the digital video camera 190 are detected. In the criteria for selecting the color of the stripe, for the reason described below, the chrominance signals Cr and Cb of the colors of the cylinder 301 and the stripes S ₁ , S ₂ and S ₃ are synthesized using the Pythagorean distance. It is preferable to use the synthesized chrominance value D obtained by this. The synthesized chrominance value D is obtained by the following equation.

  In this way, the degree of separation in the two-dimensional chrominance color space used by the digital video camera 190 can be defined.

By later synthesized chrominance value D to select a color to become maximum, it is possible to determine the threshold D _t. Color transitions corresponding to the stripes S ₁ , S ₂ and S ₃ are detected only when a certain degree of separation is exceeded such that the value of the combined chrominance value D is greater than the threshold value D _t . Accordingly, the pixels along the line L of the cylinder 301 by using such a threshold D _t is filtered, large color transition corresponding to the stripe S _1, S ₂ and S ₃ are detected.

As shown in FIG. 6B, at each position where the color transition occurs along the line L, two spikes corresponding to the stripes S ₁ , S ₂ and S ₃ are detected. The center point of these spikes is considered to be the position of the stripe. Once the stripe positions are determined, the lengths (distances) l ₁ and l ₂ between the stripes are determined. The total length of the cylinder is the sum of l ₁ and l ₂ .

Next, based on the data of the lengths (distances) l ₁ and l ₂ between the stripes, the values of the depth value z, the angle θ, and the inclination φ necessary for obtaining information on the position and direction of the three-dimensional object are obtained. A method will be described.

  First, it is assumed that the data of the line L defined by the continuous pixels over the entire length of the cylinder 301 has already been obtained and the digital video camera 190 is oriented in the normal direction with respect to the XY plane. Then, as in the case of the above-described embodiment, the angle θ can be directly obtained by regarding the angle θ as the angle between the line L in the longitudinal direction of the cylinder 301 and the Y axis.

In order to obtain the inclination in the φ direction, the length ratio l ₁ : l ₂ is used. For example, when the cylinder 301 is inclined in the φ direction toward the digital video camera 190, the upper end of the cylinder 301 is closer to the digital video camera 190 than the lower end. That is, since the length l ₁ is closer to the digital video camera 190 than the length l ₂ , the length l ₁ is longer. The lengths l ₁ and 1 ₂ depend on the distance z between the object and the digital video camera 190, but if the angle φ is constant, the ratio of these lengths l ₁ : l ₂ is unchanged. The inclination of the cylinder 301 in the φ direction can always be obtained using this ratio l ₁ : l ₂ as an index.

The depth value z can be obtained by the same method as in the first embodiment. The depth value z can be obtained by obtaining a value lφ obtained by weighting the entire length l (l = l ₁ + l ₂ ) by φ. That is, first, the degree of influence of the inclination (angle) φ on the apparent overall length l of the object is obtained. The total length appropriately weighted by the degree of influence of the inclination (angle) φ is proportional to the distance (depth value) z between the object and the digital video camera 190, so the depth value z is obtained. be able to.

More simply, the slope φ is obtained by the ratio of the lengths l ₁ and l ₂ , and if the value of the slope φ is obtained, the depth value z can be obtained by the sum of the lengths l ₁ and l _2. .

Next, a method for obtaining the rotation component of the columnar object will be described with reference to FIG. This method can be applied to any of the above-described embodiments by attaching a spiral stripe _SH to a columnar object.

The acquisition method described above is used to obtain five of the six degrees of freedom that an object has. The remaining one degree of freedom is rotation about the axis of the cylinder 301. Since the cylinder 301 is formed symmetrically about the axis, it seems difficult to determine the rotation of the cylinder 301. Methods employed in the present invention in order to obtain the rotational component of the cylinder 301 is to use a helical stripe S _H. This stripe _SH is wound around the cylinder 301 only once. When the cylinder 301 is rotated, the height of the stripe S _H corresponds to the angle of rotation of the cylinder.

Specifically, as shown in FIG. 7, (in the case of FIGS. 5A and 5B, the cylinder portion of the prop) cylinder 301 helical stripes S _H of one to being wound once. Information about the helical stripe S _H from the entire pixel group 313 constituting the helical stripe S _H, or is extracted by using the color transitions corresponding to the helical stripe S _H. The information about the helical stripe S _H, can be obtained best fitting helix H in a stripe S _H using the above-described geometry processing.

  In addition to the helix H, the center line L of the pixel group 313 corresponding to the cylinder 301 is obtained by the method described above. Further, the total length l of the pixel group 313 is obtained.

In order to obtain the degree of rotation of the cylinder 301, the height h (only h ₁ and h ₂ are shown for simplicity of explanation) at each part is obtained. These heights h are defined by the distance between one end of the cylinder 301 and a point Pa where the center line L intersects the helix H.

As shown on the right side of FIG. 7, from the digital video camera 190 side, only one side (normally projected state) of the cylinder 301 can be seen at a time. Therefore, after extracting the region from the camera image to obtain the helix H, the rotation angle of the cylinder 301 is obtained based on this helix H. In FIG. 7, when it is assumed that there is no rotation (rotational component is 0), the first height h ₁ indicates from one end (lower end) of the cylinder 301 to a certain point Pa on the helix H. When the object rotates 45 degrees, the height h ₂ of the center line L from the lower end to the intersection at the point of the helix H becomes shorter. The rightmost part of FIG. 7 shows a case where the object is rotated 90 degrees. In this case, a special state occurs in which the center line L intersects the helix H at two points Pa and Pa. In this way, by calculating the height of the center line L, the rotation component of the cylinder 301 (or another object that rotates while being fixed to the cylinder 301) can be obtained.

A specific numerical value used for obtaining the rotation is obtained by a ratio of the height from the lower end to the point Pa on the helix H with respect to the entire length l of the pixel group 313. The rotation in the range of 0 to 360 degrees can be obtained directly by the numerical value k (k = h _max / l) obtained by using this ratio. Therefore, information on the direction of the cylinder 301 in the three-dimensional space can be obtained as additional information regarding the object. Such information can be used, for example, for controlling the rotation of the virtual object displayed by the game program.

  Next, a method for determining the color of the stripe in the embodiment shown in FIGS. 6A and 6B will be described with reference to FIGS. 8A and 8B. Specifically, FIG. 8A is a diagram showing a color space defined by luminance and radial coordinates of hue and saturation. Luminance is defined by the brightness or intensity of the color, hue is defined by the change in the dominant wavelength of the spectral distribution, and saturation is defined by the density of the spectral distribution at a wavelength.

  FIG. 8B shows a two-dimensional chrominance color space corresponding to the output signals (chrominance signals Cr and Cb) of the digital video camera 190. Those skilled in the art will understand that the digital video camera 190 outputs a signal for controlling the color of each pixel constituting the video image. As shown in the hue circle diagram of FIG. 8A, a color is defined by each radial coordinate corresponding to hue and saturation. However, it is not necessary to complicate image processing by a computer by using radial coordinates. Other criteria based on the more useful YCrCb scheme can also be used to define the color. This method is the most commonly used method for representing colors in the video industry. In the YCrCb system, each color is represented by one luma component Y and two chrominance components Cr and Cb. Y roughly corresponds to lightness and brightness, and Cr and Cb roughly correspond to hue. These components are the ITU-R Recommendation BT. 601-4 (a digital television studio encoding parameter for a standard system with an aspect ratio of 4: 3 and a wide screen system with a 16: 9 aspect ratio). Thus, the chrominance signals Cr and Cb for each pixel are defined by Cartesian coordinates, which can be used to determine the position in the hue circle corresponding to a particular hue and saturation.

According to the present invention, the colors of the stripes S ₁ , S ₂ and S _{3 and} the colors of the cylinders are determined so that the digital video camera 190 can most easily detect the stripes. Color-based capture has the major problem that the apparent color changes depending on the illumination. As a result, for example, when a blue color corresponding to an object is to be detected under a certain illumination, the blue color captured by the camera changes, making it difficult to accurately detect the object. In the present invention, since an absolute color is not detected but a color transition is detected, it is possible to capture a more reliable color. For example, in the embodiment shown in FIGS. 6A and 6B, when the color of the cylinder 301 is blue and the colors of the stripes S ₁ , S _2, and S ₃ are orange, the illumination conditions change and the apparent As the colors change, these color transitions remain clear, as shown in FIG. 6B.

  As described above, the digital video camera 190 captures data using the two-dimensional chrominance color space shown in FIG. 8B. In this color space, the color transition detection performance is greatly improved by determining the color of the object and the stripe so that the color separation degree D is maximized.

Specifically, as shown in FIG. 8B, the highly saturated blue and orange colors are located at substantially opposite ends of the hue circle, and the distance D is greatly separated in the color space. As described in the equation (4), the actual distance D is calculated based on the side ΔCr (the difference between the chrominance signal values Cr corresponding to the two colors of blue and orange) and the side ΔCb (the two colors of blue and orange). As the hypotenuse of the triangle with the difference between each corresponding chrominance signal value Cb), ie the actual distance D is calculated as the square root of (ΔCr) ² + (ΔCb) ² .

  While a blue and orange combination has been described as an example, it will be appreciated that other color combinations may be used. For example, similarly, green and magenta that are largely separated in the color space can be used. That is, this method provides a general criterion for determining colors using the chrominance signals Cr and Cb so that the color separation in the color space is maximized.

  In other words, the method for calculating the distance between two colors and the method for determining the color are such that the distance between the two colors is calculated as the distance projected on a specific spoke (diameter) in the wheel of the hue (hue ring). To be done. First, the spoke (diameter) of a wheel (hue ring) of a hue having a specific inclination (angle) θ is determined. By determining the slope of the diameter determined in the hue circle, the color transition to be detected can be determined. For example, assuming that green is (1, 1) and magenta is (-1, -1), the diameter of the spoke is set to an inclination (angle) θ of 45 degrees. The color separation (distance) is then calculated by projecting each color onto a 45 degree line. Thus, in the case of green and magenta, the calculated distance is exactly the same as the Pythagorean distance D described above. However, if the diameter (line) direction is 45 degrees, the distance between blue and orange is zero. This is because blue and orange are projected on the origin. That is, when a line having a diameter of 45 degrees is determined, green and magenta are optimal colors for detection. This is because at this diameter, green and magenta have the greatest degree of separation in color space.

Therefore, the separation degree of the two colors (Cr ₁ , Cb ₁ ) and (Cr ₂ , Cb ₂ ) at a specific angle θ determined from the range of 0 to 180 degrees is obtained by the following equation.

Therefore, the distance calculated by equation (5) is also used to set the threshold value D _t based on a predetermined direction defined by the angle theta. For example, in practice, if the color transition of an object is green and magenta, using the general distance calculation methods described above, it can be set a threshold value D _t decide the angle θ in the equation 45 degrees.

  This document has described several methods for determining the position and orientation of a real object operated in front of a video camera by mapping a two-dimensional image of the object captured by the camera into a three-dimensional space. . Three-dimensional information such as the position and direction of the object is used for controlling actions in the game program.

  One easy-to-understand example of game program control is to associate a “virtual object” that forms part of the video displayed on the game screen with the motion and position of a “real” object. It will be appreciated that such information may also be used to control the game program in other different ways that can be devised by those skilled in the art. For example, a change in the position or direction of an object due to an operation can be associated with the volume, tone, pitch, rhythm, etc. of the sound generated by the sound processor, thereby creating a music effect such as “telemin”. By using such a musical sound effect or a sound effect based on rhythm in combination with a visual effect displayed on the screen of the game machine, the effect experienced by the user (for example, a game player) is enhanced. Also good.

  It will be readily understood by those skilled in the art that the contents of the embodiments described in the present specification can be modified or modified without departing from the scope and spirit of the present invention. Therefore, the scope of the claims should not be limited to what is described in this specification, but should be construed to include all features that are reasonably judged to be equivalent by those skilled in the art. It is.

It is a block diagram which shows the structural example of the principal part of the video game machine suitable for reception of the input from a digital video camera. It is a figure which shows operation | movement of the handheld columnar object as an auxiliary input device in front of a digital video camera for generating an action on the video screen of a game program. It is a block diagram which shows the function required for the capture and identification of the columnar object operated by the user. FIG. 4A is a diagram illustrating a columnar input device according to an embodiment of the present invention, and FIG. 4B maps two-dimensional pixel data of a cylinder corresponding to the columnar input device illustrated in FIG. 4A to a three-dimensional space. It is a figure which shows a method. FIG. 5A is a diagram showing a columnar input device according to another embodiment of the present invention, and FIG. 5B is a two-dimensional pixel data of a combination of a sphere and a cylinder corresponding to the columnar input device shown in FIG. 5A. It is a figure which shows the method of mapping to 3D space. FIG. 6A is a diagram showing a columnar input device according to still another embodiment of the present invention, and FIG. 6B is a two-dimensional stripe pixel attached to a cylinder corresponding to the columnar input device shown in FIG. 6A. It is a figure which shows the method of mapping data to three-dimensional space based on the color transition of a stripe. It is a figure which shows the columnar input device with which the spiral stripe was attached | subjected, and also demonstrates the principle which calculates | requires the rotation component of the columnar input device in another embodiment of this invention. 8A and 8B are explanatory diagrams of a two-dimensional chrominance color space showing the principle for determining the color transition that is most easily detected based on the color of the stripe attached to the object to be operated.

Explanation of symbols

DESCRIPTION OF SYMBOLS 50 ... Modem 60 ... Video game machine 70 ... Optical disk 80 ... Monitor 110 ... Entertainment system 112 ... MPU
114 ... Main memory 116 ... Vector arithmetic unit 120 ... Image processor 122 ... GIF
124 ... Input / output port 126 ... OSDROM
128 ... RTC 130 ... data bus 132 ... input device 136 ... optical disk drive 138 ... IPU 140 ... communication interface 170 ... rendering engine 171 ... SPU
172 ... Memory interface 173 ... Sound buffer 174 ... Image memory 175 ... Speaker 176 ... Display controller 178 ... First bus 180 ... Second bus 190 ... Digital video camera 301 ... Cylinder 303 ... Handle 305 ... Image 307, 311, 313 ... Pixel Group 309 ... Sphere object

Claims (15)

Manipulating a physical object in the field of view of a single video camera connected to a processing unit having a memory and a display;
Providing two-dimensional pixel information captured by the video camera to the processing unit;
Extracting at least one pixel group corresponding to the object from the two-dimensional pixel information;
Determining a set of parameters for defining a two-dimensional geometric shape based on the at least one pixel group;
The manipulated object by executing a predetermined algorithm based on the set of parameters and mapping the two-dimensional geometric shape to three-dimensionally defined data by geometric numerical values Obtaining the geometric numerical information in three dimensions corresponding to the position and direction of
Storing the three-dimensionally defined data in the memory;
And a step of controlling an action on the display unit using the geometric numerical information. The method of claim 1, wherein the object is a solid cylinder, the geometric shape defined by the extracted pixel group is a quadrangle, and the predetermined algorithm is:
Obtaining a length l of the pixel group;
Determining widths w ₁ and w ₂ of the pixel groups of at least two different lengths along the pixel group;
Obtaining an inclination (angle) θ between the Y axis and the direction defined by the length l in the XY plane of the image;
Obtaining the inclination φ of the object in the YZ plane based on the ratio of the two widths w ₁ : w ₂ ;
Obtaining a depth value z of the object based on a length lφ of the pixel group weighted by the inclination φ or a total number Nφ of pixels constituting the pixel group weighted by the inclination φ. A method for controlling a game program. The method of claim 1, wherein the object is a solid cylinder, the geometric shape defined by the extracted pixel group is a quadrangle, and the predetermined algorithm is:
Obtaining a length l of the pixel group;
Determining widths w ₁ and w ₂ of the pixel groups of at least two different lengths along the pixel group;
Obtaining an inclination (angle) θ between the Y axis and the direction defined by the length l in the XY plane of the image;
_Obtaining a depth value z of the object based on an average width w _avg of the pixel group;
The ratio l of the of the pixel group and the length l and the average width w _avg: the control method of a game program based on w _avg characterized in that it comprises the step of determining the inclination φ of the object in the Y-Z plane . 2. The method according to claim 1, wherein the object includes a plain cylinder and a plain sphere attached to the cylinder, and the color of the cylinder is different from the color of the sphere. Of these, the geometric shape defined by the portion corresponding to the cylinder is a square, the geometric shape defined by the portion corresponding to the sphere is a circle, and the predetermined algorithm is:
Obtaining a center point (point P _S ) of the circle defined by a portion corresponding to the sphere in the extracted pixel group and a pixel area of the entire circle;
Obtaining another point P at the center line of the rectangle defined by the portion of the extracted pixel group corresponding to the cylinder;
Determining a length l between the points P _S and P;
Obtaining an inclination θ of the object between the Y axis and the direction defined by the length l in the XY plane of the image;
Obtaining a depth value z of the object based on a total number N of pixels constituting the circle defined by a portion corresponding to the sphere in the extracted pixel group;
A step of obtaining an inclination φ of the object in a YZ plane based on the length l and the depth value z between the point P _S and the point P. Control method. The method according to claim 1, wherein the object includes a plain cylinder provided with a plurality of plain stripes in the circumferential direction, the color of the stripe is different from the color of the cylinder, and the pixel group extraction step. A plurality of color transitions between the cylinder and the stripe are determined, and the predetermined algorithm is:
Setting a threshold D _t that defines a maximum degree of separation in a two-dimensional chrominance color space;
Detecting a color transition based on a value D determined by combined chrominance signals Cr and Cb corresponding to the color of the stripe and the color of the cylinder;
And determining that the value D when it exceeds the predetermined threshold value D _t, the color transition of sufficient degree occurs,
Defining a respective length l ₁ and l ₂ between the stripes by the sufficient degree of color transition;
Determining the inclination φ of the object in the YZ plane based on the ratio l ₁ : l ₂ of the respective lengths l ₁ and l ₂ between the stripes;
A method for controlling a game program, comprising: calculating a depth value z of the object based on a sum of the lengths l ₁ and l ₂ and an inclination φ of the object in the YZ plane.   2. The method according to claim 1, wherein the object comprises a plain cylinder with a spiral plain stripe, and the color of the stripe and the color of the cylinder are different from each other in the pixel group extraction step. A method for controlling a game program, comprising: extracting at least one pixel group corresponding to a stripe; and executing a predetermined algorithm for obtaining a rotation degree of the object. The method of claim 6, wherein the step of executing a predetermined algorithm for determining the degree of rotation of the object comprises:
Extracting a pixel group corresponding to the cylinder;
Defining a helix H based on a group of pixels corresponding to the stripe;
Obtaining a center line of a pixel group corresponding to the cylinder;
Determining a height h between an end of the extracted pixel group corresponding to the cylinder and a point on the helix H at which the center line intersects the helix H;
A method of controlling a game program, wherein the degree of rotation of the cylinder is obtained based on a ratio of the height h to a total length l of a pixel group corresponding to the cylinder. In a method of selecting a color to be applied to an object captured using a video camera and an image processing system that detects colors using chrominance coordinates (Cr, Cb) defined in a two-dimensional color space,
Capturing in the color space by the video camera an object having at least two colors defined by coordinates (Cr ₁ , Cb ₁ ) and (Cr ₂ , Cb ₂ );
Selecting a diameter having a predetermined slope θ on a hue circle defined by the color space;
Projecting a pair of lines in a direction normal to the diameter from each point defined by the coordinates (Cr ₁ , Cb ₁ ) and (Cr ₂ , Cb ₂ );
Calculating a distance D between each point at which the line is projected onto the diameter;
Selecting only a color for which the calculated distance D exceeds a predetermined threshold as a color suitable for detection. 9. The color selection method according to claim 8, wherein the distance D is calculated by the following equation.
D = [Cr ₁ · cos θ + Cb ₁ · sin θ] − [Cr ₂ · cos θ + Cb ₂ · sin θ] An input device comprising a cylinder body with a stripe and a handle,
The color of the stripe is different from the color of the cylinder body,
It is possible to input three-dimensional data by inputting two-dimensional image data consisting of at least a pixel group corresponding to the stripe and a pixel group corresponding to the cylinder body to the image processing unit via the camera. An input device characterized by that.   The input device according to claim 10, wherein the stripe is spiral, and rotation data about the axis of the cylinder body can be input. Manipulating a three-dimensional object composed of a cylinder body with a stripe and a handle;
Capturing the motion of the three-dimensional object with a camera to obtain two-dimensional image data comprising at least a pixel group corresponding to the cylinder body and a pixel group corresponding to the stripe;
Analyzing the two-dimensional image data and calculating three-dimensional data of the object, and inputting a motion of the three-dimensional object. The method of claim 12, wherein the stripe is helical.
The two-dimensional image data analysis step analyzes the relationship between the pixel group corresponding to the cylinder body and the pixel group corresponding to the stripe, and calculates data relating to rotation about the axis of the cylinder body. A method of inputting a motion of a three-dimensional object characterized by comprising steps. An input device comprising a cylinder body with a stripe;
A camera that captures the operation of the input device;
An image processing unit,
The camera captures the operation of the input device and supplies two-dimensional image data consisting of a pixel group corresponding to the cylinder body and a pixel group corresponding to the stripe to the image processing unit,
A system for inputting an operation of a three-dimensional object, wherein the image processing unit analyzes the two-dimensional image data and calculates three-dimensional data based on the operation of the input device.   15. The system according to claim 14, wherein the stripe is spiral, and the image processing unit analyzes a relationship between a pixel group corresponding to the cylinder body and a pixel group corresponding to the stripe to determine the axis of the cylinder body. A system for inputting a motion of a three-dimensional object, characterized by calculating data relating to rotation at the center.

Images