JP2012064199A - Position calculating system, position calculating device, position calculating program and position calculating method - Google Patents

Abstract

A position in a three-dimensional space can be easily specified.
A position calculation system calculates a position in a three-dimensional virtual space based on an operation on an operation device. In the position calculation system, the position in the virtual space is calculated based on the attitude of the controller device and the input position with respect to a predetermined input surface provided in the controller device. More specifically, the position calculation system may calculate the position on the surface determined according to the attitude of the controller device in the virtual space as the position in the virtual space based on the input position.
[Selection] Figure 13

Description

  The present invention relates to a position calculation system, a position calculation apparatus, a position calculation program, and a position calculation method that calculate a three-dimensional position based on an operation on an operation apparatus.

  2. Description of the Related Art Conventionally, there is a technique for inputting an arbitrary position on a screen using an input device such as a touch panel or a touch pad. For example, Patent Document 1 describes a technique for controlling the movement of a character in a game space based on a trajectory drawn by an input on a touch panel in a game device including a touch panel. According to this, the player can perform a game operation by freely specifying a position in the game space on the two-dimensional plane.

Japanese Patent No. 3770499

  In the prior art, it was possible to designate a position in a two-dimensional plane by using an input device such as a touch panel, but it was difficult to designate a position in a three-dimensional space. For example, in the technique described in Patent Document 1, it is possible to specify an arbitrary position in the game space when the game space is a two-dimensional plane, but in the case of a three-dimensional game space, the game space The position in cannot be specified properly.

  Therefore, an object of the present invention is to provide a position calculation system that can easily specify a position in a three-dimensional space.

  The present invention employs the following configurations (1) to (15) in order to solve the above problems.

(1)
An example of the present invention is a position calculation system that calculates a position in a three-dimensional virtual space based on an operation on an operation device. The position calculation system calculates a position in the virtual space based on the attitude of the controller device and the input position with respect to a predetermined input surface provided in the controller device.

  In the embodiment described later, the “position calculation system” is not limited to being realized by a plurality of devices such as a terminal device that is an operation device and a game device that is an information processing device, and is realized by a single device. Also good. For example, the position calculation system may be realized by a portable information processing device that functions as an operation device and an information processing device.

  According to the configuration of (1) above, the position in the three-dimensional space can be specified by the operation of changing the attitude of the controller device and the operation of inputting the position with respect to the input surface of the controller device. That is, according to the configuration of (1) above, the user can easily specify the position in the three-dimensional space using the operation device.

(2)
The position calculation system may include an operation device and an information processing device.
The operating device includes a position detection unit, a sensor unit, and a data transmission unit. The position detection unit detects an input position with respect to the input surface. The sensor unit outputs data for calculating the posture. The data transmission unit transmits data output from the position detection unit and the sensor unit to the information processing apparatus.
The information processing apparatus includes an attitude calculation unit and a position calculation unit. The posture calculation unit calculates the posture of the controller device based on the data output from the sensor unit. The position calculation unit calculates a position in the virtual space based on the posture and the input position.

The “operation device” is any device provided with a position detection unit and a sensor unit in addition to the terminal device 7 in an embodiment to be described later and having a function of transmitting data to the information processing device. Also good.
The “information processing device” may be any device as long as it is a device that executes processing in each of the above-described units. The information processing apparatus may be an information processing apparatus dedicated to a game or a multipurpose information processing apparatus such as a general personal computer.
The “position detection unit” may be a touch pad in addition to a touch panel in an embodiment described later. That is, the operating device does not have to include a display screen.
The “sensor unit” is a concept including an arbitrary sensor capable of calculating a posture based on the detection result, such as an inertial sensor such as an acceleration sensor or a gyro sensor, or a magnetic sensor described later.

  According to the configuration of (2) above, data from the controller device is received by the information processing device, and the information processing device calculates a three-dimensional position based on the data. According to this, since the operating device does not have to have a function of executing the position calculation process, the configuration of the operating device can be simplified, and the size and weight of the operating device can be reduced. it can.

(3)
The sensor unit may include an inertial sensor.

  With configuration (3) above, the attitude of the controller device can be easily calculated (estimated) by using the output of an inertial sensor such as an acceleration sensor or a gyro sensor.

(4)
The operating device may further include a display unit that displays an image representing the virtual space.

  According to the configuration (4) above, the image representing the virtual space is displayed on the operating device, so that the user can check the state of the virtual space with the operating device at hand.

(5)
The position detection unit may be a touch panel provided on the screen of the display unit.

  According to the configuration of (5) above, since the touch panel is provided on the screen of the operating device, the player can specify the three-dimensional position by an operation of touching the screen. Therefore, a touch input operation for designating a three-dimensional position becomes easier.

(6)
The position calculation system may include a position calculation unit that calculates a position on the surface determined according to the attitude of the controller device in the virtual space based on the input position as the position in the virtual space.

  The “surface” may be a flat surface, a curved surface, or a shape whose shape is changed according to the attitude of the operating device. In addition, the “surface” may not be displayed on the display device. That is, the “surface” may not be arranged as an object in the virtual space, or may be arranged as a transparent object.

  According to the configuration of (6) above, the user can specify a position on a surface in the virtual space by an input to the input surface of the controller device, and this surface changes according to the attitude of the controller device. To do. Therefore, the user can easily specify the position in the three-dimensional virtual space by specifying the position on the input surface while changing the attitude of the controller device. According to the configuration of (6) above, the position in the three-dimensional space can be easily specified based on the input to the planar input surface such as the touch panel or the touch pad.

(7)
The position calculation system may further include a surface setting unit that sets the surface such that the posture of the surface changes according to the posture of the controller device.

  With configuration (7) above, the user can change the posture of the surface by an operation that changes the posture of the controller device. According to this, it is possible to give the user a sense of operation as if the surface in the virtual space is actually tilted by the operation of tilting the operation device, and it is possible to provide a more intuitive and easy-to-understand operation.

(8)
The operating device may include a display unit and a touch panel. The display unit displays an image representing a virtual space including a surface area. The touch panel is provided on the screen of the display unit. At this time, the position calculation unit calculates the position so that the input position on the touch panel and the position on the surface calculated based on the input position match on the screen.

  With configuration (8) above, the user can designate (touch) the position of the display unit on the screen, thereby designating the position in the virtual space corresponding to the position. Thus, the user can specify a position in the three-dimensional space by a more intuitive and easy-to-understand operation.

(9)
The position calculation system may include a process execution unit that executes a predetermined process using a position in the virtual space based on the input position as an input.

  According to the configuration of (9) above, the position calculation system can execute various processes using the three-dimensional position input by the user as an input.

(10)
The process execution unit identifies a region including a position in the virtual space based on the input position from a plurality of regions set in the virtual space, and causes a predetermined sound output unit to output a sound corresponding to the identified region. Also good.

  The “predetermined audio output unit” may be provided in a device different from the device included in the position calculation system. That is, the position calculation system may include a predetermined audio output unit or may not include the predetermined sound output unit. For example, a predetermined information processing apparatus may include a process execution unit, and a television speaker connected to the information processing apparatus may be used as the “predetermined audio output unit”. Further, a speaker provided in the operation device may be used as the “predetermined audio output unit”.

  According to the configuration of (10) above, the user can output sound by an operation that designates a three-dimensional position in the virtual space using the operation device. Thereby, for example, an operation of playing a virtual musical instrument can be provided to the user by designating each region arranged in the virtual space.

(11)
The plurality of regions may be arranged in a substantially arc shape. At this time, the position calculation system includes a surface setting unit and a position calculation unit. The surface setting unit sets a surface in the virtual space so as to move on a plurality of regions in accordance with a change in the attitude of the controller device. The position calculation unit calculates a position on the surface based on the input position.

  According to the configuration of (11) above, the user can rotate the surface on a plurality of regions arranged in a substantially arc shape by an operation of rotating the operating device (changing the posture). According to this, both the movement of the operation device and the movement of the surface are rotational movements, and the movements of both correspond to each other, so that the operation of the operation device is more intuitive and easy to understand for the player.

(12)
The position calculation system may include an object setting unit and an image generation unit. The object setting unit sets a predetermined object in the virtual space. The image generation unit generates an image representing a virtual space including a predetermined object as an image to be displayed on a predetermined display device. At this time, the process execution unit executes the predetermined process when the position of the predetermined object is calculated as the position in the virtual space based on the input position.

  According to the configuration of (12) above, the user can perform an operation of inputting an object in the virtual space by an operation of specifying a three-dimensional position in the virtual space using the operation device. In addition, since the object to be operated is displayed on the display device, the user can easily operate the object.

(13)
The image generation unit may include a first generation unit and a second generation unit. A 1st production | generation part produces | generates the image showing the partial range of an object so that a range may change according to the attitude | position of an operating device as an image for displaying on the display apparatus with which an operating device is provided. A 2nd production | generation part produces | generates the image showing the range wider than the one part range of objects as an image for displaying on a display apparatus different from the display apparatus with which an operating device is provided.

  According to the configuration of (13) above, since the relatively enlarged image is displayed on the operation device, it is easy to perform an input operation. Furthermore, since a relatively wide range of images are displayed on another display device, the user can also know the situation around the display range of the operation device. Thus, according to the configuration of (13) above, an input system with good operability can be provided.

(14)
The position calculation system may include an object placement unit and an image generation unit. The object placement unit places an object whose posture is controlled corresponding to the posture of the controller device in the virtual space. The image generation unit generates an image representing a virtual space including the object as an image to be displayed on a predetermined display device.

  According to the configuration of (14) above, since the object whose posture changes in accordance with the posture of the operating device is displayed on the display device, the posture of the operating device can be confirmed on the screen, and the operation on the operating device can be performed. Can be performed more easily.

(15)
The operating device may include a display unit that displays a keyboard arranged in the virtual space. At this time, when the position of the keyboard is calculated as a position in the virtual space based on the input position, the position calculation system outputs a sound corresponding to the keyboard corresponding to the position to a predetermined sound output unit.

  According to the configuration of (15) above, sound is output from the sound output unit by an operation of designating a keyboard arranged in the virtual space using the operation device. According to this, the user can perform an operation of playing the keyboard instrument arranged in the virtual space using the operation device.

  Another example of the present invention is an information processing apparatus included in the position calculation system described in (1) to (15) above, or an information processing apparatus (position calculation apparatus) having the same function as the position calculation system. Also good. Further, an information processing program (position calculation program) that causes a computer to function as means equivalent to each unit in the information processing apparatus may be used. Furthermore, another example of the present invention may be a form of a position calculation method performed in the position calculation system or the information processing apparatus.

  According to the present invention, the position in the three-dimensional space can be specified by the operation for changing the attitude of the operating device and the operation for inputting the position with respect to the input surface of the operating device. The position in the three-dimensional space can be easily specified using

External view of game system 1 Block diagram showing the internal configuration of the game apparatus 3 The perspective view which shows the external appearance structure of the controller 5 The perspective view which shows the external appearance structure of the controller 5 The figure which shows the internal structure of the controller 5 The figure which shows the internal structure of the controller 5 Block diagram showing the configuration of the controller 5 The figure which shows the external appearance structure of the terminal device 7 The figure which shows the external appearance structure of the terminal device 7 The figure which shows a mode that the user hold | gripped the terminal device 7 sideways The block diagram which shows the internal structure of the terminal device 7 Explanatory drawing of the position calculation method in this embodiment The figure which shows a mode that the attitude | position of the terminal device 7 was changed from the state shown in FIG. The figure which shows the television 2 and the terminal device 7 in case a game operation is performed in the 1st game example A view of the keyboard object 91 as viewed from above in the virtual space. The figure which shows an example of the image displayed on the terminal device 7 Diagram showing various data used in game processing Main flowchart showing a flow of game processing executed in the game apparatus 3 The flowchart which shows the detailed flow of the game control process (step S3) shown in FIG. The figure which shows the television 2 and the terminal device 7 in case a game operation is performed in this game example The figure which shows the television 2 and the terminal device 7 at the time of tilting the terminal device 7 from the state shown in FIG. The figure which shows the television 2 and the terminal device 7 at the time of tilting the terminal device 7 from the state shown in FIG. The figure which shows the television 2 and the terminal device 7 at the time of rotating a figure from the state shown in FIG. Diagram showing various data used in game processing The flowchart which shows the detailed flow of the game control process (step S3) in the 2nd game example. The figure for demonstrating the method of calculating a designated position based on the input position on the input surface of the touch panel 52. The figure which shows the positional relationship of the surface 201 and the virtual camera 205 for terminals. The flowchart which shows the detailed flow of the figure movement process (step S23) shown in FIG. The flowchart which shows the flow of the game control process in the modification of the said 2nd game example. The figure which shows the movement of the surface 201 in this modification The figure which shows the movement of the surface 201 in other embodiment. The figure which shows the movement of the surface 201 in other embodiment. The figure which shows the game image displayed on the television 2 in the game which draws a figure so that it may become the same as a sample The figure which shows the game image displayed on the television 2 in the game which designates the object (coin) 225 arrange | positioned in the three-dimensional virtual space by operation with respect to a terminal device. The figure which shows the game image displayed on the television 2 in the game in which an object is produced | generated based on the figure which the player drew. The figure which shows the game image displayed on the television 2 in the game in which an object is produced | generated based on the figure which the player drew. The figure which shows the screen of the television 2 and the handheld device 9 in a 3rd game example. The figure which shows the relationship between the attitude | position of the handheld device 9 and the attitude | position of the control surface 302 The figure which shows a mode that a line is drawn on the touch panel 52 of the handheld device 9. The figure which shows the control position and control direction which are calculated on the control surface 302 when the line shown in FIG. 39 is drawn on the touch panel 52 Diagram showing various data used in game processing A flowchart showing a detailed flow of the game control process The figure which shows the screen of the television 2 and the handheld device 9 in a 4th game example. Diagram showing virtual camera and control surface in game space The flowchart which shows the detailed flow of the game control process in a 4th game example. The figure which shows a mode that the position and attitude | position of the control surface 335 change.

[1. Overall configuration of game system]
Hereinafter, a game system 1 according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is an external view of the game system 1. In FIG. 1, a game system 1 includes a stationary display device (hereinafter referred to as “TV”) 2 typified by a television receiver, a stationary game device 3, an optical disc 4, a controller 5, and a marker device. 6 and the terminal device 7. The game system 1 executes game processing in the game device 3 based on a game operation using the controller 5 and the terminal device 7, and displays a game image obtained by the game processing on the television 2 and / or the terminal device 7. is there.

  An optical disk 4 that is an example of an information storage medium that can be used interchangeably with the game apparatus 3 is detachably inserted into the game apparatus 3. The optical disc 4 stores an information processing program (typically a game program) to be executed in the game apparatus 3. An insertion slot for the optical disk 4 is provided on the front surface of the game apparatus 3. The game apparatus 3 executes the game process by reading and executing the information processing program stored in the optical disc 4 inserted into the insertion slot.

  The game apparatus 3 is connected to the television 2 via a connection cord. The television 2 displays a game image obtained by a game process executed in the game device 3. The television 2 has a speaker 2a (FIG. 2), and the speaker 2a outputs game sound obtained as a result of the game processing. In other embodiments, the game apparatus 3 and the stationary display apparatus may be integrated. The communication between the game apparatus 3 and the television 2 may be wireless communication.

  A marker device 6 is installed around the screen of the television 2 (upper side of the screen in FIG. 1). Although details will be described later, the user (player) can perform a game operation to move the controller 5, and the marker device 6 is used for the game device 3 to calculate the movement, position, posture, and the like of the controller 5. The marker device 6 includes two markers 6R and 6L at both ends thereof. The marker 6 </ b> R (same for the marker 6 </ b> L) is specifically one or more infrared LEDs (Light Emitting Diodes), and outputs infrared light toward the front of the television 2. The marker device 6 is connected to the game device 3 by wire (may be wireless), and the game device 3 can control lighting of each infrared LED provided in the marker device 6. The marker device 6 is portable, and the user can install the marker device 6 at a free position. Although FIG. 1 shows a mode in which the marker device 6 is installed on the television 2, the position and orientation in which the marker device 6 is installed are arbitrary.

  The controller 5 gives operation data representing the content of the operation performed on the own device to the game apparatus 3. The controller 5 and the game apparatus 3 can communicate with each other by wireless communication. In the present embodiment, for example, Bluetooth (registered trademark) technology is used for wireless communication between the controller 5 and the game apparatus 3. In other embodiments, the controller 5 and the game apparatus 3 may be connected by wire. In FIG. 1, the game system 1 includes one controller 5, but the game system 1 may include a plurality of controllers 5. That is, the game apparatus 3 can communicate with a plurality of controllers, and a plurality of people can play a game by using a predetermined number of controllers simultaneously. A detailed configuration of the controller 5 will be described later.

  The terminal device 7 is portable and has a size that can be held by a user. The user can use the terminal device 7 by holding it in his hand or by placing the terminal device 7 in a free position. Although the detailed configuration will be described later, the terminal device 7 includes an LCD (Liquid Crystal Display: liquid crystal display) 51 which is a display means, and input means (a touch panel 52, a gyro sensor 74 and the like which will be described later). The terminal device 7 and the game device 3 can communicate wirelessly (may be wired). The terminal device 7 receives data of an image (for example, a game image) generated by the game device 3 from the game device 3 and displays the image on the LCD 51. In the present embodiment, an LCD is used as the display device. However, the terminal device 7 may have any other display device such as a display device using EL (Electro Luminescence). . In addition, the terminal device 7 transmits operation data representing the content of the operation performed on the own device to the game device 3.

[2. Internal configuration of game device 3]
Next, the internal configuration of the game apparatus 3 will be described with reference to FIG. FIG. 2 is a block diagram showing an internal configuration of the game apparatus 3. The game apparatus 3 includes a CPU (Central Processing Unit) 10, a system LSI 11, an external main memory 12, a ROM / RTC 13, a disk drive 14, an AV-IC 15, and the like.

  The CPU 10 executes a game process by executing a game program stored on the optical disc 4, and functions as a game processor. The CPU 10 is connected to the system LSI 11. In addition to the CPU 10, an external main memory 12, a ROM / RTC 13, a disk drive 14, and an AV-IC 15 are connected to the system LSI 11. The system LSI 11 performs processing such as control of data transfer between components connected thereto, generation of an image to be displayed, and acquisition of data from an external device. The internal configuration of the system LSI 11 will be described later. The volatile external main memory 12 stores a program such as a game program read from the optical disc 4 or a game program read from the flash memory 17, or stores various data. Used as a work area and buffer area. The ROM / RTC 13 includes a ROM (so-called boot ROM) in which a program for starting the game apparatus 3 is incorporated, and a clock circuit (RTC: Real Time Clock) that counts time. The disk drive 14 reads program data, texture data, and the like from the optical disk 4 and writes the read data to an internal main memory 11e or an external main memory 12 described later.

  The system LSI 11 is provided with an input / output processor (I / O processor) 11a, a GPU (Graphics Processor Unit) 11b, a DSP (Digital Signal Processor) 11c, a VRAM (Video RAM) 11d, and an internal main memory 11e. Although not shown, these components 11a to 11e are connected to each other by an internal bus.

  The GPU 11b forms part of a drawing unit and generates an image according to a graphics command (drawing command) from the CPU 10. The VRAM 11d stores data (data such as polygon data and texture data) necessary for the GPU 11b to execute the graphics command. When an image is generated, the GPU 11b creates image data using data stored in the VRAM 11d. In the present embodiment, the game device 3 generates both an image (game image) displayed on the television 2 and an image (game image) displayed on the terminal device 7. Hereinafter, an image displayed on the television 2 may be referred to as a “television image”, and an image displayed on the terminal device 7 may be referred to as a “terminal image”.

  The DSP 11c functions as an audio processor, and generates sound data using sound data and sound waveform (tone color) data stored in the internal main memory 11e and the external main memory 12. In the present embodiment, as for the sound (game sound), both the sound output from the speaker of the television 2 and the sound output from the speaker of the terminal device 7 are generated as in the case of the game image. Hereinafter, the audio output from the television 2 may be referred to as “television audio”, and the audio output from the terminal device 7 may be referred to as “terminal audio”.

  Of the images and sounds generated by the game apparatus 3 as described above, image and sound data output by the television 2 is read by the AV-IC 15. The AV-IC 15 outputs the read image data to the television 2 via the AV connector 16, and outputs the read audio data to the speaker 2 a built in the television 2. Thus, an image is displayed on the television 2 and a sound is output from the speaker 2a. The game apparatus 3 and the television 2 may be connected by any method, but the game apparatus 3 transmits a control command for controlling the television 2 to the television 2 by wire or wirelessly. Also good. For example, an HDMI cable conforming to the HDMI (High-Definition Multimedia Interface) standard may be used. In the HDMI standard, it is possible to control a connected device by a function called CEC (Consumer Electronics Control). Accordingly, when the game apparatus 3 can control the television 2 as in the case where an HDMI cable is used, the game apparatus 3 turns on the television 2 at an appropriate timing or inputs the television 2. Can be switched.

  Of the images and sounds generated by the game apparatus 3, the image and sound data output from the terminal apparatus 7 is transmitted to the terminal apparatus 7 by the input / output processor 11a and the like. Data transmission to the terminal device 7 by the input / output processor 11a and the like will be described later.

  The input / output processor 11a performs transmission / reception of data to / from components connected to the input / output processor 11a and downloads data from an external device. The input / output processor 11a is connected to the flash memory 17, the network communication module 18, the controller communication module 19, the expansion connector 20, the memory card connector 21, and the codec LSI 27. An antenna 22 is connected to the network communication module 18. An antenna 23 is connected to the controller communication module 19. The codec LSI 27 is connected to a terminal communication module 28, and an antenna 29 is connected to the terminal communication module 28.

  The game device 3 can connect to a network such as the Internet and communicate with an external information processing device (for example, another game device or various servers). In other words, the input / output processor 11a can be connected to a network such as the Internet via the network communication module 18 and the antenna 22, and can communicate with other information processing apparatuses connected to the network. The input / output processor 11a periodically accesses the flash memory 17 to detect the presence / absence of data that needs to be transmitted to the network. If there is such data, the input / output processor 11a communicates with the network via the network communication module 18 and the antenna 22. Send. Further, the input / output processor 11a receives data transmitted from the external information processing apparatus or data downloaded from the download server via the network, the antenna 22 and the network communication module 18, and receives the received data in the flash memory 17. Remember. By executing the game program, the CPU 10 reads out the data stored in the flash memory 17 and uses it in the game program. In addition to data transmitted and received between the game apparatus 3 and the external information processing apparatus, the flash memory 17 stores game save data (game result data or intermediate data) played using the game apparatus 3. May be. The flash memory 17 may store a game program.

  The game apparatus 3 can receive operation data from the controller 5. That is, the input / output processor 11a receives the operation data transmitted from the controller 5 via the antenna 23 and the controller communication module 19, and stores (temporarily stores) it in the buffer area of the internal main memory 11e or the external main memory 12.

  Further, the game apparatus 3 can transmit and receive data such as images and sounds to and from the terminal device 7. When transmitting a game image (terminal image) to the terminal device 7, the input / output processor 11 a outputs the game image data generated by the GPU 11 b to the codec LSI 27. The codec LSI 27 performs predetermined compression processing on the image data from the input / output processor 11a. The terminal communication module 28 performs wireless communication with the terminal device 7. Therefore, the image data compressed by the codec LSI 27 is transmitted to the terminal device 7 via the antenna 29 by the terminal communication module 28. In the present embodiment, the image data transmitted from the game apparatus 3 to the terminal apparatus 7 is used for the game, and if the displayed image is delayed in the game, the operability of the game is adversely affected. For this reason, it is preferable that the transmission of image data from the game apparatus 3 to the terminal device 7 is as little as possible. Therefore, in this embodiment, the codec LSI 27 is, for example, H.264. The image data is compressed using a highly efficient compression technique such as H.264 standard. Other compression techniques may be used, and when the communication speed is sufficient, the image data may be transmitted without compression. The terminal communication module 28 is a communication module that has received, for example, Wi-Fi authentication. For example, the terminal communication module 28 uses a MIMO (Multiple Input Multiple Output) technique adopted in the IEEE802.11n standard. Wireless communication may be performed at high speed, or another communication method may be used.

  In addition to the image data, the game apparatus 3 transmits audio data to the terminal device 7. That is, the input / output processor 11 a outputs the audio data generated by the DSP 11 c to the terminal communication module 28 via the codec LSI 27. The codec LSI 27 performs compression processing on the audio data in the same manner as the image data. The compression method for the audio data may be any method, but a method with a high compression rate and less deterioration of the sound is preferable. In other embodiments, audio data may be transmitted without being compressed. The terminal communication module 28 transmits the compressed image data and audio data to the terminal device 7 via the antenna 29.

  Further, the game apparatus 3 transmits various control data to the terminal apparatus 7 as necessary in addition to the image data and the sound data. The control data is data representing a control instruction for a component included in the terminal device 7, and for example, an instruction to control lighting of the marker unit (marker unit 55 shown in FIG. 11) or a camera (camera 56 shown in FIG. 11). Indicates an instruction to control imaging. The input / output processor 11 a transmits control data to the terminal device 7 in accordance with an instruction from the CPU 10. With respect to this control data, the codec LSI 27 does not perform data compression processing in the present embodiment, but may perform compression processing in other embodiments. Note that the above-described data transmitted from the game device 3 to the terminal device 7 may or may not be encrypted as necessary.

  The game apparatus 3 can receive various data from the terminal device 7. Although details will be described later, in the present embodiment, the terminal device 7 transmits operation data, image data, and audio data. Each data transmitted from the terminal device 7 is received by the terminal communication module 28 via the antenna 29. Here, the image data and audio data from the terminal device 7 are subjected to the same compression processing as the image data and audio data from the game device 3 to the terminal device 7. Therefore, these image data and audio data are sent from the terminal communication module 28 to the codec LSI 27, subjected to expansion processing by the codec LSI 27, and output to the input / output processor 11a. On the other hand, the operation data from the terminal device 7 has a smaller amount of data than images and sounds, and therefore may not be subjected to compression processing. Further, encryption may or may not be performed as necessary. Accordingly, the operation data is received by the terminal communication module 28 and then output to the input / output processor 11 a via the codec LSI 27. The input / output processor 11a stores (temporarily stores) the data received from the terminal device 7 in the buffer area of the internal main memory 11e or the external main memory 12.

  Further, the game apparatus 3 can be connected to another device or an external storage medium. That is, the expansion connector 20 and the memory card connector 21 are connected to the input / output processor 11a. The expansion connector 20 is a connector for an interface such as USB or SCSI. A network such as an external storage medium is connected to the expansion connector 20, a peripheral device such as another controller is connected, or a wired communication connector is connected to replace the network communication module 18 with a network. You can communicate with. The memory card connector 21 is a connector for connecting an external storage medium such as a memory card. For example, the input / output processor 11a can access an external storage medium via the expansion connector 20 or the memory card connector 21 to store data in the external storage medium or read data from the external storage medium.

  The game apparatus 3 is provided with a power button 24, a reset button 25, and an eject button 26. The power button 24 and the reset button 25 are connected to the system LSI 11. When the power button 24 is turned on, power is supplied to each component of the game apparatus 3 from an external power source by an AC adapter (not shown). When the reset button 25 is pressed, the system LSI 11 restarts the boot program for the game apparatus 3. The eject button 26 is connected to the disk drive 14. When the eject button 26 is pressed, the optical disk 4 is ejected from the disk drive 14.

  In other embodiments, some of the components included in the game apparatus 3 may be configured as expansion devices that are separate from the game apparatus 3. At this time, the expansion device may be connected to the game apparatus 3 via the expansion connector 20, for example. Specifically, the extension device includes, for example, each component of the codec LSI 27, the terminal communication module 28, and the antenna 29, and may be detachable from the extension connector 20. According to this, the said game device can be set as the structure which can communicate with the terminal device 7 by connecting the said expansion apparatus with respect to the game device which is not provided with said each component.

[3. Configuration of controller 5]
Next, the controller 5 will be described with reference to FIGS. FIG. 3 is a perspective view showing an external configuration of the controller 5. FIG. 4 is a perspective view showing an external configuration of the controller 5. 3 is a perspective view of the controller 5 as seen from the upper rear side, and FIG. 4 is a perspective view of the controller 5 as seen from the lower front side.

  3 and 4, the controller 5 includes a housing 31 formed by plastic molding, for example. The housing 31 has a substantially rectangular parallelepiped shape whose longitudinal direction is the front-rear direction (the Z-axis direction shown in FIG. 3), and is a size that can be gripped with one hand of an adult or a child as a whole. The user can perform a game operation by pressing a button provided on the controller 5 and moving the controller 5 itself to change its position and posture (tilt).

  The housing 31 is provided with a plurality of operation buttons. As shown in FIG. 3, a cross button 32a, a first button 32b, a second button 32c, an A button 32d, a minus button 32e, a home button 32f, a plus button 32g, and a power button 32h are provided on the upper surface of the housing 31. It is done. In the present specification, the upper surface of the housing 31 on which these buttons 32a to 32h are provided may be referred to as a “button surface”. On the other hand, as shown in FIG. 4, a recess is formed on the lower surface of the housing 31, and a B button 32i is provided on the rear inclined surface of the recess. A function corresponding to the information processing program executed by the game apparatus 3 is appropriately assigned to each of the operation buttons 32a to 32i. The power button 32h is for remotely turning on / off the main body of the game apparatus 3. The home button 32 f and the power button 32 h are embedded in the upper surface of the housing 31. Thereby, it is possible to prevent the user from pressing the home button 32f or the power button 32h by mistake.

  A connector 33 is provided on the rear surface of the housing 31. The connector 33 is used to connect another device (for example, another sensor unit or controller) to the controller 5. Further, locking holes 33a are provided on both sides of the connector 33 on the rear surface of the housing 31 in order to prevent the other devices from being easily detached.

  A plurality (four in FIG. 3) of LEDs 34 a to 34 d are provided behind the upper surface of the housing 31. Here, the controller type (number) is assigned to the controller 5 to distinguish it from other controllers. The LEDs 34a to 34d are used for the purpose of notifying the user of the controller type currently set in the controller 5 and notifying the user of the remaining battery level of the controller 5. Specifically, when a game operation is performed using the controller 5, any one of the plurality of LEDs 34a to 34d is turned on according to the controller type.

  Further, the controller 5 has an imaging information calculation unit 35 (FIG. 6), and a light incident surface 35a of the imaging information calculation unit 35 is provided on the front surface of the housing 31, as shown in FIG. The light incident surface 35a is made of a material that transmits at least infrared light from the markers 6R and 6L.

  A sound release hole 31a is formed between the first button 32b and the home button 32f on the upper surface of the housing 31 for emitting sound from the speaker 47 (FIG. 5) built in the controller 5 to the outside.

  Next, the internal structure of the controller 5 will be described with reference to FIGS. 5 and 6 are diagrams showing the internal structure of the controller 5. FIG. FIG. 5 is a perspective view showing a state in which the upper housing (a part of the housing 31) of the controller 5 is removed. FIG. 6 is a perspective view showing a state in which the lower casing (a part of the housing 31) of the controller 5 is removed. The perspective view shown in FIG. 6 is a perspective view of the substrate 30 shown in FIG.

  In FIG. 5, a substrate 30 is fixed inside the housing 31, and operation buttons 32 a to 32 h, LEDs 34 a to 34 d, an acceleration sensor 37, an antenna 45, and a speaker 47 are provided on the upper main surface of the substrate 30. Etc. are provided. These are connected to a microcomputer (microcomputer) 42 (see FIG. 6) by wiring (not shown) formed on the substrate 30 and the like. In the present embodiment, the acceleration sensor 37 is disposed at a position shifted from the center of the controller 5 with respect to the X-axis direction. This makes it easier to calculate the movement of the controller 5 when the controller 5 is rotated about the Z axis. The acceleration sensor 37 is disposed in front of the center of the controller 5 in the longitudinal direction (Z-axis direction). Further, the controller 5 functions as a wireless controller by the wireless module 44 (FIG. 6) and the antenna 45.

  On the other hand, in FIG. 6, an imaging information calculation unit 35 is provided at the front edge on the lower main surface of the substrate 30. The imaging information calculation unit 35 includes an infrared filter 38, a lens 39, an imaging element 40, and an image processing circuit 41 in order from the front of the controller 5. These members 38 to 41 are respectively attached to the lower main surface of the substrate 30.

  Further, the microcomputer 42 and the vibrator 46 are provided on the lower main surface of the substrate 30. The vibrator 46 is, for example, a vibration motor or a solenoid, and is connected to the microcomputer 42 by wiring formed on the substrate 30 or the like. When the vibrator 46 is actuated by an instruction from the microcomputer 42, vibration is generated in the controller 5. As a result, a so-called vibration-compatible game in which the vibration is transmitted to the user's hand holding the controller 5 can be realized. In the present embodiment, the vibrator 46 is disposed slightly forward of the housing 31. That is, by arranging the vibrator 46 on the end side of the center of the controller 5, the entire controller 5 can be vibrated greatly by the vibration of the vibrator 46. The connector 33 is attached to the rear edge on the lower main surface of the substrate 30. 5 and 6, the controller 5 includes a crystal resonator that generates a basic clock of the microcomputer 42, an amplifier that outputs an audio signal to the speaker 47, and the like.

  The shape of the controller 5, the shape of each operation button, the number of acceleration sensors and vibrators, and the installation positions shown in FIGS. 3 to 6 are merely examples, and other shapes, numbers, and installation positions may be used. May be. In the present embodiment, the imaging direction by the imaging unit is the positive Z-axis direction, but the imaging direction may be any direction. That is, the position of the imaging information calculation unit 35 in the controller 5 (the light incident surface 35a of the imaging information calculation unit 35) does not have to be the front surface of the housing 31, and other surfaces can be used as long as light can be taken in from the outside of the housing 31. May be provided.

  FIG. 7 is a block diagram showing the configuration of the controller 5. The controller 5 includes an operation unit 32 (operation buttons 32a to 32i), an imaging information calculation unit 35, a communication unit 36, an acceleration sensor 37, and a gyro sensor 48. The controller 5 transmits data representing the content of the operation performed on the own device to the game apparatus 3 as operation data. Hereinafter, the operation data transmitted from the controller 5 may be referred to as “controller operation data”, and the operation data transmitted from the terminal device 7 may be referred to as “terminal operation data”.

  The operation unit 32 includes the operation buttons 32a to 32i described above, and the operation button data indicating the input state (whether or not each operation button 32a to 32i is pressed) to each operation button 32a to 32i is transmitted to the microcomputer of the communication unit 36. Output to 42.

  The imaging information calculation unit 35 is a system for analyzing the image data captured by the imaging unit, discriminating a region having a high luminance in the image data, and calculating a center of gravity position, a size, and the like of the region. Since the imaging information calculation unit 35 has a sampling period of, for example, about 200 frames / second at the maximum, it can track and analyze even a relatively fast movement of the controller 5.

  The imaging information calculation unit 35 includes an infrared filter 38, a lens 39, an imaging element 40, and an image processing circuit 41. The infrared filter 38 passes only infrared rays from the light incident from the front of the controller 5. The lens 39 collects the infrared light transmitted through the infrared filter 38 and makes it incident on the image sensor 40. The image sensor 40 is a solid-state image sensor such as a CMOS sensor or a CCD sensor, for example, and receives the infrared light collected by the lens 39 and outputs an image signal. Here, the marker unit 55 and the marker device 6 of the terminal device 7 to be imaged are configured by a marker that outputs infrared light. Therefore, by providing the infrared filter 38, the image sensor 40 receives only the infrared light that has passed through the infrared filter 38 and generates image data. Therefore, the image of the imaging target (the marker unit 55 and / or the marker device 6) is captured. More accurate imaging can be performed. Hereinafter, an image captured by the image sensor 40 is referred to as a captured image. Image data generated by the image sensor 40 is processed by the image processing circuit 41. The image processing circuit 41 calculates the position of the imaging target in the captured image. The image processing circuit 41 outputs coordinates indicating the calculated position to the microcomputer 42 of the communication unit 36. The coordinate data is transmitted to the game apparatus 3 as operation data by the microcomputer 42. Hereinafter, the coordinates are referred to as “marker coordinates”. Since the marker coordinates change corresponding to the direction (tilt angle) and position of the controller 5 itself, the game apparatus 3 can calculate the direction and position of the controller 5 using the marker coordinates.

  In other embodiments, the controller 5 may not include the image processing circuit 41, and the captured image itself may be transmitted from the controller 5 to the game apparatus 3. At this time, the game apparatus 3 may have a circuit or a program having the same function as the image processing circuit 41, and may calculate the marker coordinates.

  The acceleration sensor 37 detects the acceleration (including gravity acceleration) of the controller 5, that is, detects the force (including gravity) applied to the controller 5. The acceleration sensor 37 detects the value of the acceleration (linear acceleration) in the linear direction along the sensing axis direction among the accelerations applied to the detection unit of the acceleration sensor 37. For example, in the case of a multi-axis acceleration sensor having two or more axes, the component acceleration along each axis is detected as the acceleration applied to the detection unit of the acceleration sensor. The acceleration sensor 37 is, for example, a capacitive MEMS (Micro Electro Mechanical System) type acceleration sensor, but other types of acceleration sensors may be used.

  In the present embodiment, the acceleration sensor 37 has a vertical direction (Y-axis direction shown in FIG. 3), a horizontal direction (X-axis direction shown in FIG. 3), and a front-back direction (Z-axis direction shown in FIG. 3) with reference to the controller 5. ) Linear acceleration is detected in each of the three axis directions. Since the acceleration sensor 37 detects acceleration in the linear direction along each axis, the output from the acceleration sensor 37 represents the linear acceleration value of each of the three axes. That is, the detected acceleration is represented as a three-dimensional vector in an XYZ coordinate system (controller coordinate system) set with the controller 5 as a reference.

  Data representing the acceleration detected by the acceleration sensor 37 (acceleration data) is output to the communication unit 36. Note that the acceleration detected by the acceleration sensor 37 changes corresponding to the direction (inclination angle) and movement of the controller 5 itself, so the game apparatus 3 calculates the direction and movement of the controller 5 using the acquired acceleration data. can do. In the present embodiment, the game apparatus 3 calculates the attitude, tilt angle, and the like of the controller 5 based on the acquired acceleration data.

  A computer such as a processor of the game apparatus 3 (for example, the CPU 10) or a processor of the controller 5 (for example, the microcomputer 42) is processed based on an acceleration signal output from the acceleration sensor 37 (the same applies to an acceleration sensor 73 described later). It can be easily understood by those skilled in the art from the description of the present specification that by performing the above, it is possible to estimate or calculate (determine) further information regarding the controller 5. For example, when processing on the computer side is executed on the assumption that the controller 5 on which the acceleration sensor 37 is mounted is stationary (that is, the processing is executed assuming that the acceleration detected by the acceleration sensor is only gravitational acceleration). When the controller 5 is actually stationary, it can be determined whether or not the attitude of the controller 5 is inclined with respect to the direction of gravity based on the detected acceleration. Specifically, whether or not the controller 5 is inclined with respect to the reference depending on whether or not 1G (gravity acceleration) is applied, based on the state in which the detection axis of the acceleration sensor 37 is directed vertically downward. It is possible to know how much it is inclined with respect to the reference according to its size. Further, in the case of the multi-axis acceleration sensor 37, it is possible to know in detail how much the controller 5 is inclined with respect to the direction of gravity by further processing the acceleration signal of each axis. . In this case, the processor may calculate the tilt angle of the controller 5 based on the output from the acceleration sensor 37, or may calculate the tilt direction of the controller 5 without calculating the tilt angle. Good. Thus, by using the acceleration sensor 37 in combination with the processor, the tilt angle or posture of the controller 5 can be determined.

  On the other hand, when it is assumed that the controller 5 is in a dynamic state (a state where the controller 5 is moved), the acceleration sensor 37 detects an acceleration corresponding to the movement of the controller 5 in addition to the gravitational acceleration. Therefore, the moving direction of the controller 5 can be known by removing the gravitational acceleration component from the detected acceleration by a predetermined process. Even if it is assumed that the controller 5 is in a dynamic state, the direction of gravity is obtained by removing the acceleration component corresponding to the movement of the acceleration sensor from the detected acceleration by a predetermined process. It is possible to know the inclination of the controller 5 with respect to. In another embodiment, the acceleration sensor 37 is a built-in process for performing a predetermined process on the acceleration signal before outputting the acceleration signal detected by the built-in acceleration detection means to the microcomputer 42. An apparatus or other type of dedicated processing apparatus may be provided. A built-in or dedicated processing device converts the acceleration signal into a tilt angle (or other preferred parameter) if, for example, the acceleration sensor 37 is used to detect static acceleration (eg, gravitational acceleration). It may be a thing.

  The gyro sensor 48 detects angular velocities about three axes (XYZ axes in the present embodiment). In this specification, with the imaging direction (Z-axis positive direction) of the controller 5 as a reference, the rotation direction around the X axis is the pitch direction, the rotation direction around the Y axis is the yaw direction, and the rotation direction around the Z axis is the roll direction. Call. The gyro sensor 48 only needs to be able to detect angular velocities about three axes, and any number and combination of gyro sensors may be used. For example, the gyro sensor 48 may be a three-axis gyro sensor or a combination of a two-axis gyro sensor and a one-axis gyro sensor to detect an angular velocity around three axes. Data representing the angular velocity detected by the gyro sensor 48 is output to the communication unit 36. Further, the gyro sensor 48 may detect an angular velocity around one axis or two axes.

  The communication unit 36 includes a microcomputer 42, a memory 43, a wireless module 44, and an antenna 45. The microcomputer 42 controls the wireless module 44 that wirelessly transmits data acquired by the microcomputer 42 to the game apparatus 3 while using the memory 43 as a storage area when performing processing.

  Data output from the operation unit 32, the imaging information calculation unit 35, the acceleration sensor 37, and the gyro sensor 48 to the microcomputer 42 is temporarily stored in the memory 43. These data are transmitted to the game apparatus 3 as operation data (controller operation data). That is, the microcomputer 42 outputs the operation data stored in the memory 43 to the wireless module 44 when the transmission timing to the controller communication module 19 of the game apparatus 3 arrives. The wireless module 44 modulates a carrier wave of a predetermined frequency with operation data using, for example, Bluetooth (registered trademark) technology, and radiates a weak radio signal from the antenna 45. That is, the operation data is modulated by the wireless module 44 into a weak radio signal and transmitted from the controller 5. The weak radio signal is received by the controller communication module 19 on the game apparatus 3 side. By demodulating and decoding the received weak radio signal, the game apparatus 3 can acquire operation data. Then, the CPU 10 of the game apparatus 3 performs a game process using the operation data acquired from the controller 5. Note that wireless transmission from the communication unit 36 to the controller communication module 19 is sequentially performed at predetermined intervals, but game processing is generally performed in units of 1/60 seconds (one frame time). Therefore, it is preferable to perform transmission at a period equal to or shorter than this time. The communication unit 36 of the controller 5 outputs the operation data to the controller communication module 19 of the game apparatus 3 at a rate of once every 1/200 seconds, for example.

  As described above, the controller 5 can transmit the marker coordinate data, the acceleration data, the angular velocity data, and the operation button data as the operation data representing the operation on the own device. Further, the game apparatus 3 executes a game process using the operation data as a game input. Therefore, by using the controller 5, the user can perform a game operation for moving the controller 5 itself in addition to the conventional general game operation for pressing each operation button. For example, an operation of tilting the controller 5 to an arbitrary posture, an operation of instructing an arbitrary position on the screen by the controller 5, an operation of moving the controller 5 itself, and the like can be performed.

  In the present embodiment, the controller 5 does not have a display unit that displays a game image, but may include a display unit that displays, for example, an image representing the remaining battery level.

[4. Configuration of Terminal Device 7]
Next, the configuration of the terminal device 7 will be described with reference to FIGS. FIG. 8 is a plan view showing an external configuration of the terminal device 7. 8A is a front view of the terminal device 7, FIG. 8B is a top view, FIG. 8C is a right side view, and FIG. 8D is a bottom view. FIG. 9 is a rear view of the terminal device 7. FIG. 10 is a diagram illustrating a state in which the user holds the terminal device 7 sideways.

  As shown in FIG. 8, the terminal device 7 includes a housing 50 that is generally a horizontally-long rectangular plate shape. That is, it can be said that the terminal device 7 is a tablet-type information processing device. The housing 50 may have a curved surface as long as it has a plate shape as a whole, or may have a protrusion or the like in part. The housing 50 is large enough to be gripped by the user. Therefore, the user can move the terminal apparatus 7 or change the arrangement position of the terminal apparatus 7. The length of the terminal device 7 in the vertical direction (z-axis direction) is preferably 100 to 150 [mm]. In the present embodiment, the length is 133.5 [mm]. The lateral (x-axis direction) length of the terminal device 7 is preferably 200 to 250 [mm], and in this embodiment is 228.26 [mm]. The thickness (length in the y-axis direction) of the terminal device 7 is preferably about 15 to 30 [mm] at the plate-like portion, and about 30 to 50 [mm] including the thickest portion. In this embodiment, 23.6 (the thickest part is 40.26) [mm]. The weight of the terminal device 7 is about 400 to 600 [g], and is 530 [g] in the present embodiment. Although details will be described later, even if the terminal device 7 is a relatively large terminal device (operation device) as described above, the user device is easy to hold and operate.

  The terminal device 7 has an LCD 51 on the surface (front side) of the housing 50. The screen size of the LCD 51 is preferably 5 inches or more, and here is 6.2 inches. The terminal device 7 of the present embodiment is easy to operate even if a large LCD is provided due to the configuration that it is easy to hold and operate. In another embodiment, a smaller LCD 51 may be provided so that the size of the terminal device 7 is relatively small. The LCD 51 is provided near the center of the surface of the housing 50. Therefore, the user can move the terminal device 7 while viewing the screen of the LCD 51 by holding the housings 50 on both sides of the LCD 51 as shown in FIG. FIG. 10 shows an example in which the user holds the terminal device 7 horizontally by holding the housings 50 on both the left and right sides of the LCD 51 (in a long horizontal direction), but the terminal device 7 is held vertically. It can also be held (with a long vertical orientation).

  As illustrated in FIG. 8A, the terminal device 7 includes a touch panel 52 on the screen of the LCD 51 as an operation unit. In the present embodiment, the touch panel 52 is a resistive film type touch panel. However, the touch panel is not limited to the resistive film type, and any type of touch panel such as a capacitance type can be used. The touch panel 52 may be a single touch method or a multi-touch method. In the present embodiment, the touch panel 52 having the same resolution (detection accuracy) as the resolution of the LCD 51 is used. However, the resolution of the touch panel 52 and the resolution of the LCD 51 are not necessarily matched. The input to the touch panel 52 is normally performed using the touch pen 60, but the input to the touch panel 52 is not limited to the touch pen 60 and can be performed by a user's finger. The housing 50 is provided with a storage hole 60a for storing the touch pen 60 used for operating the touch panel 52 (see FIG. 8B). Here, the storage hole 60a is provided on the upper surface of the housing 50 so that the touch pen 60 does not fall, but may be provided on the side surface or the lower surface. Thus, since the terminal device 7 includes the touch panel 52, the user can operate the touch panel 52 while moving the terminal device 7. That is, the user can directly input (by the touch panel 52) to the screen while moving the screen of the LCD 51.

  As shown in FIG. 8, the terminal device 7 includes two analog sticks 53A and 53B and a plurality of buttons (keys) 54A to 54M as operation means (operation unit). Each analog stick 53A and 53B is a device capable of indicating a direction. In each of the analog sticks 53A and 53B, a movable member (stick part) operated by a user's finger can slide in any direction (any angle in the up / down / left / right and diagonal directions) with respect to the surface of the housing 50. It is configured. That is, it is a directional input device sometimes called a slide pad. The movable members of the analog sticks 53A and 53B may be of a type that tilts in an arbitrary direction with respect to the surface of the housing 50. In the present embodiment, an analog stick of a type in which the movable member slides is used. Therefore, the user can operate each of the analog sticks 53A and 53B without moving the thumb greatly, and the housing 50 is held firmly. The operation can be performed. When the analog sticks 53A and 53B are of a type in which the movable member tilts, the degree of input (degree of tilt) is easy for the user to understand, and detailed operations can be performed more easily.

  The left analog stick 53A is provided on the left side of the screen of the LCD 51, and the right analog stick 53B is provided on the right side of the screen of the LCD 51. Therefore, the user can make an input for instructing the direction with the left and right hands using the analog stick. Also, as shown in FIG. 10, each analog stick 53A and 53B is provided at a position where the user can operate with the left and right portions of the terminal device 7 (the left and right side portions of the LCD 51) being gripped. Even when the terminal device 7 is held and moved, the analog sticks 53A and 53B can be easily operated.

  Each button 54A to 54L is an operation means (operation unit) for performing a predetermined input, and is a key that can be pressed. As shown below, each button 54A-54L is provided in the position which a user can operate in the state which hold | gripped the left-right part of the terminal device 7 (refer FIG. 10). Therefore, the user can easily operate these operation means even when the user moves the terminal device 7.

  As shown in FIG. 8A, a cross button (direction input button) 54A and buttons 54B to 54H and 54M among the operation buttons 54A to 54L are provided on the surface of the housing 50. That is, these buttons 54A to 54H and 54M are arranged at positions where they can be operated with the user's thumb (see FIG. 10).

  The cross button 54A is provided on the left side of the LCD 51 and below the left analog stick 53A. That is, the cross button 54A is arranged at a position where it can be operated with the left hand of the user. The cross button 54A has a cross shape, and is a button capable of instructing at least the vertical and horizontal directions.

  The buttons 54B to 54D are provided below the LCD 51. These three buttons 54B to 54D are arranged at positions that can be operated by both the left and right hands. The terminal device 7 has a power button 54M for turning on / off the power of the terminal device 7. It is also possible to remotely turn on / off the game apparatus 3 by operating the power button 54M. The power button 54M is provided on the lower side of the LCD 51 like the buttons 54B to 54D. The power button 54M is provided on the right side of the buttons 54B to 54D. Therefore, the power button 54M is disposed at a position where it can be operated (easy to operate) with the right hand. The four buttons 54E to 54H are provided on the right side of the LCD 51 and below the right analog stick 53B. That is, the four buttons 54E to 54H are arranged at positions that can be operated with the right hand of the user. Further, the four buttons 54E to 54H are arranged so as to have a vertical / left / right positional relationship (relative to the center position of the four buttons 54E to 54H). Therefore, the terminal device 7 can also function the four buttons 54 </ b> E to 54 </ b> H as buttons for instructing the user in the up / down / left / right directions.

  In the present embodiment, the analog sticks 53A and 53B are arranged above the cross button 54A and the buttons 54E to 54H. Here, each of the analog sticks 53A and 53B protrudes in the thickness direction (y-axis direction) from the cross button 54A and the buttons 54E to 54H. Therefore, if the arrangement of the analog stick 53A and the cross button 54A is reversed, when the user operates the cross button 54A with the thumb, the thumb may hit the analog stick 53A and the operation may be mistaken. is there. The same problem occurs when the arrangement of the analog stick 53B and the buttons 54E to 54H is reversed. On the other hand, in the present embodiment, the analog sticks 53A and 53B are arranged above the cross button 54A and the buttons 54E to 54H, so that when the user operates the analog sticks 53A and 53B, The possibility that the finger hits each of the buttons 54E to 54H is lower than that in the above case. Thus, in this embodiment, the possibility of erroneous operation can be reduced, and the operability of the terminal device 7 can be improved. However, in another embodiment, the analog stick 53A may be disposed opposite to the cross button 54A, or the analog stick 53B may be disposed opposite to the buttons 54E to 54H as necessary.

  Here, in this embodiment, some operation units (respective analog sticks 53A and 53B, cross button 54A, and three buttons 54E to 54G) are arranged on the upper and lower sides of the housing 50 on both the left and right sides of the display unit (LCD 51). It is provided above the center of the direction (y-axis direction). When operating these operation units, the user mainly holds the upper side of the terminal device 7 from the vertical center. Here, when the user grips the lower side of the housing 50, the gripped terminal device 7 becomes unstable (especially when the terminal device 7 has a relatively large size as in the present embodiment). , It becomes difficult for the user to hold the terminal device 7. On the other hand, in this embodiment, when operating the operation unit, the user mainly holds the upper side of the terminal device 7 from the center in the vertical direction, and the housing 50 is laterally held by the palm. Can support. Therefore, the user can hold the housing 50 in a stable state, and the terminal device 7 can be easily held, so that the operation unit can be easily operated. In other embodiments, at least one operation unit may be provided on each of the left and right sides of the display unit above the center of the housing 50. For example, only the analog sticks 53 </ b> A and 53 </ b> B may be provided above the center of the housing 50. Further, for example, when the cross button 54A is provided above the left analog stick 53A and the four buttons 54E to 54H are provided above the right analog stick 53B, the cross button 54A and the four buttons 54E to 54H are provided. It may be provided above the center of the housing 50.

  In the present embodiment, a protrusion (a flange 59) is provided on the back side of the housing 50 (on the opposite side of the surface on which the LCD 51 is provided) (see FIG. 8C and FIG. 9). As shown in FIG. 8C, the flange portion 59 is a mountain-shaped member provided so as to protrude from the back surface of the substantially plate-shaped housing 50. The protrusion has a height (thickness) that can be hooked on a user's finger that holds the back surface of the housing 50. The height of the protrusion is preferably 10 to 25 [mm], and in this embodiment is 16.66 [mm]. Moreover, it is preferable that the lower surface of the protruding portion has an inclination of 45 ° or more (more preferably 60 ° or more) with respect to the back surface of the housing 50 so that the protruding portion can be easily caught on the user's finger. As shown in FIG. 8C, the lower surface of the protrusion may be formed to have a larger inclination angle than the upper surface. As shown in FIG. 10, the user gets tired even when the terminal device 7 is a relatively large size by holding his / her finger on the buttocks 59 (with the buttocks 59 on the finger). The terminal device 7 can be gripped in a stable state. That is, it can be said that the collar part 59 is a support member for supporting the housing 50 with a finger, and can also be called a finger hook part.

  Further, the flange portion 59 is provided above the center in the vertical direction of the housing 50. The flange portion 59 is provided at a position approximately opposite to the operation portion (each analog stick 53A and 53B) provided on the surface of the housing 50. That is, the protrusion is provided in a region including a position on the opposite side of the operation unit provided on the left and right of the display unit. Therefore, when operating the said operation part, the user can hold | grip the terminal device 7 so that the collar part 59 may be supported with a middle finger or a ring finger (refer FIG. 10). As a result, the terminal device 7 becomes easier to hold, and the operation unit becomes easier to operate. Further, in the present embodiment, the protrusion has a hook-like shape extending from side to side (the protruding portion), so the user holds the terminal device 7 with the middle finger or the ring finger along the lower surface of the protrusion. And the terminal device 7 becomes easier to hold. In addition, the collar part 59 should just be formed so that the (protruded part) may be extended in the left-right direction, and is not restricted to the shape extended in a horizontal direction as shown in FIG. In other embodiments, the flange portion 59 may extend in a direction slightly inclined from the horizontal direction. For example, the collar portion 59 may be provided so as to be inclined upward (or downward) from the left and right ends toward the center.

  In the present embodiment, the hook 59 having a hook-like shape is adopted as the protrusion formed on the back surface of the housing because a locking hole described later is provided in the hook 59. The part may have any shape. For example, in another embodiment, on the back side of the housing 50, two protrusions may be provided on both the left and right sides (no protrusion is provided in the center in the left-right direction) (see FIG. 29). . In another embodiment, the cross-sectional shape of the protrusion (the shape in the cross section perpendicular to the x-axis direction) is such that the terminal device 7 can be more firmly supported by the user's finger (the protrusion is It may be saddle-shaped (a shape in which the lower surface is recessed).

  In addition, the width | variety regarding the up-down direction of a projection part (hook part 59) may be what. For example, the protrusion may be formed up to the upper side of the housing 50. That is, the upper surface of the protrusion may be formed at the same position as the upper side surface of the housing 50. At this time, the housing 50 has a two-stage configuration in which the lower side is thin and the upper side is thick. As described above, it is preferable that the housing 50 is formed with surfaces (lower surfaces of the protrusions) facing downward on the left and right sides of the back surface. Accordingly, the user can easily grip the operating device by placing a finger on the surface. The “surface facing downward” may be formed at any position on the back surface of the housing 50, but is preferably positioned above the center of the housing 50.

  Further, as shown in FIGS. 8A, 8B, and 8C, the first L button 54I and the first R button 54J are provided on both the left and right sides of the upper surface of the housing 50, respectively. . In the present embodiment, the first L button 54 </ b> I and the first R button 54 </ b> J are provided in an obliquely upper portion (upper left portion and upper right portion) of the housing 50. Specifically, the first L button 54I is provided at the left end of the upper side surface of the plate-shaped housing 50 and is exposed from the upper left side surface (in other words, exposed from both the upper and left side surfaces). ) The first R button 54J is provided at the right end of the upper side surface of the housing 50, and is exposed from the upper right side surface (in other words, exposed from both the upper and right side surfaces). In this way, the first L button 54I is disposed at a position operable with the user's left index finger, and the first R button 54J is disposed at a position operable with the user's right hand index finger (see FIG. 10). In other embodiments, the operation units provided on the left and right sides of the upper surface of the housing 50 do not need to be provided at the left and right ends, and may be provided at positions other than the ends. In addition, operation portions may be provided on the left and right side surfaces of the housing 50, respectively.

  Further, as shown in FIG. 8 (c) and FIG. 9, the second L button 54K and the second R button 54L are disposed on the protruding portion (the collar portion 59). The second L button 54K is provided in the vicinity of the left end of the collar 59. The second R button 54L is provided in the vicinity of the right end of the collar portion 59. That is, the second L button 54K is provided slightly above the left side (left side when viewed from the front side) of the housing 50, and the second R button 54L is right side (when viewed from the front side) of the housing 50. On the right side). In other words, the second L button 54K is provided at a position (substantially) opposite to the left analog stick 53A provided on the surface, and the second R button 54L is provided (substantially) opposite to the right analog stick 53B provided on the surface. It is provided in the position. As described above, the second L button 54K is disposed at a position operable with the user's left middle finger (or index finger), and the second R button 54L is disposed at a position operable with the user's right middle finger (or index finger). (See FIG. 10). Moreover, the 2nd L button 54K and the 2nd R button 54L are provided in the upper surface of the said collar part 59, as shown to the (c) figure of FIG. Therefore, the second L button 54K and the second R button 54L have button surfaces that face upward (obliquely upward). When the user holds the terminal device 7, the middle finger or index finger is considered to move up and down, so that the user can easily press the second L button 54 </ b> K and the second R button 54 </ b> L by turning the button surface upward.

  As described above, in the present embodiment, the operation units (analog sticks 53A and 53B) are provided on the left and right sides of the display unit (LCD 51) above the center of the housing 50, and the operation is performed on the back side of the housing 50. Separate operation units (second L button 54K and second R button 54L) are provided at positions opposite to the unit. According to this, since the said operation part and another operation part are arrange | positioned in the position where the front side and back side of the housing 50 mutually oppose, when operating these operation parts, the user sets the housing 50 to the front side. It can be gripped so as to be pinched from the back side. Further, when operating these operation units, the user grips the upper side of the housing 50 with respect to the vertical center, so that the terminal device 7 can be gripped on the upper side and the terminal device 7 can be supported by the palm ( (See FIG. 10). As described above, the user can stably hold the housing 50 in a state in which the user can operate at least four operation units, can be easily held by the user, and has high operability. (Terminal device 7) can be provided.

  As described above, in the present embodiment, the user can easily hold the terminal device 7 by holding the terminal device 7 with the finger placed on the lower surface of the protrusion (the collar portion 59). In addition, since the second L button 54K and the second R button 54L are provided on the upper surface of the protrusion, the user can easily operate these buttons in the above state. For example, the user can easily hold the terminal device 7 in the following manner.

  That is, as shown in FIG. 10, the user can hold the terminal device 7 by placing the ring finger on the lower surface of the heel part 59 (the dashed line shown in FIG. 10) (supporting the heel part 59 with the ring finger). It is. At this time, the user can operate the four buttons (first L button 54I, first R button 54J, second L button 54K, and second R button 54L) with the index finger and the middle finger. For example, when a required game operation has many buttons to be used and is relatively complicated, many buttons can be easily operated by gripping as shown in FIG. Since each analog stick 53A and 53B is provided above the cross button 54A and the buttons 54E to 54H, the user can operate the analog sticks 53A and 53B with the thumb when a relatively complicated operation is required. Convenient. In FIG. 10, the user places a thumb on the surface of the housing 50, an index finger on the upper surface of the housing 50, a middle finger on the upper surface of the collar portion 59 on the rear surface of the housing 50, and a ring finger on the lower surface of the collar portion 59. The terminal device 7 is held by placing a little finger on the back surface of 50. Thus, the user can firmly hold the terminal device 7 so as to wrap the housing 50 from all sides.

  In addition, the user can hold the terminal device 7 by placing the middle finger on the lower surface of the collar portion 59. At this time, the user can easily operate the two buttons (the second L button 54K and the second R button 54L) with the index finger. For example, when the requested game operation is relatively simple with few buttons to be used, the terminal device 7 may be gripped by placing the middle finger on the lower surface of the collar portion 59. At this time, since the user can hold the lower side of the housing 50 with two fingers (ring finger and little finger), the user can hold the terminal device 7 firmly.

  In the present embodiment, the lower surface of the collar portion 59 is positioned between the analog sticks 53A and 53B and the cross button 54A and the four buttons 54E to 54H (more than the analog sticks 53A and 53B). And provided above the cross button 54A and the four buttons 54E to 54H). Therefore, when holding the terminal device 7 by placing the ring finger against the buttocks 59 (FIG. 10), the analog sticks 53A and 53B can be easily operated with the thumb, and the terminal device 7 is placed by placing the middle finger against the buttocks 59. When grasping, it is easy to operate the cross button 54A and the four buttons 54E to 54H with the thumb. That is, in any of the above two cases, the user can perform a direction input operation while firmly holding the terminal device 7.

  Further, as described above, the user can also hold the terminal device 7 while holding it vertically. That is, the user can hold the terminal device 7 in the vertical direction by holding the upper side or the lower side of the terminal device 7 with one hand. As described above, since the user can hold the terminal device 7 with one hand, for example, the user can perform an operation of holding the terminal device 7 with one hand and performing an input to the touch panel 52 with the other hand. Is possible.

  In addition, when gripping the upper side of the terminal device 7, the user can grip the terminal device 7 firmly by placing a finger other than the thumb against the lower surface of the collar portion 59. In particular, in the present embodiment, since the collar portion 59 is formed to extend to the left and right, the user places a finger other than the thumb on the collar portion 59 regardless of the position on the upper side of the terminal device 7. And the terminal device 7 can be firmly held. That is, when the terminal device 7 is used in a vertical orientation, the collar portion 59 can be used as a handle. On the other hand, when gripping the lower side of the terminal device 7 with one hand, the user can operate the buttons 54B to 54D with the hand. Therefore, for example, it is possible to perform operations on the buttons 54B to 54D with a hand holding the terminal device 7 while performing an input on the touch panel 52 with one hand, and more operations can be performed.

  In addition, regarding the terminal device 7 in the present embodiment, since the protrusion (the flange portion 59) is provided on the back surface, the terminal device 7 is placed with the screen of the LCD 51 (the surface of the housing 50) facing upward. The screen is slightly tilted. This makes it easier to see the screen when the terminal device 7 is placed. Further, it becomes easy to perform an input operation on the touch panel 52 in a state where the terminal device 7 is placed. In another embodiment, an additional protrusion having the same height as the flange 59 may be formed on the back surface of the housing 50. According to this, in a state where the screen of the LCD 51 is facing upward, the terminal device 7 can be placed so that the screen is horizontal as each projection comes into contact with the floor surface. Further, the additional protrusions may be removable (or foldable). According to this, the terminal device can be placed both in a state where the screen is slightly inclined and in a state where the screen is horizontal. That is, when the terminal device 7 is placed and used, the heel portion 59 can be used as a leg portion.

  Functions corresponding to the game program are appropriately assigned to the buttons 54A to 54L. For example, the cross button 54A and the buttons 54E to 54H may be used for a direction instruction operation or a selection operation, and the buttons 54B to 54E may be used for a determination operation or a cancel operation. Further, the terminal device 7 may have a button for turning on / off the screen display of the LCD 51 and a button for performing connection setting (pairing) with the game device 3.

  As illustrated in FIG. 8A, the terminal device 7 includes a marker portion 55 including a marker 55 </ b> A and a marker 55 </ b> B on the surface of the housing 50. The marker unit 55 is provided on the upper side of the LCD 51. Each of the markers 55A and 55B is composed of one or more infrared LEDs, like the markers 6R and 6L of the marker device 6. Infrared LEDs constituting the markers 55A and 55B are disposed inside a window portion that transmits infrared light. The marker unit 55 is used for the game device 3 to calculate the movement of the controller 5 and the like, similar to the marker device 6 described above. Further, the game apparatus 3 can control lighting of each infrared LED included in the marker unit 55.

  The terminal device 7 includes a camera 56 that is an imaging unit. The camera 56 includes an imaging element (for example, a CCD image sensor or a CMOS image sensor) having a predetermined resolution, and a lens. As shown in FIG. 8, in this embodiment, the camera 56 is provided on the surface of the housing 50. Therefore, the camera 56 can take an image of the face of the user who has the terminal device 7, and can, for example, take an image of the user who is playing the game while looking at the LCD 51. In the present embodiment, the camera 56 is disposed between the two markers 55A and 55B.

  The terminal device 7 includes a microphone 79 that is a voice input unit. A microphone hole 50 c is provided on the surface of the housing 50. The microphone 79 is provided inside the housing 50 behind the microphone hole 50c. The microphone 79 detects sounds around the terminal device 7 such as a user's voice.

  The terminal device 7 includes a speaker 77 which is an audio output unit. As shown in FIG. 8D, a speaker hole 57 is provided below the surface of the housing 50. The output sound of the speaker 77 is output from the speaker hole 57. In the present embodiment, the terminal device 7 includes two speakers, and speaker holes 57 are provided at positions of the left speaker and the right speaker. The terminal device 7 includes a knob 64 for adjusting the volume of the speaker 77. The terminal device 7 includes an audio output terminal 62 for connecting an audio output unit such as an earphone. Here, considering that an additional device is connected to the lower side surface of the housing, the audio output terminal 62 and the knob 64 are provided on the upper side surface of the housing 50. It may be provided on the side surface.

  The housing 50 is provided with a window 63 for emitting an infrared signal from the infrared communication module 82 to the outside of the terminal device 7. Here, the window 63 is provided on the upper side surface of the housing 50 so that an infrared signal is emitted forward of the user when both sides of the LCD 51 are gripped. However, in other embodiments, the window 63 may be provided at any position such as the back surface of the housing 50.

  Further, the terminal device 7 includes an expansion connector 58 for connecting other devices to the terminal device 7. The extension connector 58 is a communication terminal for transmitting / receiving data (information) to / from other devices connected to the terminal device 7. In the present embodiment, the extension connector 58 is provided on the lower side surface of the housing 50 as shown in FIG. Note that any other additional device connected to the expansion connector 58 may be used. For example, a controller (gun-type controller or the like) used for a specific game or an input device such as a keyboard may be used. . If it is not necessary to connect an additional device, the expansion connector 58 may not be provided. The expansion connector 58 may include a terminal for supplying power to the additional device and a terminal for charging.

  Further, the terminal device 7 includes a charging terminal 66 for acquiring power from the additional device, separately from the expansion connector 58. When the charging terminal 66 is connected to a charging stand (not shown), power is supplied from the charging stand to the terminal device 7. In the present embodiment, the charging terminal 66 is provided on the lower side surface of the housing 50. Therefore, when the terminal device 7 and the additional device are connected, in addition to transmitting and receiving information via the extension connector 58, it is also possible to supply power from one to the other. Thus, by providing the charging terminal 66 around the expansion connector 58 (on the left and right sides), when the terminal device 7 and the additional device are connected, it is possible to supply power as well as transmit and receive information. . Further, the terminal device 7 has a charging connector, and the housing 50 has a cover portion 61 for protecting the charging connector. The charging connector can be connected to a charger 86 to be described later. When the charging connector is connected to the charger, power is supplied from the charger 86 to the terminal device 7. In this embodiment, considering that the additional device is connected to the lower side surface of the housing, the charging connector (cover portion 61) is provided on the upper side surface of the housing 50. It may be provided on the side surface.

  As shown in FIGS. 8D and 9, locking holes 59 a and 59 b that can lock the claw portions of the additional device are provided on the lower surface of the protruding portion (the flange portion 59). The locking holes 59 a and 59 b are used when connecting another additional device to the terminal device 7. That is, the additional device has a claw portion that can be locked in the locking holes 59a and 59b. When the additional device is connected to the terminal device 7, the claw portion is locked in the locking holes 59a and 59b. The terminal device 7 and the additional device are fixed. Further, screw holes may be further provided inside the locking holes 59a and 59b, and the additional device may be firmly fixed with screws. Here, the protrusion provided on the back surface of the terminal device 7 is a flange 59 having a hook shape. That is, the collar part 59 is provided extending in the left-right direction. As shown in FIG. 9, the locking holes 59 a and 59 b are provided near the center (with respect to the left-right direction) of the lower surface of the flange portion 59. Note that the number of the locking holes 59a and 59b provided on the lower surface of the flange portion 59 may be any number, but when it is one, it is preferably provided in the center of the flange portion 59, and there are a plurality Are preferably arranged symmetrically. According to this, it is possible to stably connect the additional device while keeping the left and right balance equal. Further, when the locking hole is provided near the center, the size of the additional device can be reduced as compared with the case where the locking hole is provided at both the left and right ends. As described above, the flange portion 59 can be used as a locking member of the additional device.

  In the present embodiment, as shown in FIG. 8D, locking holes 50 a and 50 b are provided on the lower surface of the housing 50. Therefore, when connecting the additional device to the terminal device 7, the terminal device 7 and the additional device are fixed by the four claw portions being respectively locked in the four locking holes. As a result, the additional device can be firmly connected to the terminal device 7. It should be noted that screw holes may be provided inside the locking holes 50a and 50b, and the additional device may be screwed. The screwing position may be anywhere, but, for example, the supporting portion of the additional device that contacts the back surface of the housing 50 and the flange portion 59 may be screwed. In other embodiments, the locking holes provided in the housing may be arranged in any manner.

  Further, the terminal device 7 has a battery lid 67 that can be attached to and detached from the housing 50. A battery (battery 85 shown in FIG. 11) is disposed inside the battery lid 67. In the present embodiment, the battery lid 67 is provided on the back side of the housing 50 and is provided on the lower side of the protrusion (the flange 59).

  Further, the housing 50 of the terminal device 7 is provided with holes 65a and 65b for tying strap strings. As shown in FIG. 8D, in the present embodiment, the holes 65 a and 65 b are provided on the lower surface of the housing 50. In the present embodiment, two holes 65 a and 65 b are provided on each of the left and right sides of the housing 50. That is, the hole 65 a is provided on the left side from the center of the lower surface of the housing 50, and the hole 65 b is provided on the right side of the center of the lower surface of the housing 50. The user may tie a strap to either of the holes 65a and 65b and tie the strap to his / her wrist. Thereby, even if the user drops the terminal device 7 or the terminal device 7 is removed from the hand, the terminal device 7 can be prevented from dropping or colliding with another object. In the present embodiment, since the holes are provided on both the left and right sides, the user can tie the strap to either hand, which is convenient.

  In addition, regarding the terminal device 7 shown in FIGS. 8 to 11, the shapes of the operation buttons and the housing 50, the number of components, the installation positions, and the like are merely examples, and other shapes, numbers, and installation positions are provided. It may be.

  Next, the internal configuration of the terminal device 7 will be described with reference to FIG. FIG. 11 is a block diagram illustrating an internal configuration of the terminal device 7. As shown in FIG. 11, in addition to the configuration shown in FIG. 8, the terminal device 7 includes a touch panel controller 71, a magnetic sensor 72, an acceleration sensor 73, a gyro sensor 74, a user interface controller (UI controller) 75, a codec LSI 76, a speaker. 77, a sound IC 78, a microphone 79, a wireless module 80, an antenna 81, an infrared communication module 82, a flash memory 83, a power supply IC 84, and a battery 85. These electronic components are mounted on an electronic circuit board and stored in the housing 50.

  The UI controller 75 is a circuit for controlling input / output of data to / from various input / output units. The UI controller 75 is connected to the touch panel controller 71, analog stick 53 (analog sticks 53A and 53B), operation buttons 54 (operation buttons 54A to 54L), a marker unit 55, a magnetic sensor 72, an acceleration sensor 73, and a gyro sensor 74. Is done. The UI controller 75 is connected to the codec LSI 76 and the extension connector 58. A power supply IC 84 is connected to the UI controller 75, and power is supplied to each unit via the UI controller 75. A built-in battery 85 is connected to the power supply IC 84 to supply power. In addition, a charger 86 or a cable that can acquire power from an external power source can be connected to the power supply IC 84 via a charging connector. The terminal device 7 can be connected to the external power source using the charger 86 or the cable. Power supply and charging from can be performed. The terminal device 7 can also be charged by mounting the terminal device 7 on a cradle having a charging function (not shown). That is, although not shown, a cradle capable of acquiring power from an external power source can be connected to the power supply IC 84 via the charging terminal 66, and the terminal device 7 supplies power from the external power source using the cradle. And can be charged.

  The touch panel controller 71 is a circuit that is connected to the touch panel 52 and controls the touch panel 52. The touch panel controller 71 generates input position data in a predetermined format based on a signal from the touch panel 52 and outputs the input position data to the UI controller 75. The input position data represents the coordinates of the position where the input is performed on the input surface of the touch panel 52. The touch panel controller 71 reads a signal from the touch panel 52 and generates input position data at a rate of once per predetermined time. Various control instructions for the touch panel 52 are output from the UI controller 75 to the touch panel controller 71.

  The analog stick 53 outputs to the UI controller 75 stick data representing the direction and amount in which the stick unit operated by the user's finger has slid (or tilted). In addition, the operation button 54 outputs operation button data representing the input status (whether or not it has been pressed) to each of the operation buttons 54 </ b> A to 54 </ b> L to the UI controller 75.

  The magnetic sensor 72 detects the direction by detecting the magnitude and direction of the magnetic field. The azimuth data indicating the detected azimuth is output to the UI controller 75. In addition, a control instruction for the magnetic sensor 72 is output from the UI controller 75 to the magnetic sensor 72. For the magnetic sensor 72, an MI (magnetic impedance) element, a fluxgate sensor, a Hall element, a GMR (giant magnetoresistance) element, a TMR (tunnel magnetoresistance) element, an AMR (anisotropic magnetoresistance) element, or the like was used. Although there is a sensor, any sensor may be used as long as it can detect the direction. Strictly speaking, in a place where a magnetic field is generated in addition to the geomagnetism, the obtained azimuth data does not indicate the azimuth, but even in such a case, the terminal device 7 moves. Since the orientation data changes, the change in the attitude of the terminal device 7 can be calculated.

  The acceleration sensor 73 is provided inside the housing 50 and detects the magnitude of linear acceleration along the direction of three axes (xyz axes shown in FIG. 8A). Specifically, the acceleration sensor 73 has a linear acceleration of each axis with the long side direction of the housing 50 as the x axis, the direction perpendicular to the surface of the housing 50 as the y axis, and the short side direction of the housing 50 as the z axis. Detect the size of. Acceleration data representing the detected acceleration is output to the UI controller 75. A control instruction for the acceleration sensor 73 is output from the UI controller 75 to the acceleration sensor 73. The acceleration sensor 73 is, for example, a capacitive MEMS acceleration sensor in the present embodiment, but other types of acceleration sensors may be used in other embodiments. Further, the acceleration sensor 73 may be an acceleration sensor that detects a uniaxial or biaxial direction.

  The gyro sensor 74 is provided inside the housing 50 and detects angular velocities around the three axes of the x axis, the y axis, and the z axis. Angular velocity data representing the detected angular velocity is output to the UI controller 75. Further, a control instruction for the gyro sensor 74 is output from the UI controller 75 to the gyro sensor 74. Any number and combination of gyro sensors may be used to detect the three-axis angular velocity, and the gyro sensor 74 is similar to the gyro sensor 48 in that a two-axis gyro sensor, a one-axis gyro sensor, It may be constituted by. Further, the gyro sensor 74 may be a gyro sensor that detects a uniaxial or biaxial direction.

  The UI controller 75 outputs operation data including input position data, stick data, operation button data, azimuth data, acceleration data, and angular velocity data received from each of the above components to the codec LSI 76. When another device is connected to the terminal device 7 via the extension connector 58, the operation data may further include data representing an operation on the other device.

  The codec LSI 76 is a circuit that performs compression processing on data transmitted to the game apparatus 3 and expansion processing on data transmitted from the game apparatus 3. The codec LSI 76 is connected to the LCD 51, the camera 56, the sound IC 78, the wireless module 80, the flash memory 83, and the infrared communication module 82. The codec LSI 76 includes a CPU 87 and an internal memory 88. Although the terminal device 7 is configured not to perform the game process itself, it is necessary to execute a minimum program for management and communication of the terminal device 7. When the power is turned on, the program stored in the flash memory 83 is read into the internal memory 88 and executed by the CPU 87, whereby the terminal device 7 is activated. A part of the internal memory 88 is used as a VRAM for the LCD 51.

  The camera 56 captures an image in accordance with an instruction from the game apparatus 3 and outputs the captured image data to the codec LSI 76. Control instructions for the camera 56 such as an image capturing instruction are output from the codec LSI 76 to the camera 56. Note that the camera 56 can also capture moving images. That is, the camera 56 can repeatedly capture images and repeatedly output image data to the codec LSI 76.

  The sound IC 78 is a circuit that is connected to the speaker 77 and the microphone 79 and controls input / output of audio data to and from the speaker 77 and the microphone 79. That is, when audio data is received from the codec LSI 76, the sound IC 78 outputs an audio signal obtained by performing D / A conversion on the audio data to the speaker 77, and outputs sound from the speaker 77. The microphone 79 detects a sound (such as a user's voice) transmitted to the terminal device 7 and outputs a sound signal indicating the sound to the sound IC 78. The sound IC 78 performs A / D conversion on the audio signal from the microphone 79 and outputs audio data in a predetermined format to the codec LSI 76.

  The infrared communication module 82 emits an infrared signal and performs infrared communication with other devices. Here, the infrared communication module 82 includes, for example, a function of performing infrared communication according to the IrDA standard and a function of outputting an infrared signal (control signal) for controlling the television 2.

  The codec LSI 76 transmits the image data from the camera 56, the audio data from the microphone 79, and the operation data from the UI controller 75 to the game apparatus 3 via the wireless module 80 as terminal operation data. In the present embodiment, the codec LSI 76 performs the same compression processing as the codec LSI 27 on the image data and the audio data. The terminal operation data and the compressed image data and audio data are output to the wireless module 80 as transmission data. An antenna 81 is connected to the wireless module 80, and the wireless module 80 transmits the transmission data to the game apparatus 3 via the antenna 81. The wireless module 80 has the same function as the terminal communication module 28 of the game apparatus 3. That is, the wireless module 80 has a function of connecting to a wireless LAN by a method compliant with, for example, the IEEE 802.11n standard. The data to be transmitted may or may not be encrypted as necessary.

  As described above, the transmission data transmitted from the terminal device 7 to the game apparatus 3 includes operation data (terminal operation data), image data, and audio data. When another device is connected to the terminal device 7 via the extension connector 58, the data received from the other device may be further included in the transmission data. The codec LSI 76 may transmit the data received by the infrared communication by the infrared communication module 82 to the game apparatus 3 by including the data in the transmission data as necessary.

  Further, as described above, compressed image data and audio data are transmitted from the game apparatus 3 to the terminal apparatus 7. These data are received by the antenna 81 (receiving unit) and sent to the codec LSI 76 via the wireless module 80. The codec LSI 76 decompresses the received image data and audio data. The expanded image data is output to the LCD 51, and the image is displayed on the LCD 51. That is, the codec LSI 76 (CPU 87) displays the received image data on the display unit. The expanded audio data is output to the sound IC 78, and the sound IC 78 outputs sound from the speaker 77.

  When the control data is included in the data received from the game apparatus 3, the codec LSI 76 and the UI controller 75 issue a control instruction to each unit according to the control data. As described above, the control data is a control instruction for each component (in this embodiment, the camera 56, the touch panel controller 71, the marker unit 55, the sensors 62 to 64, and the infrared communication module 82) included in the terminal device 7. It is data to represent. In the present embodiment, as the control instruction represented by the control data, an instruction to operate each of the above components or to stop (stop) the operation can be considered. That is, components that are not used in the game may be paused in order to reduce power consumption. In that case, the transmission data transmitted from the terminal device 7 to the game device 3 includes data from the paused components. Do not let it. In addition, since the marker part 55 is infrared LED, control may just be ON / OFF of supply of electric power.

  The game apparatus 3 can control the operation of the television 2 by controlling the output of the infrared communication module 82. That is, the game apparatus 3 outputs an instruction (the control data) for causing the infrared communication module 82 to output an infrared signal corresponding to a control command for controlling the television 2 to the terminal apparatus 7. In response to this instruction, the codec LSI 76 causes the infrared communication module 82 to output an infrared signal corresponding to the control command. Here, the television 2 includes an infrared light receiving unit capable of receiving an infrared signal. When the infrared signal output from the infrared communication module 82 is received by the infrared light receiving unit, the television 2 performs an operation according to the infrared signal. The instruction from the game apparatus 3 may indicate an infrared signal pattern, or when the terminal device 7 stores an infrared signal pattern, the instruction indicates the pattern. Also good.

  As described above, the terminal device 7 includes operation means such as the touch panel 52, the analog stick 53, and the operation button 54. However, in other embodiments, instead of these operation means or together with these operation means. The configuration may include other operation means.

  In addition, the terminal device 7 includes a magnetic sensor 72, an acceleration sensor 73, and a gyro sensor 74 as sensors for calculating the movement of the terminal device 7 (including changes in position and posture, or position and posture). In other embodiments, the configuration may include only one or two of these sensors. Moreover, in other embodiment, it may replace with these sensors or the structure provided with another sensor with these sensors may be sufficient.

  The terminal device 7 includes the camera 56 and the microphone 79. However, in other embodiments, the terminal device 7 may not include the camera 56 and the microphone 79, or may include only one of them. Good.

  Further, the terminal device 7 is configured to include the marker unit 55 as a configuration for calculating the positional relationship between the terminal device 7 and the controller 5 (the position and / or orientation of the terminal device 7 viewed from the controller 5). In other embodiments, the marker unit 55 may not be provided. In another embodiment, the terminal device 7 may include other means as a configuration for calculating the positional relationship. For example, in another embodiment, the controller 5 may include a marker unit, and the terminal device 7 may include an image sensor. Furthermore, in this case, the marker device 6 may be configured to include an imaging element instead of the infrared LED.

[5. Position Calculation Method in 3D Space]
Next, a method for calculating a position in a three-dimensional virtual space performed in the present embodiment will be described. In the present embodiment, the game system 1 calculates a position in a three-dimensional virtual space based on an operation on the terminal device 7 that is an operation device. Note that the game system 1 may not include the controller 5. The game system 1 calculates a position in the virtual space based on the attitude of the terminal device 7 and the input position with respect to a predetermined input surface (touch panel 52) provided in the terminal device 7. According to this, the user (player) can easily specify the position in the three-dimensional space by the operation of changing the attitude of the terminal device 7 and the input operation on the input surface of the terminal device 7. The outline of the position calculation method will be described below.

  FIG. 12 is an explanatory diagram of a position calculation method in the present embodiment. FIG. 13 is a diagram illustrating a state in which the attitude of the terminal device 7 is changed from the state illustrated in FIG. As shown in FIGS. 12 and 13, in the present embodiment, a portable terminal device 7 is used as the operation device. Although details will be described later, the game system 1 can calculate (estimate) the attitude of the terminal device 7. The terminal device 7 includes a touch panel 52 as an input device that can input a position with respect to a predetermined input surface. Therefore, the user can perform an operation for changing the posture and an input operation on the input surface of the terminal device 7 with respect to the terminal device 7.

  12 and 13, the surface S is arranged in the virtual space. The surface S (the position and / or orientation of the surface S) is determined according to the orientation of the terminal device 7 as shown in FIGS. 12 and 13. That is, the position and / or posture of the surface S is changed by an operation for changing the posture of the terminal device 7. The method for determining the surface S based on the attitude of the terminal device 7 may be any method, but in this embodiment, the attitude of the surface S changes (at least) according to the attitude of the terminal device 7. Surface S is set.

  As shown in FIGS. 12 and 13, when an input (touch input) is performed on the input surface of the touch panel 52, the position Q on the surface S is calculated based on the input position P. Hereinafter, this position Q is referred to as a “designated position”. Any method may be used to calculate the designated position Q based on the input position P. For example, the input surface and the surface S are associated with each other, and the position on the surface S corresponding to the input position P is the designated position Q. May be calculated as

  As described above, the game system 1 calculates the position on the surface S determined according to the attitude of the terminal device 7 in the virtual space as the position in the virtual space based on the input position. Here, in the state shown in FIG. 12, the user can designate a region on the surface S in the three-dimensional virtual space by touch input. On the other hand, in the state shown in FIG. 13, since the posture of the surface S is different from that in FIG. 12, the user can specify a region different from the region of the surface S shown in FIG. As described above, in the present embodiment, the user changes the posture of the surface S by an operation for changing the posture of the terminal device 7 and designates the position on the surface S by touch input. You can specify the position. Further, in the present embodiment, the posture of the surface S changes depending on the posture of the terminal device 7, and the position on the surface S changes depending on the input position on the input surface. As a result, the user can specify the posture and the input position of the surface S by an intuitive operation, and can easily specify the three-dimensional space.

  Further, although details will be described later, in the present embodiment, the display unit (LCD 51) of the terminal device 7 has an input position with respect to the touch panel 52 and a specified position Q on the surface S calculated based on the input position. Are designated so that they match on the screen (steps S16, S25 and S26, S64, and S86 described later). That is, the user can designate the position in the virtual space corresponding to the position by designating (touching) the position of the display unit on the screen. Thus, the user can specify a position in the three-dimensional space by a more intuitive and easy-to-understand operation.

  In the present embodiment, the game system 1 (game device 3) executes a predetermined process (game process) with the specified position calculated as described above as an input. This predetermined process may be any process. For example, the specified position may be used as a game input. Hereinafter, first to fourth game examples will be described as examples of the game process using the designated position.

[6. First game example]
Hereinafter, a first game example executed in the game system 1 of the present embodiment will be described with reference to FIGS. In the first game example, a player (user) performs input to a keyboard arranged in a three-dimensional virtual space (game space) by the terminal device 7. That is, when the player designates the position of the keyboard in the virtual space using the terminal device 7, a sound corresponding to the designated keyboard is output. In this game example, the terminal device 7 is used as an operation device. Therefore, the controller 5 may not be used, and the game system 1 may be configured not to include the controller 5.

  FIG. 14 is a diagram illustrating the television 2 and the terminal device 7 when a game operation is performed in the first game example. As shown in FIG. 14, the television 2 displays an image of the virtual space including the keyboard object 91 and the terminal object 92. A part of the keyboard object 91 is displayed on the terminal device 7 (LCD 51). In this game example, each keyboard of the keyboard object 91 is arranged in an arc shape. Further, the television 2 displays an image of the arc-shaped keyboard object 91 viewed from the inside of the arc (the side where the center of the arc is located). Thus, the user can perform an operation as if standing at the center of the arcuate keyboard object 91. In addition, sounds with different pitches are associated with each keyboard, and when a keyboard is designated, a sound corresponding to the designated keyboard is output.

  Further, the terminal object 92 is controlled according to the attitude of the terminal device 7. In the present game example, the terminal object 92 relates to the orientation of the terminal device 7 and the orientation of the terminal object 92 with respect to rotation about an axis (y-axis shown in FIG. 8) perpendicular to the screen of the terminal device 7 (terminal object 92). Are controlled to match. Further, the terminal object 92 is controlled to move while rotating above the keyboard object 91 arranged in an arc shape according to the attitude of the terminal device 7 (see the arrow shown in FIG. 14). Therefore, the player can move the terminal object 92 while rotating left and right by an operation of tilting the terminal device 7 left and right (see arrows shown in FIG. 14). In the present game example, an image similar to the terminal image is drawn on the screen 93 of the terminal object 92 displayed on the television 2 (see FIG. 14), but nothing may be displayed on the screen 93. Good.

  As described above, in the present game example, the keys of the keyboard object 91 are arranged in an arc shape, and the terminal object 92 is moved along an arc-shaped locus in accordance with the rotation of the terminal device 7. Thereby, the movement of the terminal device 7 and the movement of the terminal object 92 can be made to correspond to each other. Further, from the arcuate arrangement of each keyboard and the arcuate movement trajectory of the terminal object 92, the player can grasp (guess) the operation method of the terminal device 7 without any special explanation. Therefore, in this game example, the operation of the terminal device 7 can be made more intuitive and easy for the player to understand.

  FIG. 15 is a view of the keyboard object 91 as viewed from above in the virtual space. As shown in FIG. 15, in the present game example, the face 94 is set on the face of the keyboard object 91. In FIG. 15, the surface 94 is indicated by a dotted line, but in the present game example, the surface 94 is not actually displayed on the display device (the television 2 and the terminal device 7). That is, the surface 94 is not necessarily displayed, and may not be arranged as an object in the virtual space, or may be handled as a transparent object.

  The surface 94 is set based on the position and orientation of the terminal object 92. That is, the position of the surface 94 is set to a position below (directly below) the terminal object 92, and the posture of the surface 94 coincides with the posture of the terminal object 92 with respect to the posture in the rotational direction around the axis perpendicular to the surface 94. Set to posture. Accordingly, in response to the terminal object 92 moving above the keyboard object 91 along an arcuate path, the surface 94 moves while rotating on the keyboard object 91 so as to include a part of the keyboard object 91 (FIG. 15). See the arrow).

  As described above, in the present game example, the position and posture of the terminal object 92 are determined according to the posture of the terminal device 7, and the position and posture of the surface 94 are determined according to the position and posture of the terminal object 92. That is, the position and orientation of the surface 94 are determined according to the orientation of the terminal device 7. Therefore, by the operation of tilting the terminal device 7 left and right, the surface 94 moves while rotating left and right on the keyboard object 91 (see the arrow shown in FIG. 15).

  FIG. 16 is a diagram illustrating an example of an image displayed on the terminal device 7. As shown in FIG. 16, an image representing a part of the keyboard object 91 is displayed on the terminal device 7. Specifically, an area within the surface 94 of the keyboard object 91 is displayed on the terminal device 7. Therefore, the area displayed on the terminal device 7 in the keyboard object 91 changes according to the attitude of the terminal device 7. Specifically, the display area of the keyboard object 91 scrolls left and right in an arc shape by an operation of tilting the terminal device 7 left and right. In this game example, an arc-shaped keyboard object 91 is displayed on the television 2, and a part of the keyboard object 91 displayed on the terminal device 7 is displayed by an operation of moving the terminal device 7 along an arc-shaped locus. Scroll in an arc. Therefore, the user can perform a position designation operation on the keyboard object 91 using the terminal device 7 as if the arc-shaped keyboard object 91 exists.

  In this game example, the keyboard displayed on the terminal device 7 at a time is a keyboard for one octave so that the user can identify the sound corresponding to each keyboard from the arrangement of the white key and the black key. Since the terminal device 7 has a relatively large screen, each keyboard is displayed with a width that is easy for the user to press with a finger even when a keyboard for one octave is displayed.

  Here, the player can output a sound by performing a touch input on the keyboard object 91 displayed on the terminal device 7. That is, when a touch input is made on the touch panel 52 of the terminal device 7, the game apparatus 3 specifies a keyboard at a designated position corresponding to the input position (displayed at the input position on the LCD 51 screen). The sound corresponding to the specified keyboard is output. The speaker that outputs sound may be the speaker 2a of the television 2, the speaker 77 of the terminal device 7, or both the speakers 2a and 77.

  As described above, in this game example, the terminal object 92 and the surface 94 move in accordance with the change in the attitude of the terminal device 7, and the area of the keyboard object 91 displayed on the terminal device 7 changes. As a result, the keyboard that can be designated by the player using the terminal device 7 changes. Therefore, the player can easily specify the position of the keyboard object 91 in the three-dimensional virtual space by the operation of changing the posture of the terminal device 7 and the operation of touch input. In the present game example, the player can specify the entire keyboard object 91 without displaying the entire keyboard object 91 on the terminal device 7. Therefore, since the keyboard object 91 can be displayed large on the terminal device 7, it is easy to perform touch input and operation becomes easy.

(Game processing in the first game example)
Next, details of the game process executed by the game system 1 in the first game example will be described. First, various data used in the game process will be described. FIG. 17 is a diagram showing various data used in the game process. In FIG. 17, it is a figure which shows the main data memorize | stored in the main memory (the external main memory 12 or the internal main memory 11e) of the game device 3. As shown in FIG. 17, a game program 100, terminal operation data 101, and processing data 108 are stored in the main memory of the game apparatus 3. In addition to the data shown in FIG. 17, the main memory stores data necessary for the game such as image data of various objects appearing in the game and sound data used in the game.

  A part or all of the game program 100 is read from the optical disk 4 and stored in the main memory at an appropriate timing after the game apparatus 3 is turned on. The game program 100 may be obtained from the flash memory 17 or an external device of the game device 3 (for example, via the Internet) instead of the optical disc 4. A part (for example, a program for calculating the attitude of the controller 5 and / or the terminal device 7) included in the game program 100 may be stored in advance in the game apparatus 3.

  The terminal operation data 101 is data representing a player's operation on the terminal device 7. The terminal operation data 101 is transmitted from the terminal device 7, acquired by the game device 3, and stored in the main memory. The terminal operation data 101 includes operation button data 102, stick data 103, input position data 104, angular velocity data 105, acceleration data 106, and azimuth data 107. The terminal operation data 101 may include data of an image captured by the camera 56 and audio data detected by the microphone 79 in addition to the data shown in FIG. The main memory may store a predetermined number of terminal operation data in order from the latest (last acquired).

  The operation button data 102 is data representing an input state for each of the operation buttons 54 </ b> A to 54 </ b> L provided on the terminal device 7. Specifically, the operation button data 102 represents whether or not each of the operation buttons 54A to 54L is pressed.

  The stick data 103 is data representing the direction and amount in which the stick portion of the analog stick 53 (analog sticks 53A and 53B) has slid (or tilted). The direction and amount may be expressed as, for example, a two-dimensional coordinate or a two-dimensional vector.

  The input position data 104 is data representing a position (input position) where an input is performed on the input surface of the touch panel 52. In the present embodiment, the input position data 104 represents a coordinate value of a two-dimensional coordinate system for representing a position on the input surface. When the touch panel 52 is a multi-touch method, the input position data 104 may represent a plurality of input positions.

  The angular velocity data 105 is data representing the angular velocity detected by the gyro sensor 74. In the present embodiment, the angular velocity data 105 represents the angular velocities around the three axes xyz shown in FIG. 8, but in other embodiments, the angular velocity data 105 represents the angular velocities around any one or more axes. I just need it.

  The acceleration data 106 is data representing the acceleration (acceleration vector) detected by the acceleration sensor 73. In the present embodiment, the acceleration data 106 represents a three-dimensional acceleration whose components are accelerations in the directions of the three axes of xyz shown in FIG. 8, but in other embodiments, any one or more of the acceleration data 106 is represented. It only has to represent acceleration in the direction.

  The azimuth data 107 is data representing the azimuth detected by the magnetic sensor 72. In the present embodiment, the azimuth data 107 represents the direction of a predetermined azimuth (for example, north) with reference to the terminal device 7. When a magnetic field other than geomagnetism is generated, the azimuth data 107 does not strictly indicate an absolute azimuth (north, etc.). However, since the relative direction of the terminal device 7 with respect to the direction of the magnetic field at the location is indicated by the orientation data 107, even in such a case, the orientation of the terminal device 7 or a change in the orientation is changed based on the orientation data 107. It is possible to calculate.

  Note that the terminal operation data 101 only needs to represent an operation on the terminal device 7, and may include only one of the data 92 to 97. Further, when the terminal device 7 has other input means (for example, a touch pad or an imaging means of the controller 5), the terminal operation data 101 includes data representing an operation on the other input means. Also good. When the movement of the terminal device 7 itself is used as a game operation as in the present embodiment, the terminal operation data 101 is the terminal device 7 itself, such as acceleration data 106, angular velocity data 105, or azimuth data 107. It is preferable to include data whose value changes according to movement.

  In the present embodiment, in addition to the terminal operation data 101, camera image data and / or microphone sound data may be transmitted from the terminal device 7 to the game device 3. The camera image data is data representing an image (camera image) captured by the camera 56 of the terminal device 7. The microphone sound data is data representing sound (microphone sound) detected by the microphone 79 of the terminal device 7. Note that the camera image data and the microphone sound data may be compressed by the codec LSI 76 and transmitted to the game apparatus 3, and may be expanded by the codec LSI 27 in the game apparatus 3 and stored in the main memory.

  In the present embodiment, the controller 5 is not used as an operating device and is not shown, but the main memory may store controller operation data representing the player's operation on the controller 5.

  The processing data 108 is data used in a game process (FIG. 18) described later. The processing data 108 includes terminal attitude data 109, terminal object data 110, surface data 111, terminal camera data 112, and designated position data 113. In addition to the data shown in FIG. 17, the processing data 108 includes various data used in the game process, such as data representing various parameters set for various objects appearing in the game.

  The terminal attitude data 109 is data representing the attitude of the terminal device 7. The attitude of the terminal device 7 may be expressed by, for example, a rotation matrix representing rotation from a predetermined reference attitude to the current attitude of the terminal device 7, or may be expressed by a cubic vector or three angles. . In this embodiment, a posture in a three-dimensional space is used as the posture of the terminal device 7, but in another embodiment, a posture in a two-dimensional plane may be used. In the present embodiment, the terminal attitude data 109 is calculated based on the angular velocity data 105 and the acceleration data 106 included in the terminal operation data 101. A method for calculating the terminal attitude data 109 will be described later in step S11.

  The terminal object data 110 is data representing the position and orientation of the terminal object 92 arranged in the virtual space. The position of the terminal object 92 is represented by a three-dimensional coordinate value indicating a position in a coordinate system (virtual space coordinate system) for representing a position in the virtual space. In this game example, the terminal object 92 changes the posture only in the rotation direction around the axis perpendicular to the screen 93 of the terminal object 92, and does not change the posture in the other rotation directions. Therefore, the attitude of the terminal object 92 is expressed by one vector (for example, a vector parallel to the screen of the terminal object 92 and facing a predetermined direction with the terminal object 92 as a reference). Further, the attitude of the terminal object 92 may be expressed by a rotation matrix or a vector in the coordinate system.

  The surface data 111 is data representing the position and orientation of the surface 94 set in the virtual space. The position of the surface 94 is represented by a three-dimensional coordinate value indicating the position in the virtual space coordinate system. In this game example, the posture of the surface 94 is changed only in the rotation direction around the axis perpendicular to the surface 94, and the posture is not changed in the other rotation directions. Therefore, the orientation of the surface 94 can be expressed by one vector (for example, a vector parallel to the screen of the terminal object 92 and pointing in a predetermined direction with the terminal object 92 as a reference), like the orientation of the terminal object 92. Good. Further, the posture of the surface 94 may be expressed by a rotation matrix or a vector in the coordinate system.

  The terminal camera data 112 is data representing the position and orientation of a virtual camera (terminal virtual camera) used to generate a terminal image. In the present embodiment, the position and orientation of the terminal virtual camera are calculated based on the plane data 111.

  The designated position data 113 is data representing the designated position. The designated position is represented by a three-dimensional coordinate value indicating the position in the virtual space coordinate system. Although the details will be described later, the designated position is calculated based on the input position data 104 and the surface data 111.

  Next, details of the game process executed in the game apparatus 3 will be described with reference to FIGS. 18 and 19. FIG. 18 is a main flowchart showing a flow of game processing executed in the game apparatus 3. When the power of the game apparatus 3 is turned on, the CPU 10 of the game apparatus 3 executes a startup program stored in a boot ROM (not shown), whereby each unit such as the main memory is initialized. Then, the game program stored in the optical disc 4 is read into the main memory, and the CPU 10 starts executing the game program. The flowchart shown in FIG. 18 is a flowchart showing a process performed after the above process is completed. Note that the game device 3 may be configured such that the game program is executed immediately after the power is turned on, or a built-in program that displays a predetermined menu screen is executed first after the power is turned on. The game program may be executed in response to an instruction to start the game by a selection operation.

  Note that the processing of each step in the flowcharts (including the first to fourth game examples) described below is merely an example, and if the same result is obtained, the processing order of each step is changed. Also good. Moreover, the value of the variable and the threshold value used in the determination step are merely examples, and other values may be adopted as necessary. In the present embodiment, the processing of each step in the flowchart is described as being executed by the CPU 10. However, the processing of some steps in the flowchart may be executed by a processor or a dedicated circuit other than the CPU 10. Good.

  First, in step S1, the CPU 10 executes an initial process. The initial process is a process of constructing a virtual space, placing each object appearing in the virtual space at an initial position, and setting initial values of various parameters used in the game process. In the first game example, the three objects of the terminal object 92, the surface 94, and the terminal virtual camera are arranged in the virtual space at a predetermined position and a predetermined posture. That is, data representing the initial positions and initial postures of the three objects are stored in the main memory as terminal object data 110, plane data 111, and terminal camera data 112, respectively. In addition, a virtual camera (television virtual camera) used to generate a television image is set at a predetermined position and posture. The television virtual camera is set to a predetermined position and orientation such that the keyboard object 91 and the terminal object 92 are included in the visual field range. Following step S1, the process of step S2 is executed. Thereafter, a processing loop consisting of a series of processes in steps S2 to S8 is repeatedly executed at a rate of once per predetermined time (one frame time, for example, 1/60 seconds).

  In step S <b> 2, the CPU 10 acquires terminal operation data transmitted from the terminal device 7. Since the terminal device 7 repeatedly transmits the terminal operation data to the game device 3, the game device 3 sequentially receives the terminal operation data. In the game apparatus 3, the terminal communication module 28 sequentially receives the terminal operation data, and the input / output processor 11a sequentially stores the terminal operation data in the main memory. The CPU 10 reads the terminal operation data 101 from the main memory at an appropriate timing as necessary. Following step S2, the process of step S3 is executed.

  In step S3, the CPU 10 executes a game control process. The game control process is a process of progressing the game by controlling various object movements or calculating various game parameters in accordance with an operation by the player. Hereinafter, the game control process will be described in detail with reference to FIG.

  FIG. 19 is a flowchart showing a detailed flow of the game control process (step S3) shown in FIG. In the game control process, first, in step S <b> 11, the CPU 10 calculates the attitude of the terminal device 7. The attitude of the terminal device 7 is calculated based on a physical quantity for calculating the attitude represented by the terminal operation data. In the present embodiment, the angular velocity represented by the angular velocity data 105 and the acceleration represented by the acceleration data 106 are used as the physical quantities for calculating the posture. Details of the posture calculation process will be described below.

  In the attitude calculation process, first, the CPU 10 calculates the attitude of the terminal device 7 based on the angular velocity data 105. Any method may be used to calculate the posture based on the angular velocity, but the posture includes the previous posture (the posture calculated in step S11 in the previous processing loop) and the current angular velocity (current processing). And the angular velocity obtained in step S2 in the loop). Specifically, the CPU 10 calculates the posture by rotating the previous posture by a unit time at the current angular velocity. The previous posture is represented by the terminal posture data 109 stored in the main memory, and the current angular velocity is represented by the angular velocity data 105 stored in the main memory. Therefore, the CPU 10 reads the terminal attitude data 109 and the angular velocity data 105 from the main memory, and calculates the attitude of the terminal device 7. Data representing the posture calculated as described above is stored in the main memory.

  When calculating the attitude from the angular velocity, it is preferable to determine the initial attitude. That is, when calculating the attitude of the terminal device 7 from the angular velocity, the CPU 10 first sets the initial attitude of the terminal device 7. The initial posture of the terminal device 7 may be calculated based on the acceleration data 106, or the predetermined operation is performed by causing the player to perform a predetermined operation with the terminal device 7 in a specific posture. A specific posture at the time may be set as the initial posture. In addition, when calculating the attitude of the terminal device 7 as an absolute attitude based on a predetermined direction in space, it is preferable to calculate the initial attitude, but for example, the attitude of the terminal device 7 at the start of the game When the attitude of the terminal device 7 is calculated as the relative attitude, the initial attitude need not be calculated.

  After calculating the attitude based on the angular velocity, the CPU 10 corrects the calculated attitude based on the acceleration of the terminal device 7. Here, when the terminal device 7 is substantially stationary, the acceleration applied to the terminal device 7 corresponds to the gravitational acceleration. That is, in this state, the acceleration vector represented by the acceleration data 106 represents the direction of gravity in the terminal device 7. Therefore, the CPU 10 performs correction so that the downward direction (gravity direction) of the posture calculated based on the angular velocity approaches the gravitational direction represented by the acceleration vector. That is, the posture is rotated so that the downward direction approaches the gravitational direction represented by the acceleration vector at a predetermined rate. Accordingly, the posture based on the angular velocity can be corrected so as to be a posture considering the gravity direction based on the acceleration. The predetermined ratio may be a predetermined fixed value or may be set according to detected acceleration or the like. For example, if the detected acceleration magnitude is close to the gravitational acceleration magnitude, the CPU 10 increases the ratio of lowering the posture to the gravitational direction represented by the acceleration vector, and detects the detected acceleration magnitude. In the case where the distance from the gravitational acceleration is large, the ratio may be reduced.

  Specifically, the CPU 10 reads out data representing the posture calculated based on the angular velocity and the acceleration data 106 from the main memory, and performs the above correction. Then, data representing the posture after correction is performed is stored in the main memory as terminal posture data 109. Following step S11, the process of step S12 is executed.

  In the present embodiment, the CPU 10 calculates the attitude of the terminal device 7 based on the detection results of the inertial sensors (the acceleration sensor 73 and the gyro sensor 74). Here, in other embodiments, the calculation method of the attitude of the terminal device 7 may be any method (the same applies to other game examples described later). For example, in another embodiment, when the terminal device 7 includes another sensor unit (for example, the magnetic sensor 72 or the camera 56), the attitude of the controller 5 using the detection result of the other sensor unit. May be calculated. That is, the CPU 10 can know a predetermined orientation with respect to the terminal device 7 (that is, the attitude of the terminal device 7 with reference to the predetermined orientation) from the orientation data 107 detected by the magnetic sensor 72. Therefore, in another embodiment, the CPU 10 may further calculate the attitude of the terminal device 7 using the azimuth data 107 in addition to the angular velocity data 105 and the acceleration data 106. When the orientation of the terminal device 7 is calculated using the azimuth data 107, the absolute orientation based on a predetermined direction in the real space can be calculated, and thus the orientation of the terminal device 7 is calculated more accurately. be able to. Regarding the azimuth data 107, in a place where a magnetic field other than geomagnetism is generated, the azimuth data does not strictly indicate an absolute azimuth (north, etc.), but a terminal for the direction of the magnetic field at that place. Since the relative direction of the device 7 is indicated, the posture of the terminal device 7 can be calculated even in such a case. In another embodiment, the posture may be calculated based on one or two of the three data. For example, when the game system 1 includes a camera that images the terminal device 7, the game device 3 acquires an imaging result obtained by imaging the terminal device 7 with the camera, and uses the imaging result to obtain the terminal device 7. May be calculated.

  In step S <b> 12, the CPU 10 places the terminal object 92 in the virtual space based on the attitude of the terminal device 7. The terminal object 92 may be controlled in any way as long as it is controlled based on the attitude of the terminal device 7. In the present game example, the position and posture of the terminal object 92 are controlled corresponding to the posture of the terminal device 7.

  In this game example, the posture of the terminal object 92 changes in the posture related to the rotation direction around the axis perpendicular to the screen according to the posture of the terminal device 7, and does not change (fixed) in the other rotation directions. . That is, the attitude of the terminal object 92 does not change from the attitude perpendicular to the axis. When the terminal device 7 assumes a predetermined reference posture with respect to the rotation direction around the axis perpendicular to the screen, the posture of the terminal object 92 is a posture that faces the front direction (depth direction) of the virtual space displayed on the television 2. . When the terminal device 7 is rotated from a predetermined reference posture with respect to the rotation direction around the axis perpendicular to the screen, the posture of the terminal object 92 changes in the direction corresponding to the rotation direction by an amount corresponding to the rotation amount. . That is, the attitude of the terminal object 92 changes so as to correspond to the attitude of the terminal device 7. The reference posture may be set in any way, but may be, for example, the posture of the terminal device 7 at the start of the game or may be a predetermined posture.

  In the present game example, the attitude of the terminal object 92 is determined based on the attitude of the terminal device 7 with respect to the rotation direction around the axis perpendicular to the screen. Therefore, for example, the CPU 10 calculates the attitude of the terminal object 92 based on a vector that faces the front direction of the terminal device 7 (parallel to the screen of the terminal device 7 and upward of the screen; z-axis direction shown in FIG. 8). can do. That is, the orientation of the terminal object 92 may be calculated so that the front direction of the terminal object 92 is a direction corresponding to the direction of the vector. If the front direction of the terminal device 7 is in the vertical direction, the CPU 10 replaces the vector with the depth direction of the terminal device 7 (vertical to the screen of the terminal device 7 and from the near side of the screen to the back. The orientation of the terminal object 92 may be calculated based on a vector pointing in the direction toward the side (the negative y-axis direction shown in FIG. 8). According to this, even when the player uses the terminal device 7 in a state where the screen of the terminal device 7 is set up (in a vertical state), the terminal object 92 is controlled according to an operation for changing the attitude of the terminal device 7. be able to.

  Further, the position of the terminal object 92 is determined so as to be a position corresponding to the attitude of the terminal object 92 calculated as described above. Specifically, when the attitude of the terminal device 7 becomes the reference attitude, the terminal object 92 is arranged above the center of the arc of the keyboard object 91. In addition, when the terminal device 7 rotates clockwise from a predetermined reference posture (as viewed from the upper side of the screen) with respect to the rotation direction about the axis perpendicular to the screen, the terminal object 92 has the keyboard moved by a movement amount corresponding to the rotation amount. Move on the arc of the object 91 in the right direction. On the other hand, when the terminal device 7 rotates counterclockwise from a predetermined reference posture (as viewed from the top of the screen), the terminal object 92 moves leftward on the arc of the keyboard object 91 by a movement amount corresponding to the rotation amount. .

  As a specific process in step S <b> 12, the CPU 10 reads the terminal posture data 109 from the main memory, and calculates the position and posture of the terminal object 92 based on the posture of the terminal device 7. Then, data representing the calculated position and orientation is stored in the main memory as terminal object data 110. Following step S12, the process of step S13 is executed.

  As described above, in the present game example, the game apparatus 3 arranges an object (terminal object 92) whose attitude (here, attitude and position) is controlled corresponding to the attitude of the terminal device 7 in the virtual space. (Step S12). And the game device 3 produces | generates the image showing the virtual space containing the said object as an image for displaying on a predetermined | prescribed display apparatus (TV 2) (step S4 mentioned later). As a result, the player can check the posture of the terminal device 7 on the screen, and can easily check the posture of the surface 94 that changes in accordance with the posture.

  In step S <b> 13, the CPU 10 sets a surface 94 according to the attitude of the terminal device 7. The surface 94 may be set in any way as long as it is set so as to change according to the attitude of the terminal device 7. In the present game example, the surface 94 is set based on the position and orientation of the terminal object 92 that changes according to the orientation of the terminal device 7. Specifically, the position of the surface 94 is an area on the keyboard object 91 and is set to a position below (directly below) the terminal object 92. As described above, since the terminal object 92 moves in an arc shape above the keyboard object 91 according to the change in the attitude of the terminal device 7, the surface 94 corresponds to the keyboard object according to the change in the attitude of the terminal device 7. It is set to move on the area of each of the 91 keys. Further, the posture of the surface 94 is set so as to coincide with the posture of the terminal object 92 with respect to the posture in the rotational direction around the axis that is parallel to the keyboard object 91 and perpendicular to the surface 94. That is, in the present game example, the posture of the surface 94 is set to correspond to the posture of the terminal device 7 according to the posture of the terminal device 7.

  As a specific process in step S13, the CPU 10 reads the terminal object data 110 from the main memory, and calculates the position and orientation of the surface 94 based on the position and orientation of the terminal object 92. Then, the data representing the calculated position and orientation is stored in the main memory as the surface data 111. Following step S13, the process of step S14 is executed.

  In step S14, the CPU 10 sets a terminal virtual camera. The terminal virtual camera may be set based on the position and orientation of the surface 94, and in the present game example, the terminal virtual camera is set to a position including the region of the surface 94 in the visual field range. That is, the terminal virtual camera is set so that the outer edge of the visual field range coincides with the outer edge of the surface 94. Specifically, the CPU 10 reads the surface data 111 from the main memory, and calculates the position and orientation of the terminal virtual camera based on the position and orientation of the surface 94. Then, data representing the calculated position and orientation is stored in the main memory as terminal camera data 112. Following step S14, the process of step S15 is executed.

  In step S <b> 15, the CPU 10 determines whether or not there is an input (touch input) to the touch panel 52. Specifically, the latest input position data 104 stored in the main memory is read and referred to, and it is determined whether the input position data 104 represents the coordinate value of the input position or no input. . If the determination result of step S15 is affirmative, the process of step S16 is executed. On the other hand, when the determination result of step S15 is negative, the CPU 10 ends the game control process.

  In step S <b> 16, the CPU 10 calculates a designated position on the surface 94. The designated position is a position on the surface 94 and is calculated as a position corresponding to the input position. The specific calculation method of the designated position may be any method. In this game example, the designated position is calculated so that the positional relationship of the input position with respect to the four sides of the input surface of the touch panel 52 is the same as the positional relationship of the designated position with respect to the four sides of the surface 94. In this game example, the CPU 10 calculates the designated position so that the input position on the touch panel 52 and the designated position calculated based on the input position match on the screen of the LCD 51. According to this, since the position touched on the screen by the player becomes the designated position, it becomes easier for the player to designate the position in the virtual space. In this game example, the shape of the input surface (aspect ratio) of the touch panel 52 is made equal to the shape of the surface 94 (aspect ratio), and the input surface and the surface 94 of the touch panel 52 are on the screen of the LCD 51. Match. Thus, by calculating the designated position as described above, the input position and the control position can be matched on the screen of the LCD 51.

  As a specific process of step S16, the CPU 10 first reads the latest input position data 104 stored in the main memory, and the coordinates of the input position are represented by 2 in the two-dimensional coordinate system indicating the position on the surface 94. Convert to dimensional coordinates. Next, the CPU 10 converts the two-dimensional coordinates obtained by the conversion into the three-dimensional coordinates of the virtual space coordinate system indicating the position of the three-dimensional virtual space. The three-dimensional coordinates can be calculated using the two-dimensional coordinates and the position and orientation of the surface 94 (or a plane expression representing the surface 94). Therefore, the CPU 10 reads the surface data 111 from the main memory, and calculates the three-dimensional coordinates based on the two-dimensional coordinates and the surface data 111. Then, data representing the calculated three-dimensional coordinates is stored in the main memory as designated position data 113. Following step S16, the process of step S17 is executed.

  In step S17, the CPU 10 executes a predetermined process with the designated position as an input. In the first game example, the predetermined process is a process of outputting a sound corresponding to the keyboard at the designated position. Specifically, the CPU 10 reads the designated position data 113 from the main memory, and specifies the keyboard designated by the designated position (including the designated position) from each keyboard constituting the keyboard object 91. And the sound corresponding to the specified keyboard is specified. The main memory stores association data indicating the correspondence between the keyboard and the sound to be output when the keyboard is identified, and the CPU 10 identifies the sound with reference to the association data. Further, the CPU 10 causes the DSP 11c to generate the specified sound. Thus, in the present game example, the CPU 10 specifies an area including the designated position from a plurality of areas (keyboard areas) set in the virtual space, and outputs a sound corresponding to the specified area. After step S17, the CPU 10 ends the game control process.

  According to the above game control process, the surface 94 moves in the virtual space in accordance with the change in the attitude of the terminal device 7 (step S13), and the terminal virtual camera moves (step S14). As a result, the area of the keyboard object 91 displayed on the terminal device 7 changes (steps S5 and S7 described later). That is, the keyboard that can be designated by the player using the touch panel 52 changes. Therefore, the player can easily specify a desired position of the keyboard object 91 in the virtual space by an operation for changing the attitude of the terminal device 7 and an operation for touch input.

  In this game example, a plurality of areas (areas of each keyboard) set in the virtual space are arranged in an arc shape (on the arc), and the CPU 10 responds to a change in the attitude of the terminal device 7. The surface 94 is set so as to move on the area (step S13). Then, the CPU 10 calculates a designated position on the surface 94 based on the input position (step S16). Therefore, in this game example, the player can rotate and move the surface 94 on a plurality of regions arranged in a substantially arc shape by an operation of rotating the terminal device 7 (changing the posture). According to this, both the movement of the terminal device 7 and the movement of the surface 94 are rotational movements, and both movements correspond to each other, so that the operation of the terminal apparatus 7 becomes more intuitive and easy to understand for the player. Furthermore, the player can grasp (guess) the operation method of the terminal device 7 from the arcuate arrangement of each keyboard, and for this reason, the operation of the terminal device 7 is more intuitive and easy to understand for the player. It becomes. In another embodiment, each keyboard may be arranged on a straight line, and at this time, the surface 94 may be set so as to move on the straight line in accordance with a change in the attitude of the terminal device 7. .

  Further, in the present game example, the game system 1 calculates the movement of the terminal device 7 using the acceleration sensor 73 and the gyro sensor 74, and therefore calculates the movement of the terminal device 7 with high accuracy with respect to the parallel movement. You may not be able to. In order to calculate the movement of the terminal device 7 with high accuracy, it is preferable that the terminal device 7 performs rotational movement (change in posture) and detects it. Here, in the present game example, it is possible to cause the player to guess and execute the operation of rotating the terminal device 7 from the arcuate arrangement of each keyboard as described above. The movement of the terminal device 7 can be calculated well.

  Returning to the description of FIG. 18, the process of step S4 is executed after the game control process of step S3. In step S4, a television image is generated. That is, the CPU 10 and the GPU 11b read from the main memory data representing the result of the game control process in step S3 (surface 94, object data, etc. in the game space) and store data necessary for generating a game image in the VRAM 11d. To generate a TV image. In this game example, the television virtual camera is set to include the entire keyboard object 91 and the terminal object 92 in the visual field range. Therefore, a television image representing an area in the virtual space including the keyboard object 91 and the terminal object 92 is generated (see FIG. 14). The generated television image is stored in the VRAM 11d. Following step S4, the process of step S5 is executed.

  In this game example, the TV virtual camera is fixedly set. However, in other embodiments, the TV virtual camera may be controlled to be movable. Further, the shooting range of the TV virtual camera is preferably set to be wider than the shooting range of the terminal virtual camera among the keyboard objects 91, but the entire keyboard object 91 may not be included. Good. For example, in another embodiment, the CPU 10 sets a television virtual camera to include a part of the keyboard object 91 and the terminal object 92 in the visual field range, and the television virtual camera according to the movement of the terminal object 92. May be moved.

  In step S5, a terminal image is generated based on the game control process. That is, the CPU 10 and the GPU 11b read data representing the result of the game control process in step S3 from the main memory, read data necessary for generating a game image from the VRAM 11d, and generate a terminal image. In the present embodiment, an image representing an area where the surface 94 is set in the virtual space is generated as a terminal image. Specifically, the terminal image is generated so that the input surface of the touch panel 52 and the surface 94 coincide on the screen of the LCD 51. As a result, an image representing the keyboard object 91 in the area of the surface 94 is displayed on the terminal device 7 (see FIG. 16). The generated terminal image is stored in the VRAM 11d. Following step S5, the process of step S6 is executed.

  As described above, in the present game example, the game apparatus 3 sets a predetermined object (keyboard object 91) in the virtual space and displays it as an image to be displayed on the display device (the television 2 and the LCD 51 of the terminal device 7). Then, an image representing a virtual space including the object is generated. Further, the game apparatus 3 executes a predetermined process when the position of the predetermined object is calculated as the specified position (step S15). Therefore, in this game example, the player can perform an operation of inputting an object in the virtual space by an operation of designating a three-dimensional position in the virtual space using the terminal device 7.

  Further, in the present game example, the game apparatus 3 displays a partial range of the object (keyboard object 91) as an image to be displayed on the terminal apparatus 7 so that the range changes according to the attitude of the terminal apparatus 7. Is generated (step S4). Further, the game apparatus 3 generates an image representing a range wider than the partial range of the objects as an image to be displayed on the television 2 (step S5). According to this, since a relatively enlarged image is displayed on the terminal device 7, it is easy to perform a touch input operation, and a relatively wide range of images are displayed on the television 2. Can also know the situation around the display range in the terminal device 7. The player can grasp the positional relationship of the display range in the terminal device 7 with respect to the entire object, for example, by looking at the image on the television 2.

  In step S <b> 6, the CPU 10 outputs a game image to the television 2. Specifically, the CPU 10 sends TV image data stored in the VRAM 11 d to the AV-IC 15. In response, the AV-IC 15 outputs television image data to the television 2 via the AV connector 16. As a result, the television image is displayed on the television 2. When the sound generated in step S17 is output to the television 2, the CPU 10 sends the sound data generated in step S17 to the AV-IC 15. In response to this, the AV-IC 15 outputs audio data to the television 2 via the AV connector 16. Thereby, the game sound for TV is output from the speaker 2a. Following step S6, the process of step S7 is executed.

  In step S <b> 7, the CPU 10 outputs (transmits) the game image to the terminal device 7. Specifically, the image data of the terminal image stored in the VRAM 11d is sent to the codec LSI 27 by the CPU 10, and a predetermined compression process is performed by the codec LSI 27. The compressed image data is transmitted to the terminal device 7 through the antenna 29 by the terminal communication module 28. The terminal device 7 receives the image data transmitted from the game device 3 by the wireless module 80, and a predetermined decompression process is performed on the received image data by the codec LSI 76. The image data that has undergone the decompression process is output to the LCD 51. As a result, the terminal image is displayed on the LCD 51. When the sound generated in step S17 is output to the terminal device 7, the generated sound data is sent to the codec LSI 27 by the CPU 10, and a predetermined compression process is performed by the codec LSI 27. Further, the compressed audio data is transmitted to the terminal device 7 via the antenna 29 by the terminal communication module 28. The terminal device 7 receives the audio data transmitted from the game device 3 by the wireless module 80, and a predetermined decompression process is performed by the codec LSI 76. The audio data that has undergone the decompression process is output to the sound IC 78. As a result, the terminal game sound is output from the speaker 77. Following step S7, the process of step S8 is executed.

  In step S8, the CPU 10 determines whether or not to end the game. The determination in step S8 is made based on, for example, whether or not the player has given an instruction to stop the game. If the determination result of step S8 is negative, the process of step S2 is executed again. On the other hand, if the determination result of step S8 is affirmative, the CPU 10 ends the game process shown in FIG. When ending the game process, a process such as saving game data in a memory card or the like may be executed. Thereafter, a series of processes in steps S2 to S8 are repeatedly executed until it is determined in step S8 that the game is to be ended.

  As described above, in the present game example, the keyboard arranged in the virtual space is displayed on the display unit (LCD) of the terminal device 7. When the position of the keyboard is calculated as a position (designated position) in the virtual space based on the input position, the game system 1 outputs a sound corresponding to the keyboard corresponding to the position. Thus, the player can perform an operation of playing the keyboard instrument by the terminal device 7. Further, according to the above game processing, the position on the surface 94 in the virtual space is designated by the input to the touch panel 52 of the terminal device 7 (step S16), and the posture of the surface 94 is determined according to the posture of the terminal device 7. Changes (step S13). According to this, the player can easily specify the position of each keyboard in the three-dimensional virtual space by specifying the position using the touch panel 52 while changing the posture of the surface 94.

[7. Second game example]
Hereinafter, a second game example executed in the game system 1 of the present embodiment will be described with reference to FIGS. In the second game example, the player can use the terminal device 7 to draw points and lines in a three-dimensional virtual space (game space). The game image representing the virtual space is displayed on the two display devices of the television 2 and the terminal device 7. In the present game example, the terminal device 7 is used not only as a display device but also as an operation device. Therefore, the controller 5 may not be used, and the game system 1 may be configured not to include the controller 5.

  FIG. 20 is a diagram illustrating the television 2 and the terminal device 7 when a game operation is performed in the present game example. As shown in FIG. 20, a three-dimensional virtual space where a predetermined surface 201 is installed is displayed on the television 2, and this surface 201 is also displayed on the terminal device 7. In the terminal device 7, the surface 201 is displayed so that the surface 201 and the display screen coincide with each other. That is, the surface 201 corresponds to the input surface of the touch panel 52 included in the terminal device 7. The surface 201 is a rectangular plane having the same aspect ratio as the input surface of the touch panel 52 (screen of the LCD 51). Although details will be described later, the posture of the surface 201 corresponds to the posture of the terminal device 7, and the posture of the surface 201 in the virtual space changes by changing the posture of the terminal device 7. In FIG. 20, since the terminal device 7 has a posture in which the touch panel 52 is horizontal, the surface 201 is also horizontal in the virtual space. An icon 202 is displayed on the terminal device 7. Although details will be described later, the icon 202 is used to move (rotate) the locus generated in the virtual space.

  The player can draw points and lines on the touch panel 52 of the terminal device 7 with the touch pen 60 or a finger. When a point or line is drawn, the drawn point or line (line A1 in FIG. 20) is generated and displayed on the surface 201 in the virtual space, as shown in FIG. Specifically, when an input (touch input) is performed on the touch panel 52, a position (designated position) on the surface 201 corresponding to the position (input position) where the input is performed is calculated. In this game example, an input point or line is displayed by placing an object for representing the point or line at the specified position. As described above, the player can draw points and lines in the three-dimensional virtual space by an input operation on the touch panel 52.

  FIG. 21 is a diagram illustrating the television 2 and the terminal device 7 when the terminal device 7 is tilted from the state illustrated in FIG. 20. In the state illustrated in FIG. 20, the input surface of the touch panel 52 is horizontal, whereas in the state illustrated in FIG. 21, the attitude of the terminal device 7 is changed so that the input surface is vertical. When the attitude of the terminal device 7 is changed so that the input surface of the touch panel 52 is vertical, as shown in FIG. 21, the surface 201 in the virtual space is also changed to be vertical. In this game example, the surface 201 rotates according to the attitude of the terminal device 7 with the center position of the surface 201 as the center of rotation. Further, when the posture of the surface 201 changes, the viewpoint and line-of-sight direction of the image displayed on the terminal device 7 change so that the surface 201 and the display screen match.

  In the state shown in FIG. 21 as well, as in the state shown in FIG. 20, the player can draw points and lines in the three-dimensional virtual space by an input operation on the touch panel 52. In other words, when the player draws a line on the touch panel 52 in the state shown in FIG. 21, a line is generated and displayed on the surface 201 that is parallel to the vertical direction in the virtual space. Here, in the state shown in FIG. 21, since the posture of the surface 201 is different from the state shown in FIG. 20, the positions and orientations of lines generated by touch input are different from those in FIG. That is, the line A1 generated in the state shown in FIG. 20 is generated in a horizontal plane, whereas the line A2 generated in the state shown in FIG. 21 is generated in a vertical plane. Therefore, the figure formed by the line A1 and the line A2 is not planar but is three-dimensional.

  As described above, in the present game example, a figure (a point or a line) is generated on the surface 201 in the virtual space in accordance with the input to the touch panel 52 of the terminal device 7, and also according to the attitude of the terminal device 7. The posture of the surface 201 changes. According to this, the figure generated on the surface 201 can be made three-dimensional by changing the posture of the surface 201. That is, the player can freely draw a figure three-dimensionally in a three-dimensional virtual space by an input operation on the touch panel 52.

  In this game example, the player can also draw a line in the virtual space by an operation for changing the posture of the terminal device 7. FIG. 22 is a diagram illustrating the television 2 and the terminal device 7 when the terminal device 7 is tilted from the state illustrated in FIG. 21. In the state shown in FIG. 21, the normal direction of the input surface of the touch panel 52 faces the direction of the television 2, whereas in the state shown in FIG. It is suitable. Further, here, when the player changes the attitude of the terminal apparatus 7 from the state shown in FIG. 21 to the state shown in FIG. It shall be. In the case where the state shown in FIG. 21 is changed to the state shown in FIG. 22, the posture of the surface 201 changes according to the posture of the terminal device 7 as shown in FIG. In this case, even if the player continues to touch the same position, the designated position in the virtual space changes as the posture of the surface 201 changes. Therefore, as shown in FIG. 22, a line A3 representing a locus whose designated position has changed is generated and displayed. Thus, in this game example, in addition to the operation of drawing a line on the touch panel 52, it is also possible to draw a line in the virtual space by an operation of changing the attitude of the terminal device 7. Further, it is possible to draw a line in a direction not parallel to the surface 201 by an operation of changing the posture of the terminal device 7.

  In this game example, the player can also change the orientation (posture) of the graphic generated in the virtual space. FIG. 23 is a diagram showing the television 2 and the terminal device 7 when the figure is rotated from the state shown in FIG. In the state shown in FIG. 21, the normal direction of the input surface of the touch panel 52 faces the direction of the television 2, whereas in the state shown in FIG. 23, the normal direction is the same as the state shown in FIG. It is slightly sideways than the direction of 2. When the player changes the attitude of the terminal device 7 from the state shown in FIG. 21 to the state shown in FIG. 22, the player performs a predetermined operation (here, an operation of continuously touching the position of the icon 202 on the touch panel 52). It shall be. As described above, when the posture of the terminal device 7 is changed in a state where a predetermined operation is performed, the posture of the generated graphic changes with the posture of the surface 201 according to the posture of the terminal device 7. For example, in FIG. 23, the figure composed of the lines A1 and A2 rotates together with the surface 201 about the vertical direction in the virtual space in response to the terminal device 7 rotating about the vertical direction. At this time, the orientation of the graphic is changed and displayed on the television 2. Therefore, the player can see the created figure from various directions and confirm the shape of the figure by changing the posture of the terminal device 7 while touching the icon 202. In addition, regarding the image displayed on the terminal device 7, since the line-of-sight direction changes with the posture of the surface 201, the figure appears not to move. In the present game example, the predetermined operation is an operation of touching the icon 202. However, in other embodiments, for example, an operation of pressing a predetermined button of the terminal device 7, for example, There may be.

(Game processing in the second game example)
Next, details of the game process executed by the game system 1 in the second game example will be described. First, various data used in the game process will be described. FIG. 24 is a diagram showing various data used in the game process. In FIG. 24, it is a figure which shows the main data memorize | stored in the main memory (the external main memory 12 or the internal main memory 11e) of the game device 3. As shown in FIG. 24, the main memory of the game apparatus 3 stores a game program 210, terminal operation data 101, and processing data 211. In addition to the data shown in FIG. 24, the main memory stores data necessary for the game, such as image data of various objects appearing in the game and sound data used for the game. Hereinafter, various data shown in FIG. 24 will be described. 24, the same data as the data shown in FIG. 17 is denoted by the same reference numerals as those in FIG. 17, and detailed description thereof is omitted.

  The game program 210 is for causing the computer (CPU 10) of the game apparatus 3 to execute game processing in the second game example. A part or all of the game program 210 is read from the optical disk 4 and stored in the main memory at an appropriate timing after the game apparatus 3 is turned on. Note that the game program 210 may be obtained from the flash memory 17 or an external device of the game apparatus 3 (for example, via the Internet) instead of the optical disc 4. A part (for example, a program for calculating the attitude of the controller 5 and / or the terminal device 7) included in the game program 210 may be stored in advance in the game apparatus 3.

  The terminal operation data 101 is data representing a player's operation on the terminal device 7 as in the first game example. In the present game example, the controller 5 is not used as an operating device and is not shown. However, the main memory may store controller operation data representing the player's operation on the controller 5.

  The processing data 211 is data used in a game process (FIG. 25 and the like) described later. The processing data 211 includes terminal posture data 109, surface posture data 212, surface coordinate data 213, space coordinate data 214, graphic data 215, and terminal camera data 217 similar to those in the first game example. In addition to the data shown in FIG. 24, the processing data 211 includes various data used in the game process, such as data representing various parameters set for various objects appearing in the game.

  The surface posture data 212 is data representing the posture of the surface 201 set in the virtual space. The posture of the surface 201 represented by the surface posture data 212 may be represented by a rotation matrix, may be represented by a cubic vector or three angles, or may be represented by a plane equation. In this example game, the position of the surface 201 is fixed at a predetermined position.

  The plane coordinate data 213 is data representing the plane coordinates of the designated position. Here, the plane coordinates are coordinates that express the designated position by a coordinate system (plane coordinate system) with the plane 201 as a reference. Since the designated position is a point on the surface 201, the surface coordinate data 213 may represent a two-dimensional coordinate value. The designated position on the surface 201 corresponds to the input position on the touch panel 52, and the surface coordinate data 213 is calculated based on the input position data 104.

  The spatial coordinate data 214 is data representing the spatial coordinates of the designated position. Here, the space coordinates are coordinates representing the designated position by a coordinate system (space coordinate system) based on the virtual space. The spatial coordinate data 214 represents a designated position by a three-dimensional coordinate value. The spatial coordinate data 214 is calculated based on the surface orientation data 212 and the surface coordinate data 213.

  The graphic data 215 is data representing a graphic formed by a designated position. The graphic data 215 includes one or more trajectory data 216. The trajectory data 216 is data representing the trajectory of the calculated designated position. Specifically, the trajectory data 216, when input is continuously performed on the touch panel 52 (that is, when the input position is continuously detected), the designated position corresponding to the input position is in order from the oldest one. To express. The trajectory data 216 represents a coordinate value representing the designated position in the spatial coordinate system. Further, in this game example, when a next touch input is newly performed after a certain touch input is completed, the next touch input is performed separately from the trajectory data 216 generated according to the certain touch input. In response, new trajectory data 216 is generated and stored in the main memory.

  The terminal camera data 217 is data representing the position and orientation of a virtual camera (terminal virtual camera) used for generating a terminal image. In this example game, the position and orientation of the terminal virtual camera are set based on the position of the surface 201.

  In addition to the data 212 to 217, the main memory stores various data necessary for the game process. In this game example, for example, flag data indicating whether or not the player is in the gripping state is stored. The grasping state is a state where the icon 202 is touched, and is a state where a figure formed by the locus of the designated position can be moved.

  Next, details of the game processing executed in the game apparatus 3 will be described with reference to FIGS. In the second game example, a main flowchart showing a general process flow in the game process is the same as that in FIG. Hereinafter, the game process in the second game example will be described with a focus on differences from the first game example.

  Also in the second game example, as in the first game example, first, in step S1, the CPU 10 executes an initial process. The initial process is a process of constructing a virtual space, placing each object appearing in the virtual space at an initial position, and setting initial values of various parameters used in the game process. In the present game example, the surface 201 is arranged in the virtual space at a predetermined position and a predetermined posture. Data representing the initial position and initial posture of the surface 201 is stored in the main memory. Further, a virtual camera (television virtual camera) used for generating a television image and a virtual camera (terminal virtual camera) used for generating a terminal image have a predetermined initial position and initial posture. Set by. Data representing the initial position and initial posture of the television virtual camera is stored in the main memory, and data representing the initial position and initial posture of the terminal virtual camera is stored in the main memory as terminal camera data 217. Note that the television virtual camera is set to a predetermined position and orientation such that the surface 201 is included in the visual field range. The initial position and initial posture of the terminal virtual camera are set so that the line-of-sight direction is perpendicular to the surface 201 and the surface 201 matches the display screen of the LCD 51. In step S1, data indicating that the gripping state is not established is stored in the main memory as the flag data. Following step S1, the process of step S2 is executed.

  In step S <b> 2, the CPU 10 acquires terminal operation data transmitted from the terminal device 7. The process in step S2 is the same as that in the first game example. Following step S2, the process of step S3 is executed.

  In step S3, the CPU 10 executes a game control process. In the second game example, the game control process is a process for generating an object representing a point or a line in the virtual space or moving the object in accordance with an operation by the player. The details of the game control process will be described below with reference to FIG.

  FIG. 25 is a flowchart showing a detailed flow of the game control process (step S3) in the second game example. In the game control process, first, in step S <b> 21, the CPU 10 calculates the attitude of the terminal device 7. The process in step S21 is the same as that in step S11. Following step S21, the process of step S22 is executed.

  In step S <b> 22, the CPU 10 sets the surface 201 so that the posture changes according to the posture of the terminal device 7. In the present game example, since the position of the surface 201 does not change, the CPU 10 calculates the posture of the surface 201 based on the posture of the terminal device 7. In this game example, the posture of the surface 201 in the virtual space is controlled to correspond to the posture of the terminal device 7 in the real space. Specifically, the orientation of the terminal device 7 when the y-axis (see FIG. 9) of the terminal device 7 is horizontal and faces the television 2 is set as the reference orientation, and the terminal device 7 assumes the reference orientation. 201 becomes horizontal in the virtual space. When the posture of the terminal device 7 changes from the reference posture, the surface 201 changes in accordance with the amount of change from the posture of the surface 201 at the time of the reference posture to the direction according to the change direction of the posture of the terminal device 7. It is rotated by the amount (in this game example, the amount of rotation of the terminal device 7 is equal to the amount of rotation of the surface 201). Note that the posture of the surface 201 may be controlled in any way as long as it is controlled to change in accordance with the change in the posture of the terminal device 7.

  As a specific process of step S <b> 22, the CPU 10 reads the terminal attitude data 109 from the main memory, and calculates the attitude of the surface 201 based on the attitude of the terminal device 7. Data representing the calculated posture of the surface 201 is stored in the main memory as the surface posture data 212. In the present game example, the posture of the surface 201 may be expressed as, for example, three vectors representing the three-axis directions in the three-dimensional surface coordinate system with the surface 201 as a reference by a spatial coordinate system. Following step S22, the process of step S23 is executed.

  In step S23, the CPU 10 executes a graphic movement process. The graphic movement process is a process of moving (rotating) a graphic formed by a designated position set in accordance with an input by the player according to the attitude of the terminal device 7. Details of the figure moving process will be described later. Following step S23, the process of step S24 is executed.

  In step S <b> 24, the CPU 10 determines whether touch input has been performed on the touch panel 52. This determination can be made by referring to the input position data 104 read in step S2. If the determination result of step S24 is affirmative, the process of step S25 is executed. On the other hand, when the determination result of step S24 is negative, the processes of steps S25 to S30 are skipped, and the process of step S31 described later is executed.

  In step S25 and step S26, the designated position is calculated based on the input position. As described above, in the present game example, the position on the surface 201 determined by the input position is calculated as the designated position. FIG. 26 is a diagram for explaining a method of calculating the designated position based on the input position on the input surface of the touch panel 52. As shown in FIG. 26, the designated position P2 on the surface 201 is calculated as a position on the surface 201 corresponding to the input position P1 on the input surface of the touch panel 52. In this example game, first, the plane coordinates (Ps, Pt) of the designated position P2 are calculated based on the input position P1. The plane coordinates (Ps, Pt) are coordinates expressed in a plane coordinate system (stu coordinate system) with the plane 201 as a reference. Here, the surface coordinate system includes an s-axis that faces the horizontal direction of the surface 201, a t-axis that faces the vertical direction of the surface 201, and a u-axis that faces a direction perpendicular to the surface 201. Therefore, the position on the surface 201 is expressed by two-dimensional coordinates (Ps, Pt) of the s direction component and the t direction component. The u-axis of the surface coordinate system is set for the purpose of representing a position not on the surface 201 in the surface coordinate system in the graphic movement process described later. The surface coordinates (Ps, Pt) are calculated by, for example, scaling the coordinate value of the input position P1 according to the ratio between the size of the surface 201 and the size of the input surface of the touch panel 52 (multiply by a predetermined constant). Is done. When the plane coordinate system and the spatial coordinate system are set so that the ratio is “1”, the coordinate value of the input position P1 can be used as the plane coordinates (Ps, Pt) as it is.

When the plane coordinates (Ps, Pt) of the designated position P2 are calculated, the plane coordinates (Ps, Pt) are then converted into spatial coordinates (PS, PT, PU). Spatial coordinates (PS, PT, PU) are coordinate values representing the designated position P2 in a spatial coordinate system (STU coordinate system). Note that the position of the origin of the surface coordinate system and the direction of each axis in the spatial coordinate system are determined by the position and orientation of the surface 201. Therefore, based on the position (the following (DS, DT, DU)) and orientation (the following vector A and vector B) of the surface 201 and the surface coordinates (Ps, Pt), the spatial coordinates (PS, PT, PU) are calculated. Can be calculated. Specifically, the spatial coordinates (PS, PT, PU) can be calculated according to the following equation (1).
PS = Ps × AS + Pt × AS + DS
PT = Ps × AT + Pt × AT + DT
PU = Ps × AU + Pt × AU + DU (1)
In the above equation (1), each component of the unit vector A that faces the positive direction of the s-axis in the plane coordinate system is (AS, AT, AU), and the unit vector B that faces the positive direction of the t-axis in the plane coordinate system Let each component be (BS, BT, BU). The position of the origin (in the space coordinate system) in the plane coordinate system is defined as (DS, DT, DU).

  Specifically, first, in step S25, the CPU 10 calculates the plane coordinates of the designated position corresponding to the input position. As described above, the surface coordinates are calculated as the position P2 on the surface 201 corresponding to the input position P1 on the input surface of the touch panel 52. The specific method of calculating the surface coordinates may be any method, but the surface coordinates are based on the positional relationship of the input position P1 with respect to the four sides of the input surface of the touch panel 52 and the position of the designated position P2 with respect to the four sides of the surface 201. Calculated so that the relationship is the same. Specifically, in the present game example, as described above, the plane coordinates are calculated by scaling the coordinate value of the input position P1 according to the above ratio. As a specific process of step S25, the CPU 10 uses the surface coordinate system to express the position on the surface 201 corresponding to the input position from the input position represented by the input position data 104 read in step S2. Is calculated. Then, data representing the calculated two-dimensional coordinates is stored in the main memory as plane coordinate data 213. Following step S25, the process of step S26 is executed.

  In step S26, the CPU 10 calculates the spatial coordinates of the designated position from the surface coordinates. As described above, the spatial coordinates can be calculated based on the surface coordinates and the position and orientation of the surface 201 (that is, the origin position of the surface coordinate system and the orientation of each coordinate axis in the space coordinate system). That is, the CPU 10 reads the surface orientation data 212 and the surface coordinate data 213 from the main memory, and calculates the spatial coordinates by substituting the surface coordinates and the position and orientation of the surface 201 into the above equation (1). Data representing the calculated spatial coordinates is stored in the main memory as spatial coordinate data 214. Following step S26, the process of step S27 is executed.

  According to steps S25 and S26, the CPU 10 calculates a position (in the virtual space) corresponding to the input position on the display screen of the touch panel 52 (LCD 51) as the designated position in the virtual space. In other words, on the display screen, the position touched by the player is the designated position, so that the player can designate the designated position with an intuitive and easy-to-understand operation.

  In step S27, the CPU 10 determines whether or not touch input has been started (whether or not it is immediately after touch input has been started). Specifically, the CPU 10 determines whether or not there is a touch input in the previous processing loop (a processing loop including a series of processes in steps S2 to S8). In consideration of erroneous detection of the touch panel 52 and an erroneous operation by the player, the CPU 10 may determine whether there has been a touch input continuously for a predetermined number of times in addition to the previous touch input. If the determination result of step S27 is affirmative, the process of step S28 is executed. On the other hand, when the determination result of step S27 is negative, the process of step S29 is executed.

  In step S28, the CPU 10 stores the designated position calculated in steps S25 and S26 as the start point of a new locus. Specifically, the CPU 10 stores the spatial coordinate data 214 stored in the main memory as new trajectory data 216 in the main memory. As a result, one trace data 216 included in the graphic data 215 is added. That is, one locus constituting the figure is added. Following step S28, the process of step S30 is executed.

  In step S29, the CPU 10 stores the designated position calculated in steps S25 and S26 as the end point of the existing trajectory. Specifically, the CPU 10 updates the latest one of the trajectory data 216 included in the graphic data 215 so as to add the spatial coordinate data 214 to the trajectory data 216. As a result, the specified position is added to the end point of the latest locus. Following step S29, the process of step S30 is executed.

  In step S30, the CPU 10 places a predetermined object at the designated position calculated in steps S25 and S26. Here, the predetermined object may have any shape, but in the present game example, an object representing the locus of the designated position is arranged. That is, the CPU 10 arranges objects so that one or more designated positions included in one trajectory data 216 form one line. Specifically, in one process of step S30, a linear object connecting the latest specified position and the previous specified position is arranged. The linear object may be, for example, a cylindrical object so that it looks linear when viewed from any direction. When the latest specified position is the start point of the locus, an object representing the point (for example, a spherical object) is arranged. In the present game example, the object placed in step S30 continues to be placed in the subsequent processing, but in other embodiments, for example, it may be deleted in accordance with an instruction from the player. Further, the objects arranged in step S30 may be sequentially (automatically) moved in a predetermined direction in response to a predetermined operation performed by the player. According to this, for example, the player can draw a straight line simply by specifying one point in the virtual space, or draw a wavy line by reciprocating the input position in a direction perpendicular to the moving direction of the object, More colorful lines can be drawn easily. Following step S30, the process of step S31 is executed.

  In step S <b> 31, the CPU 10 controls the terminal virtual camera according to the orientation of the surface 201. FIG. 27 is a diagram illustrating a positional relationship between the surface 201 and the terminal virtual camera 205. FIG. 27 is a diagram of the surface 201 viewed from a direction parallel to the surface 201. As illustrated in FIG. 27, when the posture of the surface 201 changes, the position and posture of the terminal virtual camera 205 change according to the posture of the surface 201. Specifically, the terminal virtual camera 205 is set so that the surface 201 is included in the visual field range. In this game example, the terminal virtual camera 205 is set so that the outer periphery of the surface 201 matches the outer edge of the visual field range of the terminal virtual camera 205. Therefore, the position of the terminal virtual camera 205 is set to a position that is on a straight line that passes through the center position P3 of the surface 201 and is perpendicular to the surface 201, and that the distance from the surface 201 is a predetermined distance corresponding to the visual field range. The Further, the attitude of the terminal virtual camera 205 is set so as to face the center position P3 of the surface 201. By setting the terminal virtual camera 205 as described above, the surface 201 is displayed on the LCD 51 of the terminal device 7 so that the input surface of the touch panel 52 and the surface 201 coincide on the screen. (See FIG. 21 etc.). According to this, the designated position is a position on the screen touched by the player, and an object representing a point or a line is generated and displayed at the position, so that the operation of the touch panel 52 is made intuitive and easy to understand. Can do.

  As a specific process of step S31, the CPU 10 reads the surface orientation data 212 from the main memory, and calculates the position and orientation of the terminal virtual camera 205 based on the position and orientation of the surface 201. Data representing the calculated position and orientation is stored in the main memory as terminal camera data 217. After step S31, the CPU 10 ends the game control process.

  Next, details of the graphic movement process will be described. FIG. 28 is a flowchart showing a detailed flow of the graphic movement process (step S23) shown in FIG. In the graphic movement process, first, in step S41, the CPU 10 determines whether or not the current state is a gripping state. This determination can be made by reading the flag data stored in the main memory and referring to the flag data. If the determination result of step S41 is affirmative (the gripping state), the process of step S45 described later is executed. On the other hand, when the determination result of step S41 is negative (not the gripping state), the process of step S42 is executed.

  In step S42, the CPU 10 determines whether or not the gripping state has been started. Here, the case where the gripping state is started means that there is no touch input to the icon 202 in the previous processing loop (the processing loop of steps S2 to S8), and there is a touch input to the icon 202 in the current processing loop. It is. Specifically, the CPU 10 reads the flag data from the main memory, indicates that the flag data is not in the grabbing state, and if the input position data 104 read in step S2 represents the position of the icon 202, the grabbing is performed. It is determined that the state has started. On the other hand, if the flag data indicates that it is in the gripping state or if the input position data 104 read in step S2 indicates a position other than the icon 202, it is determined that it is not the time when the gripping state is started. . When the determination result of step S42 is affirmative, the process of step S43 is executed. On the other hand, when the determination result of step S42 is negative, the CPU 10 ends the graphic movement process.

  In step S43, the CPU 10 saves the graphic formed at the designated position. In this game example, the CPU 10 stores each designated position constituting the figure as a coordinate value in the plane coordinate system. That is, the CPU 10 reads the graphic data 215 and the surface orientation data 212 from the main memory, and converts the spatial coordinates of one or more specified positions represented by the graphic data 215 into surface coordinates. This conversion is performed using the posture of the surface 201 represented by the surface posture data 212. The CPU 10 stores data representing the converted plane coordinates in the main memory. Note that this data is stored in a format in which the designated positions are grouped for each locus, similarly to the locus data 216 included in the graphic data 215. Following step S43, the process of step S44 is executed.

  In step S44, the CPU 10 sets the flag data so as to indicate a gripping state. That is, the CPU 10 updates the flag data stored in the main memory so as to indicate that it is in the gripping state. Thus, until then, until there is no touch input to the icon 202, it is determined in the determination process of step S41 that the gripping state is established. Following step S44, a process of step S47 described later is executed.

  On the other hand, in step S45, the CPU 10 determines whether or not the gripping state has ended. That is, if the input position data 104 read in step S2 represents a position other than the icon 202 or indicates that there is no touch input, the CPU 10 determines that the gripping state has ended. On the other hand, when the input position data 104 indicates the position of the icon 202, it is determined that the gripping state has not ended (continues). If the determination result of step S45 is affirmative, the process of step S46 is executed. On the other hand, if the determination result of step S45 is negative, the process of step S46 is skipped and the process of step S47 is executed.

  In step S46, the CPU 10 sets the flag data so as to release the gripping state. That is, the CPU 10 updates the flag data stored in the main memory so as to indicate that it is not in the gripping state. As a result, in the subsequent processing, it is determined that the gripping state is not established in the determination processing in step S41. Following step S46, the process of step S47 is executed.

  In step S47, the CPU 10 moves the figure formed at the designated position. In this example game, the figure rotates according to a change in the attitude of the controller device 7 (that is, according to a change in the attitude of the surface 201). Here, the rotation center of the figure is the center position of the surface 201 as with the surface 201. Here, as described above, the spatial coordinates of the current (after rotation) designated position can be calculated based on the surface coordinates of the designated position and the current posture of the surface 201. Therefore, the CPU 10 reads out the data representing the surface coordinates stored in step S43 and the surface orientation data 212 from the main memory, and converts the surface coordinates into spatial coordinates using the orientation of the surface 201. As a result, the spatial coordinates of the designated position that forms the rotated figure according to the orientation of the surface 201 can be calculated. The CPU 10 stores data representing one or more calculated spatial coordinates in the main memory as new graphic data 215. After step S47, the CPU 10 ends the graphic movement process.

  According to the graphic movement process, the CPU 10 moves (rotates) the object representing the graphic based on an operation on the terminal device 7 (step S47). Therefore, the player can see the drawn figure from various positions or directions, and can easily confirm the shape of the drawn figure. In this example game, since the figure has a three-dimensional shape, it is particularly effective to see the figure from various positions or directions.

  Further, in this game example, the CPU 10 satisfies a predetermined condition (specifically, that an operation for designating the icon 202 has been performed) regarding the operation on the terminal device 7 (Yes in step S41 or step S42). Yes), the graphic object is moved based on the attitude of the terminal device 7 (step S47). On the other hand, when the predetermined condition is not satisfied (No in step S42), the CPU 10 does not move the object based on the attitude of the terminal device 7. Based on the above, the player can change the orientation of the figure by an intuitive and easy operation of changing the posture of the terminal device 7. Furthermore, according to the above, since the player can change the orientation of the figure by the same operation as the operation of changing the posture of the surface 201, the operability can be further improved.

  In this game example, in order to move a figure in the figure movement process, the CPU 10 stores the surface coordinates of each designated position constituting the figure. That is, the surface coordinates of the designated position are stored in the process of step S43, and the process of step S47 for moving the figure uses the surface coordinates and the posture of the current surface 201 to determine the position of the designated position after the movement. Spatial coordinates were calculated. Here, in another embodiment, the CPU 10 may store the spatial coordinates of each designated position constituting the figure in the process of step S43. At this time, in the process of step S47, the CPU 10 may rotate the figure in accordance with the change from the attitude of the surface 201 in the previous process loop to the attitude of the surface 201 in the current process loop. Specifically, the figure may be moved (rotated) in the same direction as the change direction of the posture of the surface 201 by the same amount as the change amount of the posture of the surface 201. Here, in the method of storing the spatial coordinates, an error occurs between the change of the surface 201 and the change of the figure in the process of step S47, and the error may be accumulated by repeating the process of step S47. Therefore, there is a possibility that the figure cannot be accurately rotated in accordance with the rotation of the surface 201. On the other hand, according to the present game example, by saving the plane coordinates in step S43, the designated position after movement can be calculated using the saved plane coordinates and the current posture of the plane 201. According to this, since the error is not accumulated, the figure can be accurately rotated in accordance with the rotation of the surface 201.

  Moreover, in the process of step S47 in the present game example, the entire figure arranged in the virtual space has been moved. Here, in other embodiments, only a part of the graphic may be moved. For example, the CPU 10 may store only the trajectory designated by the player in step S43 and move only the trajectory in step S47. The trajectory may be specified by any method, but for example, the trajectory displayed at the input position on the screen may be specified.

  Returning to the description of the main flowchart (steps S1 to S8), the process of step S4 is executed after the game control process of step S3. In step S4, a television image is generated. Also in the second game example, the specific method for generating an image for television is the same as in the first game example. Following step S4, the process of step S5 is executed.

  Further, in the present game example, by placing a predetermined object at the designated position (step S30), an image of the object is generated at a position on the television image corresponding to the designated position. Therefore, the player can easily confirm the position where the input has been performed on the input surface of the touch panel 52, and can easily confirm the graphic drawn on the input surface. The player can draw a three-dimensional figure in the virtual space by drawing a point or a line on the input surface. Further, in the present game example, the object is generated so as to represent a locus of each designated position determined by each input position (input position) continuously input to the input surface (step S30). Therefore, an image representing the locus is generated and displayed on the television 2. Thereby, the player can easily confirm the line drawn on the touch panel 52. In addition, since the displayed images are different depending on whether the player inputs a point with respect to the input surface or when inputting with a line, the game apparatus more accurately reflects the input on the input surface. Can be displayed.

  In this game example, the surface 201 is displayed so that the player can easily confirm the posture of the surface 201 (the posture of the terminal device 7). Furthermore, the surface 201 is generated as a translucent object so that the player can easily understand the depth relationship between the surface 201 and the figure. However, in other embodiments, the surface 201 may not be displayed (may be treated as being transparent) or may be generated as an opaque object.

  In step S5, a terminal image is generated based on the game control process. Also in the second game example, the specific method for generating the terminal image is the same as in the first game example. In this game example, an image representing an area where the surface 201 is set in the virtual space is generated as a terminal image. Specifically, the terminal image is generated so that the input surface of the touch panel 52 and the surface 201 coincide on the screen of the LCD 51. Thereby, the image of the surface 201 is displayed on the terminal device 7 (see FIG. 21 and the like). Similar to the television image, the terminal image includes the image of the object arranged in step S30. Therefore, the player can also confirm the graphic drawn on the input surface by looking at the screen of the terminal device 7. In the terminal image, the surface 201 may not be displayed, or may be generated as a translucent or opaque object. Following step S5, the process of step S6 is executed.

  In this game example, in order to generate an image representing a designated position in a television image (and a terminal image), an object is arranged at the designated position in the virtual space. Here, the image representing the designated position may be generated by any method. For example, in another embodiment, the game apparatus 3 calculates the position on the television image (and the terminal image) corresponding to the specified position, and places the predetermined image at the position, thereby determining the specified position. An image to represent may be generated.

  According to the processing in steps S4 and S5, the CPU 10 generates a television image using a television virtual camera in which the viewing direction is set independently of the orientation of the surface 201, and the positional relationship with the surface 201 is constant. The terminal image is generated using the terminal virtual camera that is set so that the line-of-sight direction is substantially perpendicular to the surface 201 (more specifically, the surface 201 is substantially perpendicular). Therefore, when the player changes the posture of the terminal device 7, the line-of-sight direction does not change in the image displayed on the television 2, while the line-of-sight direction changes in the image displayed on the terminal device 7 according to the posture. (FIGS. 20 to 23, etc.). According to this, since the orientation of the surface 201 is displayed as being constant in the terminal device 7, an operation for designating the position on the surface 201 is facilitated. Further, since the posture of the surface 201 is changed and displayed on the television 2, the posture of the surface 201 in the virtual space can be easily grasped. That is, according to the present game example, the player can easily perform an operation of designating a position on the surface 201 using the terminal device 7, and can easily make the posture of the surface 201 by looking at the screen of the television 2. Therefore, the position in the three-dimensional virtual space can be easily specified.

  In the second game example, the processes in steps S6 to S8 are the same as in the first game example. Also in the second game example, similar to the first game example, the series of processes in steps S2 to S8 is repeatedly executed until it is determined in step S8 that the game is to be ended.

  As described above, according to the game process in the second game example, the position on the surface 201 in the virtual space is designated by the input to the touch panel 52 of the terminal device 7 (step S26), and the terminal device 7 The posture of the surface 201 changes according to the posture (step S22). According to this, the player can designate a position in the three-dimensional virtual space by designating the position using the touch panel 52 while changing the posture of the surface 201. Thus, according to the present game example, the position in the three-dimensional space can be easily input using the input device having a flat input surface called the touch panel 52. According to this, for example, as in the above-described embodiment, the player inputs a point or a line to the touch panel 52 and appropriately changes the posture of the terminal device 7 to appropriately change the three-dimensional picture in the virtual space. It is also possible to draw.

(Modification of the second game example)
In the second game example, the posture of the surface 201 is changed according to the posture of the terminal device 7, but in another embodiment, the surface 201 may be moved by other methods. Hereinafter, with reference to FIG. 29 to FIG. 32, a modified example regarding the moving method of the surface 201 will be described.

  FIG. 29 is a flowchart showing the flow of the game control process in a modification of the second game example. In FIG. 29, the same processes as those shown in FIG. 25 are denoted by the same step numbers, and detailed description thereof is omitted.

  In this modification, the process of step S51 is executed after step S21. In step S <b> 51, the CPU 10 calculates the position and orientation of the surface 201 according to the orientation of the terminal device 7. That is, the CPU 10 controls the posture of the surface 201 so as to correspond to the posture of the terminal device 7 and controls the position of the surface 201 according to the posture of the terminal device 7. Here, the posture of the surface 201 is controlled based on the posture of the terminal device 7 as in the second game example. Further, the position of the surface 201 is controlled according to the posture of the surface 201 so that the surface 201 rotates and moves with the position outside the surface 201 as the center of rotation.

  FIG. 30 is a diagram illustrating movement of the surface 201 in the present modification. In FIG. 30, a point Pc is a center point of rotation of the surface 201, and is fixedly set in the game space. In FIG. 30, the point Pc is set on the same side as the terminal virtual camera 205 with respect to the plane 201. When the attitude of the terminal device 7 changes, the position of the surface 201 changes so as to rotate around the reference point Pc as the attitude changes, as shown in FIG. Specifically, the position of the surface 201 is such that the length of the perpendicular line from the reference point Pc to the surface 201 becomes a predetermined value (that is, the distance from the reference point Pc is constant). Calculated. Thus, in this modification, the position and orientation of the surface 201 change so as to rotate on the spherical surface centered on the point Pc (see FIG. 30). Therefore, the player can obtain a feeling of drawing a picture on the inner surface of the sphere by inputting to the touch panel 52 while changing the attitude of the terminal device 7.

As a specific process in step S51, the CPU 10 first calculates the attitude of the surface 201 by the same method as in step S22. Next, the CPU 10 calculates the position of the surface 201 so that the length of the perpendicular becomes a predetermined value based on the calculated posture and the position of the reference point Pc. For example, when the coordinates of the reference point Pc are (ES, ET, EU) and the distance from the reference point Pc to the surface 201 is k, the position Pd (DS, DT, DU) of the surface 201 is expressed by the following equation (2). Is calculated according to
DS = -k x CS + ES
DT = −k × CT + ET
DU = −k × CU + EU (2)
In the above equation (2), each component of the unit vector C (see FIG. 30) facing the positive direction of the u-axis in the variable plane coordinate system is (CS, CT, CU). Data representing the calculated position and orientation of the surface 201 is stored in the main memory. Following step S51, the process of step S52 is executed.

  In the present modification, the reference point Pc is set on the same side as the terminal virtual camera 205 with respect to the plane 201, but the position of the reference point may be anywhere. 31 and 32 are diagrams showing movement of the surface 201 in another embodiment. In another embodiment, as illustrated in FIG. 31, the reference point Pc may be set on the side opposite to the terminal virtual camera 205 with respect to the surface 201. According to this, the player can obtain a sensation of drawing a picture on the outer surface of the sphere by inputting to the touch panel 52 while changing the attitude of the terminal device 7. In the case shown in FIG. 31, since the direction from the reference point Pc to the position Pd of the surface 201 is the same as the direction of the vector C, “−k” in the above equation (2) is changed to “k”. By changing, the position Pd of the surface 201 can be calculated. When k = 0, the surface 201 rotates around the position Pd (reference point Pc) as in the second game example (see FIG. 27).

  In another embodiment, as shown in FIG. 32, the surface 201 rotates around the reference point Pc in a predetermined rotation direction (in FIG. 32, the rotation direction about the vertical direction). The surface 201 may be moved in parallel with respect to other directions. According to this, the player can obtain a feeling of drawing a picture on the inner side surface of the cylinder. Note that the parallel movement of the surface 201 in FIG. 32 may be performed according to a change in the attitude of the terminal device 7, or may be performed according to a direction input operation in step S52 described later.

  In step S <b> 52, the CPU 10 moves the surface 201 based on a direction input operation on the terminal device 7. The direction input operation may be performed by any method, but may be an input operation to the analog stick 53 or the cross button 54A, for example. Specifically, the surface 201 is controlled to move in the up / down / left / right / front / rear direction of the surface 201 in response to a direction input operation in the up / down / left / right / front / back direction. That is, the CPU 10 determines the direction instructed by the direction input operation based on the terminal operation data 101 acquired in step S2. Then, data representing the position and orientation of the surface 201 is read from the main memory, and the surface 201 is moved in a direction corresponding to the instructed direction. The data stored in the main memory is updated to represent the position of the surface 201 after movement. Following step S52, the process of step S53 is executed.

  According to the process of step S52, the surface 201 can be freely translated by a direction input operation on the terminal device 7. In other embodiments, in step S52, the CPU 10 may rotate the surface 201 (instead of translation). For example, as illustrated in FIGS. 30 to 32, the CPU 10 may control the surface 201 so that the surface 201 moves on a spherical surface or a side surface of a cylinder.

  In step S <b> 53, the CPU 10 controls the television virtual camera based on the position and orientation of the surface 201. The television virtual camera may be controlled in any way as long as the position and orientation are controlled so that the surface 201 is included in the visual field range. For example, the position of the virtual camera for television may be controlled such that the line-of-sight direction faces the surface 201 and is a predetermined distance from the surface 201. Further, the attitude of the television virtual camera does not change, that is, the television virtual camera may be controlled to move in parallel. According to this, as in the second game example, the player can confirm the posture of the surface 201 in the virtual space with the television image. As a specific process of step S53, the CPU 10 reads data representing the position and orientation of the surface 201 from the main memory, and calculates the position and orientation of the television virtual camera according to the position of the surface 201. Television camera data representing the calculated position and orientation is stored in the main memory. At this time, in the process of step S4, a television image is generated using the television camera data. Following step S53, the process of step S23 is executed. After the process of step S23, the same process as in the second game example is executed.

  In other embodiments, the television virtual camera may be controlled in accordance with a direction input operation on the terminal device 7. That is, the CPU 10 may translate, rotate, or rotate the television virtual camera in accordance with the direction input operation. According to this, the player can see the figure from various directions as in the case of rotating the figure by the above-described figure movement process, and can easily confirm the shape of the figure.

(Modification regarding game processing using specified position)
In the second game example, the drawing game that can draw a picture in the three-dimensional virtual space has been described as an example of the game using the designated position as an input. That is, in the second game example, the process of placing an object at the designated position (step S30) has been described as an example of the game process using the designated position as an input. Here, the game process executed with the designated position as an input may be any process, and for example, the following game examples are conceivable.

  FIG. 33 is a diagram showing a game image displayed on the television 2 in a game in which a figure is drawn so as to be the same as the sample. As shown in FIG. 33, a surface 201 and a sample object 221 are displayed on the television 2. The sample object 221 is an object representing a sample of a graphic drawn by the player, and a plurality of types of sample objects are prepared in the game program. The image displayed on the terminal device 7 is the same as that in the second game example. In this game, the player uses the terminal device 7 in the same manner as in the second game example to draw a line (object 222 representing the locus of the specified position) in the virtual space so as to be the same as the sample object 221. Draw. At this time, the CPU 10 may execute a game process for calculating the similarity between the sample object 221 and the object 222 generated in response to an input by the player, or calculating a score based on the similarity. Good. As described above, the game apparatus 3 may execute a game process in which the locus representing the input designated position is input.

  FIG. 34 is a diagram showing a game image displayed on the television 2 in a game in which an object (coin) 225 arranged in a three-dimensional virtual space is designated by an operation on the terminal device. As shown in FIG. 34, a surface 201 and a coin 225 are displayed on the television 2. The image displayed on the terminal device 7 is the same as that in the second game example. In this game, the player designates a position in the virtual space in the same manner as in the second game example. Then, the coin 225 can be acquired by designating the position of the coin 225. For example, the player plays the game by correctly specifying the coin 225 or competing for the number of coins 225 to be acquired within the time limit. In this case, the CPU 10 determines whether or not the designated position is the position of the coin 225 as a game process executed with the designated position as an input. The process of counting the number of sheets is executed. Thus, the game process is not limited to the process of displaying an image at the designated position as in the second game example, but may be a process of designating (selecting) an object in the virtual space.

  35 and 36 are diagrams showing game images displayed on the television 2 in a game in which an object is generated based on a graphic drawn by a player. In this game, the player first generates a figure in the virtual space by the same operation as in the second game example. In FIG. 35, an object 227 representing a figure input by the player is generated and displayed. Here, the CPU 10 generates a three-dimensional object based on each designated position forming the graphic as a game process executed with the designated position as an input. For example, the CPU 10 generates a three-dimensional object by setting a surface based on the locus of each designated position. For example, when a closed curve is formed by the trajectory, a surface having the closed curve as a circumference may be set. FIG. 36 shows a game image when a three-dimensional object 228 is generated based on the object 227 shown in FIG. As described above, the game system 1 may execute a game process for generating a three-dimensional object based on a designated position. At this time, the generated object may be used in the game. For example, the generated object may be used as an item by the player character, or the generated object may be used as the player character.

  In addition to the above, the game process executed with the designated position as an input may be any process. For example, in another embodiment, the CPU 10 may execute a game process for moving the player character according to the locus of the specified position. That is, the game executed in the game apparatus 3 may be a game in which the player character in the virtual space moves along the locus when the player inputs the locus of the specified position.

[8. Third game example]
Hereinafter, a third game example executed in the game system 1 of the present embodiment will be described with reference to FIGS. 37 to 42. In the third game example, a game process will be described in which a player operates a handheld device 9 to execute a game in which an object in a virtual game space is moved.

  In the third game example, the game system 1 will be described as having a handheld device 9 that is a portable game machine instead of the terminal device 7. Like the terminal device 7, the handheld device 9 is a device that can be held by the player, and includes a touch panel 52 similar to the terminal device 7. That is, the operating device used by the player is not limited to the terminal device 7, and is a portable device that can be held by the player, and any operating device that can input to the input surface may be used. .

  In the game example shown below, the controller 5 is not used for game operations, and the controller operation data from the controller 5 is not used for game processing. That is, the game system 1 may be configured without the controller 5. In other embodiments, the game process may use both the handheld device 9 and the controller 5 for game operations.

  First, an outline of the game in the third game example will be described. This game is a game in which an object (shuriken) is skipped in the game space by operating the handheld device 9. The player can change the posture of the handheld device 9 and can instruct the direction (and speed) of firing the shuriken by an operation of drawing a line on the touch panel 52. In this embodiment, the operation of drawing a line means an operation of touching the touch panel or touch pad with a finger or a touch pen and moving the input position on the surface by an operation of drawing a line while touching, Actual lines need not be drawn.

  FIG. 37 is a diagram showing the screen of the television 2 and the handheld device 9 in the third game example. In FIG. 37, an image of the game space is displayed as a game image on the lower LCD of the television 2 and the handheld device 9 (LCD provided with the touch panel 52). On the television 2, a shuriken 301, a control surface 302, and a target 303 are displayed. A control surface 302 (and shuriken 301) is displayed on the lower LCD. In the third example game, the player plays by hitting the shuriken 301 and hitting the target 303 by an operation using the handheld device 9.

  When flying the shuriken 301, the player first changes the posture of the control surface 302 by operating the posture of the handheld device 9 to a desired posture. Furthermore, the player performs an operation of drawing a line on the touch panel 52 as shown in FIG. By this operation, the direction on the control surface 302 is determined, and the shuriken 301 is fired in this direction. In the third game example, the player can fly the shuriken 301 in a desired direction by these operations. Although FIG. 37 shows a state in which the handheld device 9 is used in a vertical orientation, the handheld device 9 may be used in a horizontal orientation and can be used in an arbitrary posture.

  In the present embodiment, the attitude of the control surface 302 is determined according to the attitude of the handheld device 9. FIG. 38 is a diagram illustrating the relationship between the posture of the handheld device 9 and the posture of the control surface 302. As shown in the upper column of the table of FIG. 38, when the input surface of the touch panel 52 of the handheld device 9 is horizontal, the control surface 302 is horizontally disposed in the game space. Also, as shown in the middle column of FIG. 38, when the handheld device 9 is rotated about the front-rear direction (specifically, tilted so that the right side is lowered), the control surface 302 is about the front-rear direction. It is rotated (specifically, tilted so that the right side is lowered). Furthermore, as shown in the lower column of the table of FIG. 38, when the handheld device 9 is rotated about the left-right direction (specifically, the rear side is tilted so that the rear side is lowered), the control surface 302 is rotated left-right. (Specifically, the rear side is inclined so as to be lowered). Although not shown, when the handheld device 9 rotates about the vertical direction, the control surface 302 is also rotated and arranged about the vertical direction. Thus, the attitude of the control surface 302 is controlled to correspond to the attitude of the handheld device 9. In the present embodiment, the position of the control surface 302 is fixed at a predetermined position in the game space.

  The firing direction of the shuriken 301 is determined by determining the direction on the control surface 302 whose posture is determined as described above. That is, the direction in the three-dimensional game space is determined by determining the direction on the control surface 302 in which the posture in the three-dimensional game space is set. Here, the direction on the control surface 302 is calculated based on a line (input locus) input to the touch panel 52 of the handheld device 9.

  FIG. 39 is a diagram illustrating a state in which a line is drawn on the touch panel 52 of the handheld device 9. FIG. 40 is a diagram showing control positions and control directions calculated on the control surface 302 when the lines shown in FIG. 39 are drawn on the touch panel 52. As shown in FIG. 39, a control surface 302 is displayed on the lower LCD of the handheld device 9. Here, the control surface 302 is displayed so that the input surface of the touch panel 52 and the control surface 302 coincide on the screen (that is, the control surface 302 is displayed on the entire screen).

  When firing the shuriken 301, the player performs an operation of drawing a line on the touch panel 52 as shown in FIG. In the present embodiment, when an input is performed on the touch panel 52, the shuriken 301 is displayed at the input position. In FIG. 39, the line drawn by the player is represented by a dotted line, but this line may not actually be displayed.

  When a line is drawn by the player, that is, when there is an input to the touch panel 52, the game apparatus 3 determines the position on the control surface 302 corresponding to the input position (designated position. In the third game example, “control position”. Calculated). The control position is calculated as a position (on the control surface 302) corresponding to the input position on the lower LCD screen. In FIG. 40, a dotted line indicates a line (on the control surface 302) corresponding to a line drawn on the input surface. The control position is calculated at a position on this line, for example, points P11 to P14 shown in FIG. Note that the game apparatus 3 repeatedly executes the calculation of the control position while an input to the touch panel 52 is performed (while a line is drawn).

  When the input of drawing a line to the touch panel 52 is completed, the game apparatus 3 specifies the control direction (vector v1 shown in FIG. 40) based on the control position. The control direction is specified to be a direction represented by each control position corresponding to the drawn line. That is, the control direction is calculated so as to face the direction of the line. Although details will be described later, in the present embodiment, the control direction is calculated as a vector representing the average speed of the control position. Since the control surface 302 is arranged in the three-dimensional game space, the control direction is also calculated as a three-dimensional direction in the three-dimensional game space. As a result, the firing direction of the shuriken 301 in the game space is determined.

  When the control direction is calculated, the shuriken 301 is fired. A state in which the shuriken 301 moves in the game space is displayed on the television 2. In the present embodiment, the posture of the shuriken 301 is determined to be a posture corresponding to the control surface 302. Therefore, the shuriken 301 is fired (moved) in the control direction in a posture corresponding to the posture of the control surface 302. Note that the behavior of the shuriken 301 after being fired may be determined in any way. For example, the shuriken 301 may be controlled so as to move straight in the control direction, or may fall with time by virtually applying the influence of gravity. Further, the trajectory may be bent by rotation. The movement direction may be controlled according to the posture of the shuriken 301 (for example, turn right when the shuriken 301 is tilted to the right and turn left when the shuriken 301 is tilted to the left). .

  As described above, in the present embodiment, the player can instruct the orientation of the control surface 302 by the orientation of the handheld device 9 and can instruct the direction on the control surface 302 by a line input to the touch panel 52. . The shuriken 301 is controlled to move in the direction (control direction) on the instructed control surface 302. Therefore, according to the present embodiment, the player actually adjusts the posture of the handheld device 9 with one hand and inputs the direction with a line to the touch panel 52 with the other hand. The direction can be easily indicated by an intuitive operation such as inputting. Particularly in this embodiment, the operation can be performed as if the shuriken is actually released toward the screen with the hand. In addition, since the posture operation of the handheld device 9 and the input operation on the touch panel 52 can be simultaneously performed in parallel, the player can quickly perform an operation of instructing a direction in the three-dimensional space.

(Game processing in the third game example)
Next, details of the game process in the third game example will be described. First, various data used in the game process will be described. FIG. 41 is a diagram showing various data used in the game process. 41 shows main data stored in the main memory (external main memory 12 or internal main memory 11e) of the game apparatus 3 in FIG. As shown in FIG. 41, the main memory of the game apparatus 3 stores a game program 310, terminal operation data 311 and processing data 316. In addition to the data shown in FIG. 41, the main memory stores data necessary for the game such as image data of various objects appearing in the game and sound data used for the game.

  The game program 310 is for causing the computer (CPU 10) of the game apparatus 3 to execute game processing in the third game example. A part or all of the game program 310 is read from the optical disc 4 and stored in the main memory at an appropriate timing after the game apparatus 3 is turned on. Note that the game program 310 may be obtained from an external device of the game apparatus 3 (for example, via the Internet) instead of the optical disc 4. Further, a part (for example, a program for calculating the attitude of the controller 5 and / or the handheld device 9) included in the game program 310 may be stored in the game apparatus 3 in advance.

  The terminal operation data 311 is data representing a user operation on the handheld device 9. The terminal operation data 311 is transmitted from the handheld device 9 and acquired by the game apparatus 3. The terminal operation data 311 includes acceleration data 312, angular velocity data 313, input position data 314, and operation button data 315. The main memory may store a predetermined number of terminal operation data in order from the latest (last acquired).

  The acceleration data 312 is data representing the acceleration (acceleration vector) detected by the acceleration sensor included in the handheld device 9. Here, the acceleration data 312 represents a three-dimensional acceleration having accelerations in predetermined three-axis directions as components, but in other embodiments, represents acceleration in any one or more directions. If it is.

  The angular velocity data 313 is data representing the angular velocity detected by the gyro sensor included in the handheld device 9. Here, the angular velocity data 313 represents the respective angular velocities around the predetermined three axes, but in other embodiments, the angular velocity data 313 only needs to represent the angular velocities around any one or more axes.

  The input position data 314 is data representing a position (input position) where an input is performed on the input surface of the touch panel 52. Here, the input position data 314 represents coordinate values of a two-dimensional coordinate system for indicating the input position. When no input is made on the touch panel 52, the input position data 314 indicates that there is no input.

  The operation button data 315 is data representing an input state with respect to each operation button provided in the handheld device 9.

  The terminal operation data 311 includes data representing the input position of the player with respect to the input surface (in this embodiment, input position data 314) and data whose value changes according to the movement of the handheld device 9 (in this embodiment). Any data including acceleration data 312 and angular velocity data 313) may be used. Therefore, the terminal operation data 311 may not include the operation button data 315, or may include only one of the acceleration data 312 and the angular velocity data 313. That is, the handheld device 9 may be configured not to include an operation button, or may be configured to include only one of an acceleration sensor and a gyro sensor. Further, the handheld device 9 includes the imaging information calculation unit 35 instead of the acceleration sensor and the gyro sensor, thereby including the marker coordinate data in the terminal operation data 311 instead of the acceleration data 312 and the angular velocity data 313. Also good.

  The processing data 316 is data used in a game process (FIG. 42) described later. The processing data 316 includes terminal attitude data 317, television camera data 318, terminal camera data 319, control surface data 320, control position data 321, control direction data 322, and object data 323. In addition to the data shown in FIG. 41, the processing data 316 includes various data used in game processing, such as data representing parameters set for various objects.

  The terminal attitude data 317 is data representing the attitude of the handheld device 9. In the present embodiment, the terminal attitude data 317 is calculated based on the acceleration data 312 and the angular velocity data 313 included in the terminal operation data 311. A method for calculating the terminal attitude data 317 will be described later.

  The TV camera data 318 is data representing the arrangement of a TV virtual camera for generating a TV image displayed on the TV 2. Specifically, the TV camera data 318 represents the position and posture of the TV virtual camera in the virtual game space. Note that the television camera data 318 may include data representing the angle of view (view range) of the television virtual camera.

  The terminal camera data 319 is data representing the arrangement of the terminal virtual camera for generating the terminal image displayed on the lower LCD of the handheld device 9. Specifically, the terminal camera data 319 represents the position and orientation of the terminal virtual camera in the game space. Note that the terminal camera data 319 may include data representing the angle of view (field of view range) of the terminal virtual camera. In the present embodiment, the terminal virtual camera is arranged according to the position and orientation of the control surface 302 so that the control surface 302 is displayed on the lower LCD.

  The control surface data 320 is data representing the position and orientation of the control surface 302 in the game space. Specifically, the control surface data 320 represents a plane equation representing the control surface 302. In the present embodiment, the position of the control surface 302 is fixed at a predetermined position, and the posture of the control surface 302 is calculated based on the posture of the handheld device 9.

  The control position data 321 is data representing a designated position (control position) on the control surface 302 corresponding to the input position. In the third game example, the designated position is used to calculate a control direction, which will be described later, and therefore the designated position is referred to as a “control position”. The control position data 321 represents a three-dimensional coordinate value indicating a position in the game space. In another embodiment, the control position data 321 may represent a coordinate value of a two-dimensional coordinate system that represents a position on the control surface. In the present embodiment, when input is continuously performed on the touch panel 52 (that is, when input positions are continuously detected), a plurality of input positions corresponding to a predetermined number of input positions in order from the latest one. Data representing the control positions is stored as control position data 321. Further, when the input to the touch panel 52 is completed, the contents of the control position data 321 are reset.

  The control direction data 322 is data representing a control direction for determining the firing direction (movement direction) of the shuriken 301. Specifically, the control direction data 322 represents a three-dimensional vector indicating a direction in the three-dimensional game space. Hereinafter, a vector indicating the control direction is referred to as a “control direction vector”. The control direction data 322 is calculated based on the control position data 321.

  The object data 323 is data representing the state of an object (shuriken 301 in the present embodiment) that is moved based on the control direction. Specifically, the object data 323 represents the position and posture of the shuriken 301 in the game space. The game apparatus 3 arranges the shuriken 301 in the game space based on the object data 323, and generates and displays an image of the game space in which the shuriken 301 is arranged.

  Next, details of the game process performed in the game apparatus 3 will be described with reference to FIG. In the third game example, a main flowchart showing a schematic process flow in the game process is the same as that in FIG. Hereinafter, the game process in the third game example will be described with a focus on differences from the first game example.

  Also in the third game example, as in the first game example, first, in step S1, the CPU 10 executes an initial process. The initial process is a process of constructing a virtual space, placing each object appearing in the virtual space at an initial position, and setting initial values of various parameters used in the game process. In this game example, the position and orientation of the television virtual camera are set in the initial process. The position and orientation of the television virtual camera are set so that the control surface 302 is included in the imaging range (the range of the virtual camera viewing volume). Data representing the set position and orientation is stored in the main memory as television camera data 318. Note that since the attitude of the control surface 302 is variable but the position is fixed, it is possible to set the television virtual camera so that the control surface 302 is included in the shooting range in the initial processing. Following step S1, the process of step S2 is executed.

  In step S <b> 2, the CPU 10 acquires terminal operation data from the handheld device 9. Since the handheld device 9 repeatedly transmits each data output from the acceleration sensor, the gyro sensor, the touch panel 52, and the operation buttons to the game apparatus 3 as terminal operation data, the game apparatus 3 sequentially receives the data from the handheld device 9. Received and sequentially stored in the main memory as terminal operation data 311. In step S2, the CPU 10 may acquire controller operation data from the controller 5 in addition to the terminal operation data. In step S2, the CPU 10 reads the latest terminal operation data 311 from the main memory. Following step S2, the process of step S3 is executed.

  In step S3, the CPU 10 executes a game control process. The game control process is a process for advancing the game by a process such as operating an object (shuriken 301 or the like) in the game space in accordance with an operation by the player. Hereinafter, the game control process will be described in detail with reference to FIG. In the following, the game control process will be described focusing on the process related to the calculation of the control direction, and detailed description of the process that can be performed in the same manner as the conventional process will be omitted. For example, processing related to movement control of the shuriken 301 after being fired according to the control direction, processing for determining whether the shuriken 301 hits the target 303, score calculation processing when the shuriken 301 hits the target 303, etc. Since it can be performed in the same manner as conventional processing, detailed description thereof is omitted.

  FIG. 42 is a flowchart showing a detailed flow of the game control process. In the game control process, first, in step S60, the CPU 10 calculates the attitude of the handheld device 9. The attitude of the handheld device 9 may be calculated by any method as long as it is calculated based on the terminal operation data 311. In this game example, the attitude is calculated by the following method. Hereinafter, a method for calculating the attitude of the handheld device 9 will be described.

  First, the CPU 10 calculates the attitude of the handheld device 9 based on the angular velocity data 313 stored in the main memory. Any method may be used for calculating the posture of the handheld device 9 from the angular velocity, but the posture can be calculated by a method similar to that of the terminal device 7 in the first game example. Following step S60, the process of step S61 is executed.

  In step S <b> 61, the CPU 10 calculates the attitude of the control surface 302 based on the attitude of the handheld device 9. In this game example, the posture of the control surface 302 in the game space is set so as to correspond to the posture of the handheld device 9 in the real space. Specifically, when the handheld device 9 is in the reference posture, the posture of the control surface 302 is calculated to be a predetermined reference posture. In the present game example, a posture in which the input surface of the touch panel 52 is horizontal is a reference posture of the handheld device 9, and a posture that is horizontally arranged in the game space is a predetermined reference posture of the control surface 302 (FIG. 38). (See above). When the handheld device 9 is other than the reference posture, the posture of the control surface 302 is set to the predetermined reference by the rotation amount corresponding to the rotation amount in the direction corresponding to the direction in which the handheld device 9 is rotated from the reference posture. It is calculated so as to be a posture rotated from the posture (see the middle and lower columns in FIG. 38). As described above, in this game example, the posture of the control surface 302 is calculated so that the posture of the control surface 302 in the virtual space matches the posture of the handheld device 9 in the real space. However, in other embodiments, the attitude of the handheld device 9 and the attitude of the control surface 302 do not have to coincide with each other. Good.

  As a specific process of step S <b> 61, the CPU 10 reads the terminal attitude data 317 from the main memory, and calculates the attitude of the control surface 302 based on the terminal attitude data 317. Further, the position of the control surface 302 is set to a predetermined position. Data representing the calculated posture and the set position is stored in the main memory as control surface data 320. Following step S61, the process of step S62 is executed.

  In step S <b> 61, the CPU 10 fixes the position of the control surface 302 and changes the posture of the control surface 302 according to the posture of the handheld device 9. Here, in another embodiment, the position may be changed according to the posture of the handheld device 9 together with the posture of the control surface 302. For example, when rotating the control surface 302 to the left, the CPU 10 may move the position to the left while rotating the direction of the control surface 302 to the left.

  In step S <b> 62, the CPU 10 controls the terminal virtual camera according to the attitude of the control surface 302. Here, as described above, the control surface 302 is displayed on the lower LCD of the handheld device 9, and specifically, the control surface 302 so that the input surface of the touch panel 52 and the control surface 302 coincide on the screen. Is displayed. Accordingly, the position and orientation of the terminal virtual camera are set so that the control surface 302 is included in the visual field range (specifically, the outer periphery of the control surface 302 matches the outer edge of the visual field range of the terminal virtual camera). ) And controlled according to the attitude of the control surface 302. Further, the terminal virtual camera is controlled so that the line-of-sight direction and the control surface 302 are orthogonal.

  In addition, according to the process of the said step S62, the terminal virtual camera is moved according to the change of the control surface 302 so that the positional relationship between the control surface 302 and the terminal virtual camera becomes constant. According to this, since the correspondence relationship between the input position and the control position does not change regardless of the posture of the control surface 302, the operability of the touch operation on the touch panel 52 can be improved.

  As a specific process of step S62, the CPU 10 reads the control surface data 320 from the main memory, and calculates the position and orientation of the terminal virtual camera based on the orientation of the control surface 302. Then, data representing the calculated position and orientation is stored in the main memory as terminal camera data 319. Following step S62, the process of step S63 is executed.

  In step S <b> 63, the CPU 10 determines whether or not there is an input (touch input) to the touch panel 52. Specifically, the latest input position data 314 stored in the main memory is read out and referred to, and it is determined whether the input position data 314 represents the coordinate value of the input position or no input. . If the determination result of step S63 is affirmative, the process of step S64 is executed. On the other hand, when the determination result of step S63 is negative, a process of step S67 described later is executed.

  In step S64, the CPU 10 calculates a control position on the control surface 302 based on the input position. The control position is calculated as a position on the control surface 302 corresponding to the input position. Specifically, the control position is calculated so that the positional relationship of the input position with respect to the four sides of the input surface of the touch panel 52 is the same as the positional relationship of the control position with respect to the four sides of the control surface 302.

  As a specific process of step S64, the CPU 10 first reads the latest input position data 314 stored in the main memory, and the input position represented by the input position data 314 indicates 2 on the control surface 302. Convert to coordinates in dimensional coordinate system. Next, the CPU 10 converts the two-dimensional coordinates obtained by the conversion into three-dimensional coordinates indicating the position of the three-dimensional game space. The three-dimensional coordinates can be calculated using the coordinates of the two-dimensional coordinate system and a plane expression (control surface data 320) representing the control surface 302. That is, the CPU 10 reads the control surface data 320 from the main memory, and calculates the three-dimensional coordinates based on the two-dimensional coordinates and the control surface data 320. The three-dimensional coordinates calculated in this way represent the control position. Following step S64, the process of step S65 is executed.

  In step S64, the control position is preferably calculated so that the input position becomes the control position on the screen of the lower LCD. According to this, it becomes easier for the player to grasp the control position in the virtual game space, and it becomes easier to instruct the control direction. In this game example, the shape (aspect ratio) of the input surface of the touch panel 52 is made equal to the shape (aspect ratio) of the control surface 302, and the input surface of the touch panel 52 and the control surface 302 are connected to the lower LCD. On the screen. Thus, by calculating the control position as described above, it is possible to make the input position correspond to the control position on the screen of the lower LCD. In other embodiments, the method for calculating the control position from the input position may be any method, and may be a method in which the input position and the control position do not correspond on the screen.

  In step S65, the CPU 10 stores data representing the control position calculated in step S64 in the main memory. Here, when continuous input is performed on the touch panel 52, the CPU 10 stores a plurality of control positions corresponding to a predetermined number of input positions in order from the latest one. Therefore, the CPU 10 reads the control position data 321 from the main memory, and when the control position indicated by the control position data 321 is smaller than the predetermined number, the CPU 10 sets a new control position at the control position indicated by the control position data 321 (step S64). (Control position calculated in step 1) Create the added data. On the other hand, when the control position represented by the read control position data 321 is equal to or more than the predetermined number, the oldest control position is deleted from each control position represented by the control position data 321 and a new control position is set (step (Control position calculated in S64) The added data is created. The data created as described above is stored in the main memory as new control position data 321. Therefore, the control position data 321 represents a control position corresponding to the input position input within a predetermined time from the present time. Following step S65, the process of step S66 is executed.

  In step S66, the CPU 10 calculates the position and posture of the shuriken 301 in the game space. Here, the shuriken 301 is arranged at the control position in a posture corresponding to the control surface 302 in a period from when input to the touch panel 52 is performed until the touch panel 52 is fired. That is, the CPU 10 reads the control surface data 320 and the control position data 321 from the main memory, and calculates the position and posture of the shuriken 301. Data representing the calculated position and orientation is stored as object data 323 in the main memory. In the third game example, the shuriken 301 is arranged and displayed on the control surface 302 during the period when the input to the touch panel 52 is performed. However, in other embodiments, the shuriken is performed during the period. 301 may not be arranged / displayed. In another embodiment, the CPU 10 may control only the position of the shuriken 301 according to the control position, and may not control the posture of the shuriken 301 (the posture is fixed). After step S66, the CPU 10 ends the game control process.

  On the other hand, if the determination result of step S66 is negative, the process of step S67 is executed. In step S67, the CPU 10 determines whether or not there is a touch input in the previous processing loop (a processing loop including a series of processes in steps S2 to S8). The determination in step S67 can be made, for example, based on whether or not the control position data 321 is stored in the main memory. Note that not only the previous touch input, but also whether or not there has been a touch input in the past a predetermined number of times may be determined. If the determination result of step S67 is affirmative, the process of step S68 is executed. On the other hand, if the determination result of step S67 is negative, the processes of steps S68 to S70 are skipped, and the CPU 10 ends the game control process.

  In step S68, the CPU 10 calculates a control direction. In the present game example, the control direction (control direction vector) is calculated as a vector indicating the average speed of the control position. That is, the CPU 10 reads the control position data 321 from the main memory, and calculates a vector indicating the average speed from a plurality of control positions represented by the control position data 321. Specifically, the vector indicating the average speed can be calculated by dividing the vector having the oldest start point and the latest end point among the plurality of control positions by the number of control positions. . Data representing the calculated vector is stored as control direction data 322 in the main memory. Following step S68, the process of step S69 is executed.

  In step S68, a method of calculating the control direction as the average speed of the plurality of control positions represented by the control position data 321 is adopted. In this method, if the player stops without moving the input position in the middle of drawing a line on the touch panel, the calculated average speed may be very small. Therefore, in another embodiment, the CPU 10 may calculate the average speed by removing the control position corresponding to the input position that is estimated not to move. Specifically, in step S65, if the new control position calculated in step S64 has not moved (almost) from the control position calculated immediately before, the new control position is determined. The position may not be stored. That is, when the distance between the new control position and the previously calculated control position is within a predetermined value, the CPU 10 may not update the control position data 321.

  In step S69, the CPU 10 fires the shuriken 301 in the control direction. In this example game, the moving direction and moving speed of the shuriken 301 are determined based on the control direction vector. That is, the shuriken 301 is moved in the direction of the control direction vector by the amount of movement corresponding to the control direction vector. Specifically, in step S69, the CPU 10 reads the object data 323 and the control direction data 322, and moves the position represented by the object data 323 to the direction of the control direction vector by the amount of movement corresponding to the control direction vector. The calculated position is calculated. Then, the position represented by the object data 323 is updated to the moved position, and the updated object data is stored in the main memory. As a result, the position of the shuriken 301 is updated, and the shuriken 301 is moved in the control direction. In this game example, the shuriken 301 is arranged in a posture corresponding to the control surface 302 as described in step S66. Accordingly, the shuriken 301 is fired in a direction determined by the control surface in a direction determined by the control direction. When the magnitude of the control direction vector is smaller than a predetermined value, it is considered that the player does not intend to perform an operation for drawing a line on the touch panel 52 (that is, an operation for firing the shuriken 301). . For this reason, the CPU 10 may not fire the shuriken 301 in this case. Following step S69, the process of step S70 is executed.

  In step S70, the CPU 10 resets the control position. Specifically, the control position data 321 stored in the main memory is deleted. As a result, when touch input is performed next time, the control position is newly stored from the beginning in step S65. After step S70, the CPU 10 ends the game control process.

  According to the game control process shown in FIG. 42, the attitude of the control surface 302 is controlled according to the attitude of the handheld device 9 (steps S61 and S62), and the line input to the touch panel 52 on the control surface 302 is displayed. A control direction corresponding to the direction is calculated (step S68). As a result, the player can easily and intuitively instruct the direction in the three-dimensional space by operating the posture of the handheld device 9 and operating the touch panel 52.

  In the game control process, the control direction is calculated as the average speed of a plurality of control positions. Here, the method of calculating the control direction from the control position is not limited to the above, and any method may be used. For example, in the case of causing the player to input a line as in the present game example, the CPU 10 sets the latest control position among the plurality of control positions represented by the control position data 321 as the end point, and then performs the new control. A vector starting from the position may be calculated as the control direction vector. Further, the player may be able to instruct the control direction by designating two points on the touch panel 52. That is, the CPU 10 may calculate, as a control direction vector, a vector having a control position corresponding to the input position touched first as a start point and a control position corresponding to the input position touched next as an end point. .

  Returning to the description of the main flowchart (steps S1 to S8), the process of step S4 is executed after the game control process of step S3. In step S4, a television image is generated. Also in the third game example, the specific method for generating an image for television is the same as in the first game example. In the third game example, an image of the game space including the control surface 302 is generated as a television image (see FIG. 37). Note that, as in the third game example, by preventing the position and posture of the television virtual camera from changing even if the posture of the control surface 302 changes, the player can change the posture of the control surface 302 in the game space. It can be easily grasped. Following step S4, the process of step S5 is executed.

  In step S5, a terminal image is generated based on the game control process. Also in the third game example, the specific method for generating the terminal image is the same as in the first game example. In the present game example, an image of the game space viewed from the terminal virtual camera set in step S62 is generated as a terminal image. As a result, an image of the control surface 302 is generated as a terminal image (see FIG. 37). Following step S5, the process of step S6 is executed.

  In the third game example, the processes in steps S6 to S8 are the same as in the first game example. In the handheld device 9, the process of receiving the image data and the sound transmitted from the game apparatus 3 and outputting them to the display unit and the lower LCD is the same as the process in the terminal device 7. Also in the third game example, similar to the first game example, the series of processes in steps S2 to S8 are repeatedly executed until it is determined in step S8 that the game is to be ended.

  According to the game process of the third game example described above, the posture of the control surface 302 is determined by the posture of the handheld device 9, and the control direction is determined on the control surface 302 by an input to the touch panel. Then, the shuriken 301 is fired in the determined control direction. Thus, according to the present game example, the player can easily instruct the direction in the three-dimensional space by an intuitive operation using the touch panel 52.

  Further, according to the game process of the third game example, since the control direction vector on the control surface 302 is calculated, the movement amount can be controlled in addition to the moving direction of the object by the control direction vector. . Therefore, the object can be controlled in more detail using the handheld device 9.

  Further, according to the game process of the third game example, an image including the control surface 302 is displayed on the television 2 as a television image, so that the player confirms the posture and launch direction of the shuriken 301 with the television image. Can be performed more easily.

[9. Fourth game example]
Hereinafter, with reference to FIGS. 43 to 46, a fourth game example executed in the game system 1 of the present embodiment will be described. The fourth game example is a game in which an object (cannon bullet) is fired in the game space by operating the handheld device 9. The player can instruct the direction of firing the bullet by changing the posture of the handheld device 9 and designating (touching) the position on the touch panel 52. In the fourth game example as well, as in the third game example, the game system 1 will be described as having a handheld device 9 that is a portable game machine instead of the terminal device 7.

  FIG. 43 is a diagram showing the screen of the television 2 and the handheld device 9 in the fourth game example. In FIG. 43, an image of the game space is displayed as a game image on the lower LCD of the television 2 and the handheld device 9. On the television 2, a cannon 331, a bullet 332, and a target 333 are displayed. A bullet 332 and a target 333 are displayed on the lower LCD. In FIG. 43, the terminal image displayed on the lower LCD is an image of the game space viewed from the position of the cannon 331. In the fourth game example, a control surface is set in the game space as in the third game example, but unlike the third game example, the control surface is not displayed. Therefore, in this embodiment, the player can play the game from a subjective viewpoint while looking at the lower LCD of the handheld device 9. The image displayed on the television 2 may be an image from the same viewpoint as the lower LCD, but if an image from another viewpoint is displayed, a game that makes use of both screens can be provided. . For example, if a range that cannot be seen on the lower LCD is displayed on the television 2, it is possible to realize a game play in which the player hides and aims at a portion that cannot be seen from the subjective viewpoint while watching the television 2. Although the operation feeling is different, it is also possible to display an image of a subjective viewpoint on the television 2 and perform input using a touch pad without providing a screen on the handheld device.

  When firing the bullet 332 from the cannon 331, the player first changes the attitude of the control surface to a desired attitude by manipulating the attitude of the handheld device 9. That is, also in the fourth game example, as in the third game example, the attitude of the control surface is determined according to the attitude of the handheld device 9. Although details will be described later, in the fourth game example, the position of the control surface changes (moves) according to the attitude of the control surface.

  In the fourth game example, the player further performs an operation of touching a desired position on the touch panel 52 as shown in FIG. By this operation, the position in the game space corresponding to the input position (control position on the control surface) is determined. Also in the fourth game example, the control position determination method is the same as in the third game example.

  Here, in the fourth game example, the control direction is determined as the direction from the position of the terminal virtual camera to the control position. FIG. 44 is a diagram showing a virtual camera and a control surface in the game space. In FIG. 44, a point P represents a control position set on the control surface 335, and a vector v2 represents a control direction. In the fourth game example, as shown in FIG. 44, the control direction (control direction vector v2) is calculated based on the position Pc and the control position P of the terminal virtual camera. Specifically, the control direction vector v2 is calculated as a vector having the position Pc of the terminal virtual camera as the start point and the control position P as the end point. The bullet 332 is fired in the calculated control direction.

  As described above, in the fourth game example, the player can fly the bullet 332 in a desired direction by the operation of the posture of the handheld device 9 and the operation of designating the position on the touch panel 32. Accordingly, in the fourth game example as well, as in the third game example, the player can easily indicate the direction in the three-dimensional space by an intuitive operation using the touch panel 52. In addition, since the posture operation of the handheld device 9 and the input operation on the touch panel 52 can be simultaneously performed in parallel, the player can quickly perform an operation of instructing a direction in the three-dimensional space.

(Game processing in the fourth game example)
Next, details of the game process in the fourth game example will be described. First, with reference to FIG. 41, various data used in the game process in the fourth game example will be described focusing on differences from the third game example.

  In the fourth game example, a game program for executing the game process shown in FIG. 45 is stored as the game program 310. The terminal operation data 311 is the same as that in the third game example.

  As the processing data 316, the same data as in the third game example is stored in the fourth game example. However, in the fourth game example, the control position data 321 may represent the latest one control position. The object data 323 represents the position of the bullet 332.

  Next, details of the game process in the fourth game example will be described with reference to FIGS. 45 and 46. In the fourth game example, similarly to the third game example, steps S1 to S8 shown in FIG. 18 are executed. Hereinafter, the game process in the fourth game example will be described focusing on the differences from the game process in the third game example.

  In step S1, the CPU 10 executes an initial process similar to that in the third game example. However, in the fourth game example, the television virtual camera is controlled according to the position and orientation of the control surface (step S84 to be described later), and therefore the position and orientation of the television virtual camera are not set in the initial process. . The process of step S2 in the fourth game example is the same as that in the third game example.

  In step S3, the CPU 10 executes a game control process. The game control process in the fourth game example is a process for advancing the game by a process such as operating an object (such as a bullet 332) in the game space according to an operation by the player. Hereinafter, the game control process in the fourth game example will be described in detail with reference to FIG. In the following, the game control process will be described focusing on the process related to the calculation of the control direction, and detailed description of the process that can be performed in the same manner as the conventional process will be omitted. For example, processing related to movement control of the bullet 332 after being fired according to the control direction, processing for determining whether the bullet 332 hits the target 333, score calculation processing when the bullet 332 hits the target 333, etc. Since it can be performed in the same manner as conventional processing, detailed description thereof is omitted.

  FIG. 45 is a flowchart showing a detailed flow of the game control process in the fourth game example. In the game control process, first, in step S <b> 81, the CPU 10 calculates the attitude of the handheld device 9. The process of step S81 is the same as the process of step S60 in the third game example. Following step S81, the process of step S82 is executed.

  In step S <b> 82, the CPU 10 calculates the position and orientation of the control surface 335 based on the orientation of the handheld device 9. Here, the process of calculating the attitude of the control surface 335 based on the attitude of the handheld device 9 is the same as the process of step S61 in the third game example. That is, also in the fourth game example, the posture of the control surface 335 in the game space is set so as to correspond to the posture of the handheld device 9 in the real space.

  Further, the position of the control surface 335 is calculated as follows according to the attitude of the control surface 335. FIG. 46 is a diagram showing how the position and orientation of the control surface 335 change. In FIG. 46, a point Pa is a reference point for determining the position of the control surface 335, and is fixedly set in the game space. As shown in FIG. 46, when the posture of the handheld device 9 changes, the position of the control surface 335 also changes so as to rotate around the reference point Pa as the posture changes. Specifically, the position of the control surface 335 is such that the length of the perpendicular from the reference point Pa to the control surface 335 is a predetermined value (that is, the distance from the reference point Pa is constant). To be calculated.

  As a specific process in step S82, the CPU 10 first calculates the attitude of the control surface 335 by the same method as in step S61. Next, based on the calculated posture and the position of the reference point Pa, the CPU 10 calculates the position of the control surface 335 so that the length of the perpendicular becomes a predetermined value. Data representing the calculated position and orientation of the control surface 335 is stored as control surface data 320 in the main memory. Following step S82, the process of step S83 is executed.

  According to the process of step S82, the control surface 335 is controlled so that the position changes with the attitude of the control surface 335. By changing the position of the control surface 335 according to the attitude of the handheld device 9 around the position of the terminal virtual camera (or the position of the reference point), the player feels as if looking around from a predetermined position. Game operation can be performed. In another embodiment, for example, it is possible to realize a game with a sense of operation such that a specific position in the virtual space is viewed from various angles. That is, the position of the terminal virtual camera may be changed according to the attitude of the handheld device 9 without changing the position of the control surface 335. In addition, the position of the control surface 335 and the position of the terminal virtual camera are moved according to the posture, and the game operation is performed on the player with an operational feeling of moving in the virtual space and changing the line of sight while changing the viewpoint. You may make it let.

  In step S <b> 83, the CPU 10 controls the terminal virtual camera according to the position and orientation of the control surface 335. Here, a point Pc shown in FIG. 46 is a point indicating the position of the terminal virtual camera. As shown in FIG. 46, the terminal virtual camera is such that the control surface 302 is included in the visual field range (more specifically, the outer periphery of the control surface 335 matches the outer edge of the visual field range of the terminal virtual camera. Controlled). Therefore, the position of the terminal virtual camera is calculated on a straight line passing through the reference point Pa and the center of the control surface 335 so that the distance from the control surface 335 is a distance corresponding to the visual field range. Note that the terminal virtual camera only needs to be on the straight line, and depending on the field of view range (viewing angle), the terminal virtual camera may be located behind the reference point Pa (side far from the control surface 335).

  By setting the terminal virtual camera as described above, the control surface 335 is displayed on the lower LCD so that the input surface of the touch panel 52 and the control surface 335 coincide on the screen ( (See FIG. 43). According to this, since the bullet 332 is fired toward the position in the game space corresponding to the position touched by the player, an intuitive and easy-to-understand game operation can be provided. Also in the fourth game example, as in the third game example, the terminal virtual camera responds to changes in the control surface 302 so that the positional relationship between the control surface 302 and the terminal virtual camera is constant. Will be moved.

  As a specific process of step S83, the CPU 10 reads the control surface data 320 from the main memory, and calculates the position and orientation of the terminal virtual camera based on the position and orientation of the control surface 335. Then, data representing the calculated position and orientation is stored in the main memory as terminal camera data 319. Following step S83, the process of step S84 is executed.

  In step S <b> 84, the CPU 10 controls the television virtual camera according to the position and orientation of the control surface 335. In the fourth game example, the television virtual camera is controlled such that the control surface 335 is included in the visual field range and is at a position different from the terminal virtual camera (see FIG. 43). Specifically, the television virtual camera is arranged so as to face the control surface 335 from a position slightly behind the cannon 331 so that the cannon 331 is displayed. Here, it is assumed that the cannon 331 is arranged at the position of the terminal virtual camera. Specifically, the CPU 10 reads the control surface data 320 from the main memory, and calculates the position and orientation of the television virtual camera so that the position and orientation correspond to the position and orientation of the control surface 335. Then, data representing the calculated position and orientation is stored in the main memory as television camera data 318. Following step S84, the process of step S85 is executed.

  According to the processing in step S84, the television virtual camera is arranged at a different position from the terminal virtual camera. Thus, the player can see the game space viewed from two different viewpoints, so that it is easier to perform the game operation. For example, in the fourth game example, the television virtual camera is arranged so as to capture a wider game space range than the terminal virtual camera (see FIG. 43), so that the player views the television image. This makes it easier to grasp the state (arrangement of each object, etc.) in the game space.

  In addition, the interest of the game can be improved by making the positions of the two virtual cameras different. For example, a wall is disposed between the cannon 331 and the target 333 (the target 333 cannot be seen from the terminal virtual camera), and the television virtual camera is disposed at a position where the target 333 can be seen above the wall. Then, the target 333 can be displayed only on the television 2 side. In this case, the player performs the game operation while comparing both the screen of the handheld device 9 and the screen of the television 2, and can make the game more interesting. In other embodiments, the position and orientation of the television virtual camera may be controlled in any way, may be arranged at a position where the control surface 335 cannot be seen, and is the same as the terminal virtual camera. The position and orientation may be set (that is, only one virtual camera may be set).

  In step S85, the CPU 10 determines whether or not there is an input (touch input) to the touch panel 52. The process of step S85 is the same as the process of step S63 in the third game example. If the determination result of step S85 is affirmative, the process of step S86 is executed. On the other hand, when the determination result of step S85 is negative, the CPU 10 ends the game control process.

  In step S86, the CPU 10 calculates a control position on the control surface 335 based on the input position. The calculation method of the control position in step S86 is the same as the calculation method in step S64 in the third game example. In the fourth game example, the control direction is calculated based on the latest control position. Therefore, in step S86, the CPU 10 stores data representing the calculated control position in the main memory as control position data 321. Following step S86, the process of step S87 is executed.

  In step S87, the CPU 10 calculates a control direction based on the control position calculated in step S86. The control direction is calculated as a direction connecting the predetermined position and the control position in the game space. In the fourth game example, the control direction is calculated as the direction from the position of the terminal virtual camera (cannon 331) to the control position (see FIG. 44). Specifically, the CPU 10 reads the terminal camera data 319 and the control position data 321 from the main memory, and calculates a control direction vector v2 having the position of the terminal virtual camera as the start point and the control position as the end point. Data representing the calculated control direction vector v2 is stored in the main memory as control direction data 322. Following step S87, the process of step S88 is executed.

  In step S87, the position of the terminal virtual camera is used as the predetermined position. Here, in another embodiment, the predetermined position may be another position. For example, the predetermined position may be the position of the reference point Pa shown in FIG. 46 or the position of a specific object. When a position other than the position of the virtual camera is adopted as the predetermined position, the object (bullet 332) appears to be fired from a position different from the viewpoint. On the other hand, if the start point of the control direction and the viewpoint in the terminal image are made to coincide as in the fourth game example, the player can perform an operation of shooting a bullet from a subjective viewpoint, and the player can An easy game operation can be provided.

  In step S88, the CPU 10 fires the bullet 332 in the control direction calculated in step S87. In the fourth game example, the bullet 332 is moved at a predetermined speed (a predetermined movement amount) in a predetermined movement direction. The predetermined moving direction is a direction determined by the control direction, and is a direction facing upward by a predetermined angle from the control direction. Here, in the fourth game example, it is assumed that the bullet 332 after firing is controlled to draw a parabola in consideration of gravity. When the bullet 332 flies like a parabola in this way, if the bullet 332 is fired in the control direction, the player may feel as if it was fired slightly below the control direction. Therefore, in the second embodiment, the CPU 10 fires the bullet 332 in a direction directed upward by a predetermined angle from the control direction. Thus, the moving direction of the object may be a direction determined by the control direction, and does not necessarily have to coincide with the control direction.

  As a specific process of step S88, the CPU 10 reads the control direction data 322 and calculates the movement direction of the bullet 332 from the control direction. Then, the object data 323 is read, and a position obtained by moving the position represented by the object data 323 in the direction of the calculated movement direction by a predetermined movement amount is calculated. Then, the position represented by the object data 323 is updated to the moved position, and the updated object data is stored in the main memory. As a result, the position of the bullet 332 is updated, and the bullet 332 is moved in the control direction. After step S88, the CPU 10 ends the game control process.

  Next to the game control process, the processes of steps S3 to S8 are executed in the fourth game example as well as in the third game example. That is, a television image is generated in step S4, and a terminal image is generated in step S5. Further, a television image is displayed on the television 2 in step S6, and a terminal image is displayed on the lower LCD of the handheld device 9 in step S7. In the fourth game example, the lower LCD displays the game space so that the input surface of the touch panel 52 and the control surface 335 coincide on the screen (the control surface 335 is not displayed), and the television 2 Displays a wider game space than the lower LCD (see FIG. 43). In step S8, it is determined whether or not to end the game, and a series of processes in steps S2 to S8 are repeatedly executed until it is determined in step S8 that the game is to be ended.

  According to the game process of the fourth game example described above, the posture of the control surface 335 is determined by the posture of the handheld device 9, and the control position is determined on the control surface 302 by an input to the touch panel. The bullet 332 of the cannon 331 is fired in the control direction determined by the control position. Thus, according to the present game example, the player can easily instruct the direction in the three-dimensional space by an intuitive operation using the touch panel 52.

(Modifications of the third and fourth game examples)
In the third and fourth game examples, the CPU 10 executes a game process for controlling the moving direction of a predetermined object (shuriken 301 or bullet 332) in the game space based on the control direction. Here, CPU10 should just perform the game process based on a control direction not only the game process which controls the moving direction of an object. For example, in another embodiment, the CPU 10 may execute a predetermined process on an object arranged on a straight line extending in a control direction from a predetermined position (for example, the position of a virtual camera).

  In the third and fourth game examples, the case where the handheld device 9 includes the touch panel 52 has been described as an example. However, as described above, the handheld device 9 may include a touch pad. When the handheld device 9 includes a touch pad, the handheld device 9 may or may not include a display device.

  In addition, as the touch panel or the touch pad, a system capable of detecting a plurality of input positions input at the same time (so-called multi-touch) may be used. In this case, in each of the above embodiments, the CPU 10 may calculate the control position and the control direction for each input position. Specifically, in the game control process shown in FIG. 42, the CPU 10 may execute the processes of steps S64 to S66 and the processes of steps S68 to S70 for each input position. In the game control process shown in FIG. 45, the CPU 10 may execute the processes of steps S86 to S88 for each input position.

  In the third and fourth game examples, the case where the control surface is controlled by the operation of tilting the handheld device 9 itself (changing the posture) has been described as an example. That is, the CPU 10 calculates the attitude of the handheld device 9 based on data (acceleration data 312 and angular velocity data 313) whose values change according to the movement of the handheld device 9, and controls the control surface based on the attitude of the handheld device 9. Controlled. Here, the control surface may be controlled such that the posture changes at least according to the movement of the handheld device 9. For example, in other embodiments, the control surface may be controlled in response to a change in the position of the handheld device 9. Specifically, the CPU 10 can calculate the change in the position of the handheld device 9 based on the acceleration obtained by removing the gravitational acceleration from the acceleration represented by the acceleration data 312. Therefore, for example, in the process of step S61 or S82 described above, the CPU 10 may change the position and / or posture of the control surface according to the change in the position of the handheld device 9 (for example, the position of the handheld device 9 changes). The control surface may be tilted in a direction corresponding to the selected direction). According to this, the position and / or posture of the control surface can be changed by moving the handheld device 9 up, down, left and right.

[10. Modified example]
The above-described embodiment is an example for carrying out the present invention. In other embodiments, the present invention can be implemented with, for example, the configuration described below.

(Other examples of applying the input system)
In each of the above game examples, as an example of a position calculation system for calculating a three-dimensional position in a virtual space, a game system 1 in which a player performs a game operation using an operation device (terminal device 7 or handheld device 9) is taken as an example. I explained. Here, in another embodiment, the position calculation system is not limited to a game application, and may be applied to any information processing system for calculating a three-dimensional position in a virtual space based on an operation on an operation device.

(Modifications related to faces set in virtual space)
In each of the above game examples, the plane (control plane) set in the three-dimensional virtual space is a plane. Here, in another embodiment, the surface set in the three-dimensional virtual space may be a curved surface. For example, when the surface is controlled to move on the spherical surface or the side surface of the cylinder (FIGS. 30 to 32), a curved surface along the spherical surface or the cylinder side surface may be set. Moreover, in each said game example, although the surface was the same shape as the shape of the display screen of the terminal device 7, the shape of a surface and the shape of a display screen may differ. In addition, as in the first game example, when the surface is a flat surface and the surface is controlled to move in a direction parallel to the surface itself, the user can specify the surface. The range is a range in a plane in the three-dimensional space (a range not having a length in the depth direction). On the other hand, as in the second to fourth game examples, when the surface is a plane and the surface is controlled to move in a direction not parallel to the surface itself, the user The specifiable range is a three-dimensional range (a range having lengths in the vertical, horizontal, and depth directions) in the three-dimensional space. At this time, since the surface is a flat surface, the correspondence between the position on the input surface and the position on the surface is easy for the user to understand, and the input operation of the three-dimensional position is facilitated. When the surface is a curved surface, the range that can be specified by the user is a three-dimensional range (vertical, horizontal) in the three-dimensional space even when the surface moves in parallel according to the attitude of the terminal device 7. And a range having a length in the depth direction).

  In each of the above game examples, the posture of the surface is controlled at least according to the posture of the terminal device 7. Here, in another embodiment, the plane may be determined in the virtual space in accordance with the movement of the terminal device 7, and the game system 1 determines the position, posture, and shape of the plane in accordance with the movement of the terminal device 7. At least one of may be changed. According to this, the range (range on the surface) that can be specified by input on the input surface changes according to the movement of the terminal device 7, and the user can specify the three-dimensional position. The “movement of the terminal device 7” includes the position and orientation of the terminal device 7 or changes in the position and orientation.

(Modification regarding game system configuration)
In the second game example, the game system 1 includes a portable terminal device 7 and the television 2 as display devices. Here, if the game system is configured to include an operation device that can be gripped by the user (player), for example, in another embodiment, the game system 1 displays the image on one display device (for example, the television 2). The terminal device 7 may be configured not to display an image. Further, the game system 1 may include two display devices different from the terminal device 7. At this time, the television image in the second game example is displayed on one display device, and the terminal image is displayed on the other display device.

  Moreover, although the game system 1 was the structure containing the controller 5 as an operating device different from the terminal device 7, the structure which does not contain the controller 5 may be sufficient in other embodiment. In another embodiment, the controller 5 may be further used as an operation device in addition to the terminal device 7. For example, when the player draws a figure in the virtual space by drawing a line on the touch panel 52 using the terminal device 7, the game apparatus 3 moves or deforms the drawn figure in accordance with an operation on the controller 5. You may make it do.

  Further, the position calculation system for calculating the three-dimensional position in the virtual space is not limited to that realized by a plurality of devices, and may be realized by one device. For example, the position calculation system may be realized by a portable information processing device (may be a game device) including a display unit and an input device (such as a touch panel or a touchpad) that can input on the input surface. . At this time, the information processing apparatus displays the terminal image in the second game example on the display screen provided with the touch panel, and displays the television image in the second game example on the other display screen. Further, the information processing apparatus may include a sensor (for example, a gyro sensor or an acceleration sensor) that detects a physical quantity for calculating its own posture.

(Variation regarding input position detection method on input surface)
In each of the above game examples, the operation device (the terminal device 7 or the handheld device 9) includes a touch panel 52 provided on the front surface of the LCD as an example of a position detection unit that detects an input position with respect to a predetermined input surface. there were. Here, in another embodiment, the position detection unit may be a touch pad, for example. That is, the terminal device 7 and the handheld device 9 may be configured to include a touch pad without including the display unit (LCD 51) and the touch panel 52. The operation device may display a cursor on the LCD, move the cursor according to a predetermined direction input operation (for example, an operation on an analog stick or a cross button), and use the position of the cursor as an input position.

(Modification regarding information processing apparatus for executing game processing)
In each of the above game examples, the game apparatus 3 executes a series of game processes executed in the game system 1, but a part of the game processes may be executed by another apparatus. For example, in another embodiment, the terminal device 7 may execute a part of the game processing (for example, terminal image generation processing). In another embodiment, in a position calculation system having a plurality of information processing devices that can communicate with each other, the plurality of information processing devices may share and execute game processing.

  In addition, when said information processing is performed in several information processing apparatus, it is necessary to synchronize the process performed by each information processing apparatus, and information processing will become complicated. On the other hand, when the information processing is executed by one game device 3 and the terminal device 7 receives and displays an image as in the above-described game examples (that is, the terminal device 7 or the handheld device 9). Is functioning as a thin client terminal), it is not necessary to synchronize information processing among a plurality of information processing devices, and information processing can be simplified.

  As noted above, the various systems, methods, and techniques described herein may be provided by digital electronic circuitry, computer hardware, firmware, software, or a combination of these elements. An apparatus for implementing the above technique includes a computer program product, an input / output device, and a computer processor that are embodied in a computer-readable non-transitory storage device for execution by a programmable processor. May be included. The process for realizing the above technique may be executed by a programmable processor that executes a program for executing a required function by processing input data and generating a desired output. The above technique is realized by one or more computer programs that can be executed on a computer system including a programmable processor that exchanges data and instructions with hardware resources such as an input device, an output device, and an information storage device. May be. Each computer program may be realized by a procedural or object-oriented high-level programming language, assembly language, or machine language, and may be compiled or interpreted as necessary. The above processor may be a general-purpose or dedicated microprocessor. The processor typically receives data and instructions from ROM or RAM. The storage device includes (a) a nonvolatile memory including a semiconductor memory device such as an EPROM, an EEPROM, or a flash memory, (b) a magnetic disk such as an internal hard disk or a removable external disk, and (c). It is not limited to a magneto-optical disk (d) CDROM, but includes all kinds of computer memory. The processor and the storage device described above may be supplemented by an ASIC (Application Specific Integrated Circuit) or may be implemented in a form incorporated in the ASIC.

  Further, the processing system (circuit) described in this specification is programmed for control processing such as game processing according to the contents described in this specification. A processing system including at least one CPU that executes an instruction according to the above contents may act as a “programmed logic circuit” for executing a processing operation defined by the above contents. .

  As described above, the present invention can be used as, for example, a game system, a game device, or an information processing program for the purpose of easily specifying a position in a three-dimensional space.

DESCRIPTION OF SYMBOLS 1 Game system 2 Television 3 Game device 4 Optical disk 5 Controller 6 Marker device 7 Terminal device 10 CPU
11e Internal main memory 12 External main memory 51 LCD
52 Touch Panel 73 Acceleration Sensor 74 Gyro Sensor 80 Wireless Module 91 Keyboard Object 92 Terminal Object 94, 201 Surface 100, 210, 310 Game Program 101 Terminal Operation Data 109 Terminal Attitude Data 111 Surface Data 113 Specified Position Data 302 Control Surface

Claims (32)

A position calculation system that calculates a position in a three-dimensional virtual space based on an operation on an operation device,
A position calculation system that calculates a position in the virtual space based on an attitude of the operation device and an input position with respect to a predetermined input surface provided in the operation device. Including the operating device and an information processing device;
The operating device is:
A position detector for detecting an input position with respect to the input surface;
A sensor unit for outputting data for calculating the posture;
A data transmission unit for transmitting data output from the position detection unit and the sensor unit to the information processing apparatus;
The information processing apparatus includes:
An attitude calculation unit that calculates an attitude of the operating device based on data output by the sensor unit;
The position calculation system according to claim 1, further comprising a position calculation unit that calculates a position in the virtual space based on the posture and the input position.   The position calculation system according to claim 2, wherein the sensor unit includes an inertial sensor.   The position calculation system according to claim 1, wherein the operation device further includes a display unit on which an image representing the virtual space is displayed.   The position calculation system according to claim 4, wherein the position detection unit is a touch panel provided on a screen of the display unit.   The position of the said virtual space is provided with the position calculation part which calculates the position on the surface determined according to the attitude | position of the said operating device in the said virtual space based on the said input position. The position calculation system according to any one of claims.   The position calculation system according to claim 6, further comprising a surface setting unit that sets the surface so that the posture of the surface changes according to the posture of the operation device. The operating device is:
A display unit for displaying an image representing a virtual space including the area of the surface;
A touch panel provided on the screen of the display unit,
The position calculation unit calculates the position so that an input position with respect to the touch panel and a position on the surface calculated based on the input position match on the screen. The position calculation system described.   The position calculation system according to any one of claims 1 to 8, further comprising a process execution unit that executes a predetermined process using a position in the virtual space based on the input position as an input.   The processing execution unit identifies a region including a position in the virtual space based on the input position from a plurality of regions set in the virtual space, and outputs a sound corresponding to the identified region to a predetermined audio output unit The position calculation system according to claim 9, wherein the position calculation system is configured to output the position information. The plurality of regions are arranged in a substantially arc shape,
Setting a surface in the virtual space to move on the plurality of areas in accordance with a change in the attitude of the operating device;
The position calculation system according to claim 10, wherein a position on the surface is calculated based on the input position. An object setting unit for setting the predetermined object in the virtual space;
An image generation unit for generating an image representing a virtual space including the predetermined object as an image to be displayed on the predetermined display device;
The position calculation system according to claim 9, wherein the process execution unit executes a predetermined process when the position of the predetermined object is calculated as a position in the virtual space based on the input position. . The image generation unit
A first generation unit configured to generate an image representing a partial range of the object so that the range changes according to a posture of the operating device as an image to be displayed on a display device included in the operating device;
A second generation unit configured to generate an image representing a range wider than the partial range of the object as an image to be displayed on a display device different from a display device included in the operation device; Item 13. The position calculation system according to item 12. An object placement unit for placing an object whose posture is controlled corresponding to the posture of the operating device in the virtual space;
The position calculation system according to any one of claims 1 to 3, further comprising: an image generation unit that generates an image representing the virtual space including the object as an image to be displayed on a predetermined display device. . The operating device includes a display unit that displays a keyboard arranged in a virtual space,
The sound according to the keyboard corresponding to the position is output to a predetermined sound output unit when the position of the keyboard is calculated as the position in the virtual space based on the input position. The position calculation system according to any one of claims. A position calculation method for calculating a position in a three-dimensional virtual space based on an operation on an operating device,
A position calculation method for calculating a position in the virtual space based on an attitude of the operation device and an input position with respect to a predetermined input surface provided in the operation device. The position calculation method is executed in a system including the operation device and an information processing device,
The operating device is:
A position detecting step for detecting an input position with respect to the input surface;
Performing a data transmission step of transmitting data of the input position and data for calculating the attitude to the information processing apparatus;
The information processing apparatus includes:
A posture calculating step for calculating a posture of the controller device based on data for calculating the posture;
The position calculation method according to claim 16, wherein a position calculation step of calculating a position in the virtual space based on the posture and the input position is executed.   The position calculation method according to claim 16, further comprising a display step of displaying an image representing the virtual space on a display unit included in the operation device.   The position calculation step of calculating the position on the surface determined according to the posture of the operating device in the virtual space based on the input position as the position in the virtual space. The position calculation method described.   The position calculation method according to claim 19, further comprising a surface setting step of setting the surface such that the posture of the surface changes according to the posture of the operating device. The operating device is:
A display unit for displaying an image representing a virtual space including the area of the surface;
A touch panel provided on the screen of the display unit,
The position is calculated so that the input position with respect to the touch panel and the position on the surface calculated based on the input position coincide on the screen in the position calculation step. The position calculation method according to 20.   The position calculation method according to any one of claims 16 to 21, further comprising a process execution step of executing a predetermined process using a position in the virtual space based on the input position as an input.   In the processing execution step, an area including a position in the virtual space based on the input position is specified from a plurality of areas set in the virtual space, and a sound corresponding to the specified area is output as a predetermined sound The position calculation method according to claim 22, wherein the position calculation method is output to the unit. The plurality of regions are arranged in a substantially arc shape,
Setting a surface in the virtual space to move on the plurality of areas in accordance with a change in the attitude of the operating device;
The position calculation method according to claim 23, wherein a position on the surface is calculated based on the input position. An object setting step for setting the predetermined object in the virtual space;
An image generation step of generating an image representing a virtual space including the predetermined object as an image to be displayed on a predetermined display device;
25. The position according to claim 22, wherein in the process execution step, a predetermined process is executed when a position of the predetermined object is calculated as a position in the virtual space based on the input position. Calculation method. The image generation step includes
A first generation step of generating an image representing a partial range of the object such that the range changes according to the attitude of the operating device as an image to be displayed on a display device included in the operating device;
A second generation step of generating an image representing a range wider than the partial range of the object as an image to be displayed on a display device different from a display device included in the operation device. Item 26. The position calculation method according to Item 25. An object placement step of placing in the virtual space an object whose posture is controlled corresponding to the posture of the operating device;
The position calculation method according to claim 16, further comprising: an image generation step of generating an image representing the virtual space including the object as an image to be displayed on a predetermined display device. Display the keyboard arranged in the virtual space on the display unit provided in the operating device,
When the position of the keyboard is calculated as a position in the virtual space based on the input position, a sound corresponding to the keyboard corresponding to the position is output to a predetermined sound output unit. The position calculation method described. A position calculation device that calculates a position in a three-dimensional virtual space based on an operation on an operation device,
A position calculation device that calculates a position in the virtual space based on an attitude of the operation device and an input position with respect to a predetermined input surface provided in the operation device. As an image to be displayed on a display unit provided in the operation device, an image representing a keyboard arranged in the virtual space is generated,
30. When the position of the keyboard is calculated as a position in the virtual space based on the input position, a sound corresponding to the keyboard corresponding to the position is generated as a sound to be output to a predetermined sound output unit. The position calculation device described in 1. A position calculation program for causing a computer of a position calculation device to calculate a position in a three-dimensional virtual space based on an operation on the operation device,
A position calculation program for causing the computer to function as position calculation means for calculating a position in the virtual space based on an attitude of the operation device and an input position with respect to a predetermined input surface provided in the operation device. Image generating means for generating an image representing a keyboard arranged in the virtual space, as an image to be displayed on a display unit provided in the operation device;
When the position of the keyboard is calculated as a position in the virtual space based on the input position, as sound generating means for generating sound corresponding to the keyboard corresponding to the position as sound to be output to a predetermined sound output unit The position calculation program according to claim 31, further causing the computer to function.