CN100538613C - input coordinate processing method - Google Patents

Abstract

An input coordinate processing method detects a series of coordinate information from input coordinates output from a pointing device and sets reference coordinates. When the last coordinate information indicates an area other than the reference area, the processing is determined based on a direction from the reference coordinate to the area indicated by the last coordinate information. Further, when the last coordinate information indicates that the reference area is present after the partial coordinate information indicates that the area is outside the reference area, the processing is determined based on a combination of directions of reciprocation in the reference coordinate and the area indicated by the partial coordinate information.

Description

Input coordinate processing method

technical field

The present invention relates to an input coordinate processing method, an input coordinate processing device, an input coordinate processing program, and a recording medium on which the input coordinate processing program is recorded, and in particular to input coordinates output by a pointing device such as a touch panel that outputs coordinate information of a predetermined coordinate system, An input coordinate processing method, an input coordinate processing device, an input coordinate processing program, and a recording medium recording the input coordinate processing program are determined for processing.

Background technique

Conventionally, there is known a technique for determining the processing scheduled to be performed next time (hereinafter referred to as the next processing) based on coordinates input by operating a mouse. In addition, there is also known a technique of performing display processing on a display unit based on coordinates input by operating a touch panel and a mouse. For example, such as "Mouse Gestures in Opera", [online], [retrieved on March 15, 2017], Internet <URL: http://www.opera.com/features/mouse/> (hereinafter referred to as non-patent literature As described in 1), there has been proposed a technology that enables operations called "mouse gestures" in a web browser for browsing Internet web pages.

In the browser described in the aforementioned Non-Patent Document 1, if the mouse is moved to the left while the mouse is right-clicked in the background area of the web page, and then the right button is released, the web page proceeds to the next page. In the background area of the web page, if you move the mouse to the right in the state of right-clicking the mouse, and then release the right button, you will return to the previous page. Also, if you move the mouse down while right-clicking the mouse on the background area of the web page, and then release the right button, a new window will open. Also, if you move the mouse down while right-clicking the mouse in the background area of the web page, then move it up and release the right button, you can duplicate a window.

In addition, if you right-click the mouse while the web page is linked, move the mouse upward while right-clicking, and then release the right button, the link destination will be opened in a new window. In addition, if you right-click the mouse in the link state of the web page, move the mouse down in the right-click state, move the mouse up, and release the right button, the link target will be opened in the background. These are different processes from the case of opening the link destination in the original window when the mouse is left-clicked in the linked state of the web page.

In this way, in the mouse gesture disclosed in the aforementioned Non-Patent Document 1, the next process is determined based on the direction in which the trajectory based on the input coordinates of the mouse is accumulated and stored, or a combination of these directions. This saves the trouble of clicking the buttons arranged around the web page on the display screen or operating the tool bar in the aforementioned mouse gestures.

In a browser called Sleipnir, etc., a recognized gesture is displayed on the left bottom of the screen. For example, the accumulated and stored directions of the trajectory of the input coordinates are expressed as "↑", "↓", "↑", "→" in sequence. The user can anticipate the next process by viewing this display.

Furthermore, in the mouse gesture disclosed in the aforementioned Non-Patent Document 1, the display processing by the display unit is performed according to the direction in which the trajectory based on the input coordinates of the mouse is accumulated and stored, or a combination of these directions.

In addition, for example, as shown in Japanese Patent No. 3095145 (hereinafter referred to as Patent Document 1), a technique is disclosed in which the trajectory of input coordinates based on the touch panel is used as a stroke trajectory, and when there is no return in the stroke trajectory , to move the window to the stroke moving target, and store the window in the predetermined unit when there is a turnaround in the stroke trajectory.

However, in the browser described in the aforementioned Non-Patent Document 1, no problem occurs when the user draws an appropriate trajectory only once, but occurs when an inappropriate trajectory is drawn. question. That is, as a result of accumulating the trajectory based on the input coordinates of the mouse, if the user continues to move in order to correct the inappropriate trajectory, an unexpected trajectory will be accumulated more and more, and the user will not be able to perform the desired next time after all. deal with. In addition, in order to delete the inappropriate trajectory, the user has to perform a cumbersome operation of canceling the right-click of the mouse, right-clicking again, and drawing a desired trajectory again.

On the other hand, it is also considered that instead of combining multiple directions, the trajectory based on the input coordinates of the mouse is cumulatively stored, and the next processing is determined based on only one direction obtained thereby. However, the change that determines the next processing becomes a quantity that can be distinguished by one direction, and an operation gesture indicating the next processing becomes lacking. In addition, although there is also a method of increasing the number of distinctions by one direction, since the angle used to judge each direction will become smaller, there are many cases where the direction drawn by the user is different from the direction recognized by the computer. Difficult for users to operate.

In addition, as described above, there are also browsers that display the recognized operation gestures on the bottom left side of the display screen. However, since the operation gesture is displayed at an inconspicuous position at the bottom of the left side of the display screen, it is difficult for the user to recognize it. In addition, even if the user can recognize it, in order to clearly see the display of the operation gesture, it is necessary to switch the line of sight to the bottom of the left side of the screen one by one from the browsing position of the web page, which is troublesome for the user.

In addition, in the browser described in the aforementioned Non-Patent Document 1, the user often has no sense of intuition in operations based on mouse gestures. For example, intuitively, it is difficult to make a connection between the aforementioned operation of "moving the mouse down while right-clicking the mouse in the background area of the web page" and the display process of "opening a new window". Similarly, between the aforementioned operation of "right-clicking the mouse in the background area of the web page, and then moving the mouse down and then up" and the display processing of "copying the window", it is also difficult intuitively. Create a relationship. Therefore, the user must memorize the relationship between these operations and display processing as a special constraint item. The user must memorize this special constraint, which prevents effective use of the mouse gesture functionality.

In addition, in the technology of the above-mentioned Patent Document 1, the relationship between the touch operation based on the stroke trajectory of the trajectory of the input coordinates of the touch panel and the display process of the registration window is only a special constraint item for the user. To perform corresponding processing, it is intuitively difficult to make a correlation between this operation and display processing. However, even if the correlation between the user's operation and the corresponding display processing is high, the display processing itself is useless and meaningless. Therefore, it is also expected that display processing itself, which is highly correlated with user operations, is useful.

Contents of the invention

Therefore, the first object of the present invention is to provide an input coordinate processing method that improves the operability of coordinate input while maintaining the number of processing changes that can be designated by coordinate input operations, and simplifying the processing for inappropriate trajectories. An input coordinate processing device, an input coordinate processing program, and a recording medium recording the input coordinate processing program. In addition, a second object of the present invention is to provide an input coordinate processing method, an input coordinate processing device, an input coordinate processing program, and a recording medium recording the input coordinate processing program, in which a user can easily recognize a recognized manipulation gesture. In addition, a third object of the present invention is to provide an input coordinate processing method, an input coordinate processing device, an input coordinate processing program, and an input coordinate processing program capable of performing intuitive operations and highly useful display processing through coordinate input operations. Enter the recording medium of the coordinate processing program.

To achieve the foregoing object, the present invention employs the following constitutions. The reference symbols and step numbers in parentheses (the steps are abbreviated as S, and only the step numbers are described) are used for understanding the present invention, and represent the corresponding relationship with the embodiments described later, and do not limit the scope of the present invention.

The first aspect is an input coordinate processing method that operates with input coordinates (coordinate data corresponding to a contact position) output from the pointing device 15 in accordance with a user's operation. The input coordinate processing method comprises: input coordinate detection step (S43, S63, S68, S73, S84, S87, S92, S104, S107, S114, S120, S134, S139, S144); reference coordinate storage step S44; multi-region setting Step (FIG. 11); processing determines steps S50 to S52. In the input coordinate detection step, coordinate information DC1 based on a predetermined coordinate system (screen coordinate system) is detected based on the input coordinates output from the pointing device. The reference coordinate storing step sets and stores the reference coordinate DC2 based on the first coordinate information among the series of coordinate information detected in the input coordinate detecting step. In the multi-area setting step, a plurality of areas AM, AT, AB, AL, AR are formed using the reference coordinates as a reference, and the area including the reference coordinates is set as the reference area AM. The process determination step determines the process based on the last coordinate information (coordinates when the touch is off) and the reference coordinates among the series of coordinate information detected in the input coordinate detection step. The processing determining steps include: first processing determining steps S115 , S135 , S140 , S145 and second processing determining steps S121 . In the first process determining step, when the last coordinate information indicates areas AT, AB, AL, AR other than the reference area, the process is determined based on the direction from the reference coordinates to the area indicated by the last coordinate information. In the second processing decision step, after some coordinate information in a series of coordinate information detected in the input coordinate detection step indicates the area outside the reference area (DF8 is turned on), when the last coordinate information indicates the reference area, The processing is determined based on the combination of the direction from the reference coordinates to the area indicated by the partial coordinate information and the direction from the area indicated by the partial coordinate information to the reference coordinates. After the position indicated by the coordinate information of the above-mentioned reference area is moved from the aforementioned reference area to another area, when the position indicated by the aforementioned last coordinate information moves from the other area to another area different from the aforementioned reference area and the other area, based on the The direction of the coordinates to the other area is determined. In the second process determination step, after the position indicated by the last coordinate information moves from the reference area to another area, the position indicated by the last coordinate information changes from When the other area further moves to the reference area, processing is determined based on a combination of a direction from the reference coordinates to the other area and a direction from the other area to the reference coordinates. In addition, a pointing device is an input device for specifying an input position and coordinates on a screen, and is realized by, for example, a touch panel, mouse, trackpad, trackball, or the like. The coordinate system used by each input device is a touch panel coordinate system and a screen coordinate system. In addition, the aforementioned input coordinate processing method may be an input coordinate processing program executed in the computer 21 of a device operated by input coordinates output from a pointing device in accordance with user operations, and a recording medium in which the input coordinate processing program is recorded. to fulfill.

For the second aspect, in the aforementioned first aspect, the processing decision step further includes: flag setting steps S112, S132, S137, S142; first flag updating steps S112, S132, S137, S142; second flag updating steps S118. In the flag setting step, when the position indicated by the last coordinate information moves from the reference area to another area, set and store the flags DF8, DF10 to DF12 corresponding to the other area. In the first flag updating step, after the flag is set in the flag setting step, when the position shown in the last coordinate information moves further from the area where the flag is set to another area different from the reference area, the current set The predetermined flags are updated and stored as flags DF8, DF10 to DF12 corresponding to the area indicated by the last coordinate information. In the second flag updating step, after the flag is set in the flag setting step or the first flag updating step, when the position indicated by the last coordinate information moves further from the area where the flag is set to the reference area, the current The set flag is updated and stored as flag DF9 indicating the reciprocation between the reference area and the area where the flag is set. The first processing determining step determines processing based on the flag set in the flag setting step or the first flag updating step. The second processing determining step determines processing based on the flag set in the second flag updating step.

Regarding the third aspect, in the aforementioned first aspect, further include display control steps S113, S119, S133, S138, S143. The display control step is to display on the display device 12 indicators M8 to M12 indicating directions from the reference coordinates for which the processing has been determined in the first processing determination step to the direction (upward direction, downward direction) of the aforementioned other area. Direction, Left Direction, Right Direction), and the combined direction of processing determined in the second processing decision step.

Regarding the fourth aspect, in the aforementioned first aspect, a display control step is further included. In the display control step, an image M7 representing the reference area set in the multi-area setting step is displayed on the display device.

Regarding the fifth aspect, in the aforementioned first aspect, further comprising a display control step. The display control step is to display the image based on the coordinate system on the display device. In the multi-region setting step, the peripheral image region centered on the reference region is divided into a plurality to form a plurality of regions.

A sixth aspect is an input coordinate processing method that operates with input coordinates output from a pointing device according to a user's operation. The input coordinate processing method includes: an input coordinate detection step; a reference coordinate storage step; a first processing decision step; a second processing decision step; and a display control step. In the input coordinate detection step, coordinate information based on a predetermined coordinate system is detected based on the input coordinates output from the pointing device. The reference coordinate storing step sets and stores the reference coordinates based on the first coordinate information among the series of coordinate information detected in the input coordinate detecting step. The first processing determining step determines the processing based on the direction from the position indicated by the reference coordinates to the position indicated by the last coordinate information in the series of coordinate information detected in the input coordinate detecting step. The second processing decision step is based on the direction from the position indicated by the reference coordinates to the position indicated by the coordinate information on the way among the series of coordinate information detected in the input coordinate detection step, and the direction from the position indicated by the coordinate information on the way. The combination of the position and the direction to the position indicated by the reference coordinates determines the processing. A display control step of displaying an image based on the coordinate system on the display device, and displaying, on the image near the position indicated by the last coordinate information, indicators indicating the direction determined in the first processing determination step. The direction of processing is determined, and the combined direction of processing is determined in the second processing decision step. Here, the "nearby" position indicated by the last coordinate information includes at least the position itself indicated by the last coordinate information and an area adjacent to the position. In addition, the aforementioned input coordinate processing method may be an input coordinate processing program executed in the computer 21 as a device operated by input coordinates output from a pointing device in accordance with a user operation, and a recording medium on which the input coordinate processing program is recorded. way to achieve.

A seventh aspect is an input coordinate processing device operated by input coordinates output from a pointing device in accordance with a user's operation. The input coordinate processing device has: a storage unit 24; an input coordinate detection unit; a reference coordinate storage processing unit; a multi-region setting unit; The input coordinate detection unit detects coordinate information based on a predetermined coordinate system based on input coordinates output from the pointing device. The reference coordinate storage processing unit sets the reference coordinates based on the first coordinate information among the series of coordinate information detected by the input coordinate detection unit, and stores the reference coordinates in the storage unit. The multi-region setting unit forms a plurality of regions based on the reference coordinates, and sets the region including the reference coordinates as the reference region. The processing determining unit determines processing based on last coordinate information and reference coordinates among a series of coordinate information detected by the input coordinate detecting unit. The processing determining unit includes a first processing determining unit and a second processing determining unit. The first processing determining unit determines processing based on a direction from the reference coordinates to the area indicated by the last coordinate information when the last coordinate information indicates an area other than the reference area. The second process determination unit, after the partial coordinate information in the series of coordinate information detected by the input coordinate detection unit indicates the area other than the reference area, when the last coordinate information indicates the reference area, based on the direction from the reference coordinate to the The processing is determined by a combination of the direction of the area indicated by the partial coordinate information and the direction from the area indicated by the partial coordinate information to the reference coordinates.

An eighth aspect is an input coordinate processing device operated by input coordinates output from a pointing device in accordance with a user's operation. The input coordinate processing device has: a storage unit; an input coordinate detection unit; a reference coordinate storage processing unit; a first processing determination unit; a second processing determination unit; and a display control unit. The input coordinate detection unit detects coordinate information based on a predetermined coordinate system based on input coordinates output from the pointing device. The reference coordinate storage processing unit sets the reference coordinates based on the first coordinate information among the series of coordinate information detected by the input coordinate detection unit, and stores the reference coordinates in the storage unit. The first processing determining means determines processing based on a direction from the position indicated by the reference coordinates to the position indicated by the last coordinate information in a series of coordinate information detected by the input coordinate detecting means. The second processing determining unit is based on the direction from the position indicated by the reference coordinates to the position indicated by the coordinate information on the way among the series of coordinate information detected by the input coordinate detection unit, and the direction from the position indicated by the coordinate information on the way. The combination of the position and direction indicated by the reference coordinates determines the processing. The display control unit displays an image based on the coordinate system on the display device, and displays, on the image in the vicinity of the position indicated by the last coordinate information, indicators indicating the directions of the processing determined by the first processing determining unit. direction, and the combined direction of processing determined by the second processing determining unit.

The ninth aspect is an input coordinate processing method, which is based on the input coordinates (coordinate data corresponding to the contact position) output from the pointing device 15 in accordance with the user's operation, on the first display unit 12 or 11 and the first display unit 12 or 11 constituting the display device. The screens are displayed one by one on the second display unit 11 or 12 . The input coordinate processing method comprises: input coordinate storage step S43, S63, S68, S73, S84, S87, S92, S104, S107, S114, S120, S134, S139, S144; Reference coordinate storage step S44; Processing step S69, S88, S115, S135; display control steps S69, S88, S115, S135. The input coordinate storing step detects and stores coordinate information DC1 based on the display coordinate system (screen coordinate system) of the image displayed on the display device based on the input coordinate output from the pointing device. In the reference coordinate storing step, the first coordinate information among the series of coordinate information stored in the input coordinate storing step is set as the reference coordinate DC2 and stored. In the processing step, based on at least one coordinate information in a series of coordinate information, a display image (response image corresponding to a mark and a link, and an image displayed on the first LCD 11 or the second LCD 12 ) is obtained. In the display control step, based on the reference coordinates, the operation direction indicated by a series of coordinate information in the display coordinate system indicates the reference direction (upward or downward) from the position of the first display part to the position of the second display part. direction) (Yes in S65, Yes in S83, Yes in S111, Yes in S131), display the display image obtained in the processing step on the second display part (Fig. 5, Fig. 7 , Figure 10). In addition, the display image obtained in the processing step includes a response image corresponding to the mark and the link, at least a partial image displayed on the first display unit, and the like. In addition, a pointing device is an input device for specifying an input position and coordinates on a screen, and is realized by, for example, a touch panel, mouse, trackpad, trackball, or the like. The coordinate system used by each input device is a touch panel coordinate system and a screen coordinate system. The first display unit and the second display unit may be physically separate display units, or may be formed by dividing a physically single display screen. In addition, the arrangement relationship between the first display unit and the second display unit may be two screens up and down, or two screens left and right. In addition, the aforementioned input coordinate processing method can also be implemented as an input coordinate processing program executed by a computer 21 and a recording medium on which the input coordinate processing program is recorded. The input coordinates output by the device are used to execute display processing for displaying screens one by one on the first display unit and the second display unit constituting the display device.

Regarding the tenth aspect, in the aforementioned ninth aspect, the input coordinate processing method is to perform display processing of a web browser that downloads a file via the communication unit 33 communicating with the network and browses it on the display device based on the input coordinates. . In the processing step, when the reference coordinates are located in the response area (tag, link) of the web browser (YES in S46, YES in S47), the response image corresponding to the response area is acquired. In the display control step, when the operation direction indicates the reference direction, a response image is displayed on the second display portion (FIG. 5, FIG. 7).

Regarding the eleventh aspect, in the above-mentioned tenth aspect, in the processing step, when the reference coordinates are located within the background area of the web browser (YES in S48), at least part of the image displayed on the first display unit is acquired (the image displayed by the second LCD12 or the first LCD11). In the display control step, when the operation direction indicates the reference direction, at least a part of the image displayed on the first display unit is displayed on the second display unit ( FIG. 10 ).

Regarding the twelfth aspect, in the aforementioned tenth aspect, the reference direction is an upward direction or a downward direction in which the second display portion is provided with respect to the first display portion of the display device. In the processing step, when the reference coordinates are within the background area of the web browser and the operation direction indicates the left direction or the right direction, an image that has been obtained in the past in the processing step is obtained. In the display control step, when the operation direction indicates left or right (YES in S136, YES in S141), images acquired in the past are displayed on the first display unit (S140, S145).

Regarding the thirteenth aspect, in the aforementioned ninth aspect, in the processing step, when the trajectory DC3 of a series of coordinate information surrounds the specified area displayed on the first display part (YES in S103), the specified area is obtained. An enlarged image in which the image in the area is enlarged (S108). In the display control step, an enlarged image is displayed on the second display unit (FIG. 9).

Regarding the fourteenth aspect, in the aforementioned ninth aspect, the reference direction is an upward direction or a downward direction in which the second display portion is provided with respect to the first display portion of the display device. In the processing procedure, when the operation direction indicates the left direction or the right direction, an image of another page different from the page displayed on the first display unit is acquired. In the display control step, when the operation direction indicates the left direction or the right direction, another page image is displayed on the first display unit.

Regarding the fifteenth aspect, in the aforementioned ninth aspect, based on the coordinate information of the aforementioned display coordinate system corresponding to the input coordinates output from the pointing device, only the image displayed on the first display unit and the second One of the images displayed on the display.

A sixteenth aspect is an input coordinate processing method for displaying screens one by one on a first display unit and a second display unit constituting a display device based on input coordinates output from a pointing device in response to a user operation. The input coordinate processing method includes: an input coordinate storage step; a reference coordinate storage step; a processing step (S74, S93, S121); and a display control step (S74, S93, S121). The input coordinate storing step detects and stores coordinate information based on the display coordinate system of the image displayed on the display device based on the input coordinates output from the pointing device. In the reference coordinate storing step, the first coordinate information among the series of coordinate information stored in the input coordinate storing step is set as the reference coordinate and stored. The processing step is to obtain a display image based on at least one coordinate information in the series of coordinate information. In the display control step, when using the reference coordinates as a reference, the operation direction indicated by a series of coordinate information in the display coordinate system indicates a reciprocating direction (up and down direction) reciprocating in the arrangement direction of the first display unit and the second display unit (Yes in S70, yes in S89, yes in S116 and S117), at least part of the image displayed on the second display is displayed on the first display, and at the same time displayed on the second display Display images obtained in the processing steps (Fig. 6, Fig. 8, Fig. 12). In addition, the display image obtained in the processing step includes a response image, at least a partial image displayed on the first display unit, and the like. In addition, the aforementioned input coordinate processing method can also be implemented as an input coordinate processing program executed by a computer 21 based on a pointing device that is operated by a user, and a recording medium that records the input coordinate processing program. The outputted input coordinates are used to execute display processing for displaying screens one by one on the first display unit and the second display unit constituting the display device.

A seventeenth aspect is an input coordinate processing method that displays screens one by one on a first display unit and a second display unit constituting a display device based on input coordinates output from a pointing device in response to a user operation. The input coordinate processing method includes: input coordinate storage step; reference coordinate storage step; processing step; display control step. The input coordinate storing step detects and stores coordinate information based on a display coordinate system of an image displayed on the display device based on input coordinates output from the pointing device. In the reference coordinate storing step, the first coordinate information among the series of coordinate information stored in the input coordinate storing step is set as the reference coordinate and stored. The processing step is to obtain a display image based on at least one coordinate information in the series of coordinate information. In the display control step, when using the reference coordinates as a reference, when the operation direction indicated by a series of coordinate information in the display coordinate system indicates the reference direction from the position of the first display unit to the position of the second display unit, in the second The second display unit displays the display image obtained in the processing step. The display control step is to display at least a part of the image displayed on the second display on the first display when the operation direction indicates a reciprocating direction in which the first display and the second display are arranged, and at the same time display on the second display. The display unit displays the display image acquired in the processing steps.

An eighteenth aspect is an input coordinate processing device that executes display processing of displaying screens one by one on a first display unit and a second display unit constituting a display device based on input coordinates output from a pointing device in response to a user operation. The input coordinate processing device has: a storage unit 24; an input coordinate storage processing unit; a reference coordinate storage processing unit; a processing unit; and a display control unit. The input coordinate storage processing unit detects the coordinate information of the display coordinate system based on the image displayed on the display device based on the input coordinates output from the pointing device, and stores the coordinate information in the storage unit. The reference coordinate storage processing unit sets the first coordinate information in the series of coordinate information stored in the input coordinate storage unit as the reference coordinate and stores it in the storage unit. The processing unit acquires a display image based on at least one coordinate information in the series of coordinate information. The display control unit, when using the reference coordinates as a reference, displays the operation direction indicated by a series of coordinate information in the coordinate system, indicating the reference direction from the position of the first display part to the position of the second display part. The second display unit displays the display image acquired by the processing unit.

A nineteenth aspect is an input coordinate processing device that executes display processing for displaying screens one by one on a first display unit and a second display unit constituting a display device based on input coordinates output from a pointing device in accordance with a user operation. . The input coordinate processing device has: a storage unit; an input coordinate storage processing unit; a reference coordinate storage processing unit; a processing unit; and a display control unit. The input coordinate storage processing unit detects the coordinate information of the display coordinate system based on the image displayed on the display device based on the input coordinates output from the pointing device, and stores the coordinate information in the storage unit. The reference coordinate storage processing unit sets the first coordinate information in the series of coordinate information stored by the input coordinate storage processing unit as the reference coordinate and stores it in the storage unit. The processing unit acquires a display image based on at least one coordinate information in the series of coordinate information. The display control unit is based on the reference coordinates, and when the operation direction indicated by a series of coordinate information in the display coordinate system indicates the reciprocating direction in the arrangement direction of the first display part and the second display part, in the second A display unit displays at least part of the image displayed on the second display unit, and at the same time displays the displayed image acquired by the processing unit on the second display unit.

A twentieth aspect is an input coordinate processing device that executes display processing for displaying screens one by one on the first display unit and the second display unit constituting the display device based on input coordinates output from the pointing device in response to user operations. The input coordinate processing device has: a storage unit; an input coordinate storage processing unit; a reference coordinate storage processing unit; a processing unit; and a display control unit. The input coordinate storage processing unit detects the coordinate information of the display coordinate system based on the image displayed on the display device based on the input coordinates output from the pointing device, and stores the coordinate information in the storage unit. The reference coordinate storage processing unit sets the first coordinate information in the series of coordinate information stored by the input coordinate storage processing unit as the reference coordinate and stores it in the storage unit. The processing unit acquires a display image based on at least one coordinate information in the series of coordinate information. The display control unit, when using the reference coordinates as a reference, displays the operation direction indicated by a series of coordinate information in the coordinate system, indicating the reference direction from the position of the first display part to the position of the second display part. The second display unit displays the display image acquired by the processing unit. When the operation direction indicates the reciprocating direction in which the first display unit and the second display unit are arranged, the first display unit displays the image displayed on the second display unit. at least part of the image, and simultaneously display the displayed image obtained by the processing unit on the second display unit.

According to the aforementioned first aspect, according to the second processing determining step, it is determined that a user's gesture is recognized and executed through a combination of a plurality of operation directions corresponding to an input from the pointing device. Therefore, compared with distinguishing the user's operation gesture by a single operation direction, various changes can be made to determine the processing. On the other hand, in the first processing decision step, the gesture based on the operation direction can be easily canceled, and only by operating to an area other than the reference area, only the operation direction from the reference coordinates to the area can be recognized. into effective gestures. Therefore, when the operation performed by the user is inappropriate, the user can easily cancel the operation and provide a new operation instruction by performing an operation following the operation. That is, even if the number of changes that can be determined due to the coordinate input operation increases, failures that occur when an inappropriate trajectory is drawn are reduced, and the operability of coordinate input can be improved.

According to the aforementioned second aspect, since entry and exit to a plurality of areas are managed using the markers, it is possible to more easily distinguish the user's gestures than processing based on all input coordinates.

According to the aforementioned third aspect, since the operation gesture recognized by the coordinate input operation input by the user is displayed as an indicator, the operation gesture input by the user and the determined process can be reliably recognized.

According to the foregoing fourth aspect, by displaying a reference area to be a reference for discriminating operation gestures, it is possible to form a target image for a user to input an effective operation using a pointing device.

According to the aforementioned fifth aspect, by forming a plurality of regions for distinguishing the operation directions by dividing the surrounding region centered on the reference region, the operation directions centered on the reference region can be easily distinguished.

According to the aforementioned sixth aspect, the operation gesture recognized by the coordinate input operation input by the user is displayed as an indicator near the position where the user currently performs the input operation on the displayed image, so that the operation input by the user can be reliably recognized. Handling of gestures and decisions.

According to the aforementioned ninth aspect, according to the user's operation using the pointing device, it is possible to perform highly useful display processing through intuitive operation. For example, if the user uses a pointing device to operate in the direction from the first display unit to the second display unit, the images obtained in the processing steps (response images corresponding to tags and links and the images displayed on the first display unit) At least part of the image) is displayed on the second display unit in the operating direction (direction from the first display unit toward the second display unit). In this way, the operation direction input by the user to the pointing device and the corresponding display processing are intuitive. Furthermore, the fact that the original display information is retained on the first display unit and information corresponding to the process is displayed on the second display unit is useful in various situations.

According to the aforementioned tenth aspect, in the display process of the web browser, the response image obtained by selecting the mark and the link destination can be displayed on the second display unit in the operation direction.

According to the aforementioned eleventh aspect, in the display processing of the web browser, the image displayed on the first display unit can be displayed on the second display unit in the operation direction.

According to the aforementioned twelfth aspect, in the display processing of the web browser, by operating the pointing device in the left or right direction different from the reference direction, it is possible to perform so-called "return" and "Forward" display handling.

According to the aforementioned thirteenth aspect, a desired image can be enlarged and displayed on the second display unit by the user operating around the partial image displayed on the first display unit.

According to the aforementioned fourteenth aspect, by operating the pointing device in the left or right direction different from the reference direction, it is possible to perform display equivalent to the "previous page" and "next page" which are generally expressed in the left and right directions in electronic books and the like. deal with.

According to the aforementioned fifteenth aspect, by operating one display section that is operable input, it is possible to perform display processing for the other display section that is not operable input.

According to the aforementioned sixteenth aspect, according to the user's operation using the pointing device, it is possible to perform highly useful display processing by intuitive operation. For example, if the user uses a pointing device to perform reciprocating operations in the arrangement directions of the first display part and the second display part, at least part of the image displayed on the second display part will be displayed in one operation direction (from the second display The first display part on the direction of the first display part), and the image obtained in the processing step (response image corresponding to the mark and the link and at least part of the image displayed by the first display part) is displayed on the operation The second display part in the other direction of the direction (the direction from the first display part to the second display part). Therefore, the information displayed on the display units in the reciprocating operation direction of the user (reciprocating direction in the direction where the first display unit and the second display unit are located) are alternately displayed. In this way, the operation direction input by the user to the pointing device and the corresponding display processing are intuitive. Furthermore, the fact that the original display information is retained on the first display unit and information corresponding to the process is displayed on the second display unit is useful in various situations.

According to the aforementioned seventeenth aspect, the same effects as those of the aforementioned ninth and sixteenth aspects can be obtained.

According to the input coordinate processing device of the present invention, the same effects as those of the aforementioned input coordinate processing method can be obtained. In addition, even when the aforementioned input coordinate processing method is implemented in the form of an input coordinate processing program and a recording medium recording the input coordinate processing program, the same effects as the aforementioned input coordinate processing method can be obtained.

With reference to the accompanying drawings, combined with the following detailed description, the foregoing and other objects, features, aspects and effects of the present invention can be more clearly understood.

Description of drawings

1 is an external view of a game machine device 1 that executes an input coordinate processing program of the present invention,

Fig. 2 is a block diagram showing the internal structure of the game machine device 1 of Fig. 1,

FIG. 3 is a diagram showing an example of screen display at an initial stage displayed on the first LCD 11 and the second LCD 12,

FIG. 4 is a diagram showing an example of screen displays displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 on the link destination displayed on the second LCD 12 is tapped.

FIG. 5 is a diagram of an example of screen display displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid upward from the link target displayed on the second LCD 12.

6 is a diagram showing an example of the screen displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid upwards from the link target displayed on the second LCD 12 and then reciprocated downwards.

7 is a diagram showing an example of screen displays displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid from above a tab displayed on the second LCD 12 to the bottom.

8 is a diagram showing an example of screen displays displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid from the top to the bottom of the mark displayed on the second LCD 12 and then reciprocated upwards.

9 is a diagram showing an example of screen display displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid to surround the partial information displayed on the second LCD 12,

10 is a diagram showing an example of screen displays displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid upward from the background image displayed on the second LCD 12,

11 is a diagram for explaining each area set when the touch panel 15 is slid on the background image displayed on the second LCD 12,

FIG. 12 is a diagram showing an example of screen displays displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid upwards from the background image displayed on the second LCD 12 and then reciprocated downwards.

FIG. 13 is a flow chart showing the operation of causing the game machine device 1 to perform input coordinate processing by executing the input coordinate processing program executed by the game machine device 1,

FIG. 14 is a subroutine showing the detailed operation of the link specifying process in step 50 in FIG. 13,

Fig. 15 is a subroutine showing the detailed operation of the mark specifying process in step 51 in Fig. 13,

Fig. 16 is a subroutine showing the detailed operation of the background specifying process in step 52 in Fig. 13,

Fig. 17 is a subroutine showing the detailed operation of the background specifying process in step 52 in Fig. 13,

Fig. 18 is a subroutine showing the detailed operation of the background specifying process in step 52 in Fig. 13,

FIG. 19 is a diagram showing an example of various data stored in RAM 24 by the processing operation based on FIG. 13 .

Detailed ways

An input coordinate processing method according to an embodiment of the present invention will be described with reference to the drawings. The input coordinate processing method can be realized by means of an input coordinate processing device, an input coordinate processing program, and a recording medium on which the input coordinate processing program is recorded. Hereinafter, as an embodiment for realizing the input coordinate processing method of the present invention, an input coordinate processing device that executes an input coordinate processing program will be described. The input coordinate processing program of the present invention can be applied by using any computer system that can be displayed on a display device to execute, but as an example of an information processing device (input coordinate processing device), use The input coordinate handler to illustrate. FIG. 1 is an external view of a game machine device 1 that executes an input coordinate processing program of the present invention. Here, a portable game machine is shown as an example of the game machine 1 .

In FIG. 1 , the game machine device 1 includes: a first LCD (Liquid Crystal Display: liquid crystal display device) 11, and a second LCD 12. The casing 13 is constituted by an upper casing 13a and a lower casing 13b, and the first LCD 11 is stored in the upper casing 13a, and the second LCD 12 is stored in the lower casing 13b. The resolutions of the first LCD 11 and the second LCD 12 are both 256 dots×192 dots. In addition, in the present embodiment, an LCD is used as the display device, but other arbitrary display devices such as a display device using EL (Electro Luminescence: electroluminescence), for example, may also be used. In addition, any resolution can be used for the first LCD 11 and the second LCD 12 .

Sound emission holes 18a, 18b for emitting sound from a pair of speakers (30a, 30b in FIG. 2 ) to be described later are formed in the upper case 13a.

In the lower case 13b, as an input device, there are: a cross switch 14a, a start switch 14b, a selection switch 14c, an A button 14d, a B button 14e, an X button 14f, a Y button 14g, a power switch 14h, an L button 14L, R button 14R. In addition, as a further input device, a touch panel 15 is mounted on the screen of the second LCD 12 . In addition, an insertion port for storing the memory card 17 and the rod 16 is also provided in the lower case 13b.

As the touch panel 15 , for example, a panel of any type, such as a resistive film type, an optical type (infrared type), and a capacitive coupling type, can be used. The touch panel 15 is an example of a pointing device having a function of outputting coordinate data corresponding to the touched position when its surface is touched with the stick 16 . In addition, in the following description, the user uses the stick 16 to operate the touch panel 15 as an example. However, of course, instead of the stick 16, a pen (stylus) and fingers may be used to operate the touch panel 15 . In the present embodiment, as the touch panel 15 , a panel having a resolution (detection accuracy) of 256 dots×192 dots, which is the same as that of the second LCD 12 , is used. However, the resolution of touch panel 15 does not necessarily have to match the resolution of second LCD 12 .

The memory card 17 is a recording medium on which an input coordinate processing program and the like are recorded, and is detachably attached to an insertion opening provided in the lower case 13b. The memory card 17 is an example of a recording medium on which the input processing program of the present invention is recorded.

Next, the internal configuration of the game machine device 1 will be described with reference to FIG. 2 . FIG. 2 is a block diagram showing the internal configuration of the game machine device 1 .

In FIG. 2 , a CPU core 21 is mounted on an electronic circuit board 20 stored in a casing 13 . On the CPU core 21, a connector 23 is connected via a bus 22, and is also connected with: an input/output interface circuit (marked as an I/F circuit in the accompanying drawings) 25; a first GPU (Graphics Processing Unit) 26; a second GPU 27; a RAM 24 ; LCD controller 31 ; wireless communication unit 33 . The memory card 17 is detachably connected to the connector 23 . The memory card 17 is mounted with ROM17a which stores the input coordinate processing program, and RAM17b which stores backup data rewritably. The input coordinate processing program stored in the ROM 17 a of the memory card 17 is loaded into the RAM 24 , and the input coordinate processing program loaded into the RAM 24 is executed by the CPU core 21 . In addition to the input coordinate processing program, RAM 24 appropriately stores data for generating temporary data obtained by CPU core 21 executing the program. Connected to the I/F circuit 25 are a touch panel 15, a right speaker 30a, a left speaker 30b, and an operation switch unit 14 composed of a cross switch 14a and an A button 14d in FIG. 1 . The right speaker 30a and the left speaker 30b are disposed inside the sound emitting holes 18a and 18b, respectively.

The first GPU26 is connected to the first VRAM (Video RAM) 28, and the second GPU27 is connected to the second VRAM29. The first GPU 26 generates a first display image based on the data for generating a display image stored in the RAM 24 according to an instruction from the CPU core 21 , and draws the first display image in the first VRAM 28 . The second GPU 27 also generates a second display image based on instructions from the CPU core 21 and draws the second display image in the second VRAM 29 . The first VRAM 28 and the second VRAM 29 are connected to the LCD controller 31 .

The LCD controller 31 includes registers 32 . The register 32 stores a value of 0 or 1 according to an instruction from the CPU core 21 . The LCD controller 31, when the value of the register 32 is 0, outputs the first display image drawn in the first VRAM28 to the first LCD11, and outputs the second display image drawn in the second VRAM29. to the second LCD12. In addition, when the value of the register 32 is 1, the first display image drawn in the first VRAM 28 is output to the second LCD 12, and the second display image drawn in the second VRAM 29 is output to the first LCD 12. LCD11.

The wireless communication unit 33 has a function of exchanging data used in game processing and other data with the wireless communication unit 33 of other game machine devices. As an example, a wireless communication function conforming to the wireless LAN standard of IEEE802.11 is provided. . In this way, the wireless communication unit 33 outputs the received data to the CPU core 21 . Furthermore, the wireless communication unit 33 transmits data instructed by the CPU core 21 to other game machine devices. In addition, by installing protocols such as TCP/IP (Transmission Control Protocol/Internet Protocol) and a prescribed browser in the wireless communication unit 33 and the storage unit in the game machine device 1, the game machine device 1 can, via the wireless communication unit 33, to connect to a network such as the Internet. Therefore, the game machine device 1 can download data such as documents and images published on the Internet, and can browse them on the first LCD 11 and the second LCD 12 .

In addition, the input coordinate processing program of the present invention is not only provided to the computer system through an external storage medium such as the memory card 17, but may also be provided to the computer system through a wired or wireless communication line. In addition, the input coordinate processing program may also be pre-recorded in the non-volatile storage device inside the computer system. In addition, the information storage medium storing the input coordinate processing program is not limited to the aforementioned nonvolatile semiconductor memory, and may be a CD-ROM, DVD, or similar optical disk storage medium.

Next, referring to FIGS. 3 to 12 , before describing the specific processing actions based on the input coordinate processing program executed by the game machine device 1 , the display modes displayed on the first LCD 11 and the second LCD 12 due to the processing actions will be described first. example of. To make the description concrete, an example will be described in which the user is connected to a network such as the Internet via the wireless communication unit 33 and browses data such as documents and images published on the network through the first LCD 11 and the second LCD 12 . Moreover, FIG. 3 is a figure which shows the screen display example of the initial stage which the 1st LCD11 and the 2nd LCD12 display. FIG. 4 is a diagram showing a screen display example displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 on the link displayed on the second LCD 12 is tapped. FIG. 5 is a diagram showing an example of screen displays displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid upward from a link displayed on the second LCD 12 . 6 is a diagram showing an example of screens displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid upwards from the link target displayed on the second LCD 12 and then reciprocated downwards. . 7 is a diagram showing an example of screen displays displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid from the top to the bottom of the mark displayed on the second LCD 12 . 8 is a diagram showing examples of screen displays displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid from the top to the bottom of the mark displayed on the second LCD 12 and then reciprocated upwards. FIG. 9 is a diagram showing a screen display example displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid to surround the partial information displayed on the second LCD 12 . FIG. 10 is a diagram showing a screen display example displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid upward from the background image displayed on the second LCD 12 . FIG. 11 is a diagram for explaining each area set when the touch panel 15 is slid on the background image displayed on the second LCD 12 . 12 is a diagram showing screen display examples displayed on the first LCD 11 and the second LCD 12 when the touch panel 15 is slid upward from the background image displayed on the second LCD 12 and then reciprocated downward.

In FIG. 3 , when information such as documents and images published on the network is browsed by the game machine device 1 via the wireless communication unit 33, as an example of the initial stage, only the currently selected browsing information is displayed on the second LCD 12. information. For example, on the second LCD 12, the home page information selected by the user is displayed. In the example shown in FIG. 3 , as the response area in the homepage information, a text link and an image link indicating a link destination, and a mark to change browsing to other information in the homepage information are displayed. In addition, the icons set for Internet browsing in the game machine device 1 are arranged and displayed in parallel on the left end of the display screen of the second LCD 12 . As one of these icons, an icon for surround mode is set.

In Fig. 4, when the user utilizes the stick 16 to touch the touch panel 15 on the link target displayed on the second LCD 12 (on the image link in Fig. 4), the display centered on the touch point is displayed. Circle M1 (the state of the left diagram in Fig. 4). Thus, when the user releases the stick 16 from the touch panel 15, the homepage information (response image) of the link destination of the touch operation is displayed on the second LCD 12 (the state of the right diagram in FIG. 4 ). In addition, the homepage information displayed on the second LCD 12 is changed to the homepage information of the link destination, but the display state of the first LCD 11 does not change. Hereinafter, the aforementioned operation of momentarily touching the touch panel 15 and then releasing it is referred to as a flick operation.

In Fig. 5, when the user touches the touch panel 15 on the link target displayed on the second LCD 12 (on the text link in Fig. 5), and does not leave the touch panel 15 to slide the bar 16 upwards, An upward arrow M2 is displayed near the current touch operation point (the state of the left figure in FIG. 5 ). Hereinafter, the touch operation on the touch panel 15 without sliding from the touch panel 15 is referred to as a slide operation. Specifically, the screen coordinates corresponding to the touch input coordinates at which the user touches the touch panel 15 first are set as starting point coordinates (x1, y1), and the determination line y=y1+c1 is set in the same screen coordinate system. Here, c1 is a predetermined constant. When the touch panel 15 is slid upward until the area above the determination line is reached (touch input coordinates (xt, yt) in FIG. 5 ), an upward arrow M2 is displayed.

Then, after the user has performed the aforementioned slide operation, the bar 16 is left from the touch panel 15 in the area above the aforementioned determination line, and then on the first LCD 11, the link target (i.e. , the home page information of the link target corresponding to the starting point coordinates (x1, y1). In this way, the second LCD 12 continues to display the homepage information without changing the display (the state of the right figure in FIG. 5 ). That is, the display information of the second LCD 12 still remains, and at the same time, on the display unit (first LCD 11 ) mounted on the user's sliding operation direction (from the second LCD 12 toward the top of the first LCD 11), the information displayed in the initial touch operation is displayed. The information specified in (response image).

In Fig. 6, when the user performs a touch operation on the link target displayed on the second LCD 12 (on the text link in Fig. 6), without leaving the touch panel 15, the user slides upwards on the touch panel 15, and then When a downward reciprocating sliding operation is performed, up and down arrows M3 are displayed near the current touch operation point (the state of the left diagram in FIG. 6 ). Specifically, similar to FIG. 5 , the starting point coordinates (x1, y1) and the determination line y=y1+c1 are set. In this way, on the touch panel 15, the sliding operation is performed upwards until reaching the area above the above-mentioned determination line, and then the sliding operation is performed in the opposite direction until reaching the area below the determination line (touch in FIG. 6 ). Input the coordinates (xt, yt)), and display the up and down direction arrows M3.

After the user has performed the above-mentioned up-and-down sliding operation, in the area below the above-mentioned determination line, after the bar 16 is separated from the touch panel 15, on the second LCD12, the link target (i.e. , the home page information (response image) of the link target corresponding to the starting point coordinates (x1, y1). In this way, the first LCD 11 displays the home page information previously displayed on the second LCD 12 (the state of the right figure in FIG. 6 ). That is, the original display information of the second LCD12 is displayed on the first LCD11, and the information specified in the initial touch operation is displayed on the second LCD12, and according to the user's sliding operation direction (in the configuration direction of the first LCD11 and the second LCD12) reciprocating up and down direction), to alternately display the information displayed on the upper and lower display parts (the first LCD11 and the second LCD12).

In Fig. 7, after the user touches the touch panel 15 on the mark displayed on the second LCD 12 (in Fig. 7, on the mark "xxx"), the mark (that is, corresponding to the starting point coordinates (x1, y1) Other information (response images) in the home page information being browsed corresponding to the mark "xxx") is displayed on the first LCD 11 immediately. In this way, when the operation of sliding the stick 16 downward is performed without leaving the touch panel 15 , the downward arrow M4 is displayed near the current touch operation point (the state of the left diagram in FIG. 7 ). Specifically, the screen coordinates corresponding to the touch input coordinates at which the user first touches the touch panel 15 are set as the starting point coordinates (x1, y1), and the determination line y=y1-c2 is set in the screen coordinate system. Here, c2 is a predetermined constant. When the touch panel 15 is slid downward until the area below the determination line is reached (touch input coordinates (xt, yt) in FIG. 7 ), a down arrow M4 is displayed.

After the user performs the aforementioned sliding operation, when the bar 16 is separated from the touch panel 15 in the area below the aforementioned determination line, the second LCD 12 will display the information displayed by the first LCD 11 corresponding to the mark of the touched operation. state (the state of the right figure in Figure 7). In addition, the home page information displayed on the second LCD 12 is changed to other information corresponding to the mark, but the first LCD 11 continues to display other information corresponding to the mark. That is, the display information of the first LCD 11 still remains, and at the same time, on the display unit (second LCD 12 ) mounted on the user's sliding operation direction (from the first LCD 11 to the downward direction of the second LCD 12), the first touch display is displayed. The information (response image) specified in the operation. In addition, in the right diagram of FIG. 7 , both the first LCD 11 and the second LCD 12 display the information (response image) specified in the first touch operation, but the first LCD 11 may also display the information before the touch operation again. The original information displayed.

In Fig. 8, when the user performs a touch operation on the mark displayed on the second LCD 12 (on the mark "xxx" in Fig. 8), the mark "xxx" corresponding to the mark (that is, the starting point coordinates (x1, y1) ) corresponding to other information (response images) in the browsing homepage information is immediately displayed on the first LCD11. In this way, after the touch panel 15 has been slid downward without leaving the touch panel 15, when the upward and reciprocating sliding operation is performed, the up and down direction arrows M5 are displayed near the current touch operation point (the state of the left figure in FIG. 8 ). ). Specifically, as in FIG. 7 , the starting point coordinates (x1, y1) and the determination line y=y1-c2 are set. After the touch panel 15 is slid downward until reaching the area below the above-mentioned determination line, then the sliding operation is performed in the opposite direction until reaching the area above the determination line (Fig. 8 Touch the input coordinates (xt, yt)), and the up and down arrows M5 are displayed.

After the user has performed the above-mentioned up and down sliding operation, when the stick 16 is separated from the touch panel 15 in the area above the above-mentioned determination line, on the second LCD 12, the corresponding mark corresponding to the touch operation is displayed. information. In this way, the first LCD 11 displays the original home page information previously displayed by the second LCD 12 (the state of the right figure in FIG. 8 ). That is, the original display information in the second LCD12 is displayed on the first LCD11, and the information specified in the initial touch operation is displayed on the second LCD12, and according to the user's sliding operation direction (in the configuration of the second LCD12 and the first LCD11) The upper and lower directions of reciprocating direction) to alternately display the information displayed on the upper and lower display parts (the first LCD11 and the second LCD12).

In FIG. 9 , when the user taps the icon 1a for the surround mode, the game machine device 1 shifts to the surround mode and operates. In the surrounding mode, when the user uses the stick 16 to perform a touch operation on the touch panel 15 by surrounding the partial information displayed on the second LCD 12 (in FIG. 9 , it is the text information recorded in the background image), The locus M6 is displayed based on the touch-operated locus (state of the left figure in FIG. 9 ). Specifically, the screen coordinates corresponding to the touch input coordinates at which the user first touches the touch panel 15 are set as the starting point coordinates (x1, y1), and the starting point coordinates (x1, y1) are set in the same screen coordinate system. y1) is the area near the starting point of the center. When the sliding operation is performed so as to surround the touch panel 15 until reaching the area near the aforementioned start point (in FIG. 9 , touch input coordinates (xt, yt)), the locus M6 is displayed. When the user moves the stick 16 away from the touch panel 15 in the vicinity of the starting point, the information surrounded by the aforementioned locus M6 is enlarged (magnified image) and displayed on the first LCD 11 . The second LCD 12 continues to display the homepage information as it is without changing the display (the state of the right figure in FIG. 9 ). That is, while the display information on the second LCD 12 remains, the information (magnified image) surrounded by the touch operation is displayed on the other display portion (first LCD 11 ).

In Fig. 10, when the user touches the touch panel 15 on the background image displayed on the second LCD 12, does not leave the touch panel 15, and slides upward, the guide image M7 is displayed near the coordinates of the starting point, and Near the current touch operation point, an upward arrow M8 is displayed (the state of the left figure in FIG. 10 ).

Specifically, as shown in FIG. 11 , the screen coordinates corresponding to the touch input coordinates that the user initially touches on the touch panel 15 are taken as the starting point coordinates (x1, y1), and based on this, the coordinates in the same screen coordinate system Set multiple regions in . A predetermined area AM is set around the starting point coordinates (x1, y1). In the screen coordinate system (x, y), the predetermined area AM is an area where x1-k1≦x≦x1+k1 and y1-k2<y<y1+k2. Here, k1 and k2 are predetermined constants, respectively. In this way, the guide image M7 is displayed on the boundary of the predetermined area AM.

Above the predetermined area AM, an upper area AT is set. In the screen coordinate system (x, y), the upper area AT is an area where x1-k1≦x≦x1+k1 and y≧y1+k2. Below the predetermined area AM, a lower area AB is set. In the screen coordinate system (x, y), the lower area AB is an area where x1-k1≦x≦x1+k1 and y≦y1-k2. On the left side of the predetermined area AM, a left area AL is set. In the screen coordinate system (x, y), the left area AL is an area where x<x1-k1. Then, a right area AR is set on the right side of the predetermined area AM. In the screen coordinate system (x, y), the right area AR is an area where x>x1+k1. As shown in FIG. 10, on the touch panel 15, the sliding operation is performed upward until reaching the upper area AT above the predetermined area AM (that is, the area surrounded by the guide image M7) (the touch input coordinates in FIG. 10 (xt, yt)), the up arrow M8 is displayed.

After the user has performed the aforementioned sliding operation, in the upper area AT, when the bar 16 is separated from the touch panel 15, the homepage information (that is, the homepage information specified by the starting point coordinates (x1, y1)) in the middle of browsing by the second LCD12 ) is displayed on the first LCD11. The second LCD 12 does not change the display, and still continues to display the homepage information (the state of the right figure in FIG. 10 ). That is, the display information of the second LCD12 still remains, and meanwhile, the information shown at the starting point of the user's sliding operation direction (from the second LCD12 to the upward direction of the first LCD11) (on the second LCD12 specified in the initial touch operation) The displayed information) is displayed on the display unit (first LCD 11 ) mounted in the sliding operation direction.

In Fig. 12, when the user touches the touch panel 15 on the background image displayed on the second LCD 12, the user slides upwards without leaving the touch panel 15 until reaching the upper area AT, and then slides downwards. When the sliding operation reciprocates up and down until it reaches the specified area AM (the touch input coordinates (xt, yt) in FIG. 12), the guide image M7 is displayed near the starting point coordinates, and the up and down arrows are displayed near the current touch operation point M9 (state in the left figure of Figure 12). The regions set at this time are the same as those described in FIG. 11 .

After the user has performed the above-mentioned sliding operation up and down, in the predetermined area AM, when the stick 16 is separated from the touch panel 15, the homepage information (that is, the starting point coordinates (x1, y1) specified by the second LCD 12 will be browsed. home page information), it is displayed on the first LCD11. The homepage information being browsed by the first LCD 11 is displayed on the second LCD 12 (the state of the right figure in FIG. 12 ). That is, the display information in the second LCD12 is displayed on the first LCD11, and the display information of the first LCD11 is then displayed on the second LCD12. direction) to alternately display the information displayed on the upper and lower display parts (the first LCD11 and the second LCD12).

Next, specific processing operations of the input coordinate processing program executed by the game machine device 1 will be described with reference to FIGS. 13 to 19 . In addition, FIG. 13 is a flowchart showing the operation of causing the game machine device 1 to perform input coordinate processing by executing the input coordinate processing program. FIG. 14 is a subroutine showing the detailed operation of the link specifying process in step 50 in FIG. 13 . FIG. 15 is a subroutine showing the detailed operation of the mark specifying process in step 51 in FIG. 13 . 16 to 18 are subroutines showing the detailed operation of the background specifying process in step 52 in FIG. 13 . FIG. 19 is a diagram showing an example of various data stored in RAM 24 by the processing operation based on FIG. 13 . The program for executing these processes is included in the input coordinate processing program registered in the ROM 17a, and is read from the ROM 17a to the RAM 24 when the power of the game machine 1 is turned on, and executed by the CPU core 21. In addition, in order to make the description concrete, an example of connecting to a network such as the Internet via the wireless communication unit 33, and using the first LCD 11 and the second LCD 12 to view data such as documents and images published on the network is used. Be explained.

First, when the power supply (not shown) of the game machine device 1 is turned on, the CPU core 21 executes the program (not shown), whereby the input coordinate processing program registered in the memory card 17 is uploaded to the RAM 24. . The uploaded input coordinate processing program is executed by the CPU core 21, thereby executing the steps shown in FIG. 13 (abbreviated as "S" in FIGS. 13 to 18).

In FIG. 13 , the CPU core 21 displays information such as documents and images published in the designated web page on at least the second LCD 12 according to the user's operation (step 41; refer to FIG. 3 ). Furthermore, the CPU core 21 sets the touch input flag DFt stored in the RAM 24 to off (step 42), and proceeds to the next step of processing. In addition, in the aforementioned step 42 , the CPU core 21 sets all of the first to twelfth mode flags DF1 to DF12 stored in the RAM 24 to off.

Here, as shown in FIG. 19, the coordinate data input from the touch panel 15 is duly converted into the coordinates on the image displayed on the second LCD 12 corresponding to the contact position where the touch panel 15 is touched, and used as the touch input coordinate DC1 to store in RAM24. In addition, in the RAM 24 , other than the position data DC for generating an image, a starting point coordinate DC2 , a trajectory coordinate DC3 , and the like are appropriately stored. In addition to the touch input flag DFt, the RAM 24 stores first to twelfth mode flags DF1 to DF12 as flag data DF for determining a scheduled process to be executed next time (hereinafter referred to as next process). In addition, as the image data DI for generating an image representing a recognized operation gesture, an index image DI1 , a locus image DI2 , a guide image DI3 , and the like are suitably stored.

Returning to FIG. 13 , the CPU core 21 judges whether there is a touch input from the touch panel 15 corresponding to the user's operation (step 43 ). Then, when there is a touch input, the process proceeds to the next step 44 . On the other hand, if there is no touch input, it is judged whether to end browsing of the currently displayed information (step 53). Next, when the browsing is continued, the CPU core 21 returns to the aforementioned step 43 to repeat the processing, and when the browsing is terminated, the processing based on the program is terminated.

In step 44, the CPU core 21 stores the coordinates on the image displayed on the second LCD 12 corresponding to the contact position of the currently touched touch panel 15 (that is, the current touch input coordinates DC1 ) into the RAM 24 as the starting point coordinates DC2 . Next, the CPU core 21 sets the touch input flag DFt stored in the RAM 24 to ON (step 45), and advances the process to the next step.

Next, the CPU core 21 judges whether the image corresponding to the position indicated by the starting point coordinate DC2 is a link (step 46), whether it is a mark (step 47), or whether it is a background image (step 48). When the image corresponding to the position indicated by the starting point coordinate DC2 is a link such as an image link, a text link (YES in step 46), the CPU core 21 performs link designation processing (step 50), and returns to the aforementioned step 42. Duplicate processing. If the image corresponding to the position indicated by the starting point coordinate DC2 is a mark (YES in step 47), the CPU core 21 performs mark specifying processing (step 51), and returns to the aforementioned step 42 to repeat the process. In addition, when the image corresponding to the position indicated by the starting point coordinate DC2 is a background image (YES in step 48), the CPU core 21 performs the background specifying process (step 52), and returns to the aforementioned step 42 to repeat the process. In addition, when the image corresponding to the position indicated by the starting point coordinate DC2 is not any one of a link, a mark, or a background image (No in steps 46 to 48), the CPU core 21 executes the image corresponding to the position indicated by the starting point coordinate DC2. The processing corresponding to the location (step 49), and returning to the aforementioned step 42 to repeat the processing. Hereinafter, detailed operations of link designation processing, mark designation processing, and background designation processing will be described respectively.

In FIG. 14, when performing the link specifying process in the aforementioned step 50, first, the CPU core 21 sets the first mode flag DF1 stored in the RAM 24 to ON (step 61). Then, the CPU core 21 uses the index image DI1 to display a circle M1 centered on the current touch input coordinate DC1 on the information displayed on the second LCD 12 (step 62; refer to FIG. 4 ), and judges whether the user has interrupted the touch ( Step 63). The CPU core 21 advances the processing to the next step 64 when the user interrupts the touch operation, and advances the processing to step 65 when the touch operation is continued.

In step 64, the CPU core 21 executes the first mode based on the first mode flag DF1 which is currently set to ON, and ends the processing by this subroutine. Here, the so-called first mode refers to the display process described in the right diagram of FIG. 4 , which is the process of displaying the homepage information of the link target of the touch operation on the second LCD 12 without changing the display mode of the first LCD 11 .

In step 65, the CPU core 21 judges whether or not the current touch input coordinates DC1 are in an image region above the decision line including the decision line (hereinafter referred to as the region above the decision line). If the touch input coordinates DC1 are in the area above the determination line, the CPU core 21 advances the processing to the next step 66, and in the case of the area below the determination line, return to the aforementioned step 62 and repeat the process. Here, the CPU core 21 sets the determination line (c1 is a constant) used in step 65 with y=y1+c1 based on the starting point coordinate DC2 (x1, y1). Also, if the current touch input coordinates DC1 (xt, yt) is yt≧y1+c1, the CPU core 21 determines that the current touch input coordinates DC1 is in the region above the determination line.

In step 66, the CPU core 21 turns on the second mode flag DF2 stored in the RAM 24, and turns off the first mode flag DF1 and the third mode flag DF3. Then, the CPU core 21 uses the index image DI1 to display an upward arrow M2 near the current touch input coordinate DC1 on the information displayed on the second LCD 12 (step 67; refer to FIG. 5 ), and judges whether the user interrupted Touch (step 68). The CPU core 21 advances the processing to the next step 69 when the user interrupts the touch operation, and advances the processing to step 70 when the touch operation is continued.

In step 69, the CPU core 21 executes the second mode based on the second mode flag DF2 currently set to ON, and ends the processing by this subroutine. Here, the so-called second mode refers to the processing of displaying as explained in the right diagram of FIG. 5 , which is the processing of displaying the home page information of the link destination on the first LCD 11 without changing the display mode of the second LCD 12 .

In step 70 , the CPU core 21 judges whether the current touch input coordinate DC1 is in the region below the judgment line used in the aforementioned step 65 . When the touch input coordinates DC1 are in the area below the determination line, the CPU core 21 advances the processing to the next step 71 . On the other hand, in the case of the area above the judgment line, if the second mode flag DF2 is set to be connected, then the CPU core 21 returns to the aforementioned step 67 to repeat the process. Determined to be disconnected, then return to the aforementioned step 66 to repeat the process (step 75).

In step 71, the CPU core 21 turns on the third mode flag DF3 stored in the RAM 24 and turns off the second mode flag DF2. The CPU core 21 uses the index image DI1 to display up and down arrows M3 near the current touch input coordinates DC1 on the information displayed on the second LCD 12 (step 72; refer to FIG. 6 ), and judges whether the user has interrupted the touch ( Step 73). The CPU core 21 advances the process to the next step 74 when the user interrupts the touch operation, and returns to the aforementioned step 70 to repeat the process if the touch operation is continued.

In step 74, the CPU core 21 executes the third mode based on the third mode flag DF3 currently set to ON, and ends the processing by this subroutine. Here, the so-called third mode refers to the display process described in the right diagram of FIG. 6 , which is the process of displaying the display information in the second LCD 12 on the first LCD 11 and displaying the home page information of the link target on the second LCD 12 .

Here, in step 71, after the third mode flag DF3 is set to on, and the sliding operation reaches the region above the determination line (that is, No in step 70), step 66 is executed. Therefore, the third mode flag DF3 is set to off, and the second mode flag DF2 is set to on again. That is, the downward sliding operation form using the touch panel 15 from the area above the decision line to the area below the decision line is canceled, and only the upward and downward movement from the starting point coordinate DC2 to the area above the decision line is canceled. The swipe operation of is considered as a valid operation gesture.

In FIG. 15, when performing the flag specifying process in step 51, first, the CPU core 21 sets the fourth mode flag DF4 stored in the RAM 24 to ON (step 81). Next, the CPU core 21 executes the fourth mode based on the fourth mode flag DF4 currently set to ON (step 82 ), and advances the process to the next step. Here, the so-called fourth mode refers to the following processing, that is, directly on the first LCD 11, displaying other information ( For example, refer to the left diagram of FIG. 7 and the left diagram of FIG. 8).

Next, the CPU core 21 judges whether or not the current touch input coordinates DC1 are in an image region below the decision line including the decision line (hereinafter referred to as the region below the decision line) (step 83 ). When the touch input coordinates DC1 are in the area below the determination line, the CPU core 21 advances the processing to the next step 85 . On the other hand, when the touch input coordinates DC1 are in the area above the determination line, the CPU core 21 determines whether or not the user has interrupted the touch (step 84). When the touch operation is continued, the CPU core 21 returns to the aforementioned step 83 to repeat the process, and when the user stops the touch, the process by this subroutine is terminated. Here, the CPU core 21 sets the determination line used in step 83 (c2 is a constant) based on the starting point coordinate DC2 (x1, y1) and y=y1-c2. If the current touch input coordinates DC1 (xt, yt) satisfy yt≦y1−c2, the CPU core 21 determines that the current touch input coordinates DC1 is within the area below the determination line.

In step 85, the CPU core 21 turns on the fifth mode flag DF5 stored in the RAM 24, and turns off the fourth mode flag DF4 and the sixth mode flag DF6. The CPU core 21 uses the index image DI1 to display a downward arrow M4 near the current touch input coordinate DC1 on the information displayed on the second LCD 12 (step 86; refer to FIG. 7 ), and judges whether the user has interrupted the touch ( Step 87). The CPU core 21 advances the processing to the next step 88 when the user interrupts the touch operation, and advances the processing to step 89 when the touch operation is continued.

In step 88, the CPU core 21 executes the fifth mode based on the fifth mode flag DF5 currently set to ON, and ends the processing by this subroutine. Here, the so-called fifth mode refers to the processing of displaying as described in the right diagram of FIG. 7 , which corresponds to the mark of the touch operation and displays other information displayed on the first LCD 11 on the second LCD 12 . In addition, in the fifth mode, both the first LCD 11 and the second LCD 12 display information corresponding to the mark, but the original information displayed before the touch operation may be displayed again on the first LCD 11 .

In step 89 , the CPU core 21 judges whether the current touch input coordinate DC1 is in an area above the judgment line used in step 83 . When the touch input coordinates DC1 are in the area above the determination line, the CPU core 21 advances the processing to the next step 90 . And in the case of the area below the judgment line, if the fifth mode flag DF5 is set to be connected, then the CPU core 21 returns to the aforementioned step 86 to repeat the process, if the fifth mode flag DF5 is set to be turned off , then return to the aforementioned step 85 to repeat the process (step 94).

In step 90, the CPU core 21 turns on the sixth mode flag DF6 stored in the RAM 24 and turns off the fifth mode flag DF5. The CPU core 21 uses the index image DI1 to display up and down arrows M5 near the current touch input coordinates DC1 on the information displayed on the second LCD 12 (step 91; refer to FIG. 8 ), and judges whether the user has interrupted the touch (step 92). The CPU core 21 advances the process to the next step 93 when the user interrupts the touch operation, and returns to the aforementioned step 89 to repeat the process if the touch operation is continued.

In step 93, the CPU core 21 executes the sixth mode based on the sixth mode flag DF6 currently set to ON, and ends the processing by this subroutine. Here, the so-called sixth mode refers to the display process described in the right figure of Fig. 8, which is to display the original display information in the second LCD 12 on the first LCD 11 and correspond to the mark of the touch operation, and display on the second LCD 12 Processing of other information displayed on the first LCD11.

Here, in the aforementioned step 90, after the sixth mode flag DF6 is set to ON, and the sliding operation is performed until the area below the aforementioned determination line is reached (that is, No in the aforementioned step 89), the aforementioned step 85 is executed. Therefore, the sixth mode flag DF6 is set to off, and the fifth mode flag DF5 is set to on again. That is, the upward slide operation gesture using the touch panel 15 from the area below the decision line to the area above the decision line is canceled, and only the downward movement from the starting point coordinate DC2 to the area below the decision line is canceled. The swipe operation of is considered as a valid operation gesture.

In FIG. 16, when performing the background specifying process in the aforementioned step 52, first, the CPU core 21 judges whether or not the surround mode is currently set (step 101). Next, when the mode is not around, the CPU core 21 proceeds to the next step 111 (FIG. 17). On the other hand, in the case of the surround mode, the CPU core 21 proceeds to the next step 102 . Here, the surround mode refers to an operation mode that transitions to when the user performs a sliding operation on the surround mode icon Ia as described with reference to FIG. 9 .

In step 102, the CPU core 21 judges whether the current touch input coordinate DC1 is in the vicinity of the starting point. Next, the CPU core 21 judges whether or not a surrounding curve is formed when the trajectory coordinates DC3 are respectively connected in time series (step 103 ). In the case where the touch input coordinate DC1 is in the vicinity of the starting point, and the group of trajectory coordinates DC3 forms a surrounding curve (the foregoing steps 102 and 103 are both YES; refer to the left figure of FIG. 9 ), the CPU core 21 proceeds the process to the next step 105. And when the touch input coordinate DC1 is not in the region near the starting point, or the trajectory coordinate DC3 group does not form a surrounding curve (the aforementioned steps 102 and 103 are all No), then the CPU core 21 judges whether the user has interrupted the touch (step 104 ). When the user interrupts the touch, the CPU core 21 ends the processing based on the subroutine. On the other hand, when the touch operation is continued, the CPU core 21 additionally stores the current touch input coordinate DC1 as the trajectory coordinate DC3 in the RAM 24 (step 109 ), and returns to the aforementioned step 101 to continue processing.

In step 105, the CPU core 21 sets the seventh mode flag DF7 stored in the RAM 24 to ON. The CPU core 21 utilizes the track image DI2 to display the track M6 (step 106; with reference to FIG. 9 ) on the information displayed on the second LCD 12 along the current track coordinate DC3 (step 106; with reference to FIG. 9 ), and judges whether the user has interrupted the touch (step 107) . The CPU core 21 advances the process to the next step 108 when the user interrupts the touch operation, and returns to the aforementioned step 102 to continue the process if the touch operation is continued.

In step 108, the CPU core 21 executes the seventh mode based on the seventh mode flag DF7 which is currently set to ON, and ends the processing by this subroutine. In addition, the CPU core 21 deletes all trace coordinates DC3 stored in the RAM 24 after executing the seventh mode. Here, the so-called seventh mode refers to the processing of displaying as explained in the right diagram of Fig. 9. It is a process of amplifying the information surrounded by the track M6 and displaying it on the first LCD 11 without changing the display mode of the second LCD 12. .

In FIG. 17 , in the background specifying process in the aforementioned step 52 , and when it is not the aforementioned surround mode (NO in the aforementioned step 101 ), the CPU core 21 executes the step 111 . In step 111, the CPU core 21 judges whether the current touch input coordinates DC1 are in the upper area AT (see FIG. 11 ). When the touch input coordinates DC1 are in the upper area AT, the CPU core 21 advances the processing to the next step 112 . On the other hand, when the touch input coordinates DC1 are not in the upper area AT, the CPU core 21 advances the processing to the next step 116 . Here, the CPU core 21 sets the upper area AT used in step 111 as x1-k1≦x≦x1+k1 based on the starting point coordinates DC2 (x1, y1), and y≧y1+k2 (k1, k2 is constant). If the current touch input coordinates DC1 (xt, yt) are x1-k1≦xt≦x1+k1 and yt≧y1+k2, the CPU core 21 determines that the current touch input coordinates DC1 is within the upper area AT.

In step 112 , the CPU core 21 turns on the eighth mode flag DF8 stored in the RAM 24 and turns off the ninth to twelfth mode flags DF9 to DF12 . The CPU core 21 uses the index image DI1 to display an upward arrow M8 in the vicinity of the current touch input coordinate DC1 on the information displayed on the second LCD 12, and a guide image M7 in the vicinity of the start point coordinate DC2 (step 113; refer to 10), and judge whether the user has interrupted the touch (step 114). The CPU core 21 advances the processing to the next step 115 when the user interrupts the touch operation, and advances the processing to step 116 when the touch operation is continued.

In step 115, the CPU core 21 executes the eighth mode based on the eighth mode flag DF8 currently set to ON, and ends the processing by this subroutine. Here, the so-called eighth mode refers to the display process described in the right diagram of FIG. 10 , which is the process of retaining the display information of the second LCD 12 and displaying the information displayed by the second LCD 1 on the first LCD 11 .

In step 116, the CPU core 21 judges whether or not the current touch input coordinates DC1 are within a predetermined area AM (see FIG. 11 ). When the touch input coordinates DC1 are not within the predetermined area AM, the CPU core 21 advances the processing to the next step 131 ( FIG. 18 ). On the other hand, when the touch input coordinates DC1 are in the predetermined area AM, if the eighth mode flag DF8 is set to ON, the CPU core 21 proceeds to the next step 118, and if the eighth mode flag DF8 is set to OFF, then the eighth mode flag DF8 is set to OFF. , the process proceeds to the next step 131 (step 117). Here, the CPU core 21 sets the predetermined area AM used in step 116 to x1-k1≦x≦x1+k1 based on the starting point coordinate DC2 (x1, y1), and y1-k2<y<y1+k2 (k1, k2 are constants). If the current touch input coordinate DC1 (xt, yt) is x1-k1≦xt≦x1+k1, and y1-k2<yt<y1+k2, then the CPU core 21 determines that the current touch input coordinate DC1 is in the specified area AM.

In step 118 , the CPU core 21 turns on the ninth mode flag DF9 stored in the RAM 24 and turns off the eighth mode flag DF8 . The CPU core 21 uses the index image DI1 to display the up and down arrows M9 in the vicinity of the current touch input coordinate DC1 on the information displayed on the second LCD 12, and display the guide image M7 in the vicinity of the start point coordinate DC2 (step 119; refer to 12), and judge whether the user has interrupted the touch (step 120). The CPU core 21 advances the process to the next step 121 when the user interrupts the touch operation, and advances the process to step 131 ( FIG. 18 ) if the touch operation is continued.

In step 121, the CPU core 21 executes the ninth mode based on the ninth mode flag DF9 currently set to ON, and ends the processing by this subroutine. Here, the so-called ninth mode refers to the display process explained by the right figure in FIG. deal with.

In step 131 of FIG. 18 , the CPU core 21 judges whether the current touch input coordinate DC1 is in the lower area AB (see FIG. 11 ). When the touch input coordinates DC1 are not in the lower area AB, the CPU core 21 advances the processing to the next step 136 . On the other hand, when the touch input coordinates DC1 are in the lower area AB, the CPU core 21 advances the processing to the next step 132 . Here, the CPU core 21 sets the lower area AB used in step 131 as x1-k1≦x≦x1+k1, and y≦y1-k2 (k1, k2 is constant). Thus, if the current touch input coordinates DC1 (xt, yt) are x1-k1≦xt≦x1+k1 and yt≦y1-k2, the CPU core 21 determines that the current touch input coordinates DC1 is in the lower area AB.

In step 132, the CPU core 21 sets the tenth mode flag DF10 stored in the RAM 24 to ON, and sets the eighth, ninth, eleventh, and twelfth mode flags DF8, DF9, DF11, and DF12 to ON. for disconnection. The CPU core 21 uses the index image DI1 to display a downward arrow M10 (not shown) near the current touch input coordinate DC1 on the information displayed on the second LCD 12, and displays a guide image around the starting point coordinate DC2. M7 (refer to FIG. 10 ) (step 133), and judge whether the user has interrupted the touch (step 134). The CPU core 21 advances the processing to the next step 135 when the user interrupts the touch operation, and advances the processing to step 136 when the touch operation is continued.

In step 135, the CPU core 21 executes the tenth mode based on the tenth mode flag DF10 currently set to ON, and ends the processing by this subroutine. Here, the so-called tenth mode refers to a process in which the information displayed on the first LCD 11 is still retained, and the information displayed on the first LCD 11 is also displayed on the second LCD 12 . That is, in the tenth mode, the display information in the first LCD11 still remains, and the information displayed on the starting point side of the user's sliding operation direction (from the first LCD11 toward the bottom of the second LCD12) simultaneously (displayed by the first LCD11 information) is displayed on the display unit (second LCD 12 ) mounted in the sliding operation direction.

In step 136, the CPU core 21 judges whether the current touch input coordinate DC1 is in the left area AL (see FIG. 11 ). When the touch input coordinates DC1 are not in the left area AL, the CPU core 21 advances the processing to the next step 141 . On the other hand, when the touch input coordinates DC1 are in the left area AL, the CPU core 21 advances the processing to the next step 137 . Here, the CPU core 21 sets the left area AL used in step 136 to x<x1-k1 (k1 is a constant) based on the starting point coordinate DC2 (x1, y1). In this way, if the current touch input coordinates DC1 (xt, yt) is xt<x1-k1, the CPU core 21 determines that the current touch input coordinates DC1 is within the left area AL.

In step 137 , the CPU core 21 turns on the eleventh mode flag DF11 stored in the RAM 24 , and turns off the eighth to tenth mode flags DF8 to DF10 and the twelfth mode flag DF12 . The CPU core 21 uses the index image DI1 to display a leftward arrow M11 (not shown) near the current touch input coordinate DC1 on the information displayed on the second LCD 12, and displays a guide image around the starting point coordinate DC2. M7 (refer to FIG. 10) (step 138), and judge whether the user has interrupted the touch (step 139). The CPU core 21 advances the process to the next step 140 when the user interrupts the touch operation, and advances the process to step 141 when the touch operation is continued.

In step 140, the CPU core 21 executes the eleventh mode based on the eleventh mode flag DF11 which is currently set to ON, and ends the processing by this subroutine. Here, the so-called eleventh mode means that according to the user's sliding operation to the left, the display information in the first LCD11 is still retained, and at the same time, the information displayed on the second LCD12 in the previous processing is displayed on the second LCD12, And the processing of information equivalent to the information front page displayed by the second LCD12 at present. For example, in the eleventh mode, among the information displayed on the second LCD 12, the so-called "return" operation of the browser and the operation of displaying the previous page in the electronic book and the like can be performed. In addition, when the second LCD 12 displays an electronic book, in the horizontal version electronic book, the left side of the indicator generally displays the previous page, but it can also be configured according to the document structure of the displayed electronic book (such as vertical version, comics, foreign language documents, etc.) ), sets the command to show the previous page as a swipe to the left.

In step 141, the CPU core 21 judges whether the current touch input coordinates DC1 are in the right area AR (see FIG. 11 ). When the touch input coordinates DC1 are not in the right area AR, the CPU core 21 returns to the aforementioned step 111 ( FIG. 17 ) to continue processing. On the other hand, when the touch input coordinates DC1 are in the right area AR, the CPU core 21 advances the processing to the next step 142 . Here, the CPU core 21 sets the right region AR used in step 141 to x>x1+k1 (k1 is a constant) based on the starting point coordinates DC2 (x1, y1). If the current touch input coordinate DC1 (xt, yt) is xt>x1+k1, the CPU core 21 determines that the current touch input coordinate DC1 is within the right area AR.

In step 142, the CPU core 21 turns on the twelfth mode flag DF12 stored in the RAM 24, and turns off the eighth to eleventh mode flags DF8 to DF11. The CPU core 21 uses the index image DI1 to display a right arrow M12 (not shown) near the current touch input coordinate DC1 on the information displayed on the second LCD 12, and displays a guide image M7 around the starting point coordinate DC2. (Refer to FIG. 10) (step 143), and judge whether the user has interrupted the touch (step 144). The CPU core 21 advances the processing to the next step 145 when the user interrupts the touch operation, and advances the processing to step 111 when the touch operation is continued.

In step 145, the CPU core 21 executes the twelfth mode based on the twelfth mode flag DF12 currently set to ON, and ends the processing by this subroutine. Here, the so-called twelfth mode means that according to the user's sliding operation to the right, the display information in the first LCD11 is still retained, and at the same time as the next processing, the past information displayed by the second LCD12 is displayed on the second LCD12, And the processing of information corresponding to the next page of the information currently displayed on the second LCD12. For example, in the twelfth mode, among the information displayed on the second LCD 12, the so-called "forward" operation of the browser and the display operation of the next page in electronic books and the like can be performed. In addition, when the second LCD 12 displays an electronic book, in the horizontal version of the electronic book, the next page is generally displayed on the right side of the pointer, but it is also possible to set the command to display the previous page according to the document structure of the displayed electronic book. Defined as a swipe action to the right.

Here, when any one of the eighth to twelfth mode flags DF8 to DF12 is set to on when performing background designation processing other than the surround mode, when a slide operation is performed from the currently touched area to another area, it is set to The mode flag set to be on is set to be off, and the other mode flags are set to be on. For example, in the aforementioned step 112, after the eighth mode flag DF8 is set to on, the sliding operation is performed until reaching the left area AL (that is, yes in the aforementioned step 136), the eighth mode flag DF8 is turned on. It is set to off, and the eleventh mode flag DF11 is set to on. In this case, the upward sliding gesture using the touch panel 15 from the starting point coordinate DC2 to the upper area AT is canceled, and only the leftward sliding operation from the starting point coordinate DC2 to the left area AL is canceled. Think of it as a valid manipulation gesture. That is, it is possible to easily cancel the slide operation gesture for performing a predetermined operation, and only the slide operation to another area can be performed, and only the slide operation from the starting point coordinate DC2 to this area can be regarded as a valid operation gesture. .

In addition, although gestures in four directions using the touch panel 15 can be recognized in the configuration described above, multi-directional operation gestures in five or more directions may be recognized. By further subdividing the other areas around the predetermined area AM, gestures in five or more directions using the touch panel 15 can be recognized. In this case, the guide image M7 is displayed in a rectangular pattern as the boundary of the predetermined area AM (refer to FIG. 10 ). Displayed in patterns of other shapes. For example, in the case where not only four-directional gestures using the touch panel 15 but also multi-directional operation gestures can be recognized as described above, polygonal patterns corresponding to the number of directions can be displayed.

Thus, in the game machine device 1, in the third mode of step 74, the sixth mode of step 93, the seventh mode of step 108, the ninth mode of step 121, etc., according to the input corresponding to the touch panel 15, A combination of multiple sliding operation directions (such as up and down directions) is used to recognize the user's gesture and determine the processing to be performed. Therefore, by distinguishing the user's operation gesture by a single sliding operation direction, it is possible to increase the variation of the next processing. On the other hand, the gesture of the sliding operation can be easily canceled, and only the sliding operation to other areas can be performed, and only the sliding operation from the coordinates of the starting point to the area can be regarded as an effective operation gesture. Therefore, when the sliding operation performed by the user is inappropriate, the user can easily cancel the operation and provide a new operation instruction by performing an operation following the operation. That is, according to the present invention, by increasing the change of the next process that can be specified through the coordinate input operation, it is possible to reduce the inconvenience caused when an inappropriate trajectory is drawn, and to improve the operability of the coordinate input. In addition, since the operation gesture recognized by the coordinate input operation input by the user is displayed as an indicator near the touched position, the operation gesture input by the user can be reliably recognized and processed next time.

In addition, in the game machine device 1 , according to the user's slide operation using the touch panel 15 , it is possible to execute highly useful display processing through intuitive operation. For example, when the user uses the touch panel 15 to perform an upward sliding operation, while retaining the display information in the second LCD 12, the images obtained in the processing corresponding to the starting point coordinate DC2 (response image, enlarged image, The image displayed on the second LCD 12 ) can be displayed on the first LCD 11 in the sliding operation direction (from the second LCD 12 toward the top of the first LCD 11 ). In addition, when the user performs a downward sliding operation using the touch panel 15, the image obtained in the processing corresponding to the starting point coordinate DC2 can be displayed in the direction of the sliding operation (from the first LCD 11 toward the bottom of the second LCD 12). on the second LCD12. In addition, when the user uses the touch panel 15 to perform a reciprocating sliding operation in the up and down direction, the image displayed in the second LCD 12 can be displayed on the first LCD 11, and the image (response image) obtained in the processing corresponding to the starting point coordinate DC2 , the image displayed by the first LCD11), can be displayed on the first LCD11, so the upper and lower display parts (the first LCD11 and the second The information displayed in the LCD12) can be alternately displayed. The sliding operation direction input by the user to the touch panel 15 and the corresponding display processing are intuitive. Furthermore, it is useful in various situations to display information corresponding to processing on another display unit while retaining the original display information.

In addition, in the foregoing, for the sake of specific explanation, each input coordinate processing has been described using specific operation sequences, but these are only examples, and needless to say, the present invention is not limited to these operation sequences. For example, a slide operation that can be recognized as an operation gesture on the touch panel 15 may be further added.

In addition, in the foregoing, for the sake of specific description, the example of browsing data such as documents and images published on the Internet on the first LCD 11 and the second LCD 12 has been used for description, but other information and images may also be displayed on the first LCD 11 and the second LCD 12. displayed on the second LCD12. For example, an electronic book such as a book such as a digital dictionary recorded on an electronic medium for viewing by the game machine device 1 may be displayed on the first LCD 11 and the second LCD 12 .

In addition, in the above-mentioned embodiment, as an example of the liquid crystal display unit that divides the screen into two parts, the case where the first LCD 11 and the second LCD 12 that are physically separated are arranged above and below each other (in the case of two screens up and down) has been described. ). However, other configurations may be employed for the configuration of the display screen in which the screen is divided into two. For example, the first LCD 11 and the second LCD 12 may be arranged on the left and right on one main surface of the lower case 13b. In addition, it can also be configured as follows: a vertically-elongated LCD (that is, an LCD that is physically one, but the display length is divided into two screens vertically) with the same horizontal width as the second LCD 12 and twice the vertical length is arranged in the On one main surface of the lower case 13b, the first and second display images are displayed vertically (that is, adjacently displayed without upper and lower boundary portions). In addition, it may also be configured such that a horizontally long LCD having the same vertical width as the second LCD 12 and twice the horizontal length is arranged on one main surface of the lower case 13b, and the first and second LCDs are displayed on the left and right in the horizontal direction. Images are displayed (ie, displayed adjacently without left and right border portions). That is, the first and second display images can be displayed by dividing a substantially one screen into two and using it. For any game image method, if the touch panel 15 is arranged on the display screen of the second display image, the present invention can also be realized. In addition, when the first and second display images are displayed by dividing a substantially one screen into two and using it, the touch panel 15 may be arranged on the entire screen.

In addition, in the foregoing embodiments, the touch panel 15 is provided integrally with the game machine device 1, but needless to say, the present invention can be implemented even if the game machine device and the touch panel are separately configured. In addition, the touch panel 15 may be provided on the upper surface of the 1st LCD11. In addition, in the foregoing embodiment, two display screens (the first LCD 11 and the second LCD 12 ) are provided, but according to the aspect of the present invention, one display screen may be provided. That is, in the foregoing embodiments, the first LCD 11 may not be provided, and the touch panel 15 may be provided using only the second LCD 12 as a display screen. In addition, in the foregoing embodiments, the second LCD 12 may not be provided, but the touch panel 15 may be provided only on the first LCD 11 .

Furthermore, in the foregoing embodiments, as the input means of the game machine device 1, a touch panel is used, but other pointing devices may also be used. Here, the so-called pointing device refers to an input device that specifies the input position and coordinates on the screen. For example, if a mouse, trackpad, trackball, etc. The information of the calculated screen coordinate system can also realize the present invention.

In addition, in the aforementioned embodiment, the touch panel 15 is integrally provided on the game machine device 1, but an information processing device (input coordinate processing device) such as a general personal computer using the touch panel as one of the input means may also be used.

The input coordinate processing program and input coordinate processing device of the present invention can increase user operation gestures that can be recognized by pointing devices, and at the same time can easily cancel the input operation gestures, and can be used to browse documents and images published on the Internet, etc. information or an information processing device such as a game machine device that displays display information such as an electronic document on a display device, and a program executed in the information processing device. In addition, the input coordinate processing program and the input coordinate processing device of the present invention can perform highly useful display processing through intuitive operation according to the user's operation using a pointing device, and can be used to browse documents and documents published on the Internet. Information such as images or information processing devices such as game machine devices that display information such as electronic books on a display device, and programs executed in the information processing devices.

As mentioned above, although this invention was demonstrated in detail, the said description merely exemplifies this invention in every aspect, and does not limit the range. Needless to say, various improvements and modifications can be made without departing from the scope of the present invention.

In this specification, the contents of Japanese Patent Application Nos. Japanese Patent Application No. 2005-109986 and Japanese Patent Application No. 2005-109987 are cited.

Claims (5)

1. input coordinate processing method, it is by operating from the input coordinate of pointing device output according to user's operation, and this input coordinate processing method is characterised in that, comprising:

Input coordinate detects step, and it detects the coordinate information based on the regulation coordinate system according to the input coordinate from aforementioned pointing device output;

The reference coordinate storing step, it is set and the Memory Reference coordinate based on detect the initial coordinate information in detected a series of coordinate informations in the step at aforementioned input coordinate;

Multizone is set step, and it forms a plurality of zones with aforementioned reference coordinate as benchmark, and the zone that will comprise this reference coordinate is set at reference area;

Handle deciding step, it decides processing based on detect last coordinate information and the aforementioned reference coordinate in detected a series of coordinate informations in the step at aforementioned input coordinate,

The aforementioned processing deciding step comprises:

First handles deciding step, when it represents in the zone outside the aforementioned reference area at aforementioned last coordinate information, based on deciding processing from the direction of aforementioned reference coordinate to the zone shown in this last coordinate information;

Second handles deciding step, it detects after part coordinate information in detected a series of coordinate informations in the step represented in the zone outside the aforementioned reference area at aforementioned input coordinate, when aforementioned last coordinate information is represented in the aforementioned reference area, based on from aforementioned reference coordinate to the direction in the zone shown in this part coordinate information, and from the combination of the zone shown in this part coordinate information to the direction of aforementioned reference coordinate, decide processing

Handle in the deciding step aforementioned first, in position that aforementioned last coordinate information is represented after aforementioned reference area moves to other zone, when the position that aforementioned last coordinate information is represented moves to different another other zone with aforementioned reference area and this other zone from this other zone, based on deciding processing to this another other regional direction from aforementioned reference coordinate

Handle in the deciding step aforementioned second, in position that aforementioned last coordinate information is represented after aforementioned reference area moves to other zone, when the position that aforementioned last coordinate information is represented further moves to aforementioned reference area from this other zone, based on from aforementioned reference coordinate to this other regional direction with from of the combination of this other zone to the direction of aforementioned reference coordinate, decide processing.

2. input coordinate processing method according to claim 1 is characterized in that:

The aforementioned processing deciding step also comprises:

Sign is set step, and it is regional when mobile to other from aforementioned reference area in the position shown in the aforementioned last coordinate information, sets and store this other area relative sign;

The first sign step of updating, it has set sign in aforementioned sign setting step after, in the position shown in the aforementioned last coordinate information from the zone that is set with this sign when other zone that is different from aforementioned reference area is moved further, the sign of current setting is updated to and the regional corresponding sign shown in this last coordinate information, and is stored;

The second sign step of updating, it has set sign in aforementioned sign setting step or the aforementioned first sign step of updating after, in the position shown in the aforementioned last coordinate information from the zone that is set with this sign when aforementioned reference area is moved further, the sign of current setting is updated to aforementioned reference area of expression and the reciprocal sign of having set the zone of this sign, and stored, in addition

Aforementioned first handles deciding step, based on set the sign that step or the aforementioned first sign step of updating set by aforementioned sign, decides processing,

Aforementioned second handles deciding step, and the sign based on being set by the aforementioned second sign step of updating decides processing.

3. input coordinate processing method according to claim 1, it is characterized in that, also comprise the demonstration controlled step, this step shows the index of representing following direction respectively in display device, and this direction is to have determined the combinations of directions of handling in the deciding step from handling aforementioned another other regional direction of having determined the deciding step to handle and handle aforementioned second aforementioned first.

4. input coordinate processing method according to claim 1 is characterized in that, also comprises the demonstration controlled step, and this step is shown in the image that aforementioned multizone is set the reference area of setting in the step at the display device indicator gauge.

5. input coordinate processing method according to claim 1 is characterized in that:

Also comprise the demonstration controlled step, this step shows image based on aforementioned coordinate system in display device,

Aforementioned multizone is set step, will be that the image region segmentation of the periphery at center becomes a plurality of with aforementioned reference area, and form aforementioned a plurality of zone.

Images