JP2013218468A - Information processing system, display terminal device, server, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing system in which an arrangement pattern of optionally arranged plural display terminal devices can be recognized in accordance with an easy operation, a screen formed by the arranged plural display terminal devices can be effectively used.SOLUTION: A plurality of display terminal devices transmit, to a server, operation information on a swipe operation performed to the own screen in accordance with an operation performed with a finger sequentially moved with respect to a user display part in the arranged state in a plane direction. On the basis of the operation information received from the plurality of display terminal devices, a server recognizes an arrangement pattern of screens of the plurality of display terminal devices. Then, on the basis of a recognition result of the arrangement pattern, an image to be displayed at each of the plurality of display terminal devices is determined, and a determined image is transmitted to each of the plurality of display terminal devices. The display terminal device causes a received image on a screen.

Description

  The present invention relates to an information processing system, a display terminal device, a server, an information processing method, and a program.

  By arranging a plurality of display devices in a plane direction, one screen composed of the screens of the plurality of display devices is formed, and an image is displayed on the entire formed screen. As an example, by attaching an external display device to an electronic device having a small screen, the screen of the electronic device and the screen of the external display device are arranged side by side to form one screen, and thus configured 1 There is known an electronic device system that displays an image on one screen (see, for example, Patent Document 1).

JP 2004-45913 A

However, in the electronic device system of Patent Document 1, an external display device is physically coupled to the electronic device, so that an image is displayed on a single screen configured by a user freely arranging a plurality of display devices. It is not possible.
For example, an application that displays an image on a single screen configured by arranging a plurality of smartphones is also known. If it is this application, arrangement of a smart phone is arbitrary. However, the user needs to perform an operation for setting, for each smartphone, an image area to be displayed on one screen in accordance with the arranged position, which is very troublesome. In addition, since the user sets the image area to be displayed on each smartphone by operation, the application range is narrow, for example, not suitable for games and educational materials.

  The present invention has been made in view of such circumstances, and a plurality of display terminals arranged such that an arrangement pattern of a plurality of display terminal devices arranged arbitrarily is recognized according to a simple operation. The purpose is to make effective use of the screen formed by the apparatus.

  In order to solve the above-described problem, an information processing system as one embodiment of the present invention includes a plurality of display terminal devices and a server, and the display terminal device arranges the plurality of display terminal devices in a planar direction. The in-screen moving operation performed on the screen of the self-display terminal device in response to the user performing a screen-to-screen moving operation for sequentially moving the operating tool with respect to the screens of the display units of the plurality of display terminal devices. An operation detection unit that detects the content of the detected in-screen movement operation, and an information transmission unit that transmits the generated operation information to the server. An arrangement pattern recognition unit that recognizes the arrangement patterns of the screens of the plurality of display terminal devices based on the operation information received from the display terminal devices.

  The display terminal device according to an aspect of the present invention is a display terminal device in which a plurality of display terminal devices including itself are arranged in a plane direction, and a user places an operation tool on a screen of a display unit of the plurality of display terminal devices. An operation detection unit that detects an in-screen moving operation performed on the screen of the self-display terminal device in response to performing an inter-screen moving operation, and an operation indicating the content of the detected in-screen moving operation An information transmission unit that generates information and transmits the operation information to a server that recognizes an arrangement pattern of screens of the plurality of display terminal devices based on the operation information.

  In addition, the server as one aspect of the present invention performs a screen-to-screen movement operation in which the user sequentially moves the operating tool with respect to the screens of the display units of the plurality of display terminal devices arranged in the plane direction. Each of the plurality of display terminal devices receives operation information indicating the contents of the in-screen movement operation performed on the own screen transmitted by each of the plurality of display terminal devices, and based on the received operation information, the plurality of display terminals An array pattern recognition unit that recognizes the array pattern of the screen of the apparatus is provided.

  An information processing method according to an aspect of the present invention is an information processing method in an information processing system including a plurality of display terminal devices and a server, and the display terminal device moves the plurality of display terminal devices in a planar direction. The self-screen that is performed on the screen of the self-display terminal device in response to the user performing an inter-screen movement operation that sequentially moves the operating body with respect to the screens of the display units of the plurality of display terminal devices in an arrayed state An operation detecting step for detecting an internal movement operation; and an information transmission step for generating operation information indicating the content of the detected in-screen movement operation and transmitting the generated operation information to the server. And an arrangement pattern recognition step of recognizing an arrangement pattern of screens of the plurality of display terminal devices based on the operation information received from the plurality of display terminal devices.

  According to another aspect of the present invention, there is provided an information processing method in which a user places an operating tool on a screen of a display unit of a plurality of display terminal devices in a state where a plurality of display terminal devices including the self are arranged in a plane direction. An operation detecting step for detecting an in-screen moving operation performed on the screen of the self-display terminal device in response to performing an inter-screen moving operation, and an operation indicating the content of the detected in-screen moving operation. An information transmission step of generating information and transmitting the operation information to a server that recognizes an arrangement pattern of screens of the plurality of display terminal devices based on the operation information.

  According to the information processing method as one aspect of the present invention, the user performs an inter-screen movement operation in which the user sequentially moves the operation body on the screens of the display units of the plurality of display terminal devices arranged in the planar direction. Then, each of the plurality of display terminal devices receives operation information indicating the content of the in-screen movement operation performed on the own screen transmitted, and based on the received operation information, the plurality of the plurality of display terminal devices An arrangement pattern recognition step for recognizing the arrangement pattern of the screen of the display terminal device is provided.

  According to another aspect of the present invention, there is provided a program in which a user operates an operating body on a screen of a display unit of a plurality of display terminal devices in a state where a plurality of display terminal devices including the computer are arranged in a plane direction on a computer. An operation detecting step for detecting an in-screen moving operation performed on the screen of the self-display terminal device in response to performing an inter-screen moving operation for sequentially moving the screen, and an operation indicating the content of the detected in-screen moving operation This is for generating information and causing an information transmission step of transmitting the operation information to a server that recognizes an arrangement pattern of the screens of the plurality of display terminal devices based on the operation information.

  In addition, a program as one aspect of the present invention is a program for performing a screen-to-screen moving operation in which a user sequentially moves an operating tool with respect to screens of display units of a plurality of display terminal devices arranged in a plane direction on a computer. In response, operation information indicating the content of the movement operation within the own screen performed on the own screen transmitted by each of the plurality of display terminal devices is received, and the plurality of the plurality of display terminal devices are based on the received operation information. This is to execute an array pattern recognition step for recognizing the array pattern on the screen of the display terminal device.

  According to the present invention, it is possible to recognize an array pattern of a plurality of display terminal devices arranged arbitrarily according to a simple operation, and to effectively use a screen formed by the plurality of display terminal devices arranged. The effect of being obtained.

It is a figure which shows the structural example of the image display system in embodiment of this invention. It is a figure which shows the structural example of the display terminal device and server in this embodiment. It is a figure which shows the example of an arrangement pattern of the display terminal device in the extended display mode of this embodiment, and the example of screen swipe operation. It is a figure which shows the example of the content of the terminal information which an arrangement | sequence pattern recognition part memorize | stores. It is a figure which shows the example of the content of the operation information which an arrangement | sequence pattern recognition part memorize | stores. It is a figure for demonstrating the determination method of the partial image area and its rotation angle by the terminal image determination part. It is a figure which shows the example of a production | generation of the partial image by the image transmission part classified by terminal. It is a figure which shows the example of a production | generation of the image classified by terminal by the image transmission part classified by terminal. It is a figure which shows the example of a display mode of the display target image in the extended screen formed by the arranged display terminal device. It is a figure which shows the example of a process sequence which a display terminal device and a server respond | correspond to extended display mode. It is a figure which shows the outline | summary about the 1st application example of the extended display mode in this embodiment. It is a figure which shows the example of a process sequence which a display terminal device and a server perform corresponding to a 1st application example. It is a figure which shows the outline | summary about the 2nd application example of the extended display mode in this embodiment. It is a figure which shows the example of the arrangement | sequence pattern which can be recognized in this embodiment.

[Image display system configuration]
FIG. 1 shows a configuration example of an image display system (information processing system) in an embodiment of the present invention.
The image display system shown in this figure includes, for example, four display terminal devices 100-1, 100-2, 100-3, 100-4 and a server 200. The display terminal devices 100-1 to 100-4 and the server 200 are connected via a network 300 so that they can communicate with each other.

Hereinafter, the display terminal devices 100-1 to 100-4 will be referred to as the display terminal device 100 unless otherwise distinguished. Further, the screens 111-1 to 111-4 are referred to as screens 111 when not particularly distinguished.
Moreover, although the four display terminal devices 100 are shown in this figure, this is an example to the last, and if it is multiple, there will be no limitation in particular about the number.

The display terminal device 100 includes, for example, a screen 111 on the front surface thereof, and an image is displayed on the screen 111. Further, the screen 111 is provided with a touch panel so that various operations can be performed by touching the screen 111 with an operating body such as a finger.
Note that the operating body for performing operations on the touch panel may be, for example, a stylus pen in addition to a finger.
Moreover, as a specific example of a device in which the display terminal device 100 is embodied, for example, a mobile phone having an information processing function called a smartphone, an information processing device having a display unit with a touch panel called a tablet terminal, or the like. .

  In the image display system of the present embodiment, the display terminal devices 100-1 to 100-4 are arranged in an arbitrary positional relationship in the planar direction with the screens 111-1 to 111-4 facing the front surface. Thereby, one screen (extended screen) is formed by the screens 111-1 to 111-4 of the display terminal devices 100-1 to 100-4. The user performs a swipe operation (inter-screen swipe (move) operation) so as to sequentially move the extended screen formed in this way while touching the screen 111 with a finger. Each of the display terminal devices 100-1 to 100-4 displays the contents of the swipe operation (swipe (move) operation within the own screen) performed on its own screen 111-1 to 111-4 in accordance with the swipe operation between the screens. The operation information shown is transmitted to the server 200.

The server 200 recognizes the arrangement pattern of the display terminal devices 100-1 to 100-4 based on the received operation information. That is, the server 200 recognizes in what positional relationship the display terminal devices 100-1 to 100-4 are arranged in correspondence with the image display direction.
Further, the server 200 determines which one of the display terminal devices 100-1 to 100-4 in the display target image to be displayed on the extended screen based on the recognition result of the arrangement pattern of the display terminal devices 100-1 to 100-4. Decide whether to display the image area portion. Further, the rotation angle of the image region portion is determined according to the arrangement direction of the display terminal devices 100-1 to 100-4. And based on these determination results, the server 200 generates a terminal-specific image, which is an image to be displayed for each of the display terminal devices 100-1 to 100-4, from the display target image. It transmits to each of the display terminal devices 100-1 to 100-4.

  The display terminal devices 100-1 to 100-4 display the received terminal-specific images on the screens 111-1 to 111-4, respectively. Accordingly, for example, one image is enlarged and displayed on the extended screen formed by the screens 111-1 to 111-4.

[Configuration of display terminal and server]
FIG. 2 shows a configuration example of the display terminal device 100 and the server 200. In this figure, the network 300 between the display terminal device 100 and the server 200 is not shown.

  First, the display terminal device 100 will be described. The display terminal device 100 includes a display unit 110, a touch panel 120, an information transmission unit 130, a time measurement unit 140, a display control unit 150, a communication unit 160, and a storage unit 170.

  The display unit 110 displays an image according to the control of the display control unit 150. A screen 111 provided on the front surface of the display terminal device 100 is a screen portion in the display unit 110. As the display unit 110, for example, a liquid crystal display device or the like is employed.

The touch panel (operation detection unit) 120 is an operation input device arranged on the screen 111 of the display unit 110, and detects a position where a finger (operation body) comes into contact. By combining the touch panel 120 with the display unit 110 in this manner, the user can perform an operation on the image displayed on the screen 111.
Then, the touch panel 120 detects the in-screen swipe operation performed on the screen 111 in response to the user performing the inter-screen swipe operation when the extended display mode is set in the display terminal device 100.
The extended display mode is a mode that is set when the screen 111 of the display terminal device 100 functions as a part of the enlarged screen, as described with reference to FIG. It is set in response to an operation for setting the mode for 100-4.

The information transmission unit 130 generates operation information indicating the content of the in-screen swipe operation detected by the touch panel 120 and transmits the operation information to the server 200. Further, the terminal information 180 is transmitted in response to the setting of the extended display mode. The terminal information 180 is stored in the storage unit 170 and includes predetermined information about the display terminal device 100.
In addition, when transmitting operation information or terminal information, the information transmission unit 130 delivers the operation information to the communication unit 160 and instructs transmission. In response to this instruction, the communication unit 160 transmits the passed operation information to the server 200 via the network 300.

  The time measuring unit 140 measures time. In addition, in order to make the time measurement part 140 measure time, the server 200 transmits reference time with respect to each display terminal device 100-1 to 100-4. The reference time is a time used as a reference when the time measuring unit 140 measures time, and is the time measured in the server 200. Each time measurement part 140 of the display terminal devices 100-1 to 100-4 starts time measurement at the timing when the reference time is received. Thereby, the time which each time measurement part 140 of the display terminal devices 100-1 to 100-4 measures (synchronizes).

  The display control unit 150 controls the display unit 110 to display an image. In addition, when the extended display mode is set, the display control unit 150 receives the terminal-specific image transmitted from the server 200 in response to the transmission of the operation information, and the received terminal-specific image is displayed on the screen 111. The display unit 110 is controlled in such a manner.

The communication unit 160 communicates with the server 200 via the network 300.
The storage unit 170 is a part that stores various types of information, and terminal information 180 is shown as one of the stored information.

Next, the configuration of the server 200 will be described.
The server 200 includes an arrangement pattern recognition unit 210, a synchronization control unit 220, a terminal-specific image determination unit 230, a terminal-specific image transmission unit 240, a storage unit 250, and a communication unit 260.

  The operation information transmitted from the display terminal devices 100-1 to 100-4 is received by the communication unit 260. The array pattern recognition unit 210 recognizes the array pattern of the screens 111 of the plurality of display terminal devices 100 based on the operation information received from the plurality of display terminal devices 100 as described above.

  The synchronization control unit 220 transmits the reference time to the plurality of display terminal devices 100. The synchronization control unit 220 measures (clocks) the reference time. Then, in response to the reception of the terminal information from the display terminal device 100 as described later, the reference time measured at that time is transmitted to the display terminal device 100 that is the transmission source of the terminal information.

  The terminal-specific image determination unit 230, based on the recognized arrangement pattern of the screens 111 of the plurality of display terminal devices 100, the partial image regions to be displayed on each of the screens 111 in the display target image to be displayed on the extended screen, and these The rotation angle for each partial image area is determined.

  The terminal-specific image transmission unit 240 generates a terminal-specific image obtained by rotating each of the partial images corresponding to the partial image region generated based on the display target image by the rotation angle, and the generated terminal-specific images are Transmit to each of the display terminal devices.

The storage unit 250 stores image data used as a display target image. The image data may be prepared and stored in advance by a person who operates the server 200, for example, or uploaded from the display terminal device 100 or another terminal device via the network 300, for example. Also good.
The communication unit 260 communicates with each of the display terminal devices 100 via the network 300.

[Extended display mode]
Subsequently, an operation example of the user of the display terminal device 100 and an operation example of the display terminal device 100 and the server 200 in the extended display mode of the present embodiment will be described.
FIG. 3 shows an example of an aspect in which the user arranges the display terminal devices 100-1 to 100-4 for image display in the extended display mode. In this case, each of the display terminal devices 100-1 to 100-4 is turned sideways. In addition, the display terminal device 100-2 is arranged on the upper left, and the display terminal device 100-4 is arranged on the right side thereof. Further, the display terminal device 100-3 is arranged directly below the display terminal device 100-2, and further, the display terminal device 100-3 is positioned immediately below the display terminal device 100-4 and on the right side of the display terminal device 100-3. 100-1 is arranged.
That is, in this case, the display terminal devices 100-1 to 100-4, which are each in the horizontal direction, are arranged in a 2 × 2 state. And as these positional relationships, the display terminal device 100-2 is the upper left, the display terminal device 100-4 is the upper right, the display terminal device 100-3 is the lower left, and the display terminal device 100-1 is the lower right.

Further, the number of pixels of the screens 111-1 to 111-4 of the display terminal devices 100-1 to 100-4 is 480 × 800, respectively.
The pixel coordinates ps of the display terminal device 100-2 arranged at the upper left are ps (0,800) at the upper right vertex and ps (480 at the lower left vertex) with respect to the origin coordinate ps (0, 0) of the upper left vertex. , 0), and the lower right vertex is ps (480, 800). That is, in the display terminal device 100-2, the upper left vertex on the screen 111-2 in the landscape state is the origin coordinate ps (0, 0), and the downward direction from the origin coordinate ps (0, 0) is the x direction. The rightward direction is the y direction. About this point, the display terminal device 100-3 arrange | positioned at the lower left is also the same.

  The pixel coordinates ps of the display terminal device 100-4 arranged at the upper right and the display terminal device 100-1 arranged at the lower right are respectively the origin coordinates ps (0, 0) at the lower right vertex. The upper left vertex is ps (480, 800), the upper right vertex is ps (480, 0), and the lower left vertex is ps (0, 800).

That is, in FIG. 3, the display terminal devices 100-1 to 100-4 are all in the same horizontal direction, but are the display terminal devices 100-2 and 100-3 and the display terminal devices 100-4 and 100-1. The state in which the vertical direction is opposite is shown.
This means that in the present embodiment, the user does not need to align the top and bottom of the screen 111 when arranging the plurality of display terminal devices 100. That is, the user can arrange a plurality of display terminal devices 100 without being particularly aware of the upper and lower sides of the screen 111.

The user sets the extended display mode by performing a predetermined operation on each of the display terminal devices 100-1 to 100-4 arranged as shown in FIG. Note that the extended display mode can be easily set by a predetermined operation on the touch panel 120, for example.
In response to the setting of the extended display mode as described above, the information transmission unit 130 of the display terminal devices 100-1 to 100-4 reads the terminal information 180 from the storage unit 170 and transmits it to the server 200 from the communication unit 160. To do. Thereby, the communication unit 260 of the server 200 receives the terminal information 180 transmitted from each of the display terminal devices 100-1 to 100-4. The array pattern recognition unit 210 stores the received terminal information of the display terminal devices 100-1 to 100-4.

FIG. 4 shows an example of the contents of terminal information stored in the array pattern recognition unit 210.
As shown in this figure, the terminal information stored by the array pattern recognition unit 210 has a structure in which a screen size and a corresponding image format are associated with each terminal identifier. For example, the contents of one line indicated by broken lines in this figure correspond to the terminal information 180 transmitted by the display terminal device 100-1.
The terminal identifier is an identifier that uniquely identifies the display terminal device 100 that is the transmission source of the terminal information 180. In this figure, the terminal identifier “0x01” indicates the display terminal device 100-1. The terminal identifier “0x02” indicates the display terminal device 100-2. The terminal identifier “0x03” indicates the display terminal device 100-3. The terminal identifier “0x04” indicates the display terminal device 100-4.
The terminal identifier is included in the terminal information 180 transmitted by the display terminal device 100, for example. Alternatively, the terminal identifier may be generated and assigned by the array pattern recognition unit 210 in response to reception of the terminal information 180.
The screen size indicates the size (resolution) of the screen 111 of the display terminal device 100. FIG. 4 shows an example in which each of the display terminal devices 100-1 to 100-4 has a screen size of 480 × 800.
The compatible image format indicates an image format that can be displayed in correspondence with the display terminal device 100.

In response to the terminal information 180 being transmitted from the display terminal device 100 as described above, the synchronization control unit 220 sets the reference time counted at that time to the display terminal device 100 that is the transmission source of the terminal information 180. On the other hand, it is transmitted from the communication unit 260.
This reference time is received by the communication unit 160 of the display terminal device 100. In the display terminal device 100, the time measuring unit 140 measures time using the received reference time. That is, at the timing when the reference time is received, the value of the time measured by itself is reset to the reference time, and thereafter, the measurement is performed so that the value is incremented at regular intervals. When the time measuring units 140 of the display terminal devices 100-1 to 100-4 execute such measurement, the times measured by the display terminal devices 100-1 to 100-4 are synchronized.

  The user transmits and receives the terminal information 180 and the reference time between the display terminal devices 100-1 to 100-4 and the server 200 as described above according to the setting of the extended display mode. Perform a swipe operation.

The inter-screen swipe operation refers to touching the screen 111 of the display terminal device 100 arranged at a predetermined operation start array position with a finger, and thereafter screens 111 of all the remaining display terminal devices 100 arranged. This is an operation of moving the finger so as to sequentially pass through.
In the present embodiment, the operation start arrangement position is determined to be the upper left of the arranged display terminal devices 100. Therefore, the user performs an inter-screen swipe operation, for example, as shown as an operation locus by a broken line in FIG.
That is, first, the user touches his / her finger on the screen 111-2 of the upper left display terminal device 100-2, and then moves the touched finger to the right to display the screen of the display terminal device 100-4. To reach above 111-4. When the finger reaches the screen 111-4, the user moves the finger downward to reach the screen 111-1 of the display terminal device 100-1. Furthermore, from this state, the user moves his / her finger to the left to reach the screen 111-3 of the display terminal device 100-3.
By the movement of the finger so far, the swipe operation has been performed on the screens 111 of all the display terminal devices 100 arranged. Therefore, the user stops moving the finger on the screen 111-3 of the display terminal device 100-3, and further releases the finger from the screen 111-3 to end the inter-screen swipe operation.

In response to the inter-screen swipe operation, on each of the screens 111-1 to 111-4 of the display terminal devices 100-1 to 100-4, as shown in FIG. 3, a certain trajectory is obtained from a certain operation start position coordinate ps_st. A self-screen swipe operation is performed to trace and end at the operation end position coordinate ps_ed.
The touch panels 120 of the display terminal devices 100-1 to 100-4 respectively detect coordinates corresponding to the trajectory of the in-screen swipe operation.

Therefore, each of the information transmission units 130 of the display terminal devices 100-1 to 100-4 inputs the values of the operation start position coordinate ps_st and the operation end position coordinate ps_ed of the in-screen swipe operation from the detection output of the touch panel 120. .
Each of the information transmission units 130 inputs the time (operation start time) when the operation start position coordinate ps_st is detected, that is, the time when the in-screen swipe operation is started from the time measurement unit 140.
Each of the information transmission units 130 inputs the time (operation end time) when the operation end position coordinate ps_ed is detected, that is, the time when the in-screen swipe operation is ended from the time measurement unit 140.
Each of the information transmitting units 130 generates operation information including the operation start position coordinates ps_st, the operation start time, the operation end position coordinates ps_ed, and the operation end time input as described above. Is transmitted from the communication unit 160 to the server 200.

  Thereby, the communication part 260 in the server 200 receives the operation information transmitted from each of the display terminal devices 100-1 to 100-4. The array pattern recognition unit 210 stores the received operation information.

FIG. 5 shows a specific example of the contents of the operation information stored in the array pattern recognition unit 210. The operation information has a structure in which operation start information and operation end information are associated with each terminal identifier. The operation start information includes an operation start position and an operation start time. The operation end information includes an operation end position and an operation end time.
In this figure, one line associated with the terminal identifier “0x01” corresponds to the operation information transmitted by the display terminal device 100-1. Similarly, each row associated with the terminal identifiers “0x02” to “0x04” corresponds to operation information transmitted by the display terminal devices 100-2 to 100-2, respectively.

The content of the operation information in FIG. 5 indicates that an inter-screen swipe operation has been performed as follows.
That is, in the operation information of FIG. 5, the terminal identifier corresponding to the earliest operation start time is 0x02.
That is, in order to start the inter-screen swipe operation, as shown in FIG. 3, the finger is touched on the operation start position coordinates ps_st (250, 200) on the screen 111-2 of the display terminal device 100-2. The time at this time is the operation start time “158” associated with the terminal identifier “0x02” in FIG.

  Thereafter, the finger touched on the screen 111-2 is moved to the right, so that the in-screen swipe operation is ended at the operation end position coordinates ps_ed (270, 800) on the screen 111-2. The time at this time is the operation end time “161” associated with the terminal identifier “0x02” in FIG.

  Next, as the finger is moved further to the right as it is, the in-screen swipe operation is started from the operation start position coordinates ps_st (260, 800) on the screen 111-4 of the display terminal device 100-4. The time at this time is the operation start time “163” associated with the terminal identifier “0x04” in FIG.

  Thereafter, the finger touched on the screen 111-4 is moved so as to change from the left direction to the lower direction in accordance with the trajectory shown in FIG. 3, so that the operation end position coordinates ps_ed (0 , 400), the self-screen swipe operation is terminated. The time at this time is the operation end time “165” associated with the terminal identifier “0x04” in FIG.

  Next, when the finger is moved further down, the finger moves to the screen 111-1 of the display terminal device 100-1, and the in-screen swipe operation is started from the operation start position coordinates ps_st (480, 400). It was. The time at this time is the operation start time “168” associated with the terminal identifier “0x01” in FIG.

  Thereafter, the finger touched on the screen 111-1 is moved while changing from the downward direction to the left direction according to the trajectory shown in FIG. 3, so that the operation end position coordinates ps_ed (240, 800), the in-screen swipe operation is terminated. The time at this time is the operation end time “172” associated with the terminal identifier “0x01” in FIG.

  Next, when the finger is moved further leftward, the finger moves to the screen 111-3 of the display terminal device 100-3, and the in-screen swipe operation is started from the operation start position coordinates ps_st (250, 800). It was. The time at this time is the operation start time “173” associated with the terminal identifier “0x03” in FIG.

  Thereafter, the finger touched on the screen 111-3 is further moved to the left according to the trajectory shown in FIG. 3, and the operation end time “175” is set at the operation end position coordinates ps_ed (240, 200) on the same screen 111-3. The finger is released at the timing “” and the in-screen swipe operation on the screen 111-3 is finished. In addition, since there is no operation information of the remaining display terminal devices 100 thereafter, it is indicated that the inter-screen swipe operation is also ended.

  First, the array pattern recognition unit 210 recognizes that four display terminal devices 100-1 to 100-4 are used for the extended display mode from the contents of the terminal information shown in FIG. Next, by referring to the operation start time and the operation end time in the operation information shown in FIG. 5, it is possible to recognize the order of the display terminal devices 100 that have been swiped by the inter-screen swipe operation. Then, according to the recognized order, the coordinates of the operation end position corresponding to one operation end time and the coordinates of the operation start position corresponding to the operation start time following this operation end time are in accordance with the movement of the finger. It is virtually arranged to pass the same axis in the direction. By executing this processing in the order in which the in-screen swipe operation is performed, for example, the array pattern recognition unit 210 arranges the display terminal devices 100-1 to 100-4 according to the array pattern shown in FIG. Recognize that

Depending on the above processing, the relative positional relationship among the screens 111 of the plurality of display terminal devices 100 is recognized. Only by this relative positional relationship, the display direction of the extended screen formed by the screens 111 of the arranged display terminal devices 100 is not specified.
However, in the present embodiment, as described above, it is defined that the inter-screen swipe operation should be started from the one arranged at the upper left in the arranged display terminal devices 100. This corresponds to designating that the screen 111 of the display terminal device 100 in which the inter-screen swipe operation has been started is arranged at the upper left in the screen 111 forming the enlarged screen.
As described above, in this embodiment, the screen 111-2 of the display terminal device 100-2 in which the inter-screen swipe operation is started is designated as being arranged at the reference position, so that an image is displayed on the extended screen. The direction is also specified. In addition, as the screen 111 at the reference arrangement position on the extended screen is designated as described above, it is also recognized whether each screen 111 in the arrangement pattern is in the portrait orientation or the landscape orientation.
Then, if the user simply starts an inter-screen swipe operation from the display terminal device 100 arranged at a predetermined arrangement position, the display direction can be designated accordingly. In other words, in the present embodiment, it is not necessary for the user to perform a step-by-step operation for specifying the display direction.

Contrary to the broken-line arrows shown in FIG. 3, the same arrangement as described above may be performed even if the screen swipe operation is performed in the order of the display terminal devices 100-2, 100-3, 100-1, and 100-4. A pattern recognition result is obtained.
Further, the reference arrangement arrangement position for starting the inter-screen swipe operation is not limited to the upper left. For example, the lowermost left, the uppermost right, and the lowermost right may be used.

  When the arrangement pattern of the screens 111 of the plurality of display terminal devices 100 is recognized as described above, the terminal-specific image determination unit 230 uses the recognition result of the arrangement pattern to each of the screens 111 as follows. Determine the image to be displayed. At this stage, the arrangement pattern recognition unit 210 recognizes that the arrangement patterns of the screens 111-1 to 111-4 of the display terminal devices 100-1 to 100-4 shown in FIG. is there.

  FIG. 6 shows a display target image 400 to be displayed on the extended screen. The size of the display target image 400 is 768 × 1024. In the display target image 400 in this case, for example, the pixel coordinate of the upper left vertex is px (0, 0), the pixel coordinate of the upper right vertex is px (0, 1024), and the pixel coordinate of the lower left vertex is px (768). , 0), and the pixel coordinates of the lower right vertex are px (768, 1024).

  The terminal-specific image determination unit 230 applies, to the display target image 400, an extended screen formed by the array patterns of the screens 111-1 to 111-4 in FIG. 3 that are the recognition results of the array pattern recognition unit 210. Match. As a result of this collation, the terminal-specific image determination unit 230 recognizes which partial image area of the display target image 400 corresponds to each of the screens 111-1 to 111-4.

  For this purpose, for example, the terminal-specific image determination unit 230 recognizes the correspondence between the coordinates of the four vertices of each of the screens 111-1 to 111-4 forming the extended screen and the coordinates in the display target image 400. In the case of FIG. 6, the terminal-specific image determination unit 230 obtains the following recognition result.

  First, the coordinates ps (0,0), ps (0,800), ps (480,0), and ps (480,800) of the four vertices of the screen 111-2 arranged at the upper left of the extended screen are respectively , Corresponding to the pixel coordinates px (0, 0), px (0, 512), px (384, 0), px (384, 512) of the display target image 400. That is, the screen 111-2 corresponds to the upper left image region portion of the image region portion obtained by dividing the display target image 400 into two rows and two columns.

  The coordinates ps (480,800), ps (480,0), ps (0,800), and ps (0,0) of the four vertices of the screen 111-4 arranged at the upper right of the extended screen are respectively , Corresponding to pixel coordinates px (0, 513), px (0, 1024), px (384, 513), px (384, 1024) of the display target image 400. That is, the screen 111-4 corresponds to the upper right image region portion of the image region portion obtained by dividing the display target image 400 into two rows and two columns.

  The coordinates ps (0,0), ps (0,800), ps (480,0), and ps (480,800) of the four vertices of the screen 111-3 arranged at the lower left of the extended screen are respectively , Corresponding to the pixel coordinates px (385, 0), px (385, 512), px (768, 0), px (768, 512) of the display target image 400. That is, the screen 111-3 corresponds to the lower left image region portion of the image region portion obtained by dividing the display target image 400 into two rows and two columns.

  Further, the coordinates ps (480, 800), ps (480, 0), ps (0, 800), and ps (0, 0) of the four vertices of the screen 111-1 arranged at the lower right of the extended screen are: These correspond to the pixel coordinates px (385, 513), px (385, 1024), px (768, 513), and px (768, 1024) of the display target image 400, respectively. That is, the screen 111-1 corresponds to the lower right image region portion of the image region portion obtained by dividing the display target image 400 into two rows and two columns.

  Then, the terminal-specific image determination unit 230 converts the image portion of the display target image 400 having the coordinates recognized for each of the screens 111-1 to 111-4 as described above, for each of the screens 111-1 to 111-4. Are determined to be partial image areas 410a to 410d to be displayed.

  The terminal-specific image determination unit 230 also determines a rotation angle when displaying the partial image area on the screen 111. For this purpose, for example, the terminal-specific image determination unit 230 recognizes the coordinate direction of the arranged screen 111 and recognizes how many rotation angles the coordinate direction is relative to the standard.

Specifically, in the case of the screen 111-2 in FIG. 6, since the coordinates of the upper left vertex are ps (0, 0) and the coordinates of the lower left vertex are ps (480, 0), the direction of the x coordinate is downward. . In the present embodiment, the downward direction of the x-coordinate direction is a reference direction when displaying an image in a horizontal state.
Therefore, the partial image area 410a corresponding to the screen 111-2 may be displayed on the screen 111-2 without giving a rotation angle. That is, the terminal-specific image determination unit 230 determines that the rotation angle of the partial image region 410a is 0 degree.

On the other hand, in the case of the screen 111-4 of FIG. 6, since the coordinates of the upper left vertex are ps (480, 800) and the coordinates of the lower left vertex are ps (0, 800), the direction of the x coordinate is upward. In the present embodiment, the direction in which the x-coordinate direction is upward is a direction in which a rotation angle of 180 degrees is given with respect to a reference for displaying an image in a landscape state.
When the partial image area 410b is displayed as it is on the screen 111-4 in this direction, the image portion displayed on the screen 111-4 is inverted 180 degrees on the extended screen. Therefore, the terminal-specific image determination unit 230 determines that the rotation angle of the partial image region 410b is 180 degrees.
Similarly, the terminal-specific image determination unit 230 in FIG. 6 determines that the rotation angle of the partial image region 410c corresponding to the screen 111-1 is 180 degrees, and the partial image region 410d corresponding to the screen 111-3. Is determined to be 0 degree.

  The terminal-specific image transmission unit 240 generates individual partial images 411 a to 411 d corresponding to the partial image areas 410 a to 410 d determined by the terminal-specific image determination unit 230. For this purpose, for example, the terminal-specific image transmission unit 240 cuts out an area portion corresponding to the partial image areas 410 a to 410 d from the display target image 400.

  FIG. 7 shows partial images 411a to 411d generated by the terminal-specific image transmission unit 240. Further, in the figure, an example in which the partial images 411a to 411d are generated as image data having a resolution of 1/4 of the display target image 400 is shown. That is, in the partial images 411a to 411d, the coordinates of each vertex are the same in px (0,0), px (0,512), px (384,512), and px (384,0), respectively, in the x direction. The resolution in the y direction is half that of the display target image 400.

Next, the terminal-specific image transmission unit 240 rotates the partial images 411a to 411d generated as illustrated in FIG. 7 in accordance with the rotation angle determined by the terminal-specific image determination unit 230, whereby the terminal illustrated in FIG. Separate images 412a to 412d are generated.
That is, since the rotation angles of the partial images 411a and 411d shown in FIG. 7 are determined to be 0 degrees, these images become the terminal-specific images 412a and 412d as they are.
On the other hand, since the rotation angles of the partial images 411b and 411c are determined to be 180 degrees, the terminal-specific images 412b and 412c are generated by rotating the partial images 411b and 411c by 180 degrees, respectively. . As described above, the terminal-specific image transmission unit 240 generates a terminal-specific image by, for example, extracting an image from the display target image 400 and rotating the extracted image.

  Then, the terminal-specific image transmission unit 240 transmits the terminal-specific image 412a to the display terminal device 100-2. Similarly, the terminal-specific image transmission unit 240 transmits the terminal-specific image 412b to the display terminal apparatus 100-4, transmits the terminal-specific image 412c to the display terminal apparatus 100-1, and displays the terminal-specific image 412d. -3.

  The communication units 160 in the display terminal devices 100-1 to 100-4 receive the terminal-specific images 412c, 412a, 412d, and 412b, respectively. Then, the display control unit 150 in each of the display terminal devices 100-1 to 100-4 displays the received terminal-specific images 412c, 412a, 412d, and 412b on the display unit 110, respectively.

  FIG. 9 shows images displayed on the screens 111-1 to 111-4 of the display terminal devices 100-1 to 100-4 at this time. In the arrangement of the display terminal devices 100-1 to 100-4 in this case, the top and bottom of the display terminal devices 100-4 and 100-1 are reversed, but the terminal-specific image 412b displayed on them. Since 412c is inverted 180 degrees from the original state, it is in the correct orientation for the displayed state.

In this way, on the extended screen formed by the screens 111-1 to 111-4 of the display terminal devices 100-1 to 100-4, for example, an image having the same content as the display target image 400 illustrated in FIG. 6 is displayed. The
Then, when displaying an image on the extended screen as shown in FIG. 9, the user performs troublesome operations such as image enlargement, trimming, and movement for setting the image to be displayed on the screen 111 for each display terminal device 100. There is no need to do this. Specifically, a simple operation for setting the extended display mode is performed for each of the display terminal devices 100-1 to 100-4, and the display terminal device 100 is arranged to form an extended screen. Just swipe between screens.
In addition, the server 200 according to the present embodiment recognizes the arrangement pattern of the screens 111 based on the operation information detected for each screen 111 by the screen-to-screen swipe operation. Accordingly, the orientation of each screen 111 is also changed. It has come to be recognized. Then, the terminal-specific image to be displayed is rotated according to the recognized orientation. Thereby, when arranging the display terminal devices 100, the user does not need to be particularly aware of the orientation, for example, by aligning the orientation. This point also greatly contributes to improvement in operability and convenience when displaying an image on the extended screen.
Further, in the present embodiment, since the image to be displayed on the display terminal device 100 is transmitted from the server 200 side in the extended display mode, for example, common image data used for display between the display terminal devices 100 is used. There is no need to store them in advance.

The flowchart of FIG. 10 shows an example of a processing procedure executed by the display terminal device 100 and the server 200 in correspondence with the extended display mode. As can be understood from the above description, a plurality of display terminal devices 100 are used in the extended screen mode, but here, for the sake of simplicity of description, one display terminal device 100 is used. Indicates processing.
In the following description, the partial image areas are described as partial image areas 410 unless they are particularly distinguished. Similarly, when the individual images are not particularly distinguished from each other, they are described as partial images 411. When the individual images are not particularly distinguished from each other, they are described as terminal-specific images 412.

  First, in the display terminal device 100, the information transmission unit 130 waits for an operation for setting the extended display mode on the touch panel 120 (step S101—NO). Then, in response to the operation for setting the extended display mode being performed (step S101—YES), the information transmission unit 130 reads the terminal information 180 from the storage unit 170 and the communication unit 160 to the server 200. Transmit (step S102).

  Next, the time measuring unit 140 receives the reference time transmitted by the server 200 in response to the transmission of the terminal information 180 in step S102 (step S103), and measures the time from the reference time at the timing of receiving this reference time. Is started (step S104).

Next, the information transmission unit 130 waits for a swipe operation within the own screen to be performed by monitoring operation information on the touch panel 120 (step S105—NO). Then, in response to the in-screen swipe operation being performed, the information transmitting unit 130 first acquires, as the operation start position, the operation position detected on the touch panel 120 when the in-screen swipe operation is started. At the same time, the time measured at that time is acquired as the operation start time (step S106). Next, the information transmitting unit 130 acquires the operation position detected on the touch panel 120 when the in-screen swipe operation is ended as the operation end position, and the time measured at that time is the operation end time. (Step S107).
Then, the information transmission unit 130 transmits the operation start position, operation start time, operation end position, and operation end time acquired in steps S106 and S107 as operation information from the communication unit 160 to the server 200 (step S108).

  Next, the display control unit 150 waits for reception of a terminal-specific image transmitted from the server 200 as a transmission destination (step S109—NO). Then, in response to receiving the terminal-specific image (step S109-YES), the display control unit 150 displays the terminal-specific image on the screen 111 of the display unit 110 (step S110).

  In the server 200, the array pattern recognition unit 210 is on standby for reception of the terminal information transmitted in step S102 (step S201-NO). And according to receiving terminal information (step S201-YES), arrangement pattern recognition part 210 memorizes received terminal information (step S202). In addition, at the timing according to the reception of the terminal information, the synchronization control unit 220 transmits the reference time measured at that time from the communication unit 260 to the display terminal device 100 that is the transmission source of the terminal information (Step S203). ).

  Next, the array pattern recognizing unit 210 waits for operation information from all the display terminal devices 100 corresponding to the plurality of terminal information stored in step S202 to be received (step S204—NO). When operation information from all the display terminal devices 100 is received (step S204—YES), the array pattern recognition unit 210 recognizes the array patterns on the screens 111 of the plurality of display terminal devices 100 (step S205). . For this purpose, as described above, the array pattern recognition unit 210 performs one operation based on the operation start position, operation start time, operation end position, and operation end time included in the operation information from each display terminal device 100. The positional relationship between other screens adjacent to the screen is specified.

  Next, as described with reference to FIG. 6, the terminal-specific image determination unit 230 is a part to be displayed for each display terminal device 100 based on the recognition result of the arrangement pattern of the screens 111 of the plurality of display terminal devices 100. The rotation angle of the image area 410 and each partial image area 410 is determined (step S206).

Next, the terminal-specific image transmission unit 240 generates the terminal-specific image 412 for each display terminal device 100 according to the determination result of the terminal-specific image determination unit 230 (step S207). In other words, the terminal-specific image transmission unit 240 corresponds to the partial image region 410 determined by the terminal-specific image determination unit 230 and corresponds to each display terminal device 100 based on the display target image 400 as described with reference to FIG. A partial image 411 to be generated is generated. Next, the terminal-specific image transmission unit 240 rotates the partial image 411 generated as described above by the determined rotation angle. In this way, the terminal-specific image 412 is generated.
Then, the terminal-specific image transmission unit 240 transmits the terminal-specific image 412 generated as described above from the communication unit 260 to each display terminal device 100 (step S208).

[First application example]
Hereinafter, application examples of the extended display mode of the present embodiment will be described. First, the first application example will be described.
FIG. 11 shows an overview of the first application example. In the first application example, the extended display mode of the present embodiment is applied to a game in which a correct kanji is created by combining the deviation and kanji of kanji.
FIG. 11A shows three display terminal devices 100-1 to 100-3 used for the game. On the screens 111-1 to 111-3 of these display terminal devices 100-1 to 100-3, images of predetermined kanji bias or wrinkles (problem images) are displayed, respectively. Specifically, a problem image is displayed on the screen 111-1 of the display terminal device 100-1. On the screen 111-2 of the display terminal device 100-2, a problem image of the most important thing is displayed. On the screen 111-3 of the display terminal device 100-3, for example, a habit problem image used for “green” or “record” is displayed. These problem images are transmitted from the server 200.

  The problem presented in this case is that the Chinese character “green” is created using the display terminal devices 100-1 to 100-3. In order to answer this problem, for example, the user arranges the display terminal device 100-2 on the left side as shown in FIG. 11B and arranges the display terminal device 100-3 on the right side thereof. Suppose that In addition, as in the previous embodiment, the user performs an inter-screen swipe operation starting from the display terminal device 100 arranged at the upper left in the array as a declaration of an answer to the question. That is, in this case, the screen 111-2 of the display terminal device 100-2 is touched on the left side, and the screen swipe operation is performed so that the screen 111-3 of the right display terminal device 100-3 is moved from this state. I do.

  The display terminal device 100-2 transmits to the server 200 operation information regarding the in-screen swipe operation performed on the screen 111-2 by the inter-screen swipe operation. Similarly, the display terminal device 100-3 transmits, to the server 200, operation information regarding the in-screen swipe operation performed on the screen 111-3 by the inter-screen swipe operation.

  Based on the operation information, the server 200 recognizes that the display terminal devices 100-2 and 100-3 are arranged according to the arrangement pattern shown in FIG. Here, since the server 200 stores which question image is transmitted to the display terminal devices 100-1 to 100-3, it is understood that the array pattern shown in FIG. 11B is correct. Yes.

In this case, the server 200 determines that the arrangement pattern recognized based on the operation information is correct. Therefore, the server 200 selects an answer image having a content for notifying a correct answer as a display target image. In addition, the server 200 determines an image to be displayed on the display terminal devices 100-2 and 100-3 based on the display target image, and for each display terminal device 100-2 and 100-3 according to the determination result. A corresponding terminal-specific image is generated. Then, the server 200 transmits the generated terminal-specific images to the display terminal devices 100-2 and 100-3.
The display terminal devices 100-2 and 100-3 each display the received terminal-specific image. As a result, as shown in FIG. 11C, the answer image for notifying that the extended screen formed by the screens 111-2 and 111-3 of the display terminal devices 100-2 and 100-3 is correct. Is displayed.

On the other hand, after the user arranges the display terminal device 100-1 on the left side and the display terminal device 100-3 on the right side as shown in FIG. Assume that the same swipe operation between screens as in the case is performed.
In the server 200 that has received the operation information corresponding to this operation from the display terminal devices 100-1 and 100-3, the display terminal devices 100-1 and 100-3 are arranged according to the arrangement pattern shown in FIG. Recognize that. Further, the server 200 determines that the recognized array pattern is different from the correct array pattern.
Therefore, the server 200 selects an image having contents to notify that the answer is incorrect as a display target image, and determines an image to be displayed on the display terminal devices 100-1 and 100-3 based on the display target image. Then, a terminal-specific image is generated. Then, the server 200 transmits the generated terminal-specific images to the display terminal devices 100-1 and 100-3.
The display terminal devices 100-1 and 100-3 each display the received terminal-specific image. As a result, as shown in FIG. 11 (e), the answer image for notifying that the answer is incorrect in the extended screen formed by the screens 111-1 and 111-3 of the display terminal devices 100-1 and 100-3. Is displayed.
Here, the game content is that the user selects and arranges the kanji bias and 偏, but the game content includes other things such as associating a picture such as an animal with a word representing the animal. Is also conceivable.
As described above, the extended display mode of the present embodiment can be applied to a game in which a user selects and arranges a plurality of images having a correspondence relationship, such as picture matching.

FIG. 12 shows an example of a processing procedure executed by the display terminal device 100 and the server 200 corresponding to the first application example described with reference to FIG. The configurations of the display terminal device 100 and the server 200 in the first application example may be the same as those in FIG.
First, the information transmission unit 130 in the display terminal device 100 waits for an operation (game use notification operation) for notifying the touch panel 120 that the display terminal device 100 is used for a game to the server 200. (Step S301-NO). This game use notification operation corresponds to the operation for setting the extended display mode in the previous embodiment. Then, in response to the game use notification operation being performed (step S301—YES), the information transmission unit 130 reads the terminal information 180 from the storage unit 170 and transmits the terminal information 180 to the server 200 from the communication unit 160 (step S302). ).

  Next, the time measuring unit 140 receives the reference time transmitted by the server 200 (step S303), and starts measuring time (step S304).

  In this case, the server 200 transmits the problem image in response to the transmission of the terminal information in step S302. Therefore, the display control unit 150 displays the received problem image on the screen 111 of the display unit 110 (step S305).

The subsequent processes from step S306 to S311 are the same as steps S105 to S108 in FIG.
As described above, the in-screen swipe operation waiting in step S306 is performed by the user performing an inter-screen swipe operation as an answer, as shown in FIG. 11B or FIG. This is performed on the screen 111 as it is performed.
Further, the terminal-specific image received corresponding to step S310 is a part of the answer image, and a part of the answer image is displayed on the screen 111 depending on step S311.

  As processing executed by the server 200, the processing from step S401 to S403 is the same as the processing from step S201 to S203 in FIG. In addition, the terminal-specific image transmission unit 240 in the server 200 transmits a different problem image to each of the display terminal devices 100 that is the transmission source of the terminal information (step S404). At this time, which problem image is to be transmitted to which display terminal device 100 may be determined at random using, for example, a random number. The problem image data is stored in advance in the storage unit 250, for example.

  In addition, the array pattern recognition unit 210 determines the array pattern of the display terminal device 100 corresponding to the correct answer (correct answer array pattern) based on the correspondence between the problem image transmitted in step S404 and the display terminal device 100 that is the transmission destination. ) Then, the array pattern recognition unit 210 stores the recognized correct array pattern (step S405).

In response to the inter-screen swipe operation performed as an answer as described with reference to FIG. 11B or 11D, the display terminal device 100 performs the intra-screen operation performed on itself. Sends swipe operation information.
The array pattern recognizing unit 210 waits for this operation information to be received (step S406—NO), and in response to the operation information being received (step S406—YES), based on this operation information. The arrangement pattern of the display terminal device 100 is recognized (step S407). Then, the array pattern recognition unit 210 determines whether the recognized array pattern is correct or incorrect depending on whether the recognized array pattern matches the correct array pattern stored in step S405 (step S408).

  In this case, the terminal-specific image determination unit 230 selects an answer image corresponding to the determination result in step S408 from the answer images stored in the storage unit 250 (step S409). That is, when it is determined that the answer is correct in step S408, the answer image that notifies the correct answer is selected, and when it is determined that the answer is incorrect, the answer image that notifies the incorrect answer is selected.

The terminal-specific image transmission unit 240 determines the partial image region and the rotation angle for each display terminal device 100 using the answer image selected in step S409 as a display target image (step S410).
According to the determination result, the terminal-specific image transmission unit 240 generates a terminal-specific image corresponding to each display terminal device 100 (step S411), and transmits the generated terminal-specific image to each display terminal device 100 ( Step S412).

[Second application example]
FIG. 13 shows an overview of a second application example of the extended display mode of the present embodiment. The second application example uses the image display in the extended display mode of the present embodiment as, for example, a coupon ticket presented to a store.
In this case, the user is registered as a member of a service such as a coupon site, and accordingly, the storage unit 250 of the server 200 stores the name, gender, and terminal identifier of the display terminal device 100 owned by the user as the member. Such information is registered (stored).
In addition, the store that issues the coupon registers (stores) the coupon image corresponding to the coupon and the condition for acquiring the coupon in the storage unit 250 of the server 200.

  Here, it is assumed that the condition for acquiring a coupon that can be used in a certain store is that two men and women couples have visited the store. Two users of a couple of men and women who want to obtain coupons at this store visit the store and move the two display terminal devices 100-1 and 100-2 to the left and right, for example, as shown in FIG. It arranges by the arrangement pattern arranged in order. In addition, similarly to the description so far, the user performs, for example, an inter-screen swipe operation from the screen 111 of the display terminal device 100 located in the upper left corner (in this case, the screen 111-1 of the display terminal device 100-1). Do. In response to this operation, the display terminal devices 100-1 and 100-2 each transmit operation information corresponding to the in-screen swipe operation to the server 200.

  In response to receiving the operation information from the display terminal devices 100-1 and 100-2, the array pattern recognition unit 210 of the server 200 first displays the display terminal devices 100-1 and 100-2 that are the transmission sources of the operation information. However, it authenticates whether the pre-registered coupon acquisition conditions are satisfied. That is, in this case, since the coupon acquisition condition is a couple of men and women, the array pattern recognition unit 210 determines whether each owner of the display terminal devices 100-1 and 100-2 has a combination of men and women. Determine whether or not.

For this purpose, the array pattern recognition unit 210 refers to the gender in the registration information of the user associated with the identifiers of the transmission source display terminal devices 100-1 and 100-2 included in the operation information. If the sex in the registered information of the referred user is different between male and female, the coupon acquisition condition is satisfied. In this case, the authentication is established when the coupon acquisition condition is satisfied.
In response to the successful authentication, the array pattern recognition unit 210 displays the display terminal devices 100-1 and 100- based on the contents of the operation information transmitted from the display terminal devices 100-1 and 100-2. It is recognized that the arrangement pattern 2 is, for example, that shown in FIG.

In response to the recognition of the arrangement pattern as described above, the terminal-specific image determination unit 230 reads the coupon image registered in the storage unit 250 as a display target image. Then, the terminal-specific image determination unit 230 determines a partial image area corresponding to the arrangement pattern of the display terminal devices 100-1 and 100-2 recognized by the arrangement pattern recognition unit 210, and a rotation angle for each partial image area.
Next, according to the determination result of the terminal-specific image determination unit 230, the terminal-specific image transmission unit 240 generates a terminal-specific image for each of the display terminal devices 100-1 and 100-2 based on the coupon image. It transmits with respect to the display terminal devices 100-1 and 100-2.

The display terminal devices 100-1 and 100-2 display the received terminal-specific images on the screen 111 of the display unit 110. Thereby, as shown in FIG.13 (b), a coupon image is displayed in the enlarged screen formed by the screens 111-1 and 111-2 of the display terminal devices 100-1 and 100-2.
The male and female users show the clerk the coupon image displayed as shown in FIG. When the clerk confirms the coupon image, the clerk provides a service corresponding to the coupon.

[Other sequence pattern examples]
FIG. 14 shows another arrangement pattern example of the screen 111 that can be recognized in the present embodiment.
First, in FIG. 14A, the display terminal device 100-1 is arranged in the vertical direction on the left side, and the display terminal device 100-2 is arranged in the vertical direction on the right side. In addition, a horizontal display terminal device 100-3 is arranged along the lower end face of the display terminal devices 100-1 and 100-2.
Even in such an arrangement pattern, for example, the user may perform an inter-screen swipe operation as indicated by a dashed arrow in the figure, starting from the screen 111 of the upper left display terminal device 100-1 in this arrangement, for example. . Based on the operation information transmitted from the display terminal devices 100-1 to 100-3 in response to this operation, the array pattern recognition unit 210 of the server 200 can recognize the array pattern shown in this figure.

Moreover, in this figure, the sizes of the screens 111-1, 111-2, and 111-3 are different from each other. However, in this embodiment, the terminal information includes screen size information, and the operation start position and operation end position of the in-screen swipe operation are expressed by coordinates on the screen 111. Thereby, even if the size of the screen 111 is different, the array pattern recognition unit 210 can recognize the array pattern without any problem.
In FIG. 14A, the arrangement pattern is recognized in the same manner even when the screen swipe operation is performed in the order of the display terminal devices 100-1, 100-3, and 110-2, contrary to the broken arrow. The

  Moreover, in this embodiment, as shown in FIG.14 (b), the state which the end surface which contact | connects between the display terminal devices 100-1, 100-2, 100-3 has shifted | deviated mutually is shown. Even in such a state, the array pattern recognition unit 210 is based on operation information transmitted by each of the display terminal devices 100-1 to 100-3 in response to, for example, a swipe operation between screens performed as indicated by a dashed arrow. The array pattern can be recognized without problems.

Note that various application examples of the extended display mode of the present embodiment other than those shown in FIGS. 11 to 13 can be considered.
In the above embodiment, the terminal-specific image rotated on the server 200 side is transmitted to the display terminal device 100. However, for example, the server 200 transmits the terminal-specific image that is not rotated together with the rotation angle instruction information, and displays the terminal-specific image received on the display terminal device 100 side by rotating according to the rotation angle instruction information. It is good.
Moreover, in the said embodiment, the example in which the display terminal device 100 and the server 200 are connected via a network is shown. However, for example, after a plurality of display terminal devices 100 are connected by ad hoc communication, any one of these display terminal devices 100 may have a function equivalent to that of the server 200.

  2 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed. The extended display may be performed by doing so. Here, the “computer system” includes an OS and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

DESCRIPTION OF SYMBOLS 100 Display terminal device 110 Display part 111 Screen 120 Touch panel 130 Information transmission part 140 Time measurement part 150 Display control part 160 Communication part 170 Storage part 180 Terminal information 200 Server 210 Array pattern recognition part 220 Synchronization control part 230 Image determination part 240 according to terminal Terminal-specific image transmission unit 250 Storage unit 260 Communication unit 300 Network 400 Display target image 410, 410a to 410d Partial image region 411, 411a to 411d Partial image 412, 412a to 412d Terminal-specific image

Claims (12)

A plurality of display terminal devices and servers,
The display terminal device
The self-display terminal in response to the user performing a screen-to-screen movement operation for sequentially moving the operating body with respect to the screens of the display units of the plurality of display terminal devices in a state where the plurality of display terminal devices are arranged in a plane direction An operation detection unit that detects an in-screen movement operation performed on the screen of the device;
An operation transmission unit that generates operation information indicating the content of the detected movement operation within the own screen, and an information transmission unit that transmits the generated operation information to the server;
The server
Based on the operation information received from the plurality of display terminal devices, an array pattern recognition unit that recognizes an array pattern of screens of the plurality of display terminal devices,
An information processing system characterized by this. The information transmitting unit
As the contents of the movement operation within the own screen, at least an operation start position and an operation start time at which the movement operation within the own screen is started, and an operation end position and an operation end time at which the movement operation within the own screen is completed. Generating the operation information including,
The array pattern recognition unit includes:
Recognizing an arrangement pattern of screens of the plurality of display terminal devices based on the operation start position and operation start time included in the operation information, and the operation end position and the operation end time;
The information processing system according to claim 1. The display terminal device
It further comprises a time measuring unit that measures time synchronized with other display terminal devices,
The information transmitting unit
The time measured when the movement operation within the own screen detected by the operation detection unit is started is set as the operation start time, and the movement operation within the own screen detected by the operation detection unit is completed. The time that has been measured as the operation end time,
The information processing system according to claim 2. The server
A synchronization control unit that transmits a reference time, which is a reference time for the time measurement unit to measure the time, to the plurality of display terminal devices;
The time measuring unit is
Start measuring the time using the reference time received from the server;
The information processing system according to claim 3. The server
Based on the recognized arrangement pattern, a partial image area to be displayed on each of the display target images to be displayed on an extended screen composed of the screens of the plurality of display terminal devices, and a rotation angle for each of the partial image areas. A terminal-specific image determination unit to be determined;
A terminal-specific image is generated by rotating each of the partial images corresponding to the partial image region generated based on the display target image by the rotation angle, and the generated terminal-specific images are respectively displayed in the plurality of display terminal devices. And a terminal-specific image transmission unit for transmitting to
The display terminal device
A display control unit configured to receive a terminal-specific image transmitted from the server in response to the transmission of the operation information, and to control the display unit so that the received terminal-specific image is displayed on the screen;
The information processing system according to claim 1, wherein the information processing system is an information processing system. In response to the user performing an inter-screen movement operation that sequentially moves the operating body with respect to the screens of the display units of the plurality of display terminal devices in a state where the plurality of display terminal devices including the self are arranged in a plane direction. An operation detection unit that detects a movement operation within the own screen performed on the screen of the display terminal device;
An information transmission unit that generates operation information indicating the content of the detected movement operation within the own screen, and transmits the operation information to a server that recognizes an arrangement pattern of screens of the plurality of display terminal devices based on the operation information; ,
A display terminal device comprising: Self transmitted by each of the plurality of display terminal devices in response to the user performing an inter-screen movement operation for sequentially moving the operating body on the screens of the display units of the plurality of display terminal devices arranged in the plane direction. An array pattern recognition that receives operation information indicating the contents of the in-screen moving operation performed on the screen of the display screen and recognizes an array pattern of the screens of the plurality of display terminal devices based on the received operation information Comprising a part,
A server characterized by that. An information processing method in an information processing system including a plurality of display terminal devices and a server,
The display terminal device
The self-display terminal in response to the user performing a screen-to-screen movement operation for sequentially moving the operating body with respect to the screens of the display units of the plurality of display terminal devices in a state where the plurality of display terminal devices are arranged in a plane direction. An operation detecting step for detecting a moving operation within the own screen performed on the screen of the device;
An information transmission step of generating operation information indicating the content of the detected in-screen movement operation and transmitting the generated operation information to the server;
The server
An arrangement pattern recognition step for recognizing an arrangement pattern of screens of the plurality of display terminal devices based on the operation information received from the plurality of display terminal devices;
An information processing method characterized by the above. In response to the user performing an inter-screen movement operation that sequentially moves the operating body with respect to the screens of the display units of the plurality of display terminal devices in a state where the plurality of display terminal devices including the self are arranged in a plane direction. An operation detecting step for detecting a moving operation within the own screen performed on the screen of the display terminal device;
An information transmission step of generating operation information indicating the content of the detected movement operation within the own screen and transmitting the operation information to a server that recognizes an arrangement pattern of screens of the plurality of display terminal devices based on the operation information; ,
An information processing method comprising: Self transmitted by each of the plurality of display terminal devices in response to the user performing an inter-screen movement operation for sequentially moving the operating body on the screens of the display units of the plurality of display terminal devices arranged in the plane direction. An array pattern recognition that receives operation information indicating the contents of the in-screen moving operation performed on the screen of the display screen and recognizes an array pattern of the screens of the plurality of display terminal devices based on the received operation information Comprising steps,
An information processing method characterized by the above. On the computer,
In response to the user performing an inter-screen movement operation that sequentially moves the operating body with respect to the screens of the display units of the plurality of display terminal devices in a state where the plurality of display terminal devices including the self are arranged in a plane direction. An operation detecting step for detecting a moving operation within the own screen performed on the screen of the display terminal device;
An information transmission step of generating operation information indicating the content of the detected movement operation within the own screen and transmitting the operation information to a server that recognizes an arrangement pattern of the screens of the plurality of display terminal devices based on the operation information;
A program for running On the computer,
Self transmitted by each of the plurality of display terminal devices in response to the user performing an inter-screen movement operation for sequentially moving the operating body on the screens of the display units of the plurality of display terminal devices arranged in the plane direction. An array pattern recognition that receives operation information indicating the contents of the in-screen moving operation performed on the screen of the display screen and recognizes an array pattern of the screens of the plurality of display terminal devices based on the received operation information Step,
A program for running

Images