JP2024053778A - Program and information processing device - Google Patents

Abstract

[Problem] To provide a program and an information processing device that can improve convenience while sharing the screens of multiple devices and enabling switching with simple operations. [Solution] Outputs information for enabling another information processing device to specify an image that can be displayed in response to a first operation among predetermined images, receives information for enabling an image that can be displayed on a display unit of the other information processing device in response to a first operation of the other information processing device, switches the image to be displayed on the display unit in response to a second operation to one of a plurality of types of images including a first image among the predetermined images corresponding to the first operation on an input unit of the information processing device and a second image specified from the received information, outputs and receives switching information specifying the switching, and when the other information processing device receives its own switching information that has switched to the first image of the information processing device, displays the other specified image corresponding to the other information processing device. [Selected Figure] Figure 16

Description

The present invention relates to a program and an information processing device.

Technology for sharing the screen of one terminal device with other terminal devices among multiple terminal devices has been known for some time. Specifically, there is a technique for performing a screen sharing operation on one terminal device (hereinafter referred to as terminal device A) and performing an approval operation on another terminal device (hereinafter referred to as terminal device B), thereby displaying and sharing the screen of terminal device A on terminal device B (for example, FIG. 3 of Patent Document 1).

JP 2017-175638 A

However, in conventional technology, when terminal device B shares the screen of terminal device A and displays the screen of terminal device A, for example, to display an image of its own device that was displayed on terminal device B before sharing, there is a risk that an operation to cancel sharing may be required, and a sharing operation may be required again to display the screen of terminal device A again on terminal device B, resulting in cumbersome operations. Furthermore, when a screen that was displayed on terminal device B is displayed on terminal device A and shared, an operation to cancel sharing from terminal device A to terminal device B, a screen sharing operation from terminal device B, and an approval operation on terminal device A are required. In other words, a sharing operation is required each time sharing is performed. For this reason, when screens are shared in both directions between multiple terminal devices, there is a risk that operations may become cumbersome.

Furthermore, for example, a user who owns terminal device B cannot cause an image of terminal device B itself to be displayed or an image of terminal device A to be displayed at the user's will and timing.Furthermore, a user who owns terminal device A cannot cause an image of terminal device A itself to be displayed or an image of terminal device B to be displayed at the user's will and timing.

Therefore, there is a demand for multiple devices to share their screens and be able to switch between them with simple operations. In this case, while other users can freely view the images displayed on the user's terminal device, the user cannot know which other users are viewing the images. This can lead to problems such as not being able to take appropriate action for the other users viewing the images (for example, displaying an image suitable for other users, or displaying an image that the user does not want other users to view), and this could result in a failure to improve convenience.

The present invention was conceived in light of this situation, and its purpose is to provide a program and an information processing device that can improve convenience while allowing multiple devices to share their screens and switch between them with simple operations.

(1) A program (e.g., a program included in a web page, an installed program, etc.) executed on an information processing device (e.g., a user terminal) having a processor, a memory, an input unit, and a display unit,
The information processing device includes:
A step of outputting self-image information for enabling another information processing device to identify an image that can be displayed on the display unit in response to a first operation (e.g., a scroll operation) on the input unit (e.g., step S14);
receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device (e.g., step S15);
execute a step (e.g., steps S24 to S30) of switching an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from other image information received in the receiving step, in response to a second operation (e.g., a switching operation) on the input unit;
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different (see (8) in the modified example);
The information processing device includes:
a step of outputting switching information for enabling the other information processing device to specify that the image displayed on the display unit of the information processing device has been switched to the first image of the other information processing device by switching to the second image in the switching step in response to a second operation on the input unit (e.g., step S25c of FIG. 14);
By receiving self-switching information specifying that the image to be displayed on the display unit of the other information processing device has been switched to a first image in the information processing device as switching information from the other information processing device, a step of displaying another specific image corresponding to the other information processing device that has switched to the first image in the information processing device (e.g., steps S40 and S41 in Figure 15) is executed.

According to this configuration, the image displayed on the other information processing device can be viewed at the timing of the user who owns the information processing device, and smooth communication with users who own the other information processing devices can be achieved. In addition, switching information is output and received, and when the switching information is received, another specific image corresponding to the other information processing device that has switched to the first image on the information processing device is displayed, so that the other information processing device can be identified. As a result, appropriate measures can be taken for other users who are viewing the first image on the information processing device, improving convenience and facilitating communication.

(2) In the above (1), the step of displaying the other specific image displays the other specific image corresponding to the other information processing device by receiving the self-switching information, regardless of whether the first image or the second image is displayed on the display unit of the information processing device (for example, regardless of the display mode that is set, steps S40 and S41 in FIG. 15 are executed, FIG. 16).

According to this configuration, it is possible to identify other information processing devices that have switched to the first image on the information processing device, regardless of the image displayed on the information processing device. As a result, even when the first image is not being displayed on the information processing device, it is possible to identify other information processing devices that have switched to the first image, and to take appropriate action for other users viewing the first image on the information processing device, such as by switching to the first image on the information processing device.

(3) In the above (1), the step of displaying the other specific image does not display the other specific image corresponding to the other information processing device when the first image is not displayed on the display unit of the information processing device even when the self-switching information is received, and displays the other specific image corresponding to the other information processing device when the self-switching information is received and the first image is displayed on the display unit of the information processing device (for example, in user terminal X, a process is performed to identify and manage other users who are viewing the self-terminal image of user X based on setting mode information received regardless of the set display mode, and when the display mode of user terminal X is the self-terminal mode, a user icon of the identified other user (a user viewing the self-terminal image of user X) is displayed, and a user icon of another user who has newly come to view the image is displayed).

With this configuration, when a first image is displayed on an information processing device, another specific image is displayed, making it easy to intuitively understand which other information processing device has switched to the first image.

(4) In any of (1) to (3) above, the other specific image is an image that includes user information for identifying a user who operates the other information processing device.

This configuration makes it possible to identify users who are operating (using/viewing) other information processing devices that have switched to the first image.

(5) In any of (1) to (3) above, the step of displaying the other specific image terminates the display of the other specific image corresponding to the other information processing device when it is determined, based on switching information from the other information processing device, that the image displayed on the display unit of the other information processing device has been switched from a state in which the image was switched to a first image in the information processing device to a different image.

This configuration allows the other information processing device, which had switched to the first image, to know that it has switched to another image, and prevents the other specific image from continuing to be displayed even after it has switched to the other image.

(6) In the above (5), the step of displaying the other specific image includes, when starting to display the other specific image, moving the other specific image from a predetermined position on the edge of the display area of the display unit of the information processing device toward the inside of the predetermined area, and, when ending to display the other specific image, moving the other specific image from the predetermined area toward the predetermined position (e.g., dotted arrows shown in Figures 16 (B) and (C)).

With this configuration, the movement of the other specific image allows the user to intuitively and easily understand that the display of the first image on the information processing device will begin or end on the other information processing device.

(7) In the above (5), the step of displaying the other specific image includes displaying the other specific image when it is determined from the self-switching information that the image to be displayed on the display unit of the other information processing device has been switched to a first image in the information processing device, and then temporarily hiding the other specific image, and when it is determined that the image has been switched to another image, displaying the other specific image again and then terminating the display of the other specific image.

This configuration makes it possible to minimize the display area for displaying other specific images while avoiding a situation where multiple other specific images are displayed at the same time, making it difficult to identify other information processing devices that are displaying the first image on the information processing device.

(8) In any one of (1) to (3) above, the plurality of types of images include a predetermined third image (e.g., a video call image, a chat image, etc.) different from the first image and the second image,
The third image is an image having a type image that enables identification of the type of image being displayed on the display unit of another information processing device (e.g., steps S30 and S30a in FIG. 14, FIGS. 16 (D) and (F), etc.).

With this configuration, by switching to the third image, it becomes possible to see at a glance the setting modes set on other information processing devices during communication, improving convenience.

(9) In the above (1), the information processing device is caused to execute a step of performing specific control (e.g., steps S50 to S57 in FIG. 17 ) for switching the image displayed on the display unit of the other information processing device to the first image displayed on the information processing device in response to a third operation on the input unit.

With this configuration, specific control is performed in response to the third operation, so that the first image can be efficiently displayed on the display unit of the other information processing device, improving convenience and facilitating communication.

(10) In the above (9), the specific control is control to output icon information for displaying, on a display unit of the other information processing device, an icon that accepts an operation for switching an image displayed on a display unit of the other information processing device to a first image of the information processing device (for example, step S51 in FIG. 17 ),
The information processing device includes:
A step of displaying an icon on the display unit of the information processing device by receiving the icon information (for example, step S54 in FIG. 17 );
By accepting an operation on the icon, a step (e.g., step S57 in Figure 17) is executed to switch the image to be displayed on the display unit of the information processing device to a second image which is a first image in the information processing device that outputs the icon information and is identified from the other image information, without the need for the second operation on the input unit.

According to this configuration, by receiving icon information and displaying an icon, it is possible to know that there has been a switching request (viewing request) from another information processing device. Furthermore, even when icon information is received, unless the icon is operated, the image will not be switched to the second image, which is the first image in the information processing device that has output the icon information and is identified from the other image information. Therefore, it is possible to switch to the second image, which is the first image in the information processing device that has output the icon information and is identified from the other image information, according to the user's intention. Furthermore, when switching, it is possible to switch by operating the icon without requiring a second operation, thereby improving convenience.

(11) In the above (10), the step of displaying the icon includes controlling the display of the icon to emphasize that the icon is being displayed.

This configuration makes it possible to prevent an icon from being displayed but overlooked without realizing that it is displayed.

(12) In the above (10), the step of displaying the icon displays the icon when the icon information is received, and ends the display of the icon when an operation on the icon is accepted or when a predetermined time has elapsed without accepting the operation (e.g., steps S52 to S56, S59, and S60 in FIG. 17).

This configuration makes it possible to avoid a situation where icons are displayed indefinitely, preventing the display area of the display unit from being used effectively.

(13) In the above (9), the specific control is a control to output forcing information for forcibly switching an image displayed on a display unit of another information processing device to a first image in the information processing device (for example, viewing request information for forcibly switching to an image of the user's own terminal that is a source of the viewing request information),
The information processing device includes:
By receiving the forced information, a step is executed in which the image to be displayed on the display unit of the information processing device is switched to a second image, which is a first image in the information processing device that output the forced information and is identified from the other image information, without the need for the second operation on the input unit (for example, in a user terminal that received the viewing request information, the image is forcibly switched to an image of the user's own terminal that sent the viewing request information, without the need for any operation on the user terminal).

According to this configuration, by outputting the forced information, it is possible to switch in the other information processing device to the first image in the information processing device that output the forced information. In addition, in the other information processing device that receives the forced information, it is possible to switch to the first image in the information processing device that output the forced information without requiring a second operation. This makes it possible to more reliably switch to the first image in the information processing device that output the forced information, and improves convenience.

(14) In the above (13), the information processing device executes a step of notifying information regarding switching to a second image specified from the other image information by receiving the forced information (for example, when switching to the image of user X's own terminal thereafter, displaying a message or outputting audio such as "Switching to user X's screen in response to your viewing request"; when switching to the image of user X's own terminal has already been performed, displaying a message or outputting audio such as "Switching to user X's screen in response to your viewing request").

With this configuration, information regarding the switch to the second image is notified, allowing the user to understand the reason and circumstances under which the image was forcibly switched.

(15) In any of (9) to (14) above, the step of switching to the second image identified from the other image information without requiring the second operation on the input unit switches to the second image in the same manner as when the second operation is accepted at a predetermined timing and switched to one of the multiple types of images (for example, the display mode is switched and set at a predetermined time in the same manner as when a display mode switching operation (e.g., a swipe operation from left to right) for switching from the display mode currently set in the user terminal to the other terminal mode of the user who sent the viewing request information is performed (accepted) at a predetermined time (e.g., 0.2 seconds), thereby switching and setting the display mode at a predetermined time, so that the image displayed on the user terminal (the image corresponding to the display mode) switches sequentially (stepwise) at a predetermined time).

With this configuration, the second image can be switched to in the same manner as when the second operation was accepted, which reduces the sense of discomfort felt when the image is switched to without the second operation being performed, and allows for smooth image switching.

(16) In any of (9) to (14) above, the step of switching to a second image identified from the other image information without requiring the second operation on the input unit may switch to the second image in a manner different from when the second operation is accepted and the image is switched to one of the multiple types of images (for example, by setting the other party's terminal mode of the user who sent the viewing request information as the display mode and immediately switching to an image in the other party's terminal mode of the user who sent the viewing request information without switching to an image corresponding to another display mode, etc.).

With this configuration, the second image is switched to in a manner different from when the second operation was accepted, so the user can be informed in an easily understandable manner that this is different from normal switching.

(17) In any of (9) to (14) above, the third operation can be accepted only when the first image is displayed on the display unit of the information processing device.

With this configuration, the third operation can be accepted only when the first image to be displayed on the other information processing device is being displayed on the information processing device, thereby preventing an unintended image from being displayed on the other information processing device.

(18) In any of (9) to (14) above, the third operation can be accepted regardless of which image is displayed on the display unit of the information processing device.

With this configuration, the third operation can be accepted regardless of which image is being displayed, eliminating the need to switch to the first image to be displayed, improving operability and convenience.

(19) In the above (1), a step of receiving a designation operation for an image displayed on the display unit in the information processing device (e.g., step S70 in FIG. 19 );
A step of outputting part information for enabling identification of the image displayed on the display unit and the part corresponding to the accepted designation operation (for example, step S71 in FIG. 19 );
A step of receiving part information output from another information processing device (for example, step S72 in FIG. 19 );
When the image displayed on the display unit includes a part of the image identified from the part information, a step of superimposing and displaying an effect image on that part (e.g., steps S73 and S74 in Figure 19) is executed.

With this configuration, in another information processing device that displays on its display an image including a body part identified from body part information from a certain information processing device, the effect image is superimposed on the body part, allowing other users to quickly and accurately grasp the body part that they wish to explain or introduce, etc., and facilitating communication.

(20) In the above (19), the step of accepting the designation operation can accept the designation operation regardless of whether the image displayed on the display unit is the first image or the second image.

With this configuration, a designation operation can be performed not only on the first image but also on the second image, and the effect image is displayed superimposed on the part corresponding to the designation operation, thereby improving convenience.

(21) A program (e.g., a program included in a web page, an installed program, etc.) executed in an information processing device (e.g., a user terminal) having a processor, a memory, an input unit, and a display unit,
The information processing device includes:
A step of outputting self-image information for enabling another information processing device to identify an image that can be displayed on the display unit in response to a first operation (e.g., a scroll operation) on the input unit (e.g., step S14);
receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device (e.g., step S15);
execute a step (e.g., steps S24 to S30) of switching an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from other image information received in the receiving step, in response to a second operation (e.g., a switching operation) on the input unit;
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different (see (8) in the modified example);
The information processing device includes:
In response to a third operation on the input unit, a step of performing specific control to switch the image displayed on the display unit of the other information processing device to the first image of the information processing device is executed (for example, steps S50 to S57 in Figure 17).

According to this configuration, the image displayed on the other information processing device can be viewed at the timing of the user who owns the information processing device, and smooth communication with the user who owns the other information processing device can be achieved. In addition, since specific control is performed in response to the third operation, the first image can be efficiently displayed on the display unit of the other information processing device, improving convenience and facilitating communication.

(22) A program (e.g., a program included in a web page, an installed program, etc.) executed in an information processing device (e.g., a user terminal) having a processor, a memory, an input unit, and a display unit,
The information processing device includes:
A step of outputting self-image information for enabling another information processing device to identify an image that can be displayed on the display unit in response to a first operation (e.g., a scroll operation) on the input unit (e.g., step S14);
receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device (e.g., step S15);
execute a step (e.g., steps S24 to S30) of switching an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from other image information received in the receiving step, in response to a second operation (e.g., a switching operation) on the input unit;
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different (see (8) in the modified example);
The information processing device includes:
A step of accepting a designation operation for an image displayed on the display unit (e.g., step S70 in FIG. 19 );
A step of outputting part information for enabling identification of the image displayed on the display unit and the part corresponding to the accepted designation operation (for example, step S71 in FIG. 19 );
A step of receiving part information output from another information processing device (for example, step S72 in FIG. 19 );
When the image displayed on the display unit includes a part of the image identified from the part information, a step of superimposing and displaying an effect image on that part (e.g., steps S73 and S74 in Figure 19) is executed.

With this configuration, images displayed on other information processing devices can be viewed at the user's own timing, enabling smooth communication with users who own other information processing devices. In addition, in another information processing device that displays an image including a body part identified from body part information from a certain information processing device on its display unit, an effect image is displayed superimposed on the body part, allowing the user to quickly and accurately grasp the body part that the user wishes to explain or introduce to other users, thereby facilitating communication.

(23) An information processing device (e.g., a user terminal) including a processor, a memory, an input unit, and a display unit,
a means for outputting self-image information for enabling an image that can be displayed on the display unit in response to a first operation (e.g., a scroll operation) on the input unit to be identified in another information processing device (e.g., step S14);
A means for receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device (e.g., step S15);
a means for switching an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from other image information received by the receiving means in response to a second operation (e.g., a switching operation) on the input unit (e.g., steps S24 to S30);
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different (see (8) in the modified example);
a means for outputting switching information for enabling a different information processing device to specify that an image displayed on the display unit of the information processing device has been switched to a first image of the different information processing device by the switching means in response to a second operation on the input unit, the switching information being used to enable the different information processing device to specify that the image displayed on the display unit of the information processing device has been switched to a first image of the different information processing device (e.g., step S25c of FIG. 14);
The information processing device further includes a means (e.g., steps S40 and S41 in FIG. 15) for receiving self-switching information specifying that an image to be displayed on the display unit of the other information processing device has been switched to a first image in the information processing device as switching information from the other information processing device, and displaying another specific image corresponding to the other information processing device that has switched to the first image in the information processing device.

According to this configuration, the image displayed on the other information processing device can be viewed at the timing of the user who owns the information processing device, and smooth communication with users who own the other information processing devices can be achieved. In addition, switching information is output and received, and when the switching information is received, another specific image corresponding to the other information processing device that has switched to the first image on the information processing device is displayed, so that the other information processing device can be identified. As a result, appropriate measures can be taken for other users who are viewing the first image on the information processing device, improving convenience and facilitating communication.

(24) An information processing device (e.g., a user terminal) including a processor, a memory, an input unit, and a display unit,
a means for outputting self-image information for enabling an image that can be displayed on the display unit in response to a first operation (e.g., a scroll operation) on the input unit to be identified in another information processing device (e.g., step S14);
A means for receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device (e.g., step S15);
a means for switching an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from other image information received by the receiving means in response to a second operation (e.g., a switching operation) on the input unit (e.g., steps S24 to S30);
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different (see (8) in the modified example);
The information processing device further includes a means for performing specific control (e.g., steps S50 to S57 in FIG. 17) for switching an image displayed on a display unit of another information processing device to a first image displayed on the information processing device in response to a third operation on the input unit.

According to this configuration, the image displayed on the other information processing device can be viewed at the timing of the user who owns the information processing device, and smooth communication with the user who owns the other information processing device can be achieved. In addition, since specific control is performed in response to the third operation, the first image can be efficiently displayed on the display unit of the other information processing device, improving convenience and facilitating communication.

(25) An information processing device (e.g., a user terminal) including a processor, a memory, an input unit, and a display unit,
a means for outputting self-image information for enabling an image that can be displayed on the display unit in response to a first operation (e.g., a scroll operation) on the input unit to be identified in another information processing device (e.g., step S14);
A means for receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device (e.g., step S15);
a means for switching an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from other image information received by the receiving means in response to a second operation (e.g., a switching operation) on the input unit (e.g., steps S24 to S30);
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different (see (8) in the modified example);
A means for accepting a designation operation for an image displayed on the display unit (for example, step S70 in FIG. 19 );
A means for outputting part information for enabling identification of an image displayed on the display unit and a part corresponding to a specified operation received (for example, step S71 in FIG. 19 );
A means for receiving part information output from another information processing device (for example, step S72 in FIG. 19 );
The display device further includes a means (e.g., steps S73 and S74 in FIG. 19) for, when the image displayed on the display unit includes an image part identified from the part information, superimposing an effect image on that part and displaying it.

With this configuration, images displayed on other information processing devices can be viewed at the user's own timing, enabling smooth communication with users who own other information processing devices. In addition, in another information processing device that displays an image including a body part identified from body part information from a certain information processing device on its display unit, an effect image is displayed superimposed on the body part, allowing the user to quickly and accurately grasp the body part that the user wishes to explain or introduce to other users, thereby facilitating communication.

(26) A method executed in an information processing device (e.g., a user terminal) including a processor, a memory, an input unit, and a display unit, comprising:
A step of outputting self-image information for enabling another information processing device to identify an image that can be displayed on the display unit in response to a first operation (e.g., a scroll operation) on the input unit (e.g., step S14);
receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device (e.g., step S15);
a step of switching an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from other image information received in the receiving step in response to a second operation (e.g., a switching operation) on the input unit (e.g., steps S24 to S30);
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different (see (8) in the modified example);
a step of outputting switching information for enabling the other information processing device to specify that the image displayed on the display unit of the information processing device has been switched to the first image of the other information processing device by switching to the second image in the switching step in response to a second operation on the input unit (e.g., step S25c of FIG. 14);
The method includes a step of receiving self-switching information specifying that an image to be displayed on a display unit of the other information processing device has been switched to a first image in the information processing device as switching information from the other information processing device, and displaying another specific image corresponding to the other information processing device that has switched to the first image in the information processing device (e.g., steps S40 and S41 in FIG. 15).

According to this configuration, the image displayed on the other information processing device can be viewed at the timing of the user who owns the information processing device, and smooth communication with users who own the other information processing devices can be achieved. In addition, switching information is output and received, and when the switching information is received, another specific image corresponding to the other information processing device that has switched to the first image on the information processing device is displayed, so that the other information processing device can be identified. As a result, appropriate measures can be taken for other users who are viewing the first image on the information processing device, improving convenience and facilitating communication.

(27) A method executed in an information processing device (e.g., a user terminal) including a processor, a memory, an input unit, and a display unit, comprising:
A step of outputting self-image information for enabling another information processing device to identify an image that can be displayed on the display unit in response to a first operation (e.g., a scroll operation) on the input unit (e.g., step S14);
receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device (e.g., step S15);
a step of switching an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from other image information received in the receiving step in response to a second operation (e.g., a switching operation) on the input unit (e.g., steps S24 to S30);
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different (see (8) in the modified example);
The method includes a step of performing specific control (e.g., steps S50 to S57 in FIG. 17) to switch an image displayed on a display unit of another information processing device to a first image displayed on the information processing device in response to a third operation on the input unit.

According to this configuration, the image displayed on the other information processing device can be viewed at the timing of the user who owns the information processing device, and smooth communication with the user who owns the other information processing device can be achieved. In addition, since specific control is performed in response to the third operation, the first image can be efficiently displayed on the display unit of the other information processing device, improving convenience and facilitating communication.

(28) A method executed in an information processing device (e.g., a user terminal) including a processor, a memory, an input unit, and a display unit, comprising:
A step of outputting self-image information for enabling another information processing device to identify an image that can be displayed on the display unit in response to a first operation (e.g., a scroll operation) on the input unit (e.g., step S14);
receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device (e.g., step S15);
a step of switching an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from other image information received in the receiving step in response to a second operation (e.g., a switching operation) on the input unit (e.g., steps S24 to S30);
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different (see (8) in the modified example);
A step of accepting a designation operation for an image displayed on the display unit (for example, step S70 in FIG. 19 );
A step of outputting part information for enabling identification of the image displayed on the display unit and the part corresponding to the accepted designation operation (for example, step S71 in FIG. 19 );
A step of receiving part information output from another information processing device (for example, step S72 in FIG. 19 );
When the image displayed on the display unit includes a part of an image identified from the part information, a step of superimposing an effect image on the part and displaying it (e.g., steps S73 and S74 in Figure 19) is included.

According to this configuration, images displayed on other information processing devices can be viewed at the user's own timing, and smooth communication with users who own other information processing devices can be achieved. In addition, in another information processing device that displays an image including a body part identified from body part information from a certain information processing device on its display unit, an effect image is displayed superimposed on the body part, so that the body part that the user wishes to explain or introduce to other users can be quickly and accurately grasped, and smooth communication can be achieved.

(29) The input unit is a touch panel,
the second operation is an operation of moving along a predetermined direction on the touch panel,
The switching step, when the first image is displayed on the display unit of the information processing device, does not switch to another image among the multiple types of images in response to an operation of moving in a specific direction among specified directions, but can switch to another image among the multiple types of images in response to an operation of moving in a direction other than the specific direction among specified directions.

This configuration makes it easier to understand the location of the first image, and prevents the operation to switch to the first image from becoming difficult to understand.

FIG. 2 is a diagram illustrating an example of a hardware configuration of a communication system. FIG. 2 is a diagram illustrating an example of a hardware configuration of a user terminal. 1 is a diagram for explaining a flow up to the start of WebRTC (including, for example, P2P communication) between multiple user terminals. 2 shows an example of a display screen of a touch screen of a user terminal X. 1 shows an example of a display screen of a touch screen of a user terminal X during P2P communication, and an example of a display screen of a touch screen of a user terminal Y, which is a partner terminal, in self-terminal mode. 1 shows an example of a display screen of a touch screen of a user terminal X during P2P communication, and an example of a display screen of a touch screen of a user terminal Y, which is a partner terminal, in self-terminal mode. 11 is a flowchart illustrating a drawing process. 13 is a flowchart illustrating an image specification process. 11 is a flowchart illustrating a display process. 10 is a flowchart illustrating a drawing process according to a second embodiment. 10 is a flowchart for explaining an image specification process in the second embodiment. 10 is a flowchart illustrating a display process in the second embodiment. 13 shows an example of a display screen of a touch screen of a user terminal X in a self-terminal mode during P2P communication in the second embodiment, and an example of a display screen of a touch screen of a user terminal Y which is a counterpart terminal. 13 is a flowchart illustrating a display process according to the third embodiment. 13 is a flowchart illustrating a user icon control process. 11A to 11C are diagrams for explaining a display example based on the user icon control process. 13 is a flowchart illustrating a process related to a viewing request. 13A and 13B are diagrams for explaining examples of displays performed in the browsing request-related process; 13 is a flowchart illustrating an operation part sharing process. 13A to 13C are diagrams for explaining a display example by the operation part sharing process.

The communication system according to the present disclosure is a system in which a screen is shared between information processing devices owned by multiple users and can be switched by simple operations. Hereinafter, an embodiment of the communication system will be described with reference to the drawings. Note that the present invention is not limited to the following examples, but is indicated by the claims, and it is intended that all modifications within the meaning and scope equivalent to the claims are included in the present invention. In the following description, the same elements in the description of the drawings are given the same reference numerals, and redundant description will not be repeated.

[First embodiment]
<Hardware configuration of communication system>
FIG. 1 is a diagram showing an example of a hardware configuration of a communication system 1. The communication system 1 includes a plurality of user terminals 100, a WEB server 200, and a signaling server 300. The plurality of user terminals 100, the WEB server 200, and the signaling server 300 can each be connected to a network 2 for communication and can transmit and receive information (data) in both directions. The network 2 is composed of the Internet and various mobile communication systems constructed by wireless base stations (not shown). Examples of the mobile communication system include so-called 4G and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks (e.g., Wi-Fi (registered trademark)) that can be connected to the Internet by a predetermined access point.

The user terminal 100 (computer, information processing device) may be, for example, a mobile terminal such as a smartphone, a feature phone, a PDA (Personal Digital Assistant), or a tablet computer, or may be a general-purpose computer such as a stationary personal computer. The user terminal 100 also has a web browser installed for viewing web pages.

2 is a diagram showing an example of the hardware configuration of the user terminal 100. As shown in the figure, the user terminal 100 includes a processor 101 constituting a control processing unit, a RAM 102 constituting a storage unit, a ROM 103, an EEPROM 104, a camera 105 that converts an image captured by an image sensor (e.g., a CMOS (Complementary Metal-Oxide Semiconductor) sensor, etc.) into a digital signal, a microphone 106 that converts mechanical vibrations of a diaphragm or the like generated by sound waves into an electrical signal, a speaker 107 that outputs sound, a touch screen 108 that has a display unit that displays images and detects touch operations on the display unit, an operation detection unit 109 that detects pressing operations by a user on a specific button (operation unit), a communication IF 110, and an input/output IF 111. These components of the user terminal 100 are electrically connected to each other by a communication bus. The communication IF 110 includes a NIC (Network Interface Controller) and a wireless LAN (Local Area Network) chip, etc. In addition, instead of or in addition to the touch screen 108, the user terminal 100 may be provided with an input/output IF to which a display (display unit) configured separately from the user terminal 100 main body can be connected.

The processor 101 includes a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit). For example, the processor 101 loads a program stored in the EEPROM 104 onto the RAM 102, receives data from the touch screen 108, the operation detection unit 109, the communication IF 110, etc. according to instructions from the program, calculates and processes the acquired data according to a procedure defined in the program, and outputs the calculated and processed data to the EEPROM 104, the touch screen 108, the communication IF 110, etc.

Returning to FIG. 1, the WEB server 200 and the signaling server 300 (computer, information processing device) may each be a general-purpose computer such as a workstation or a personal computer. The WEB server 200 and the signaling server 300 include at least a processor, memory, storage, a communication IF, and an input/output IF, and are electrically connected to each other by a communication bus.

The WEB server 200 is a server that provides a specific WEB page. The WEB page may be a one-level page, or a multi-level page that transitions to a linked page by clicking on a specific link within the page. The WEB page provided by the WEB server 200 includes, for example, a WEB page that includes a program (e.g., script, JavaScript (registered trademark), etc.) for providing WebRTC (Web Real-time Communications) services. By accessing the WEB server 200 using a WEB browser implemented in the user terminal 100, a user can receive communication services such as video calls and screen sharing that use WebRTC technology based on the program included in the WEB page.

The signaling server 300 is a server that mediates the exchange of information to establish an RTC Peer Connection between the web browsers implemented in each of the multiple user terminals 100. By accessing the web server 200 using the web browser, the user terminal 100 can receive html files that make up a web page and display the web page, and can also download program files for video communication, screen sharing, etc. using WebRTC technology, enabling real-time video calls, screen sharing, etc. with other user terminals 100.

In the user terminal 100 of this embodiment, based on the downloaded program, for example, a video call can be performed between multiple user terminals 100 in which an RTC Peer Connection has been established via the signaling server 300 and Peer to Peer (P2P) communication is possible, for example, by transmitting and receiving images captured by the camera 105 and audio input to the microphone 106 in real time. Furthermore, the user terminal 100 in this embodiment can transmit and receive information for displaying a WEB page image actually viewed on the user terminal 100 on another user terminal 100 by the downloaded program, and can perform screen sharing in which the image displayed on the touch screen is switched to one of multiple types of images (for example, an image of the user's terminal, an image of the other terminal, an image of a video call, etc.) in response to a switching operation (for example, a swipe operation to the left or right) from the user who owns the user terminal 100. The following describes the flow from the establishment of P2P communication to the video call and screen sharing.

Figure 3 is a diagram for explaining the flow up to the start of WebRTC (including, for example, P2P communication) between multiple user terminals. Figure 3 shows the flow up to the establishment of P2P communication between user terminal X owned by user X and user terminal Y owned by user Y among multiple user terminals. The horizontal axis of Figure 3 shows each configuration such as user terminal X, user terminal Y, WEB server 200, signaling server 300, etc., and the vertical axis shows the processing content of each configuration as time passes. Note that Figure 3 is merely an example of the flow up to the start of WebRTC (including, for example, P2P communication) between multiple user terminals, and is not limited to the one described below as long as it is possible to communicate between multiple user terminals. Also, Figure 4 shows an example of the display screen of the touch screen 108 in user terminal X.

First, at t1, a web browser is launched in user terminal X and a web page request is sent to web server 200. The web page request includes communication capability information (such as information that can identify the type of media (audio, video), codec format, IP address, port number, protocol used for data transfer (TCP, UDP, etc.), encryption key, etc. (information for establishing an RTC Peer Connection)) that enables user terminal X to perform P2P communication with other user terminals. When web server 200 receives the web page request, it transmits the communication capability information to signaling server 300.

At t2, a web page response is sent from the web server 200 to the user terminal X. The web page response includes information (html file, image data, etc.) for displaying a specific image. As a result, the web page provided by the web server 200 is displayed on the touch screen 108 of the user terminal X. The web page response also includes a program file (script, JavaScript (registered trademark), etc.). As a result, the user terminal 100 that has accessed the web server 200 can execute a program included in the web page to establish P2P communication, enabling video calls and screen sharing. Note that the program file included in the web page response and stored in the RAM 102 is discarded when access to the web server 200 is terminated, but this is not limited to this, and may be maintained so that it can be used the next time the user accesses the server without being discarded, depending on the user's settings, etc.

Figure 4 (a) shows an example of a screen when the web server 200 is accessed and a web page is displayed on the touch screen 108. The upper area 108a of the screen of the touch screen 108 displays the current time (for example, "18:21"), the address of the access destination (for example, "abc.com"), the name of the user who has accessed the page (in this case, "X" for user X), and an icon 401 for creating a room, which will be described later.

A partial image (screen at the start of viewing) of a web page (predetermined image) is displayed in the screen center area 108b of the touch screen 108. In this embodiment, a page of a shopping site is shown as an example, but the type of web page is not limited to this. The user terminal X can display an arbitrary portion of the web page provided by the web server 200 in the screen center area 108b in response to a scroll operation (first operation) such as a swipe operation in the up and down direction or an operation on the scroll bar on the screen center area 108b. This allows the user to view the desired portion of the web page by performing a scroll operation on the screen center area 108b.

In addition, in the upper part of the screen central region 108a, the other terminal notification image 402, the video call notification image 403, and the own terminal notification image 404 are displayed superimposed on the WEB page. The other terminal notification image 402, the video call notification image 403, and the own terminal notification image 404 are images for notifying the display mode of the screen central region 108b. As the display mode, a plurality of types of display modes are provided, including a own terminal mode for displaying an image of the own terminal, a other terminal mode for displaying an image of the other terminal, and a video call mode for displaying an image of a video call. The user terminal 100 in this embodiment can switch to one of the display modes based on a program included in the WEB page in response to a switching operation (second operation) such as a swipe operation in the left and right direction or an operation on an icon described later, and display an image corresponding to the display mode. When displaying a WEB page is started, the display mode is set to the own terminal mode. Therefore, in the central area 108a of the screen, the screen of the own terminal (a partial image of a web page) is displayed, and the own terminal notification image 404 is displayed inverted, thereby notifying the user that the screen of the own terminal is being displayed.

In the lower area 108c of the touch screen 108, a call switch icon 405a for switching between a video call in which the user's image can be displayed on the other terminal and a voice call in which a default screen is displayed without displaying the user's image on the other terminal, an end icon 405b for ending P2P communication, an input field 406 for chatting, and various icons 407 to 411 of the web browser are displayed. When P2P communication with the other terminal is established, the user can switch from a video call to a voice call or from a voice call to a video call by touching the call switch icon 405, and the user can end the P2P communication (video call) and display an image of the user's terminal by touching the end icon 405b, and can input characters, stamps, etc. by touching the input field 406, and the input characters, etc. can be sent to the other terminal.

The various icons 407 to 411 are icons displayed by the functions of the web browser. Of these, icon 409 is a sharing icon that is operated when sharing a sharing link for starting a video call, screen sharing, etc. with a partner terminal. The sharing link includes, for example, character string information for identifying a unique address of a room created by a user. Clicking on the sharing link allows access to the web server 200 and entry to the created room. As described above, when accessing the web server 200 to display a web page, multiple icons and images are displayed in addition to the web page based on the program included in the web page.

Returning to FIG. 3, in the signaling server 300 that received the communication availability information at t1, processing is performed to issue an SDP (Session Description Protocol) to the user terminal X, and at t3, the issued SDP is sent to the user terminal X and stored in a specified area of the RAM 102. The SDP includes information such as the type of media (audio, video), codec format, IP address and port number, protocol used for data transfer (TCP, UDP, etc.), and encryption key (information for establishing an RTC Peer Connection).

In addition, when user terminal X is displaying a web page provided by web server 200, as shown at t4, it is possible to establish P2P communication with a user terminal owned by another user by establishing a sharing start condition and sending a sharing link to the other user.

Figures 4(b) to (d) show example screens from when the sharing start conditions are met until the sharing link is sent to another user (user Y in Figure 4). First, the icon 401 shown in Figure 4(a) is touched to display a screen including a room creation icon 420 such as "Create a room" shown in Figure 4(b).

Next, the user touches the room creation icon 420 to create a unique room for use in the P2P communication connection, issues a unique address for the room (for example, a room-specific URL with an encryption key or encrypted text added to the end of a specific URL), and generates a sharing link containing information for identifying the address. In addition, by touching the room creation icon 420, the display mode is switched to a video call mode in preparation for the start of P2P communication. As a result, the image in the screen center area 108b is switched from an image of the user's own terminal to an image of a video call. Accordingly, in the screen of FIG. 4(c), the screen center area 108b is switched from the screen of the user's own terminal that displayed a WEB page to a screen containing a character string such as "AUDIO". At the stage of FIG. 4(c), P2P communication has not yet been established and a video call has not yet started, so a screen containing a character string such as "AUDIO" is displayed as the video call screen. In addition, when the mode is switched to the video call mode, the video call notification image 403 is displayed inverted.

P2P communication is established by notifying the other terminal of the sharing link and having the other terminal click on it. FIG. 4(c) illustrates an example of notifying the other terminal of the sharing link using the sharing function of the web browser. As shown in FIG. 4(c), the sharing icon 409 of the web browser is touched to display the screen shown in FIG. 4(d) using the function of the web browser. In the screen shown in FIG. 4(d), a window 430 is superimposed on the lower half of the screen. In the window 430, application icons 431 to 434 corresponding to applications that can be used to send a sharing link including information for identifying the unique address of the created room to another specific user are displayed. User X touches any of the application icons to start the application, selects the destination of the other user (here, user Y), and sends the sharing link. When the sending of the sharing link is completed using the application, the screen shown in FIG. 4(c) is displayed again on the touch screen 108, and the touch screen 108 is in a state of waiting for a connection from user Y. In addition, while the connection waiting state is in progress, the screen shown in FIG. 4(c) is displayed for a while, but even during this time, a switching operation can be performed to switch to the local terminal mode, display an image of the local terminal, and enable browsing of web pages, and when the connection of user Y is completed, the mode can be automatically switched to the video call mode.

As described above, user X can satisfy the sharing start conditions by operating icon 401 → creating a room (see FIG. 4(b)) → operating share icon 409 (see FIG. 4(c)) → operating the app icon, and can send a sharing link to other users using the app. Note that the method of notifying the sharing link is not limited to that shown in FIG. 4(c) and FIG. 4(d), and may be by email or the like. When user terminal X owned by user X creates a room, it is in a state of accessing the URL unique to that room.

In user terminal Y that has received the sharing link, the sharing link is touched (clicked) to access the web server 200, enter the room created by user X (access the room's unique URL), and start processing to establish P2P communication with user terminal X. When the sharing link is touched, as shown at t5 in FIG. 3, a web browser is launched in user terminal Y and a web page request is sent to web server 200. The web page request includes communication availability information that enables user terminal Y to perform P2P communication with user terminal X that sent the sharing address. When the web page request is received, web server 200 transmits the communication availability information to signaling server 300.

At t6, a web page response is sent from the web server 200 to the user terminal Y. This web page response also includes information for displaying the above-mentioned specific image and program files (scripts, JavaScript (registered trademark), etc.). As a result, the web page provided by the web server 200 is displayed on the touch screen 108 of the user terminal Y (as in FIG. 4(a)), and P2P communication is established, making it possible to carry out video calls and screen sharing.

Meanwhile, in the signaling server 300 that received the communication availability information at t5, processing is performed to issue an SDP to user terminal Y, and at t7, the issued SDP is sent to user terminal Y and stored in a specified area of RAM 102. Following t7, at t8, the received SDP is sent via the signaling server 300 to user terminal X, which is the source of the sharing link, and as shown at t9, the SDP from user terminal Y is registered in user terminal X.

At t10 following t9, user terminal X transmits the SDP of user terminal X, which was already stored at t3, to user terminal Y via signaling server 300, and as shown at t11, the SDP from user terminal X is registered in user terminal Y. This causes a connection check to be performed between user terminal X and user terminal Y, and a pair of IP address and port number that can communicate via P2P is determined, and an RTC Peer Connection is established. As a result, P2P communication is established between user terminal X and user terminal Y, a video call is started, and screen sharing becomes possible.

Figure 4(e) shows a screen in which P2P communication has been established and a video call with the other party, user Y, has begun. When P2P communication begins, the user's terminal transmits image data captured by camera 105 and audio data collected by microphone 106 to the other party's terminal, and receives image and audio data similarly transmitted from the other party's terminal, making it possible to display the image from the other party's terminal in central screen area 108b and output audio from speaker 107. The transmission and reception of image and audio data is performed at predetermined intervals (e.g., 50 msec). This makes it possible to display the video call screen. Figure 4(e) shows an image (moving image) including user Y being displayed on user terminal X.

In addition, when P2P communication is initiated, the local terminal transmits to the other terminal identification information that enables the local terminal to identify an image (area) of a web page that can be displayed in response to a scroll operation in the local terminal mode, and also receives identification information transmitted in the same manner from the other terminal.

The identification information transmitted by the own terminal is, for example, information that specifies the coordinate position of a predetermined position (e.g., the upper left corner position, etc.) of an image displayed in the screen center area 108b in response to a scroll operation in the own terminal mode, when a specific position (e.g., the position of the upper left corner of the web page, the position of coordinate 0 on the x-axis and y-axis, etc.) of a web page received when the web server 200 is accessed is taken as the base point. The identification information received by the own terminal is, for example, information that specifies the coordinate position of a predetermined position (e.g., the position of the upper left corner, etc.) of an image displayed in the screen center area 108b of the other terminal in response to a scroll operation in the own terminal mode, when a specific position (e.g., the position of the upper left corner, the position of coordinate 0 on the x-axis and y-axis, etc.) of a web page received when the web server 200 is accessed in the other terminal is taken as the base point. The transmission and reception of the identification information is performed at a predetermined time interval (e.g., 100 msec). This makes it possible to specify not only the image of the web page being viewed on the own terminal, but also the image being viewed on the other terminal in real time.

Based on a program included in a web page, the user terminal 100 switches to and sets a display mode according to a switching operation, and an image corresponding to the display mode can be displayed in the central area 108b of the screen of the user terminal 100. Note that a user terminal 100 in P2P communication transmits at least image and audio data and identification information to a partner terminal, and receives image and audio data and identification information from the partner terminal, regardless of the display mode that is set.

Figures 5 and 6 show an example of the display screen of the touch screen 108 of user terminal X during P2P communication, and an example of the display screen of the touch screen 108 of the partner terminal, user terminal Y, in self-terminal mode.

Figure 5 (a) shows an example of the display screen of user terminal X, with Figure 5 (a1) being an example of the display screen in the other terminal mode, Figure 5 (a2) being an example of the display screen in the video call mode, and Figure 5 (a3) being an example of the display screen in the own terminal mode. When the set display mode is, for example, the video call mode, as shown in Figure 5 (a2), an image (moving image) of the other user Y is displayed in the central area 108b of the screen based on the image and audio data transmitted from user terminal Y, and audio is output from speaker 107. This causes the video call screen to be displayed.

When the display screen for the video call mode is displayed, for example, when a switching operation of swiping from right to left is received, the display mode is switched to the local terminal mode as shown in FIG. 5(a3), the display screen for the local terminal mode is displayed, and the local terminal notification image 404 is switched to an inverted display state. FIG. 5(a3) shows an example in which a screen at the start of browsing of a web page is displayed.

In the self-terminal mode, any part of the web page can be displayed in response to the scroll operation on the user terminal X, which is the self-terminal. As described above, regardless of which display mode is set, the user terminal 100 transmits identification information capable of identifying the part of the web page (the image on the web page) in response to the scroll operation in the self-terminal mode to the user terminal Y, which is the other terminal, at predetermined time intervals during P2P communication. This makes it possible for the user terminal Y to identify the image of the part that the user X is viewing in response to the scroll operation on the user terminal X in the self-terminal mode. Note that even in the self-terminal mode, the voice from the user Y is output from the speaker 107 based on the voice data transmitted from the user terminal Y.

In addition, when the display screen in the own terminal mode is displayed, for example, when a switching operation of swiping from left to right is received, the display mode is switched to the video call mode as shown in FIG. 5 (a2), the display screen in the video call mode is displayed, and the video call notification image 403 is switched to a state in which it is displayed inverted. As a result, an image from the other terminal is displayed in the central area 108b of the screen, and sound is output from the speaker 107.

When the display screen in the video call mode is displayed, for example, when a switching operation of swiping from left to right is received, the display mode is switched to the other terminal mode as shown in FIG. 5 (a1), the display screen in the other terminal mode is displayed, and the other terminal notification image 402 is switched to a state in which it is displayed inverted. As described above, the user terminal Y, which is the other terminal, transmits, like the user terminal X, identification information capable of identifying a portion of the WEB page corresponding to the scroll operation in the user terminal Y's own terminal mode at predetermined time intervals during P2P communication, to the user terminal X, which is the other terminal. Therefore, the image of the portion of the WEB page that the user Y is viewing in response to the scroll operation in the user terminal Y's own terminal mode can be identified on the user terminal X side.

As shown in FIG. 5(a'), an example is shown in which the screen of the web page at the start of browsing is displayed on the user terminal Y, and identification information that can identify this screen is sent to the user terminal X. Therefore, when the remote terminal mode is set as the display mode of the user terminal X, as shown in FIG. 5(a1), the screen of the web page at the start of browsing is displayed in the central area 108b of the screen of the user terminal X based on the identification information sent from the user terminal Y. This makes it possible for the user terminal X to identify the image of the part of the web page that the user Y is viewing according to the operation of the user terminal Y in the local terminal mode. Note that even in the remote terminal mode, the voice from the user Y is output from the speaker 107 based on the voice data sent from the user terminal Y.

Furthermore, when the display screen in the other party's terminal mode is displayed, for example, when a swipe operation from right to left is received, the display mode is switched to the video call mode, the display screen in the video call mode is displayed, and the video call notification image 403 is switched to an inverted state. As a result, an image from the other party's terminal is displayed in the central area 108b of the screen, and sound is output from the speaker 107. As described above, the image displayed in the central area 108b of the screen of the user terminal 100 can be switched to any one of an image of the user's terminal, an image of the other party's terminal, and an image of the video call, by a switching operation on the user terminal 100.

Note that even if an operation such as a swipe from right to left is performed on the display screen in the own terminal mode, the display mode will not be switched to another display mode such as the other terminal mode. Also, even if an operation such as a swipe from left to right is performed on the display screen in the other terminal mode, the display mode will not be switched to another display mode such as the own terminal mode. In other words, switching from the display screen in the own terminal mode to the display screen in the other terminal mode and switching from the display screen in the other terminal mode to the display screen in the own terminal mode are restricted.

5(b) and 5(b') show examples of display screens when the part being viewed by user Y has changed. FIG. 5(b') shows an example of a display screen in which user Y scrolls in the self-terminal mode of user terminal Y to display a part of the WEB page, for example, including a "Today's Recommendation" product image, in the screen center area 108b of user terminal Y. As described above, regardless of the display mode set, user terminal Y transmits identification information for identifying an image that may be displayed in the screen center area 108b of user terminal Y in the self-terminal mode at predetermined intervals, and user terminal X receives the identification information. Therefore, by switching the display mode of user terminal X to the remote terminal mode, user X can grasp the part being viewed by user Y in real time by displaying a part of the WEB page, including a "Today's Recommendation" product image, in the screen center area 108b of user terminal X, as shown in FIG. 5(b1).

Figure 6 shows an example of a display screen when the part being viewed by user X and the part being viewed by user Y have changed. Figure 6 (c') shows an example of a display screen in which user Y scrolls while user terminal Y is in the local terminal mode, and a part of the web page including, for example, a coupon image of "Coupon recommended for you" is displayed in the central area 108b of the screen of user terminal Y. In this case, by switching the display mode of user terminal X to the remote terminal mode, the part of the web page including the coupon image of "Coupon recommended for you" is displayed in the central area 108b of the screen of user terminal X as shown in Figure 6 (c1), allowing user X to grasp the part being viewed by user Y in real time.

Similarly, FIG. 6(c3) shows an example of a display screen in which user X scrolls in the user terminal X's own terminal mode to display a portion of the web page including, for example, a "New Product!" product image in the central area 108b of the screen of user terminal X. As described above, user terminal X transmits identification information for identifying the image displayed in the central area 108b of the screen at predetermined time intervals, and user terminal Y receives the identification information. For this reason, although not shown, user Y can know in real time the portion of the web page that user X is viewing by switching the display mode of user terminal Y to the remote terminal mode, whereby a portion of the web page including a "New Product!" product image is displayed in the central area 108b of the screen of user terminal Y.

Similarly, in Fig. 6(d) and Fig. 6(d'), by switching the display mode of user terminal X to the other terminal mode, a portion of the web page including product images of "recently checked products" is displayed in the central area 108b of the screen of user terminal X as shown in Fig. 6(d1), allowing user X to grasp the portion of the web page that user Y is viewing in real time. Also, although not shown, by switching the display mode of user terminal Y to the other terminal mode, a portion of the web page including coupon images of "coupons recommended for you" (the same portion as Fig. 6(d3)) is displayed in the central area 108b of the screen of user terminal Y, allowing user Y to grasp the portion of the web page that user X is viewing in real time.

The user terminal 100 controls the display screen as illustrated in FIGS. 4 to 6 by performing various processes based on the program included in the aforementioned WEB page. Below, as an example of processes executed based on the program included in the WEB page, a drawing process for drawing in the RAM 102 an image to be displayed on the touch screen 108, an image identification process for identifying an image (location) of the WEB page that can be displayed in the central area 108b of the screen in the local terminal mode in each of the multiple user terminals 100, and a display process for switching to a display mode corresponding to a switching operation and displaying an image corresponding to the display mode in the central area 108b of the screen.

Figure 7 is a flowchart for explaining the drawing process. From the time when the user terminal 100 accesses the web server 200 and receives a web page until the user finishes viewing the web page, the user terminal 100 repeatedly executes the drawing process at predetermined intervals (e.g., every 50 msec) based on the program included in the web page.

In step S01, the web server 200 is accessed, and it is determined whether or not it is time to start browsing in response to the web page response. If it is determined that it is time to start browsing, in step S02, the entire web page, which is a specified image, is drawn in a predetermined first area of the RAM 102 of the user terminal 100. In the self-terminal mode, an image of an area of the web page drawn in the first area that corresponds to a scroll operation can be displayed in the central area 108b of the screen. On the other hand, if it is not determined that it is time to start browsing, the web page is already being viewed and the entire web page has been drawn, so the process proceeds to step S03.

In step S03, it is determined whether P2P communication has been established and it is time to start the P2P communication. If it is determined that it is time to start the P2P communication, in step S04, a video call image is drawn in a predetermined second area of RAM 102 based on image data received from the other terminal. In video call mode, the image drawn in the second area can be displayed in central screen area 108b.

In step S05, the entire web page, which is the specified image, is drawn in the third area of the RAM 102, which has been determined in advance, and the drawing process is terminated. In the remote terminal mode, an image of the area of the web page drawn in the third area that corresponds to the identification information received from the remote terminal can be displayed in the central area 108b of the screen.

On the other hand, if it is not determined in step S03 that it is time to start P2P communication, then in step S06 it is determined whether or not P2P communication is in progress. If it is determined that P2P communication is in progress, then in step S07 the video call image in the second area is updated to a video call image based on the image data received from the other terminal, and the drawing process ends. This makes it possible to display a moving image as the video call image. On the other hand, if it is not determined that P2P communication is in progress, then the drawing process ends.

As described above, the drawing process draws the base image for display in the central screen area 108b. Note that, in the drawing process of this embodiment, an example has been described in which drawing is also performed in the first area used in the local terminal mode, but drawing in the first area is performed using the functions of the web browser, and steps S01 and S02 may not be performed in this drawing process.

Figure 8 is a flowchart for explaining the image identification process. From the time when the user terminal 100 accesses the web server 200 and receives a web page until the user finishes viewing the web page, the user terminal 100 repeatedly executes the image identification process at predetermined intervals (e.g., every 50 msec) based on the program included in the web page.

In step S11, it is determined whether the display mode is the local terminal mode. The display mode is set in the display process (steps S21, S23 to S25, etc.) described later. When it is determined in step S11 that the display mode is the local terminal mode, the area (image) to be displayed in the screen center area 108b in response to the scroll operation on the local terminal among the WEB page drawn in the first area of the RAM 102 in step S12 is specified, and the process proceeds to step S13. This specifies the image to be displayed in the screen center area 108b in the local terminal mode. The area (image) specified in step S12 is maintained even after switching to another display mode, and when switching back to the local terminal mode, the maintained area (image) is displayed in the screen center area 108b. On the other hand, when it is not determined in step S11 that the display mode is the local terminal mode, the process proceeds directly to step S13.

In step S13, it is determined whether or not a predetermined time for transmitting and receiving identification information has elapsed during P2P communication. If it is determined that the predetermined time has not elapsed during P2P communication, the image identification process is terminated. On the other hand, if it is determined that the predetermined time has elapsed during P2P communication, then in step S14, regardless of the type of display mode that is set, identification information is sent to the other terminal to enable identification of the area (image) identified in step S12 that corresponds to the scroll operation. This makes it possible to identify the image displayed in the screen center area 108b in the local terminal mode on the other terminal as well.

In step S15, the area (image) to be displayed in the screen center area 108b of the web page drawn in the third area of the RAM 102 of the own terminal is identified according to the identification information from the other terminal, and the image identification process is terminated. This makes it possible to identify the image to be displayed in the screen center area 108b in the other terminal mode. The identification information from the other terminal is transmitted from the other terminal and received by the own terminal, regardless of whether the image is displayed on the other terminal or the own terminal.

As described above, the image identification process identifies an image to be displayed in the central screen area 108b. Note that, in the image identification process of this embodiment, an example has been described in which the area (image) in the local terminal mode is also identified, but the process of identifying the area (image) in the local terminal mode is performed by the function of the web browser, and the processes of steps S11 and S12 may not be performed in this image identification process. In this case, in S14, the area (image) identified by the function of the web browser in response to the scroll operation may be used to transmit identification information for identifying the area (image).

Figure 9 is a flowchart for explaining the display process. The user terminal 100 executes the display process based on the program included in the web page from the time the user accesses the web server 200 and receives the web page until the user finishes viewing the web page.

In step S21, since browsing of a web page has started, the local terminal mode is set as the display mode. In the local terminal mode, a process of displaying the local terminal notification image 404 inverted is also performed. In step S22, it is determined whether or not P2P communication has started. In step S22, it is determined that P2P communication has started when a room is created, but it may also be determined that P2P communication has started when P2P communication is actually established.

If it is not determined in step S22 that P2P communication has started, in step S27, an image of the area of the web page drawn in the first area of RAM 102 that is identified in step S12 in response to a scroll operation on the own terminal is displayed in the central area 108b of the screen. As a result, when P2P communication is not in progress or when the own terminal mode is set, an image of the part of the web page that corresponds to the scroll operation is displayed in the central area 108b of the screen and can be viewed (see FIG. 4(a) etc.).

On the other hand, when it is determined in step S22 that P2P communication is about to start, in step S23, the video call mode is set as the default display mode at the start (see FIG. 4(c) etc.). Also, a process of displaying the video call notification image 403 inverted is performed. Note that the default display mode is not limited to the video call mode, and may be another display mode, or may maintain the own terminal mode.

In step S24, it is determined whether a switching operation has been detected. If a switching operation has been detected, in step S25, a display mode corresponding to the switching operation is set from the own terminal mode, the other terminal mode, and the video call mode, and the process proceeds to step S26. In addition, a process is performed to invert the notification image corresponding to the set display mode from the other terminal notification image 402, the video call notification image 403, and the own terminal notification image 404. Note that switching from the own terminal mode to the other terminal mode without going through the video call mode, and switching from the other terminal mode to the own terminal mode without going through the video call mode, are restricted. On the other hand, if it is not determined in step S24 that a switching operation has been detected, the process proceeds to step S26.

In step S26, it is determined whether the display mode that has been set is the local terminal mode. If it is determined that it is the local terminal mode, the process proceeds to step S27, where an image from the local terminal is displayed as described above. In addition, the image from the local terminal can be displayed by performing a switching operation to switch to the local terminal mode at the user's will and timing. On the other hand, if it is not determined that it is the local terminal mode, it is determined in step S28 whether the display mode that has been set is the remote terminal mode.

When it is determined in step S28 that the remote terminal mode is selected, in step S29, an image of the area of the web page drawn in the third area of RAM 102 that is identified in step S15 according to the identification information from the remote terminal is displayed in the central screen area 108b. This makes it possible to grasp the image displayed on the remote terminal according to a scroll operation in the remote terminal mode on the remote terminal when the remote terminal is in the remote terminal mode when the remote terminal is in the remote terminal mode. In addition, by performing a switching operation to switch to the remote terminal mode at the user's will and timing, it is possible to switch to and display an image from the remote terminal regardless of whether or not an operation is performed on the remote terminal (without requiring a sharing operation on the remote terminal, etc.).

On the other hand, if it is not determined in step S28 that the other party's terminal mode is selected, then video call mode is selected, and so in step S30, the video call image drawn in step S04 in the second area of RAM 102 is displayed in the central screen area 108b. As a result, when video call mode is selected during P2P communication, the other party's status can be confirmed using an image (moving image) captured on the other party's terminal. As described above, during P2P communication, the voice of the other party can be output based on the voice data from the other party's terminal, regardless of the display mode.

Even if the video call mode is set, if the call switching icon 405a is operated on the other terminal to switch to a voice call without video, in step S30, a predetermined screen (for example, a screen including a character string such as "AUDIO" as shown in FIG. 4(c), a screen displaying the user name of the other terminal or a character image set by the user, etc.) is displayed in the central area 108b of the screen.

In step S31, it is determined whether the end icon 405b has been touched and the P2P communication has ended. If it is determined that the P2P communication has ended, the process proceeds to step S21, where processing is performed to display an image on the own terminal. On the other hand, if it is not determined that the P2P communication has ended, it is determined in step S32 whether browsing has ended. In step S32, it is determined that browsing has ended, for example, when the web browser is closed or access to the web server 200 is terminated.

If it is not determined that viewing has ended, the process proceeds to step S24, where processing is performed to switch the display mode and display an image according to the display mode. On the other hand, if it is determined that viewing has ended, the display processing is terminated.

<Example of effect of the first embodiment>
(1) In the above embodiment, the user terminal 100 (e.g., user terminal X) outputs, based on the program included in the web page response from the web server 200, identification information for enabling another user terminal 100 (e.g., user terminal Y engaged in P2P communication) to identify an area (image) of the web page that can be displayed in response to a scrolling operation on the touch screen 108 of the user terminal X, as shown in step S14 of FIG. 8, and receives identification information for enabling another user terminal 100 to identify an area (image) that can be displayed in response to a scrolling operation on the touch screen 108 of the user terminal Y, as shown in step S15 of FIG. 8. Also, based on the program included in the WEB page response from the WEB server 200, the user terminal X switches the image displayed on the touch screen 108 (screen central area 108b) of the user terminal X to one of a plurality of types of images including an image corresponding to a scroll operation on the user terminal X on the WEB page and an image specified from the identification information received from the user terminal Y, in response to a switching operation on the touch screen 108, as shown in steps S24 to S30 of FIG. 9. This allows a user (e.g., user X) to share with another user (e.g., user Y) which part of the WEB page he is viewing by accessing the WEB server 200 in a two-way manner, and further allows the image (part) of the WEB page being viewed on the user terminal (user terminal X) to be switched and displayed to the image (part) being viewed on the other terminal (user terminal Y) by a simple operation such as a swipe operation according to the user X's intention. As a result, the user X and the user Y can recognize the image being viewed by the other user at their own timing, and can communicate smoothly.

(2) The multiple types of images that can be switched by a switching operation include video call images between the user's terminal and the other terminal, as shown in FIG. 5, etc. This allows the user to grasp the state of the other user in addition to the image (part) being viewed on the other terminal, enabling smoother communication. Furthermore, when the call switching icon 405a is operated on the other terminal to switch to a voice call without video, the multiple types of images that can be switched by a switching operation include predetermined images rather than video call images, as shown in FIG. 4(c). This makes it possible for the user to prevent their own image from being displayed on the other terminal according to their convenience and preferences.

(3) Switching from your own terminal mode to the other terminal mode without going through the video call mode, or switching from the other terminal mode to your own terminal mode without going through the video call mode, is restricted. This prevents direct switching from your own terminal mode to the other terminal mode, or direct switching from the other terminal mode to your own terminal mode, reducing the chance of mistaking your own terminal's screen for the other terminal's screen, or mistaking the other terminal's screen for your own terminal's screen.

(4) Regardless of the type of display mode that is set, multiple types of notification images 402-404 corresponding to each display mode are displayed, and the notification image corresponding to the currently set display mode is displayed inverted (displayed in a manner different from other notification images). This makes it possible to easily notify the user of the currently set display mode and prevent erroneous recognition.

In addition, the notification images 402 to 404 corresponding to the multiple display modes are arranged and displayed in a predetermined order horizontally, as shown in FIG. 4(a) and other figures, and the switching operation for switching the display mode is also set to a horizontal movement operation, i.e., a left/right swipe operation. This makes the switching operation intuitively easy to understand.

(5) The identification information transmitted by the own terminal and the other terminal is information that identifies the coordinate position of a predetermined position of an image displayed in the screen center area 108b in response to a scroll operation in the own terminal mode, based on a specific position of the web page received when accessing the web server 200, and is area information that identifies a display area in response to a scroll operation. This makes it possible to reduce the amount of data in the identification information and the processing load for identifying an image.

[Second embodiment]
In the first embodiment, an example (program, method, and information processing device) was described in which a user terminal with which communication is established by WebRTC (including P2P communication) or the like performs a process (for example, sending identification information to other terminals) for making the image of the user terminal (image corresponding to a scroll operation) identifiable on other terminals in all (all) of the user terminals. However, for example, assuming a user who explains/introduces a specific image and other users who receive the explanation/introduction, the user who explains/introduces may not need to grasp the image of the user terminal used by other users in order to focus on the explanation/introduction, and in this case, the process for making the image of the user terminal used by other users identifiable on other terminals (for example, the terminal of the user who explains/introduces) is wasted. Taking into consideration such usage conditions, the second embodiment illustrates a program, method, and information processing device in which, among user terminals with which communication is established, at least one or more user terminals perform processing to make the image of the user terminal identifiable on the other terminal, while the other user terminals do not perform processing to make the image of the user terminal identifiable on the other terminal (for example, do not transmit identification information to the other terminal).

In the following, it is assumed that communication is established between user terminal X and user terminal Y by user X creating a room using user terminal X and sending a sharing link to user terminal Y of user Y in order for user X described in the first embodiment to explain and introduce the contents of a web page as an example of a predetermined image to user Y.

FIGS. 10 to 12 are flowcharts for explaining the drawing process, image specification process, and display process in the second embodiment. In the following, redundant explanations of the configurations that are the same as in the first embodiment will be omitted, and different configurations will be explained. In FIG. 10 to FIG. 12, the configurations surrounded by dotted lines differ from the corresponding processes in the first embodiment.

In the drawing process of FIG. 10, the process proceeds from step S04 to step S04a to determine whether the user terminal executing the drawing process is the user terminal that created the room. In the user terminal, by creating a room as shown in FIG. 4(b), information that can identify the creation of the room is stored in a predetermined area of RAM 102. The information that can identify the creation of the room is cleared when communication with another terminal is not established within a predetermined time period or when communication with another terminal is terminated. In step S04a, it is determined whether information that can identify the creation of a room is stored, thereby determining whether the user terminal executing the drawing process is the user terminal that created the room.

When it is determined in step S04a that the user terminal is the one that created the room (e.g., user terminal X is performing the process), the image of the other terminal (e.g., user terminal Y) is not displayed, and the drawing process is terminated. On the other hand, when it is not determined in step S04a that the user terminal is the one that created the room (e.g., user terminal Y is performing the process), in order to be able to display the image of the other terminal (e.g., user terminal X), in step S05, the entire web page, which is a specified image, is drawn in the third area of RAM 102, and the drawing process is terminated.

In the image identification process of FIG. 11, when it is determined in step S13 that P2P communication is in progress and a predetermined time has elapsed, the process proceeds to step S13a, where it is determined whether the user terminal performing the image identification process is the user terminal of the room creator. When it is determined in step S13a that the user terminal is the room creator, in step S14a, identification information is sent to the other terminal to enable identification of the area (image) identified in step S12 that corresponds to the scroll operation, and the image identification process ends. As a result, the user terminal of the room creator can make the image of its own terminal identifiable on the other terminal as soon as communication begins, without requiring any operation for sharing, etc.

On the other hand, if it is not determined in step S13a that the user terminal is the room creator, then in step S15a, the area (image) to be displayed in the screen center area 108b of the web page drawn in the third area of the RAM 102 of the user terminal is identified according to the identification information from the other terminal, and the image identification process is terminated. As a result, in a user terminal that is not the room creator, the image to be displayed in the screen center area 108b in the other terminal mode can be identified upon starting communication without requiring an operation to request sharing permission. Note that if it is not determined in step S13a that the user terminal is the room creator, the process corresponding to step S14a is not performed, and the identification information for enabling identification of the area (image) identified in step S12 and corresponding to the scroll operation is not transmitted to the user terminal of the room creator. This makes it possible to prevent unnecessary processing and reduce the processing load.

In the display process of FIG. 12, when it is determined in step S24 that a switching operation has been detected, the process proceeds to step S24a, where it is determined whether the user terminal executing the display process is the user terminal of the room creator. When it is determined in step S24a that the user terminal is the user terminal of the room creator, a display mode corresponding to the switching operation is set from the two display modes of the own terminal mode and the video call mode in step S25a, and the process proceeds to step S26. In this way, the user terminal of the room creator (the user terminal of the party providing the explanation, etc.) can switch between the own terminal mode and the video call mode.

On the other hand, if it is not determined in step S24a that the user terminal is the one that created the room, then in step S25b, a display mode corresponding to the switching operation is set from among the three display modes of the own terminal mode, the other terminal mode, and the video call mode, and the process proceeds to step S26. In this way, a user terminal that is not the one that created the room (a user terminal that receives an explanation, etc.) can switch to any of the own terminal mode, the other terminal mode, and the video call mode.

Figure 13 shows an example of a display screen of the touch screen 108 on user terminal X, the room creator, during P2P communication, and an example of a display screen of the touch screen 108 on user terminal Y, the non-room creator.

Figure 13 (a') shows a screen of user terminal X, in which user X is explaining on a WEB page. In user terminal X, as shown in step S25a of Figure 12, it is possible to switch between the own terminal mode and the video call mode, and regardless of the mode, the identification information is transmitted as shown in step S14a of Figure 11. In addition, in user terminal X, regardless of the mode, as shown in step S14a of Figure 11, the video call image is updated as shown in step S07 of Figure 10, and the voice (voice data) input to the microphone 106 of user terminal X is transmitted, and the voice based on the voice data from user terminal Y, which is the other terminal, is output from the speaker 107. As a result, the voice from user X can be transmitted to user Y, and the voice from user Y can be heard, so that user X can communicate smoothly with user Y. In FIG. 13(a'), the remote terminal notification image 402 is displayed, but since the user terminal X in the second embodiment cannot switch to the remote terminal mode, the remote terminal notification image 402 may be displayed in gray or may be hidden.

On the other hand, in the user terminal Y, as shown in step S25b of FIG. 12, the user can switch between the self-terminal mode, the other terminal mode, and the video call mode at the user's will and timing. In addition, in the user terminal Y, regardless of the mode, identification information for identifying the image of the user terminal Y is not transmitted (output), but identification information from the other terminal, user terminal X, is received. In addition, in the user terminal Y, regardless of the mode, the video call image is updated as shown in step S07 of FIG. 10, and the voice (voice data) input to the microphone 106 of the user terminal Y is transmitted, and the voice based on the voice data from the other terminal, user terminal X, is output from the speaker 107. This allows the user Y to hear the voice from user X and to transmit the voice from user Y to user X, allowing smooth communication between user Y and user X.

Fig. 13(a) shows an example of a display screen of user terminal Y, with Fig. 13(a1) being an example of a display screen in the other party's terminal mode, Fig. 13(a2) being an example of a display screen in the video call mode, and Fig. 13(a3) being an example of a display screen in the own terminal mode. User Y can check the part that user X is explaining by switching to the other party's terminal mode at his/her own will and timing without the need for a sharing operation on user terminal X, and by switching to the own terminal mode, user Y can check the part that user Y wants to see, which is different from the part that user X is explaining, and can also check the part that user X is explaining by switching to the other party's terminal mode.

13(b) and 13(b') show examples of display screens when the part being explained by user X has changed. FIG. 13(b') shows an example of a display screen in which user X scrolls the screen of user terminal X to display a part of the WEB page, for example, a product image of "recently checked products", in the central area 108b of the screen of user terminal X. As described above, regardless of the display mode set, user terminal X transmits identification information for identifying an image that may be displayed in the central area 108b of the screen of user terminal X in the self-terminal mode at predetermined intervals, and user terminal Y receives the identification information. For this reason, user Y can switch the display mode of user terminal Y to the other terminal mode, and a part of the WEB page, including a product image of "recently checked products", is displayed in the central area 108b of the screen of user terminal Y as shown in FIG. 13(b1), and the part that user X is viewing can be grasped in real time while listening to the explanation of user X. In addition, for example, if a point of concern arises later in the explanation by user X in FIG. 13(a'), user Y can switch to the self-terminal mode at his/her own will and timing to reconfirm the part that user X was explaining at the timing in FIG. 13(a'). This improves the convenience for users who are receiving explanations or introductions. In addition, users who are giving explanations or introductions can concentrate on the explanation or introduction.

<Modification>
Modifications of the above-described embodiment are listed below.

(1) In the above embodiment, an image of the user's terminal, an image of the other terminal, and a video call image are exemplified as the multiple types of images to be switched, but as long as it includes an image of the user's terminal and an image of the other terminal, it may not include other images such as video call images, and even if it includes other images, the other images are not limited to video call images. The multiple types of images to be switched may be, for example, chat images that enable chat between multiple user terminals and display the message history sent and received through the chat, or predetermined images during voice calls (wherein video call images are not displayed), instead of or in addition to video call images.

(2) In the first embodiment and the modified example, when P2P communication is established, the user of the terminal may select whether or not to transmit the identification information to the other terminal. For example, a share icon may be arranged, and when an operation on the share icon is detected, the identification information may be transmitted to the other terminal, allowing the image of the terminal to be displayed on the other terminal. When an operation on the share icon is detected while the identification information is being transmitted to the other terminal, the identification information may not be transmitted to the other terminal, preventing the image of the terminal from being displayed on the other terminal. This allows the user to select whether or not to share the image of the terminal with the other terminal according to the user's convenience and preference, improving convenience. When P2P communication is started, the default may be set to a state in which the image of the terminal is permitted to be displayed on the other terminal, or may be set to a state in which the image of the terminal is not permitted to be displayed on the other terminal. Also, the state in which the image of the terminal is permitted to be displayed on the other terminal when P2P communication is started may be set (for example, by initial setting) before the start of P2P communication to determine whether the image of the terminal is permitted to be displayed on the other terminal.

The above-mentioned share icon may be displayed only on a screen on which a web page is not displayed (for example, a screen in video call mode) and may be hidden on a screen on which a web page is displayed in the own terminal mode or the other terminal mode. This prevents the share icon from interfering with the screen on which a web page is displayed in the own terminal mode or the other terminal mode. Not limited to this, the share icon may be displayed on a screen on which a web page is displayed, but which is in the own terminal mode. This improves convenience when a screen that you do not want the other party to see for security reasons, etc., is displayed (or is displayed) on the screen in the own terminal mode.

(3) In the above embodiment and the above modified example, the identification information transmitted and received in the user terminal 100 is information that identifies the coordinate position of a predetermined position of an image displayed in the screen center area 108b in response to a scroll operation in the own terminal mode when a specific position of a WEB page is used as a base point, and is area information that identifies a display area in response to a scroll operation. However, the identification information transmitted and received in the user terminal 100 is not limited to this as long as it is information that allows the image displayed in the screen center area 108b in response to a scroll operation in the own terminal mode to be identified in the other terminal. The identification information transmitted and received in the user terminal 100 may be, for example, operation mode information that can identify the mode (direction, scroll amount, etc.) of a swipe operation on the screen center area 108b in the own terminal mode, the mode (operation content) of a scroll operation such as a touch operation, a pinch out operation, a pinch in operation, an operation on a scroll bar, or image information that allows the image displayed in the screen center area 108b in the own terminal mode to be displayed itself. In either case, in step S15 of FIG. 8, the area (image) corresponding to the identification information is identified at any time, and in the remote terminal mode, in step S29 of FIG. 9, the image of the area displayed on the remote terminal identified is displayed, making it possible to grasp the screen of the remote terminal.

(4) In the above embodiment and modified example, the program for performing the processes shown in Figures 7 to 12 and displaying the screens shown in Figures 4 to 6 and 13 is included in a web page obtained by accessing the web server 200. However, the present invention is not limited to this. The program may be downloaded and installed in advance from a predetermined server. In this case, images from each user terminal may be shared between user terminals on which the program is installed.

(5) In the above embodiment and the above modified example, a WEB page obtained by accessing the WEB server 200 is exemplified as a predetermined image shared between user terminals. However, the predetermined image is not limited to this, and may be an image (e.g., a WEB page, a document, a business document, an instruction manual, a document for a room in a rental property, etc.) that is stored or displayed in at least one user terminal. In this case, a user terminal that stores the image among the user terminals in P2P communication may transmit image data for displaying the image to the other user terminal. The predetermined image may also be an image composed of multiple types of images (e.g., a photo and a business document, etc.), or may be an image composed of multiple layers. As an image composed of multiple layers, for example, an image in the second layer may be displayed by selecting (clicking, touching, etc.) a first link in an image in the first layer, an image in the third layer may be displayed by selecting a second link in the image in the first layer, and an image in the layer corresponding to the link may be displayed by selecting a link in the second layer and third layer images in the same manner. The specified image may also be an image displayed on the display unit by starting an application installed on the user terminal (e.g., an image of an app), or an image displayed on the display unit by opening file data of an electronic document (e.g., a photograph and business documents, etc.).

(6) In the above embodiment and the above modified example, an example of sharing an image between two user terminals has been described, but the number of user terminals sharing an image is not limited to this, and may be three or more. For example, when sharing among three users, the multiple types of images to be switched include, for example, an image of the own terminal, an image of a first other terminal owned by another first user, an image of a second other terminal owned by another second user, and other images such as a video call image, and may be arranged in the order of the image of the first other terminal from the left, the image of the second other terminal, the image of the second other terminal, and the other images are arranged next, and the image of the own terminal is arranged on the far right (that is, arranged so that the other images are sandwiched between the image of the other terminal and the image of the own terminal). As a result, by swiping from left to right to display the leftmost image, the image of the first other terminal is displayed, and each time a swipe operation is performed from right to left from that state, an image according to the user's intention can be displayed, such as the image of the second other terminal → other image → image of the own terminal. In this case, as in the above embodiment and the above modified example, when the image of the first other terminal arranged at the far left is displayed, the image will not be switched to the image of the own terminal even if the image of the own terminal arranged at the far right is displayed and the image is swiped from right to left, the image will not be switched to the image of the first other terminal (note that this does not exclude the case where the image of the own terminal arranged at the far right is displayed and the image is switched to a menu screen or a setting screen by swiping from right to left). This makes it easier to understand the arrangement position of the image of the own terminal, and prevents the operation of switching to the image of the own terminal from becoming difficult to understand. Note that if the image of the own terminal is arranged at the edge, for example, the image of the second other terminal and the image of the first other terminal may be arranged from the right in that order, followed by other images, and the image of the own terminal may be arranged at the far left. In this case, as in the above embodiment and the above modified example, when the image of the user's terminal located at the far left is displayed, the image will not be switched to that of the second other terminal even if the user swipes from left to right (this does not exclude cases where the image of the user's terminal located at the far left is displayed and the image is switched to a menu screen or a setting screen by swiping from left to right), and when the image of the second other terminal located at the far right is displayed, the image will not be switched to that of the user's terminal even if the user swipes from right to left. This makes it easier to understand the arrangement position of the image of the user's terminal, and prevents the operation of switching to the image of the user's terminal from becoming difficult to understand. In addition, the notification images may be displayed in the order of the first other terminal notification image, the second other terminal notification image, the notification image corresponding to the other image, and the notification image of the user's terminal from the left, in accordance with the arrangement order of the images. In addition, although an example of connection by P2P communication has been described between multiple user terminals, the communication format is not limited to this, and may be, for example, connected via a specified server, connected via a wireless LAN or a wired LAN, or connected by a wireless communication technology capable of short-distance data communication.

In the second embodiment and the above modified example, one user terminal (user terminal X) is exemplified as the user terminal that provides the explanation/introduction, and one user terminal (user terminal Y) is exemplified as the user terminal that receives the explanation/introduction, but the number of user terminals that can communicate is not limited to this. For example, the user terminal that provides the explanation/introduction may be two or more user terminals. In this case, only an image of one of the two or more user terminals may be shared with the user terminal that receives the explanation/introduction, or each image of a predetermined number of user terminals may be shared with the user terminal that receives the explanation/introduction, or each image of all two or more user terminals may be shared with the user terminal that receives the explanation/introduction. In addition, when only an image of one user terminal is shared or when each image of a predetermined number of user terminals is shared, the user terminal to be shared may be specified or changed by a predetermined operation on the user terminal that provides the explanation/introduction when starting P2P communication or at any timing during P2P communication. In addition, whether to share images from only one user terminal, to share images from each of a predetermined number of user terminals, or to share images from each of two or more user terminals may be specified or changed by a predetermined operation on the user terminal providing the explanation or introduction when starting P2P communication or at any time during P2P communication. The user terminal receiving the explanation or introduction may also be two or more user terminals.

(7) In the above embodiment and the above modified example, a swipe operation in the left or right direction on the touch screen 108 is exemplified as a switching operation. However, a swipe operation in the left or right direction within the central area 108b of the screen on the touch screen 108 may also be validly accepted as a switching operation.

The switching operation may also be one that effectively accepts a swipe operation into a portion of the screen center region 108b (for example, a vertical region that includes a notification image and where an image may be displayed superimposed on a predetermined image (web page), such as a region from the left end to the right end). The switching operation may also be one that effectively accepts a swipe operation into a specific region other than the screen center region 108b (for example, the upper region 108a of the screen, the lower region 108c of the screen, etc.). In this case, the region that accepts a scroll operation for displaying any part of the web page in the screen center region 108b and the region that accepts a switching operation can be separated, thereby improving operability.

The switching operation is not limited to a swipe operation, and may be, for example, a touch operation on the other terminal notification image 402, the video call notification image 403, and the own terminal notification image 404. In this case, the display mode may be switched to the one corresponding to the notification image that has been touched. The switching operation may be an operation on the operation detection unit 109 mounted on the user terminal, or an operation of shaking the user terminal itself to the left or right when a sensor (e.g., an acceleration sensor, an angular velocity sensor, etc.) is provided for identifying an operation on the user terminal itself. The switching operation may be a display operation in which a scroll bar or the like is displayed on the display unit to identify a location, and the image of the location corresponding to the location is switched by moving the scroll bar.

(8) In the above embodiment and the above modified example, in order to display the image of the other terminal, an example was described in which the entire WEB page, which is a predetermined image, is drawn in the third area of RAM 102 in step S05. However, since the image to be shared is the same image (for example, an image of the same WEB page), the process of step S05 may not be performed, and for example, in identifying the image in the other terminal, the WEB page drawn in the first area in step S02 may be used to display the image of the area of the WEB page drawn in the first area according to the identification information received from the other terminal in the screen center area 108b. Note that, although an example was described in which the image of the own terminal and the image of the other terminal are images in the same predetermined image, they are not limited to this, and may be different images (for example, when different images are selected or specified as the object to be displayed on each of the own terminal and the other terminal), or may be different images (for example, when the object to be displayed on each of the own terminal and the other terminal is fundamentally different). For example, the image of user terminal X may be an image of a predetermined WEB page, while the image of user terminal Y may be an image of a predetermined document. Also, the image of the user terminal X may be an image of a specific web page and a business document (for example, an example of an image displayed by opening file data of an electronic document), whereas the image of the user terminal Y may include an image of a specific document in addition to the image of the specific web page and the business document. Also, the image of the user terminal X may be an image of a web page with a specific URL, whereas the image of the user terminal Y may be an image of a web page with a specific URL different from the specific URL. Also, the image of the user terminal X may be an image displayed by starting application A in the user terminal X, whereas the image of the user terminal Y may be an image displayed by starting application B in the user terminal Y. Also, the image of the user terminal X may include multiple types of images such as an image of a web page with a specific URL or an image displayed by starting application A in the user terminal X, whereas the image of the user terminal Y may include multiple types of images such as an image of a web page with a specific URL different from the specific URL or an image displayed by starting application B in the user terminal Y. This increases the variety of images that can be shared, improving convenience.

(9) In the above embodiment and modified example, an example has been described in which multiple icons are displayed in the lower screen area 108c of the touch screen 108, but this is not limiting, and a predetermined image such as a web page may be displayed in the lower screen area 108c without displaying the multiple icons. This increases the screen area in which the predetermined image is displayed, improving visibility. In this case, the multiple icons displayed in the lower screen area 108c may be displayed by, for example, touching icon 401.

(10) In the above embodiment and the above modified example, an example is shown in which multiple types of images, including an image of the user's terminal and an image of the other terminal, are switched in the left-right direction (horizontal direction), and the switching operation is also defined as an operation along the left-right direction (horizontal direction). However, the switching direction is not limited to this, and may be, for example, switched along the up-down direction (vertical direction), and in this case, the switching operation may also be defined as an operation along the up-down direction (vertical direction).

(11) In the above embodiment and the above modified example, a portable smartphone or the like has been mainly described as an information processing device. However, even in the case of a stationary personal computer, for example, one of a plurality of types of images including an image of the own terminal and an image of the other terminal is displayed on a display device connected to the personal computer in response to a switching operation. In addition, it may be possible to set whether or not to display a composite image including an image of the own terminal and an image of the other terminal on one screen, and the display may be switched by a switching operation or displayed on one screen depending on the setting.

Specifically, as a control pattern for controlling the display unit of the information processing device, multiple types of control patterns may be provided, including a first control pattern for displaying one of multiple types of images including an image of the own terminal and an image of the other terminal in response to a switching operation, and a second control pattern for displaying a composite image including an image of the own terminal and an image of the other terminal on one screen, and one of the multiple types of control patterns may be set to control the display unit of the information processing device based on the set control pattern. As a result, the control pattern for controlling the display unit of the information processing device is set from among multiple types of control patterns, thereby improving convenience.

The process of setting the control pattern may involve, for example, identifying the display size (e.g., resolution, number of pixels, number of inches, etc.) of the display unit of the information processing device, and automatically setting the first control pattern when the display size is less than a predetermined size, and automatically setting the second control pattern when the display size is equal to or greater than the predetermined size. This allows the control pattern to be automatically customized according to the display size of the display unit of the information processing device being used.

In addition, the process of setting the control pattern may involve, for example, setting the control pattern selected by the user in response to the user's input operation (such as an operation on a setting icon). This allows the control pattern to be customized in response to the user's will and preference. Furthermore, in the initial stage (when sharing/P2P communication is started), a control pattern in response to the display size may be automatically set, and the control pattern may be changed later in response to the user's input operation. This improves convenience.

(12) In the above embodiment and the above modified example, an example was described in which voice data input to the microphone 106 of the terminal is transmitted to the other terminal and output from the speaker 107 based on the voice data from the other terminal. However, when the output of voice data is not permitted on the other terminal side, the process for outputting the voice input to the microphone 106 of the terminal at the other terminal is not executed (for example, even if voice is input to the microphone 106 of the terminal, the voice data is not transmitted to the other terminal), and when the output of voice data is permitted on the other terminal side, the process for outputting the voice input to the microphone 106 of the terminal at the other terminal is executed (for example, the voice data input to the microphone 106 of the terminal is transmitted to the other terminal and the voice is output from the other terminal). As an example, in the second embodiment, the user terminal X, which is the side giving the explanation, and the user terminal Y, which is the side receiving the explanation, can set whether or not to permit the output of voice data from other terminals in the user terminal X. When the output of voice data is not permitted, even if voice is input to the microphone 106 of the user terminal Y, the voice data input to the microphone 106 of the user terminal Y is not transmitted to the user terminal X, and when the output of voice data is permitted on the user terminal X side, the voice data input to the microphone 106 of the user terminal Y is transmitted to the user terminal X to output the voice from the user terminal X. On the other hand, since the user terminal Y is the side receiving the explanation, it is configured not to be possible to set whether or not to permit the output of voice data from the user terminal X. As a result, the user terminal Y can reliably output the voice from the user X, while the user terminal X does not permit the output when it is not desirable for the user Y to output voice (such as when the explanation is interrupted), so that the voice from the user Y can be blocked and the user can concentrate on the explanation, improving convenience.

In particular, when the number of user terminals sharing the image of user terminal X is two or more, that is, when the image of user terminal X giving the explanation is shared among multiple user terminals (user terminals receiving the explanation), and when the audio from user terminal X can be output from each of the multiple user terminals and the audio from any of the multiple user terminals can be output from user terminal X and each of the other user terminals, there is a risk that the audio from any of the multiple user terminals will become mixed up and interfere with user X's explanation, or that the audio from each of the multiple users will become mixed up and cause confusion. In other words, when the information for identifying an image that can be displayed on the display unit of another information processing device is information receivable by multiple information processing devices, and the audio information received from the other information processing device includes audio identified from the audio information output by the second step in the multiple information processing devices, there is a risk that the audio from the multiple information processing devices will cause various problems as described above. However, by making it possible to set on the user terminal X whether to permit audio output from each of the multiple user terminals (user terminals receiving explanations, etc.), the audio from each of the multiple users can be output from each user terminal only when the user X permits it, and is not output when the user X does not permit it, thereby improving convenience. Note that with regard to the setting of whether to permit audio output by the user terminal X, it may be possible to set whether to permit audio output for all users uniformly across all multiple user terminals, or alternatively or in addition to this, it may be possible to set whether to permit audio output for each user terminal (each user). This can further improve convenience.

(13) In the second embodiment and the modified example, the processing for making the first image (e.g., the image on the user terminal Y side) identifiable on another information processing device (e.g., user terminal X) is not executed, and as described in FIG. 11, the processing for not outputting the identification information for identifying the area (image) corresponding to the scroll operation on the user terminal Y is not executed. However, the present invention is not limited to this. It is also possible that the identification information is output from the user terminal Y but is not received on the user terminal X side, or the display processing using the identification information is not executed.

(14) In the second embodiment and the modified example described above, as explained in FIG. 13, the user terminal X is the user terminal that provides the explanation, introduction, etc., and the user terminal Y that can be switched to the other terminal mode but does not output identification information for identifying the image of its own terminal is the user terminal that receives the explanation, introduction, etc. However, the relationship is not limited to this, and the user terminal X may be the user terminal that receives the explanation, introduction, etc., and the user terminal Y that can be switched to the other terminal mode but does not output identification information for identifying the image of its own terminal is the user terminal that provides the explanation, introduction, etc.

(15) In the second embodiment and the modified example, as shown in step S04a of FIG. 10, step S13a of FIG. 11, and step S24a of FIG. 12, it is determined whether the user terminal executing the process is the user terminal that created the room, but it may be determined whether the user terminal is the user terminal that sent the sharing link, for example, as long as it is specified as the user terminal of the host of the communication. Also, in step S04a of FIG. 10, step S13a of FIG. 11, and step S24a of FIG. 12, it is determined whether the user terminal is the user terminal of the host of the communication, but this is not limited to this, and it may be determined whether the user terminal is a specific user terminal. The specific user terminal may be, for example, a user terminal that is not the initiator of the communication (user terminal Y in FIG. 13), and may be a terminal that can be designated or changed among the communicating user terminals by performing a specific setting operation on all the communicating user terminals or on a specific user terminal among all the communicating user terminals (for example, the user terminal on the initiator side of the communication) when starting P2P communication or at any time during the P2P communication.

(16) In the above embodiment and the above modified example, the example shown in the first embodiment and the example shown in the second embodiment have been described as separate technical matters, but the present invention is not limited to this. For example, the present invention may be a program, method, and information processing device that can execute both the process described in the first embodiment (the process shown in Figures 1 to 9 including the process for sharing images on both sides, hereinafter also referred to as the first process) and the second process described in the second embodiment (the process shown in Figures 10 to 13 including the process for sharing only one image, hereinafter also referred to as the second process), and can select and set whether to execute the first process or the second process by inputting an operation on a user terminal. More specifically, when starting P2P communication, for example, in response to an input operation on a user terminal of the communication initiator, during the communication that is about to start, it may be possible to select whether to execute the first process or the second process during the communication that is about to start, or it may be possible to select in advance whether to execute the first process or the second process in response to an input operation on a user terminal in initial setting or the like, and execute the selected process. As a result, in the other party's terminal (e.g., a user terminal that is not the organizer of the communication), the process selected in the user terminal of the organizer of the communication is executed without requiring any operation when the communication starts. Also, even after the start of P2P communication, at any timing during the communication, it may be possible to switch from the first process to the second process, or from the second process to the first process, in response to an input operation in all user terminals in the communication or in a specific user terminal among all user terminals in the communication (e.g., a user terminal of the organizer of the communication). As a result, in the other party's terminal (e.g., a user terminal that is not the organizer of the communication), the process selected in the user terminal of the organizer of the communication is executed without requiring any operation even during the communication. Also, convenience can be further improved.

(17) In the above embodiment and the above modified example, the other terminal notification image 402, the video call notification image 403, and the own terminal notification image 404 are displayed to notify the display mode of the screen center area 108b. However, instead of or in addition to this, an identification image that can identify which screen is displayed in the screen center area 108b among the multiple types of display modes may be displayed so that the user can grasp the current display mode. That is, among the multiple types of identification images, an identification image corresponding to the display mode switched by the switching operation may be switched and displayed. The identification image that can identify the other terminal mode may be, for example, the user name of the other terminal (such as the screen of user X), the role name of the other terminal (such as the screen of the presenter, the screen of the organizer), an icon, a mark, etc. corresponding to the other terminal. The identification image that can identify the video call mode may be, for example, the user name of the other terminal (such as the appearance of user X), the role name of the other terminal (such as the appearance of the presenter, the appearance of the organizer), or an icon, a mark, etc. corresponding to the appearance of the other terminal. The identification image for identifying the own terminal mode may be, for example, the user name of the own terminal (user Y's screen, your screen, etc.), the role name of the own terminal (viewer's screen, student's screen, etc.), an icon or mark corresponding to the own terminal, or the own terminal mode may be notified by not displaying anything as the identification image for the own terminal mode. The identification image may be displayed, for example, in a predetermined area (for example, the lower right area) of the screen center area 108b, or in a predetermined area other than the screen center area 108b. The identification image may be displayed at all times, or may be displayed until a predetermined time (for example, 10 seconds) has elapsed since the display mode was switched by performing a switching operation, and may be hidden when the predetermined time has elapsed.

[Third embodiment]
In the third embodiment, the program, method, and other functions (technical matters) of the information processing device exemplified in the first embodiment, the second embodiment, and the modified example (hereinafter collectively referred to as the above-mentioned embodiment) will be described. When the screens of multiple terminal devices are shared and can be switched by simple operations as exemplified in the above-mentioned embodiment, for example, an image that can be displayed in response to a scroll operation or the like by setting a self-terminal mode on a user terminal of a certain user (hereinafter also referred to as a self-terminal image) can be displayed on the user terminal of another user and can be freely viewed.

On the other hand, the user (the user mentioned above) whose own terminal image is being viewed by another user cannot know that the own terminal image is being viewed by another user, or the other user who is viewing it. This may result in a problem in that the user is unable to take appropriate action when the own terminal image is being viewed (for example, displaying an image suitable for other users (such as an image that the user wishes to explain or introduce, or displaying an image that the user does not wish other users to view). In order to solve this problem, the program, method, and information processing device in the third embodiment have a function (hereinafter also referred to as the first function) for notifying other users when their own terminal image is being viewed so that the other users can be identified.

In addition, even when the user wishes to immediately show the image on the user's own terminal to other users for explanation or introduction, the other users must perform normal switching operations. This may result in the image not being efficiently viewable or shared by other users. In order to solve this problem, the program, method, and information processing device in the third embodiment show an example having a function (hereinafter also referred to as the second function) for switching the image displayed on the other user's terminal to the image on the user's own terminal to make it easier for the other users to view (encourage viewing).

Furthermore, when multiple users are viewing the same image (an image that includes a specific part), even when one user explains or introduces a specific part (part of interest) of the image, the specific part cannot be quickly and accurately grasped by the other users. As a result, there is a risk that the explanation or introduction cannot be efficiently given to the other users viewing the same image. In order to solve such problems, the program, method, and information processing device in the third embodiment show an example having a function (hereinafter also referred to as the third function) for notifying the specific part to be explained so that it is possible to identify it. The first to third functions will be specifically described below with reference to Figs. 14 to 20.

Figure 14 is a flowchart for explaining the display process in the third embodiment. In the following, the same parts as those in the above-mentioned embodiments, for example, the first embodiment, will not be described repeatedly. Figure 14 differs from the corresponding process in the first embodiment in that the configuration surrounded by the dotted line is added.

In the display process of FIG. 14, after a switching operation is detected in step S24 and a display mode corresponding to the switching operation is set in step S25, setting mode information for identifying the display mode set in step S25 (the other user in the case of the other terminal mode) and the user of the terminal executing the display process (the user who switched the display mode) is transmitted to the other terminals (all terminals with which communication is established). For example, when communication is established between users X, Y, and Z and the user terminal Y of user Y switches to the other terminal mode of user X, setting mode information for identifying user X corresponding to the switched display mode and user Y who performed the switching operation is transmitted to the terminals of user X and user Z. This makes it possible to identify that user Y has switched to the other terminal mode of user X on the terminals of user X and user Z. As a result, it becomes possible to identify the display mode set for each other terminal with which communication is established (the other user in the case of the other terminal mode) on each terminal with which communication is established.

Also, when the display mode is set to a video call mode (a display mode different from the display mode corresponding to the user), an image of the other user is displayed. In addition, in the third embodiment, the setting mode set on the other user's terminal is notified in correspondence with the image of the other user. Specifically, a video call image is displayed in step S30, and the display mode currently set for each other user (the other user in the case of the other user's terminal mode) is notified based on the setting mode information received in step S30a. This makes it possible to know the display mode currently displayed on the other user's terminal by switching to video call mode (a display mode different from the display mode corresponding to the user).

In the display process of FIG. 14, a user icon control process corresponding to the first function is performed in step S30b, a view request related process corresponding to the second function is performed in step S30c, and an operation part sharing process corresponding to the third function is performed in step S30d. In this embodiment, a program, a method, and an information processing device having all of the first to third functions are shown, but the present invention is not limited to this, and may be a program, a method, and an information processing device having any one of the first to third functions, or any two of the first to third functions. That is, in the display process of FIG. 14, an example is shown in which steps S30b, S30c, and S30d are performed, but the present invention is not limited to this, and may be a program, a method, and an information processing device having only any one of steps S30b, S30c, and S30d, or any two of the steps.

(Regarding the first function)
15 is a flowchart for explaining the user icon control process. In the following, a description will be given mainly of user terminal X of user X in a situation where communication is established between a plurality of users including user X.

In step S40, it is determined whether or not setting mode information specifying switching to the own terminal image has been received. For example, when processing is being performed at user terminal X, it is determined whether or not setting mode information specifying user X as the display mode after switching has been received, based on setting mode information transmitted from another terminal (step S25c). If it is not determined in step S40 that setting mode information specifying switching to the own terminal image has been received, the process proceeds to step S42.

On the other hand, when it is determined that the setting mode information specifying the switching to the own terminal image has been received, a display process is performed in step S41 to display the user icon of the user who performed the switching operation specified from the setting mode information. The user icon is an icon corresponding to the user who performed the switching operation (the user terminal operated by the user), and is, for example, a user name for identifying the user, a facial image or illustration image of the user registered in advance by the user, or an image including a photograph or landscape image. When processing is being performed at user terminal X, for example, when it is specified from the setting mode information that user Y has switched to the own terminal image terminal of user X, the user icon of user Y is displayed on user terminal X. In this way, it is possible to know that another user has switched to the own terminal image and the other user.

In step S42, it is determined based on the received setting mode information whether or not another user (own terminal switching user) who switched to the own terminal image has identified that they have switched to another display mode. For example, when processing is being performed at user terminal X, it is determined based on setting mode information from user Y who switched to the own terminal image whether or not they have identified that they have switched to another display mode (other than user X). If it is not determined in step S42 that they have identified that they have switched to another display mode, the user icon control process is terminated.

On the other hand, when it is determined in step S42 that a switch to another display mode has been identified, in step S43, a non-display process is performed to hide (end display) the user icon of the user who switched to the other display mode. For example, when processing is being performed on user terminal X and a switch to another display mode (other than user X) is identified based on setting mode information from user Y who has switched to the user's own terminal image, the user icon of user Y displayed on user terminal X is made invisible and the display of the user icon is ended. In this way, it becomes possible to know that the other user who has switched to the user's own terminal image has switched to a display mode other than the user's own terminal image.

Figure 16 is a diagram for explaining examples of displays by the user icon control process. (a)-(d) in the upper part of Figure 16 show examples of display screens on user terminal Y, one of the partner terminals, and (A)-(F) in the lower part of Figure 16 show examples of display screens on user terminal X, which is the user's own terminal. Note that the display screens of user terminal Y and user terminal X, which are shown side by side, show display screens at the same timing; for example, Figure 16(a) and Figure 16(A) show display screens at the same timing.

In both Figures 16(a) and 16(A), the local terminal notification image is displayed inverted and local terminal mode is set, so Figure 16(a) shows user Y's local terminal image being displayed on user terminal Y, and Figure 16(A) shows user X's local terminal image being displayed on user terminal X.

Next, FIG. 16(b) shows how a display mode switching operation is performed on user terminal Y, and the display mode is switched to user X's display mode among the other terminal modes. As shown in FIG. 16(b), the other terminal notification image is displayed inverted, and user terminal Y displays the user X's own terminal image (the same image as shown in FIG. 16(A)) among the other terminals. When the display mode is switched in this manner, setting mode information is transmitted in step S25c of FIG. 14. Specifically, when user terminal Y switches to user X among the other terminals as shown in FIG. 16(b), setting mode information is transmitted to enable identification that user Y has switched to the user X's own terminal image.

On the other hand, when the user terminal X receives the setting mode information for making it possible to identify that the user terminal has been switched to user X from the user terminal Y, the result is determined as YES in step S40 of FIG. 15, and processing for displaying the user icon of user Y is performed in step S41. FIG. 16(B) shows the state in which the user icon Y of user Y is displayed to the left of the user name "X". This allows user X to understand that user Y is viewing the image of the user terminal. As a result, it is possible to take appropriate measures for the other user (user Y) viewing the image of the user terminal of user X (for example, displaying an image suitable for the other user, or displaying an image that the other user does not want to view), thereby improving convenience. The setting mode information for making it possible to identify that the user terminal has been switched to user X from user terminal Y is transmitted to the communicating user, so that, for example, when users X, Y, and Z are communicating with each other, the setting mode information is also received at the user terminal of the other user, user Z, but since the switching is not to the image of user Z's own terminal, the user icon Y of user Y is not displayed on user terminal Z.

Figure 16(c) shows a state in which the display mode is switched on user terminal Y, for example, to the user Y's own terminal mode (the same image as the image shown in Figure 16(a) before switching). When the display mode is switched in this way, setting mode information is transmitted in step S25c of Figure 14.

When user terminal X receives setting mode information that enables the user Y, who had switched to the user X's own terminal image, to specify the switching to another display mode, it determines YES in step S42 of FIG. 15, and performs a non-display process to hide user Y's user icon in step S43. FIG. 16(C) shows the state in which user icon Y of user Y, which was displayed to the left of the user name "X", is no longer displayed. In FIG. 16(C), the fact that user icon Y is no longer displayed is illustrated by displaying the area where user icon Y was displayed with a dotted line. This allows user X to know that user Y, who was viewing the user terminal image, has switched to another display mode, and prevents user Y's user icon from continuing to be displayed even after the display mode has been switched to another display mode.

In the display process in step S41, the user icon to be displayed is displayed at a predetermined position, and in the non-display process in step S43, the user icon to be hidden is hidden. However, this is not limiting. For example, in the display process in step S41, the transparency of the user icon may be gradually decreased so that the user icon becomes gradually visible at the predetermined position, and in the non-display process in step S43, the transparency of the user icon may be gradually increased so that the visibility of the user icon at the predetermined position gradually decreases, thereby hiding the user icon.

In addition, in the display process in step S41, the user icon may be moved (the dotted arrow in FIG. 16(B) represents movement) from the edge of the display area of the user terminal (for example, the upper left position) toward a predetermined position to be displayed, and in the non-display process in step S43, the user icon may be moved (the dotted arrow in FIG. 16(C) represents movement) toward the edge of the display area of the user terminal (for example, the upper left position) to be hidden. This allows the user icon to be intuitively and easily understood to start or end display of the user icon from the movement mode of the user icon. In addition, the direction of the dotted arrow shown as the movement direction of the user icon in FIGS. 16(B) and (C) is the direction along the swipe direction of the switching operation for switching the display mode. For this reason, the fun of moving from another display mode to a display mode that displays the user's own terminal image, or moving from a display mode that displays the user's own terminal image to another display mode can be expressed by the movement of the user icon, thereby improving interest.

Figure 16 (D) shows how the display mode has been switched on user terminal X to switch to video call mode. In Figure 16 (D), it is assumed that users X and Y are communicating with each other, and that user terminal Y corresponding to Figure 16 (D) has its own terminal mode set as shown in Figure 16 (d). Therefore, steps S30 and S30a in Figure 14 display a video call image as shown in Figure 16 (D), and also display a message such as "Y: Y screen displayed" to inform users that the set mode of user terminal Y is user Y's own terminal mode (Y screen).

Figure 16 (E) shows an example of how user icons are displayed when communication is taking place between users X, Y, and Z, and user X's own terminal image is being viewed not only by user Y but also by user Z. As shown in Figure 16 (E), user Y's user icon Y and user Z's user icon Z are displayed to the left of the user name "X." This allows user X to know that user Y and user Z are viewing his own terminal image.

In addition, in order to suppress the display area of the user icon assuming that many users will view the image of the own terminal, an example is shown in FIG. 16 (E) in which the user icon Z is superimposed on the left end of the user icon Y. However, this is not limited to this, and in the display process in step S41, after a predetermined time (e.g., 5 seconds) has elapsed since the user icon was displayed (e.g., the user icon is moved from the edge portion described above toward a predetermined position), the user icon is temporarily hidden, so that the user icon is hidden even if the other user is viewing, and in the non-display process in step S43, the user icon of the other user who has switched to another display mode is displayed again, and then the user icon is hidden (e.g., the user icon is moved from the predetermined position described above toward the edge portion to be hidden), and the user icon display may be terminated. In other words, the user icon of the other user may be displayed for a predetermined time after the other user starts viewing, and then temporarily hidden, and when the other user switches to another display mode, the user icon of the other user may be displayed again, and then the display of the user icon may be terminated. This allows you to know when another user has come to view your device's images or when the display mode has been switched to another mode, and also reduces the display area for user icons while minimizing the display of multiple user icons at the same time.

16(F) shows a state in which, when user Y and user Z are viewing the image of user X's own terminal as illustrated in FIG. 16(E), a display mode switching operation is performed on user terminal X to switch to video call mode. Steps S30 and S30a in FIG. 14 display the video call image as shown in FIG. 16(F), and a message such as "Y: X screen displaying" is displayed in correspondence with user Y's video image (in FIG. 16(F), the image of user Y is simplified as "Y") to inform that the setting mode of user terminal Y is user X's own terminal mode (X screen), and a message such as "Z: X screen displaying" is displayed in correspondence with user Z's video image (in FIG. 16(F), the image of user Z is simplified as "Z") to inform that the setting mode of user terminal Z is user X's own terminal mode (X screen). This allows users to see a list of the setting modes of other users with whom they are communicating by switching to a display mode that is different from the display mode that corresponds to the user, such as a video call mode, thereby improving convenience.

As illustrated in FIG. 16(F), even when the user X's own terminal mode is not set (i.e., regardless of the display mode of the user terminal X or the type of image displayed on the user terminal X), the user icons of other users viewing the user X's own terminal image are displayed on the user terminal X. In other words, step S41 in FIG. 15 is executed when step S40 is judged as YES, not only when the own terminal mode is set but also when the other terminal mode or the like is set. For this reason, for example, even when the display mode is switched on the user terminal X and switched to the other terminal mode of the user Y, the user icons of the users viewing the user X's own terminal image are displayed. This makes it possible to grasp the other users who have switched to the own terminal image on the user terminal, regardless of the display mode set on the user terminal. As a result, for example, even when the user terminal does not display the own terminal image, by grasping the other users who have switched to the own terminal image, it is possible to take appropriate measures for the other users viewing the own terminal image on the user terminal by switching the user terminal to the own terminal mode on the user terminal.

However, without being limited to this, the user icon notifying other users who are viewing the user terminal image may not be displayed when the user terminal is not set to the local terminal mode for displaying the local terminal image, but may be displayed only when the local terminal mode is set. For example, in user terminal X, a process is performed to identify and manage other users who are viewing the user terminal image of user X based on the setting mode information received regardless of the set display mode, and when the display mode of user terminal X is the local terminal mode, the user icon of the identified other user (the user viewing the user terminal image of user X) is displayed, as well as the user icon of other users who have newly come to view the image. More specifically, when the user terminal X switches to the local terminal mode, the user icons of the other users who are viewing the local terminal image of the user X at that time are displayed, and when a new user who has come to view the local terminal image of the user X is identified based on the setting mode information received during the local terminal mode, a display process (step S41) is performed to display the user icon of the user, while when a user who has switched to another display mode is identified from among the users who have been viewing, a non-display process (step S43) is performed to hide the user icon of the user, and when the user terminal X switches from the local terminal mode to the other display mode, all user icons may be hidden regardless of the viewing status of other users (even when other users continue viewing). As a result, the user icons of other users who are viewing the local terminal image are displayed only (only) when the user terminal is displaying the local terminal image, so that other users who have switched to the local terminal image can be intuitively and easily recognized.

Note that in Figures 15 and 16, an example is described in which information that enables other users viewing the user's own terminal image to be identified is notified on the user's user terminal. However, instead of or in addition to this, the number of other users viewing the user's own terminal image (for example, a message image such as "3 users are currently viewing") may be notified on the user's user terminal, or the presence or absence of other users viewing the user's own terminal image (for example, a message image such as "Other users are viewing" only if there are other users currently viewing the image) may be notified on the user's user terminal.

In addition, in FIG. 15 and FIG. 16, an example is described in which information that enables other users viewing the image of a user's own terminal to be identified is notified on the user terminal of the user. However, instead of or in addition to this, information that enables other users viewing the image of a user displayed on the user terminal of a user (including images of other users as well as the image of the user) to be identified (for example, a message image display such as "User Y is viewing this image") may be notified on the user terminal of the user, the number of other users viewing the image of a user displayed on the user terminal of a user (including images of other users as well as the image of the user) (for example, a message image display such as "3 people are currently viewing this image") may be notified on the user terminal of the user, or the presence or absence of other users viewing the image of a user displayed on the user terminal of a user (including images of other users as well as the image of the user) of a user (for example, a message image display such as "There are other users viewing this image" only if there are other users viewing the image).

In addition, while Figures 15 and 16 explain an example in which other users who are viewing a user's own terminal image are notified by an image such as a user icon corresponding to that user, instead of or in addition to this, the notification may be by audio (for example, audio output such as "Viewing by user Y has started" or "Viewing by user Y has ended").

(Regarding the second function)
17 is a flowchart for explaining the process related to a view request. In the following, the process will be explained mainly with respect to user terminal X of user X in a situation where communication is established between a plurality of users including user X.

In step S50, it is determined whether or not a viewing request icon for making a viewing request has been touched. The viewing request icon is an icon that is displayed, for example, by touching icon 401 located to the right of the user name, but it may be an icon that is constantly displayed without requiring any operation. In this embodiment, regardless of the display mode set in the user terminal (even when the local terminal mode is not set), the viewing request icon is displayed and operations on the viewing request icon can be effectively accepted. As a result, operations on the viewing request icon can be accepted regardless of which display mode is set, and operability and convenience can be improved without the need for a switching operation to switch display modes.

If it is not determined in step S50 that the view request icon has been touched, the process proceeds to step S52. On the other hand, if it is determined that the view request icon has been touched, in step S51, view request information is sent to the other terminals (all terminals with which communication is established). For example, when communication is established between users X, Y, and Z, and user X touches the view request icon, view request information for identifying that a view request has been made by user X is sent to the terminals of users Y and Z. Note that the user terminal that sent the view request information is notified that a view is being requested. The fact that a view is being requested is hidden when the image on the user's terminal is viewed by another user after the view request, or when a predetermined time has passed since the view request was made, etc.

In step S52, it is determined whether or not viewing request information from another user has been received. If it is determined in step S52 that viewing request information has not been received, the process proceeds to step S55. On the other hand, if it is determined in step S52 that viewing request information has been received, it is determined in step S53 whether or not the display mode set is the remote terminal mode of the user who sent the viewing request information. In other words, it is determined whether or not the user's own terminal image who sent the viewing request information is already being displayed. If it is determined in step S53 that the display mode set is the remote terminal mode of the user who sent the viewing request information, the user's own terminal image is already being displayed, so the process proceeds to step S55 without displaying the viewing icon.

In contrast, if it is determined that the display mode set in step S53 is not the remote terminal mode of the user who transmitted the viewing request information, a viewing icon based on the viewing request information is displayed in step S54. Therefore, the viewing request information can also be said to be information for displaying a viewing icon on another user's terminal. The viewing icon is an icon that makes it possible to identify the user who transmitted the viewing request information, and that accepts an operation to switch to the image of the user's own terminal. For example, in a situation where communication is established between users X, Y, and Z, when viewing request information is received from user Y but the image of user Y's own terminal is not being displayed, a process is performed to display a viewing icon corresponding to the viewing request from user Y on user terminal X (and user terminal Z).

In step S54, the viewing icon is simply displayed, but it is not limited to this and control may be performed to emphasize that the viewing icon is displayed. The control for emphasis may be, for example, displaying the viewing icon in an emphasis manner such as vibrating the viewing icon, or vibrating the user terminal itself that displays the viewing icon (vibration operation). This makes it possible to prevent as much as possible the viewing icon being overlooked without the user realizing that it is displayed and without considering whether or not to switch.

In step S55, it is determined whether the viewing icon has been touched. If it is not determined in step S55 that the viewing icon has been touched, it is determined in step S59 whether a predetermined time (e.g., 10 seconds) has elapsed since the viewing icon was displayed, or whether the display mode has been switched to the remote terminal mode of the user who sent the viewing request information by a display mode switching operation. If it is not determined in step S59 that the predetermined time has elapsed or that the display mode has been switched to the remote terminal mode of the user who sent the viewing request information, the process proceeds to step S61.

In contrast, when it is determined in step S59 that a predetermined time has elapsed or the mode has been switched to the other party's terminal mode of the user who sent the viewing request information, the viewing icon is hidden in step S60 and the process proceeds to step S61. This makes it possible to avoid the viewing icon remaining displayed indefinitely and the display area of the display unit being rendered ineffective, since the viewing icon is hidden when a predetermined time has elapsed without a touch operation or when the display mode is switched to the other party's terminal mode of the user who sent the viewing request information by a display mode switching operation. In addition, since the viewing icon remains displayed even if the display mode (excluding the other party's terminal mode of the user who sent the viewing request information) is switched after the viewing icon is displayed, it is possible to encourage an operation on the viewing icon. Note that the conditions for hiding the viewing icon are not limited to the above.

On the other hand, when it is determined in step S55 that the viewing icon has been touched, the viewing icon is hidden in step S56, and the process proceeds to step S57. In step S57, a switching process is performed to switch to the other terminal mode for displaying the image of the user who sent the viewing request information. For example, when communication is established between users X, Y, and Z, and the viewing icon based on the reception of viewing request information from user Y is touched by user X, a switching process is performed to switch the display mode to the other terminal mode of user Y in order to display the image of user Y's own terminal. This makes it possible to switch to the display mode of the user who sent the viewing request information without requiring a display mode switching operation to display the image of user Y's own terminal.

In the switching process in step S57, the display mode is switched and set at predetermined intervals in the same manner as when a display mode switching operation (e.g., a swipe operation from left to right) for switching from the display mode currently set on the user terminal to the partner terminal mode of the user who sent the viewing request information is performed (accepted) at predetermined intervals (e.g., 0.2 seconds), so that the images displayed on the user terminal (images corresponding to the display modes) are switched sequentially (in stages) at predetermined intervals. This allows the images to be switched in the same manner as when the display mode switching operation was accepted, thereby reducing the sense of discomfort that may be felt when the display mode is switched despite no display mode switching operation being performed, and allowing images to be switched smoothly.

However, the switching process may be a process in which the image is switched in a manner different from the manner in which it is switched when the display mode switching operation is received. For example, the switching process may be a process in which the display mode is set to the other terminal mode of the user who sent the viewing request information, and the image is immediately switched to the other terminal mode of the user who sent the viewing request information without switching to an image corresponding to another display mode, or the display mode may be set to the other terminal mode of the user who sent the viewing request information, and an image including a message to switch to the other terminal mode of the user who sent the viewing request information is displayed, and then the image is switched to the other terminal mode of the user who sent the viewing request information. Alternatively, the display mode may be set to the other terminal mode of the user who sent the viewing request information, and the image including a message to switch to the other terminal mode of the user is displayed for a predetermined period (e.g., 3 seconds). As a result, the image is switched in a manner different from that when the display mode switching operation is received, and the user can be easily informed that it is different from normal switching.

In addition, in the switching process in step S57, the display mode that was set before the switching is stored in a memory or the like, and setting mode information is sent to the other terminals (all terminals with which communication is established) to enable identification of the display mode that was switched and set (the user who sent the viewing request information) and the user of the terminal that executed the switching process (the user who switched the display mode based on the request), in the same manner as in step S25c of FIG. 14.

After the switching process in step S57 is performed, in step S58, a back icon is displayed superimposed on the image corresponding to the other terminal mode after the switching process. The back icon is an icon for immediately returning to the display mode that was set before the switching.

In step S61, it is determined whether the back icon has been touched. If it is not determined in step S61 that the back icon has been touched, it is determined in step S64 whether a predetermined time (a time different from the predetermined time in step S59, for example, a time longer than the predetermined time in step S59, for example, 5 minutes) has elapsed since the back icon was displayed, or whether the display mode has been switched to the display mode before the switching process by a display mode switching operation. If it is not determined in step S64 that the predetermined time has elapsed or that the display mode has been switched to the display mode before the switching process, the viewing request-related process is terminated, but if it is determined that the predetermined time has elapsed or that the display mode has been switched to the display mode before the switching process, the back icon is hidden in step S65 and the viewing request-related process is terminated. As a result, the back icon is hidden when a predetermined time has elapsed without being touched, or when the display mode has been switched to the display mode before the switching process in step S57 by a display mode switching operation, and so on, so that it is possible to avoid the back icon being displayed indefinitely.

On the other hand, when it is determined in step S61 that the back icon has been touched, the back icon is hidden in step S62, and the process proceeds to step S63. In step S63, a return process is performed to return to the display mode set before the switching process in step S57. For example, when a viewing icon based on the reception of viewing request information from user Y is touched by user X in a situation where communication is established between users X, Y, and Z, a switching process is performed to switch the display mode to the other terminal mode of user Y in order to display the image of user Y's own terminal, but when the back icon is touched by user X after that, a return process is performed to return to the display mode before the switching process that was stored when the switching process was performed. This makes it possible to easily switch to the display mode set before the switching process without requiring a display mode switching operation to display the image displayed before the switching process, improving the user's convenience and reducing the resistance to touching the viewing icon. In addition, after the back icon is displayed, even if the display mode (excluding the display mode before the switching process) is switched until YES is determined in step S64, the display of the back icon is maintained, so that convenience can be maintained.

In the return process in step S63, the display mode is switched and set at predetermined intervals in the same manner as when a display mode switching operation for switching from the remote terminal mode of the user who sent the viewing request information to the display mode set before the switching process is performed (accepted) at predetermined intervals (e.g., 0.2 seconds), so that the images displayed on the user terminal are switched in sequence (stepwise) at predetermined intervals. Note that, without being limited to this, the return process may set the display mode set before the switching process as the display mode, and immediately switch to the image of the display mode set before the switching process without switching to an image corresponding to another display mode, or may set the display mode set before the switching process as the display mode, display an image including a message to return to the display mode before the switching process, and then switch to the image of the display mode set before the switching process, or may set the display mode set before the switching process as the display mode, switch to the image of the display mode before the switching process, and display an image including a message to return to the display mode before the switching process for a predetermined period (e.g., 3 seconds). Furthermore, in the restoration process in step S63, similar to step S25c in FIG. 14, the restored display mode and setting mode information that enables identification of the user of the terminal that executed the restoration process (the user who restored the display mode) are sent to the partner terminals (all terminals with which communication is established).

Figure 18 is a diagram for explaining examples of displays resulting from processing related to a viewing request. (a)-(d) in the upper part of Figure 18 show examples of display screens on user terminal Y, one of the other terminals, and (A)-(E) in the lower part of Figure 18 show examples of display screens on user terminal X, which is the user's own terminal. Note that the display screens of user terminal Y and user terminal X, which are shown arranged vertically, show display screens at the same timing; for example, Figure 18(a) and Figure 18(A) show display screens at the same timing.

In both Fig. 18(a) and Fig. 18(A), the local terminal notification image is displayed inverted and local terminal mode is set, so Fig. 18(a) shows user Y's local terminal image being displayed on user terminal Y, and Fig. 18(A) shows user X's local terminal image being displayed on user terminal X. Also, in Fig. 18(A), a viewing request icon R1 containing a message such as "Please take a look!" is displayed superimposed on user X's local terminal image.

Next, FIG. 18(B) shows an example of the display when the viewing request icon R1 displayed on user terminal X is touched. When the viewing request icon R1 is touched, viewing request information for identifying that a viewing request has been made by user X in step S51 of FIG. 17 is sent to the terminal of the other user with whom communication is ongoing, and a message R2 such as "Requesting..." is displayed to indicate that a viewing request is being made.

On the other hand, in the user terminal of the other user that received the viewing request information and that does not have the remote terminal mode of user X set as the display mode, a viewing icon B1 including the name of the user who made the viewing request, such as "Look at the X screen!", and a message indicating that a viewing request has been received, is displayed based on the viewing request information in step S54 of FIG. 17, as exemplified in FIG. 18(b). This allows other users, including user Y, to know that they have received a viewing request from user X.

Figure 18 (c) shows an example of the display when user Y touches the viewing icon B1 in Figure 18 (b). When user Y touches the viewing icon B1, a switching process is performed in step S57 of Figure 17 to switch to the remote terminal mode of user X who sent the viewing request information, and as shown in Figure 18 (c), the remote terminal notification image is displayed inverted and is switched to the image of user X's own terminal (the same image as shown in Figure 18 (A) etc.) among the remote terminals on user terminal Y. Also, step S58 of Figure 17 displays a back icon B2 containing a message such as "Go back!".

As described above, by touching the viewing request icon R1, user X can cause the viewing icon B1 to be displayed on the other user's device as a control for switching to the image of user X's own device, and the other user can switch to the image of user X's own device by touching the viewing icon B1. This allows the image of the user's own device to be efficiently displayed on the user device of the other user, improving convenience.

In addition, by displaying the viewing icon B1, the user can know that there has been a viewing request from another user (user X). Even when viewing request information is received, the image of the user's own terminal (user X) that sent the viewing request information cannot be switched to unless the viewing icon B1 is operated. This allows the image to be switched to the image of the user's own terminal (user X) that sent the viewing request information according to the user's will. Furthermore, when switching, the image can be switched by operating the viewing icon B1 without the need for the normal display mode switching operation, improving convenience.

When the switching process is performed, setting mode information that can identify user X who sent the viewing request information and user Y who switched is sent to the other terminal (all terminals with which communication is established). As a result, on user terminal X that sent the viewing request information, user icon Y of user Y is displayed as shown in FIG. 18 (C). Note that since the switching of display mode for user Y is based on an operation on viewing icon B1, it is also possible to notify the user that the switching has been made based on the viewing request (for example, by displaying a message image such as "Y has responded to the viewing request!" for a certain period of time).

After that, when user Z who is communicating also touches the viewing icon B1 displayed on user terminal Z, a switching process is performed on user terminal Z to switch to the remote terminal mode of user X, and setting mode information that can identify user X who sent the viewing request information and user Z who switched is sent to the remote terminal. As a result, on user terminal X that sent the viewing request information, user icon Z of user Z is displayed together with user icon Y, as shown in FIG. 18 (D).

Figure 18 (d) shows an example of the display when user Y touches back icon B2 in Figure 18 (c). When user Y touches back icon B2, a return process is performed in step S63 of Figure 17 to return to the image corresponding to the display mode before the switching process of step S57 (the same image as shown in Figure 18 (b)). When returning to the display mode before the switching process, setting mode information capable of identifying the display mode and user Y who switched is sent to the other terminals (all terminals with which communication is established). As a result, user icon Y of user Y is hidden in user terminal X, as shown in Figure 18 (E).

Note that, although Figs. 17 and 18 show an example in which a viewing icon is displayed on the user terminal that received the viewing request information, and an operation on the viewing icon switches to an image of the user's own terminal that sent the viewing request information, this is not limited to this. For example, the user terminal that received the viewing request information may be forced to switch to an image of the user's own terminal that sent the viewing request information, without requiring an operation to switch to the user terminal.

In addition, in the user terminal that received the viewing request information, before forcibly switching to the image of the user's own terminal that sent the viewing request information, it may be notified that the image of the user's own terminal that sent the viewing request information will be switched to (for example, when switching to the image of the user's own terminal of user X, a message such as "The screen will be switched to the screen of user X in response to the viewing request" or voice output, etc.). In this case, after notifying for a certain period of time (for example, 2 seconds) that the image of the user's own terminal that sent the viewing request information will be switched to, the image of the user's own terminal that sent the viewing request information may be automatically switched to, or after notifying that the image of the user's own terminal that sent the viewing request information will be switched to, it may be switched to the image of the user's own terminal that sent the viewing request information by accepting a touch operation to end the notification. In addition, it may be forcibly switched to the image of the user's own terminal that sent the viewing request information and notified that the image of the user's own terminal that sent the viewing request information has been switched to (for example, when switching to the image of the user's own terminal of user X, a message such as "The screen of user X has been switched to in response to the viewing request" or voice output, etc.). In this way, by notifying the user who sent the request to view that the image will be switched to the image on their own device, or by notifying the user who sent the request to view that the image has been switched to the image on their own device, the user can understand the reason and circumstances under which the image was forcibly switched.

In the above embodiment, an example was shown in which a viewing request icon is displayed and operations on the viewing request icon can be validly accepted regardless of the display mode set on the user terminal (even when the local terminal mode is not set), but this is not limited to the above. Control may be performed so that the viewing request icon can be displayed and operations on the viewing request icon can be validly accepted only when the display mode set on the user terminal is the local terminal mode. This makes it possible to prevent unintended images from being displayed on other user terminals, since operations on the viewing request icon can be accepted only when the local terminal image that is to be displayed on other user terminals is displayed. Note that if the viewing request icon can be displayed only when the display mode set on the user terminal is the local terminal mode, the viewing request icon may be hidden when the display mode is switched to another display mode after the viewing request icon is displayed.

In addition, in FIG. 17 and FIG. 18, an example in which the viewing request information is basically sent to all user terminals in communication has been described. Alternatively or in addition, when making a viewing request, an operation for specifying a user to which the viewing request information is to be sent from among the users in communication can be accepted, and the viewing request information can be sent only to the user terminal of the specified user. For example, by touching the icon 401 located to the right of the user name, it is possible to display the user icons of users who are not viewing the user's own terminal image, and it is possible to specify a user according to the touch operation on the user icon, and the viewing request information can be sent to the user terminal of the user icon that was touched. This makes it possible to send the viewing request information to a pinpointed user (not limited to one user, but including multiple users), improving convenience.

In addition, in Figures 17 and 18, an example is described in which a user can request other users to view images of the user's own terminal. However, instead of or in addition to this, it is also possible to request other users to view images of a user displayed on a user's user terminal (images of a user according to the set display mode, including images of not only the user in question but also other users).

(Regarding the third function)
19 is a flowchart for explaining the operation part sharing process. In the following, the process will be explained mainly with respect to the user terminal X of the user X in a situation where communication is established between a plurality of users including the user X.

In step S70, it is determined whether or not a touch operation (designation operation) on the display image displayed on the display unit of the user terminal performing the operation part sharing process has been accepted. If it is not determined in step S70 that a touch operation on the display image has been accepted, the process proceeds to step S72. On the other hand, if it is determined in step S70 that a touch operation on the display image has been accepted, in step S71, operation part information for identifying the user of the display mode (for example, limited to the self-terminal mode or the other-terminal mode) set in the user terminal and the operation part of the display image is transmitted to the other terminal (all terminals with which communication is established). For example, in a situation in which communication is established between users X, Y, and Z, for example, when the self-terminal mode is set in user terminal X of user X and a touch operation is performed on the self-terminal image of user X, operation part information for identifying user X, who is the user of the set display mode, and the touched part (for example, coordinate position) of the display image (user X's own terminal image) is transmitted to the terminals of user Y and user Z. As another example, when the remote terminal mode of user Y is set on user terminal X of user X and a touch operation is performed on the image of user Y's own terminal, operation part information for identifying user Y, who is the user of the set display mode, and the part (e.g., coordinate position) of the display image (user Y's own terminal image) that was touched is transmitted to the terminals of user Y and user Z. The operation part information is transmitted in response to the touch operation regardless of whether the set display mode is the local terminal mode or the remote terminal mode.

Next, in step S72, it is determined whether operation part information from another user terminal has been received. When it is determined in step S72 that operation part information has been received, in step S73, it is determined whether the display mode set in the user terminal is the display mode of the user identified from the operation part information. The determination in step S73 can be said to be a determination of whether the self-terminal image of the same user as the user terminal that transmitted the operation part information is displayed, or a determination of whether an image including a touch operation part in the user terminal that transmitted the operation part information is displayed. For example, when operation part information for identifying user X, who is the user of the set display mode, and the touch-operated part of the display image (user X's own terminal image), is received, in step S73, it is determined whether the display mode for displaying user X's own terminal image is set. Also, when operation part information for identifying user Y, who is the user of the set display mode, and the touch-operated part of the display image (user Y's own terminal image), is received, in step S73, it is determined whether the display mode for displaying user Y's own terminal image is set.

When it is not determined in step S72 that the operation part information has been received, or when it is not determined in step S73 that the display mode of the user identified from the operation part information has been set, the operation part sharing process is terminated. On the other hand, when it is determined in step S73 that the display mode of the user identified from the operation part information has been set, since an image including the touch operation part in the user terminal that transmitted the operation part information is displayed in the user terminal, the effect image is superimposed on the touch-operated part identified from the operation part information in step S74, and the operation part sharing process is terminated. For example, when operation part information is received in user terminal Y by touching part A of the user X's own terminal image, and when the user terminal X has set the own terminal mode and the user X's own terminal image is displayed (when the part A is displayed on the display unit of user terminal X), the effect image is superimposed on the part A displayed on the display unit of user terminal X. In this case, for example, when the remote terminal mode of user X is set on user terminal Z and the image of user X's own terminal is displayed (when the part A is displayed on the display unit of user terminal Z), the effect image is displayed superimposed on the part A displayed on the display unit of user terminal Z. The effect image for the touch operation part is displayed regardless of whether the display mode set on the user terminal is the local terminal mode or the remote terminal mode.

Figure 20 is a diagram for explaining a display example by the operation part sharing process. (a) to (d) in the upper part of Figure 20 show an example of a display screen in user terminal Y, which is one of the partner terminals, and (A) to (D) in the lower part of Figure 20 show an example of a display screen in user terminal X, which is the user's own terminal. Note that the display screens of user terminal Y and user terminal X, which are illustrated side by side, show display screens at the same timing; for example, Figure 20(a) and Figure 20(A) show display screens at the same timing.

In FIG. 20(A), the local terminal notification image is displayed inverted and the local terminal mode is set, so that the local terminal image of user X is displayed on user terminal X, whereas FIG. 20(a) shows the other terminal notification image is displayed inverted and the user X display mode of the other terminal mode is set, so that the local terminal image of user X of the other terminal (the same image as shown in FIG. 20(A)) is displayed on user terminal Y. Therefore, as shown in FIG. 20(A), user icon Y of user Y is displayed on user terminal X.

In this way, in a state where the same image is displayed on the user terminal X and the user terminal Y, FIG. 20(B) shows a state where the user X performs a touch operation on the display image of the display unit of the user terminal X. In FIG. 20, the position where the touch operation was performed is indicated by an arrow. FIG. 20(B) shows a state where a touch operation is performed on the image of the partition among the display images. By performing the touch operation, in step S71 of FIG. 19, operation part information for identifying the user of the display mode set in the user terminal where the touch operation was performed and the operation part where the touch operation was performed on the display image is transmitted to the communicating terminal including the terminal of the user Y. On the other hand, in the user terminal Y that received the operation part information, since the display mode of the user X identified from the operation part information is set, the process proceeds to step S74 of FIG. 19, and the effect image E1 is displayed in the part identified from the operation part information (near the image of the partition) as shown in FIG. 20(b). Note that the effect image is displayed in the part identified in the same manner as in FIG. 20(b) not only in the user terminal Y but also in other user terminals that display the same image (image of the same user).

The effect image when a touch operation is performed on a specific part is, for example, an image that displays the inside of a circle of a specific size centered on the coordinate position where the touch operation was performed in a specific color (e.g., fluorescent color, etc.), but is not limited to this. Note that the effect image is displayed while the touch operation is being performed, and if the touch operation is performed for less than a specific time (e.g., 2 seconds), the effect image may be displayed until the specific time has elapsed even after the touch operation is released.

20C shows a state where user X performs a touch operation on another part of the display image on the display unit of user terminal X. In FIG. 20C, the touch operation started at the position of the dotted arrow is moved (dragged) to the position of the solid arrow and released, and the touch operation is performed to trace the characters such as "3,500 yen OFF" on the display image. During the touch operation, in step S71 of FIG. 19, operation part information for identifying the user of the display mode set in the user terminal where the touch operation was performed and the operation part where the touch operation was performed on the display image is transmitted from time to time to the communicating terminal including the terminal of user Y. Meanwhile, in user terminal Y that has received the operation part information, in step S74 of FIG. 19, the effect image E2 is displayed on the part ("3,500 yen OFF") identified from the operation part information as shown in FIG. 20C. In FIG. 20(c), first, an effect image similar to that in FIG. 20(b) is displayed in the area corresponding to the position of the dotted arrow shown in FIG. 20(C), and then, based on the operation area information transmitted continuously, the effect image is extended to the position of the solid arrow shown in FIG. 20(C), and an effect image E2 that is a substantially ellipse that surrounds the specified area ("3,500 yen OFF") is displayed. Note that the effect image is displayed in the specified area similar to that in FIG. 20(c) not only in user terminal Y but also in other user terminals that display the same image. In this way, by performing a touch operation on the area that user X wants to explain or introduce to other users, the effect image can be displayed on the area on the terminals of other users viewing the image including the same area, and as a result, the area that user X wants to explain or introduce can be easily and clearly conveyed to other users.

Figure 20 (d) shows a state where user Y performs a touch operation on the display image of the display unit of user terminal Y. Figure 20 (d) shows a state where a touch operation is performed on an image of a screen facing diagonally forward, which is displayed approximately in the center of the display image. By performing a touch operation, in step S71 of Figure 19, operation part information for identifying the user (user X) of the display mode set in the user terminal where the touch operation was performed and the operation part touched on the display image is transmitted to the communicating terminal including the user X's terminal. Meanwhile, in user terminal X that has received the operation part information, the effect image E3 is displayed in the part identified from the operation part information (near the image of the screen facing diagonally, which is displayed approximately in the center) as shown in Figure 20 (D) by step S74 of Figure 19. Note that the effect image is displayed not only in user terminal X but also in other user terminals that display the same image, in the same manner as in Figure 20 (D). In this way, by performing a touch operation on a part of the image on User X's own device that User X wishes to explain or introduce, not only User X but also other users (User Y) can cause an effect image to be displayed on that part on the device of the other user viewing an image that includes the same part. As a result, users other than User X can easily and clearly communicate the part that they wish to explain or introduce to other users.

As described above, by performing a touch operation on an image displayed on a user terminal, operation part information is transmitted to the other user terminal being notified, and an effect image is displayed superimposed on the part of the image identified from the operation part information on the other user terminal that is displaying the image. This allows the part that the user wishes to explain or introduce to other users to be quickly and accurately identified, facilitating communication.

In addition, regardless of whether the image displayed on a user terminal is an image of the user's own terminal or an image of the other terminal, operation part information is transmitted by performing a touch operation on the image, and an effect image is superimposed and displayed on the corresponding part on the other user terminal. This allows, for example, users to perform touch operations in both directions to explain or introduce things to other users, regardless of the type of image being displayed, thereby improving convenience.

In addition, in Figs. 19 and 20, an example is described in which an effect image is displayed according to the operation mode on the image, such as displaying the inside of a circle of a predetermined size in a specific color by a touch operation on the image, and displaying the inside of an approximately ellipse in a specific color by a drag operation. However, instead of or in addition to this, it may be possible to specify the type and display mode of the effect image (for example, color, size, or inverted display by tracing text, etc.) when performing an operation on the image. Also, in Figs. 19 and 20, an example is described in which an effect image generated by a touch operation on an image is displayed only on the user terminals of other users who are displaying the same image, but this is not limiting and the effect image may also be displayed on the user terminal of the user who accepted the touch operation.

The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is indicated by the claims rather than the above description, and it is intended to include all modifications within the meaning and scope of the claims.

1 Communication system, 2 Network, 100 User terminal, 200 Web server, 300 Signaling server.

Claims (25)

A program executed in an information processing device including a processor, a memory, an input unit, and a display unit,
The information processing device includes:
outputting self-portrait information for enabling an image that can be displayed on the display unit in response to a first operation on the input unit to be identified in another information processing device;
receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device;
executing a step of switching, in response to a second operation on the input unit, an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from the other image information received in the receiving step;
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different from each other;
The information processing device includes:
outputting switching information for enabling a different information processing device to specify that an image displayed on the display unit of the information processing device has been switched to a first image of the different information processing device by switching to the second image in the switching step in response to a second operation on the input unit;
a program for receiving self-switching information from another information processing device, the self-switching information specifying that an image to be displayed on a display unit of the other information processing device has been switched to a first image in the information processing device, and executing a step of displaying another specific image corresponding to the other information processing device that has switched to the first image in the information processing device. The program according to claim 1, wherein the step of displaying the other specific image displays the other specific image corresponding to the other information processing device by receiving the self-switching information, regardless of whether the first image or the second image is displayed on the display unit of the information processing device. The program according to claim 1, wherein the step of displaying the other specific image does not display the other specific image corresponding to the other information processing device when the first image is not displayed on the display unit of the information processing device even when the self-switching information is received, and displays the other specific image corresponding to the other information processing device when the self-switching information is received and the first image is displayed on the display unit of the information processing device. The program according to any one of claims 1 to 3, wherein the other specific image is an image including user information for identifying a user who operates the other information processing device. The program according to any one of claims 1 to 3, wherein the step of displaying the other specific image terminates the display of the other specific image corresponding to the other information processing device when it is determined that the image displayed on the display unit of the other information processing device has been switched from a state in which the image was switched to a first image in the other information processing device to a state in which the image was switched to another image based on switching information from the other information processing device. The program according to claim 5, wherein the step of displaying the other specific image comprises moving the other specific image from a predetermined position on the edge of the display area of the display unit of the information processing device toward the inside of the predetermined area when starting to display the other specific image, and moving the other specific image from the inside of the predetermined area toward the predetermined position when ending to display the other specific image. The program according to claim 5, wherein the step of displaying the other specific image includes displaying the other specific image when it is determined from the self-switching information that the image to be displayed on the display unit of the other information processing device has been switched to a first image in the information processing device, and then temporarily hiding the other specific image, and when it is determined that the image has been switched to another image, displaying the other specific image again and then terminating the display of the other specific image. the plurality of types of images includes a predetermined third image different from the first image and the second image,
4. The program according to claim 1, wherein the third image is an image having a type image that enables identification of a type of an image displayed on a display unit of another information processing device. The information processing device includes:
The program according to claim 1 , further comprising a step of performing specific control for switching an image displayed on a display unit of another information processing device to a first image displayed on the information processing device in response to a third operation on the input unit. the specific control is control to output icon information for causing a display unit of the other information processing device to display an icon accepting an operation for switching an image displayed on a display unit of the other information processing device to a first image of the information processing device,
The information processing device includes:
receiving the icon information and displaying an icon on the display unit of the information processing device;
The program of claim 9, further comprising: a step of switching an image to be displayed on the display unit of the information processing device to a second image identified from the other image information, which is a first image in the information processing device that outputs the icon information, without requiring the second operation on the input unit, by accepting an operation on the icon. The program according to claim 10, wherein the step of displaying the icon includes controlling the display of the icon to emphasize that the icon is being displayed. The program according to claim 10, wherein the step of displaying the icon displays the icon when the icon information is received, and ends the display of the icon when an operation on the icon is received or when a predetermined time has elapsed without receiving the operation. the specific control is control for outputting forcing information for forcibly switching an image displayed on a display unit of the other information processing device to a first image of the information processing device,
The information processing device includes:
The program according to claim 9, further comprising: a step of switching an image to be displayed on the display unit of the information processing device to a second image, which is a first image in the information processing device that output the forced information and is identified from the other image information, without requiring the second operation on the input unit, by receiving the forced information. The information processing device includes:
The program according to claim 13 , further comprising a step of notifying information regarding switching to a second image specified from the other image information by receiving the forced information. The program according to any one of claims 9 to 14, wherein the step of switching to a second image identified from the other image information without requiring the second operation on the input unit switches to the second image in the same manner as when the second operation is accepted at a predetermined timing and the step of switching to one of the multiple types of images switches to the second image. The program according to any one of claims 9 to 14, wherein the step of switching to a second image identified from the other image information without requiring the second operation on the input unit may switch to the second image in a manner different from when the second operation is accepted and the image is switched to one of the multiple types of images in the step of switching. The program according to any one of claims 9 to 14, wherein the third operation can be accepted only when the first image is displayed on the display unit of the information processing device. The program according to any one of claims 9 to 14, wherein the third operation can be accepted regardless of the image displayed on the display unit of the information processing device. The information processing device includes:
receiving a designation operation for an image displayed on the display unit;
outputting part information for enabling identification of the image displayed on the display unit and the part corresponding to the accepted designation operation;
receiving the body part information output from the other information processing device;
The program according to claim 1 , further comprising: a step of, when the image displayed on the display unit includes an image part identified from the part information, superimposing and displaying an effect image on the part. The program according to claim 19, wherein the step of accepting the designation operation is capable of accepting the designation operation regardless of whether the image displayed on the display unit is the first image or the second image. A program executed in an information processing device including a processor, a memory, an input unit, and a display unit,
The information processing device includes:
outputting self-portrait information for enabling an image that can be displayed on the display unit in response to a first operation on the input unit to be identified in another information processing device;
receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device;
executing a step of switching, in response to a second operation on the input unit, an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from the other image information received in the receiving step;
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different from each other;
The information processing device includes:
A program that executes a step of performing specific control to switch an image displayed on a display unit of another information processing device to a first image displayed on the information processing device in response to a third operation on the input unit. A program executed in an information processing device including a processor, a memory, an input unit, and a display unit,
The information processing device includes:
outputting self-portrait information for enabling an image that can be displayed on the display unit in response to a first operation on the input unit to be identified in another information processing device;
receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device;
executing a step of switching, in response to a second operation on the input unit, an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from the other image information received in the receiving step;
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different from each other;
The information processing device includes:
receiving a designation operation for an image displayed on the display unit;
outputting part information for enabling identification of the image displayed on the display unit and the part corresponding to the accepted designation operation;
receiving the body part information output from the other information processing device;
and when the image to be displayed on the display unit includes a part of the image identified from the part information, displaying an effect image by superimposing the part. An information processing device including a processor, a memory, an input unit, and a display unit,
a means for outputting self-portrait information for enabling an image that can be displayed on the display unit in response to a first operation on the input unit to be specified in another information processing device;
a means for receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device;
a means for switching an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from other image information received by the receiving means, in response to a second operation on the input unit;
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different from each other;
a means for outputting switching information for enabling a different information processing device to specify that an image displayed on the display unit of the information processing device has been switched to a first image of the different information processing device by the switching means switching to the second image in response to a second operation on the input unit;
The information processing device further includes a means for displaying another specific image corresponding to the other information processing device that has switched to the first image in the information processing device by receiving self-switching information from the other information processing device that specifies that the image to be displayed on the display unit of the other information processing device has been switched to a first image in the information processing device. An information processing device including a processor, a memory, an input unit, and a display unit,
a means for outputting self-portrait information for enabling an image that can be displayed on the display unit in response to a first operation on the input unit to be specified in another information processing device;
a means for receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device;
a means for switching an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from other image information received by the receiving means, in response to a second operation on the input unit;
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different from each other;
The information processing device further includes a means for performing specific control to switch an image displayed on a display unit of another information processing device to a first image displayed on the information processing device in response to a third operation on the input unit. An information processing device including a processor, a memory, an input unit, and a display unit,
a means for outputting self-portrait information for enabling an image that can be displayed on the display unit in response to a first operation on the input unit to be specified in another information processing device;
a means for receiving other image information for enabling identification of an image that can be displayed on a display unit of the other information processing device in response to a first operation on an input unit of the other information processing device;
a means for switching an image to be displayed on the display unit to one of a plurality of types of images including a first image corresponding to a first operation on the input unit of the information processing device and a second image specified from other image information received by the receiving means, in response to a second operation on the input unit;
an image to be displayed in response to a first operation on the input unit of the information processing device and an image to be displayed in response to a first operation on the input unit of the other information processing device may be different from each other;
means for accepting a designation operation for an image displayed on the display unit;
a means for outputting part information for enabling identification of an image displayed on the display unit and a part corresponding to the received designation operation;
A means for receiving part information output from another information processing device;
The information processing device further comprises: a means for, when an image part identified from the part information is included in the image displayed on the display unit, superimposing an effect image on the part and displaying it.

Images