JP2008071313A - REMOTE OPERATION SYSTEM, REMOTE OPERATION DEVICE, REMOTE OPERATION METHOD, REMOTE OPERATION PROGRAM, AND RECORDING MEDIUM - Google Patents

Abstract

An object of the present invention is to check a display screen of a plurality of types of remotely operated devices having a display and a specific input unit in real time, and to remotely control a user interface via a GUI that is common in appearance with a user interface installed on these remotely operated devices. To provide a remote operation system for remotely operating an operation device.
A remotely operated device M1 including a display unit D1 and unique input units HK1 to HK4 that is remotely operated by a remote operation device via a network includes image data of the unique input units HK1 to HK4 and a display unit D1. Image data transmission means S1 for transmitting the image data output to the remote control device, and whether or not the pseudo operation of the specific input units HK1 to HK4 is realized based on the operation content signal transmitted by the remote control device And a control unit 53 for controlling the remote operation device M1 based on a determination result by the determination unit 61.
[Selection] Figure 6

Description

  The present invention relates to a remote operation system, a remote operation device constituting the remote operation system, a remote operation method, a remote operation program, and a recording medium, and in particular, a plurality of types of remote operation devices including a display and a specific input unit. A remote operation system for remotely operating a plurality of types of remotely operated devices on a common application program without using various dedicated application programs corresponding to each, a remotely operated device constituting the remote operation system, and a remote The present invention relates to a remote operation method for an operation device, a remote operation program, and a recording medium storing the program.

  Conventionally, there is a remote controller with a learning function that combines functions of remote controllers attached to various devices such as a television and an air conditioner by assigning and storing infrared signals of a plurality of remote controllers to predetermined hard keys. .

  In addition, there is a soft remote controller that uses a touch panel that is not limited in shape or arrangement, such as a unique input unit, and various operation screens are displayed on a display with a touch panel provided on the remote controller to remotely control various devices. A remote control device that realizes the above is also known (see, for example, Patent Document 1).

  Furthermore, a network device (for example, refer to Patent Document 2) including a web server that distributes an operation screen for realizing remote operation from a host computer as WWW (World Wide Web) content, and an operation panel including a display. There is also known a remote operation system that increases the degree of freedom of operation panel arrangement by separating the main body of the apparatus and connecting them via a wireless local area network (LAN) (see, for example, Patent Document 3).

  However, the above-described remote control device, network device, and remote operation system do not have means for sharing an image displayed by the remotely operated device, and are an OA (Office Automation) device having a unique input unit and a display. It cannot be used for the purpose of remotely assisting the operation of various devices such as a ticket vending machine while confirming the image displayed on the display.

  VNC (Virtual Network Computing) is known as a technology for remotely supporting a remote device while confirming an image displayed on the display of the remote device.

  FIG. 1 is a diagram for explaining the principle of operation of a VNC. A system equipped with a VNC has a screen on a PC (Personal Computer) (hereinafter referred to as a “remotely operated PC”) that is remotely operated. A screen sharing client C is provided on a PC (hereinafter referred to as “remote control PC”) that has a shared server S and is remotely operated.

  Regarding screen sharing, the screen sharing server S of the remotely operated PC includes an image data acquisition unit 11 and an image data transmission unit 12, and the screen sharing client C of the remote operation PC includes an image data receiving unit 15 and an image data display. Part 16.

  The image data acquisition unit 11 of the remotely operated PC acquires image data of a screen displayed on the display of the remotely operated PC, and the image data transmission unit 12 transmits this image data to the screen sharing client C of the remote operation PC. Send. The image data receiving unit 15 of the remote operation PC receives image data transmitted from the remote operation PC, and the image data display unit 16 displays the received image data on the display of the remote operation PC.

  In this way, the screen of the remotely operated PC on which the screen sharing server S is activated can be shared on the remotely operated PC. Note that the size of image data may be compressed using a known compression technique or using difference data from the immediately preceding image data.

  On the other hand, regarding the input operation, the screen sharing server S of the remotely operated PC has the input event execution unit 13 and the input event receiving unit 14, and the screen sharing client C of the remote operation PC includes the input event transmitting unit 17 and the input event. An acquisition unit 18 is included.

  The input event acquisition unit 18 of the remote operation PC acquires an input event signal generated by mouse or keyboard operation input on the remote operation PC, and the input event transmission unit 17 transmits the input event signal to the remote operation PC. To do. The input event receiving unit 14 of the remotely operated PC receives an input event signal transmitted from the remote operated PC, and the input event executing unit 13 receives an input event corresponding to the received input event signal. Run on.

  As described above, the system including the VNC can remotely support the remotely operated PC while confirming the image displayed on the display of the remotely operated PC in the remote place on the display of the remotely operated PC.

  However, unlike a PC having a general-purpose input device such as a keyboard or a mouse on both the remotely operated side and the remote-operated side, the OA does not have a general-purpose input device only having a unique operation panel near the display. Embedded devices such as devices, ticket vending machines, and facility reservation machines cannot use the VNC as described above, and need to use dedicated screen sharing servers and dedicated screen sharing clients corresponding to each of the embedded devices. There is.

  For example, FIG. 2 is a diagram showing a relationship between a facility reservation machine used when reserving a sports facility or the like and a remote operation PC that remotely operates the facility reservation machine. FIG. It is a figure which shows an example of the network with which the facility reservation machine and remote control PC were connected.

  Here, first, the basic configuration of the facility reservation machine will be described with reference to FIGS. 4 and 5.

  FIG. 4 is a diagram showing an example of the appearance of the facility reservation machine. The facility reservation machine M includes a touch panel display D having soft keys (referred to as buttons displayed on the liquid crystal display) SK1 and SK2, and And hard keys HK1 to HK4.

  FIG. 5 is an example of a functional block diagram illustrating a configuration of the dedicated control unit P included in the facility reservation machine M. The dedicated control unit P includes a display D with a touch panel and the facility reservation machine M including the hard keys HK1 to HK4. A program that controls various processes, and includes a soft key input unit 50, a hard key input unit 51, an input processing unit 52, an input control unit 53, a screen generation unit 54, and a screen display unit 55.

  The soft key input unit 50 detects an input operation to the soft key SK1 or SK2 displayed on the display D with a touch panel and outputs an input signal to the input processing unit 52. Similarly, the hard key input unit 51 detects an input operation to any of the hard keys HK1 to HK4 and outputs an input signal to the input processing unit 52.

  The input processing unit 52 receives an input signal from the soft key input unit 50 or the hard key input unit 51, determines which button operation has been performed, and outputs the determination result to the input control unit 53.

  The input control unit 53 causes the facility reservation machine M to execute a predetermined process based on the output of the input processing unit 52, and outputs a screen generation command to the screen generation unit 54 as one of the processes, for example.

  The screen generation unit 54 generates image data to be displayed on the display D in response to a screen generation command from the input control unit 53 and outputs the image data to the screen display unit 55.

  The screen display unit 55 receives the image data from the screen generation unit 54 and causes the display D to display a predetermined screen.

  Next, referring again to FIG. 2, the facility reservation machine M that does not have a general-purpose input device as described above has a plurality of types of facility reservation machines M in addition to the screen sharing server S (see FIG. 1). It is necessary to provide an operation content signal receiving application RA corresponding to the above.

  In addition to the screen sharing client C (see FIG. 1), the remote operation PC needs to include an operation content signal transmission application SA corresponding to each of a plurality of types of facility reservation machines M.

  This is because each type of facility reservation machine M has a unique operation panel with a different number of buttons and operation method.

  For example, the operation content signal transmission application SA generates a GUI (Graphical User Interface) corresponding to the hard key of the facility reservation machine M that cannot be visually recognized by the remote operation PC sharing only the screen, and performs a predetermined GUI for the GUI using a mouse, a keyboard, or the like. When the input event is performed, a predetermined operation content signal is transmitted to the facility reservation machine M.

  For example, the operation content signal reception application RA receives the operation content signal transmitted by the operation content signal transmission application SA and causes the facility reservation machine M to execute a predetermined process.

Therefore, as shown in FIG. 3, if the facility reservation machine M _A ~M _C of connected three types in the network to remotely control, remote control PC, operation content signal transmitted for each type of facility reservation device M The applications SA1 to SA3 need to be activated, and the operation content signal transmission applications SA1 to SA3 need to be switched each time the facility reservation machine M to be remotely operated is changed.
JP 2002-345057 A JP 2002-232635 A JP 2002-366287 A

  However, developing a dedicated operation content signal receiving application for each type of facility reservation machine is inefficient in terms of cost and development period. Also, multiple types of operation content signal transmission applications for each type of facility reservation machine are started up by a remote operation PC, and these multiple types of operation content signal transmission applications are switched for each type of facility reservation machine to be remotely operated. It is complicated.

  In addition, the user needs to learn two types of operation methods, when the facility reservation machine is directly operated and when the facility reservation machine is remotely operated using a remote operation PC.

  In view of the above points, the present invention has a common external appearance with the user interface installed on the remotely operated devices while allowing the display screens of the plurality of types of remotely operated devices including the display and the unique input unit to be confirmed in real time. A remote operation system for remotely operating a remote operation device via a GUI, a remote operation device constituting the remote operation system, a remote operation method for the remote operation device, a remote operation program, and a remote operation program thereof An object is to provide a recording medium for recording.

  Another object of the present invention is to provide means for remotely operating a plurality of types of remotely operated devices with a single remote operation software.

  Another object of the present invention is to reduce the development cost of a plurality of types of remotely operated devices that can be remotely operated by sharing an application program installed in the remotely operated device.

  Another object of the present invention is to provide means for remotely operating a remote operation device while displaying the same GUI as the user interface installed in the remote operation device on the remote operation device in real time.

  In order to achieve the above object, a remote operation system according to a first aspect of the present invention is a remote operation system that remotely operates a plurality of types of remote operation devices having a display unit and a specific input unit and a remote operation device via a network. A remote operation system including an operation device, wherein the remote operation device includes image data transmission means for transmitting image data of the specific input unit and image data output to the display unit to the remote operation device; Determining means for determining whether or not the operation for the specific input unit is to be realized in a pseudo manner based on an operation content signal transmitted from the remote operation device; and the remote operation device based on a determination result by the determination means And an input control means for controlling the remote control device, wherein the remote operation device receives the image data transmitted by the remote operation device and displays the unique input portion image and the remote operation screen image. Image data display means to be displayed, and operation content signal transmission means for transmitting an operation content signal indicating the content of the operation to the remote operation device when an operation is performed on the displayed unique input unit image. It is characterized by that.

  The remote control device according to the second invention is a remote control device including a display unit and a specific input unit that is remotely operated by the remote control device via a network, and is an image of the specific input unit. Image data transmitting means for transmitting data and image data output to the display unit to a remote operation device, and an operation for the specific input unit is realized in a pseudo manner based on an operation content signal transmitted by the remote operation device Determination means for determining whether or not, and input control means for controlling the remote operation device based on a determination result by the determination means.

  The third invention is a remotely operated device according to the second invention, and includes a dedicated control unit unique to each of the plurality of remotely operated devices and a common control unit common to the plurality of remotely operated devices. And the common control unit realizes an operation on the specific input unit in a pseudo manner based on a specific input unit image generation unit that generates image data of the specific input unit and an operation content signal transmitted by the remote operation device. And determining means for determining whether or not to perform the operation.

  The fourth invention is a remotely operated device according to the second invention, and includes a dedicated control unit unique to each of the plurality of remotely operated devices and a common control unit common to the plurality of remotely operated devices. And the dedicated control unit artificially realizes an operation on the specific input unit based on an operation content signal transmitted from a specific input unit image generation unit that generates image data of the specific input unit and the remote operation device. And determining means for determining whether or not to perform the operation.

  The fifth invention is a remotely operated device according to the second invention, and includes a dedicated control unit unique to each of the plurality of remotely operated devices and a common control unit common to the plurality of remotely operated devices. And the dedicated control unit simulates an operation on the specific input unit based on an operation content signal transmitted from the specific input unit image generation unit that generates image data of the specific input unit or the remote operation device. It has the determination means which determines whether to implement | achieve.

  The sixth invention is a remotely operated device according to the second invention, wherein the image data transmitted by the other remotely operated device is received and the unique input portion image and the others in the other remotely operated device are received. When an operation is performed on the image data display means for displaying the remote operation screen image in the remote operation device on the display unit and the unique input unit image on the other remote operation device displayed on the display unit And an operation content signal transmitting means for transmitting an operation content signal to the other remotely operated device.

  A seventh invention is a remotely operated device according to any one of the second to sixth inventions, wherein the display unit is a display with a touch panel.

  A remote operation method according to the eighth invention is a remote operation method of a plurality of types of remotely operated devices that are remotely operated by a remote operation device via a network and are provided with a display unit and a specific input unit, Based on the image data transmission step of transmitting the image data of the unique input unit and the image data output to the display unit from the remote operation device to the remote operation device, and the image data transmitted in the image data transmission step When the operation on the unique input portion image displayed on the remote operation device is performed, the operation on the unique input portion is simulated based on the operation content signal transmitted from the remote operation device to the remote operation device. And a control step for controlling the remote operation device based on a determination result in the determination step. Characterized in that it has Tsu and up, the.

  Further, the remote operation program according to the ninth invention is a remote operation program for a plurality of types of remotely operated devices having a display unit and a specific input unit, which are remotely operated by a remote operation device via a network, An image data transmission step for transmitting the image data of the unique input unit and the image data output to the display unit from the remote operation device to the remote operation device, and the image data transmitted in the image data transmission step. Based on an operation content signal transmitted from the remote operation device to the remote operation device when an operation is performed on the unique input portion image displayed on the remote operation device based on the operation content signal, A determination step for determining whether or not to realize the control, and controlling the remotely operated device based on a determination result in the determination step Remote operation program characterized by comprising: a control step that, a.

  The tenth invention is a computer-readable recording medium recording the remote control program according to the ninth invention.

  By the above-mentioned means, the present invention has the same appearance as the user interface installed on the remotely operated devices while allowing the display screens of the plurality of types of remotely operated devices having the display and the unique input unit to be confirmed in real time. REMOTE OPERATION SYSTEM FOR REMOTE OPERATION OF REMOTE OPERATION DEVICE THROUGH GUI, REMOTE OPERATION DEVICE CONSTRUCTING THE REMOTE OPERATION SYSTEM, REMOTE OPERATION METHOD FOR REMOTE OPERATION DEVICE, REMOTE OPERATION PROGRAM, AND RECORDING THE REMOTE OPERATION PROGRAM A recording medium can be provided.

  In addition, the user can remotely operate a plurality of types of remotely operated devices using a single PC software for remote operation.

  In addition, the manufacturer can reduce the development cost of a plurality of types of remotely operated devices that can be remotely operated by sharing an application program installed in the remotely operated device.

  In addition, the user can remotely operate the remote operation device using the remote operation PC software with the same GUI and the same operation method as directly operating the remote operation device. You don't have to learn two ways of operation.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 6 is a diagram illustrating a configuration example of a remotely operated device in the remote operation system according to the present invention. The remotely operated device M1 includes a dedicated control unit P1, a screen sharing server S1, and a communication unit T. To a remote control PC as shown in FIG.

  The dedicated control unit P1 is a computer including a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), an NVRAM (Non-Volatile Random Access Memory), etc., and a program stored in the ROM Is expanded in the RAM and executed by the CPU, thereby executing processing unique to each remotely operated device such as ticket issuing processing and facility reservation processing.

  Further, the dedicated control unit P1 is different from the dedicated control unit P of the facility reservation machine M shown in FIG. 5 in the screen generation unit 60 and the input processing unit 61. Other components are the same as the dedicated control unit P in FIG.

  The screen generation unit 60 is a means for acquiring image data to be transmitted to the remote operation PC, and includes a screen image generation unit 600 and a hard key image generation unit 601, and the screen image generation unit 600 is remotely operated. A screen image to be displayed on the display D1 of the device M1 is generated, and the hard key image generation unit 601 generates a hard key image representing the appearance of the hard key group of the remote operation device M1.

  7 is a diagram illustrating an example of an image generated by the screen generation unit 60. FIG. 7A illustrates a screen image generated by the screen image generation unit 600, and FIG. 7B illustrates a hard key image generation unit. FIG. 7C illustrates a remote operation image displayed in a single window on the remote operation PC.

  The screen image shown in FIG. 7A is the same as the screen image displayed on the display D1 of the remote operation device M1, and the hard key image shown in FIG. 7B is the hard key HK1 of the remote operation device M1. To HK4.

  The hard key image shown in FIG. 7B may be registered in the ROM in advance or may be generated at any time, but is read from the ROM or generated only for transmission to the remote control PC. Therefore, it is not displayed on the display D1 of the remotely operated device M1.

  The hard key image preferably has the same size (including similar shape), shape, and arrangement as the physical hard key, but the image corresponding to each of the operable hard keys can be identified. As long as they are arranged, the size, shape, and arrangement may be different.

  For example, a power button (hard key) for turning ON / OFF the power of the remotely operated device M1 may not be included in the hard key image.

  7C is an image obtained by combining the screen image of FIG. 7A and the hard key image of FIG. 7B. As described above, the screen generation unit 60 generates the remote operation image by arranging the screen image and the hard key image in the front-rear direction or the left-right direction, and outputs the remote operation image to the screen sharing server S1.

  The input processing unit 61 is a means for performing a pseudo operation input to the remote operation device M1 based on the operation content signal received from the remote operation PC. The determination unit 610, the soft key input generation unit 611, and the hardware A key input generation unit 612 is included.

  The “operation content signal” is a signal including the content of operation input on the remote operation PC, and includes, for example, a character string input with a keyboard and a coordinate point clicked with a mouse.

  Based on the operation content signal received from the remote operation PC, the determination unit 610 determines whether the operation input on the remote operation PC is for the hard key group of the remote operation device M1 or the soft key group of the touch panel display D1 in the remote operation device M1. It is determined whether or not.

  When the determination unit 610 determines that the operation input on the remote operation PC is for the display D1 with the touch panel in the remote operation device M1, the soft key input generation unit 611 is software corresponding to the coordinates indicated by the operation content signal. This is means for executing the process assigned to the key.

  The soft key input generation unit 611 specifies, for example, a soft key including coordinates indicated by the operation content signal, and an input generated when the soft key is actually pressed on the touch panel display D1 in the remote operation device M1. The same pseudo input signal as the signal is generated and output to the input control unit 53.

  When the determination unit 610 determines that the operation input on the remote operation PC is to the hard key of the remote operation device M1, the hard key input generation unit 612 sets the hard key input generation unit 612 to the hard key corresponding to the coordinates indicated by the operation content signal. This is means for executing the assigned processing.

  The hard key input generation unit 612 specifies, for example, a hard key including coordinates indicated by the operation content signal, and the same pseudo input as the input signal generated when the hard key is actually pressed in the remote operation device M1. A signal is generated and output to the input control unit 53.

  The screen sharing server S1 is server software that is generally used in a plurality of types of remotely operated devices. For example, the screen sharing server S1 is the screen sharing server S of FIG. 1, and remotely controls the image data generated by the screen generation unit 60. It is shared by the PC and the remotely operated device M1.

  The screen sharing server S1 receives a signal transmitted by the screen sharing client C of the remote operation PC and converts it into a signal that can be handled by the input processing unit 52 of the dedicated control unit P.

  The communication unit T is software for controlling communication between the remote operation device M1 and the remote operation PC, for example, HTTP (Hyper Text Transfer Protocol) over TCP / IP (Transmission Control Protocol / Internet Protocol). Controls communication using FTP or File Transfer Protocol (FTP).

  The dedicated control unit P1 may be configured by software, and the screen sharing server S1 and the communication unit T may be configured by electronic circuits.

  Next, the flow of processing in which the remote operation PC remotely operates the remote operation device M1 will be described with reference to FIGS.

  FIG. 8 is a sequence diagram showing the flow of the remote operation process, showing the passage of time from the top to the bottom of the figure, the process by the remote operation device M1 on the left side, and the process by the remote operation PC on the right side.

  First, when receiving a signal requesting image data from the remote operation PC, the remotely operated device M1 receives the screen image and the hard key image in the screen image generation unit 600 and the hard key image generation unit 601 in the screen generation unit 60, respectively. Generate (step ST1).

  Next, the remotely operated device M1 transmits image data including the generated screen image and hard key image to the remote operation PC (step ST2).

  The remote operation PC receives image data from the remote operation device M1 (step ST3), and displays a remote operation image (see FIG. 7C) including a screen image and a hard key image on the display of the remote operation PC. (Step ST4).

  Next, the remote operation PC waits until an operation input is performed on the image for remote operation via a mouse or a keyboard by the remote operator, and when the operation input is performed, the content of the character string or mouse input by the keyboard Is acquired as the clicked coordinate point (step ST5), and the content is transmitted as an operation content signal to the remotely operated device M1 (step ST6).

  The remote operation device M1 receives the operation content signal from the remote operation PC (step ST7), determines the content of the operation input indicated by the operation content signal (step ST8), and performs a corresponding process on the remote operation device M1. It is executed (step ST9).

  FIG. 9 is a flowchart showing a flow of processing for determining the content of the operation input indicated by the operation content signal received from the remote operation PC.

  First, the determination unit 610 of the input processing unit 61 in the remotely operated device M1 acquires an operation content signal transmitted by the remote operation PC via the screen sharing server S1, and performs remote operation based on the coordinates indicated by the operation content signal. It is determined whether or not the position coordinate clicked with the PC mouse is within the hard key image area (step ST11).

  The “hard key image area” refers to an area including one or more hard key images, and includes, for example, the entire area shown in FIG.

  The operation content signal transmitted from the remote operation PC includes the coordinate points of the four corners of the remote operation image and the coordinate points of the four corners of the hard key image area (both are the coordinates in the remote operation PC) (various types). It may be possible to include values indicating the coordinates and height and width of the upper left corner of the region.), So that the relative relationship between the position clicked with the mouse of the remote operation PC and the image for remote operation can be recognized. To do.

  When it is determined that the position coordinates clicked with the mouse are within the hard key image area (YES in step ST11), the determination unit 610 determines whether or not a hard key image exists at the position coordinates clicked with the mouse. Determine (step ST12).

  When a hard key image exists at the position coordinate clicked with the mouse (YES in step ST12), the determination unit 610 uses the hard key input generation unit 612 to input a pseudo input signal corresponding to the hard key (image) to the input control unit. It outputs to 53 and performs a predetermined | prescribed process (step ST13).

  If the hard key image does not exist at the position coordinate clicked with the mouse (NO in step ST12), the determination unit 610 ends the operation input content determination process without causing the input control unit 53 to output the pseudo input signal. Let

On the other hand, when it is determined that the position coordinates clicked with the mouse are not within the hard key image area (NO in step ST11), the determination unit 610 determines that the position coordinates are within the screen image area and softkeys the position coordinates. It is determined whether an image exists (step ST14).
The “screen image area” refers to an area including one or more soft key images, and includes, for example, the entire area shown in FIG.

  When a soft key image exists at the position coordinate clicked with the mouse (YES in step ST14), the determination unit 610 uses the soft key input generation unit 611 to input a pseudo input signal corresponding to the soft key (image) to the input control unit. It outputs to 53 and performs a predetermined | prescribed process (step ST15).

  If no soft key image exists at the position coordinate clicked with the mouse (NO in step ST14), the determination unit 610 ends the operation input content determination process without causing the input control unit 53 to output a pseudo input signal. Let

  As described above, in the remote operation system according to the first embodiment, when an operation input (for example, click operation with a mouse) is performed on a hard key image on the remote operation PC, an operation on the hard key in the remote operation device M1 is performed. The remote control device M1 can be made to execute the same processing as when the input (for example, pressing) is actually performed.

  For example, in the remote operation system of the first embodiment, when the reset key image (see FIG. 7B) in the hard key image area is clicked with the mouse on the remote operation PC, the hard key in the remote operation device M1 is displayed. The remote control device M1 is caused to execute the same reset process as when HK3 (see FIG. 4) is pressed.

  In the remote operation system according to the present invention, the display D1 of the remote operation device M1 includes a touch panel, and an operation input (for example, click operation with a mouse) is performed on a screen image on the remote operation PC. In this case, it is possible to cause the remote operation device M1 to execute the same processing as when an operation input (for example, touch input) to a soft key on the touch panel display D1 of the remote operation device M1 is actually performed.

  Further, the manufacturer of the remote control system according to the present invention does not design the screen sharing client C in the remote control PC so as to correspond to each of the multiple types of remote control devices, and is common to the multiple types of remote control devices. By designing to do so, the development cost of a plurality of types of remotely controlled devices can be reduced. The sharing of the screen sharing client C is enabled by displaying a GUI (see FIG. 7C) that has the same or approximate appearance as a hard key that is uniquely installed in each of a plurality of types of remotely controlled devices. .

  In addition, the user of the remote operation system according to the present invention can perform the same operation method as when directly operating the user interface installed in the remote operation device via the GUI on the remote operation PC (see FIG. 7C). Since it can be used, it is not necessary to learn two kinds of operation methods in the case of direct operation and remote operation, and it is only necessary to understand the one way of operation.

  Next, another embodiment of the remote control system according to the present invention will be described.

  FIG. 10 is a diagram showing another configuration example of the remotely operated device in the remote operation system according to the present invention. The remotely operated device M2 includes a dedicated control unit P2, a screen sharing server S2, and a communication unit T. The remote control PC as shown in FIG. 1 is connected via the network.

  Further, in the remote operation device M2, the image data acquisition unit 70 of the screen sharing server S2 includes a screen image acquisition unit 700 and a hard key image generation unit 701 as compared with the remote operation device M1 shown in FIG. The difference is that the input event execution unit 71 of the shared server S2 includes a determination unit 710, a soft key input generation unit 711, and a hard key input generation unit 712. Other components are common to the remotely operated device M1 of FIG.

  The image data acquisition unit 70 is a means for acquiring a remote operation image to be transmitted to the remote operation PC, and includes a screen image acquisition unit 700 and a hard key image generation unit 701. A screen image (for example, see FIG. 7A) displayed on the display D2 of the remote operation device M2 is acquired from the screen generation unit 54 of the remote operation device M2, and the hard key image generation unit 701 performs the remote operation. A hard key image representing the appearance of the hard key group of the device M2 is generated.

  The screen image acquisition unit 700 acquires from the screen generation unit 54 the same image data as the image data output to the screen display unit 55 by the screen generation unit 54 of the remotely operated device M2.

  The hard key image generation unit 701 refers to a signal for identifying the type of the remote operation device M2 from the screen generation unit 54 of the remote operation device M2 (type ID indicating the type of the remote operation device M2, etc. From the image database stored in advance in the ROM or the like and managed by the screen sharing server S2, the hardware corresponding to the hard key group on the operation panel of the remote operation device M2 is received. A key image group (see, for example, FIG. 7B) is read.

  Note that the hard key image group corresponding to the hard key group on the operation panel of the remote operation device M2 may be set in advance when the screen sharing server S2 is incorporated into the dedicated control unit P2.

  As described above, the image data acquisition unit 70 performs the remote operation image based on the screen image acquired by the screen image acquisition unit 700 and the hard key image generated by the hard key image generation unit 701 (for example, FIG. 7C). Reference.)

  Further, by storing in advance in the image database hard key images corresponding to the hard keys possessed by each of the plurality of types of remotely operated devices, the screen sharing server S2 can be used universally with a plurality of types of remotely operated devices. Can be used.

  The input event execution unit 71 emulates an input signal output from the soft key input unit 50 or the hard key input unit 51 based on the operation content signal transmitted from the remote operation PC and received by the input event reception unit 14. A determination unit 710, a soft key input generation unit 711, and a hard key input generation unit 712.

  “Emulate” means that operation input of a hard key or soft key is realized in a pseudo manner by another method (for example, operation input by a mouse or a keyboard).

  The determination unit 710 receives the operation content signal received by the input event reception unit 14, and the operation input by the mouse or the keyboard in the remote operation PC is for any of the soft key in the display D2 with a touch panel or the hard key in the operation panel. Determine whether.

  The determination unit 710 determines whether or not the position coordinates included in the operation content signal exist within the screen image area or the hard key image area. If the position coordinates exist within the screen image area, It is assumed that the operation input for the key image has been performed, and if the position coordinates exist in the hard key image area, the operation input for the hard key image has been performed.

  When the determination unit 710 determines that an operation input to the screen image area has been performed, the soft key input generation unit 711 determines whether any soft key image exists in the position coordinates included in the operation content signal. Is generated, and a soft key pseudo input signal corresponding to the soft key image at the position coordinate is generated and output to the input processing unit 52 of the dedicated control unit P2.

  Similarly to the soft key input generation unit 711, the hard key input generation unit 712 displays any position coordinates included in the operation content signal when the determination unit 710 determines that an operation input to the hard key image area has been performed. It is determined whether or not such a hard key image exists, and a hard key pseudo input signal corresponding to the hard key image at the position coordinate is generated and output to the input processing unit 52 of the dedicated control unit P2.

  Note that the soft key input generation unit 711 and the hard key input generation unit 712 do not generate a pseudo input signal when it is determined that no soft key image or hard key image exists in the position coordinates included in the operation content signal. This is to emulate the soft key or hard key operation input (key press) only when the soft key image or hard key image in the remote operation image is correctly input (clicked with the mouse).

  With this configuration, the remote operation system replaces the screen sharing server S2 having the image data acquisition unit 70 and the input event execution unit 71 with the dedicated control unit P of FIG. 5 without making any changes to the dedicated control unit P of FIG. The remote operation of the remotely operated device M2 by the remote operation PC can be realized only by connecting.

  Next, still another embodiment of the remote control system according to the present invention will be described.

  FIG. 11 is a diagram showing still another configuration example of the remotely operated device in the remote operation system according to the present invention. The remotely operated device M3 includes a dedicated control unit P3, a screen sharing server S3, and a communication unit T. Then, it is connected to a remote control PC as shown in FIG.

  Further, the remote operation device M3 is different from the remote operation device M1 shown in FIG. 6 in that the image data acquisition unit 70 of the screen sharing server S3 includes a screen image acquisition unit 700 and a hard key image generation unit 701. However, it is different from the remotely operated device M2 shown in FIG. 10 in that the input processing unit 61 of the dedicated control unit P3 includes a determination unit 610, a soft key input generation unit 611, and a hard key input generation unit 612.

  With this configuration, the remote operation system changes the input processing unit 52 in the dedicated control unit P of FIG. 5 to be the dedicated control unit P3 and uses the screen sharing server S3 having the image data acquisition unit 70 as the dedicated control unit P3. The remote operation of the remotely operated device M3 by the remote operation PC can be realized only by adding to the above.

  Further, this configuration has a greater degree of change with respect to the dedicated control unit P3 than the remote control device M2 of FIG. 10 (no change is made to the dedicated control unit P of FIG. 5). Compared with the operating device M1 (the input processing unit 52 and the screen generation unit 54 in the dedicated control unit P in FIG. 5 are changed), the degree of change with respect to the dedicated control unit P3 can be suppressed.

  Next, still another embodiment of the remote control system according to the present invention will be described.

  FIG. 12 is a diagram showing still another configuration example of the remotely operated device in the remote operation system according to the present invention. The remotely operated device M4 includes a dedicated control unit P4, a screen sharing server S4, and a communication unit T. Then, it is connected to a remote control PC as shown in FIG.

  Further, in the remote operation device M4, as compared with the remote operation device M1 shown in FIG. 6, the input event execution unit 71 of the screen sharing server S4 has a determination unit 710, a soft key input generation unit 711, and a hard key input generation unit 712. Compared with the remotely operated device M2 shown in FIG. 10, the screen generation unit 60 of the dedicated control unit P4 is different in that it includes a screen image generation unit 600 and a hard key image generation unit 601.

  With this configuration, the remote operation system changes the screen generation unit 54 in the dedicated control unit P of FIG. 5 to be the dedicated control unit P4, and uses the screen sharing server S4 having the input event execution unit 71 as the dedicated control unit P4. The remote operation of the remotely operated device M4 by the remote operation PC can be realized only by adding to the above.

  Further, this configuration has a greater degree of change with respect to the dedicated control unit P3 than the remote control device M2 of FIG. 10 (no change is made to the dedicated control unit P of FIG. 5). Compared with the operating device M1 (the input processing unit 52 and the screen generation unit 54 in the dedicated control unit P in FIG. 5 are changed), the degree of change with respect to the dedicated control unit P3 can be suppressed.

  Next, still another embodiment of the remote control system according to the present invention will be described.

  FIG. 13 is a diagram showing still another configuration example of the remotely operated device in the remote operation system according to the present invention. The remotely operated device M5 includes a dedicated control unit P5, a screen sharing server S5, and a communication unit T. Then, it is connected to a remote control PC as shown in FIG.

  Further, in the remotely operated device M5, the display D5 does not include a touch panel as compared with the remotely operated device M1 shown in FIG. 6 (therefore, the soft key input unit 50 (see FIG. 5) is not provided). ), Except that the input processing unit 130 includes only the determination unit 1300 and the hard key input generation unit 1301.

  With this configuration, the remote operation system can realize remote operation by the remote operation PC, similarly to the remote operation device having the touch panel, even when the remote operation device M5 does not have the touch panel.

  Next, still another embodiment of the remote control system according to the present invention will be described.

  FIG. 14 is a diagram showing still another configuration example of the remotely operated device in the remote operation system according to the present invention. The remotely operated device M6 includes a dedicated control unit P6, a screen sharing server S6, and a communication unit T. Then, it is connected to a remote control PC as shown in FIG.

  Further, the remotely operated device M6 is different from the remotely operated device M1 shown in FIG. 6 in that a remote controller R is provided instead of the hard key input unit 51.

  The remote controller R is, for example, an infrared remote controller provided with hard keys.

  15 is a diagram illustrating an example of an image generated by the screen generation unit 60. FIG. 15A illustrates a screen image generated by the screen image generation unit 600, and FIG. 15B illustrates a hard key image generation unit. FIG. 15C shows a remote operation image displayed in a single window on the remote operation PC.

  The screen image shown in FIG. 15A is common to the screen image displayed on the display D6 of the remotely operated device M6, and the hard key image shown in FIG. 15B is common to the appearance of the remote controller R.

  15C is an image obtained by combining the screen image of FIG. 15A and the hard key image of FIG. 15B. As described above, the screen generation unit 60 generates the remote operation image by arranging the screen image and the hard key image in the front-rear direction or the left-right direction, and outputs the remote operation image to the screen sharing server S6.

  When the hard key image “1” of the remote operation image shown in FIG. 15C is clicked with the mouse, the input processing unit 61 acquires the operation content signal via the screen sharing server S6, and the determination unit 610 It is determined that the hard key image area corresponding to the hard key of the remote controller R is clicked, and the pseudo input signal indicating that the hard key “1” of the remote controller R is pseudo pressed by the hard key input generation unit 612 Is output to the input control unit 53.

  After that, the input control unit 53 that has received the pseudo input signal causes the process assigned to the hard key “1” of the remote controller R to be executed.

  With this configuration, the remote operation system includes the remote operation device having the operation panel even when the remote operation device M6 has only the remote controller R instead of the operation panel (including hard keys) provided. Similarly, remote operation by a remote operation PC can be realized.

  Further, the manufacturer of the remote control system according to the present invention does not design the screen sharing client C in the remote control PC so as to correspond to each of the multiple types of remote control devices, and is common to the multiple types of remote control devices. By designing to do so, the development cost of a plurality of types of remotely controlled devices can be reduced. The screen sharing client C can be shared by displaying a GUI (see FIG. 15C) having the same or similar appearance as the remote controller attached to each of a plurality of types of remotely controlled devices.

  Further, the user of the remote operation system according to the present invention can perform the same operation method as when directly operating the user interface installed in the remote operation device via the GUI on the remote operation PC (see FIG. 15C). Since it can be used, it is not necessary to learn two kinds of operation methods in the case of direct operation and remote operation, and it is only necessary to understand the one way of operation.

  Next, still another embodiment of the remote control system according to the present invention will be described.

  FIG. 16 is a diagram showing still another configuration example of the remotely operated device in the remote operation system according to the present invention. The remotely operated device M7 has a dedicated control unit P7, a screen sharing server S7, and a communication unit T. Then, it is connected to a remote control PC as shown in FIG.

  Further, the remotely operated device M7 is different from the remotely operated device M1 shown in FIG. 6 in that it includes a screen sharing client C7.

  The remote operation device M7 is, for example, an MFP (Multi Function Printer). The image data receiving unit 15 in the screen sharing client C7 receives screen images and hard key images of other MFPs, and the image data display unit 16 uses the dedicated remote operation device M7. Image data is output to the screen display unit 55 of the control unit P7 to display a remote operation image on the display D7.

  Further, the remote operation device M7 outputs an input event signal from the soft key input unit 50 to the input event acquisition unit 18 in the screen sharing client C7, and sends an operation content signal to another MFP via the input event transmission unit 17. The other MFP is remotely operated by transmitting and emulating the soft key or hard key in the other MFP.

  With this configuration, the remotely operated device M7 can be remotely operated from a remote operation PC or another MFP, and can also remotely operate another MFP.

  As described above, each of the remote operation systems according to the present invention shown in the plurality of embodiments is provided with a VNC without providing a dedicated operation content signal transmission application corresponding to each of a plurality of types of remotely operated devices. By providing the common screen sharing client as described above in the remote operation PC, a plurality of types of remotely operated devices can be remotely operated from the remote operation PC.

  Each of the remote operation systems shown in the plurality of embodiments emulates the soft key and the hard key in the remote operation device by the remote operation PC by software, so that the single remote operation system does not depend on the type of the remote operation device. Remote operation of multiple types of remotely operated devices by a screen sharing client is enabled.

  Each of the remote operation systems shown in the plurality of embodiments is configured such that an operation panel installed on the remote operation device or an image displayed on the display screen of the remote operation device can be confirmed in real time by the remote operator. The remote control device can be remotely controlled via a GUI that has the same appearance as the remote controller. If there is experience that the remote operator has input the actual remote control device, a new operation method can be remotely controlled. It is possible to remotely operate the remote operation device without requiring the user to remember.

  Further, the manufacturer of each remote operation system shown in the plurality of embodiments does not design the screen sharing client C in the remote operation PC to correspond to each of the plurality of types of remotely operated devices, and does not design multiple types of devices. By designing to be common to the remote control devices, it is possible to reduce the development cost of multiple types of remote control devices. The sharing of the screen sharing client C is achieved by using a GUI (FIG. 7C) or FIG. 15C (C) that has the same or similar appearance as a hard key or an attached remote controller that is uniquely installed in each of a plurality of types of remotely controlled devices. ))) Is displayed.

  Further, the user of each remote operation system shown in the plurality of embodiments uses the same operation method as that in the case of directly operating the user interface installed in the remote operation apparatus on the GUI on the remote operation PC (FIG. 7C). Or (see FIG. 15 (C)), it is not necessary to learn two ways of operation in the case of direct operation and in the case of remote operation, just to understand the way of operation. Good.

  As mentioned above, although the Example of this invention was explained in full detail, this invention is not limited to the above specific embodiment, In the range of the summary of this invention described in the claim, Various modifications or changes can be made.

  For example, in the above-described embodiment, the remote operation is executed while displaying the hard key image of either the hard key arranged on the remote controller or the hard key arranged on the operation panel on the remote operation PC. For the remote operation device having both the operation panel and the remote operation device, the remote operation may be realized while displaying both hard key images on the remote operation PC.

  The remote operation PC may be a dedicated remote operation terminal, and may be a general-purpose desktop PC, notebook PC, mobile phone, PDA (Personal Digital Assistant), or the like.

  Further, the remote operation system according to the present invention may compress the image data transmitted from the remote operation device to the remote operation PC by a known compression method or may be encrypted by a known encryption method. Even if the image for remote operation displayed on the remote operation PC (see FIG. 7C) is adjusted to an arbitrary window size, the display magnification of the screen image and the hard key image can be set independently. Good.

  Further, the remote operation system prohibits remote operation on a soft key or hard key to which a specific function is assigned, or performs remote operation on a soft key or hard key to which a specific function is assigned after authenticating the remote operator. May be allowed.

  Further, the remote operation device may be an OA device such as an MFP, a printer, a FAX, or a scanner. In this case, the remote operation system according to the present invention is a trouble support system for dealing with a user's trouble, and a device for the user. It is used as a user support system that teaches how to use and a maintenance system that maintains a system built by the user.

  Further, the remotely operated device may be a ticket vending machine, a reservation machine, etc. In this case, the remote operation system according to the present invention concentrates reservations for hospital consultations and public facilities such as gymnasiums and tennis courts. Used as a public facility reservation management system to manage.

  Further, the remote operation device may be an AV (Audio Visual) device such as a DVD recorder or a hard disk recorder. In this case, the remote operation system according to the present invention includes a video displayed on a television and a hardware of a remote controller. Used as a system for remotely operating a TV while displaying key images on a PC or mobile phone. For example, if there is not enough free space on the hard disk, a part of the recorded program can be deleted by remote operation before new recording. Used for operations such as making reservations.

  Further, the remote operation system according to the present invention may be used in a system for remotely operating a video device such as a projector or a DVD player, or an audio device such as a stereo, CD player, MD player, or portable music player via a network. .

  In addition, the remote control system according to the present invention may be used as a support system for handling a mobile phone failure or supporting operation for elderly people, or locking a mobile phone lost by remote control or using a GPS signal. The present invention may be used for a service for finding a lost mobile phone by acquiring location information of the lost mobile phone or performing a photo shoot by remote control.

  The remote operation system according to the present invention may be used as a home appliance remote operation system for remotely operating home appliances such as an air conditioner, a washing machine, and a microwave oven.

  FIG. 17 is a diagram illustrating a configuration example of a public facility reservation management system, and FIG. 18 is a diagram illustrating a configuration example of a home appliance remote control system.

  As shown in FIG. 17, the public facility reservation management system includes three types of facility reservation machines installed in public facilities, a copy from a remote control PC installed in a management center located physically away from the public facilities. The machine, ticket vending machine, security device, air conditioner, etc. will be able to be remotely controlled centrally.

  Further, as shown in FIG. 18, the home appliance remote control system is a remote control PC or mobile phone installed in a company or Internet cafe, a DVD recorder at home, an air conditioner, or a bathroom that manages hot water supply for a bath. Allows centralized remote control of management devices.

It is a figure for demonstrating the operating principle of VNC. It is a figure which shows the relationship between a facility reservation machine and the remote control PC which operates the facility reservation machine remotely. It is a figure which shows an example of the network with which multiple types of facility reservation machines and remote control PC were connected. It is a figure which shows an example of the external appearance of a facility reservation machine. It is a functional block diagram which shows the structural example of the exclusive control part which a facility reservation machine has. It is FIG. (1) which shows the structural example of the remotely operated apparatus in the remote control system which concerns on this invention. FIG. 6 is a first diagram illustrating an example of an image generated by a screen generation unit. It is a sequence diagram which shows the flow of the remote operation process in the remote operation system which concerns on this invention. It is a flowchart which shows the flow of the process which determines the content of the operation input which the operation content signal received from remote control PC shows. It is FIG. (2) which shows the structural example of the remotely operated apparatus in the remote control system which concerns on this invention. It is FIG. (The 3) which shows the structural example of the remotely operated apparatus in the remote control system which concerns on this invention. It is FIG. (4) which shows the structural example of the remotely operated apparatus in the remote control system which concerns on this invention. It is FIG. (5) which shows the structural example of the remotely operated apparatus in the remote control system which concerns on this invention. It is FIG. (6) which shows the structural example of the remotely operated apparatus in the remote control system which concerns on this invention. It is FIG. (2) which shows the example of an image which a screen production | generation part produces | generates. It is FIG. (7) which shows the structural example of the remotely operated apparatus in the remote control system which concerns on this invention. It is a figure which shows the structural example of a public facility reservation management system. It is a figure which shows the structural example of a household appliance remote control system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11, 70 Image data acquisition part 12 Image data transmission part 13, 71 Input event execution part 14 Input event reception part 15 Image data reception part 16 Image data display part 17 Input event transmission part 18 Input event acquisition part 50 Soft key input part 51 Hard key input unit 52, 61, 130 Input processing unit 53 Input control unit 54, 60 Screen generation unit 55 Screen display unit 600 Screen image generation unit 601, 701 Hard key image generation unit 610, 710, 1300 Determination unit 611, 711 Software Key input generation unit 612, 712, 1301 Hard key input generation unit 700 Screen image acquisition unit C, C7 Screen sharing client D, D1-D7 Display HK1-HK4 Hard key M, M _A -M _C , M1-M7 Remote operation Device P, P1 to P7 Dedicated controller RA In operation Content signal receiving application S, S1 to S7 Screen sharing server SA, SA1 to SA3 Operation content signal transmitting application SK1, SK2 Soft key T Communication unit

Claims (10)

A remote operation system having a plurality of types of remotely operated devices having a display unit and a unique input unit, and a remote operation device for remotely operating the remotely operated device via a network,
The remote operation device includes image data transmission means for transmitting image data of the specific input unit and image data output to the display unit to the remote operation device, and an operation content signal transmitted by the remote operation device. A determination unit that determines whether or not the operation for the specific input unit is to be realized in a pseudo manner, and an input control unit that controls the remote operation device based on a determination result by the determination unit,
The remote operation device receives image data transmitted from the remote operation device and displays image data display means for displaying a unique input portion image and a remote operation screen image, and an operation on the displayed unique input portion image. An operation content signal transmitting means for transmitting an operation content signal indicating the content of the operation to the remote operation device when performed,
A remote control system characterized by that. A remotely operated device having a display unit and a specific input unit that is remotely operated by a remote operation device via a network,
Image data transmitting means for transmitting image data of the unique input unit and image data output to the display unit to a remote control device;
Determining means for determining whether or not the operation for the specific input unit is to be realized in a pseudo manner based on an operation content signal transmitted by the remote operation device;
Input control means for controlling the remotely operated device based on a determination result by the determination means;
A remotely operated device characterized by comprising: A dedicated control unit unique to each of the plurality of remotely operated devices and a common control unit common to the plurality of remotely operated devices,
Whether the common control unit realizes an operation on the specific input unit in a pseudo manner based on an operation content signal transmitted from the specific input unit image generation unit that generates image data of the specific input unit and the remote operation device. Determination means for determining whether or not
The remotely operated device according to claim 2. A dedicated control unit unique to each of the plurality of remotely operated devices and a common control unit common to the plurality of remotely operated devices,
Whether or not the dedicated control unit realizes an operation on the specific input unit in a pseudo manner based on a specific input unit image generation unit that generates image data of the specific input unit and an operation content signal transmitted by the remote operation device. Determination means for determining whether or not
The remotely operated device according to claim 2. A dedicated control unit unique to each of the plurality of remotely operated devices and a common control unit common to the plurality of remotely operated devices,
The dedicated control unit artificially realizes an operation on the specific input unit based on an operation content signal transmitted from the specific input unit image generation unit that generates image data of the specific input unit or the remote operation device. Having determination means for determining whether or not
The remotely operated device according to claim 2. Image data display means for receiving image data transmitted by another remotely operated device and displaying the unique input portion image in the other remotely operated device and the remotely operated screen image in the other remotely operated device on the display portion When,
An operation content signal transmitting means for transmitting an operation content signal to the other remote operation device when an operation is performed on the unique input unit image in the other remote operation device displayed on the display unit;
The remotely operated device according to claim 2, further comprising: The display unit is a display with a touch panel.
The remotely operated device according to any one of claims 2 to 6, wherein: A remote operation method for a plurality of types of remotely operated devices that are remotely operated by a remote operation device via a network and have a display unit and a specific input unit,
An image data transmission step of transmitting the image data of the unique input unit and the image data output to the display unit from the remote operation device to the remote operation device;
Operation contents transmitted from the remote operation device to the remote operation device when an operation is performed on the unique input unit image displayed on the remote operation device based on the image data transmitted in the image data transmission step A determination step of determining whether or not to artificially realize an operation on the specific input unit based on a signal;
A control step of controlling the remotely operated device based on a determination result in the determination step;
A remote operation method characterized by comprising: A remote operation program for a plurality of types of remotely operated devices including a display unit and a specific input unit, which is remotely operated by a remote operation device via a network,
An image data transmission step for transmitting the image data of the unique input unit and the image data output to the display unit from the remote operation device to the remote operation device;
Operation contents transmitted from the remote operation device to the remote operation device when an operation is performed on the unique input unit image displayed on the remote operation device based on the image data transmitted in the image data transmission step A determination step of determining whether or not to artificially realize an operation on the specific input unit based on a signal;
A control step of controlling the remotely operated device based on a determination result in the determination step;
A remote operation program characterized by comprising:   A computer-readable recording medium on which the program according to claim 9 is recorded.

Images