JP2006058668A - Image transfer using drawing command hook - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for improving the transfer speed when transferring an image to an image display device via a network.
A hook processing module 400 hooks a specific drawing command issued by an application program 122 in advance and draws an image in an image data storage area 106b in a RAM 106 according to the acquired drawing command. For the image portion of the area that is not drawn by the hook processing module 400 and drawn by the normal drawing modules 124, 126, and 128, the image portion is acquired from the VRAM 114 and written to the image data storage area 106b. . The VNC server 130 acquires an image from the image data storage area 106b and transfers the acquired image to the projector via the network.
[Selection] Figure 6

Description

  The present invention relates to an image display system including an image supply device and an image display device connected via a network.

  As a kind of projector for projecting and displaying an image, a projector capable of projecting and displaying an image supplied from a personal computer via a network (hereinafter also referred to as “network projector”) has been proposed (for example, Patent Document 1). . In addition, a technique called VNC (Virtual Network Computing) is used to supply such a network projector with an image from a personal computer via a network.

International Publication No. 01/093583 Pamphlet

  However, when using VNC, an image is temporarily transferred from a VRAM (frame memory) in the computer to a RAM (system memory), and then the image is acquired from the RAM and transferred to the projector via the network. . Since transfer of image data from the VRAM to the RAM takes a considerable time, there is a case where a sufficient transfer speed cannot be obtained as a whole. Such a problem is not limited to a network projector system, and is generally a problem common to systems that supply and display images from an image supply apparatus to an image display apparatus via a network.

  An object of the present invention is to provide a technique for improving the transfer speed when transferring an image to an image display device via a network.

In order to achieve at least a part of the above object, a first image supply device according to the present invention is an image supply device for supplying an image to an image display device via a network for display.
An application program capable of issuing image drawing commands;
A drawing module for processing a drawing command issued by the application program;
A hook processing module that hooks a specific drawing command issued by the application program in advance and draws an image in a specific transfer image storage area in the general-purpose memory according to the acquired drawing command;
An image transfer processing module that acquires an image from the transfer image storage area and transfers the acquired image to the image display device via the network;
With
The hook processing module stores, in the general-purpose memory, first change area information indicating a first change area that is an area in which an image is drawn in response to the specific drawing command in the transfer image storage area. A function of writing, and a function of supplying the drawing command to the drawing module after processing the specific drawing command;
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module
(I) Referring to the first and second change area information stored in the general-purpose memory, the second change area corresponds to an area not included in the first change area. Obtaining an image portion from the frame memory;
(Ii) obtaining an image portion corresponding to the first change area from the transfer image storage area without obtaining from the frame memory;
(Iii) The acquired image portion is transferred to the image display device via the network together with screen update area information indicating a screen update area that is a sum of the first and second change areas.

  According to the first image supply device, when a specific drawing command is issued, the hook processing module hooks and acquires the drawing command instead of the drawing module, and draws the image in the general-purpose memory. Therefore, the speed of the entire transfer process can be improved as compared with the conventional case where the entire image to be transferred is transferred from the VRAM to the RAM (general purpose memory). Of the second change area, the area not included in the first change area is an area that is not drawn by the hook processing module. Since the image of this area is acquired from the frame memory, the image is not lost. Can be transferred.

A second image supply device according to the present invention is an image supply device for supplying and displaying an image on an image display device via a network,
An application program capable of issuing image drawing commands;
A drawing module for processing a drawing command issued by the application program;
A hook processing module that hooks a specific drawing command issued by the application program in advance and draws an image in a specific transfer image storage area in the general-purpose memory according to the acquired drawing command;
An image transfer processing module that acquires an image from the transfer image storage area and transfers the acquired image to the image display device via the network;
With
The hook processing module stores, in the general-purpose memory, first change area information indicating a first change area that is an area in which an image is drawn in response to the specific drawing command in the transfer image storage area. A function of writing, and a function of supplying the drawing command to the drawing module after processing the specific drawing command;
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module
(I) Referring to the first and second change area information stored in the general-purpose memory, the second change area corresponds to an area not included in the first change area. Obtaining an image portion from the frame memory;
(Ii) An image portion corresponding to an area included in the first change area among the second change areas is acquired from the transfer image storage area without acquiring from the frame memory;
(Iii) The acquired image portion is transferred to the image display device via the network together with screen update area information indicating a screen update area that is the same as the second change area.

  Even in the second image supply device, when a specific drawing command is issued, the hook processing module hooks and acquires the drawing command instead of the drawing module, and draws the image in the general-purpose memory. Thus, compared to the conventional case where the entire image to be transferred is transferred from the VRAM to the RAM (general purpose memory), the speed of the entire transfer process can be improved. Of the second change area, the area not included in the first change area is an area that is not drawn by the hook processing module. Since the image of this area is acquired from the frame memory, the image is not lost. Can be transferred.

  The image transfer processing module obtains an image portion corresponding to an area that is not included in the first change area in the second change area from the frame memory, and writes it in the transfer image storage area. Thereafter, an image portion corresponding to the screen update area may be acquired from the transfer image storage area.

  According to this configuration, when the image is finally transferred via the network, the image can be acquired from one transfer image storage area, so that the processing is simplified.

The image transfer processing module is
The hook processing module may be loaded into the application program when the image transfer processing module is activated, and the hook processing module may be unloaded from the application program when the execution of the image transfer processing module is completed.

  According to this configuration, since the hook processing module works only when the image transfer process is performed, it is possible to prevent the hook processing module from desirably executing the hook process when the hook process is unnecessary.

  The present invention can be realized in various forms, for example, an image display system, an image supply device and an image display device that constitute the image display system, a hook processing module and an image transfer used in the image supply device. The present invention can be realized in various modes such as a processing module, an image supply method, a computer program for realizing the method or apparatus, and a recording medium on which the computer program is recorded.

Hereinafter, embodiments of the present invention will be described in the following order.
A. First embodiment:
B. Second embodiment:
C. Variations:

A. First embodiment:
FIG. 1 is an explanatory diagram showing the configuration of an image display system as an embodiment of the present invention. The image display system 10 of the present embodiment includes a personal computer 100 as an image supply device, a projector 200 as an image display device, and a network 300 that connects the computer 100 and the projector 200. The computer 100 has a function of supplying an image to the projector 200 via the network 300 and causing the projector 200 to project the image and display it on the projection display screen 70.

  FIG. 2 is a block diagram showing the internal configuration of the computer 100 and the projector 200. The computer 100 includes a CPU 102, a ROM 104, a RAM 106 as a general-purpose memory (also referred to as “system memory”), a hard disk drive 108, an input unit 110 including a keyboard and a pointing device, a network interface unit 112, A VRAM 114 as a frame memory, a graphic controller 116, a display device 118 such as a liquid crystal display, and a bus 120 for connecting these elements are provided.

  The RAM 106 stores various computer programs including an application program 122, a graphical device interface (GDI) 124, a display driver 126, a projector driver 128, and a VNC server 130 as an image transfer processing module. Has been. The GDI 124, the display driver 126, and the projector driver 128 function as a part of the operating system. In this embodiment, Windows (registered trademark) provided by Microsoft Corporation is assumed as the operating system. Such various computer programs are provided in a form recorded on a computer-readable recording medium such as a flexible disk or a CD-ROM.

  The projector 200 includes a CPU 202, a ROM 204, a RAM 206, an input unit 210 including various operation buttons, a network interface unit 212, an image processing unit 214, a light source lamp, a liquid crystal panel, and a projection unit 216 including a projection optical system. And a bus 218 for connecting these elements. The ROM 204 stores various computer programs including the image processing driver 228 and the VNC client 230.

  Hereinafter, before describing the function of the computer 100 in the embodiment, the function of a conventional computer using a VNC server will be described as a comparative example.

  FIG. 3 is an explanatory diagram showing a hierarchical structure of computer software and hardware in a comparative example. The application program 122 and the VNC server 130 belong to an application program layer (also referred to as “user application program layer”). The GDI 124, the display driver 126, and the projector driver 128 belong to the kernel layer. Further, the RAM 106 as a general-purpose memory, the network interface unit 112, the VRAM 114, the graphic controller 116, and the display device 118 belong to the hardware layer.

  In the first embodiment, a case where a presentation program (for example, Microsoft PowerPoint) is used as the application program 122 will be described.

  The application program 122 issues various drawing requests to the GDI 124. The GDI 124 is a computer program that uniformly manages drawing on a display device (the display device 118 or the projector 200 of the computer 100) or a printing device (not shown). As is well known, the GDI 124 provides an application program interface (API: Application Program Interface) relating to drawing called “GDI function” to various application programs. Note that an API generally refers to a collection of procedures for an application program to use various functions of an operating system.

  For example, the application program 122 issues a drawing request for a presentation sheet image included in the presentation file to the GDI 124. Usually, the drawing request also includes information regarding the output destination of the image (that is, information specifying whether to output the image to the display device or the printing device).

  Note that other general-purpose drawing APIs may be used instead of the GDI 124. The “general-purpose drawing API” means an API that is commonly used for many application programs.

  The GDI 124, the display driver 126, and the projector driver 128 all function as a drawing module that executes a drawing process in accordance with a drawing command. Specifically, the GDI 124 receives the drawing request issued from the application program 122, checks the output destination of the image based on the drawing request, and if the output destination is a display device, the display driver 126 and the projector driver A drawing request is passed to each of 128.

  The display driver 126 draws image data in the VRAM 114 in accordance with the received drawing request. As is well known, the drawing request is not limited to drawing the entire screen, but frequently drawing a part of the screen. When receiving a drawing command for drawing a part of the screen, the display driver 126 draws only an image portion of an area to be drawn (also referred to as “change area”) in the VRAM 114. The graphic controller 116 displays an image on the display device 118 based on image data (that is, bitmap data) drawn in the VRAM 114. On the other hand, the projector driver 128 writes the change area information 106 a indicating the position and size of the drawing target area (that is, the change area) in the VRAM 114 in the RAM 106 in accordance with the drawing request received from the GDI 124.

  4A and 4B are explanatory diagrams showing examples of the change area information 106a and the image data storage area 106b in the RAM 106. FIG. Here, it is assumed that the image to be projected and displayed by the projector changes from FIG. 4 (A) to FIG. 4 (B). As will be described later, the image data storage area 106 b stores image data transferred from the VRAM 114 by the VNC server 130. The image data storage area 106b preferably has the same data capacity as the VRAM 114 (frame memory).

  The change area Ra is an area of an image portion drawn by a drawing command when changing the image from FIG. 4 (A) to FIG. 4 (B). The change area information 106a regarding the change area Ra includes the x coordinate Xa and the y coordinate Ya of the reference point (usually the upper left point) of the change area Ra, and the width Wa and the height Ha of the change area Ra. The coordinates (Xa, Ya) of the reference point are data indicating the position of the change area Ra, and the width Wa and the height Ha are data indicating the size of the change area Ra.

  The VNC server 130 refers to the change area information 106 a in the RAM 106 and transfers the image data in the change area Ra from the VRAM 114 to the image data storage area 106 b in the RAM 106. The image data storage area 106b is a memory area having a size equal to the display resolution of the computer 100 or the projector 200, for example. The VNC server 130 further reads out the image data of the change area Ra from the image data storage area 106b, and creates a screen update message including the change area information 106a of the change area Ra and the image data.

  FIG. 5 is an explanatory diagram showing the structure of the screen update message. This screen update message conforms to the structure of the frame buffer update “FramebufferUpdate” defined in the VNC protocol. The screen update message has a message header portion indicating a message type and a parameter portion indicating a message parameter.

  The message header part includes a message type (1 byte) indicating the type of message and a change area number (2 bytes) indicating the number of change areas. The data value of the message type indicating the screen update message is “0”. The data value of the number of change areas is normally set to “1”. However, when the change area information 106a regarding a plurality of change areas remains unprocessed in the RAM 106, the screen update message includes information about the change areas (change area information and image data). It is also possible to construct an update message. The change area information included in the screen update message is also referred to as “screen update area information”. The message type data and the change area number data are divided through 1-byte padding.

  In the parameter section, the x coordinate (2 bytes) and y coordinate (2 bytes) of the reference point of the change area, the width (2 bytes) and height (2 bytes) of the change area, and the transfer of the image data of the change area Time encoding format (4 bytes) and change area image data (n bytes) are included. A predetermined format such as a JPEG format (data value “7”) is used as the encoding format.

  The VNC server 130 transfers the screen update message thus created to the projector 200 via the network interface unit 112.

  As described above, in the comparative example shown in FIG. 3, the image data once stored in the VRAM 114 is transferred to the RAM 106 which is a general-purpose memory, and then the image data in the RAM 106 is transferred to the projector 200. For this reason, it takes a considerable time to transfer data from the VRAM 114 to the RAM 106, and a sufficient transfer speed may not be obtained.

  FIG. 6 is an explanatory diagram showing a hierarchical structure of computer software and hardware in the first embodiment. 3 is different from the comparative example shown in FIG. 3 in that a hook processing module 400 is added between the application program 122 and the GDI 124, and two types of change area information 106a1 and 106a2 (hereinafter, “ 3) and other configurations are the same as those in FIG. 3. The hook processing module 400 hooks a specific drawing command issued by the application program 122 in advance and executes processing described later. Other drawing commands are processed by the GDI 124 as usual. Note that the hook processing module 400 is preferably configured exclusively for each application program so as to hook only a specific drawing command appropriate for the application program 122.

  The hook processing module 400 can also be considered as a kind of drawing API, but is distinguished from a general-purpose drawing API such as the GDI 124. That is, the hook processing module 400 is typically configured to process only specific drawing commands, whereas the GDI 124 is configured to process all drawing commands. .

  The hook processing module 400 is loaded (installed) into the application program 122 as a DLL (dynamic link library), for example. In general, an application program operates in units of “processes”. During execution of an application program, multiple modules (GDI32.DLL, Kernel32.DLL, etc.) are loaded in the process, and the application program performs various operations by using the functions in those modules. Is going. The “loading” of the hook processing module 400 means that the module 400 in which the hook procedure is mounted is incorporated in the process of the application program 122.

  When the hook processing module 400 is loaded, the function address of a specific drawing command issued from the application program 122 is the address of another function for executing a unique function implemented in the hook processing module 400. Is replaced. In the function of this unique function, processing for writing change information and image data into the RAM (described later) is executed, and then a normal GDI function is executed. When the hook processing module 400 is unloaded, the hook processing module 400 loaded in the process is unloaded after the function address replaced at the time of loading is restored.

  The hook processing module 400 can be loaded into the application program 122 and unloaded at any time as necessary. However, when the VNC server 130 is activated, the hook processing module 400 is loaded into a specific application program 122 (presentation program in this embodiment) that is operating, and when the VNC server 130 is terminated, the hook processing module 400 is loaded. Preferably, the VNC server 130 is configured to unload. In this way, since the hook processing module 400 can be operated only during the period when the VNC server 130 is being executed, a drawing command is hooked when the VNC server 130 is not executed (not activated) on the computer 100. And undesirable results can be avoided. In the present specification, the phrase “VNC server 130 is running on computer 100” means that the process of VNC server 130 is activated.

  FIG. 7 is a flowchart showing operations of the hook processing module 400 and the VNC server 130 in the first embodiment. Steps S10 to S16 are executed by the hook processing module 400, and steps S20 to S26 are periodically executed by the VNC server 130.

  When a specific drawing command is issued from the application program 122, the hook processing module 400 hooks the drawing command in step S10 and acquires the drawing command instead of the GDI 124. Note that it is preferable that the drawing command hooked by the hook processing module 400 is not limited to all drawing commands, but only limited specific drawing commands. This is because there are many drawing commands issued by the application program 122. If all of them are hooked, there is a possibility that the processing speed may be lowered or the desired drawing may not be performed. Because.

  In step S12, a process of registering the change area information is executed according to the acquired drawing command. Specifically, the hook processing module 400 acquires the position and size of the area (the change area Ra in FIG. 4B) where drawing is performed according to the drawing command, and changes area information A1 representing the change area is stored in the RAM 106. Execute the direct write process.

  In step S14, the writing process of the image data of the change area Ra is executed. Specifically, the hook processing module 400 executes the process of expanding the image data in the change area Ra in accordance with the drawing command and directly writing the image data in the image data storage area 106b in the RAM 106.

  In step S16, the hook processing module 400 calls a normal GDI function according to the drawing command acquired in step S10, and executes the drawing process. Note that the display driver 126 and the projector driver 128 (FIG. 6) execute respective processes in accordance with the drawing command received from the GDI 124, as in the comparative example shown in FIG. That is, the display driver 126 draws an image in the VRAM 114, and the projector driver 128 writes the change area information A2 indicating the drawing area in the RAM.

  On the other hand, in step S20, the VNC server 130 determines an area of image data to be transferred to the projector 200 (referred to as a “screen update area”). FIG. 8 is a flowchart showing the detailed procedure of step S20. In steps S30 and S32, the first and second change area information A1 and A2 are acquired. As described above, the first change area information A1 is information representing an area drawn in the image data storage area 106b in the RAM 106 by the hook processing module 400, and is written in the RAM 106 by the hook processing module 400. It is. On the other hand, the second change area information A2 is information representing an area drawn in the VRAM 114 by the display driver 126, and is written in the RAM 106 by the projector driver 128.

  FIG. 9A shows an example of two change areas RA1 and RA2 represented by two change area information A1 and A2. Here, the two change areas RA1 and RA2 are arranged in the same screen area SCA. Here, the “screen area SCA” means the entire area of the image stored in the VRAM 114, which is equal to the entire area of the image data storage area 106b. Strictly speaking, the first change area RA1 is an area defined by an address in the image data storage area 106b, and the second change area RA2 is an area defined by an address in the VRAM 114. It can be expressed by the same screen area SCA.

  In the example of FIG. 9A, the two change regions RA1 and RA2 partially overlap, but the two change regions RA1 and RA2 may take various other relationships. For example, there are cases where the two change regions RA1 and RA2 do not overlap at all, and there are also cases where the two change regions RA1 and RA2 completely overlap. For example, when the hook processing module 400 performs drawing in accordance with a specific drawing command, and then the normal drawing modules 124, 126, and 128 perform drawing in the same region in accordance with the same drawing command, the two change regions RA1, RA2 Are the same. However, even when the same drawing command is executed, there may be a slight difference between the two change areas RA1 and RA2 depending on the setting of each module.

  FIG. 9B shows an area TG1 where the VNC server 130 directly acquires an image from the RAM 106 during image transfer, and FIG. 9C shows an area TG2 where the image is acquired from the VRAM 114. These areas TG1 and TG2 are also referred to as “transfer areas”. As can be understood from this example, the image of the first change region RA1 (= TG1) is acquired directly from the RAM 106. On the other hand, the image of the area TG2 of the second change area RA2 that is not included in the first change area RA1 is acquired from the VRAM 114. The reason for this distinction is to reduce the number of images acquired from the VRAM 114 as much as possible and reduce the time required for the acquisition. For example, when the two change areas RA1 and RA2 are completely overlapped, it is not necessary to acquire an image from the VRAM 114, so that the transfer process can be performed at high speed. FIG. 9D shows the entire area (screen update area TTG) transferred by the VNC server 130. This screen update area TTG corresponds to the sum area of the first and second change areas RA1 and RA2.

  The first change area information A1 may include a plurality of information corresponding to a plurality of drawing commands. In this case, the sum area of the drawing areas of the plurality of drawing commands becomes the first change area RA1. The same applies to the second change region RA2.

  In step S34 in FIG. 8, the difference between the change areas RA1 and RA2 represented by the two types of change area information A1 and A2 is calculated. Specifically, a region TG2 (FIG. 9C) that is not included in the first change region RA1 in the second change region RA2 is calculated. In step S36, the optimization process of the screen update area TTG (FIG. 9D) is executed.

  FIG. 10 is an explanatory diagram showing the contents of the optimization process of the screen update area. FIG. 10A shows the same screen update area TTG as FIG. 9D. In the optimization process, as shown in FIG. 10B, the screen update area TTG is divided into adjacent rectangular areas R11 to R13 that do not overlap each other. And these rectangular area | regions R11-R13 are employ | adopted as a screen update area | region after optimization. Since each of the screen update areas R11 to R13 is rectangular, it is possible to reduce the amount of image data to be transferred compared to the case of transferring a rectangular image including the screen update area TTG before optimization. is there.

  In step S22 of FIG. 7, image data of the image portion corresponding to the screen update area is acquired. FIG. 11 is a flowchart showing the detailed procedure of step S22. In step S40, it is checked whether or not the second transfer area TG2 (FIG. 9C) is empty. When the second change area information A2 does not exist and when the second change area A2 is completely included in the first change area A1, the second transfer area TG2 is empty. If the second transfer area TG2 is empty, the process proceeds to step S44 described later. On the other hand, if the second transfer area TG2 is not empty, step S42 is executed. In step S42, the VNC server 130 acquires the image data in the second transfer area TG2 from the VRAM 114 and writes it in the image data storage area 106b in the RAM 106. As a result, all the images in the screen update area TTG shown in FIG. 9D are stored in the image data storage area 106b. In step S44, the VNC server 130 acquires the image data of the screen update area TTG from the image data storage area 106b of the RAM 106. Note that what is actually acquired is the image data of the individual screen update areas R11 to R13 (FIG. 10B) after the optimization processing.

  In step S24 of FIG. 7, the image data is converted in accordance with a preset encoding format. In step S <b> 26, a screen update message (FIG. 5) including the image data thus converted and the change area information is created and transferred to the projector 200 via the network interface unit 112.

  When the network interface unit 212 (FIG. 2) of the projector 200 receives the communication data supplied via the network 300, the network interface unit 212 (FIG. 2) extracts a screen update message included in the communication data and passes it to the VNC client 230. The VNC client 230 passes the image data included in the screen update message to the image processing driver 228. The image processing driver 228 controls the image processing unit 214 according to the image data, and causes the image processing unit 214 to develop the image data in a display memory (not shown) in the image processing unit 214. As a result, the projector 200 can display an image on the projection display screen 70 as shown in FIG.

  As described above, in the first embodiment, the hook processing module 400 hooks a specific drawing command to prefetch and executes a process of directly writing the image data of the change area to the RAM 106. As a result, the time required to transfer the image data from the VRAM 114 to the RAM 106 in the comparative example shown in FIG. 3 can be saved. As a result, there is an advantage that the transfer speed when image data is transferred to the projector 200 can be sufficiently increased.

  In a general personal computer, data transfer from the RAM 106 to the VRAM 114 can be executed at a very high speed, whereas data transfer from the VRAM 114 to the RAM 106 can be executed only several times slower than that. There are many. In the first embodiment, since the data transfer from the low-speed VRAM to the RAM can be reduced, the time required to transfer the image data to the projector 200 can be greatly reduced.

  In the first embodiment, the image portion of the area TG2 (FIG. 9C) not drawn by the hook processing module 400 is acquired from the VRAM 114 and transferred to the projector 200. As a result, the images drawn by the normal drawing modules 124, 126, and 128 can be transferred to the projector 200 without omission.

B. Second embodiment:
FIG. 12 is an explanatory diagram showing the relationship between the change area and the screen update area in the second embodiment, and corresponds to FIG. 9 in the first embodiment. In the second embodiment, the overall apparatus configuration and processing flow are the same as those in the first embodiment, but the area to be transferred is different from that in the first embodiment. Specifically, in the second embodiment, the first transfer region TG1 (FIG. 12B) in which an image is directly acquired from the RAM 106 is a portion where the first and second change regions RA1 and RA2 overlap. Set to In other words, the first transfer area TG1 is set to an area included in the first change area RA1 in the second change area RA2. On the other hand, the second transfer area TG2 (FIG. 12C) from which an image is acquired from the VRAM 114 is included in the first change area RA1 in the second change area RA2, as in the first embodiment. It is set to the part that is not. Accordingly, the screen update area TTG (FIG. 12D), which is the entire area to be transferred, is the same as the second change area RA2.

  Thus, in the second embodiment, the image in the second change area RA2, which is an area drawn in the VRAM 114 by the GDI 124 and the display driver 126, is transferred by the VNC server 130. The reason for performing the processing in this way is as follows. That is, since the drawing command hooked by the hook processing module 400 is again executed by the GDI 124 and the display driver 126, basically all drawing commands are executed by the GDI 124 and the display driver 126. Therefore, if the image of the second change area RA2 is transferred, the projector 200 can display a correct image. However, if all the image data is acquired from the VRAM 114 at this time, there arises a problem that a considerable time is required for data transfer. Thus, in the second embodiment, the time required for data acquisition is reduced by directly acquiring image data from the RAM 106 for the portion included in the first change region RA1 in the second change region RA2. is doing. Further, the second embodiment is preferable to the first embodiment in that the amount of image data transferred is much smaller than that of the first embodiment.

  In many cases, the area drawn by the hook processing module 400 (first change area RA1) and the area drawn by the GDI 124 and the display driver 126 (second change area RA2) coincide with each other. Yes. Therefore, as in the second embodiment, even if the image transfer of the portion not included in the second change area RA2 in the first change area RA1 is omitted, the image displayed on the projector 400 is uncomfortable. The possibility is negligible. On the other hand, in the first embodiment described above, the image is transferred also in the portion of the first change area RA1 that does not overlap with the second change area RA2. Therefore, the first and second change areas RA1 are assumed. , RA2 has the advantage of being able to transfer a more complete image even if it is significantly different.

  The VNC server 130 includes a plurality of transfer modes including the transfer process of the first embodiment (also referred to as “first transfer mode”) and the transfer process of the second embodiment (also referred to as “second transfer mode”). It may be configured such that one can be selected and executed. The transfer mode may be selected by the user or automatically by the VNC server 130 in accordance with the processing capability (CPU processing speed, bus transfer speed, etc.) of the computer 100 (generally an image supply device). Alternatively, the transfer mode may be selected. In this way, it is possible to select a transfer mode in which a preferable image can be transferred at an appropriate transfer speed.

C. Variations:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

C1. Modification 1:
As the screen update message supplied from the computer 100 to the projector 200, various messages other than those used in the above embodiment can be used. For example, a screen update message that includes at least image data can be used. Specifically, for example, when transferring image data of the entire screen, there is no need to transfer change area information.

  The screen update message preferably includes at least information indicating the position and size of the change area (screen update area) and image data in the change area. This is because the amount of data to be transferred is reduced if only the data relating to the change area (change area information and its image data) is transferred.

C2. Modification 2:
In the above embodiment, the image portion of the second transfer area TG2 (FIG. 9C, FIG. 12C) is acquired from the VRAM 114 and once written in the image data storage area 106b in the RAM 106, then the RAM 106 The image of the screen update area TTG is acquired from the image data and transferred via the network. However, the process of writing in the image data storage area 106b can be omitted. However, according to the procedure of the above-described embodiment, when an image is finally transferred through the network, the image can be acquired from one image data storage area 106b.

C3. Modification 3:
In the above embodiment, the drawing module is divided into three modules, the GDI 124, the display driver 126, and the projector driver 128. However, the modules can be arbitrarily classified, and these functions can be combined into one module. It is also possible to combine the functions of the display driver 126 and the projector driver 128 into one module.

C4. Modification 4:
In the above embodiment, a personal computer is used as the image supply device, but other types of computers (mobile computer, handheld computer, workstation, etc.) may be used instead. In addition to these computers, devices that can be connected to a network and have the same functions as the computer may be used. Such devices include, for example, information mobile terminals, mobile phones, mail terminals, game machines, set top boxes, and the like. As the image display device, various display devices other than the projector can be used.

C5. Modification 5:
As the network, in addition to a local area network (LAN), various networks such as a wide area network (WAN), the Internet, and an intranet can be applied. The network may be configured by wire or may be configured by radio.

C6. Modification 6:
In the above embodiment, a part of the functions realized by software may be realized by hardware, or a part of the functions realized by hardware may be realized by software.

It is explanatory drawing which shows the structure of an image display system as one Example of this invention. It is a block diagram which shows the internal structure of a computer and a projector. It is explanatory drawing which shows the hierarchical structure of the software and hardware of a computer in a comparative example. It is explanatory drawing which shows the example of the change area information and image data storage area in an Example. It is explanatory drawing which shows the structure of a screen update message. It is explanatory drawing which shows the hierarchical structure of the software and hardware of a computer in an Example. It is a flowchart which shows operation | movement of the hook process module and VNC server in 1st Example. It is a flowchart which shows the detailed procedure of step S20. It is explanatory drawing which shows the relationship between the change area | region and screen update area | region in 1st Example. It is explanatory drawing which shows the content of the optimization process of a screen update area | region. It is a flowchart which shows the detailed procedure of step S22. It is explanatory drawing which shows the relationship between the change area | region and screen update area | region in 2nd Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Image display system 60 ... Display screen 70 ... Projection display screen 100 ... Computer 102 ... CPU
104 ... ROM
106 ... RAM
106a ... Change area information 106b ... Image data storage area 108 ... Hard disk drive 110 ... Input unit 112 ... Network interface unit 114 ... VRAM
116: Graphic controller 118 ... Display device 120 ... Bus 122 ... Application program 124 ... GDI (graphics device interface)
126 ... Display driver 128 ... Projector driver 130 ... VNC server 200 ... Projector 202 ... CPU
204 ... ROM
206 ... RAM
210: Input unit 212 ... Network interface unit 214 ... Image processing unit 216 ... Projection unit 218 ... Bus 228 ... Image processing driver 230 ... VNC client 300 ... Network 400 ... Hook processing module

Claims (10)

An image supply device for supplying and displaying an image on an image display device via a network,
An application program capable of issuing image drawing commands;
A drawing module for processing a drawing command issued by the application program;
A hook processing module that hooks a specific drawing command issued by the application program in advance and draws an image in a specific transfer image storage area in the general-purpose memory according to the acquired drawing command;
An image transfer processing module that acquires an image from the transfer image storage area and transfers the acquired image to the image display device via the network;
With
The hook processing module stores, in the general-purpose memory, first change area information indicating a first change area that is an area in which an image is drawn in response to the specific drawing command in the transfer image storage area. A function of writing, and a function of supplying the drawing command to the drawing module after processing the specific drawing command;
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module
(I) Referring to the first and second change area information stored in the general-purpose memory, the second change area corresponds to an area not included in the first change area. Obtaining an image portion from the frame memory;
(Ii) obtaining an image portion corresponding to the first change area from the transfer image storage area without obtaining from the frame memory;
(Iii) The acquired image portion is transferred to the image display device via the network together with screen update area information indicating a screen update area that is a sum of the first and second change areas. Feeding device. An image supply device for supplying and displaying an image on an image display device via a network,
An application program capable of issuing image drawing commands;
A drawing module for processing a drawing command issued by the application program;
A hook processing module that hooks a specific drawing command issued by the application program in advance and draws an image in a specific transfer image storage area in the general-purpose memory according to the acquired drawing command;
An image transfer processing module that acquires an image from the transfer image storage area and transfers the acquired image to the image display device via the network;
With
The hook processing module stores, in the general-purpose memory, first change area information indicating a first change area that is an area in which an image is drawn in response to the specific drawing command in the transfer image storage area. A function of writing, and a function of supplying the drawing command to the drawing module after processing the specific drawing command;
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module
(I) Referring to the first and second change area information stored in the general-purpose memory, the second change area corresponds to an area not included in the first change area. Obtaining an image portion from the frame memory;
(Ii) An image portion corresponding to an area included in the first change area among the second change areas is acquired from the transfer image storage area without acquiring from the frame memory;
(Iii) The obtained image portion is transferred to the image display device via the network together with screen update area information indicating a screen update area that is the same as the second change area. The image supply device according to claim 1, wherein:
The image transfer processing module obtains an image portion corresponding to an area not included in the first change area in the second change area from the frame memory and writes the image portion in the transfer image storage area. An image supply device that acquires an image portion corresponding to the screen update area from the transfer image storage area. The image supply device according to any one of claims 1 to 3,
The image transfer processing module
An image supply configured to load the hook processing module into the application program when the image transfer processing module is activated and unload the hook processing module from the application program when the execution of the image transfer processing module is completed. apparatus. An image transfer processing module used in an image supply device for supplying and displaying an image on an image display device via a network,
The image supply device hooks an application program capable of issuing an image drawing command, a drawing module for processing a drawing command issued by the application program, and a specific drawing command issued by the application program. And a hook processing module for drawing an image in a specific transfer image storage area in the general-purpose memory in accordance with the acquired drawing command.
The image transfer processing module has a function of acquiring an image from the transfer image storage area and transferring the acquired image to the image display device via the network;
The hook processing module has a function of drawing an image in the transfer image storage area in accordance with a drawing command acquired by the hook, and an image is drawn in the transfer image storage area in accordance with the specific drawing command. A function of writing first change area information indicating a first change area, which is an area, into the general-purpose memory; and a function of supplying the drawing command to the drawing module after processing the specific drawing command ,
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module
(I) Referring to the first and second change area information stored in the general-purpose memory, the second change area corresponds to an area not included in the first change area. Obtaining an image portion from the frame memory;
(Ii) obtaining an image portion corresponding to the first change area from the transfer image storage area without obtaining from the frame memory;
(Iii) The acquired image portion is transferred to the image display device via the network together with screen update area information indicating a screen update area that is a sum of the first and second change areas. Transfer processing module. An image transfer processing module used in an image supply device for supplying and displaying an image on an image display device via a network,
The image supply device hooks an application program capable of issuing an image drawing command, a drawing module for processing a drawing command issued by the application program, and a specific drawing command issued by the application program. And a hook processing module for drawing an image in a specific transfer image storage area in the general-purpose memory in accordance with the acquired drawing command.
The image transfer processing module has a function of acquiring an image from the transfer image storage area and transferring the acquired image to the image display device via the network;
The hook processing module has a function of drawing an image in the transfer image storage area in accordance with a drawing command acquired by the hook, and an image is drawn in the transfer image storage area in accordance with the specific drawing command. A function of writing first change area information indicating a first change area, which is an area, into the general-purpose memory; and a function of supplying the drawing command to the drawing module after processing the specific drawing command ,
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module
(I) Referring to the first and second change area information stored in the general-purpose memory, the second change area corresponds to an area not included in the first change area. Obtaining an image portion from the frame memory;
(Ii) An image portion corresponding to an area included in the first change area among the second change areas is acquired from the transfer image storage area without acquiring from the frame memory;
(Iii) An image transfer processing module that transfers the acquired image portion to the image display device via the network together with screen update area information indicating a screen update area that is the same as the second change area. . A method of supplying and displaying an image on an image display device via a network,
(A) A drawing module for processing a drawing command issued by an application program and a specific drawing command issued by the application program are hooked in advance, and a specific transfer in the general-purpose memory is performed according to the acquired drawing command. Preparing a hook processing module for drawing an image in the image storage area,
(B) the hook processing module rendering an image in a specific transfer image storage area in the general-purpose memory in response to a specific rendering command issued by the application program;
(C) The hook processing module uses the first change area information indicating the first change area, which is an area in which an image is drawn in accordance with the specific drawing command, in the transfer image storage area. Writing into the memory;
(D) the hook processing module supplying the drawing command to the drawing module after processing the specific drawing command;
(E) the drawing module drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module;
(F) the drawing module writing second change area information indicating a second change area, which is an area in which an image is drawn in the frame memory, in accordance with the drawing command, into the general-purpose memory; ,
(G) An image corresponding to an area of the second change area that is not included in the first change area with reference to the first and second change area information stored in the general-purpose memory. Obtaining a portion from the frame memory;
(H) obtaining an image portion corresponding to the first change area from the transfer image storage area without obtaining from the frame memory;
(I) transferring the acquired image portion to the image display device via the network together with screen update area information indicating a screen update area that is a sum of the first and second change areas;
An image supply method comprising: A method of supplying and displaying an image on an image display device via a network,
(A) A drawing module for processing a drawing command issued by an application program and a specific drawing command issued by the application program are hooked in advance, and a specific transfer in the general-purpose memory is performed according to the acquired drawing command. Preparing a hook processing module for drawing an image in the image storage area,
(B) the hook processing module rendering an image in a specific transfer image storage area in the general-purpose memory in response to a specific rendering command issued by the application program;
(C) The hook processing module uses the first change area information indicating the first change area, which is an area in which an image is drawn in accordance with the specific drawing command, in the transfer image storage area. Writing into the memory;
(D) the hook processing module supplying the drawing command to the drawing module after processing the specific drawing command;
(E) the drawing module drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module;
(F) the drawing module writing second change area information indicating a second change area, which is an area in which an image is drawn in the frame memory, in accordance with the drawing command, into the general-purpose memory; ,
(G) An image corresponding to an area of the second change area that is not included in the first change area with reference to the first and second change area information stored in the general-purpose memory. Obtaining a portion from the frame memory;
(H) obtaining an image portion corresponding to an area included in the first change area in the second change area from the transfer image storage area without obtaining from the frame memory;
(I) transferring the acquired image portion to the image display device via the network together with screen update area information indicating a screen update area that is the same as the second change area;
An image supply method comprising: A computer-readable recording medium recording an image transfer processing module used in an image supply device for supplying an image to an image display device via a network for display.
The image supply device hooks an application program capable of issuing an image drawing command, a drawing module for processing a drawing command issued by the application program, and a specific drawing command issued by the application program. And a hook processing module for drawing an image in a specific transfer image storage area in the general-purpose memory in accordance with the acquired drawing command.
The image transfer processing module obtains an image from the transfer image storage area, and causes a computer to execute a process of transferring the acquired image to the image display device via the network.
The hook processing module has a function of drawing an image in the transfer image storage area in accordance with a drawing command acquired by the hook, and an image is drawn in the transfer image storage area in accordance with the specific drawing command. A function of writing first change area information indicating a first change area, which is an area, into the general-purpose memory; and a function of supplying the drawing command to the drawing module after processing the specific drawing command ,
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module
(I) Referring to the first and second change area information stored in the general-purpose memory, the second change area corresponds to an area not included in the first change area. Obtaining an image portion from the frame memory;
(Ii) obtaining an image portion corresponding to the first change area from the transfer image storage area without obtaining from the frame memory;
(Iii) A function of transferring the acquired image portion to the image display device via the network together with screen update area information indicating a screen update area that is a sum of the first and second change areas. A recording medium. A computer-readable recording medium recording an image transfer processing module used in an image supply device for supplying an image to an image display device via a network for display.
The image supply device hooks an application program capable of issuing an image drawing command, a drawing module for processing a drawing command issued by the application program, and a specific drawing command issued by the application program. And a hook processing module for drawing an image in a specific transfer image storage area in the general-purpose memory in accordance with the acquired drawing command.
The image transfer processing module obtains an image from the transfer image storage area, and causes a computer to execute a process of transferring the acquired image to the image display device via the network.
The hook processing module has a function of drawing an image in the transfer image storage area in accordance with a drawing command acquired by the hook, and an image is drawn in the transfer image storage area in accordance with the specific drawing command. A function of writing first change area information indicating a first change area, which is an area, into the general-purpose memory; and a function of supplying the drawing command to the drawing module after processing the specific drawing command ,
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module
(I) Referring to the first and second change area information stored in the general-purpose memory, the second change area corresponds to an area not included in the first change area. Obtaining an image portion from the frame memory;
(Ii) An image portion corresponding to an area included in the first change area among the second change areas is acquired from the transfer image storage area without acquiring from the frame memory;
(Iii) Recording having a function of transferring the acquired image portion to the image display device via the network together with screen update area information indicating a screen update area that is the same as the second change area Medium.

Images