JPH10126773A - Image data transfer coding system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress the information quantity of data by coding separately and simultaneously the data with a plurality of kinds of compression coding algorithm sets, selecting bits resulting from a compression coding algorithm providing a least code quantity and transferring the bits. SOLUTION: An operating system OS 2A receives a drawing instruction issued by an application shared program menu reproduction display device AP2D via a path P034. The instruction is converted into an instruction to be issued to a graphics hardware GP2A(A) and issued again to a path P035 and given to a processor hardware and a system bus CPU 2A(B). Image drawing data decoded and stored in a memory are written to the (A) connecting to the (B) via a path P036. (A) provides an output to a display device DP2A via a path P037. Thus, the same image as that outputted to a display device DP1A is displayed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an encoding system for screen data transfer, and more particularly to a system for communicating between a plurality of computers, sharing one application program, and supporting a joint work between remote locations. And an encoding method for screen data transfer.

[0002] A system for supporting collaborative work between remote locations (hereinafter referred to as "application sharing" or "AP sharing" in this specification) includes a plurality of computers all operating the same application program to operate the user. There are a distributed execution method in which only information is shared, and a centralized execution method in which an application program is operated on only one computer and its screen information and operation information are shared by other computers. The present invention relates to the latter centralized execution method.

[0003]

2. Description of the Related Art As shown in FIG. 7, a plurality of users share a computer system PC1 on the side of providing a shared application.
In the environment where the computer system PC2 on the side of the shared application using the remote application is connected via the communication line NET1 and the user works together, the user at the remote site can use the existing user application program APU1 running on the computer system PC1 on the side of the shared application.
The information displayed by the user application program APU1 on the display device PC1a of the computer system PC1, which is information necessary for operating the
If the information is not displayed on the display device PC2a of the remote application computer system PC2, the user of the computer system PC2 cannot operate the same information. That is, the computer system PC1
Some kind of "display information sharing means" for displaying the same information on the PC and the PC 2 is required.

[0004] In FIG. 7, an application program APU shared by an AP in order for a user to cooperate.
1 and a computer PC on which APU1 is actually operating
1b, a computer PC2b using the APU1 operating on the computer PC1b from a remote location, and an application program APU1 operating on the computer system PC1 and transmitting the application program APU1 to the computer system PC2.
AP-side sharing means APM1, which is a means for sharing an AP with the host, and guest-side AP sharing means APM2, which operates on the computer system PC2 and shares the application program APU1 with the computer system PC1. Are operating respectively, and the computer systems PC1 and PC2 are connected to the communication line N.
Through ET1, information necessary for sharing the application program is exchanged.

[0005] Here, in synchronization with the frame rate displayed by the display device PC1a, a sufficient data transfer band capacity for continuously sending a display image to the computer system PC2 like a moving image at the resolution of the shared screen PC1c is provided. There are few communication lines NET1, and in many cases, it is desired to minimize the amount of data to be transmitted and received and to achieve efficient communication.

As means for solving this problem, a first display information sharing means (hereinafter also referred to as a "drawing command method") for converting information to be drawn on the display device PC2a of the computer system PC2 into a drawing processing command and transmitting the same. ) Is conventionally known (for example, JP-A-4-237793 and JP-A-4-281516), and by acquiring bitmap graphics as a drawing result on the display device PC1a of the computer system PC1. A second display information sharing unit (hereinafter, referred to as a second display information sharing unit) that compresses this by some compression unit
A "bitmap method" is also known (for example, Japanese Patent Application Laid-Open No. 7-302233).

[0007] First, the first display information sharing means (drawing command method) will be described. In order for the application program APU1 to output to the display device PC1a, the operation (P01) of issuing a drawing processing command through the operating system (OS) of the computer system PC1 is performed by the OS.
If there is a function (P02A) given to PM1, it becomes possible to acquire (P02A) this drawing processing command by the application sharing means APM1 and transmit it to the computer system PC2 (P03). According to this method, it is not necessary to flow screen information, which tends to increase the amount of information, on the communication line NET1, so that efficient data transfer is possible.

In the computer system PC2, the remote application sharing means APM2 issues the same drawing processing command as P01 (P04), so that equivalent screen information is always displayed on the display devices PC1a and PC2a. Become.

However, in the first display information sharing means (drawing command method), the existing user application program APU1 issues a drawing instruction P01 to the OS running on the computer PC1b. Existing user application program APU1
Does not issue a drawing command to the application sharing unit APM1,
1 is an OS running on the computer PC1b,
Drawing command information (P02) equivalent to the drawing command P01 issued by the existing user application program APU1
A) must be obtained.

Next, the second display information sharing means (bitmap method) will be described. Until the existing user application program APU1 issues the drawing command P01, the OS executes the drawing process command of P01 and the display device PC1a, as in the first display information sharing means.
The application sharing unit APM1 acquires the image information as the result drawn above from the display system of the computer PC1b as bitmap image data (P02).
B) is different from the first display information sharing means.

However, if the obtained bitmap image data is passed through the communication line NET1 as it is, the information amount of the bitmap image data easily increases according to the display area of the shared screen PC1c.
M1 has some information compression means, so that the communication line NE
It is necessary to reduce the amount of information (P03) flowing through T1.

[0012]

A first problem is that the first display information sharing means (drawing command method) is a key to realization of obtaining drawing command information issued by the existing user application program APU1. That is, the technology must depend on the OS of the computer PC1b.

For example, Japanese Patent Laid-Open Publication No.
In JP-A-37932 and JP-A-4-281516, it is assumed that the OS has this function. The OS itself operates as another application program (FIG. 7).
In FIG. 7, the drawing processing command (corresponding to P01 in FIG. 7) issued by the APU 1 is acquired (P02 in FIG. 7).
A) (for example, see Japanese Patent Application Laid-Open No. 4-28).
It is also obvious from the fact that JP 1516 describes a "X window system developed and published at the Massachusetts Institute of Technology". In the X-window system, the OS itself has a mechanism for exchanging drawing processing instructions with another computer via a network, so that the drawing command method can be easily realized.

On the other hand, for example, when Microsoft Windows (Microsoft Windows) version 3.1, Microsoft Windows NT version 3.5, or the like is used as the OS, a drawing processing instruction is sent from another application program. It is difficult to apply the drawing command method because there is no method for obtaining it.

The reason is that if the OS does not provide these methods, an operation of rewriting the OS is required. For example, a method of rewriting a jumping part (entry point) of an OS drawing process or embedding a module for stealing (hooking) is known, but it is similar to the enhancement of the function of the OS itself. Rewriting is not a preferable method if the security aspect of the OS is emphasized.

The second problem is that the second display information sharing means (bitmap system) uses a computer system PC.
When the screen size to be shared between the PC 1 and the PC 2 increases, the bitmap image data increases in proportion to the area of the screen size, a load is imposed on the communication line NET1, and data transfer for sharing the screen information (P03) ) Takes a long time, and as a result, the operability of the user of the computer system PC2 on the remote side of the shared application is significantly reduced.

The reason is that the user of the computer system PC2 on the remote side of the shared application handles the information of the user application program APU1 running on the computer system PC1 at the remote place, so that the contents (P05) operated by himself / herself are different. Application program AP of the computer system PC1
The screen is reflected on U1 (P06, P08), the screen of the display device PC1a of PC1 is changed (P01), the result is obtained as bitmap information (P02B), and the bitmap information is transferred from the computer system PC1 to the computer system PC2. This is because the change is reflected on the screen of the display device PC2a of the PC2 (P04), and the next operation on the application program APU1 becomes possible only at this time.

In particular, if the shared screen size is large and the communication line NET1 does not have a sufficient bit rate,
The time required to transfer the bitmap information from C1 to PC2 becomes very long, and the operability deteriorates. As described above, the bitmap method has a higher OS than the drawing command method.
Is wide, but on the other hand, the amount of information to be transferred tends to be large.

In order to solve the second problem, in the bitmap system, some compression coding is used together to suppress the amount of information flowing through the communication line NET1.

The third problem is that the graphics screen displayed by the computer, that is, the display device PC1a of the PC1
Is that the high compression efficiency may not be obtained depending on the specific encoding algorithm. In other words, the encoding algorithm depends on the user application program APU1, but this APU1 is an arbitrary application program, and what to draw depends on the user application program APU1, so that a high compression rate is required in advance. This is because it is difficult to predict and select an encoding algorithm to obtain.

The reason is that since a wide variety of data is displayed, including text information, line drawings, and in some cases, natural images, it is difficult to obtain a certain unified regularity. This is because the generated generated code amount is greatly affected.

Since the compression encoding algorithms have advantages and disadvantages, problems will be individually described for the following first to fourth encodings.

As a first encoding algorithm, there is run-length encoding compression. This run-length coding compression is an algorithm for coding according to a combination of a data element and the number of appearances in which the data element appears continuously. As shown in a relatively simple block diagram mainly composed of squares, a vertical Alternatively, a graphic having a high probability that pixels having the same value arranged horizontally or a wire frame (line drawing) graphic has a high probability that a region where the same color continues is high, so that high compression efficiency can be obtained.

For example, Windows
When a blank window is displayed as in the initialization of the notepad (notepad) or the initial screen, the code amount can be suppressed at a high compression rate. However, the compression efficiency does not increase when a graphic having many changing points is displayed, such as hatching, in which the same bitmap is not continuous.

As a second encoding algorithm, there is Huffman encoding compression. This Huffman coding compression is one of the algorithms of the variable length coding, and is known as one of the variable length codings that can shorten the average code length,
A relatively high compression efficiency can be obtained for an image in which the same bit pattern is frequently repeated in the drawing content.

However, Huffman coding itself is systematically:
Since it has a code conversion table called a codebook used for encoding (encoding) and uses the same table for decoding (decoding), the table data must be sent from the computer system PC1 to the computer system PC2 every time. , Computer system P
The remote application sharing means APM2 in C2 cannot correctly decode the transmitted data (P03).

Due to this property, Huffman coding can be performed together with the encoded data in the code book, even if the screen size to be shared is small or a screen with a very small amount of information such as a blank window is displayed. Is always added as overhead.
Therefore, when a simple and small figure is displayed, it is inferior to the run-length coding.

In the conventional coding method for screen data transfer described in JP-A-7-302233,
The description of the compression encoding method is ambiguous (it is described that the compression technique is used in combination to realize "window sharing", but there is no specific description of how to perform the compression. However, it is not clearly described that the encoding compression is used), however, it employs the bitmap method as the second display information sharing means, and performs compression using a “compression dictionary”. , Because it is described as a method of restoring using a "decompression dictionary", even if the screen size to be shared is small, the "dictionary"
(Corresponding to a code book of Huffman coding compression) and
This is a method of transmitting a type of coding table called.

For this reason, the above-mentioned fixed extra transfer time (overhead time) is required. Therefore, it can be seen that this conventional method has exactly the same problem as in the case of the Huffman coding compression.

The third encoding algorithm is a method of encoding the inter-frame difference information. According to this method, in principle, the screen information already transferred to the computer system PC2 is also stored in the memory in the computer system PC1 in advance, and a difference from the newly displayed content is obtained. It is a method of sending.

Specifically, one of the easiest methods is to take an XOR (exclusive OR) of the bitmap of the entire shared screen in pixel units. In this case, the bit value of the pixel having the difference becomes “1”, so that it is possible to extract the difference information in this manner.
1 can be reduced.

Considering a situation where a computer is actually used, for example, a character input is performed by a word processor application program between the computer systems PC1 and PC1.
In many computer application program movements, such as when shared by two, there are many operations for rewriting a part of the screen, so that a very high compression effect can be expected.

However, when updating the entire screen, even if a plain window with a small amount of information is displayed on the shared screen, if a complicated image with a large amount of information is drawn on the screen already displayed, the previous Therefore, there is a possibility that large data will be generated.

Finally, as a fourth encoding algorithm, there is an encoding method of an orthogonal transform system capable of obtaining the highest compression ratio in the compression encoding of a still natural image currently existing. In this orthogonal transform coding method, when the user application program APU1 displays a natural image or a graphic in which adjacent pixels change slowly, a value is assigned to a low spectral component on a two-dimensional spectral plane. As a result of performing orthogonal transformation (methods such as two-dimensional Fourier transform and two-dimensional discrete cosine transform), continuation of 0 (zero) is obtained in higher-order spectral components, which is referred to as run-length or Huffman code. High compression efficiency can be expected by performing compression compression.

However, since figures (especially line drawings) which are actually drawn by the user application program APU1 are pulse-like, high-order spectra tend to appear in a two-dimensional spectrum, and generally exist in the natural world. However, compression efficiency cannot be expected because of an unnatural distribution that cannot be considered in a moving image (natural image such as a photograph).

In addition, in the case of a natural image, the compression efficiency is increased by positively producing a continuous 0 (zero) in the spectrum value by quantization, but this is irreversible encoding. Problem. Lossy coding is
In particular, it is not appropriate for sharing an application program by using an input device closely related to position information such as a mouse. The reason is that the user of the computer system PC2 performs "operation" using a pointing device such as a mouse.

In order for the user to operate, it is required that the display contents on the display devices PC1a and PC2a of the computer systems PC1 and PC2 coincide with each other with an accuracy of a pixel unit. This is because if the accuracy in pixel units cannot be guaranteed, when the user of the computer system PC2 points to the small object on the screen with a pointing device such as a mouse in order to move the object, the computer system PC2 uses the pointing device. A user application program APU which indicates an incorrect position on the display device PC1a of the computer system PC1 by an error of one pixel and actually executes the operation.
The reason 1 is that a problem such as failure to grasp an object or catching another nearby object may occur.

Therefore, even with a high compression efficiency, it is possible to transmit a substantially similar shape, but it is not possible to guarantee information with pixel-level accuracy. Irreversible coding is not suitable for this application. .

The present invention has been made in view of the above points, and an object of the present invention is to provide an encoding system for screen data transfer capable of efficiently suppressing the amount of information transmitted to a communication line connecting a plurality of computers. Aim.

Another object of the present invention is to provide a screen data transfer encoding system which improves operability of a user who shares an application program with a receiving computer system.

[0041]

According to the present invention, in order to achieve the above object, a plurality of computers are connected by a communication line, and an application program is operated by only one of the computers to display screen information and operation information. In a screen data transfer encoding system in a system that supports collaborative work with another computer at a remote location, one computer on which an application program operates is drawn by an application program ram. Encoding means for encoding the screen data separately and simultaneously by a plurality of types of compression encoding algorithms before sending them to the communication line;
Selecting means for selecting a bit stream of a compression encoding algorithm having the least code amount among bit streams obtained by encoding with a plurality of types of compression encoding algorithms, and a bit stream selected by the selecting means, at least, Means for adding a header indicating the type of the compression encoding algorithm and transferring the data to another computer via the communication line, and the other computer reads the bit from the header in the signal input via the communication line. Identification means for identifying a stream encoding method; decoding means for separately and simultaneously decoding an input bit stream into screen data by a plurality of types of decoding algorithms corresponding to a plurality of types of compression encoding algorithms; Of the decrypted screen data output by the Data selecting means for selecting only the decoded screen data decoded by the decoding algorithm corresponding to the encoding method, and means for displaying the decoded screen data selected by the decoded data selecting means on the display device. It was done.

According to the present invention, a bit stream of a compression encoding algorithm having the least code amount is automatically selected from the bit streams obtained by encoding with a plurality of types of compression encoding algorithms and transmitted to a communication line. .

By the way, a plurality of computers are connected by a communication line, an application program is operated only by one of the computers, the screen information and the operation information are shared by another computer, and another computer at a remote location is used. In a system for supporting cooperative work with a computer, the amount of transfer information of screen information is reduced by a method with a small OS dependency in order to improve the operability of the other computer for the application program (shared application program). For example, it is preferable that an encoding algorithm that minimizes the amount of information can be automatically selected by a method that matches the screen transition of the window-type OS.

In view of the transition of screen information, the movements of many application programs on the window type OS can be roughly divided into the following three.
Can be divided into two cases.

The first screen transition is when a window with an extremely small amount of information is opened. This corresponds to a case where a blank screen, on which nothing is drawn, is displayed when the shared application program is initialized (including at startup).

The second screen transition is an operation for partially changing the contents. This is an operation to partially modify the graphics and characters on the screen, and this is the most common case where the user uses a computer to work while interacting with the application program, such as AP sharing. It is considered to be.

The third screen transition is for drawing a complicated figure. This may occur immediately after reading figures and characters from a file, or when pasting complicated figures and characters by cut and paste.

If there is a method for selecting the most suitable method for each case separately in these cases, the operability of the user is improved. Here, in the first screen transition, the first encoding algorithm is suitable. In the second screen transition, the third encoding algorithm is suitable. Also, in the third screen transition, the second encoding algorithm must be used.

However, in the actual use scene of the user, the third screen transition is less likely to occur if the second screen transition is performed in such a manner that a partial editing and altering operation is performed. Screen transitions often take a long time to process the shared application program itself, such as when a complicated figure is read from a file. The side user (user of another computer) feels that the processing of the computer is slightly delayed, so that it is not noticeable.

In other words, it may take a little transfer time, but since the user interface often allows a certain amount of time intuitively, it means that the probability of seriously affecting operability is small. .

Therefore, according to the present invention, at least a plurality of types of inter-frame correlation encoding algorithms having the least code amount when the current screen data is data in which only a part of the previous screen data has changed are used as encoding means. It is characterized in that it is included as one of the compression encoding algorithms.

The above-mentioned inter-frame correlation encoding algorithm corresponds to the third encoding algorithm. A part of the screen, which frequently occurs during operation by the user and requires high-speed response in a user interface, is used. During the editing reorganization operation, high-speed screen responsiveness can be secured for a screen transition occurring with a high probability.

In the present invention, a run-length encoding compression algorithm and a second encoding algorithm are used as the first encoding algorithm.
Characterized in that it has a Huffman coding compression algorithm as a coding algorithm and an engine for realizing an inter-frame coding compression algorithm as a third coding algorithm. It has a means to select the conversion method. As a result, better operability can be realized for the user of the AP sharing, in particular, the user of another computer (a user of the shared application) at a remote location.

[0054]

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 shows an overall configuration diagram of an embodiment of the system of the present invention. As shown in the figure, in this embodiment, a computer PC1A that runs a shared application program AP1A, which is an application program that actually performs cooperative work, and a computer PC2A that remotely uses the application program AP1A in cooperative work. This is an encoding method applied to a collaborative work support system.

The hardware of the computer PC1A is as follows.
Processor hardware and system bus CPU1A
A graphics hardware GP1A which is a logic circuit for generating a display signal; a keyboard input hardware KY1A which is a logic circuit for an input port from the keyboard device KB1A; and a communication port hardware COM1A which outputs data to the communication line NET1A. It is composed of The graphics hardware GP1A is connected to a display device DP1A such as a cathode ray tube (CRT), and the keyboard input hardware KY1A is connected to a keyboard device KB1A.

The software of the computer PC1A includes an operating system OS1A, a shared application program AP1A, and a screen sending side A1.
It is composed of an application sharing program screen acquisition mechanism AP1B that performs P sharing, and an application sharing program operation system reflection mechanism AP1C that shares user input information such as a keyboard and a mouse with the computer PC2A.

On the other hand, the hardware of the computer PC2A includes processor hardware and a system bus CPU.
2A, graphics hardware GP2A which is a logic circuit for generating a display signal, and a keyboard device KB2A.
And a communication port hardware COM2A for outputting data to the communication line NET1A. The graphics hardware GP2A is connected to a display device DP2A such as a cathode ray tube (CRT), and the keyboard input hardware KY2A is connected to a keyboard device KB2A.

The software of the computer PC2A includes an operating system OS2A and a PC2A.
Side, an application sharing program screen reproduction and display mechanism AP which is means for sharing screen information of the application program to be shared AP1A running on the PC 1A.
2D and PC1A keyboard input
And an application sharing program operation system acquisition mechanism AP2E to be transmitted to the side.

FIG. 2 is a block diagram showing an example of the inside of the application sharing program screen acquisition mechanism AP1B in FIG. In FIG. 2, the screen change detection mechanism AP1B01
This is a mechanism for detecting that the screen has changed. The screen is always captured, compared with screen information already captured and stored, and when a difference is recognized, the screen acquisition mechanism AP1
An event trigger is generated to activate B02.

The screen acquisition mechanism AP1B02 performs a screen capture (acquisition of a bitmap screen) procedure and
(Operating system OS1A running on PC1A in FIG. 1), obtains screen information, and stores the screen frame information storage mechanism (also referred to as “frame buffer”) A
An operation of storing the data in P1B04 and AP1B05 is performed.

The memory AP1B04 for storing the first image and the memory AP1B05 for storing the second image constituting the frame buffer are double buffers for temporarily storing screen frame information. It functions to provide necessary information to the mechanism AP1B01 and an encoding engine described later.

This double buffer is functionally composed of two frame buffers AP1B04 and AP1B05. In order to configure these two frame buffers, hardware may be configured with two memories (actually, two memory IC chips) so that hardware can be accessed completely independently and separately. If the writing / reading speed of the memory is sufficiently high, the memory may be constituted by one memory (memory IC chip), and the input / output may be arbitrated (arbitrated) by using the memory separately. In FIG. 2, since it is convenient in pursuing the operation in an embodiment described later, AP1B04 and AP1B
05.

Screen frame information switching management mechanism AP1B0
3 is a screen frame information storage mechanism AP1B04 and AP
The memory address of 1B05 is managed, and when the screen acquisition mechanism AP1B02 acquires a new screen, the head address is toggled. Then, the start address of the memory in which the information necessary for each is stored is provided to the screen change detection mechanism AP1B01, the screen acquisition mechanism AP1B02, and various encoding engines described later.

The coding engine includes a run-length coding engine AP1B06 for performing run-length coding and a Huffman coding engine AP1B07 for performing Huffman coding.
And an inter-frame correlation encoding engine AP1B08 for performing inter-frame correlation encoding. These three types of encoding engines start encoding simultaneously when screen data to be encoded is stored in the frame buffers AP1B04 and AP1B05.

Coding Engines AP1B06 and AP1B
07 are two frame buffers AP1B04 and AP1B04.
1B05, only the latest captured image information is read and encoded. On the other hand, only the inter-frame correlation encoding engine AP1B08 requires information that spans between frames, so that the most recently acquired image and the most recently acquired image are acquired from the two frame buffers AP1B04 and AP1B05. Read both information as well,
The difference between frames is coded.

Coding engines AP1B06, AP1B0
7 and AP1B08 also have a counting mechanism for counting the generated code amount inside, thereby having a mechanism for outputting the generated code amount. Further, since each of the encoding engines AP1B06, AP1B07, and AP1B08 is a module highly independent of the OS,
This means may be realized by either software or hardware.

The encoding engine AP1B0 is implemented by software.
6, when each of AP1B07 and AP1B08 is configured, if a multiprocessor environment can be used, the program is divided into a plurality of threads, and the encoding engines AP1B06, AP1B07, AP1B
08 can be assigned to different processors and executed, thereby further improving the processing speed. Each of the encoding engines AP1B06, AP1B07, and AP1B08 is functionally highly independent, and has two frame buffers AP1B04 and AP1B0 from which data is read.
Except for sharing 5, there is no mutual interference,
This is because they can be executed simultaneously.

Coding engines AP1B06, AP1B0
7 and the code amount output by AP1B08 are input to a code amount comparison mechanism AP1B09. The code amount comparison mechanism AP1B09 performs a comparison operation, and includes encoding engines AP1B06, AP1B07, and AP1B08.
Are compared with each other, and as a result,
The system having the smallest code amount is notified to the bit stream switching mechanism AP1B10.

The encoding engines AP1B06, AP1
The code strings (bit streams) respectively output by 1B07 and AP1B08 are stored in a bit stream switching mechanism A.
Output to P1B10. Based on the result obtained from the code amount comparison mechanism AP1B09, the bit stream switching mechanism AP1B10 encodes the encoding engines AP1B06 and AP1B06.
Only one of the bit streams output from 1B07 and AP1B08 is selected and output to the transmission protocol addition mechanism AP1B11. The transmission protocol addition mechanism AP1B11 performs a process of adding a header to identify an encoding method, and outputs the same from the application sharing program screen acquisition mechanism AP1B.

FIG. 3 shows an application sharing program screen playback / display mechanism AP in the computer PC2A in FIG.
FIG. 2 shows a configuration diagram of an example of the inside of 2D. As described with reference to the application sharing program screen acquisition mechanism AP1B shown in FIG. 2, the header is added to the screen information and transmitted, so the receiving side also needs to process the header.

The transmission protocol separation mechanism AP2D01 in the application sharing program screen reproduction display mechanism AP2D shown in FIG. 3 performs this header processing. By performing this header process, it is possible to know whether the currently received bit stream is run-length coded, Huffman coded, or inter-frame correlation coded, so that information necessary for this decoding is decoded. At the same time as passing the bit stream to the engine selection mechanism AP2D02, the bit stream separated from the protocol such as the header is delivered to the three decoding engines.

The above three decoding engines are run-length decoding engines AP2D for performing run-length decoding.
03, a Huffman decoding engine AP2D04 for performing Huffman decoding, and an inter-frame correlation decoding engine AP2D05 for performing inter-frame correlation decoding. These decryption engines AP2D03, AP2D04 and AP2D0
Reference numeral 5 writes the decoded image data into the screen frame information storage mechanism (memory) AP2D07 and AP2D08.

The screen frame information storage mechanism AP2D
07 and AP2D08 are the same double buffers as the two frame buffers AP1B04 and AP1B05, temporarily store screen frame information, respectively, and decode engines AP2D03, AP2D04 and AP2D05.
To store the reconstructed image information.

The decryption engines AP2D03, AP2
2D04 is the screen frame information storage mechanism AP2D0.
7 and AP2D08, but the inter-frame correlation decoding engine AP2D05 needs to read the immediately preceding screen information because it needs the inter-frame information. For this reason, the screen frame information storage mechanism (memory) has a double buffer configuration as shown by AP2D07 and AP2D08.

In FIG. 3, the screen frame information storage mechanism (memory) is divided into two parts for the sake of convenience in the embodiment described later, since it is convenient in pursuing the operation. However, the hardware configuration is the same as that of the two frame buffers AP1B04 and AP1B05 described above. Decryption engines AP1B06, AP1B07 and A
P1B08 operates with only one of them selected by the decoding engine selection mechanism AP2D02.

Bitmap drawing command issuing mechanism AP2D0
9 is reconstructed by the decoding engines AP2D03, AP2D04 and AP2D05 and the frame buffer AP
This is a mechanism for issuing a drawing command for drawing the same image information stored in 2D07 or AP2D08 on the display device DP1A as that drawn by AP1A of the AP-shared application program.

Next, the operation of the embodiment of the present invention will be described. Since the present invention solves the problem of sharing screen information, the input system is configured with existing technology. State.

In FIG. 1, a shared application program AP1A is connected to a keyboard device KB1A of a computer PC1A and a computer P1 to share an AP.
The keyboard device KB2A of C2A is operated in the same manner. The mechanism is an application shared program operation system reflection mechanism AP1C and an application shared program operation system acquisition mechanism AP2E.
Has been realized.

Input from the keyboard device KB1A,
The input value of the user of the computer PC1A is the route P02.
0, and is input to the keyboard input hardware KY1A via the path P021, the processor hardware and the system bus CPU1A, the path P022, the operating system OS1A, and the path P023.
Delivered to P1C.

Similarly, the input value of the user of the computer PC2A input from the keyboard device KB2A is input to the keyboard input hardware KY2 via the path P040.
A, and the path P041, the processor hardware and the system bus CPU2A, the path P04
2. Operating system OS2A and path P04
After that, the program is passed to the application-sharing-program operation-system acquisition mechanism AP2E. Furthermore, this is the route P04
4. Operating system OS2A, path P04
5. Processor hardware and system bus CPU2
A, path P046, communication port hardware COM2
A, and is output to the communication line NET1A via the path P047.

The computer 1A communicates with the communication line NET1.
The value output from PC2A to PCA is obtained by communication port hardware COM1A via path P024, and from this, the application is further processed via path P025, processor hardware and system bus CPU1A, path P026, operating system OS1A, path P027. Deliver to shared program operation system reflection mechanism AP1C.

The application sharing program operation system reflection mechanism AP1C combines the information from the path P023 and the information from the path P027 therein. Synthesizing means arranging data as if one user is inputting from one input device, while arbitrating appropriately in a time-division manner.

The input data generated in this way is
It is delivered to the shared application program AP1A through the path P028, the operating system OS1A, the path P029, the processor hardware and the system bus CPU1A, the operating system OS1A, and the path P030.

Application program to be shared AP1
A can perform normal processing as an application program, assuming that this is input from one input device.

In FIG. 1, the keyboard device KB1A is shown.
And in the block of KB2A, the mouse is described in parentheses, which indicates that the same route as that of the keyboard data can be applied to the mouse. Because the data exchange procedure is exactly the same as that of the keyboard, if the same thing is described in a block diagram, the whole figure will be complicated and difficult to see. 1, modules (blocks) and data paths (arrow lines) are not shown and are omitted.

[0087]

【Example】

(First Embodiment) The first embodiment is an example in which an initial screen is output after AP sharing is started. When the user of the computer PC1A on the sharing application providing side shown in FIG. 1 starts the program and starts the AP sharing, the P1A transmits the screen data of the first screen to the PC2A located at a remote place via the communication line NET.
Transfer via 1A. That is, assuming that the shared application program AP1A shown in FIG. 1 displays a plain window on the initial screen in the computer PC1A on the shared application providing side, first, the shared application program AP1A draws a window. Start with the operation. The screen frame information storage mechanisms AP1B04 and AP1B, which are memories, are provided.
05, AP2D07, and AP2D08 are initially initialized when the user starts AP sharing.

The drawing processing command (P001) issued by the computer PC1A is transferred from the shared application program AP1A to the operating system OS1A as shown in FIG. 1, and the OS1A converts the drawing processing command into data for drawing on the screen. Via the processor hardware and the system bus CPU 1A (P002), a drawing processing command is issued to the graphics hardware GP1A connected to the system bus via the path indicated by P003.

The application sharing program screen acquisition mechanism AP1B stores this information in storage means (generally, video random data) in the graphics hardware GP1A.
From the access memory (VRAM) or the like, the operation of extracting as bitmap information is started.

First, upon receiving a bitmap graphics acquisition request from the application sharing program screen acquisition mechanism AP1B, the bitmap information of the graphics hardware GP1A drawn by the computer PC1A becomes
The program is passed to the application sharing program screen acquisition mechanism AP1B via the path P005, the CPU 1A, the path P006, the operating system OS1A, and the path P007.

Here, the bitmap acquired by the application sharing program screen acquiring mechanism AP1B shown in FIG. 2 is transmitted via the path P101 to the screen change detecting mechanism AP1B0.
1 and the screen acquisition mechanism AP1B02. Here, the screen change detection mechanism AP1B01 refers to the data already stored in one of the screen frame information storage mechanisms (memory) AP1B04 and AP1B05 via the path P103, but as described above, AP1B04 and AP
Since 1B05 has been initialized, it differs from the content drawn based on the drawing process command, so that a command is output to the screen acquisition mechanism AP1B02 via the route P104, thereby enabling, for example, the memory AP1B02 on the route P106, that is, permitting the writing operation. Output a signal to

At the same time, the screen change detection mechanism AP1B
01 also notifies the signal to the path P104 to the screen frame information switching management mechanism AP1B03, and thereby the head address of the screen frame information, specifically, the memory AP1B0
4 and the start address of the memory AP1B05 are toggled in preparation for the next reading.

This screen frame information switching management mechanism AP1
The screen acquisition mechanism AP1B02 alternately enables the memories AP1B04 and AP1B05 on the path P106 according to the address input from B03 via the path P105. Therefore, the memories AP1B04 and AP1B0
Either of the contents 5 is the bitmap information of the newly changed screen, and either of them is the bitmap information of the immediately preceding screen. Regardless of which one is started, the role is toggled (reversed) each time there is writing by the screen frame information switching management mechanism AP1B03, so that the operation is performed without any problem.
It is assumed that the process starts from a state where the screen information immediately before and the newly acquired screen information are stored in the memory AP1B05.

That is, the memory AP1B04 retains the initialized state, which is the immediately preceding state of all 0 (zero), and the memory AP1B05 stores the AP1A in this phase.
Will be stored as bitmap information of the drawing result.

Next, from the memory AP1B05 to the path P10
8, the bitmap information is transferred to the encoding engine AP1B0.
6, the bitmap information input to each of AP1B07 and AP1B08 and stored in the memory AP1B04 passes through the path P109 to the encoding engine AP1.
B08 only, and each encoding engine AP1B
06, AP1B07 and AP1B08.

When the encoding is completed, the encoding engines AP1B06, AP1B07 and AP1B08 transmit the respective code amount data to the paths P110 and P1.
11, the code amount comparison mechanism AP1B09 via P112
Hand over to The code amount comparison unit AP1B09 performs a comparison operation on these three code amount data, and as a result, switches the bit stream via the path P116 so as to select the output data of the encoding engine that outputs the smallest code amount. The result is notified to the mechanism AP1B10.

Here, in the first embodiment, the screen data of this time is data continuing a solid continuous area, so that the code amount of the run-length encoding engine AP1B06 is the smallest, so that the run-length encoding engine A
The output bit stream P113 of P1B06 is selected. This bit stream P113 is directly transferred to the path P1.
17 and is passed to the transmission protocol addition mechanism AP1B11, where the header is added.
It is output to C2A via paths P118 and P008.

Although the way of attaching the header is not related to the present invention, as an example, the run-length encoding engine AP1B
06 is selected as the first byte of the data when the bit stream output from the Huffman encoding engine AP1B07 is selected, and when the bit stream output from the Huffman encoding engine AP1B07 is selected, the first byte of the data is set as the first byte of the data. 1
When the bit stream output from the inter-frame correlation encoding engine AP1B08 is selected by adding hexadecimal "02" to the first byte of the data, hexadecimal "03" is added to the first byte of the data.
Is assumed to be a convention to be added.

Then, since the data output from the run-length encoding engine AP1B06 is to be transmitted in the currently processed screen data, a hexadecimal number “01” is added to the first byte of the data in the header according to the above-mentioned rules. Then, data obtained by synthesizing the bit stream in time series is output.

Returning to FIG. 1, the data output from the application sharing program screen acquisition mechanism AP1B via the path P008 is transmitted to the operating system OS1.
A, path P009, processor hardware CPU1
A, communication port hardware COM via path P010
1A, and then put on the communication line NET1A via the path P011.

The remote application side computer PC2A transmits this data to the communication port hardware COM2 from the communication line NET1A via the path P030.
A, and further from this, the path P031, the processor hardware and the system bus CPU 2A, the path P03
2. Operating system OS2A and path P0
33 to the application sharing program screen playback / display mechanism AP2D.

As shown in FIG. 3, the application sharing program screen reproducing / displaying mechanism AP2D supplies the communication data obtained from the computer PC1A to the transmission protocol separating mechanism AP2D01 via the path P201, where the bit stream separation and transmission are performed. , By header analysis,
Information for selecting a decoding engine to which this bit stream is to be sent is extracted.

Information for selecting a decoding engine to which this bit stream is to be sent is transferred from the transmission protocol separating mechanism AP2D01 to the decoding engine selecting mechanism AP2D02 via the path P202. In the first embodiment, at the time of encoding, a run-length encoding engine AP
1B06, the decryption engine selection mechanism AP2
D02 makes only the run-length encoding engine AP2D03 operable via the path P204 based on the input selection information, and performs decoding by the run-length decoding engine AP2.
It is performed in 2D03.

Run-length decoding engine AP2D03
Is the path P20 from the transmission protocol separation mechanism AP2D01.
In step 8, the data is received and decoded, and the obtained decoding result is stored in the memory AP2D08 (or the memory AP2D07). Here, the memory AP2D08 and the memory AP2
Which of D07 is to be written depends on the memory AP2B.
In the same way as when writing to either AP04 or AP2B05, the role is reversed each time, so there is no functional inconvenience whichever is used first, but temporarily share the AP and write to the memory AP2D08 for the first time. Was assumed.

Next, the bitmap drawing command issuing mechanism AP
The 2D09 reads out the decoded drawing data stored in the memory AP2D08 via the path P213, and issues a bitmap drawing command to the path P214.

Operating system OS shown in FIG.
2A transmits the drawing command issued by the application sharing program screen reproduction display mechanism AP2D in the path P
034, which is converted into an instruction to be issued to the graphics hardware GP2A, re-issued to the path P035, and the processor hardware and the system bus CP
Handover to U2A, and further from this via route P036 to C
The decoded drawing data stored in the memory AP2D08 is read and written to the graphics hardware GP2A connected to the PU 2A.

The graphics hardware GP2A outputs to the display device DP2A via the path P037, thereby displaying the same image as the image output to the display device DP1A. Thus, the display devices DP1A and DP2A can display the same screen.

(Second Embodiment) The second embodiment is an example in which the shared application program AP1A performs drawing as shown in FIGS. 4A to 4B. FIG. 4A shows a plain window drawn in the first embodiment. FIG. 4B shows a screen as a result of newly drawing by the shared application program AP1A in the second embodiment.

In the block diagram of the application sharing program screen acquisition mechanism AP1B shown in FIG. 2, the operation will be described from the point that the application program to be shared AP1A already performs the drawing of FIG. 4B in the computer PC1A.

First, the shared application program AP1A draws the screen information shown in FIG. 4B. Then, the drawing processing command issued by the computer PC1A is:
Operating system OS via path P001
Over 1A, the OS 1A converts the drawing processing command into data to be drawn on the screen. This conversion data is stored in the route P002,
It is supplied to the processor hardware and the graphics hardware GP1A connected to the system bus of the system bus CPU1A to issue a drawing processing instruction.

The application sharing program screen acquisition mechanism AP1B extracts the information as bitmap information from the storage means in the graphics hardware GP1A.

Here, upon receiving a bitmap graphics acquisition request from the application sharing program screen acquisition mechanism AP1B, the GP drawn by the computer PC1A is displayed.
The bitmap information of 1A is stored in the path P005, the CPU 1
A, path P006, operating system OS1
A, It is delivered to the application sharing program screen acquisition mechanism AP1B via the path P007.

In FIG. 2, the bitmap acquired by the application sharing program screen acquisition mechanism AP1B passes through the path P101 and is displayed on the screen change detection mechanism AP1B01.
And transferred to the screen acquisition mechanism AP1B02. In the first embodiment, the immediately preceding screen information is stored in the memory AP1B04, and the newly captured screen information is stored in the memory AP1B05. On the other hand, in the second embodiment, since it is assumed that the operation is performed following the first embodiment, the address pointer of the screen frame information switching management mechanism AP1B03 is toggled with respect to the first embodiment and the role is reversed. Should be doing. Therefore, in the second embodiment, the immediately preceding screen information is stored in the memory AP1B05, and the newly captured screen information is stored in the memory AP1B04.

That is, the screen change detection mechanism AP1B01
Refers to data that has already been stored,
With reference to P1B05, it is detected that the image information acquired this time does not match, and the screen acquisition mechanism AP1B02 is enabled, that is, a write operation is permitted. At the same time, the screen change detection mechanism AP1B01 also notifies the signal to the path P104 to the screen frame information switching management mechanism AP1B03, whereby the head address of the screen frame information, specifically, the head address of the memory AP1B04 and the memory A
The leading address of P1B05 is toggled in preparation for the next reading.

Next, from the memory AP1B04 to the path P10
8, the bitmap information is transferred to the encoding engine AP1B0.
6, the bitmap information input to each of AP1B07 and AP1B08 and stored in the memory AP1B05 passes through the path P109 to the encoding engine AP1.
B08 only, and each encoding engine AP1B
06, AP1B07 and AP1B08.

When the encoding is completed, the encoding engines AP1B06, AP1B07 and AP1B08 transmit the respective code amount data to the paths P110 and P1.
11, the code amount comparison mechanism AP1B09 via P112
Hand over to The code amount comparison unit AP1B09 performs a comparison operation on these three code amount data, and as a result, switches the bit stream via the path P116 so as to select the output data of the encoding engine that outputs the smallest code amount. The result is notified to the mechanism AP1B10.

Here, in the second embodiment, FIG.
Since the screen data of this time has been rewritten to a very complicated graphic shown in FIG. 4B from the data in which a continuous continuous area as shown in FIG. 4B continues, the code amount comparison mechanism AP1B09
As a result, the result that the code amount of the output bit stream of the Huffman coding engine AP1B07 is the smallest is obtained.

As a result, the bit stream switching mechanism A
P1B10 selects the output bit stream of Huffman coding engine AP1B07 output to path P114. The bit stream on the path P114 is passed as it is to the transmission protocol adding mechanism AP1B11 through the bit stream switching mechanism AP1B10, where the header is added.

When the header rules described in the first embodiment are applied, the data output from the Huffman encoding engine AP1B07 is transmitted in the currently processed screen data. A hexadecimal number “02” is added as a header to the first byte of the data.

Returning to FIG. 1, the explanation will be given again.
The data output in 008 is the operating system OS1A, the path P009, the processor hardware C
It is handed over to the communication port hardware COM1A via the PU1A and the path P010, and is then put on the communication line NET1A via the path P011.

The remote application side computer PC2A transmits this data to the communication port hardware COM2 from the communication line NET1A via the path P030.
A, and further from this, the path P031, the processor hardware and the system bus CPU 2A, the path P03
2. Operating system OS2A and path P0
33 to the application sharing program screen playback / display mechanism AP2D.

As shown in FIG. 3, the application sharing program screen reproducing / displaying mechanism AP2D supplies the communication data obtained from the computer PC1A to the transmission protocol separating mechanism AP2D01 via the path P201, where the bit stream is separated and transmitted. , By header analysis,
Information for selecting a decoding engine to which this bit stream is to be sent is extracted.

Information for selecting a decoding engine to which this bit stream is to be sent is transferred from the transmission protocol separating mechanism AP2D01 to the decoding engine selecting mechanism AP2D02 via the path P202. In the second embodiment, at the time of encoding, the Huffman encoding engine AP1B
07, the decryption engine selection mechanism AP2D0
2 makes only the Huffman decoding engine AP2D04 operable via the path P205 based on the input selection information, and performs decoding by the Huffman decoding engine AP2D04.

The Huffman decoding engine AP2D04 receives and decodes the data from the transmission protocol separating mechanism AP2D01 via the path P208, and stores the obtained decoding result in the memory AP2D07 (the role is reversed since the previous embodiment uses AP2D08). I remember).

Next, the bitmap drawing command issuing mechanism AP
The 2D09 reads out the decoded drawing data stored in the memory AP2D07 via the path P213, issues a bitmap drawing command to the path P214, and performs drawing.

Returning to FIG. 1 again, the operating system OS2A receives the drawing command issued by the application shared program screen playback / display mechanism AP2D via the path P034 in this way, and sends it to the graphics hardware GP2A. The instruction is converted into an instruction to be issued, reissued to the path P035, passed to the processor hardware and the system bus CPU 2A, and further written into the graphics hardware GP2A connected to the CPU 2A via the path P036.

The graphics hardware GP2A outputs to the display device DP2A via the path P037, thereby displaying the same image as the image output to the display device DP1A. Thus, the display devices DP1A and DP2A can display the same screen.

(Third Embodiment) The third embodiment is an example in which the shared application program AP1A performs the drawing as shown in FIG. 5 immediately after the operation of the second embodiment. FIG. 5A shows a window drawn in the second embodiment. FIG. 5B shows a screen as a result of newly drawing by the shared application program AP1A in the third embodiment.

In the configuration diagram of the application sharing program screen acquisition mechanism AP1B shown in FIG. 2, the operation will be described from the point that the application program to be shared AP1A already performs the drawing of FIG. 5B in the computer PC1A.

First, the shared application program AP1A draws the screen information shown in FIG. 5B. Then, the drawing processing command issued by the computer PC1A is:
Operating system OS via path P001
Over 1A, the OS 1A converts the drawing processing command into data to be drawn on the screen. This conversion data is stored in the route P002,
It is supplied to the processor hardware and the graphics hardware GP1A connected to the system bus of the system bus CPU1A to issue a drawing processing instruction.

The application sharing program screen acquisition mechanism AP1B extracts the information from the storage means in the graphics hardware GP1A as bitmap information.

Here, upon receiving the bitmap graphics acquisition request from the application sharing program screen acquisition mechanism AP1B, the GP drawn by the computer PC1A is displayed.
The bitmap information of 1A is stored in the path P005, the CPU 1
A, path P006, operating system OS1
A, It is delivered to the application sharing program screen acquisition mechanism AP1B via the path P007.

In FIG. 2, the bitmap acquired by the application sharing program screen acquisition mechanism AP1B passes through the path P101 and is displayed on the screen change detection mechanism AP1B01.
And transferred to the screen acquisition mechanism AP1B02. In the second embodiment, the immediately preceding screen information is stored in the memory AP1B05, and the newly captured screen information is stored in the memory AP1B04. On the other hand, in the third embodiment, since it is assumed that the operation is performed following the second embodiment, the address pointer of the screen frame information switching management mechanism AP1B03 is toggled with respect to the second embodiment to reverse the role. Should be doing. Therefore, in the third embodiment, the immediately preceding screen information is stored in the memory AP1B04, and the newly captured screen information is stored in the memory AP1B05.

That is, the screen change detection mechanism AP1B01
Refers to data that has already been stored,
With reference to P1B04, it is detected that the image information acquired this time does not match, and the screen acquisition mechanism AP1B02 is enabled, that is, a write operation is permitted. At the same time, the screen change detection mechanism AP1B01 also notifies the signal to the path P104 to the screen frame information switching management mechanism AP1B03, whereby the head address of the screen frame information, specifically, the head address of the memory AP1B05 and the memory A
The leading address of P1B04 is toggled in preparation for the next reading.

Next, from the memory AP1B04 to the path P10
8, the bitmap information is transferred to the encoding engine AP1B0.
6, the bitmap information input to each of AP1B07 and AP1B08 and stored in the memory AP1B05 passes through the path P109 to the encoding engine AP1.
B08 only, and each encoding engine AP1B
06, AP1B07 and AP1B08.

When the encoding is completed, the encoding engines AP1B06, AP1B07 and AP1B08 transmit the respective code amount data to the paths P110 and P1.
11, the code amount comparison mechanism AP1B09 via P112
Hand over to The code amount comparison unit AP1B09 performs a comparison operation on these three code amount data, and as a result, switches the bit stream via the path P116 so as to select the output data of the encoding engine that outputs the smallest code amount. The result is notified to the mechanism AP1B10.

Here, in the third embodiment, the screen data of this time is a very complicated figure as shown in FIG. 5B, but compared with the screen information immediately before shown in FIG. 5A, Since a part of the data is changed so that the screen is rewritten, the code amount comparison mechanism AP1B09 obtains the result that the code amount data from the inter-frame correlation coding engine AP1B08 has the smallest code amount.

As a result, the bit stream switching mechanism A
P1B10 selects the output bit stream of the inter-frame correlation encoding engine AP1B08 output to the path P115. The bit stream on the path P115 is passed as it is to the transmission protocol addition mechanism AP1B11 through the bit stream switching mechanism AP1B10, where the header is added, and then output to the computer PC2A via the paths P118 and P008.

When the header rules described in the first embodiment are applied, the data output from the inter-frame correlation encoding engine AP1B08 is transmitted for the currently processed screen data. Therefore, a hexadecimal number “03” is added as a header to the first byte of the data.

Returning again to FIG. 1, the path sharing from the application sharing program screen acquisition mechanism AP1B to the path P
The data output in 008 is the operating system OS1A, the path P009, the processor hardware C
It is handed over to the communication port hardware COM1A via the PU1A and the path P010, and is then put on the communication line NET1A via the path P011.

The computer PC2A for sharing this data uses the communication port hardware COM2 from the communication line NET1A via the path P030.
A, and further from this, the path P031, the processor hardware and the system bus CPU 2A, the path P03
2. Operating system OS2A and path P0
33 to the application sharing program screen playback / display mechanism AP2D.

As shown in FIG. 3, the application sharing program screen reproduction / display mechanism AP2D supplies the communication data obtained from the computer PC1A to the transmission protocol separation mechanism AP2D01 via the path P201, where the bit stream separation and the transmission protocol separation are performed. , By header analysis,
Information for selecting a decoding engine to which this bit stream is to be sent is extracted.

The information for selecting the decoding engine to which this bit stream is to be sent is transferred from the transmission protocol separating mechanism AP2D01 to the decoding engine selecting mechanism AP2D02 via the path P202. In the third embodiment, at the time of encoding, an inter-frame correlation encoding engine A
Since P1B08 was used, the decryption engine selection mechanism AP
In 2D02, only the inter-frame correlation decoding engine AP2D05 is enabled via the path P206 based on the input selection information, and decoding is performed by the inter-frame correlation decoding engine AP2D05.

Inter-frame correlation decoding engine AP2D0
5 is a path P2 from the transmission protocol separation mechanism AP2D01.
08, the data is taken and decoded, and the obtained decoding result is stored in the memory AP2D08 (AP2D in the previous second embodiment).
07, the role is reversed).

Next, the bitmap drawing command issuing mechanism AP
The 2D09 reads out the decoded drawing data stored in the memory AP2D08 via the path P213, issues a bitmap drawing command to the path P214, and performs drawing.

Returning to FIG. 1, the operating system OS2A receives the drawing command issued by the application sharing program screen reproduction / display mechanism AP2D via the path P034 in this way, and sends it to the graphics hardware GP2A. The instruction is converted into an instruction to be issued, reissued to the path P035, passed to the processor hardware and the system bus CPU 2A, and further written into the graphics hardware GP2A connected to the CPU 2A via the path P036.

The graphics hardware GP2A outputs to the display device DP2A via the path P037, thereby displaying the same image as the image output to the display device DP1A. Thus, the display devices DP1A and DP2A can display the same screen.

(Fourth Embodiment) The fourth embodiment is an example in which the application program to be shared AP1A performs drawing as shown in FIG. 6 immediately after the operation of the second embodiment. FIG. 6A shows a window drawn in the second embodiment. FIG. 6B shows a screen as a result of newly drawing by the shared application program AP1A in the fourth embodiment.

In the configuration diagram of the application sharing program screen acquisition mechanism AP1B shown in FIG. 2, the operation will be described from the point that the application program to be shared AP1A already performs drawing in FIG. 6B on the computer PC1A.

First, the shared application program AP1A draws the screen information shown in FIG. 6B. Then, the drawing processing command issued by the computer PC1A is:
Operating system OS via path P001
Over 1A, the OS 1A converts the drawing processing command into data to be drawn on the screen. This conversion data is stored in the route P002,
It is supplied to the processor hardware and the graphics hardware GP1A connected to the system bus of the system bus CPU1A to issue a drawing processing instruction.

The application sharing program screen acquisition mechanism AP1B extracts the information from the storage means in the graphics hardware GP1A as bitmap information.

Here, upon receiving the bitmap graphics acquisition request from the application sharing program screen acquisition mechanism AP1B, the GP drawn by the computer PC1A is displayed.
The bitmap information of 1A is stored in the path P005, the CPU 1
A, path P006, operating system OS1
A, It is delivered to the application sharing program screen acquisition mechanism AP1B via the path P007.

In FIG. 2, the bitmap acquired by the application sharing program screen acquisition mechanism AP1B passes through the path P101 and is displayed on the screen change detection mechanism AP1B01.
And transferred to the screen acquisition mechanism AP1B02. In the second embodiment, the immediately preceding screen information is stored in the memory AP1B05, and the newly captured screen information is stored in the memory AP1B04. On the other hand, in the fourth embodiment, it is assumed that the operation is performed following the second embodiment. Therefore, the address pointer of the screen frame information switching management mechanism AP1B03 is toggled with respect to the second embodiment and the role is reversed. Should be doing. Accordingly, in the fourth embodiment, the immediately preceding screen information is stored in the memory AP1B04, and the newly captured screen information is stored in the memory AP1B05.

That is, the screen change detection mechanism AP1B01
Refers to data that has already been stored,
With reference to P1B04, it is detected that the image information acquired this time does not match, and the screen acquisition mechanism AP1B02 is enabled, that is, a write operation is permitted. At the same time, the screen change detection mechanism AP1B01 also notifies the signal to the path P104 to the screen frame information switching management mechanism AP1B03, whereby the head address of the screen frame information, specifically, the head address of the memory AP1B05 and the memory A
The leading address of P1B04 is toggled in preparation for the next reading.

Next, from the memory AP1B04 to the path P10
8, the bitmap information is transferred to the encoding engine AP1B0.
6, the bitmap information input to each of AP1B07 and AP1B08 and stored in the memory AP1B05 passes through the path P109 to the encoding engine AP1.
B08 only, and each encoding engine AP1B
06, AP1B07 and AP1B08.

When the encoding is completed, the encoding engines AP1B06, AP1B07 and AP1B08 transmit the respective code amount data to the paths P110 and P1.
11, the code amount comparison mechanism AP1B09 via P112
Hand over to The code amount comparison unit AP1B09 performs a comparison operation on these three code amount data, and as a result, switches the bit stream via the path P116 so as to select the output data of the encoding engine that outputs the smallest code amount. The result is notified to the mechanism AP1B10.

Since the current screen data is a very simple figure as shown in FIG. 6B in the fourth embodiment, it is transmitted from the run-length encoding engine AP1B06 by the code amount comparing mechanism AP1B09. Is obtained that the code amount data has the smallest code amount.

Thus, the bit stream switching mechanism A
P1B10 selects the output bit stream of the run-length encoding engine AP1B06 output to the path P113. The bit stream of the path P113 is passed as it is to the transmission protocol addition mechanism AP1B11 through the bit stream switching mechanism AP1B10, where the header is added, and then output to the computer PC2A via the paths P118 and P008.

As for the method of attaching the header, when the rules of the header described in the first embodiment are applied, the data output from the run-length encoding engine AP1B08 is transmitted in the screen data currently being processed. , A hexadecimal number “01” is added as a header to the first byte of the data.

Returning to FIG. 1, the description will be continued.
The data output in 008 is the operating system OS1A, the path P009, the processor hardware C
It is handed over to the communication port hardware COM1A via the PU1A and the path P010, and is then put on the communication line NET1A via the path P011.

The computer PC2A for sharing this data uses the communication port hardware COM2 from the communication line NET1A via the path P030.
A, and further from this, the path P031, the processor hardware and the system bus CPU 2A, the path P03
2. Operating system OS2A and path P0
33 to the application sharing program screen playback / display mechanism AP2D.

As shown in FIG. 3, the application sharing program screen reproducing / displaying mechanism AP2D supplies the communication data obtained from the computer PC1A to the transmission protocol separating mechanism AP2D01 via the path P201, where the bit stream separation and transmission are performed. , By header analysis,
Information for selecting a decoding engine to which this bit stream is to be sent is extracted.

Information for selecting a decoding engine to which this bit stream is to be sent is transferred from the transmission protocol separating mechanism AP2D01 to the decoding engine selecting mechanism AP2D02 via the path P202. In the fourth embodiment, at the time of encoding, the run-length encoding engine AP
1B06, the decryption engine selection mechanism AP2
D02 makes only the run-length decoding engine AP2D03 operable via the path P204 based on the input selection information, and sets the decoding to the run-length decoding engine AP2.
It is performed in 2D03.

Inter-frame correlation decoding engine AP2D0
5 is a path P2 from the transmission protocol separation mechanism AP2D01.
08, the data is taken and decoded, and the obtained decoding result is stored in the memory AP2D08 (AP2D in the previous second embodiment).
07, the role is reversed).

Next, the bitmap drawing command issuing mechanism AP
The 2D09 reads out the decoded drawing data stored in the memory AP2D08 via the path P213, issues a bitmap drawing command to the path P214, and performs drawing.

Returning to FIG. 1 again, the operating system OS2A receives the drawing command issued by the application shared program screen reproduction / display mechanism AP2D via the path P034 in this way, and sends it to the graphics hardware GP2A. The instruction is converted into an instruction to be issued, reissued to the path P035, passed to the processor hardware and the system bus CPU 2A, and further written into the graphics hardware GP2A connected to the CPU 2A via the path P036.

The graphics hardware GP2A outputs the image to the display device DP2A via the path P037, thereby displaying the same image as the image output to the display device DP1A. Thus, the display devices DP1A and DP2A can display the same screen.

[0168]

As described above, according to the present invention,
Since the encoding method that can suppress the amount of information most efficiently according to the display screen is automatically selected from a plurality of types of encoding methods prepared in advance, the shared application remote use side can be used from the shared application providing computer. The information amount of data transmitted on the communication line to the computer can be suppressed as compared with the conventional case, and it is particularly effective when applied to a network where the transmission band capacity (bit rate) of the communication line is not sufficient.

Further, according to the present invention, a high probability occurs when the user performs a partial editing / modifying operation on a screen that frequently occurs during operation and requires high-speed response in a user interface. Since high-speed screen responsiveness to screen transition can be ensured, better operability can be provided to a user who uses AP sharing remotely.

[Brief description of the drawings]

FIG. 1 is an overall configuration diagram of a system according to an embodiment of the present invention.

FIG. 2 is a configuration diagram of an example of an application sharing program screen acquisition mechanism in FIG.

FIG. 3 is a configuration diagram of an example of an application sharing program screen reproduction display mechanism in FIG. 1;

FIG. 4 is a diagram illustrating a screen before an application changes screen information according to a second embodiment and a screen as a result of drawing and changing by the application;

FIG. 5 is a diagram illustrating a screen before an application changes screen information according to a third embodiment and a screen as a result of drawing and changing by the application;

FIG. 6 is a diagram illustrating a screen before the application changes the screen information and a screen as a result of the drawing and the change by the application according to the fourth embodiment.

FIG. 7 is a schematic conceptual diagram of application sharing.

[Explanation of symbols]

AP1A Application-shared program AP1B Application-shared program screen acquisition mechanism AP1B01 Screen change detection mechanism AP1B02 Screen acquisition mechanism AP1B03 Screen frame information switching management mechanism (mainly managing memory addresses) AP1B04, AP1B05, AP2D07, AP2D
08 Screen frame information storage mechanism AP1B06 Run-length coding engine AP1B07 Huffman coding engine AP1B08 Inter-frame correlation coding engine AP1B09 Code amount comparison mechanism AP1B10 Bit stream switching mechanism AP1B11 Transmission protocol addition mechanism (header processing) AP1C Application sharing program operation system reflection Mechanism AP2D application sharing program screen reproduction and display mechanism AP2D01 transmission protocol separation mechanism (header processing) AP2D02 decoding engine selection mechanism AP2D03 run-length decoding engine AP2D04 Huffman decoding engine AP2D05 inter-frame correlation decoding engine AP2D06 screen frame information switching management mechanism Memory address management) AP2D09 Bitmap drawing command issued Line mechanism (drawing A
PI issuance) AP2E Application shared program operation system acquisition mechanism APM1 Existing application sharing means in operation APM2 Remote application sharing means APU1 Existing user application program running on PC1 COM1A, COM2A Communication port hardware CPU1A, CPU2A Processor hardware And system bus DP1A, DP2A Display device GP1A, GP2A Graphics hardware KB1A, KB2A Keyboard device (user input means including mouse) KY1A, KY2A Keyboard input port hardware (same for mouse) NET1, NET1A Communication line OS1A , OSA2A Operating system PC1, PC1A Computer system (shared application Shon provider) PC2, PC 2A computer system (shared application remote use side)

Claims (3)

[Claims] A computer is connected to a plurality of computers by a communication line, an application program is operated by only one of the computers, and its screen information and operation information are shared by another computer. In the encoding method for screen data transfer in a system that supports cooperative work with a computer, one computer on which the application program operates, before transmitting the screen data drawn by the application program ram to the communication line, Encoding means for encoding separately and simultaneously with a plurality of types of compression encoding algorithms; and compression encoding with the least code amount among bit streams obtained by encoding the plurality of types of compression encoding algorithms, respectively. Select the algorithm bitstream Selecting means, and means for transferring the bit stream selected by the selecting means to the other computer via the communication line after adding at least a header indicating the type of the compression encoding algorithm, The other computer includes an identification unit that identifies an encoding method of the bit stream from the header in the signal input via the communication line, and a plurality of types of decoding corresponding to the plurality of types of compression encoding algorithms. A decoding means for separately and simultaneously decoding an input bit stream into screen data by an algorithm, and a decoding algorithm corresponding to the encoding scheme identified by the identification means in output decoded screen data of the decoding means. Decoding data selecting means for selecting only the decoded screen data; Means for displaying the decoded screen data selected by the signal data selecting means on a display device. 2. The encoding means according to claim 1, wherein said current screen data is data in which only a part of the previous screen data is changed, said inter-frame correlation encoding algorithm having the least code amount being at least said plurality of types. The encoding method for screen data transfer according to claim 1, wherein the encoding method is included as one of the compression encoding algorithms. 3. The encoding unit has at least a run-length encoding engine, a Huffman encoding engine, and an inter-frame correlation encoding engine as the plurality of types of compression encoding algorithms. 2. The encoding system for screen data transfer according to claim 1, comprising at least a run-length decoding engine, a Huffman decoding engine, and an inter-frame correlation decoding engine as a plurality of types of compression decoding algorithms.