JP2018170588A - Model data distribution server and model data reception client - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve security strength in transmission / reception of model data representing an object shape and image processing in a server-client type system. A server 2 has an encoding processing unit 2b, a decoding program generation unit 2c, and a data transmission unit 2d. The encoding processing unit 2b generates encoded model data by encoding model data representing the shape of an object. The decoding program generation unit 2c is composed of a series of instructions for instructing a processing procedure for decoding the encoded model data, and is a decoding program executed by an image processing unit provided separately from the central processing unit on the client side. To generate. The data transmission unit 2d transmits the encoding model data and the decoding program to the client. [Selection diagram] Fig. 2

Description

  The present invention relates to a server and a client that transmit and receive model data representing the shape of an object.

  For example, Patent Document 1 discloses a configuration that increases the security strength when decrypting encrypted image data in a data processing system that includes a CPU (main processor device) and a GPU (sub processor device). ing. Specifically, the CPU transfers the encrypted image data to the GPU without decrypting it. The GPU expands the encrypted image data on an internal memory (device memory), and then performs processing such as decryption, decoding, and display. Since the encrypted image data is processed inside the GPU, the decrypted image data is not expanded on the memory of the CPU. As a result, it is possible to prevent image data decoded by a memory dump or the like from being acquired, and thus security strength is improved.

JP 2012-151642 A

  Patent Document 1 relates to a stand-alone system, not a server client type, and does not describe data transmission / reception between the server and the client, and details about decryption processing of encrypted image data. Is also not described.

  An object of the present invention is to improve the security strength in transmission / reception of model data representing the shape of an object and image processing in a server client type system.

  In order to solve such a problem, the first invention provides a model data distribution server having an encoding processing unit, a decoding program generation unit, and a data transmission unit. The encoding processing unit generates encoded model data by encoding model data representing the shape of the object. The decode program generation unit is composed of a series of instructions for instructing a processing procedure for decoding the encode model data, and on the client side, a decode program executed by an image processing unit provided separately from the central processing unit Generate. The data transmission unit transmits the encoding model data and the decoding program to the client.

  Here, in the first invention, the encoding processing unit encodes the model data by a different process for each distribution of the model data, and the decoding program generation unit generates a decoding program having a different processing content for each distribution target. It is preferable. The decoding program is preferably a shader program executed by the image processing unit.

  A second invention provides a model data receiving client having a data receiving unit and an image processing unit. The data receiving unit receives, from the server, encoded model data obtained by encoding model data representing the shape of an object, and a decoding program configured by a series of instructions for instructing a processing procedure for decoding encoded model data. The image processing unit is provided separately from the central processing unit, and generates an image by decoding the encoded model data by executing a decoding program when displaying the image.

  Here, in the second invention, it is preferable that the decoding program has different processing contents for each distribution target of the model data distributed from the server. The decoding program is preferably a shader program executed by the image processing unit.

  According to the present invention, eavesdropping and falsification by a third party can be prevented by transmitting and receiving model data between a server and a client in the form of encoded model data obtained by encoding model data. Also, the decoding of the encoding model data on the client side is performed by an image processing unit provided separately from the central processing unit executing a decoding program when displaying an image. The security strength can be improved by not decoding the model data on the network or on the main memory, but decoding the model data each time it is displayed (during drawing).

Overall view of server-client system Server block diagram Diagram showing an example of model data encoding Flow chart showing an example of a decoding program Client block diagram Explanatory drawing of flow of image processing in GPU

  FIG. 1 is an overall view of a server client type system according to the present embodiment. In this system 1, a server 2 that distributes model data and a large number of clients 3 that can receive the model data are connected via the Internet. In response to a request from the client 3, the server 2 distributes model data that has been encoded (hereinafter referred to as “encoded model data”) and a decoding program for decoding the encoded data as distribution data. . The feature of this embodiment is not to distribute the model data as it is, but to perform an encoding process on the model data and distribute the contents in a form that is difficult to identify (encoded model data). In this case, since the client 3 receiving the encoded model data cannot restore significant data as it is, a decoding program is transferred from the server 2 to the client 3 in order to enable the restoration. When the client 3 receives the distribution data, when displaying the image, the client 3 generates the image by restoring the encoded model data using the decoding program, and displays the image.

  Here, the “model data” is data representing the shape of the object, and may be either two-dimensional data or three-dimensional data. The 3D data may be 3D CAD data (file extensions such as IGES, STEP, PARASOLID, SAT, JT, VDA, etc.), 3DCG (3D computer graphics) data, 3D It may be point cloud data that expresses the surface shape of the image by a number of points. The “decode program” is a program for decoding the encoded model data, and is composed of a series of instructions for instructing a processing procedure for decoding the encoded model data. This “decode program” is executed on the client side by an image processing unit (typically, a graphics processing unit (hereinafter referred to as GPU)) provided separately from the central processing unit.

  The distribution of the encoded model data and the decoding program may be performed simultaneously, but may be distributed separately in order to improve the security strength, or may be obtained through a different route by accessing a predetermined site. It may be. Furthermore, when a well-known security enhancement technique such as user authentication or public key cryptography is used in combination with the model data distribution method according to the present embodiment, the security in data transmission / reception can be further enhanced.

  FIG. 2 is a block diagram of the server 2. The server 2 is mainly composed of a data request receiving unit 2a, an encoding processing unit 2b, a decoding program generating unit 2c, a data transmitting unit 2d, and a model data storage unit 2e.

  The data request receiving unit 2a receives from the client 3 a request to acquire designated model data. The encoding processing unit 2b refers to the model data storage unit 2e and acquires model data designated by the client 3 as a distribution target. Then, the encoding processing unit 2b generates encoded model data by encoding model data to be distributed using an encoding program.

  In response to the instruction from the encoding processing unit 2b, the decoding program generation unit 2c generates a decoding program for decoding the encoding model data to be distributed. This decoding program performs the reverse process of the encoding program applied to the distribution target. When the encoding program differs for each distribution target, the decoding program is also unique for each distribution target. . The data transmission unit 2d transmits the encoding model data generated by the encoding processing unit 2b and the decoding program generated by the decoding program generation unit 2c to the client 3.

  FIG. 3 is a diagram illustrating an example of encoding of model data. For example, when the model data is composed of a large number of vertices V composed of three-dimensional coordinates, as the simplest example, scrambling is possible in which the X, Y, and Z coordinates are switched according to a predetermined encoding rule. In the example of the figure, for the odd-numbered vertex V2n-1 (n is a natural number), the X coordinate is replaced with the Z coordinate, the Y coordinate is replaced with the X coordinate, and the Z coordinate and the Y coordinate are respectively replaced. Replaces the X coordinate with the Y coordinate, the Y coordinate with the Z coordinate, and the Z coordinate with the X coordinate. Further, the vertices V may be exchanged instead of the coordinate values. By performing such an encoding process, significant model data is converted into insignificant data. However, this is merely an example, and it should be noted that any encoding method may be used as long as the image processing unit on the client 3 side can perform calculation, not limited to scrambling.

  FIG. 4 is a flowchart showing an example of a decoding program corresponding to the encoding shown in FIG. This decoding program is composed of a series of instructions for instructing a processing procedure for decoding encoding model data to be distributed. Although details will be described later, in this embodiment, a shader program executed by an image processing unit (for example, a programmable shader) on the client 3 side is used as a decoding program. The example in the figure shows a case in which a vertex shader program that defines a vertex coordinate conversion method is used as a shader program.

  First, in step 1, a vertex format is defined. Next, in step 2, in order to tell the shader what kind of vertex information is to be handled, vertex data to be handled by the vertex shader is declared.

  Step 3 and subsequent steps are the shader program entity. In step 3, a vertex shader program is created and compiled into a form that can be understood by the image processing unit, and the compiled instruction is held in the buffer. In step 4, a vertex shader interface (shader handler) is generated using the shader program buffer. In step 5, the vertex shader interface is set to a stream, the fixed function pipeline is switched to a programmable pipeline, and rendering is performed.

  In the actual part after step 3, the reverse procedure of the above-described encoding is described as the decoding processing content. For example, in the case of the encoding shown in FIG. 3, for odd-numbered vertices V2n-1 (n is a natural number), the Z coordinate is replaced with the X coordinate, the X coordinate is replaced with the Y coordinate, the Y coordinate and the Z coordinate, respectively. For the second vertex V2n, the Y coordinate is replaced with the X coordinate, the Z coordinate is replaced with the Y coordinate, and the X coordinate is replaced with the Z coordinate.

  From the viewpoint of improving the security strength, it is preferable that the encoding method of the model data is changed for each distribution target. In this case, the encoding processing unit 2b encodes the model data by a different process for each delivery of the model data. Then, the decode program generation unit 2c generates a decode program having different processing contents for each distribution target in accordance with the encoding method adopted by the encoding processing unit 2b.

  FIG. 5 is a block diagram of the client 3. The client 3 is composed mainly of a CPU 3a (central processing unit), a GPU 3b as an image processing unit, a data receiving unit 3c, a storage device 3d such as a main memory, a GPU memory 3e, and a display device 3f. Has been. The storage device 3d can be accessed by the CPU 3a, and the GPU memory 3e can be accessed only by the GPU 3b.

  The data receiving unit 3c receives the encoding model data and the decoding program transmitted from the server 2. These data are stored in the storage device 3d. Since the model data stored in the storage device 3d is still encoded, it is difficult to extract significant data even if the storage device 3d is dumped. The GPU 3b is provided separately from the CPU 3a, and is exclusively responsible for image processing based on instructions from the CPU 3a. When displaying an image on the display device 3f, the GPU 3b executes the decoding program distributed from the server 2 to decode the encoded model data, that is, restore the significant model data to generate an image.

  FIG. 6 is an explanatory diagram of the flow of image processing in the GPU 3b. As an example, the flow of DirectX (particularly DirectX9), which is an API set for multimedia such as games developed and provided by Microsoft Corporation, is shown. First, “vertex data” is vertex data defined in local coordinates and stored in a vertex buffer. In the vertex buffer, positions, vertex colors, texture coordinates, and the like are set. “Primitive data” is a geometric figure composed of basic figures such as points, lines, and triangular polygons, and is defined as an order (index) buffer. Both the vertex buffer and the index buffer may be provided, but either one may be provided. These pieces of information are passed to “Tessellation”. This is a part for finely dividing the polygon. Usually this feature is often not supported, and sometimes you can get through without doing anything.

  The “fixed function pipeline” and the “programmable pipeline (vertex shader)” convert three-dimensional polygon information into a rectangular coordinate system. For programmable pipelines, programmers describe this vertex transformation in a unique way. Coordinates passing through these pipelines are condensed into a rectangular coordinate system. When the viewpoint is set to the front, it becomes a two-dimensional screen image, but at this time, a lot of unnecessary information is included. Therefore, unnecessary information is discarded while eliminating waste by “rear culling”, “clipping”, “attribute evaluation”, “rasterization”, and the like. With the above processing, vertex conversion and screen positioning are completed.

  Subsequent processes are not vertices, but various drawing processes and determinations are performed for each drawn point. The first “pixel shader” describes how to draw a polygon on the screen at the stage of rasterization, and finally “pixel color” is output. Programmers can customize the pixel shader to generate various effects. The “texture surface” is texture information, and the “texture sampler” applies various filters to the input texture. As a result, the pixel shader can extract an appropriate color for the designated texture coordinate (texel).

  After the pixel color on the screen is determined by the pixel shader, whether to draw or not is determined by "alpha test", "depth test", "stencil test", "fog test", "blending", etc. Is called. After these determinations, pixels are drawn in the buffer.

  The programmable shader inserts a unique instruction group into the “programmable pipeline (vertex shader)” and “pixel shader” portion of the GPU 3b. In the vertex shader and the pixel shader, an instruction group for giving an instruction to the GPU 3b is prepared in advance. In the present embodiment, among the shader programs executed by the programmable shader of the image processing unit 3b on the client 3 side, a vertex shader program (see FIG. 4) that defines a vertex coordinate conversion method is used as a decoding program. By executing this program, the GPU 3b performs image processing while sequentially decoding the encoded model data, and displays an image on the display device 3f. As the decoding program, a pixel shader program can be used instead of the vertex shader program.

  As described above, according to the present embodiment, model data is transmitted and received between the server 2 and the client 3 in the form of encoded model data obtained by encoding model data, thereby preventing eavesdropping and falsification by a third party. it can. Further, the decoding of the encoding model data on the client 3 side is performed by an image processing unit (typically GPU 3b) provided separately from the CPU 3a when displaying an image, by executing a decoding program. This image processing is performed using the GPU memory 3e without using the storage device 3d such as the main memory. In this way, the model data that has already been decoded is not arranged on the network or the main memory, but is decoded each time the model data is displayed (during drawing). In the encoding model data, only data necessary for forming a display screen (for one frame) is partially decoded sequentially, and all data is not expanded in the GPU memory 3e at once. Thereby, it is possible to improve the security strength in the transmission / reception of model data and image processing.

  In the above-described embodiment, the DirectX 9 in FIG. 6 has been described as an example. However, the latest DirectX 12 employs a different environment, and the WebGL operating in the browser is based on OpenGL. The environment is different. The details of the rendering pipeline depend on the environment, and the present invention does not ask the environment of the GPU 3b as the image processing unit. What is important in the present invention is that the user can insert a process created by the user before the vertex data is rasterized and displayed on the screen in the image processing unit. It should be noted that the present invention does not question the environment of the image processing unit as long as this condition is satisfied.

  In the above-described embodiment, the vertex shader is exemplified as the “program (shader) operating in the image processing unit”, and the pixel shader is also applicable. However, the present invention is not limited to this. However, there is no essential meaning to distinguish the two. For example, the GPU of the integrated shader architecture does not distinguish between these, and there is an environment in which a geometry shader other than the vertex shader and the pixel shader can be used. The present invention can widely use various shaders (programs) that operate in the environment of the image processing unit included in the client 3.

DESCRIPTION OF SYMBOLS 1 Server client type system 2 Server 2a Data request receiving part 2b Encoding process part 2c Decoding program production | generation part 2d Data transmission part 2e Model data storage part 3 Client 3a CPU
3b GPU
3c Data receiving unit 3d Storage device 3e GPU memory 3f Display device

Claims (6)

In the model data distribution server,
An encoding processing unit that generates encoded model data by encoding model data representing the shape of an object;
Decoding program generation that includes a series of instructions for instructing a processing procedure for decoding the encoding model data, and that generates a decoding program executed on the client side by an image processing unit provided separately from the central processing unit And
A model data distribution server comprising: a data transmission unit configured to transmit the encoding model data and the decoding program to a client.   The model data distribution server according to claim 1, wherein the decoding program is a shader program executed by the image processing unit. The encoding processing unit encodes the model data by a different process for each delivery of the model data,
The model data distribution server according to claim 1, wherein the decode program generation unit generates the decode program having different processing contents for each distribution target. In the model data receiving client,
A data receiving unit that receives from the server, an encoded model data that encodes model data representing the shape of an object, and a decoding program configured by a series of instructions that instruct a processing procedure for decoding the encoded model data;
An image processing unit that is provided separately from the central processing unit and that generates the image by decoding the encoded model data by executing the decoding program when the image is displayed. Model data receiving client.   The model data receiving client according to claim 4, wherein the decoding program is a shader program executed by the image processing unit.   6. The model data receiving client according to claim 4, wherein the decoding program has different processing contents for each distribution target of the model data distributed from the server.

Images