JP2021087129A - Electronic signature system, electronic signature method, electronic signature program, information processing device, and mobile information device - Google Patents

Abstract

To prevent the leakage of a private key.SOLUTION: In an electronic signature system, a first information processing device generates a message digest (MD) on the basis of target data to be signed (M05), displays an MD image indicating the MD (M07), and a second information processing device acquires the MD image captured by capturing the display surface of the first information processing device (K02), and displays a signature image indicating signature data (K04) obtained by encrypting the MD with a private key (K05). The first information processing device further acquires a signature image by capturing the display surface of the second information processing device (M08), and associates the data indicated in the signature image with target data as the signature data (M10) when data obtained by decrypting the data indicated in the signature image with a public key corresponding to the private key matches the MD (M09).SELECTED DRAWING: Figure 4

Description

The present invention relates to an electronic signature system, an electronic signature method, an electronic signature program, an information processing device, and a portable information device, and in particular, an electronic signature system for electronically signing using a private key, and an electronic signature method executed by the electronic signature system. , The electronic signature program executed by the computer that controls the information processing device included in the electronic signature system, the electronic signature program executed by the computer that controls the mobile information device included in the electronic signature system, the information processing device and the mobile information. Regarding the device.

When sending and receiving data, an electronic signature is used to guarantee the authenticity of the data. Specifically, as for the electronic signature, the electronic signature in which the hash value of the data is encrypted with the private key is transmitted together with the data. On the receiving side, the identity between the value obtained by decrypting the electronic signature received together with the data with the public key and the hash value of the received data is determined.

For example, Patent No. 3616601 is an image forming apparatus capable of digitally signing a document that can be connected to a data processing apparatus via a network, and the image forming apparatus scans at least (1) the document and scans data. It has scanning means for generating data, (2) encryption means, (3) a connector to which an external secret key storage means connected to the encryption means is connected, and (4) means for connecting to a network. And (a) a step of transmitting the scan data to the data processing device, (b) a step of receiving the message digest generated from the transmitted scan data in the data processing device, (c) the encryption. In the encryption means, the step of encrypting and signing the message digest with the private key from the secret key storage means and (d) the step of transmitting the signed message digest to the data processing device can be sequentially executed. Described is an image forming apparatus capable of digitally signing a document, which is characterized in that the above is performed.

However, the image forming apparatus may be arranged in an environment that can be used by a plurality of people, and if such an image forming apparatus is made to perform the step of encrypting using the private key, the private key is leaked to another person. Or, there is a problem that the private key is misused.

Patent No. 3616601

The present invention has been made to solve the above-mentioned problems, and one of the objects of the present invention is to provide an electronic signature system that prevents leakage of a private key used for electronic signature.

Another object of the present invention is to provide an electronic signature method that prevents leakage of a private key used for electronic signature.

Yet another object of the present invention is to provide an electronic signature program that prevents the leakage of the private key used for electronic signature.

Still another object of the present invention is to provide an information processing device that prevents leakage of a private key used for an electronic signature.

Yet another object of the present invention is to provide a portable information device that prevents leakage of a private key when digitally signing.

In order to achieve the above-mentioned object, according to an aspect of the present invention, the electronic signature system is an electronic signature system including a first information processing device and a second information processing device, and the first information processing device is a first information processing device. A first display means, a first imaging means for capturing a subject, a generation means for generating a message digest based on target data to be signed, and a message digest for displaying an MD image showing the message digest on the first display means. The second information processing apparatus includes a notification means, a second display means, a second image pickup means for capturing a subject, and an MD image pickup in which the second image pickup means captures and outputs a display surface of the first display means. The MD captured image acquisition means for acquiring the image and the signature image showing the signature data in which the message digest is encrypted with the private key are displayed on the second display means according to the reproduction of the message digest based on the MD captured image. The first information processing apparatus further comprises a signature data notification means for acquiring a signature image acquisition means for acquiring a signature image acquired by the first imaging unit capturing and outputting a display surface of the second display means. When the data shown in the signature captured image is decrypted with the public key corresponding to the private key and matches the message digest, the data shown in the signature captured image is associated with the target data as signature data. ..

According to this aspect, the MD image displayed by the first information processing device is captured by the second information processing device, so that the message digest generated based on the target data is the second information from the first information processing device. It is transmitted to the processing device. Further, by capturing the signature image displayed by the second information processing device by the first information processing device, the signature data in which the message digest is encrypted with the private key is transferred from the second information processing device to the first information processing device. Be transmitted. Therefore, the second information processing apparatus can digitally sign the target data without outputting the private key to the outside. As a result, it is possible to provide an electronic signature system that prevents leakage of the private key used for electronic signature.

Preferably, the message digest notification means generates an image showing the encrypted data in which the message digest is encrypted with the public key as an MD image, and the signature data notification means can decrypt the data shown by the MD captured image with the private key. In this case, the data indicated by the MD captured image is decrypted with the private key, and the data is reproduced as a message digest.

According to this aspect, when the data indicated by the MD captured image can be decoded by the private key in the second information processing apparatus, the data indicated by the signed captured image is reproduced as the message digest by the secret key. The same data can be processed between the first information processing device and the second information processing device.

Preferably, the second information processing apparatus further includes a certificate notification means for displaying a setting image indicating the electronic certificate on the second display means, and the first information processing apparatus further includes a first imaging means for the second. A setting image acquisition means for acquiring a setting image to acquire an image captured and an electronic certificate reproduction means for reproducing an electronic certificate from the setting image are provided, and the related means is a signature. In addition to the data, associate the digital certificate with the target data.

According to this aspect, a setting image showing the electronic certificate is displayed on the second information processing device, and the electronic certificate is associated with the target data in addition to the signature data on the first information processing device. It is possible to prove that the user of the target data is the owner of the target data.

Preferably, the set image further indicates destination data indicating the destination of the target data, and the first information processing apparatus is further associated with the destination data reproduction means for reproducing the destination data from the set captured image by the associative means. A transmission means for transmitting the obtained target data and signature data to a destination specified by the destination data is provided.

According to this aspect, the second information processing apparatus displays a setting image showing the destination data indicating the destination of the target data, and the first information processing apparatus sets the target data and the signature data as the destination specified by the destination data. Will be sent. Therefore, if the user specifies the destination in the second information processing device, the user can have the first information processing device transmit the target data to the destination. Therefore, the operation becomes easy.

Preferably, the second information processing device further includes a destination notification means for displaying a setting image indicating the destination data indicating the destination of the target data on the second display means, and the first information processing device further includes a first information processing device. The imaging means is associated with the setting captured image acquisition means for acquiring the set captured image and the destination data reproducing means for reproducing the destination data from the set captured image by the associated means. It is provided with a transmission means for transmitting the target data to the destination specified by the destination data.

According to this aspect, the second information processing apparatus displays a setting image showing the destination data indicating the destination of the target data, and the first information processing apparatus transmits the target data to the destination specified by the destination data. Therefore, if the user specifies the destination in the second information processing device, the user can have the first information processing device transmit the target data to the destination. Therefore, the operation becomes easy.

Preferably, the MD image or signature image is an image of one or more two-dimensional barcodes.

According to this aspect, the encrypted data or the signature data is represented by a two-dimensional bar code, so that the information can be accurately transmitted.

Preferably, at least a part of the display surface of the second display means is included in the imaging range of the first imaging means, and at least a part of the display surface of the first display means is included in the imaging range of the second imaging means. Further, a holding means for determining a relative position between the first information processing device and the second information processing device is further provided.

According to this aspect, the second imaging means can be made to image the MD image, and the first imaging means can be made to take the signature image. Therefore, the user does not need to adjust the imaging range of the first imaging means or the second imaging means, which facilitates the operation.

According to another aspect of the present invention, the electronic signature method is an electronic signature method executed by an electronic signature system including a first information processing device and a second information processing device, and the first information processing device is The second information processing apparatus includes a first display means and a first imaging means for capturing a subject, and the second information processing apparatus includes a second display means and a second imaging means for imaging the subject, and the first information processing apparatus. Executes a generation step of generating a message digest based on the target data to be signed and a message digest notification step of displaying an MD image showing the message digest on the first display means, and causes the second information processing apparatus to execute the step. In response to the MD captured image acquisition step of acquiring the MD captured image that the second imaging means captures and outputs the display surface of the first display means and the message digest being reproduced based on the MD captured image, the message is displayed. A signature data notification step of displaying a signature image showing signature data in which the digest is encrypted with a private key is displayed on the second display means, and the first information processing apparatus is made to execute, and the first imaging means is the second display means. When the signature image acquisition step of acquiring the signature image to be output by imaging the display surface of the data and the data obtained by decoding the data shown in the signature image with the public key corresponding to the private key match the message digest. The associated step of associating the data shown in the signature captured image with the target data as signature data is executed.

According to this aspect, it is possible to provide an electronic signature method that prevents leakage of the private key used for electronic signature.

Preferably, the message digest notification step includes generating an image showing the encrypted data in which the message digest is encrypted with the public key as an MD image, and the signature data notification step includes the data shown by the MD captured image with the private key. It includes a step of reproducing as a message digest the data obtained by decoding the data indicated by the MD captured image with the private key when the data can be decoded.

Preferably, the second information processing apparatus is further subjected to a certificate notification step of displaying the setting image indicating the electronic certificate on the second display means, and the first information processing apparatus is further subjected to the first imaging means. 2 Setting to capture and output the display surface of the display means Setting to acquire the captured image The step to acquire the captured image and the step to reproduce the digital certificate from the set captured image are executed, and the related step is , In addition to the signature data, includes the step of associating the digital certificate with the target data.

Preferably, the set image further indicates destination data indicating the destination of the target data, and the first information processing apparatus further associates the destination data reproduction step of reproducing the destination data from the set captured image with the association step. The transmission step of transmitting the target data and the signature data to the destination specified by the destination data is executed.

Preferably, the second information processing device is further subjected to a destination notification step of displaying the setting image indicating the destination data indicating the destination of the target data on the second display means, and the first information processing device is further subjected to the second. 1 The imaging means associates the setting captured image acquisition step of acquiring the set captured image and the destination data reproduction step of reproducing the destination data from the set captured image in the associated step. The transmission step of transmitting the target data to the destination specified by the destination data is executed.

Preferably, the MD image or signature image is an image of one or more two-dimensional barcodes.

Preferably, in the first information processing apparatus, at least a part of the display surface of the second display means is included in the imaging range of the first imaging means, and at least a part of the display surface of the first display means is second. A holding means for determining a relative position between the first information processing apparatus and the second information processing apparatus is provided so as to be included in the imaging range of the imaging means.

According to another aspect of the present invention, the electronic signature program is an electronic signature program executed by a computer that controls an information processing device, and the information processing device includes a display means, an imaging means for imaging a subject, and the like. A generation step of generating a message digest based on the target data to be signed, a message digest notification step of displaying an MD image showing the message digest on the display means, and a portable information device that images the display surface of the display means. While displaying the signature image showing the signature data in which the message digest is encrypted with the private key, the imaging means captures the display surface of the mobile information device and outputs the signature captured image. , When the data shown in the signature image is decrypted with the public key corresponding to the private key and matches the message digest, the associated step that associates the data shown in the signature image with the target data as signature data. Let the computer do it.

According to this aspect, when the MD image is displayed on the display means, the message digest generated based on the target data is transmitted to the mobile information device. Further, while the signature image showing the signature data in which the message digest is encrypted with the private key is displayed by the mobile information device, the display surface of the mobile information device is imaged to obtain the signature image, which is shown in the signature image. When the data obtained by decrypting the data with the public key corresponding to the private key matches the message digest, the data shown in the signature captured image is associated with the target data as signature data. Therefore, the target data can be digitally signed using the signature data generated by the mobile information device without causing the mobile information device to output the private key to the outside. As a result, it is possible to provide an electronic signature program that prevents the leakage of the private key used for the electronic signature.

Preferably, the message digest notification step includes generating an image showing the encrypted data in which the message digest is encrypted with the public key as an MD image, and the portable information device can decrypt the data shown by the MD image with the private key. If not, the signature image is displayed.

According to this aspect, the mobile information device displays the signature image when the data indicated by the MD image can be decrypted with the private key, so that the same data can be processed with the mobile information device. ..

Preferably, the imaging means captures and outputs the display surface of the mobile information device that displays the setting image indicating the digital certificate. The setting captured image acquisition step of acquiring the setting captured image is further executed by the computer, and the related step is , Setting Includes a step of associating the digital certificate with the target data in addition to the signature data when the digital certificate is reproduced from the captured image.

According to this aspect, the digital certificate is reproduced from the setting image displayed by the mobile information device, and the digital certificate is associated with the target data in addition to the signature data. Therefore, the user of the mobile information device is the owner of the target data. You can prove that there is.

Preferably, the mobile information device displays a set image further indicating the destination data, and when the destination data is reproduced from the set captured image, the target data and the signature data associated in the association step are specified by the destination data. Causes the computer to perform additional send steps to send to the destination.

According to this aspect, the destination data is reproduced from the setting image displayed by the mobile information device, and the target data and the signature data are transmitted to the destination specified by the destination data. Therefore, if the user specifies the destination on the mobile information device, the target data and the signature data can be transmitted to the destination. Therefore, the operation becomes easy.

Preferably, the imaging means captures and outputs the display surface of the mobile information device that displays the set image indicating the destination data. The set captured image acquisition step of acquiring the set captured image and the destination data are reproduced from the set captured image. In this case, the computer is further executed in the transmission step of transmitting the target data associated in the association step to the destination specified by the destination data.

According to this aspect, the destination data is reproduced from the setting image displayed by the mobile information device, and the target data is transmitted to the destination specified by the destination data. Therefore, if the user specifies the destination on the mobile information device, the target data and the signature data can be transmitted to the destination. Therefore, the operation becomes easy.

Preferably, the MD image or signature image is an image of one or more two-dimensional barcodes.

According to this aspect, the encrypted data or the signature data is represented by a two-dimensional bar code, so that the information can be accurately transmitted.

According to still another aspect of the present invention, the electronic signature program is an electronic signature program executed by a computer that controls a portable information device that stores a private key, and the portable information device displays a display means and a subject. An MD image acquisition step of acquiring an MD image acquired by the image device that captures and outputs a display surface of an information processing device, and a message digest are reproduced based on the MD image. Depending on the situation, the computer is made to perform a signature data notification step of displaying a signature image indicating signature data in which the message digest is encrypted with the private key on the display means.

According to this aspect, it is possible to provide an electronic signature program that prevents leakage of the private key.

According to still another aspect of the present invention, the information processing apparatus comprises a display means, an imaging means for imaging a subject, a generation means for generating a message digest based on target data to be signed, and a message digest. The image pickup means while the message digest notification means for displaying the MD image on the display means and the portable information device for imaging the display surface of the display means are displaying the signature image showing the signature data in which the message digest is encrypted with the private key. The MD image acquisition means for acquiring the MD image that captures and outputs the display surface of the portable information device and the data obtained by decoding the data shown in the MD image with the public key corresponding to the private key match the message digest. In this case, an associative means for associating the data shown in the MD captured image with the target data as signature data is provided.

According to this aspect, it is possible to provide an information processing device that prevents leakage of the private key used for the electronic signature.

According to still another aspect of the present invention, the portable information device acquires a display means, an image pickup means for capturing a subject, and an MD image captured by the image pickup means for capturing and outputting the display surface of the information processing device. Captured image acquisition means, signature data notification means for displaying a signature image indicating signature data in which the message digest is encrypted with a private key on the display means in response to playback of the message digest based on the MD captured image. To be equipped.

According to this aspect, it is possible to provide a portable information device that prevents leakage of a private key when digitally signing.

It is a figure which shows the whole outline of the electronic signature system in one of the Embodiments of this invention. It is a block diagram which shows the outline of the hardware structure of the MFP. It is a block diagram which shows the outline of the structure of the portable information device. It is a figure which shows an example of the flow of the electronic signature processing. It is a figure which shows an example of a transmission setting screen. It is a figure which shows an example of a setting instruction screen. FIG. 1 is a first diagram showing an example of a data format indicated by a two-dimensional bar code. It is a figure which shows an example of the document reading guidance screen. It is a figure which shows an example of the MD notification screen. FIG. 2 is a second diagram showing an example of a data format indicated by a two-dimensional bar code. It is a figure which shows an example of the signature notification screen. FIG. 3 is a third diagram showing an example of a data format indicated by a two-dimensional bar code. It is a block diagram which shows an example of the function of the CPU provided in the MFP in this embodiment. It is a block diagram which shows an example of the function which a CPU included in a mobile information device has. It is a flowchart which shows an example of the flow of the signature processing on a device side. It is a flowchart which shows an example of the flow of transmission information acquisition processing. It is a flowchart which shows an example of the flow of the signature processing on the mobile phone side. It is a figure which shows the state before holding the portable information device in a holding member. It is a figure which shows the state which held the portable information device in a holding member.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are designated by the same reference numerals. Their names and functions are the same. Therefore, detailed explanations about them will not be repeated.

FIG. 1 is a diagram showing an overall outline of an electronic signature system according to one of the embodiments of the present invention. With reference to FIG. 1, the electronic signature system 1 includes an MFP (Multi Function Peripheral) 100 and a portable information device 200. The MFP 100 is an example of an information processing device. The MFP 100 includes a document reading unit 130 for reading a document, an automatic document transfer device 120 for transporting a document to the document reading unit 130, and an image forming unit 140 for forming an image on paper or the like based on image data. A paper feed unit 150 for supplying paper to the image forming unit 140, and an operation panel 160 as a user interface are included. The operation panel 160 includes a camera 119 that captures a subject and a holding member 300 that holds the portable information device 200.

The automatic document transfer device 120 automatically transports a plurality of sheets of paper set on the document tray 125 one by one to the document reading position of the document reading unit 130, and the image formed on the document by the document reading unit 130 is displayed. The scanned document is ejected onto the document ejection tray 127.

The document reading unit 130 includes a light source that irradiates light and a photoelectric conversion element that receives light, and scans an image formed on the document placed on the reading surface. When a document is placed in the reading area, the light emitted from the light source is reflected by the document, and the reflected light is imaged by the photoelectric conversion element. When the photoelectric conversion element receives the light reflected by the document, it generates image data in which the received light is converted into an electric signal.

The paper feed unit 150 conveys the paper stored in the paper feed tray to the image forming unit 140. The image forming unit 140 forms an image by a well-known electrophotographic method, forms an image on the paper conveyed by the paper feeding unit 150 based on the image data, and ejects the paper on which the image is formed. Discharge to tray 155.

The mobile information device 200 is a portable computer carried and used by a user, such as a smartphone or a PDA (Personal Digital Assistant). In the present embodiment, the mobile information device 200 is a smartphone.

The holding member 300 is fixed to the operation panel 160 and can hold the portable information device 200.

FIG. 2 is a block diagram showing an outline of the hardware configuration of the MFP. With reference to FIG. 2, the MFP 100 includes a main circuit 110. The main circuit 110 includes a CPU (Central Processing Unit) 111 that controls the entire MFP 100, a communication interface (I / F) unit 112, a ROM (Read Only Memory) 113, a RAM (Random Access Memory) 114, and the like. It includes a hard disk drive (HDD) 115 as a large-capacity storage device, a facsimile unit 116, and an external storage device 117. The CPU 111 is connected to the camera 119, the automatic document transfer device 120, the document reading unit 130, the image forming unit 140, the paper feeding unit 150, and the operation panel 160, and controls the entire MFP 100.

The ROM 113 stores a program executed by the CPU 111 or data necessary for executing the program. The RAM 114 is used as a work area when the CPU 111 executes a program. Further, the RAM 114 temporarily stores image data continuously sent from the document reading unit 130.

The operation panel 160 is provided on the upper surface of the MFP 100. The operation panel 160 includes a display unit 161 and an operation unit 163. The display unit 161 is, for example, a liquid crystal display (LCD), and displays an instruction menu for the user, information on acquired image data, and the like. If the device displays an image instead of the LCD, for example, an organic EL (electroluminescence) display can be used.

The operation unit 163 includes a touch panel 165 and a hard key unit 167. The touch panel 165 is a capacitance type. The touch panel 165 is not limited to the capacitance method, and other methods such as a resistance film method, a surface acoustic wave method, an infrared method, and an electromagnetic induction method can be used.

The touch panel 165 is provided with its detection surface superimposed on the display unit 161 on the upper surface or the lower surface of the display unit 161. Here, the size of the detection surface of the touch panel 165 and the size of the display surface of the display unit 161 are the same. Therefore, the coordinate system of the display surface and the coordinate system of the detection surface are the same. The touch panel 165 detects a position on the detection surface that indicates the display surface of the display unit 161 by the user, and outputs the coordinates of the detected position to the CPU 111. Since the coordinate system of the display surface and the coordinate system of the detection surface are the same, the coordinates output by the touch panel 165 can be replaced with the coordinates of the display surface.

The hard key unit 167 includes a plurality of hard keys. The hard key is, for example, a contact switch. The touch panel 165 detects a position designated by the user on the display surface of the display unit 161.

The camera 119 is arranged on the upper surface of the operation panel 160 below the display surface of the display unit 161. The camera 119 includes a lens and a photoelectric conversion element, and images the light collected by the lens on the photoelectric conversion element, and the photoelectric conversion element photoelectrically converts the received light and outputs image data to the CPU 111. The photoelectric conversion element is a CMOS (Complementary Metal Oxide Sensor) sensor, a CCD (Charge Coupled Device) sensor, or the like.

The communication I / F unit 112 is an interface for connecting the MFP 100 to the network. The communication I / F unit 112 communicates with another computer or data processing device connected to the network by a communication protocol such as TCP (Transmission Control Protocol) or FTP (File Transfer Protocol). The network to which the communication I / F unit 112 is connected is a local area network (LAN), and the connection form may be wired or wireless. The network is not limited to a LAN, and may be a wide area network (WAN), a public switched telephone network (PSTN), the Internet, or the like.

The facsimile unit 116 is connected to the public switched telephone network (PSTN) and transmits facsimile data to or receives facsimile data from the PSTN. The facsimile unit 116 stores the received facsimile data in the HDD 115, converts it into print data that can be printed by the image forming unit 140, and outputs it to the image forming unit 140. As a result, the image forming unit 140 forms an image of the facsimile data received by the facsimile unit 116 on the paper. Further, the facsimile unit 116 converts the data stored in the HDD 115 into facsimile data and transmits the data to the facsimile apparatus connected to the PSTN.

The external storage device 117 is controlled by the CPU 111, and is equipped with a semiconductor memory such as a CD-ROM (Compact Disk Read Only Memory) 118 or a USB (Universal Serial Bus) memory. In the present embodiment, an example in which the CPU 111 executes a program stored in the ROM 113 will be described. However, the CPU 111 controls an external storage device 117 to read a program for the CPU 111 to execute from the CD-ROM 118. , The read program may be stored in the RAM 114 and executed.

The recording medium for storing the program to be executed by the CPU 111 is not limited to the CD-ROM118, but is a flexible disc, a cassette tape, an optical disc (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versailles Disc). )), An IC card, an optical card, a mask ROM, a medium such as a semiconductor memory such as an EPROM (Erasable Program ROM) may be used. Further, the CPU 111 downloads the program from the computer connected to the network and stores it in the HDD 115, or the computer connected to the network writes the program to the HDD 115 and loads the program stored in the HDD 115 into the RAM 114. Then, it may be executed by the CPU 111. The program referred to here includes not only a program that can be directly executed by the CPU 111, but also a source program, a compressed program, an encrypted program, and the like.

FIG. 3 is a block diagram showing an outline of the configuration of the portable information device. With reference to FIG. 3, the mobile information device 200 according to the present embodiment non-volatilely displays data in a CPU 201 for controlling the entire mobile information device 200, a ROM 202, a RAM 203 used as a work area of the CPU 201, and data. A flash memory 204 for storing, a wireless communication unit 206 connected to the call unit 205, a camera 207, a display unit 208 for displaying information, an operation unit 209 for receiving user operations, and an external storage device 210. Including.

The display unit 208 is a display device such as a liquid crystal display (LCD) or an organic ELD, and displays an image. The operation unit 209 includes a main key 209A and a touch panel 209B. The touch panel 209B is a capacitance type. The touch panel 209B is not limited to the capacitance method, and other methods such as a resistance film method, a surface acoustic wave method, an infrared method, and an electromagnetic induction method can be used.

The touch panel 209B is provided with its detection surface superimposed on the display unit 208 on the upper surface or the lower surface of the display unit 208. Here, the size of the detection surface of the touch panel 209B and the size of the display surface of the display unit 208 are the same. Therefore, the coordinate system of the display surface and the coordinate system of the detection surface are the same. The touch panel 209B detects a position on the detection surface that indicates the display surface of the display unit 208 by the user, and outputs the coordinates of the detected position to the CPU 201. Since the coordinate system of the display surface and the coordinate system of the detection surface are the same, the coordinates output by the touch panel 209B can be replaced with the coordinates of the display surface. Hereinafter, the coordinates detected and output by the touch panel 209B on the detection surface when the user instructs the display surface of the display unit 208 are also referred to as the coordinates of the display surface.

The camera 207 includes a lens and a photoelectric conversion element, and images the light collected by the lens on the photoelectric conversion element, and the photoelectric conversion element photoelectrically converts the received light and outputs image data to the CPU 201. The photoelectric conversion element is a CMOS sensor, a CCD sensor, or the like.

The camera 207 is arranged on the same surface as the surface on which the display surface of the display unit 208 of the mobile information device 200 is arranged. Therefore, the mobile information device 200 can have the camera 207 image the user's face while the user is viewing the image displayed on the display surface of the display unit 208.

The wireless communication unit 206 wirelessly communicates with a mobile phone base station connected to the telephone communication network. The wireless communication unit 206 connects the mobile information device 200 to the telephone communication network and enables a telephone call using the communication unit 205. The wireless communication unit 206 decodes the voice signal obtained by demodulating the wireless signal received from the mobile phone base station and outputs it to the communication unit 205. Further, the wireless communication unit 206 encodes the voice input from the communication unit 205 and transmits it to the mobile phone base station. The call unit 205 includes a microphone and a speaker, outputs the sound input from the wireless communication unit 206 from the speaker, and outputs the sound input from the microphone to the wireless communication unit 206. Further, the wireless communication unit 206 is controlled by the CPU 201 to connect the mobile information device 200 to the e-mail server to send and receive e-mail.

The flash memory 204 stores a program executed by the CPU 201 or data necessary for executing the program. The CPU 201 loads the program recorded in the flash memory 204 into the RAM included in the CPU 201 and executes the program.

The external storage device 210 is detachable from the mobile information device 200, and a CD-ROM 211 storing the signature program on the mobile phone side can be attached. The CPU 201 can access the CD-ROM 211 via the external storage device 210. The CPU 201 can load the portable signature program recorded on the CD-ROM 211 mounted on the external storage device 210 into the RAM included in the CPU 201 and execute the program.

Although the program recorded in the flash memory 204 or the CD-ROM 211 has been described as the program executed by the CPU 201, another computer connected to the network rewrites the program stored in the flash memory 204, or a program. , It may be a new program additionally written. Further, the mobile information device 200 may be a program downloaded from another computer connected to the network. The program referred to here includes not only a program that can be directly executed by the CPU 201, but also a source program, a compressed program, an encrypted program, and the like.

The medium for storing the program executed by the CPU 201 is not limited to the CD-ROM211 but may be a semiconductor memory such as an optical disk (MO / MD / DVD), an IC card, an optical card, a mask ROM, or an EPROM.

The electronic signature system 1 electronically signs the data stored in the MFP 100 using the private key stored in the mobile information device 200. The private key assigned to the user of the mobile information device 200 is stored in the flash memory 204 in advance. Further, the public key corresponding to the private key is stored in the flash memory 204. The mobile information device 200 may generate the public key from the private key instead of storing the public key in the flash memory 204. Here, a case where the MFP 100 electronically signs and transmits an e-mail to which the data obtained by scanning the original is attached will be described as an example.

Further, the mobile information device 200 has a mobile-side signature program installed, and the MFP 100 has a device-side signature program installed. The mobile-side signature program and the device-side signature program are part of the electronic signature program.

FIG. 4 is a diagram showing an example of the flow of processing the electronic signature. In the left half of FIG. 4, the process executed by the MFP 100 is indicated by adding a code to the device side process column, and the image displayed on the display unit 161 while the process is executed is displayed on the device side. Shown in the column. Further, in the right half of FIG. 4, the process executed by the mobile information device 200 is indicated by adding a code to the column of the mobile information device 200, and the image displayed on the display unit 208 while the process is executed. Is shown in the display column on the mobile phone side. With reference to FIG. 4, when the mobile phone side signature program is executed in the mobile information device 200, an image of the transmission setting screen is displayed on the display unit 208. Then, the mobile information device 200 accepts the setting for transmitting the e-mail (K01).

FIG. 5 is a diagram showing an example of a transmission setting screen. With reference to FIG. 5, the transmission setting screen 500 includes an area 501 in which the e-mail address of the transmission source of the e-mail can be input and an area 502 in which the e-mail address of the destination can be input. The selected e-mail address is set in each of the area 501 or the area 502 according to the user selecting the e-mail address registered in the address book stored in the flash memory 204 in advance. Further, it may be set by the user directly inputting an e-mail address in the area 501 or the area 502 according to the transmission setting screen. An e-mail address preset in the mobile information device 200 may be automatically set in the area 501 as the e-mail address of the transmission source. The image of the setting instruction screen 510 is displayed on the display unit 208 by the mobile information device 200 in response to the user inputting the e-mail address to the area 501 or the area 502 according to the transmission setting screen.

FIG. 6 is a diagram showing an example of a setting instruction screen. With reference to FIG. 6, the setting instruction screen 510 includes the setting images 511A and 511B. The setting images 511A and 511B are two-dimensional bar codes indicating the sender's e-mail address, the destination's e-mail address, and the digital certificate. The digital certificate is information issued by a certification body that is a third-party organization, and is information for certifying the user. This digital certificate contains the public key. The digital certificate is stored in advance in the flash memory 204 of the mobile information device 200. The upper limit of the data indicated by the two-dimensional bar code is predetermined. Therefore, when the amount of data indicated by the two-dimensional bar code exceeds the upper limit value, the data is indicated by a plurality of two-dimensional bar codes. The format of the data indicated by the two-dimensional bar code is predetermined. As the format, for example, a format is used in which each type of data is separated by a tag indicating each.

FIG. 7 is a first diagram showing an example of a data format indicated by a two-dimensional bar code. FIG. 7A shows an example of the data format shown by the setting image 511A, and FIG. 7B shows an example of the data format shown by the setting image 511B. The data shown by the setting image 511A is separated by two destination tags, two sender tags, and two certificate 1 tags. The destination email address is set between the two destination tags, the sender email address is set between the two sender tags, and the digital certificate is set between the two certificate 1 tags. The first half of is set. The data shown in the setting image 511B is separated by two tags of the certificate 2. The latter half of the digital certificate is set between the tags of the two certificates 2.

Returning to FIG. 6, the setting instruction screen 510 shows a case where the setting images 511A and 511B are included as the two-dimensional bar code. However, the setting instruction screen 510 may include at least a two-dimensional bar code indicating a source e-mail address and a destination e-mail address. In this case, the setting instruction screen 510 may include one setting image 511A as a two-dimensional bar code. Further, when the setting instruction screen 510 includes a two-dimensional bar code indicating the digital certificate, the digital certificate does not have to include the public key. In this case, the setting instruction screen 510 may or may not include a two-dimensional bar code indicating the public key.

Returning to FIG. 4, the user causes the mobile information device 200 to be held by the holding member 300 of the MFP 100. Specifically, the user causes the holding member 300 to hold the mobile information device 200 so that the display surface of the display unit 208 of the mobile information device 200 faces the operation panel 160. In the state where the holding member 300 holds the mobile information device 200, at least a part of the display surface of the display unit 208 of the mobile information device 200 fits within the imaging range of the camera 119, and at least the display surface of the display unit 161 of the MFP 100. The relative position between the mobile information device 200 and the operation panel 160 is determined so that a part of the mobile information device 200 is within the imaging range of the camera 207 of the mobile information device 200.

When the mobile information device 200 is held by the holding member 300, the setting instruction screen 510 is displayed on the display unit 208 of the mobile information device 200. The MFP 100 acquires a set captured image by causing the camera 119 to image the display surface of the display unit 208 of the mobile information device 200 (M01). Since the set images 511A and 511B are displayed on the display unit 208 of the mobile information device 200, the set captured image includes the set images 511A and 511B. The user may have the MFP 100 image the setting images 511A and 511B displayed on the display unit 208 of the mobile information device 200 with the camera 119 without holding the mobile information device 200 on the holding member 300.

Then, the MFP 100 reproduces the set images 511A and 511B and acquires the transmission information (M02). Specifically, by analyzing the set images 511A and 511B, the destination e-mail address, the source e-mail address, and the digital certificate are reproduced as transmission information. Then, the MFP 100 displays the document reading guide screen 520 on the display unit 161.

FIG. 8 is a diagram showing an example of a document reading guide screen. The document reading guide screen 520 is a screen for instructing the scanning of a document. With reference to FIG. 8, the document reading guidance screen 520 includes a message 521 instructing to read the document, a destination e-mail address 522, and an e-mail address 523 of the source of the e-mail. The destination e-mail address 522 and the source e-mail address 523 are obtained by playing back the set images 511A and 511B.

Returning to FIG. 4, when the start button of the operation unit 163 is pressed by the user, the MFP 100 starts scanning and acquires image data obtained by reading the original (M03). Then, the MFP 100 generates an e-mail with the image data attached (M04). Further, the MFP 100 generates a message digest (MD) based on the e-mail to which the image data is attached (M05). The message digest is a hash value generated from an e-mail with image data attached. Then, the MFP 100 generates encrypted data in which the message digest is encrypted with the public key included in the digital certificate (M06). Then, the MFP 100 generates an MD image 532 showing the encrypted data (M07). The MD image 532 is a two-dimensional bar code indicating encrypted data. Then, the MFP 100 displays the image of the MD notification screen 530 on the display unit 161.

FIG. 9 is a diagram showing an example of the MD notification screen. The MD notification screen 530 is a screen for notifying the mobile information device 200 of the message digest. With reference to FIG. 9, the MD notification screen 530 includes an MD image 532, a destination e-mail address 533, and a source e-mail address 534.

FIG. 10 is a second diagram showing an example of a data format indicated by a two-dimensional bar code. FIG. 10 shows an example of the data format shown by MD image 532. The data represented by the MD image 532 is separated by two destination tags, two sender tags, and two message digest tags. The destination email address 533 is set between the two destination tags, the source email address 534 is set between the two sender tags, and the message digest is between the two message digest tags. The encrypted data is set.

Returning to FIG. 4, the mobile information device 200 acquires an MD image by causing the camera 207 to image the display surface of the display unit 161 of the MFP 100 (K02). Since the MD notification screen 530 is displayed on the display unit 161 of the MFP 100, the MD captured image includes the MD image 532. When the mobile information device 200 is not held by the holding member 300, the user may have the mobile information device 200 image the MD image 532 displayed on the display unit 161. The mobile information device 200 analyzes the MD image 532, acquires the encrypted data, and acquires the message digest by decrypting the encrypted data with the private key (K03). Then, the mobile information device 200 generates signature data by encrypting the message digest with the private key (K04). Then, the mobile information device generates a signature image 544 showing the signature data (K05). The signature image 544 is a two-dimensional bar code indicating signature data. Then, the mobile information device 200 displays the image of the signature notification screen 540 on the display unit 208 on the display unit 161.

FIG. 11 is a diagram showing an example of the signature notification screen. The signature notification screen 540 is a screen for notifying the MFP 100 of the signature data. With reference to FIG. 10, the signature notification screen 540 includes a message 541, a signature image 544, an e-mail address 542 to which an e-mail is sent, and an e-mail address 543 from which the e-mail is sent. The message 541 is a message prompting an operation for causing the MFP 100 to read the signature image 544.

FIG. 12 is a third diagram showing an example of a data format indicated by a two-dimensional bar code. FIG. 12 shows an example of the data format shown by the signature image 544. The data represented by the signature image 544 is separated by two destination tags, two sender tags, and two signature tags. The destination email address 542 is set between the two destination tags, the source email address 543 is set between the two sender tags, and the signature data is between the two signature tags. Set.

Returning to FIG. 4, the MFP 100 acquires a signature image by causing the camera 119 to image the display surface of the display unit 208 of the mobile information device 200 (M08). When the mobile information device 200 is not held by the holding member 300, the user may have the MFP 100 image the signature image 544 displayed on the display unit 208. Since the signature notification screen 540 is displayed on the display unit 208 of the mobile information device 200, the signature captured image includes the signature image 544. The MFP 100 analyzes the signature image 544, acquires the signature data, and acquires the message digest by decrypting the signature data with the public key. Then, the MFP 100 verifies the signature data (M09). Specifically, the MFP 100 collates the message digest acquired from the signature data with the message digest generated from the e-mail to which the image data is attached. If the two match, the MFP 100 determines that the signature data is genuine, generates an e-mail, and transmits the e-mail (M10). Specifically, the signature data and the digital certificate are further attached to the e-mail, the e-mail to which the signature data is attached is transmitted, and the mail transmission completion notification screen is displayed on the display unit 161. The e-mail transmission completion notification screen includes a message indicating that the e-mail has been sent, an e-mail address from which the e-mail is sent, and an e-mail address to which the e-mail is sent.

The user who receives the e-mail compares the hash value of the part of the e-mail excluding the e-mail certificate with the data obtained by decrypting the signature data set in the e-mail with the public key included in the e-mail attached to the e-mail. To do. The user who receives the e-mail can confirm that the part of the e-mail excluding the digital certificate is generated by the sender user by confirming that the two match. The user who receives the e-mail confirms that the two do not match, so that the part of the e-mail other than the digital certificate is not generated by the user who sent the e-mail, or has been tampered with. It can be confirmed that it is a thing.

In the electronic signature system 1 of the present embodiment, in order to notify the mobile information device 200 of the message digest from the MFP 100, the MFP 100 displays an MD image showing the encrypted data encrypted with the public key, and the mobile information device 200 displays the MD image. Image it. In this way, since the message digest is optically notified between the MFP 100 and the mobile information device 200, it is possible to prevent the message digest from leaking to another computer. Similarly, in order to notify the signature data from the mobile information device 200 to the MFP 100, the mobile information device displays a signature image showing the signature data, and the MFP 100 images the signature image. In this way, since the signature data is optically notified between the MFP 100 and the mobile information device 200, it is possible to prevent the signature data from being leaked to another computer.

Further, since the signature data is generated based on the message digest, the MFP 100 cannot generate the signature data unless the MD image is acquired, and cannot display the signature image indicating the signature data. Further, the MFP 100 cannot confirm that the signature data is genuine unless the signature image is acquired. Therefore, since the order of a series of processes for adding the signature data to the target data is determined between the MFP 100 and the mobile information device 200, the series of processes is executed synchronously between the MFP 100 and the mobile information device 200. can do.

FIG. 13 is a block diagram showing an example of a CPU function included in the MFP in the present embodiment. The function shown in FIG. 13 is a function formed in the CPU 111 by the CPU 111 included in the MFP 100 executing the device-side signature program stored in the ROM 113, the HDD 115, and the CD-ROM 118. The device-side signature program is a part of the electronic signature program.

With reference to FIG. 13, the CPU 111 includes a first imaging control unit 51, a document reading control unit 53, a transmission information acquisition unit 55, a certificate acquisition unit 57, an e-mail generation unit 59, and an MD generation unit 61. A public key encryption unit 63, an MD notification unit 65, an association unit 67, and an e-mail transmission unit 69 are included.

The first imaging control unit 51 controls the camera 119 to cause the camera 119 to image a subject, and acquires an captured image output by the camera 119. The subject of the camera 119 is the display surface of the display unit 208 of the portable information device 200.

The first image pickup control unit 51 includes a set image pickup image acquisition unit 71 and a signature image pickup image acquisition unit 73. The setting captured image acquisition unit 71 causes the camera 119 to image the display surface of the display unit 208 at the stage when the setting instruction screen 510 is displayed on the display unit 208 of the mobile information device 200, and obtains the set captured image output by the camera 119. get. The set captured image acquisition unit 71 outputs the set captured image to the transmission information acquisition unit 55 and the certificate acquisition unit 57. Since the setting captured image is an image obtained by capturing the setting instruction screen 510, the setting images 511A and 511B are included.

The transmission information acquisition unit 55 analyzes the setting image 511A and acquires the transmission source e-mail address and the transmission destination e-mail address as transmission information. The transmission information acquisition unit 55 outputs the transmission information to the e-mail generation unit 59.

The certificate acquisition unit 57 analyzes the setting image 511A and the setting image 511B to acquire an electronic certificate. The certificate acquisition unit 57 synthesizes the information between the tags of the two certificates 1 shown in the setting image 511A and the information between the tags of the two certificates 2 shown in the setting image 511B, thereby electronically combining the information. Get a certificate. The certificate acquisition unit 57 outputs the digital certificate to the e-mail generation unit 59. The certificate acquisition unit 57 outputs the public key included in the digital certificate to the public key encryption unit 63 and the related unit 67.

The document scanning control unit 53 controls the automatic document transporting device 120 and the document scanning unit 130 in response to the user pressing the start button of the operation unit 163 to cause the automatic document transporting device 120 to transport the document. , The document reading unit 130 is made to read the document. The document reading control unit 53 acquires the image data output by the document reading unit 130 and outputs the image data to the e-mail generation unit 59.

The e-mail generation unit 59 generates an e-mail according to the transmission information input from the transmission information acquisition unit 55. Specifically, an e-mail in which the sender's e-mail address and the destination's e-mail address are set is generated. The e-mail generation unit 59 generates an e-mail with the image data input from the document reading control unit 53 and the electronic certificate input from the certificate acquisition unit 57 attached, and the generated e-mail is generated by the MD generation unit. Output to 61 and the related part 67.

The MD generation unit 61 generates a hash value of data obtained by removing the digital certificate from the e-mail as a message digest. The MD generation unit 61 outputs the message digest to the public key encryption unit 63 and the association unit 67.

The public key encryption unit 63 encrypts the message digest with the public key input from the certificate acquisition unit 57, and generates encrypted data. The public key encryption unit 63 outputs the encrypted data to the MD notification unit 65.

The MD notification unit 65 includes an MD image generation unit 83 and an MD image display unit 81. The MD image generation unit 83 generates a two-dimensional bar code indicating the encryption data input from the public key encryption unit 63 as the MD image 532, and outputs the MD image 532 to the MD image display unit 81. The MD image display unit 81 generates an MD notification screen 530 including the MD image 532, and displays the MD notification screen 530 on the display unit 161. The MD notification unit 65 displays the MD image 532 on the display surface of the display unit 161 at a position overlapping the imaging range of the camera 207 of the mobile information device 200. As a result, the MD image 532 is captured by the camera 207 of the portable information device 200.

The signature captured image acquisition unit 73 acquires a signature captured image to be output by the camera 119 capturing the display surface of the display unit 208 at the stage when the signature notification screen 540 is displayed on the display unit 208 of the mobile information device 200. The signature captured image acquisition unit 73 outputs the signature captured image to the association unit 67. Since the signature captured image is an image obtained by capturing the signature notification screen 540 by the camera 119, the signature image 544 is included.

The association unit 67 includes a signature image acquisition unit 75, a public key decryption unit 77, and a collation unit 79. The signature image acquisition unit 75 acquires the signature image 544 included in the signature image, and outputs the data (signature data) indicated by the signature image 544 to the public key decryption unit 77. The public key decryption unit 77 outputs the data (signature data) input from the signature image acquisition unit 75 and the data decrypted with the public key input from the certificate acquisition unit 57 to the collation unit 79. If the decryption of the data input from the signature image acquisition unit 75 fails, the public key included in the digital certificate and the data input from the signature image acquisition unit 75 do not match. In this case, the data input from the signature image acquisition unit 75 may not be the signature data. Therefore, the collation unit 79 does not output the e-mail to the e-mail transmission unit 69, assuming that the data input from the signature image acquisition unit 75 is not the signature data. For example, the collation unit 79 displays a message indicating that the electronic signature has failed on the display unit 161.

The collation unit 79 collates the message digest input from the MD generation unit 61 with the data input from the public key decryption unit 77. When the two match, the collation unit 79 sets the data input from the signature image acquisition unit 75 as the signature data in the e-mail input from the e-mail generation unit 59, and outputs the e-mail to the e-mail transmission unit 69. To do. The e-mail transmission unit 69 controls the communication I / F unit 112 to transmit an e-mail to a predetermined e-mail server.

FIG. 14 is a block diagram showing an example of the functions of the CPU included in the mobile information device. The function shown in FIG. 14 is a function realized by the CPU 201 when the CPU 201 included in the mobile information device 200 executes a mobile-side signature program stored in the ROM 202, the flash memory 204, or the CD-ROM 211. The mobile signing program is part of the signing program. With reference to FIG. 14, the CPU 201 includes an operation reception unit 251, a setting image generation unit 253, a setting image display unit 255, a second imaging control unit 257, an MD image acquisition unit 259, and a secret key decryption unit 261. And a private key encryption unit 263 and a signature data notification unit 265.

The operation reception unit 251 controls the operation unit 209 and receives an operation input by the user to the operation unit 209. The operation reception unit 251 includes a transmission destination reception unit 271, a transmission source reception unit 273, and a certificate designation reception unit 275. The destination reception unit 271 accepts the e-mail address of the destination of the e-mail. The operation reception unit 251 displays the transmission setting screen 500 on the display unit 208. The destination reception unit 271 accepts the e-mail address input to the area 502 of the transmission setting screen 500 as the destination e-mail address. Further, the destination reception unit 271 is selected in response to the selection of the e-mail address of the destination user from the address book stored in advance in the flash memory 204 in a state where the area 502 is selected by the user. Accept the e-mail address as the destination e-mail address. The destination reception unit 271 outputs the destination e-mail address to the setting image generation unit 253.

The sender reception unit 273 accepts the e-mail address of the sender of the e-mail. The sender reception unit 273 accepts the e-mail address input to the area 501 as the e-mail address of the sender while the transmission setting screen 500 is displayed on the display unit 208. Further, the sender reception unit 273 may accept the e-mail address stored in advance in the flash memory 204 as the e-mail address of the sender as the e-mail address of the sender with the area 501 selected by the user. .. The sender reception unit 273 outputs the sender's e-mail address to the setting image generation unit 253.

The certificate designation reception unit 275 accepts the digital certificate stored in advance in the flash memory 204 as a processing target. Further, when the digital certificate stored in advance in the flash memory 204 is designated by the user, the certificate designation reception unit 275 may accept the digital certificate as a processing target. Further, when a plurality of digital certificates are stored in the flash memory 204, the certificate designation reception unit 275 accepts the digital certificate selected by the user from the plurality of digital certificates as the processing target. The certificate designation reception unit 275 outputs the digital certificate accepted as the processing target to the setting image generation unit 253.

The setting image generation unit 253 generates a two-dimensional bar code indicating the sender's e-mail address, the destination's e-mail address, and the electronic certificate as the setting images 511A and 511B, and sets the setting images 511A and 511B as the setting image display unit. Output to 255.

The setting image display unit 255 generates a setting instruction screen 510 including the setting images 511A and 511B input from the setting image generation unit 253, and displays the setting instruction screen 510 on the display unit 208. The set image display unit 255 displays the set images 511A and 511B at positions on the display surface of the display unit 208 that overlap the imaging range of the camera 119 of the MFP 100. As a result, the set images 511A and 511B are imaged by the camera 119 of the MFP100.

The second imaging control unit 257 controls the camera 207 to cause the camera 207 to image the subject, and acquires the captured image output by the camera 207. The subject of the camera 207 is the display surface of the display unit 161 of the MFP 100. The second image pickup control unit 257 includes an MD image capture image acquisition unit 279. The MD captured image acquisition unit 279 causes the camera 207 to image the display surface of the display unit 161 while the MFP 100 is displaying the MD notification screen 530 on the display unit 161, and acquires the MD captured image output by the camera 207. The MD image acquisition unit 279 outputs the MD image to the MD image acquisition unit 259. Since the MD captured image is an image obtained by capturing the MD notification screen 530, the MD image 532 is included.

The MD image acquisition unit 259 acquires the MD image 532 from the MD image captured image, analyzes the MD image 532, and outputs the encrypted data indicated by the MD image 532 to the secret key decryption unit 261. The private key decryption unit 261 outputs the data obtained by decrypting the encrypted data with the private key to the private key encryption unit 263. The data obtained by decrypting the encrypted data with the private key is a message digest. When the decryption of the encrypted data by the private key decryption unit 261 fails, the encrypted data and the private key are inconsistent. In this case, the private key decryption unit 261 displays a message indicating that the electronic signature has failed on the display unit 208. The private key encryption unit 263 generates signature data by encrypting the message digest input from the private key decryption unit 261 with the private key. The private key encryption unit 263 outputs the signature data to the signature data notification unit 265.

The signature data notification unit 265 includes a signature image generation unit 283 and a signature image display unit 281. The signature image generation unit 283 generates a two-dimensional bar code indicating the signature data as the signature image 544 based on the signature data input from the private key encryption unit 263, and outputs the signature image 544 to the signature image display unit 281. To do. The signature image display unit 281 generates a signature notification screen 540 including the signature image 544, and displays the signature notification screen 540 on the display unit 208. The signature data notification unit 265 displays the signature image 544 on the display surface of the display unit 208 at a position overlapping the imaging range of the camera 119 of the MFP 100. As a result, the signature image 544 is imaged by the camera 119 of the MFP 100.

FIG. 15 is a flowchart showing an example of the flow of signature processing on the device side. The device-side signature process is a process executed by the CPU 111 when the CPU 111 included in the MFP 100 executes the device-side signature program stored in the ROM 113, HDD 115, and CD-ROM 118. With reference to FIG. 15, the CPU 111 included in the MFP 100 displays the mobile phone installation guide on the display unit 161 (step S01). For example, the mobile installation guide includes the message "Please place the smartphone on the holding member 300 with the display surface facing down." As a result, the user can be notified of the work of installing the mobile information device 200 on the holding member in a state where the setting instruction screen 510 is displayed on the display unit 208 of the mobile information device 200.

In the next step S02, the transmission information acquisition process is executed, and the process proceeds to step S03. The details of the transmission information acquisition process will be described later, but it is a process of reproducing the setting images 511A and 511B included in the setting instruction screen 510 to acquire the transmission information.

In step S03, the document reading guide is displayed, and the process proceeds to step S04. Specifically, the document reading guide screen 520 is displayed on the display unit 161. As a result, the user can be notified of the work of causing the MFP 100 to read the manuscript to be transmitted.

In step S04, the document is scanned and the process proceeds to step S05. When the start key of the operation unit 163 is pressed by the user, the document reading unit 130 is made to read the document, and the image data output by the document reading unit 130 is acquired.

In step S05, an email is generated. Based on the transmission information acquired in step S02, an e-mail with the image data acquired in step S04 is generated.

In step S06, MD is generated, and the process proceeds to step S07. The hash value of the e-mail to which the image data is attached is calculated as a message digest (MD). In step S07, the message digest is encrypted with the public key, and the process proceeds to step S08. The public key is included in the digital certificate included in the transmission information acquired in step S02. The public key may be acquired by being input to the operation unit 163 by the user. In step S08, the MD image 532 is generated, and the process proceeds to step S09. The MD image 532 is a two-dimensional bar code indicating encrypted data in which the message digest is encrypted.

In step S09, the MD image 532 is displayed on the display unit 161 and the process proceeds to step S10. Specifically, the MD notification screen 530 is displayed on the display unit 161. The CPU 111 causes the camera 119 to take an image (step S10), and acquires the signature image taken by the camera 119 (step S11). Then, it is determined whether or not the signature captured image includes the two-dimensional bar code (step S12). If the signature captured image contains a two-dimensional bar code, the process proceeds to step S13, otherwise the process returns to step S10.

In step S13, the signature data is acquired, and the process proceeds to step S14. The signature data is acquired by reproducing the two-dimensional bar code included in the signature captured image. At this stage, since the signature notification screen 540 is displayed on the display unit 208 of the mobile information device 200, the signature captured image output by the camera 119 includes the signature image 544. Therefore, the signature data shown in the signature image 544 is acquired.

In step S14, it is determined whether or not the signature data can be decrypted with the public key. If it can be decrypted with the public key, the process proceeds to step S15, otherwise the process proceeds to step S20. In step S20, an error message is displayed on the display unit 161 and the process ends. In step S15, the signature data is decrypted with the public key, and the process proceeds to step S16. In step S16, it is determined whether or not the decrypted data whose signature data is decrypted with the public key is the same as the message digest (MD) generated in step S06. If the decoded data is the same as MD, the process proceeds to step S17, otherwise the process proceeds to step S20. In step S20, an error message is displayed on the display unit 161 and the process ends.

In step S17, the signature data acquired in step S13 is attached to the e-mail, and the process proceeds to step S18. In step S18, the e-mail is transmitted and the process proceeds to step S19. In step S19, the transmission completion guide is displayed on the display unit 161 and the process ends. The transmission completion notice includes, for example, a message indicating that the transmission of the e-mail is completed.

FIG. 16 is a flowchart showing an example of the flow of the transmission information acquisition process. The transmission information acquisition process is a process executed in step S02 of the device-side signature process. The CPU 111 included in the MFP 100 controls the camera 119 to take an image (step S21). Then, the set captured image output by the camera 119 is acquired (step S22). Then, it is determined whether or not the set captured image includes the two-dimensional bar code (step S23). If the set captured image contains a two-dimensional bar code, the process proceeds to step S24, otherwise the process returns to step S21.

In step S24, the two-dimensional bar code included in the set captured image is reproduced. At this stage, since the setting instruction screen 510 is displayed on the display unit 208 of the mobile information device 200, the setting captured image output by the camera 119 includes the setting images 511A and 511B.

Then, the e-mail address of the destination is acquired (step S25), the e-mail address of the sender is acquired (step S26), and the process proceeds to step S27. The e-mail address of the destination and the e-mail address of the sender shown in the setting image 511A are acquired. In step S27, the digital certificate is acquired, and the process returns to the device-side signature process. The digital certificate shown in the setting image 511A and the setting image 511B is acquired.

FIG. 17 is a flowchart showing an example of the flow of the signature processing on the mobile phone side. The mobile phone signature process is a process executed by the CPU 201 when the CPU 201 included in the mobile information device 200 executes a mobile phone signature program stored in the ROM 202, the flash memory 204, or the CD-ROM 211. With reference to FIG. 17, the CPU 201 displays the transmission setting screen 500 on the display unit 208 (step S31), and proceeds to the process in step S32.

In step S32, the destination information is acquired, and the process proceeds to step S33. The e-mail address input to the area 501 of the transmission setting screen 500 is acquired as the destination information. In step S33, the source information is acquired, and the process proceeds to step S34. The e-mail address input to the area 502 of the transmission setting screen 500 is acquired as the sender information.

In step S34, the digital certificate is acquired, and the process proceeds to step S35. When the user specifies the digital certificate stored in the flash memory 204 in advance, the digital certificate is acquired as the processing target.

In step S35, the setting images 511A and 511B are generated, and the process proceeds to step S36. A setting image 511A showing a part of the destination information, a source information, and the digital certificate is generated, and a setting image 511B showing the remaining part of the digital certificate is generated. In step S36, the set images 511A and 511B are displayed on the display unit 208. Specifically, the setting instruction screen 510 is displayed on the display unit 208.

In the next step S37, the CPU 201 controls the camera 207 to image the display surface of the display unit 161 of the MFP 100. Then, the MD captured image output by the camera 207 is acquired (step S38). Then, it is determined whether or not the MD captured image includes the two-dimensional bar code (step S39). If the MD captured image contains a two-dimensional bar code, the process proceeds to step S40, otherwise the process returns to step S37.

In step S40, the encrypted data is acquired, and the process proceeds to step S41. The encrypted data is acquired by reproducing the two-dimensional bar code included in the MD captured image. At this stage, since the MD notification screen 530 is displayed on the display unit 161 of the MFP 100, the MD captured image output by the camera 207 includes the MD image 532. Therefore, the encrypted data shown in the MD image 532 is acquired.

In step S41, it is determined whether or not the encrypted data can be decrypted with the private key. If the private key can be decrypted, the process proceeds to step S42, otherwise the process proceeds to step S47. In step S47, an error message is displayed on the display unit 208, and the process ends. In step S42, the message direct (MD) is acquired, and the process proceeds to step S43. Since the encrypted data is data in which the MD is encrypted with the public key, the MD is acquired by decrypting the encrypted data with the private key.

In step S43, signature data is generated, and the process proceeds to step S44. Signature data is generated by encrypting the MD with the private key. Then, the signature image 544 is generated, and the process proceeds to step S45. A two-dimensional bar code indicating the signature data is generated as the signature image 544.

In step S45, the signature image 544 is displayed on the display unit 208, and the process proceeds to step S46. Specifically, the signature notification screen 540 is displayed on the display unit 161. In step S46, it is determined whether or not the end operation has been accepted. When the end operation is input to the operation unit 209 by the user, the end operation is accepted. It is in a standby state until the end operation is accepted (NO in step S46), and when the end operation is accepted (YES in step S46), the process ends.

FIG. 18 is a diagram showing a state before the portable information device is held by the holding member. FIG. 18 is a view of the operation panel 160 as viewed from above. FIG. 19 is a diagram showing a state in which the portable information device is held by the holding member. FIG. 19 is a side view of the operation panel 160. 18 and 19 show a state in which the MD image 532 is displayed on the MFP 100 and a state in which the signature image 544 is displayed on the mobile information device 200 for the sake of explanation. With reference to FIGS. 18 and 19, the camera 119 is arranged on the same surface as the surface on which the display surface of the display unit 161 is arranged on the operation panel 160. The camera 119 is arranged below the display surface of the display unit 161. In the mobile information device 200, the camera 207 is arranged on the same surface as the surface on which the display surface of the display unit 208 is arranged. The camera 207 is arranged above the display surface of the display unit 208.

The holding member 300 includes a device holding portion 301 and a connecting portion 303. The device holding unit 301 has the same shape as the outer circumference of the display surface of the display unit 208 of the mobile information device 200. The device holding unit 301 holds a portion outside the display surface of the display unit 208 of the mobile information device 200. The device holding unit 301 may be capable of holding at least two portions facing each other with the display surface on the outside of the display surface of the display unit 208 of the mobile information device 200.

One end of the connecting portion 303 is connected to the device holding portion 301, and the other end is connected to the housing of the operation panel 160. The connecting unit 303 connects the device holding unit 301 and the housing of the operation panel 160 so that the distance between the device holding unit 301 and the display surface of the display unit 161 is a predetermined distance L.

In a state where the mobile information device 200 is held by the holding member 300, the distance between the camera 207 of the mobile information device 200 and the display surface of the display unit 161 of the MFP 100 becomes a distance L, and the distance L of the display unit 208 of the mobile information device 200. The distance between the display surface and the camera 119 of the MFP 100 is the distance L. Further, the imaging range of the camera 207 of the mobile information device 200 includes a part of the display surface of the display unit 161 of the MFP 100, and the imaging range of the camera 119 of the MFP 100 includes a part of the display surface of the display unit 208 of the mobile information device 200. Is included.

In a state where the mobile information device 200 is held by the holding member 300, the setting images 511A and 511B displayed on the mobile information device 200 are read by the MFP 100, and the MD image 532 displayed on the MFP 100 is read by the mobile information device 200. , The signature image 544 displayed on the mobile information device 200 is read by the MFP 100. Therefore, after setting the destination e-mail address, the sender's e-mail address, and the digital certificate in the mobile information device 200, the user places the mobile information device 200 on the holding member 300 and puts the manuscript on the MFP 100. The start key of the operation unit 163 may be pressed for reading.

<First modification>
The data attached to the e-mail is image data obtained by reading the document by the document reading unit 130, but the data is not limited to this, and may be data stored in the HDD 115 or attached to the external storage device 117. The data may be stored in the CD-ROM 118 or the semiconductor memory.

Further, in the present embodiment, the data to be digitally signed is an e-mail to which the data is attached, but the data is not limited to the e-mail, and the data stored in the HDD 115 and the document reading unit 130 read the document. The image data to be output may be the data stored in the CD-ROM 118 or the semiconductor memory mounted on the external storage device 117.

Further, in the present embodiment, an example of transmitting an electronically signed e-mail has been shown, but the process is not limited to the transmission of the e-mail, and any process may be used as long as the electronically signed data is output to the outside. For example, the electronically signed data may be stored in the semiconductor memory mounted in the external storage device 117. Further, the electronically signed data may be transmitted to another computer via the communication I / F unit 112.

<Second modification>
In the present embodiment, the MD image 532 and the signature image 544 show an example in which a two-dimensional bar code is used, but the MD image 532 and the signature image 544 are not limited to the two-dimensional bar code. For example, it may be an image of a character string of text. In this case, the character string of the text is reproduced by recognizing the character of the MD image 532 or the signature image 544. Further, the MD image 532 and the signature image 544 may be moving images in which character strings of text are displayed one by one at different times in order. A character string can be reproduced by recognizing a moving image as a character for each frame.

As described above, the electronic signature system 1 in the present embodiment includes the MFP 100 and the portable information device 200, and the MFP 100 is an image obtained by reading the document by the document reading unit 130 for the target data to be signed. Data, e-mail, HDD 115, and data stored in the CD-ROM 118. The MFP 100 generates a message digest based on the target data, and displays an MD image 532 showing the message digest on the display unit 161. The mobile information device 200 causes the camera 207 to image the display surface of the display unit 161, acquires the MD image captured by the camera 207, and encrypts the message digest indicated by the MD image 532 included in the MD image with the private key. The signature image 544 showing the converted signature data is displayed on the display unit 208. The MFP 100 further causes the camera 119 to image the display surface of the display unit 208 of the portable information device 200, acquires the signature image captured by the camera 119, and publishes the data indicated by the signature image 544 included in the signature image. When the data decrypted by the key matches the message digest, the data indicated by the signature image 544 is associated with the target data as signature data.

Therefore, when the MD image 532 displayed by the MFP 100 is captured by the mobile information device 200, the message digest generated based on the target data is transmitted from the MFP 100 to the mobile information device 200. Further, the signature image 544 displayed by the mobile information device 200 is imaged by the MFP 100, so that the signature data is transmitted from the mobile information device 200 to the MFP 100. Therefore, the target data can be digitally signed without outputting the private key stored in the mobile information device 200 to the outside from the mobile information device 200.

Further, the MFP 100 generates an image showing encrypted data in which the message digest is encrypted with the public key corresponding to the private key stored in the mobile information device 200 as an MD image, and the mobile information device 200 is shown by the MD image 532. When the data to be decrypted can be decrypted with the private key, the data shown by the MD image decrypted with the private key is reproduced as a message digest. Therefore, the same data can be processed between the MFP 100 and the portable information device 200.

Further, the mobile information device 200 displays the setting images 511A and 511B indicating the electronic certificate on the display unit 208, and the MFP 100 causes the camera 119 to image the display surface of the display unit 208, and the setting captured image output by the camera 119. Is obtained, the electronic certificate is reproduced from the setting captured image, and the electronic certificate is associated with the target data in addition to the signature data. Therefore, it is possible to prove that the user of the mobile information device 200 is the owner of the target data.

Further, the mobile information device 200 displays the setting image 511A indicating the e-mail address of the destination of the e-mail on the display unit 208, the MFP 100 causes the camera 119 to image the display surface of the display unit 208, and the camera 119 outputs the image. Acquires the set captured image, reproduces the destination data from the set captured image, and transmits the target data to the destination specified by the destination data. Therefore, if the user specifies the destination e-mail address on the mobile information device 200, the MFP 100 can be made to send the target data to the destination e-mail address. Therefore, the operation becomes easy.

Further, the MD image 432 and the signature image 544 are images of one or more two-dimensional bar codes. Therefore, since the encrypted data and the signature data are indicated by a two-dimensional bar code, the information can be accurately transmitted.

Further, in the holding member 300, while holding the mobile information device 200, at least a part of the display surface of the display unit 208 is included in the imaging range of the camera 119, and at least a part of the display surface of the display unit 161 is the camera. The relative position between the MFP 100 and the portable information device 200 is determined so as to be included in the imaging range of 207. Therefore, the camera 207 can image the MD image 532, and the camera 119 can image the signature image 544. Therefore, the user does not need to adjust the imaging range of the camera 119 and the camera 207, which facilitates the work.

It should be considered that the embodiments disclosed this time are exemplary in all respects and not restrictive. The scope of the present invention is shown by the scope of claims rather than the above description, and it is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

1 Electronic signature system, 100 MFP, 111 CPU, 112 communication I / F section, 113 ROM, 114 RAM, 115 HDD, 116 facsimile section, 117 external storage device, 119 camera, 120 automatic document transfer device, 130 document reader, 140 image forming unit, 150 paper feeding unit, 160 operation panel, 161 display unit, 163 operation unit, 200 mobile information device, 201 CPU, 202 ROM, 203 RAM, 204 flash memory, 205 call unit, 206 wireless communication unit, 207 Camera, 208 display unit, 209 operation unit, 51 first imaging control unit, 53 document reading control unit, 55 transmission information acquisition unit, 57 certificate acquisition unit, 59 e-mail generation unit, 61 MD generation unit, 63 public key cryptography Cryptography unit, 65 MD notification unit, 67 related unit, 69 e-mail transmission unit, 71 setting image acquisition unit, 73 signature image acquisition unit, 75 signature image acquisition unit, 77 public key decryption unit, 79 collation unit, 81 MD image display unit, 83 MD image generation unit, 210 external storage device, 251 operation reception unit, 253 setting image generation unit, 255 setting image display unit, 257 second imaging control unit, 259 MD image acquisition unit, 261 private key decryption 263 Private key cryptography unit, 265 signature data notification unit, 271 destination reception unit, 273 source reception unit, 275 certificate designation reception unit, 279 MD image acquisition unit, 281 signature image display unit, 283 signature image Generation unit, 300 holding member, 301 device holding unit, 303 connecting unit, 500 transmission setting screen, 510 setting instruction screen, 511A, 511B setting image, 520 document reading guidance screen, 530 MD notification screen, 532 MD image, 540 signature notification Screen, 544 signature image.

Claims (23)

An electronic signature system including a first information processing device and a second information processing device.
The first information processing device includes a first display means and
The first imaging means for imaging the subject,
A generation method that generates a message digest based on the target data to be signed,
A message digest notification means for displaying an MD image showing the message digest on the first display means is provided.
The second information processing device includes a second display means and
A second imaging means for imaging the subject,
An MD image acquisition means for acquiring an MD image image that the second image pickup means captures and outputs the display surface of the first display means, and
A signature data notification means for displaying a signature image showing signature data in which the message digest is encrypted with a private key on the second display means in response to the reproduction of the message digest based on the MD captured image. With
The first information processing apparatus further includes a signature image acquisition means for acquiring a signature image image that the first image pickup means captures and outputs a display surface of the second display means.
When the data shown in the signature captured image is decrypted with the public key corresponding to the private key and matches the message digest, the data shown in the signature captured image is associated with the target data as the signature data. An electronic signature system equipped with means of attachment. The message digest notification means generates an image showing the encrypted data obtained by encrypting the message digest with the public key as the MD image.
When the data indicated by the MD image can be decoded by the secret key, the signature data notification means reproduces the data indicated by the MD image captured by the secret key as the message digest. Item 1. The electronic signature system according to item 1. The second information processing device further includes a certificate notification means for displaying a setting image showing an electronic certificate on the second display means.
The first information processing apparatus further includes a set captured image acquisition means for acquiring a set captured image that the first imaging means captures and outputs a display surface of the second display means.
A digital certificate reproduction means for reproducing the digital certificate from the set captured image is provided.
The electronic signature system according to claim 1 or 2, wherein the related means associates the digital certificate with the target data in addition to the signature data. The setting image further shows destination data indicating a destination of the target data.
The first information processing apparatus further includes a destination data reproducing means for reproducing the destination data from the set captured image.
The electronic signature system according to claim 3, further comprising a transmission means for transmitting the target data and the signature data associated with the related means to a destination specified by the destination data. The second information processing device further includes a destination notification means for displaying a setting image indicating the destination data indicating the destination of the target data on the second display means.
The first information processing apparatus further includes a set captured image acquisition means for acquiring a set captured image that the first imaging means captures and outputs a display surface of the second display means.
A destination data reproduction means for reproducing the destination data from the set captured image, and
The electronic signature system according to claim 1 or 2, further comprising a transmission means for transmitting the target data associated with the related means to a destination specified by the destination data. The electronic signature system according to any one of claims 1 to 5, wherein the MD image or the signature image is an image of one or more two-dimensional bar codes. At least a part of the display surface of the second display means is included in the imaging range of the first imaging means, and at least a part of the display surface of the first display means is included in the imaging range of the second imaging means. The electronic signature system according to any one of claims 1 to 6, further comprising a holding means for determining a relative position between the first information processing device and the second information processing device. An electronic signature method executed by an electronic signature system including a first information processing device and a second information processing device.
The first information processing device includes a first display means and a first image pickup means for photographing a subject.
The second information processing device includes a second display means and a second image pickup means for photographing a subject.
A generation step of generating a message digest based on the target data to be signed in the first information processing device, and
A message digest notification step of displaying an MD image showing the message digest on the first display means is executed.
An MD image acquisition image acquisition step of acquiring an MD image image to be output to the second information processing apparatus by the second image pickup means capturing the display surface of the first display means.
A signature data notification step of displaying a signature image showing signature data in which the message digest is encrypted with a private key on the second display means in response to the reproduction of the message digest based on the MD captured image. To execute,
A signature imaging image acquisition step of acquiring a signature imaging image to be output to the first information processing apparatus by the first imaging means capturing an image of the display surface of the second display means.
When the data shown in the signature captured image is decrypted with the public key corresponding to the private key and matches the message digest, the data shown in the signature captured image is associated with the target data as the signature data. With attached steps, and the electronic signature method to execute. The message digest notification step includes a step of generating an image showing encrypted data obtained by encrypting the message digest with the public key as the MD image.
The signature data notification step is a step of reproducing the data indicated by the MD image captured by the MD image as the message digest when the data indicated by the MD image can be decoded by the secret key. The electronic signature method according to claim 8, which includes. The second information processing apparatus is further made to execute a certificate notification step of displaying a setting image showing a digital certificate on the second display means.
A setting imaging image acquisition step of acquiring a setting imaging image to be output to the first information processing apparatus by the first imaging means capturing and outputting the display surface of the second display means.
The digital certificate reproduction step of reproducing the digital certificate from the set captured image is executed.
The electronic signature method according to claim 8 or 9, wherein the related step includes, in addition to the signature data, a step of associating the digital certificate with the target data. The setting image further shows destination data indicating a destination of the target data.
The first information processing apparatus further includes a destination data reproduction step of reproducing the destination data from the set captured image.
The electronic signature method according to claim 10, wherein a transmission step of transmitting the target data and the signature data associated in the association step to a destination specified by the destination data is executed. The second information processing apparatus is further made to execute a destination notification step of displaying a setting image indicating the destination data indicating the destination of the target data on the second display means.
The first information processing apparatus further includes a set image acquisition step stage for acquiring a set image acquired by the first image pickup means by capturing and outputting the display surface of the second display means.
A destination data reproduction step for reproducing the destination data from the set captured image, and
The electronic signature method according to claim 8 or 9, wherein the transmission step of transmitting the target data associated in the association step to the destination specified by the destination data is executed. The electronic signature method according to any one of claims 8 to 12, wherein the MD image or the signature image is an image of one or more two-dimensional bar codes. In the first information processing apparatus, at least a part of the display surface of the second display means is included in the imaging range of the first image pickup means, and at least a part of the display surface of the first display means is said. The electronic signature according to any one of claims 8 to 13, further comprising a holding means for determining a relative position between the first information processing device and the second information processing device so as to be included in the imaging range of the second information processing device. Method. An electronic signature program executed by a computer that controls an information processing device.
The information processing device includes a display means and an image pickup means for photographing a subject.
A generation step that generates a message digest based on the target data to be signed,
A message digest notification step of displaying an MD image showing the message digest on the display means, and
While the mobile information device that captures the display surface of the display means displays a signature image showing signature data in which the message digest is encrypted with a private key, the imaging means captures the display surface of the mobile information device. MD captured image acquisition step to acquire the MD captured image to be output
When the data shown in the MD captured image is decoded with the public key corresponding to the private key and matches the message digest, the data shown in the MD captured image is associated with the target data as the signature data. An electronic signature program that causes the computer to execute the attached step. The message digest notification step includes a step of generating an image showing encrypted data obtained by encrypting the message digest with the public key as the MD image.
The electronic signature program according to claim 15, wherein the mobile information device displays the signature image when the data indicated by the MD image can be decrypted with the private key. The computer is further made to perform a setting image acquisition step of acquiring a setting image acquired by the imaging means capturing and outputting a display surface of the mobile information device displaying a setting image indicating a digital certificate.
The related step according to claim 15 or 16, further comprising associating the digital certificate with the target data in addition to the signature data when the digital certificate is reproduced from the set captured image. Electronic signature program. The mobile information device displays the setting image further showing the destination data, and displays the setting image.
When the destination data is reproduced from the set captured image, the computer further comprises a transmission step of transmitting the target data and the signature data associated in the association step to the destination specified by the destination data. 17. The electronic signature program according to claim 17. The setting imaging image acquisition step of acquiring the setting imaging image that the imaging means captures and outputs the display surface of the mobile information device that displays the setting image indicating the destination data, and
A claim that further causes the computer to perform a transmission step of transmitting the target data associated in the association step to a destination specified by the destination data when the destination data is reproduced from the set captured image. The electronic signature program according to 15 or 16. The electronic signature program according to any one of claims 15 to 19, wherein the MD image or the signature image is an image of one or more two-dimensional bar codes. An electronic signature program that runs on a computer that controls a mobile information device that stores a private key.
The portable information device includes a display means and an image pickup means for photographing a subject.
An MD image acquisition step of acquiring an MD image image that the image pickup means captures and outputs the display surface of the information processing device, and
In response to the message digest being reproduced based on the MD captured image, the computer includes a signature data notification step of displaying a signature image indicating signature data in which the message digest is encrypted with a private key on the display means. An electronic signature program to be executed by. Display means and
An imaging means that captures the subject,
A generation method that generates a message digest based on the target data to be signed,
A message digest notification means for displaying an MD image showing the message digest on the display means, and
While the mobile information device that captures the display surface of the display means displays a signature image showing signature data in which the message digest is encrypted with a private key, the imaging means captures the display surface of the mobile information device. MD captured image acquisition means for acquiring MD captured images to be output
When the data shown in the MD captured image is decoded with the public key corresponding to the private key and matches the message digest, the data shown in the MD captured image is associated with the target data as the signature data. An information processing device equipped with an accessory means. Display means and
An imaging means that captures the subject,
An MD image acquisition unit that acquires an MD image image that the image pickup means captures and outputs the display surface of the information processing device, and
In response to the message digest being reproduced based on the MD captured image, the signature data notification means for displaying the signature image showing the signature data in which the message digest is encrypted with the private key is provided on the display means. Mobile information device.

Images