JP7564916B1 - Information processing device, information processing method, and information processing program - Google Patents

Abstract

An information processing device, an information processing method, and an information processing program are provided that are capable of accurately detecting and preventing unauthorized photographing by a worker of data displayed on a user's terminal screen.
[Solution] The information processing device 10 includes a camera information acquisition unit 13 that acquires an image of the front of the screen of the user terminal 20 captured via a camera 21 of the user terminal; a face authentication unit 14 that performs face authentication of a person based on the user's face information when the person appears in the acquired image; a lens extraction unit 15 that stores lens information of the imaging device and uses the lens information to determine whether the lens outline of the imaging device can be extracted in the image; a fraud determination unit 16 that detects fraud when the lens outline is extracted in the vicinity of a person whose face authentication has been successful; and a display control unit 17 that controls data on the screen of the user terminal when fraud is detected.
[Selected Figure] Figure 1

Description

Embodiments of the present invention relate to an information processing device, an information processing method, and an information processing program.

A working style known as teleworking, in which company work is carried out from a remote location away from the company's base, is becoming more common. Teleworking allows employees to work from anywhere over the network, and has the advantage of allowing them the flexibility to choose where to carry out their work, such as their own homes.

However, in telework, where employees work from home or elsewhere, it is difficult to constantly keep track of the status of each employee's work compared to the traditional way of working where employees gather at a company base to perform their work, and it is difficult to detect when an employee behaves in a manner that violates company policies regarding the performance of their work. In particular, it is difficult to detect improprieties even if a device screen is illegally photographed while the employee is working.

Previously, technology has been disclosed that detects an incident when a third party other than the person performing the work is caught on camera during teleworking.

JP 2015-222555 A

However, while the method of detecting an incident when a third party other than the worker is caught on camera can detect suspicious third party actions in advance, if a teleworking worker illegally photographs the screen of a work terminal using their own mobile device such as a smartphone or tablet, it is difficult to detect the fraud because the worker is the legitimate user of the work terminal, and there is a risk of work information being leaked.

The present invention has been made in consideration of these circumstances, and aims to provide an information processing device, information processing method, and information processing program that can accurately detect and prevent unauthorized capture of data displayed on a user's terminal screen by a worker.

An information processing device according to an embodiment of the present invention includes a camera information acquisition unit that acquires an image of the front of the screen of a user terminal captured via the camera of the user terminal; a face authentication unit that performs face authentication of a person based on the user's face information when a person appears in the acquired image; a lens extraction unit that stores lens information of an imaging device and determines whether the lens outline of the imaging device is extracted in the image using the lens information; a fraud determination unit that detects fraud when the lens outline is extracted in the vicinity of a person whose face authentication has been successful; and a display control unit that hides data on the screen of the user terminal when fraud is detected, and is characterized in that the display control unit hides an application that executes the data for the data being operated on the user terminal when fraud is detected .

Embodiments of the present invention provide an information processing device, an information processing method, and an information processing program that can accurately detect and prevent unauthorized capture of data displayed on a user's terminal screen by a worker.

FIG. 1 is a diagram showing an example of the configuration of an information processing device according to an embodiment of the present invention. 1 is a schematic diagram showing a method for detecting unauthorized photography according to the present embodiment; (A) An explanatory diagram showing an example of judging an image taken by the camera of a user terminal, and (B) an explanatory diagram showing another example of judging an image taken by the camera of a user terminal, in which it is determined that the image was taken illegally by a worker. 4 is a flowchart showing an example of an information processing method according to the embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.
FIG. 1 is a diagram showing an example of the configuration of an information processing device 10 according to the present embodiment.
The information processing device 10 is a server that performs face authentication of a person in an image captured by the camera 21 of the user terminal 20, determines whether the lens outline of the imaging device is extracted near a person whose face authentication has been successful, and controls data on the screen of the user terminal 20 when fraud is detected. The data refers to various data stored in the information processing device 10, including files, image data, video data, and the like. The data is executed and displayed by an application in the user terminal 20. The imaging device refers to an optical device for taking photos and videos, and a terminal that has the device built in, and includes cameras (including digital cameras), smartphones with built-in cameras, tablets, and other mobile terminals.

The user terminal 20 refers to a terminal such as a PC, smartphone, or tablet terminal used by employees of a company, organization, etc. for business purposes, and is a terminal capable of communicating with the information processing device 10 via a network. The user terminal 20 may also be a headset terminal that is worn and operated on the user's head.

The user terminal 20 is equipped with a camera 21. The camera 21 is capable of constant communication with the user terminal 20 and is an imaging unit that captures an image in front of the display screen (display) of the terminal. The camera 21 may be built into the display screen of the user terminal 20, or may be arranged on the side of the display screen of the user terminal 20 and connected to the main body of the user terminal 20 from the outside. When the user terminal 20 is a headset terminal, it is equipped with a camera that captures an image of the surroundings of the user wearing the terminal, in particular, the rear of the user.

The images captured by the camera 21 include still images, videos, etc. The acquired image data is transmitted to the information processing device 10.

Note that in FIG. 1, three user terminals 20 (20a, 20b, 20c) and three cameras 21 (21a, 21b, 21c) are shown on each user terminal 20, but this configuration is not limited to this, and multiple user terminals 20 and cameras 21 may be connected.

A specific configuration of the information processing device 10 according to this embodiment will be described.
The information processing device 10 includes a data transmission/reception unit 11 , a user information registration unit 12 , a camera information acquisition unit 13 , a face authentication unit 14 , a lens extraction unit 15 , a fraud determination unit 16 , and a display control unit 17 .

The functions of each unit constituting the information processing device 10 may be realized by executing a specific program code using a processor, and are not limited to such software processing. For example, the functions may be realized by hardware processing using an ASIC or the like, or may be realized by a combination of software processing and hardware processing. Furthermore, each unit constituting the information processing device 10 may be realized by a separate server, and for example, data accessed from the user terminal 20 may be stored in a server separate from the main body of the information processing device 10.

The data transmission/reception unit 11 transmits data such as files and video data in response to an access request from the user terminal 20, and receives image data captured by the camera 21 of the user terminal 20.

The user information registration unit 12 registers authentication information for authenticating the user of the user terminal 20. A login ID, a password, etc. are registered in association with a user ID for identifying each user. In addition, face information of the user to be used for face authentication is registered in association with the user ID. When the user terminal 20 connects to the information processing device 10 and accesses data, it may be authenticated based on the registered authentication information.

Users with access rights may also be set in association with each piece of data. Multiple users may be set as groups, and data access rights may be set on a group basis.

The camera information acquisition unit 13 acquires images captured by the camera 21 of the user terminal 20. The camera information acquisition unit 13 starts capturing images with the camera 21 of the user terminal 20 when access to data from the user terminal 20 starts, and causes the camera 21 to continue capturing images until the access from the user terminal 20 ends. The camera information acquisition unit 13 acquires images captured by the camera 21 at any time.

The face authentication unit 14 extracts a person's face from an image acquired by the camera 21, and performs face authentication by comparing the extracted face with the user's face information for authentication. When multiple people are extracted from an image, face authentication is performed for each person based on the user's face information. A person whose face is successfully authenticated in the image becomes a user who has access authority to the data, while a person whose face is not successfully authenticated becomes a user who does not have access authority. When a person whose face is successfully authenticated is present in the image, the face authentication unit 14 transmits an image including that person to the lens extraction unit 15. When a person whose face is successfully authenticated is present, access to the data may be denied.

The facial authentication method may be performed using facial information of the user that has been registered in advance in association with the user, or may be performed using facial information for authentication that is stored in the user terminal 20.

In addition, when multiple people are viewing the data, the multiple users who will be viewing the data may be registered in advance, and facial authentication may be performed on the image captured by the camera 21 based on the facial information of each registered user.

The lens extraction unit 15 stores lens information of the imaging device, and uses the lens information to determine whether the outer shape of the lens of the imaging device can be extracted in an image. The lens information of the imaging device is information for identifying the lens of the imaging device from within an image, and refers to information such as the shape (e.g., a circle with a diameter of 1 cm) and color of the lens mounted on the imaging device. The lens information may also include information such as the number and arrangement of lenses in a rectangular imaging device (e.g., a smartphone or tablet). In this case, the outer shape (length and width, etc.) of the imaging device is set, and information such as the arrangement of the lenses within that outer shape and the number of lenses is set.

The lens extraction unit 15 determines whether the lens outer shape is extracted in an image captured by the camera 21 based on the lens information. The lens extraction unit 15 may also learn from multiple images capturing the lens outer shape of the imaging device by machine learning, and determine whether the lens outer shape is extracted in an image captured by the camera 21 using the generated image determination model. The lens extraction unit 15 may also determine that the lens outer shape of the imaging device is present in the image when an object matching the shape of the lens stored in the lens information is present in the image and an image of reflected light is present in the object.

In addition, if the lens outline of the imaging device is present in the image, the lens extraction unit 15 may determine the orientation of the lens outline in the captured image. The lens orientation means the direction of the front surface of the lens (the direction of the normal to the front surface of the lens). The lens extraction unit 15 then determines whether the direction of the front surface of the lens is oriented in the direction of the display screen of the user terminal 20.

The lens extraction unit 15 may determine that the front of the lens is facing the display screen of the user terminal 20 if the front of the lens faces the display screen of the user terminal 20 at least once within a certain period of time (e.g., within 10 seconds).

The fraud determination unit 16 detects fraud when the lens outline of the imaging device is extracted in the vicinity (for example, within a distance of 30 cm) of a person whose face has been successfully authenticated. It determines that a person with access rights to the data is pointing their imaging device at the screen of the user terminal 20, and determines that there is a risk of fraudulent photography.

The fraud determination unit 16 may also detect fraud when the lens outline of the imaging device is extracted near a person whose face has been successfully authenticated, and the front of the lens faces the display screen of the user terminal 20.

The fraud determination unit 16 may also detect fraud when the lens outline of the imaging device is extracted near a person whose face has been successfully authenticated and that person remains in the same position for a predetermined period of time or more (e.g., one minute or more).

When the display control unit 17 determines that there is a risk of unauthorized photography, it controls the data displayed on the user terminal 20 so that it cannot be accessed. Specifically, when unauthorized photography is detected, the application that executes the data being operated by the user of the user terminal 20 is hidden. One method of hiding the data is to cover the entire application being operated with a black image so that the application that executes the data cannot be viewed. Alternatively, only the application at the forefront of the screen may be hidden. When unauthorized photography is no longer detected, the application is unhidden and the data can be viewed.

In addition, when data is displayed on the display screen of the user terminal 20, the screen may be controlled to switch to a warning screen.

The display control unit 17 may also control the user terminal 20 to stop copying or expanding the display screen if it determines that unauthorized photography has occurred.

As a result, if a user of the user terminal 20 is working with data displayed on the terminal screen and there is a risk of the user engaging in fraudulent activity such as taking a photo of the terminal screen, the data can be made immediately unviewable, thereby preventing fraudulent activity from occurring.

Figure 2 is a schematic diagram showing a method for detecting unauthorized photography according to this embodiment (see Figure 1 as appropriate). Here, we consider a case in which user A of user terminal 20 connects to information processing device 10 and views data using a specific application.

When data viewing begins, the camera information acquisition unit 13 acquires an image of the front of the display screen of the user terminal 20 from the camera 21. User A and others are captured in the camera image. At this time, face authentication is performed on the captured camera image using the face information of user A. Face authentication is successful for user A in the image.

Then, using the lens information, it is determined whether the lens outline of the imaging device 50 is extracted near the image of user A, whose face has been successfully authenticated. Because the lens outline of the imaging device 50 is extracted near user A, it is determined that there is a risk of unauthorized photography by user A.

If it is determined that unauthorized photography has occurred, the application that executes the data displayed on the user terminal 20 is hidden so that the data cannot be viewed.

Figures 3 (A) and (B) are explanatory diagrams showing an example of determining an image captured by the camera 21 of the user terminal 20. Here, it is assumed that user A of the user terminal 20 is viewing data.

In the example of FIG. 3(A), while face authentication is successful for user A of user terminal 20, the lens outline of the imaging device is not extracted in the vicinity of user A, so unauthorized photography is not determined.

In FIG. 3(B), face authentication is successful for user A of user terminal 20, and the outline of a smartphone is extracted in the vicinity of user A, so it is determined that there is a risk of unauthorized photography by user A. The data displayed on user terminal 20 is controlled so that it cannot be viewed.

In this way, it is possible to accurately determine the actions of a worker attempting to illegally photograph the screen of the user terminal 20, and to prevent such unauthorized photographing from occurring.

Next, the operation of the information processing device 10 according to this embodiment will be described.

Figure 4 is a flowchart showing an example of an information processing method according to this embodiment (see Figure 1 as appropriate).

Upon receiving a request from the user terminal 20, the data transmission/reception unit 11 transmits the work data to the user terminal 20. The user terminal 20 displays the work data (S10).

When the user terminal 20 accesses the work data, the camera information acquisition unit 13 acquires an image captured by the camera 21 of the terminal (S11).

The face authentication unit 14 determines whether face authentication is successful for the person in the acquired image based on the face information of the user of the user terminal 20 (S12). If face authentication is not successful for the person in the image, the display of the work data is hidden (S12: NO, S14).

When a person whose face has been successfully recognized is present in the image, the lens extraction unit 15 determines whether the lens outline of the smartphone is extracted near the person (S12: YES, S13). If the lens outline is not extracted near the person, the display of the work data continues (S13: NO).

The fraud determination unit 16 determines that the operation is fraudulent if the lens outline is extracted near a person whose face has been successfully authenticated. The display control unit 17 hides the display of the work data when fraud is determined (S13: YES, S14).

According to the information processing device of the embodiment described above, face recognition is performed on a person in an image captured by the camera of the user terminal, and it is determined whether the lens outline of the imaging device is extracted near a person whose face recognition has been successful. If fraudulent activity by a worker is detected, the display of the application being executed on the user terminal 20 is controlled. As a result, if fraudulent photography is performed by a user performing work on the user terminal 20, the display of the application being used for work can be hidden, thereby preventing fraudulent activity in advance.

The program executed by the information processing device 10 is provided by being pre-installed in a storage circuit such as a ROM. Alternatively, the program may be provided by being stored in a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD, or flexible disk as a file in an installable or executable format. The program executed by the information processing device 10 may also be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

In addition, each of the units, the camera information acquisition unit 13, the face authentication unit 14, the lens extraction unit 15, and the fraud determination unit 16, may be implemented in the user terminal 20 to control the data displayed on the user terminal 20.

Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims.

10...information processing device, 11...data transmission/reception unit, 12...user information registration unit, 13...camera information acquisition unit, 14...face authentication unit, 15...lens extraction unit, 16...fraud determination unit, 17...display control unit, 20 (20a, 20b, 20c)...user terminal, 21 (21a, 21b, 21c)...camera, 50...imaging device (smartphone).

Claims (3)

a camera information acquisition unit that acquires an image of a front portion of a screen of the user terminal captured via a camera of the user terminal;
a face authentication unit that performs face authentication of a person based on face information of a user when the person is captured in the acquired image;
a lens extraction unit that stores lens information of an imaging device and determines whether or not an outer shape of a lens of the imaging device is extracted in the image by using the lens information;
a fraud determination unit that detects fraud when the lens outline is extracted near a person whose face has been successfully authenticated;
A display control unit that hides data on the screen of the user terminal when fraud is detected ,
The display control unit, when fraud is detected, hides an application that executes the data for which an operation is being executed on the user terminal.
23. An information processing apparatus comprising: acquiring an image of a front screen of the user terminal captured via a camera of the user terminal;
When a person is captured in the captured image, performing face authentication on the person based on face information of the user;
storing lens information of an imaging device, and determining whether or not an outer shape of a lens of the imaging device is extracted in the image using the lens information;
detecting fraud when the lens outline is extracted in the vicinity of a person whose face has been successfully authenticated;
When fraud is detected, hiding data on the screen of the user terminal ;
When fraud is detected, an application that executes the data is hidden for the data being operated on the user terminal.
23. An information processing method comprising: Computer,
A function for acquiring an image of the front of the screen of the user terminal captured via a camera of the user terminal;
A function to perform face recognition of a person based on the user's facial information when the person appears in the captured image;
a function of storing lens information of an imaging device and determining whether or not the outer shape of the lens of the imaging device is extracted in the image by using the lens information;
A function to detect fraud when the lens outline is extracted near a person whose face has been successfully recognized;
a function of hiding data on the screen of the user terminal when fraud is detected ,
When fraud is detected, an application that executes the data is hidden for the data being operated on the user terminal.
2. An information processing program comprising:

Images