WO2025191769A1 - Image retrieval device, image retrieval method, image retrieval program, and recording medium - Google Patents

Abstract

This image retrieval device comprises: an input/output and image comparison unit for storing face image data representing a face image of a user registered by the user and a keyword registered by the user as the cause of leakage of the face image; and an image extraction unit for extracting a plurality of pieces of still-image data from moving-image data acquired at a website that is a sexual site on the Internet and contains the keyword registered by the user, and acquiring, from the extracted still-image data, image data in which face information is included. The input/output and image comparison unit is configured to compare the image data that is acquired by the image extraction unit and contains face information with the face image data that is registered by the user and stored in the input/output and the image comparison unit, and to extract the URL of face image data having a high degree of similarity. The image extraction unit is configured to extract still-image data from the moving-image data using a plurality of mutually different processing procedures, and to integrate the plurality of pieces of still-image data extracted using the plurality of mutually different processing procedures.

Description

Image search device, image search method, image search program, and recording medium

The present invention relates to an image search device, an image search method, an image search program, and a recording medium on which an image search program is recorded, for searching for user images that have been leaked onto the Internet.

In recent years, with the increasing performance of cameras and the widespread use of smartphones, there has been an increase in the number of cases of voyeurism and revenge porn. Furthermore, because it is easy to post photos and videos on adult sites and social networking services (SNS), users' own photos and videos can sometimes be uploaded without their permission and leaked to pornographic websites on the Internet.

Patent Document 1 discloses technology for searching for personal images of a client that have been made public on the Internet against the client's will. This image search technology extracts feature data of people from all public images on the Internet, as well as feature data of the client's personal image, and compares these extracted feature data with each other to determine whether the client's personal image is the same as multiple public images, thereby searching for the client's personal image.

Patent No. 5150572

The image search technology described in Patent Document 1 searches the entire Internet to check whether or not an image of the requester exists. However, the amount of images posted on the Internet is enormous, and extracting features of people from all of these public images, extracting features from the extracted features, and then matching them not only requires a significant amount of computing resources, but also takes a significant amount of time, making it extremely difficult to implement image search with a simple configuration.

Furthermore, the image search technology described in Patent Document 1 is a search technology designed to prevent the violation of the client's privacy, and therefore is unable to quickly detect the leaking of the client's facial image, especially in the early stages, in order to prevent the client from becoming a victim of voyeurism or revenge porn.

Therefore, an object of the present invention is to provide an image search device, image search method, image search program, and recording medium that can reliably detect when a user's facial image has been leaked onto the Internet using a simple configuration.

Another object of the present invention is to provide an image search device, an image search method, an image search program, and a recording medium that can quickly detect, at an early stage, that a user's facial image has been leaked onto the Internet.

The present invention provides an image search device that includes an input/output and image comparison unit that stores facial image data representing the user's own facial image registered by the user and keywords registered by the user as the cause of the facial image leak, and an image extraction unit that extracts multiple still image data from video data acquired from websites that are sexually explicit on the Internet and include keywords registered by the user, and acquires image data containing facial information from the extracted still image data. The input/output and image comparison unit is configured to compare the image data containing facial information acquired by the image extraction unit with the user's registered facial image data stored in the input/output and image comparison unit, and extract the URL (web page address) of facial image data that has a high degree of similarity, and the image extraction unit is configured to extract still image data from the video data using multiple different processing procedures and integrate the still image data extracted by these multiple different processing procedures.

In this invention, video data is obtained from websites that contain keywords registered as being the cause of facial image leaks. Instead of searching the entire Internet, only websites for preventing voyeurism and revenge pornography are searched. This allows for rapid detection of user facial images leaked on the Internet at an early stage and simplifies the configuration of the image search device. Furthermore, in this invention, multiple still image data are extracted from the acquired video data, and when image data containing facial information is obtained from the extracted still image data, multiple different processing procedures are used to extract still image data from the video data, and the still image data extracted by these multiple different processing procedures is integrated. In this way, different processing procedures result in different final images, making facial image detection significantly more reliable than when facial images are extracted using only a single procedure. This makes it possible to reliably detect when a user's facial image has been leaked on the Internet with a simple configuration.

It is preferable that the multiple different processing procedures performed by the image extraction unit include a process of playing back video data at high speed and extracting still image data determined by the machine learning model to contain a face, and a process of playing back video data at high speed and extracting still image data at any fixed frame interval.

It is also preferable that the multiple different processing procedures performed by the image extraction unit include a process of playing back video data at high speed and extracting a still image when the machine learning model determines that a face is visible; a process of playing back video data at high speed and extracting a still image of a face at the timing determined by the machine learning model that a face is most visible; a process of playing back video data at high speed and extracting facial features at the timing determined by the machine learning model that a face is most visible, thereby extracting a still image of the face; and a process of playing back video data at high speed and extracting still images of the entire video at regular frame intervals, and extracting the still image with the highest probability of showing a face for each extracted still image using the machine learning model.

It is also preferable that the image extraction unit is configured to compare the confidence value of image data containing facial information with a threshold value and extract image data that most closely resembles a facial image.

In this case, it is more preferable that the image extraction unit be configured to divide the image data containing facial information into a predetermined number of pieces starting from the beginning, select the best image data from the predetermined number of divided pieces of image data, and compare the confidence value of the selected image data with a threshold value.

It is also preferable that the system further includes a URL crawler unit that collects URLs of websites on the Internet that are sexually explicit and contain keywords registered by the user, and that the image extraction unit is configured to obtain video data from websites that correspond to the URLs collected by the URL crawler unit.

It is also preferable that the input/output and image comparison unit is configured to store facial image data and keywords sent from the user terminal, and to send the extracted URL to the user terminal.

The present invention further provides an image search method that stores facial image data representing the user's own facial image registered by the user and keywords registered by the user as the cause of the facial image leak, extracts multiple still image data from video data acquired from websites on the Internet that are sexually explicit and include the keywords registered by the user, and acquires image data containing facial information from the extracted still image data. The acquired image data containing the facial information is compared with the facial image data registered by the user, URLs of facial image data with a high degree of similarity are extracted, still image data is extracted from the video data using multiple different processing procedures, and the still image data extracted using the multiple different processing procedures is integrated.

It is preferable that the multiple different processing procedures include a process of playing back video data at high speed and extracting still image data determined by the machine learning model to contain a face, and a process of playing back video data at high speed and extracting still image data at any fixed frame interval.

It is also preferable that the multiple different processing procedures include a process of playing back video data at high speed and extracting a still image when a machine learning model determines that a face is shown; a process of playing back video data at high speed and extracting a still image of a face at the timing determined by the machine learning model that a face is most likely to be shown; a process of playing back video data at high speed and extracting facial features at the timing determined by the machine learning model that a face is most likely to be shown, thereby extracting a still image of the face; and a process of playing back video data at high speed and extracting still images from the entire video at regular frame intervals, and extracting the still image with the highest probability of showing a face for each extracted still image using the machine learning model.

It is also preferable to compare the confidence value of image data containing facial information with a threshold value to extract the image data that most closely resembles a facial image.

In this case, it is more preferable to divide the image data containing facial information into a predetermined number of parts starting from the beginning, select the image data with the highest accuracy from among the predetermined number of divided image data parts, and compare the confidence value of the selected image data with a threshold value.

It is also preferable to collect URLs of websites on the Internet that are sexually explicit and contain keywords registered by the user, and obtain video data from the sites corresponding to the collected URLs.

It is also preferable to store the facial image data and keywords sent from the user terminal and send the extracted URL to the user terminal.

The present invention also provides a program for causing a computer to function as an input/output and image comparison means for storing facial image data representing a user's own facial image registered by the user and keywords registered by the user as the cause of the facial image leak, and an image extraction means for extracting multiple still image data from video data acquired from a website on the Internet that is a sexually explicit site and includes keywords registered by the user, and acquiring image data containing facial information from the extracted still image data. The input/output and image comparison means compares the acquired image data containing facial information with the facial image data registered by the user and extracts the URL of facial image data that has a high degree of similarity, and the image extraction means extracts still image data from video data using multiple different processing procedures and integrates the still image data extracted by the multiple different processing procedures.

The present invention also provides a computer-readable recording medium having recorded thereon a program for causing a computer to function as an input/output and image comparison means for storing facial image data representing the user's own facial image registered by the user and keywords registered by the user as the cause of the facial image leak, and an image extraction means for extracting multiple still image data from video data acquired from websites on the Internet that are sexually explicit and include keywords registered by the user, and acquiring image data containing facial information from the extracted still image data. The input/output and image comparison means compares the acquired image data containing facial information with stored facial image data registered by the user, and extracts the URL of facial image data that has a high degree of similarity, and the image extraction means extracts still image data from video data using multiple different processing procedures and integrates the still image data extracted by the multiple different processing procedures.

In this invention, only websites for preventing voyeurism and revenge pornography are searched, making it possible to quickly detect a user's facial image leaked onto the Internet at an early stage and simplifying the configuration of the image search device. Furthermore, since different processing procedures result in different images being ultimately obtained, the reliability of facial image detection is significantly increased, making it possible to reliably detect when a user's facial image has been leaked onto the Internet with a simple configuration.

1 is a block diagram schematically illustrating the overall configuration of an image search device according to an embodiment of the present invention. FIG. 2 is an explanatory diagram illustrating the function of the image search device of FIG. 1; 2 is a flowchart showing the processing operation of an input/output and image comparison unit in the image search device of FIG. 1 . 2 is a flowchart showing the processing operation of an input/output and image comparison unit in the image search device of FIG. 1 . 2 is a flowchart showing the processing operation of a URL crawler unit in the image search device of FIG. 1 . 10 is a flowchart showing the processing operation of a still image crawler unit in the image search device of FIG. 1 . 10 is a flowchart showing the processing operation of a video crawler unit in the image search device of FIG. 1 . 10 is a flowchart showing a modified example of a part of the processing operation of the video crawler unit in the image search device of FIG. 1 . FIG. 10 is a diagram illustrating a registration screen for a user's own face image on a user terminal. FIG. 10 is a diagram illustrating a keyword registration screen on a user terminal. FIG. 10 is a diagram illustrating a search result display screen on a user terminal.

FIG. 1 shows a schematic diagram of the overall configuration of an image search device according to one embodiment of the present invention, and FIG. 2 explains the functions of the image search device according to this embodiment.

In Figure 1, 10 denotes a user terminal, such as a smartphone, tablet terminal, or computer terminal operated by a user, on which the image search app of this embodiment is installed, 11 denotes an image search server on the cloud that can communicate with the user terminal 10, and 12 denotes the Internet, which can be accessed by the image search server 11.

As shown in Figure 1, the image search server 11 includes at least an input/output and image comparison unit 13, a cloud server 14, a URL crawler unit 15, a still image crawler unit 16, and a video crawler unit 17.

As shown in FIG. 2, the input/output and image comparison unit 13 includes a database (AWS/RDS) 13a and storage (AWS/S3) 13b provided by a cloud computing service (AWS). This input/output and image comparison unit 13 is communicatively connectable to the user terminal 10, and is configured to store the user's facial image data transmitted from the user terminal 10 and keywords registered by the user in association with this facial image data. The input/output and image comparison unit 13 is further configured to store, in association with the user, the URLs of videos containing facial images highly similar to the user's facial image, extracted by a cloud server (Azure) 14 provided by the cloud computing service.

As mentioned above, the user's facial image data sent from the user terminal 10 is stored in storage (AWS/S3) 13b, and the keywords registered by the user are stored in database (AWS/RDS) 13a. The cloud server (Azure) 14 is configured to learn the images collected by the still image crawler unit 16 or video crawler unit 17, assign a Face ID, and compare these collected images with images registered by the user to determine their similarity.

The URL crawler unit 15 uses Library Selenium, an automatic crawling program that utilizes one of the web crawlers present on the AWS cloud server (AWS/EC2) 15a. This URL crawler unit 15 is configured to collect URLs of videos containing registered keywords from multiple predetermined websites on the Internet 12 that deal with sexually explicit videos and have the potential for facial image leaks. Examples of predetermined websites include Pornhub, FC2, Tokyo Motion, Twitter Video Tools, XVIDEOS, and Twitter. A list of the collected URLs is stored in the URL list unit 15b within the URL crawler unit 15.

The URL list unit 15b is made up of an AWS database (AWS/RDS) and a local file (configuration file within local program files) storage unit, and collected URLs are stored in this RDS and local file storage unit. The reason for writing to the RDS is that various systems such as video crawlers and face similarity systems require access to URLs and information such as associated video data.

In this embodiment, the still image crawler unit 16 is primarily composed of AWS storage (AWS/S3) 16a, an image search unit 16b, and a machine learning unit 16c.

The image search unit 16b uses a search engine described in a configuration file within the local program files to perform an image search on websites using specified keywords stored in the input/output memory unit, and stores the displayed images in storage (AWS/S3) 16a. The search engines used are primarily Google, Yandex, Yahoo, Baidu, etc.

The machine learning unit 16c utilizes a deep learning algorithm and is built using the FaceBoxes model, a library specialized for human face detection. This machine learning model learns about human facial features from a vast number of face images, making it possible to detect facial information contained in unknown input image data. The detected facial information is accompanied by a numerical value called confidence, which indicates the certainty that a face is contained in the detected area. In other words, the higher this numerical value, the more likely the detected area is to be a face. The FaceBoxes model is a publicly available machine learning model library and was built by learning from images of Western faces. As described below, a model that has been fine-tuned using images of Asian faces is used to improve the accuracy of facial image detection compared to the original model.

The machine learning unit 16c is further configured to store all image data whose confidence value exceeds a predetermined threshold in storage (AWS/S3) 16a.

In this embodiment, the video crawler unit 17 is primarily composed of a video crawler located on the cloud server (AWS/EC2) 15a, AWS storage (AWS/S3) 17a, an image extraction unit 17b, and the aforementioned machine learning unit 16c.

The video crawler uses a search engine (Google Chrome) described in a configuration file within the local program files to play numerous videos at URLs stored in the URL list section 15b, and obtains a huge number of still image data for each video data. It then uses the FaceBoxes model, a library specialized for human face detection, to obtain only images containing facial information from the resulting image group, and is configured to select the image with the highest confidence value (the image with the clearest face).

There are two ways to obtain still images from the many videos at the URLs stored in the URL list unit 15b: downloading and playing the videos, or crawling and playing the videos from a website. Either method is acceptable. The image extraction unit 17b plays the many obtained videos and obtains many still images.

The machine learning unit 16c utilizes a deep learning algorithm and is built using the FaceBoxes model, a library specialized for human face detection. This machine learning model learns about human facial features from a vast number of facial images, enabling it to detect facial information contained in input, unknown image data. The detected facial information is accompanied by a numerical value called confidence, which indicates the certainty that a face is contained in the detected area. In other words, the higher this numerical value, the more likely the detected area is to be a face. Note that the FaceBoxes model is a publicly available machine learning model library and was built by learning from images of Western faces, so its face detection accuracy for Asian people is not very high. Therefore, in this embodiment, independently collected facial images of Asian people are input back into the FaceBoxes model as training data, and fine-tuning is performed to adjust the model weights. This improves the face image detection accuracy compared to the original model.

The machine learning unit 16c is further configured to store all image data whose confidence value exceeds a predetermined threshold in storage (AWS/S3) 17a.

Figures 3 and 4 show the processing operations of the input/output and image comparison unit 13, and these figures will be used below to explain the processing operations of this input/output and image comparison unit 13.

First, the user registers an image of their face via the user terminal 10. That is, as shown in FIG. 9, multiple pieces of face image data, including not only straight faces but also smiling and profile images, are sent to the image search server 11, if possible. Furthermore, as shown in FIG. 10, keywords indicating "who," "when," "where," "what," etc. are sent to the image search server 11 via the user terminal 10. For example, keywords such as "ex-boyfriend," "classmate," "listener," and "personal photo" are sent to the image search server 11 as the photographer (if known); keywords such as "JD," "female college student," "office worker," "delivery health," and "sugar daddy" are sent to the image search server 11 as the identity of the photographer (if known); keywords such as "hotel," "Shibuya," "Shinjuku," "shop name," "live chat," "amateur video," and "voyeur" are sent to the image search server 11 as the location of the photo (if known); keywords such as "matching app," "SNS," "Twitter," and "cosplay," and keywords such as the user's stage name or nickname.

As shown in FIG. 3, the input/output and image comparison unit 13 receives multiple pieces of facial image data sent from the user terminal 10 and keywords sent together with this facial image data (step S1).

Next, the input/output and image comparison unit 13 stores the received user's facial image data in storage (AWS/S3) 13b, and stores the keywords registered by the user in association with this data in database (AWS/RDS) 13a (step S2).

Furthermore, similar keywords that are similar to the keywords registered by the user and that are determined to be appropriate through crawling processing, either automatically or manually, are stored in database (AWS/RDS) 13a (step S3).

Meanwhile, as shown in Figure 4, the input/output and image comparison unit 13 inputs the user's facial image stored in storage (AWS/S3) 13b into Azure Similar on the cloud server 14, and outputs the ID of the facial image with the highest degree of similarity. Based on this ID, the database (AWS/RDS) 13a is referenced, and the URL of the video in which the facial image exists and the probability that it is the user themselves are extracted (step S11). That is, the URL and probability are output from Azure, and the posting date and time are read and output from the database (AWS/RDS).

The input/output and image comparison unit 13 then transmits the URL of the video containing the highly similar facial image, the probability that the person is the same person, the posting date and time, and the discovery date and time to the user terminal 10 (step S12). The user terminal 10 then displays the search results (discovery date, whether or not the information was checked, posting date, URL, and identity probability) as shown in FIG. 11.

Figure 5 shows the processing operations of the URL crawler unit 15, and the processing operations of this URL crawler unit 15 will be explained below using this figure.

The URL crawler unit 15 first determines in advance the multiple websites to be accessed (step S21).

Next, for these determined websites, the URLs are crawled based on the keywords registered by the user and similar keywords stored in the database (AWS/RDS) 13a, and URLs of videos containing these keywords are obtained (step S22).

The acquired URLs are then stored as a URL list in the URL list section 15b (step S23).

Figure 6 shows the processing operation of the still image crawler unit 16, and the processing operation of this still image crawler unit 16 will be explained below using this figure.

The still image crawler unit 16 uses a predetermined search engine in the image search unit 16b to perform an image search for still images using specified keywords stored in the database (AWS/RDS) 13a (step S31). Because there is a wide variety of websites from which images can be collected, the specified keyword, for example, "secretly taken" in the case of images taken secretly, is used in the search.

Next, the retrieved still images are temporarily stored in local cache memory (step S32).

Next, a deep learning machine learning algorithm for face detection (e.g., the FaceBoxes model) is applied to the numerous still images stored in this cache memory to extract still images containing faces and confidence values representing the probability that a face is included (step S33).

Next, still images (image data) whose extracted confidence values exceed the threshold are stored in storage (AWS/S3) 16a and a list registration process is performed (step S34).

Meanwhile, asynchronously with the processing of steps S31-34, when a user registers their own facial image data and keywords, Azure's Find Similar is called up, and the image is trained in this Azure Find Similar model. The image's original URL, posting date and time, keywords, and the image ID trained by Azure are recorded in database (AWS/RDS) 13a, and the image data is uploaded to and stored in storage (AWS/S3) 13b (step S35).

Figure 7 shows the processing operation of the video crawler unit 17, and the processing operation of this video crawler unit 17 will be explained below using this figure.

The video crawler unit 17 first uses a predetermined image search engine (Google Chrome) to retrieve a large number of videos using the specified keywords from the URLs stored in the URL list unit 15b (step S41).

Next, the acquired multiple video images are played back at high speed to acquire still images that show the faces identified by the machine learning model (step S42).

Meanwhile, the acquired multiple video images are played back at high speed, and still images are acquired at any fixed frame interval (step S43).

Next, the large number of still images extracted using the processing procedure of step S42 (a processing procedure different from step S43) and the large number of still images extracted using the processing procedure of step S43 (a processing procedure different from step S42) are integrated and temporarily stored in local cache memory (step S44). In this way, in this embodiment, still images are extracted from moving images using two different processing procedures, and the still images extracted using these two processing procedures are integrated to extract still images. Because the final images obtained differ depending on the processing procedures, the reliability of facial image detection is significantly higher than when facial images are extracted using only one procedure.

As the volume of image data is too large to record directly in storage, in steps S42 and S43, only images that show faces are selected from a group of images that do not necessarily show faces, and the image with the highest confidence value is ultimately retained. Step S44 is similar: from the group of images that do show faces, the image that best shows the face (with the highest confidence value) is ultimately retained. This is ultimately uploaded to storage, but more precisely, after the image with the highest confidence value for each of the videos listed in the URL crawler has been obtained, it is trained by Azure through a list registration process and stored in storage (AWS/S3) 17a.

When obtaining the image with the highest confidence value, rather than evaluating all images, it is desirable to divide the multiple image groups obtained from the video into groups of a predetermined number (for example, every 100 images) from the beginning and select the image with the highest accuracy from among them. Then, for the multiple facial images obtained from each image group, the one with the highest confidence is finally selected. The reason for dividing into image groups and evaluating each division unit in this way, rather than selecting the image with the highest accuracy from all images, is that processing can be stopped midway if a threshold is exceeded, allowing for efficient processing overall. For example, if the image with the highest accuracy is selected from 1,000 images, simply evaluating the image data of 1,000 images would be necessary, but if the group is divided into 10 groups of 100 images each, and the threshold is exceeded on the 10th image, then it will be sufficient to evaluate 10 x 10 = 100 images.

After processing in step S44, a deep learning machine learning algorithm for face detection (e.g., the FaceBoxes model from the library) is applied to the numerous still images stored in the cache memory to extract still images containing faces and confidence values representing the probability that a face is included (step S45).

Next, still images (image data) whose confidence values exceed the threshold are stored in storage (AWS/S3) 17a (step S46).

Meanwhile, asynchronously with the processing of steps S41-46, when a user registers their own facial image data and keywords, Azure's Find Similar is called up, and the image is trained in this Azure Find Similar model. The image's original URL, posting date and time, keywords, and the image ID trained by Azure are recorded in database (AWS/RDS) 13a, and the image data is uploaded and stored in storage (AWS/S3) 13b (step S47).

Figure 8 shows a modified version of the processing operations in which some of the processing operations of the video crawler unit 17 have been modified.

In this modified embodiment, instead of the processing operation in step S42 of FIG. 7, the following processes are performed in parallel: as shown in step S42a of FIG. 8, a process of playing back a large number of acquired moving images at high speed and using a machine learning model to extract still images when it is determined that a face is shown; as shown in step S42b of FIG. 8, a process of playing back a large number of acquired moving images at high speed and using a machine learning model to extract still images of a face at the timing when it is determined that a face is most likely to be shown; and as shown in step S42c of FIG. 8, a process of playing back a large number of acquired moving images at high speed and using a machine learning model to extract facial features at the timing when it is determined that a face is most likely to be shown and extract still images of a face.

In addition, instead of the processing operation in step S43 in Figure 7, as shown in step S43a in Figure 8, a large number of acquired moving images are played back at high speed, still images from the entire moving image are extracted at regular frame intervals, and a machine learning model is used for each extracted still image to extract the still image with the highest probability of showing a face.

Then, the multiple still images extracted using the processing procedure of steps S42a to S42c (a processing procedure different from the other steps) and the multiple still images extracted using the processing procedure of step S43a (a processing procedure different from the other steps) are integrated and temporarily stored in local cache memory. In this modified form, still images are extracted from moving images using four different processing procedures, and the still images extracted using these four processing procedures are integrated to extract still images. Because the final image obtained differs depending on the processing procedures, this provides significantly higher reliability than when facial images are extracted using only one procedure.

As explained above, in this embodiment, the video crawler of the video crawler unit 17 acquires video data from websites corresponding to URLs stored in the URL list unit 15b, i.e., websites containing keywords registered as the cause of facial image leaks. In other words, the entire Internet is not searched. This allows for rapid detection of user facial images leaked onto the Internet at an early stage and simplifies the configuration of the image search device. Furthermore, in this embodiment, when multiple still image data are extracted from the acquired video data and image data containing facial information is obtained from the extracted still image data, multiple different processing procedures are used to extract the still image data from the video data, and the still image data extracted by these multiple different processing procedures is integrated. In this way, different processing procedures result in different final images, which significantly increases the reliability of facial image detection compared to when facial images are extracted using only a single procedure. This makes it possible to reliably detect that a user's facial image has been leaked onto the Internet with a simple configuration.

In the above-described embodiment and modified aspects, the image search device of the present invention is constructed using servers, storage, and databases provided by a cloud computing service, but the present invention may also be realized using a dedicated image search device constructed by assembling local servers, storage, and databases without using a cloud computing service.

The above-described embodiments are intended to be illustrative of the present invention and are not limiting, and the present invention can be implemented in a variety of other modified and altered forms. Therefore, the scope of the present invention is defined only by the claims and their equivalents.

It can be used to search for images of users that have been leaked onto the Internet and prevent users from becoming victims of voyeurism or revenge porn.

10 User terminal 11 Image search server 12 Internet 13 Input/output and image comparison unit 13a Database (AWS/RDS)
13b, 16a, 17a Storage (AWS/S3)
14 Cloud server 15 URL crawler unit 15a Cloud server (AWS/EC2)
15b URL list unit 16 Still image crawler unit 16b Image search unit 16c Machine learning unit 17 Video crawler unit 17b Image extraction unit

Claims (16)

The device comprises an input/output and image comparison unit that stores facial image data representing the user's own facial image registered by the user and keywords registered by the user as the cause of the leakage of the facial image, and an image extraction unit that extracts a plurality of still image data from video data acquired from a website that is a sexually explicit site on the Internet and includes the keyword registered by the user, and acquires image data including facial information from the extracted still image data,
the input/output and image comparison unit is configured to compare the image data including the face information acquired by the image extraction unit with the face image data registered by the user and stored in the input/output and image comparison unit, and to extract a URL of the face image data having a high degree of similarity;
The image extraction unit is configured to extract still image data from the video data using a plurality of different processing procedures, and to integrate the still image data extracted by the plurality of different processing procedures. The image search device of claim 1, characterized in that the multiple different processing procedures performed by the image extraction unit include a process of playing back the video data at high speed and extracting still image data determined by a machine learning model to contain a face, and a process of playing back the video data at high speed and extracting still image data at any fixed frame interval. The image search device of claim 1, wherein the multiple different processing procedures performed by the image extraction unit include: a process of playing back the video data at high speed and extracting a still image when a machine learning model determines that a face is shown; a process of playing back the video data at high speed and extracting a still image of a face at the timing determined by the machine learning model that a face is most likely to be shown; a process of playing back the video data at high speed and extracting facial features at the timing determined by the machine learning model that a face is most likely to be shown, thereby extracting a still image of a face; and a process of playing back the video data at high speed, extracting still images from the entire video at regular frame intervals, and extracting, for each extracted still image, a still image that has the highest probability of showing a face according to the machine learning model. The image search device of claim 1, characterized in that the image extraction unit is configured to compare the confidence value of the image data containing the facial information with a threshold value to extract the image data that most closely resembles a facial image. The image search device of claim 4, wherein the image extraction unit is configured to divide the image data containing the facial information into a predetermined number of parts starting from the beginning, select the image data with the highest accuracy among the predetermined number of divided image data, and compare the confidence value of the selected image data with a threshold value. The image search device of claim 1 further comprises a URL crawler unit that collects URLs of websites that are sexually explicit on the Internet and that contain keywords registered by the user, and the image extraction unit is configured to acquire video data from websites that correspond to the URLs collected by the URL crawler unit. The image search device described in claim 1, characterized in that the input/output and image comparison unit is configured to store facial image data and keywords sent from a user terminal, and to send the extracted URL to the user terminal. An image search method comprising: storing face image data representing a user's own face image registered by the user and a keyword registered by the user as a cause of the face image being leaked; extracting a plurality of still image data from video data acquired from a web site on the Internet that is a sexually explicit site and includes the keyword registered by the user; and acquiring image data including face information from the extracted still image data,
The image data including the acquired face information is compared with the face image data registered by the user, and the URL of the face image data having a high degree of similarity is extracted.
An image retrieval method comprising: extracting still image data from moving image data using a plurality of mutually different processing procedures; and integrating the still image data extracted by the plurality of mutually different processing procedures. The image search method of claim 8, wherein the plurality of different processing procedures include a process of playing back the video data at high speed and extracting still image data determined by a machine learning model to contain a face, and a process of playing back the video data at high speed and extracting still image data at any fixed frame interval. The image search method of claim 8, wherein the multiple different processing procedures include: a process of playing back the video data at high speed and extracting a still image when a machine learning model determines that a face is shown; a process of playing back the video data at high speed and extracting a still image of a face at the timing determined by the machine learning model that a face is most likely to be shown; a process of playing back the video data at high speed and extracting facial features at the timing determined by the machine learning model that a face is most likely to be shown, thereby extracting a still image of the face; and a process of playing back the video data at high speed, extracting still images from the entire video at regular frame intervals, and extracting, for each extracted still image, a still image that has the highest probability of showing a face according to the machine learning model. The image search method described in claim 8, characterized in that the confidence value of the image data containing the facial information is compared with a threshold value to extract the image data that most closely shows the facial image. The image search method described in claim 11, characterized in that the image data containing the facial information is divided into a predetermined number of parts starting from the beginning, the best image data is selected from the predetermined number of divided image data parts, and the confidence value of the selected image data is compared with a threshold value. The image search method described in claim 8, characterized in that the URLs of websites on the Internet that are sexually explicit and contain keywords registered by the user are collected, and the video data is obtained from the sites corresponding to the collected URLs. The image search method described in claim 8, characterized in that facial image data and keywords sent from a user terminal are stored, and the extracted URL is sent to the user terminal. Computer,
A program for functioning as an input/output and image comparison means for storing facial image data representing a user's own facial image registered by the user and a keyword registered by the user as the cause of the leakage of the facial image, and an image extraction means for extracting a plurality of still image data from video data acquired from a website on the Internet that is a sexual site and includes a keyword registered by the user, and for acquiring image data including facial information from the extracted still image data,
the input/output and image comparison means is means for comparing image data including the acquired face information with face image data registered by the user, and extracting a URL of face image data having a high degree of similarity;
The image extraction means extracts still image data from moving image data using a plurality of different processing procedures, and integrates the still image data extracted by the plurality of different processing procedures. Computer,
A computer-readable recording medium having recorded thereon a program for functioning as an input/output and image comparison means for storing facial image data representing a user's own facial image registered by the user and a keyword registered by the user as the cause of the leakage of the facial image, and an image extraction means for extracting a plurality of still image data from video data acquired from a website on the Internet that is a sexually explicit site and includes a keyword registered by the user, and for acquiring image data including facial information from the extracted still image data,
the input/output and image comparison means is means for comparing image data including the acquired face information with face image data registered by the user, and extracting a URL of face image data having a high degree of similarity;
A computer-readable recording medium characterized in that the image extraction means is a means for extracting still image data from video data using a plurality of different processing procedures and integrating the still image data extracted by the plurality of different processing procedures.