WO2025154857A1 - System and method for managing de-identified image data of personal information - Google Patents

Abstract

This system for managing de-identified image data of personal information comprises: an image collection device for detecting a personal information area from collected original-image data, and generating de-identified image data by de-identifying the personal information area; and an operating server, which is connected to the image collection device through a communication network, receives the de-identified image data from the image collection device and stores same, and provides same according to a request of the image collection device, wherein the image collection device is an information processing device for collecting image data, and includes: an image collection unit for generating or collecting original-image data; a personal information area detection unit for detecting a personal information area from the original-image data; a de-identification unit for generating de-identified image data by de-identifying the personal information area; a restoration unit for restoring the original-image data from the de-identified image data by using a key value; a control unit for controlling each constituent element of the image collection device; and a communication unit for communicating with external devices through the communication network.

Description

Personal information de-identification video data management system and method

The present invention relates to a method for managing de-identified personal information image data, and more specifically, to a management system and method for de-identified personal information image data and an image restoration method.

Personal information is being used as important data in various fields such as marketing, big data, and artificial intelligence. As the fields in which personal information is used increase, the risk of personal information leakage also increases, so it is protected by the Personal Information Protection Act. In particular, in the case of video data, if it is exploited using deepfake technology, there is a risk of fatal human rights violations for individuals, so special care is required.

Various video collection devices such as CCTV and smartphones can be used to acquire images or video information. Saving the collected video content in the local storage of the video collection device does not violate the Personal Information Protection Act. However, distributing the video content in the local storage device or transmitting and saving it from the collection device to another remote storage device is a violation of the Personal Information Protection Act. Personal information (human faces, car license plates) in the content must be distributed or transmitted after being anonymized so that it cannot be identified.

Accordingly, video data containing personal information is stored in anonymized form in the video collection device at the edge, managed so that only the administrator can access the video and download the data, and provided by anonymizing the requested video when requested from an external device. In cases where video acquired from CCTV in an apartment or kindergarten needs to be exported externally, the administrator (a person with authority) and the anonymization worker are present together, anonymizing the original video from the storage device, and then copying it to a removable storage device before exporting it externally. In the case of another method of storing on a remote server, the streamed video is anonymized, and both the original and the anonymized video are stored and managed.

The above methods have the problem that workers must always work on site and that management costs increase because both original and de-identified images are stored.

Meanwhile, autonomous driving special vehicles refer to agricultural vehicles such as tractors and rice transplanters, or vehicles that can clean indoors and outdoors, and are capable of autonomous driving and operation. Due to the nature of autonomous driving special vehicles, they continuously acquire and collect images, and these images may contain personal information. Images acquired from edge devices such as autonomous driving special vehicles have the characteristic that the camera is constantly operating, so there are limitations to storing them in local storage devices. In other words, the data in the local storage device must be directly deleted or stored in a remote storage, and it must be stored/transmitted after being de-identified. However, since the de-identified image is transmitted instead of the original, and the original must be deleted, there is a problem in that the image cannot be confirmed when the original image is required.

(Patent Document 1) KR 2020-0036656 A

The present invention aims to solve the above problems by providing a method for performing an anonymization operation in real time in an image collection device, storing the anonymized image as an image that can be restored using a related key value, and protecting the image content from illegal access from the outside.

The present invention seeks to provide a method for managing image data cost-effectively while protecting personal information.

The present invention also seeks to provide a method for managing image data that allows a person with legitimate authority to efficiently access and use the image data.

In order to solve the above problem, a personal information de-identified image data management system according to one aspect of the present invention is provided. The de-identified image data management system includes an image collection device that detects a personal information area from collected original image data and performs de-identification processing on the personal information area to generate de-identified image data; and an operation server that is connected to the image collection device through a communication network, receives the de-identified image data from the image collection device, stores it, and provides it upon a request from the image collection device.

The above image collection device is an information processing device that collects image data, and preferably includes an image collection unit that generates or collects original image data, a personal information area detection unit that detects a personal information area in the original image data, a de-identification unit that de-identifies the personal information area to generate de-identified image data, a restoration unit that restores the original image data from the de-identified image data using a key value, a control unit that controls each component of the image collection device, and a communication unit that communicates with external devices through a communication network.

The above image collection device transmits device information and related key values together with the de-identified image data to the operation server, and deletes the de-identified original image data and the de-identified image data.

The above device information is generated based on the MAC address and serial number of the image collection device. The above operation server includes a data storage unit, a control unit, an authentication unit, an input/output interface, and a communication module, and the data storage unit includes a program storage unit and a database. The database stores the device information and related key values together with the de-identified image data. It is preferable that the authentication unit performs authentication using the device information and related key values.

The above image data management system may further include a user device. The user device is an information processing device used by a user who has management or access authority for the image collection device, and provides device information and related key values to the operation server to search for the original image data.

The above operation server executes authentication using the above device information, and when authentication is completed, it receives a specific image search or restoration request including a related key value from the user device, determines whether the key value matches, and if the key value matches, creates a request queue including device information and related key value for the image collection device.

The above image collection device receives a request queue matching the device information, verifies whether the related key value matches, receives the matching de-identified image data from the operation server, and restores the original image data from the de-identified image data using the related key value.

The above device information is the MAC address and serial number of the encrypted image collection device, and the related key value includes the meta file key value of the de-identified image data and the de-identification key value.

The above image collection device may further include an authentication verification unit.

The above authentication verification unit checks whether the relevant key value matches the request queue from the operation server, and if it matches, the control unit downloads the de-identified image data from the operation server and controls the restoration unit to restore the original image data.

The above-detected personal information area information includes coordinates and pixel information of the personal information area for each frame of the original image data, and the coordinates and pixel information of the personal information area are stored in a meta file, and the meta file is encrypted using a meta file key value. The personal information area is de-identified using a shuffling table in a shuffling manner, and it is preferable that the de-identification key value is an index of the shuffling table.

The above operation server further includes a device registration unit that registers device information of one or more image collection devices to which a user has access authority, in response to a request from a user device.

The above device information is generated based on the MAC address and serial number of the image collection device, and is an encrypted value obtained by inputting the value obtained by adding the MAC address and serial number (combineString = Mac address + serial Number) into a hash algorithm.

According to another aspect of the present invention, a method for managing de-identified image data is provided, which is executed by an operating server that receives and stores de-identified image data in which personal information is de-identified from an image collection device.

The above-described de-identified image data management method includes: a step of receiving and storing de-identified image data together with device information and related key values from an image collection device; an authentication step of receiving an authentication request including device information from a user device and executing authentication; a step of receiving a specific image search or restoration request including related key values from the user device when the authentication is completed; and a step of determining whether the related key values match, and if the related key values match, generating a request queue including device information and related key values for the image collection device.

The above-described method for managing de-identified image data may further include a step of providing the de-identified image data to the image collection device in response to a request for de-identified image data based on confirmation of a match between related key values of the image collection device; and a step of receiving restored original image data from the image collection device and providing the restored original image data to the user device.

The above device information is the MAC address and serial number of the encrypted image collection device, and the related key value includes the meta file key value and the de-identification key value of the de-identified image data. The meta file of the de-identified image data, in which the personal information is de-identified using the meta file key value, is encrypted. The meta file includes the coordinates and pixel information of the personal information area for each frame of the original image data.

The above personal information area is preferably anonymized by shuffling using a shuffling table, and the anonymization key value is preferably an index of the shuffling table.

According to one aspect of the present invention, an efficient and privacy-enhanced image data management method is provided.

According to another aspect of the present invention, an image data management method and system that are efficient and have enhanced security are provided, by enabling restoration of de-identified image data in a remote storage using device information and related key values of an image collection device, and enabling restoration of the image from a device that initially acquired the image upon a restoration request from an image owner.

Figure 1 is a schematic structural diagram of the image data management system of the present invention.

FIG. 2 is a block diagram showing the structure of an image collection device according to one embodiment of the present invention.

Figure 3 is a block diagram showing the structure of an operating server according to one embodiment of the present invention.

FIG. 4 is a schematic diagram showing a process of registering advance device information in a video data management method according to one embodiment of the present invention.

FIG. 5 is a schematic diagram showing the sequence and data transmission process of an image data management method according to one embodiment of the present invention.

FIG. 6 is a diagram illustrating a method for de-identifying image data according to one embodiment of the present invention.

Hereinafter, various preferred embodiments of the present invention will be described in detail with reference to the attached drawings so that a person having ordinary skill in the art to which the present invention pertains can easily practice the present invention.

The terms first, second, etc. used in the description of this specification are merely identifiers to distinguish one component from another. When a component is referred to as being "connected" or "connected" to another component in this specification, it should be understood that the component may be directly connected or directly connected to the other component, but unless there is a specific description to the contrary, it should be understood that the component may also be connected or connected via another component in between.

Among the terms used in this specification, "module," "part," "interface," etc. generally mean computer-related objects, and may mean, for example, hardware, software, and a combination thereof. In this specification, an information processing device is a device that includes a processor, a memory, and a communication module to process data and communicate with an external device, and may be a computer, a PC, a set-top box, a smartphone, etc.

In general, video image data acquired from cameras, etc. are de-identified and then transmitted to other devices, including external servers, or stored and managed in memory. Video data that has undergone personal information area detection and de-identification processing for each frame is encrypted and encoded and then stored, managed, utilized, and transmitted. The targets of de-identification processing are personal information areas such as faces and vehicle license plates, and the detection of personal information areas is performed by various algorithms such as artificial intelligence learning models, face recognition engines, or number recognition engines. Meanwhile, when personal information areas such as human faces or vehicle license plates are detected, de-identification processing is performed by the processor, and can be performed in various ways such as masking, scrambling, blurring, categorization, replacement, and deletion.

Referring to FIG. 1, an image data management system according to a preferred embodiment of the present invention includes an image collection device (100), a user device (200), and an operation server (300) that are connected to each other via a communication network.

The image collection device (100) is an information processing device that collects image data, and in terms of hardware, includes a processor, memory, input/output interface, and communication module. The image collection device (100) may include an image acquisition device such as a camera, but is not limited thereto, and may be a device that is connected to a separate image acquisition device and collects image data from it. For example, the image collection device may be a smartphone, a set-top box that is connected to a CCTV and collects image data acquired from the CCTV, or an ECU system of an autonomous vehicle.

Referring to FIG. 2, the structure of the image collection device (100) includes an image collection unit, a personal information area detection unit, an anonymization unit, a restoration unit, an authentication confirmation unit, a control unit, and a communication unit, and each of the above components can be implemented by software and/or hardware.

The image collection unit collects image data, and may be a camera, but is not limited thereto, and may also be a component that collects and stores image data from a separate device. The personal information area detection unit detects a personal information area, which is an anonymized area, from the collected image data, and may include various algorithms such as an artificial intelligence learning model, a face recognition engine, or a number recognition engine. The detected personal information area information (coordinates of the personal information area for each frame, pixel information, etc.) is stored in a metafile, and the metafile is encrypted using a metafile key value.

The metafile key value and the de-identification key value can be specified by the user who is the image data manager. It is preferable that the security key be input or specified by the user in advance when the original image data is created, produced or acquired, but it is not limited thereto and may be automatically generated or input during the de-identification process and the user may obtain it.

The de-identification unit performs de-identification processing on the personal information area of the image data detected by the personal information area detection unit using the de-identification key value. However, the de-identified image data can be restored using the de-identification key value. According to one embodiment of the present invention, de-identification processing can be performed using a shuffling method in which the personal information area is divided into predetermined shuffle areas as shown in FIG. 6 and the pixel values in the shuffle areas are shuffled using a shuffle table to create a fake copy. At this time, the selected shuffle table is extracted with an index calculated using the de-identification key value. In other words, an index is calculated using the de-identification key value, and shuffling is performed by a shuffle table according to the index. The shuffling method will be described in detail later.

The authentication verification unit checks for authentication and key value matching for the request queue from the operation server, and if there is a match, the control unit downloads the de-identified image data from the operation server and causes the restoration unit to restore the original image data. The communication unit communicates with external devices through a communication network.

The user device (200) may be an information processing device used by an administrator or user who has access and processing authority for image data of the image collection device (100).

The operation server (300) is connected to one or more image collection devices (100) and one or more user devices (200) through a communication network, and may be an information processing device that processes and stores data from one or more image collection devices (100) and one or more user devices (200) and manages access to and use of the image collection devices (100) by the user devices (200).

According to FIG. 3, an operation server (300) according to one embodiment of the present invention includes a data storage unit, a control unit, an authentication unit, an input/output interface, a communication module, and a device registration unit. The data storage unit includes a program file storage unit and a database, and the database stores de-identified image data, device information related thereto, and key values.

The above device registration section registers device information of an image collection device (100) to which a user has access rights upon a request from a user device. The above device information is generated based on the Mac address and serial number of the image collection device (100). The above device information may be, for example, a value obtained by adding the MAC address and serial number of the image collection device (100) (combineString = Mac address + serial Number) to a hash algorithm and encrypting the value (licenseString = hash algorithm (combinString)), but is not limited thereto.

The authentication unit performs authentication processing based on the device information in response to an authentication request from a user device.

Hereinafter, an image data management method according to one embodiment of the present invention will be described in detail with reference to FIGS. 4 and 5.

Referring to Figure 4, a user uses a user authentication service to authenticate himself/herself and then proceeds with the management service subscription procedure in order to use the video data management service provided by the operating server through the user device. This user authentication and subscription procedure is the same as the general user authentication and membership registration procedure, so a detailed description is omitted.

Thereafter, the user proceeds with the procedure of registering the management device (image collection device) for which he/she has management authority. When registering the management device, the MAC address and serial number of the image collection device are registered so that the device can be distinguished. However, the MAC address and serial number are encrypted in the user device and transmitted to the operation server. The encryption is performed by the hash algorithm as described above. If there are multiple image collection devices, which are devices for which he/she has management authority, the user can proceed with the registration procedure for each image collection device. That is, the MAC address and serial number of each image collection device can be encrypted and a request for device registration can be made to the operation server. The device registration section of the operation server stores the device information of the image collection device (100) for which the user has access/management authority according to the registration request from the user device. The key value can be composed of 16-digit numbers + uppercase/lowercase alphabets using an advanced standard encryption technique based on the AES (Advanced Encryption Standard) algorithm.

Referring to FIG. 5 below, a method for managing image data according to one embodiment of the present invention will be described in detail.

The image collection device (100) detects a personal information area in the collected image and performs de-identification processing to generate de-identified image data. The generated de-identified image data, along with device information (encrypted MAC address and serial number of the image collection device) and related key values (meta file key value and de-identification key value), are transmitted to the operation server. Since the image collection device (100) continuously collects image data, it has a limit on the storage capacity at the edge. Therefore, if necessary, both the de-identified image data and the original image data can be deleted and only the related key value and the de-identified image data (or original image data) unique number can be stored. New image data is continuously collected to extract a personal information area, de-identify it, and transmit it to the operation server.

According to FIG. 5, the operation server receives de-identified image data and device information (encrypted MAC address and serial number of the image collection device) and related key values (meta file key value and de-identified key value) from the image collection device and stores them as structured data in the data storage unit.

Meanwhile, a user with legitimate authority over a specific video collection device can access the original video collected by the video collection device through the user device or request the original video data. In other words, the user can request video viewing or restoration from the operation server through the user device. In this case, an authentication procedure is first performed. In this authentication procedure, the user device transmits authentication information including device information to the operation device, and the authentication unit of the operation server performs authentication using the authentication information. In this authentication step, the authentication unit first checks whether there is device information in the database that matches the device information transmitted from the user device, i.e., the MAC address and serial number of the encrypted video collection device. If there is matching device information, it secondarily checks whether the user device is a user device that has registered the device information stored in the device registration unit. However, this second verification is not mandatory and has the effect of further strengthening security as an option. This is because the management authority is recognized only when the user device has previously registered the device information of the video collection device managed by the user.

When authentication is completed, the operation server receives a specific image search or restoration request including the relevant key value from the user device. The authentication unit of the operation server determines whether the relevant key value matches, and if the relevant key value matches, it generates a request queue including device information and the relevant key value for the image collection device. That is, it checks whether the key value related to the specific de-identified image data received from the user device matches the relevant key value stored in the database, and if they match, it generates a request queue, and if they do not match, it sends a message to the user device rejecting the specific image search or restoration request.

The above related key value includes a meta file key value of the de-identified image data and a de-identification key value. Here, the meta file key value is a key value used by the image collection device to encrypt the meta file of the de-identified image data. The meta file includes personal information area information (coordinates and pixel information of the personal information area for each frame of the original image data) detected by the image collection device. The de-identification key value is a de-identification key value used by the de-identification unit of the image collection device to de-identify the personal information area by a shuffling method.

For the above-mentioned generated request queue, the image collection device receives only request queues whose device information () matches, and the authentication confirmation unit re-verifies the related key value included in the request queue. That is, the related key value included in the request queue and the related key value matching the unique number of the image data and the unique number of the de-identified image data (or the original image data) are checked. If a complete match is confirmed, the control unit requests the corresponding de-identified image data to the operation server and downloads it, and if they do not match, the mismatch is notified to the operation server again, and the restoration of the original image data is stopped. Since the restoration work is performed only in the image collection device, and the related key value is double-checked in the operation server and the image collection device, it is easy to improve security more than when the image data is restored in the operation server.

When the relevant key value is reconfirmed in the authentication confirmation unit of the image collection device, the control unit receives the de-identified image data from the operation server and controls the restoration of the original image data using the relevant key value by the restoration unit. The original image data restored by the image collection device is transmitted to the user device that requested the image search or restoration through the operation server. Therefore, a user with management authority can easily search or obtain the original image data remotely.

Below, anonymization and restoration using the shuffling method are explained with reference to Fig. 6. The shuffling method is an anonymization processing method that creates fake copies using a shuffling table.

The shuffling method includes a step of calculating a shuffle index using a de-identified key value, a step of selecting a shuffle table corresponding to the shuffle index, a shuffling step of shuffling the personal information area using the selected shuffle table, and a step of encrypting de-identified information including coordinates and the number of pixels of the personal information area using a meta file key value and storing the encrypting information as a meta file.

The method for restoring the above-described de-identified image data includes a step of decrypting a metafile of the de-identified image data using a metafile key value, a step of obtaining personal information region information from the decrypted metafile, a step of separating a personal information region of the de-identified image data using the personal information region information, a step of restoring the separated personal information region using a de-identification key value, and a step of generating original image data by combining the restored personal information region with the de-identified image data.

However, the anonymization processing of the present invention is not limited to a shuffling method using a shuffling table, and can be performed in various ways, such as other restorable methods, such as masking and blurring.

The steps of a method or algorithm described in connection with the embodiments of the present invention may be implemented directly in hardware, implemented in software executed by hardware, or implemented by a combination of these. The software may reside in a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), a Flash Memory, a hard disk, a removable disk, a CD-ROM, or any other form of computer-readable recording medium well known in the art to which the present invention pertains.

Above, while the embodiments of the present invention have been described with reference to the attached drawings, it will be understood by those skilled in the art that the present invention may be implemented in other specific forms without changing the technical idea or essential features thereof. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

The present invention relates to an image data management system and method, and is applicable to the information and communication industry, and therefore has industrial applicability.

Claims (10)

As a video data management system, An image collection device that detects a personal information area from collected original image data and performs anonymization processing on the personal information area to generate anonymized image data; and An operation server, which is connected to the image collection device through a communication network, receives and stores the de-identified image data from the image collection device, and provides it upon request from the image collection device; The above image collection device is an information processing device that collects image data. An image collection unit that generates or collects original image data; A personal information area detection unit that detects personal information areas in the above original image data, A de-identification unit that de-identifies the above personal information area to create de-identified image data. A restoration unit that restores original image data from de-identified image data using key values. A control unit that controls each component of the above image collection device, and An image data management system characterized by including a communication unit that communicates with external devices through a communication network. In the first paragraph, The above image collection device transmits the device information and related key values together with the de-identified image data to the operation server, and deletes the de-identified original image data and the de-identified image data. The above device information is generated based on the MAC address and serial number of the image collection device. The above operation server includes a data storage unit, a control unit, an authentication unit, an input/output interface, and a communication module, and the data storage unit includes a program storage unit and a database, and the database stores the device information and related key values together with the de-identified image data. An image data management system characterized in that the authentication unit performs authentication using the above device information and related key values. In paragraph 1 or 2, The above image data management system further includes a user device, The above user device is an information processing device used by a user who has management or access rights to the image collection device, and provides device information and related key values to the operation server to search the original image data. The above operation server executes authentication using the above device information, and when authentication is completed, it receives a specific image search or restoration request including a related key value from the user device and determines whether the key value matches, and if the key value matches, it creates a request queue including device information and related key value for the image collection device, An image data management system characterized in that the image collection device receives a request queue matching the device information, checks whether the related key value matches, receives the matching de-identified image data from the operation server, and restores the original image data from the de-identified image data using the related key value. In the second or third paragraph, The above device information is the MAC address and serial number of the encrypted video collection device. The above related key values include the meta file key value of the de-identified image data and the de-identified key value, The above image collection device further includes an authentication verification unit, A video data management system characterized in that the above authentication verification unit checks whether the related key value matches the request queue from the operation server, and if it matches, the control unit downloads the de-identified video data from the operation server and controls the restoration unit to restore the original video data. In any one of claims 1 to 4, The above-detected personal information area information includes coordinates and pixel information of the personal information area for each frame of the original image data, and the coordinates and pixel information of the personal information area are stored in a meta file, and the meta file is encrypted using a meta file key value. An image data management system, characterized in that the above personal information area is anonymized in a shuffling manner using a shuffling table, and the anonymization key value is an index of the shuffling table. In any one of the provisions of paragraphs 2 to 5, The above operation server further includes a device registration section for registering device information of each video collection device to which the user has access rights according to a request from the user device. An image data management system characterized in that the above device information is generated based on the MAC address and serial number of the image collection device, and is an encrypted value obtained by inputting the value obtained by adding the MAC address and the serial number (combineString=Mac address+serial Number) into a hash algorithm. A method for managing de-identified image data executed by an operating server that receives and stores de-identified image data from an image collection device, A step of receiving and storing device information and related key values together with de-identified image data from an image collection device; An authentication step that receives an authentication request including device information from a user device and executes authentication; Upon completion of the above authentication, a step of receiving a specific image search or restoration request including a relevant key value from the user device; and A method for managing de-identified image data, comprising: a step of determining whether the related key values match, and if the related key values match, generating a request queue including device information and related key values for the image collection device. In Article 7, A step of providing the de-identified image data to the image collection device in response to a request for de-identified image data based on confirmation of the match between the relevant key values of the image collection device; and A method for managing de-identified image data, comprising: a step of receiving original image data restored from the image collection device and providing the same to the user device. In Article 8, The above device information is the MAC address and serial number of the encrypted video collection device. A method for managing de-identified image data, characterized in that the above-mentioned related key value includes a meta file key value of de-identified image data and a de-identified key value. In Article 9, The meta file of the de-identified image data, which de-identifies the personal information using the above meta file key value, is encrypted, and the meta file includes coordinates and pixel information of the personal information area for each frame of the original image data, A method for managing de-identified image data, characterized in that the above personal information area is de-identified in a shuffling manner using a shuffling table, and the de-identified key value is an index of the shuffling table.