CN117278342B - Multi-environment Hadoop KMS proxy service method and system - Google Patents
Multi-environment Hadoop KMS proxy service method and system Download PDFInfo
- Publication number
- CN117278342B CN117278342B CN202311572773.2A CN202311572773A CN117278342B CN 117278342 B CN117278342 B CN 117278342B CN 202311572773 A CN202311572773 A CN 202311572773A CN 117278342 B CN117278342 B CN 117278342B
- Authority
- CN
- China
- Prior art keywords
- kms
- module
- edek
- hadoop
- party
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/06—Network architectures or network communication protocols for network security for supporting key management in a packet data network
- H04L63/062—Network architectures or network communication protocols for network security for supporting key management in a packet data network for key distribution, e.g. centrally by trusted party
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/08—Key distribution or management, e.g. generation, sharing or updating, of cryptographic keys or passwords
- H04L9/0816—Key establishment, i.e. cryptographic processes or cryptographic protocols whereby a shared secret becomes available to two or more parties, for subsequent use
- H04L9/0819—Key transport or distribution, i.e. key establishment techniques where one party creates or otherwise obtains a secret value, and securely transfers it to the other(s)
- H04L9/0822—Key transport or distribution, i.e. key establishment techniques where one party creates or otherwise obtains a secret value, and securely transfers it to the other(s) using key encryption key
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L2209/00—Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
- H04L2209/76—Proxy, i.e. using intermediary entity to perform cryptographic operations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L2463/00—Additional details relating to network architectures or network communication protocols for network security covered by H04L63/00
- H04L2463/062—Additional details relating to network architectures or network communication protocols for network security covered by H04L63/00 applying encryption of the keys
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Storage Device Security (AREA)
Abstract
The invention provides a multi-environment Hadoop KMS proxy service method and system, comprising a service cluster, wherein the service cluster comprises a data access module, a data management module and a Ranger KMS proxy module, and the service cluster is used for data management and key proxy management; and the third-party Hadoop KMS is used for key management. The method for the multi-environment Hadoop KMS proxy service is also provided, file data can be encrypted or decrypted in a KMS proxy mode, and data safety is guaranteed. The invention provides a multi-environment Hadoop KMS proxy service method and system, which protect the security of a secret key through KMS proxy service in the process of transparent encryption or decryption of an HDFS file, thereby guaranteeing the security of data.
Description
Technical Field
The invention relates to the technical field of information security, in particular to a multi-environment Hadoop KMS proxy service method and system.
Background
HDFS is a distributed file system used by Hadoop that can store and process large-scale data. The design goal of HDFS is to run on standard hardware, providing high fault tolerance, and to be able to handle large amounts of data that are already stored. The data in the HDFS is stored in a local disk, typically in the clear, and it is difficult to prevent users from accessing the data that is involved. The KMS key management service is a one-stop key management and data encryption service platform and can provide simple, safe and reliable and compliant data encryption protection capability. By means of an HDFS transparent encryption mechanism realized by the KMS, key data or some special files can be encrypted, and only encrypted ciphertext data is stored in a DataNode of an HDFS layer, so that even if an illegal user copies the files from the level of an operating system, the illegal user can only see a string of encrypted bytes and cannot check the real content before the files are encrypted, and the confidentiality and the security of data resources are improved to a great extent. KMS key management services are typically integrated in Hadoop clusters and can be used once properly configured and booted. KMS is responsible for key management work of the whole HDFS transparent decryption process, so the saved keys are crucial for confidentiality of user data and file resources. In the current configuration, KMS services are deployed in the same cluster as HDFS, and clients encrypt and decrypt files of the KMS services by connecting to KMS services built in the HDFS local cluster. However, once an illegitimate user has obtained the rights or highest rights of the local cluster, the client's key may be exposed to a risk of leakage. Although the data has been encrypted by the key, there is still a potential risk that the key itself may be obtained improperly, resulting in a threat to the security of the data.
In the prior art, as a key management method for transparent encryption and decryption of a Hadoop distributed file system disclosed in a chinese published patent CN108111479a, a load of a key management protocol of a KMS is changed into a load which is lighter and higher in efficiency, but the security of a client key is not enhanced by the method. The security enhancement method for Hadoop key management service based on SGX disclosed in China patent publication No. CN109981579B enhances the security of key management by adding an SGX security zone, but the method is only aimed at the security enhancement of a KMS in the process of creating encryption zone keys, data encryption keys and encryption and decryption of encryption data encryption keys and cannot solve the security requirement scene mentioned in the application.
Disclosure of Invention
The invention aims to provide a multi-environment Hadoop KMS proxy service method and system, and the security of a client key is improved in a KMS proxy service mode.
The multi-environment Hadoop KMS proxy service system comprises an encryption method and a decryption method, and is used,
the encryption method comprises the following steps:
a1, a data management module sends an EDEK generation request to a Ranger KMS proxy module;
a2, the Ranger KMS agent module agents the request for accessing the EZ Key to a third party Hadoop KMS to acquire the EZ Key;
a3, the Ranger KMS agent module generates a new EDEK request agent to the third-party Hadoop KMS, and the third-party Hadoop KMS generates the new EDEK and returns the new EDEK to the Ranger KMS agent module;
a4, the Ranger KMS agent module stores the EDEK in a NameNode of the data management module.
Preferably, the decryption method includes the steps of:
b1, the data access module reads EDEK from NameNode of the data management module;
b2, the data access module sends a request for decrypting the EDEK to the Ranger KMS proxy module;
b3, the Ranger KMS agent module agents the request for accessing the EZ Key to a third party Hadoop KMS to acquire the EZ Key;
b4, the Ranger KMS agent module agents the EDEK decrypting request to a third-party Hadoop KMS, and the third-party Hadoop KMS decrypts the EDK into DEK and returns the DEK to the Ranger KMS agent module;
and B5, returning the decrypted DEK to the data access module by the Ranger KMS proxy module.
Also provided is a multi-environment Hadoop KMS proxy service system, comprising:
the service cluster comprises a data access module, a data management module and a Ranger KMS proxy module, and is used for data management and key proxy management;
and the third-party Hadoop KMS is used for key management.
Preferably, the data access module comprises an EZ Key management module, a DEK acquisition module and a data file encryption and decryption module, and is used for ensuring that data is protected in the storage and transmission processes and preventing unauthorized access and data leakage.
Further preferably, the ezkey management module may add, delete, alter or query the ezkey and the mapping relationship between the ezkey and KMS. The ezkey is an encryption area Key used to encrypt the data file.
Further preferably, the DEK obtaining module may send the request of the EDEK to be decrypted to the range KMS proxy module according to the mapping relationship between the EZ Key and the KMS, and receive the decrypted DEK from the range KMS proxy module. The DEK is a data encryption Key, the EDEK is an encrypted data encryption Key, and the EDEK can be obtained through combination of the DEK and the EZ Key.
Further preferably, the data file encryption and decryption module performs transparent encryption or transparent decryption operation on the data file, and the transparent encryption operation ensures that the data is kept in an encrypted state in the transmission and storage processes, so that a user or an application program can correctly process the data.
Preferably, the data management module includes a NameNode for metadata management, maintaining file encryption status and obtaining the EDEK, and a DataNode for storing actual plaintext or ciphertext data blocks.
Preferably, the NameNode comprises an EDEK acquisition module, a metadata management module and a transparent processing module.
Further preferably, the EDEK acquisition module may communicate with the range KMS proxy module, send a request for acquiring the EDEK to the range KMS proxy module according to the mapping relationship between the EZ Key and the KMS, and receive the EDEK returned by the range KMS proxy module.
Further preferably, the metadata management module may record metadata information such as encrypted files or directories and used EDEKs.
Further preferably, the transparent processing module may process details of encryption and decryption such that the encryption and decryption operations are insensitive to HDFS users and applications, and the process of data encryption and decryption does not affect the user's operations.
Preferably, the range KMS proxy module includes a third party Hadoop KMS access module and an EDEK proxy module, and the range KMS proxy module is used for EZ Key proxy management.
Further preferably, the third party Hadoop KMS access module is configured to access the third party Hadoop KMS, proxy the HTTP request for accessing the EZ Key to the third party Hadoop KMS, and return the EZ Key from the third party Hadoop KMS.
Further preferably, the EDEK proxy module is configured to proxy a request for generating or decrypting the EDEK to a third party Hadoop KMS, and the third party Hadoop KMS returns the generated or decrypted DEK.
Preferably, the third-party Hadoop KMS is an independent third-party self-built KMS module or a third-party built-in KMS module built in a third-party cluster.
Preferably, the third-party self-built KMS module includes an EZ Key access module and an EDEK generation module, and the third-party self-built KMS module is used for the EZ Key access module, the EDEK generation module and the EDEK decryption module.
Further preferably, the ezkey access module is configured to access the stored ezkey.
Further preferably, the EDEK generating module is configured to generate the EDEK.
Further preferably, the EDEK decryption module is configured to decrypt the EDEK to generate the DEK.
Preferably, the third party built-in KMS module includes an EZ Key access module, an EDEK generation module and an EDEK decryption module.
Further preferably, the ezkey access module is configured to access the stored ezkey.
Further preferably, the EDEK generating module is configured to generate the EDEK.
Further preferably, the EDEK decryption module is configured to decrypt the EDEK to generate the DEK.
The beneficial effects are that:
1. the invention provides a multi-environment Hadoop KMS proxy service method and system, which protect the security of a secret key through KMS proxy service in the process of transparent encryption or decryption of an HDFS file, thereby guaranteeing the security of data.
2. In the prior art, KMS service and HDFS are deployed in the same cluster, and a client encrypts and decrypts a file thereof by connecting to KMS service built in the HDFS local cluster. But once an illegitimate user has obtained the rights or highest rights of the local cluster, the client's key may be exposed to a risk of leakage. According to the invention, the security of the stored data can be improved while the transparent encryption or decryption of the HDFS can be realized. The local service cluster does not store the secret key, the secret key of the client is stored in the client hand, but not stored locally, the KMS in the service cluster is only used as a proxy, the highest authority of the service cluster can not be obtained in time, the secret key can not be stolen, the secret key safety is protected, and therefore the data safety is guaranteed.
3. The method has various use scenes, can be applied to an independent third-party self-built KMS module or a third-party built-in KMS module built in a third-party cluster, and has wider applicability.
4. The data access module may ensure that data is protected during storage and transmission to prevent unauthorized access and leakage of data.
5. The NameNode in the data management module can process the details of encryption and decryption, so that the encryption and decryption operations are insensitive to HDFS users and application programs.
Drawings
Fig. 1 is a system structure diagram of a service cluster in embodiment 1 of a multi-environment Hadoop KMS proxy service system provided by the present invention.
Fig. 2 is a system structure diagram of a third party self-built KMS module in embodiment 1 of the multi-environment Hadoop KMS proxy service system provided by the present invention.
Fig. 3 is a system structure diagram of a third party cluster in embodiment 2 of a multi-environment Hadoop KMS proxy service system provided by the present invention.
Fig. 4 is a flowchart of a method of embodiment 1 of a multi-environment Hadoop KMS proxy service method provided by the present invention.
Fig. 5 is a flowchart of a method of embodiment 2 of a multi-environment Hadoop KMS proxy service method provided by the present invention.
Detailed Description
The contents of the present invention can be more easily understood by referring to the following detailed description of preferred embodiments of the present invention and examples included. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. In case of conflict, the present specification, definitions, will control.
Example 1:
the invention provides a multi-environment Hadoop KMS proxy service method and system, which protect the security of a secret key through KMS proxy service in the process of transparent encryption or decryption of an HDFS file, thereby guaranteeing the security of data.
The invention provides a multi-environment Hadoop KMS proxy service system as shown in figures 1-2, which comprises:
the service cluster comprises a data access module, a data management module and a Ranger KMS proxy module, and is used for data management and key proxy management; and the third-party Hadoop KMS is used for key management.
The data access module comprises an EZ Key management module, a DEK acquisition module and a data file encryption and decryption module, and is used for ensuring that data is protected in the storage and transmission processes and preventing unauthorized access and data leakage. The EZ Key management module can add, delete, change or inquire the EZ Key and the mapping relation between the EZ Key and the KMS. The ezkey is an encryption area Key used to encrypt the data file. The DEK acquisition module can send the EDEK request to be decrypted to the Ranger KMS proxy module according to the mapping relation between the EZ Key and the KMS, and receive the decrypted DEK from the Ranger KMS proxy module. The DEK is a data encryption Key, the EDEK is an encrypted data encryption Key, and the EDEK can be obtained through combination of the DEK and the EZ Key. The data file encryption and decryption module performs transparent encryption or transparent decryption operation on the data file, the transparent encryption operation ensures that the data is kept in an encrypted state in the transmission and storage processes, and the data decryption operation enables a user or an application program to process the data correctly.
The data management module comprises NameNode and DataNode, nameNode for metadata management, file encryption state maintenance and EDEK acquisition, and data Node for storing actual plaintext or ciphertext data blocks. The NameNode comprises an EDEK acquisition module, a metadata management module and a transparent processing module. The EDEK acquisition module can be communicated with the Ranger KMS proxy module, and sends an EDEK acquisition request to the Ranger KMS proxy module according to the mapping relation between the EZ Key and the KMS and receives the EDEK returned by the Ranger KMS proxy module. The metadata management module may record metadata information such as encrypted files or directories and used EDEKs. The transparent processing module may process the details of the encryption and decryption, making the encryption and decryption operations insensitive to HDFS users and applications.
The Ranger KMS proxy module comprises a third-party Hadoop KMS access module and an EDEK proxy module, and is used for EZ Key proxy management. The third-party Hadoop KMS access module is used for accessing the third-party Hadoop KMS, and for proxy of HTTP requests for accessing the EZ Key to the third-party Hadoop KMS, and the third-party Hadoop KMS returns the EZ Key. The EDEK proxy module is used for proxy of the EDEK or decryption request to the third-party Hadoop KMS, and the third-party Hadoop KMS returns the generated EDEK or decryption DEK.
The third-party self-built KMS module comprises an EZ Key access module, an EDEK generation module and an EDEK decryption module, wherein the EZ Key access module is used for accessing the EZ Key. The EDEK generation module is used for generating the EDEK. The EDEK decryption module is used for decrypting the EDEK to generate DEK. The third party self-built KMS module is used as a self-built independent service module, the EZ Key is stored in a client, the safety of the EZ Key is improved, the volume is small, and the construction is convenient.
As shown in fig. 4, there is also provided a multi-environment Hadoop KMS proxy service method, including an encryption method and a decryption method, a multi-environment Hadoop KMS proxy service system used,
the encryption method comprises the following steps:
a1, a data management module sends an EDEK generation request to a Ranger KMS proxy module;
the EDEK acquisition module in the NameNode in the data management module sends a request for acquiring the EDEK to the Ranger KMS proxy module;
a2, the Ranger KMS agent module agents the request for accessing the EZ Key to a third party Hadoop KMS to acquire the EZ Key;
a third-party Hadoop KMS access module in the Ranger KMS agent module sends an access request to a third-party self-built KMS module, and an EZ Key access module in the third-party self-built KMS module acquires an EZ Key;
a3, the Ranger KMS agent module generates a new EDEK request agent to the third-party Hadoop KMS, and the third-party Hadoop KMS generates the new EDEK and returns the new EDEK to the Ranger KMS agent module;
the EDEK agent module in the range KMS agent module provides an EDEK generation request, the third-party Hadoop KMS access module sends the EDEK generation request to the third-party self-built KMS module, and the EDEK generation module in the third-party self-built KMS module generates EDEK and feeds the EDEK back to the range KMS agent module;
a4, storing the EDEK in a NameNode of the data management module by the Ranger KMS proxy module;
the metadata management module of NameNode manages metadata of EDEK; the transparent processing module conceals the encrypted details.
The decryption method comprises the following steps:
b1, the data access module reads EDEK from NameNode of the data management module;
b2, the data access module sends a request for decrypting the EDEK to the Ranger KMS proxy module;
the DEK acquisition module in the data access module sends a DEK acquisition request to the Ranger KMS proxy module
B3, the Ranger KMS agent module agents the request for accessing the EZ Key to a third party Hadoop KMS to acquire the EZ Key;
a third-party Hadoop KMS access module in the Ranger KMS agent module sends an access request to a third-party self-built KMS module, and an EZ Key access module in the third-party self-built KMS module acquires an EZ Key;
b4, the Ranger KMS agent module agents the EDEK decrypting request to a third-party Hadoop KMS, and the third-party Hadoop KMS decrypts the EDK into DEK and returns the DEK to the Ranger KMS agent module;
the EDEK agent module in the range KMS agent module provides an EDEK decryption request, the third-party Hadoop KMS access module sends the EDEK decryption request to the third-party self-built KMS module, and the EDEK decryption module in the third-party self-built KMS module generates DEK and feeds the DEK back to the range KMS agent module;
b5, the Ranger KMS agent module returns the decrypted DEK to the data access module
The data file decryption module in the data access module converts the ciphertext data block into a plaintext data block by using the DEK, and a user can normally use the data file.
Example 2
Embodiment 2 differs from embodiment 1 in that the third party Hadoop KMS is a third party built-in KMS module built into the third party cluster.
The third-party built-in KMS module comprises an EZ Key access module, an EDEK generation module and an EDEK decryption module. The EZ Key access module is used for accessing the stored EZ Key. The EDEK generation module is used for generating the EDEK. The EDEK decryption module is used for decrypting the EDEK to generate DEK. The third party cluster is an independent cluster of the client, and the built-in KMS module of the third party is built in the third party cluster, so that the client can use the inside conveniently.
As shown in fig. 3 and fig. 5, the third party cluster is provided with the same data access module and data management module as the service cluster, and besides the encryption method in embodiment 1, the data file may be encrypted in the third party cluster by a KMS module built in the third party, the encryption method is the same as the conventional KMS encryption method, the encrypted data file is stored in the data management module in the third party cluster, and then the encrypted data file is copied to the DataNode of the data management module in the service cluster for encryption storage in a manner of DISTCP or uploading, and the EDEK corresponding to the encrypted data file is stored in the NameNode. The decryption method is the same as in example 1.
Claims (8)
1. The multi-environment Hadoop KMS proxy service method comprises an encryption method and a decryption method, and is characterized in that,
the encryption method comprises the following steps:
a1, a data management module sends an EDEK generation request to a Ranger KMS proxy module;
a2, the Ranger KMS agent module agents the request for accessing the EZ Key to a third party Hadoop KMS so that the third party Hadoop KMS can acquire the EZ Key;
a3, the Ranger KMS agent module generates a new EDEK request agent to the third-party Hadoop KMS, and the third-party Hadoop KMS generates the new EDEK and returns the new EDEK to the Ranger KMS agent module;
a4, storing the EDEK in a data management module by using a Ranger KMS proxy module;
the decryption method comprises the following steps:
b1, the data access module reads the EDEK from the data management module;
b2, the data access module sends a request for decrypting the EDEK to the Ranger KMS proxy module;
b3, the Ranger KMS agent module agents the request for accessing the EZ Key to a third-party Hadoop KMS so that the third-party Hadoop KMS can acquire the EZ Key;
b4, the Ranger KMS agent module agents the EDEK decrypting request to a third-party Hadoop KMS, and the third-party Hadoop KMS decrypts the EDK into DEK and returns the DEK to the Ranger KMS agent module;
b5, returning the decrypted DEK to the data access module by the Ranger KMS agent module;
the data management module, the data access module and the Ranger KMS agent module are deployed in the same service cluster, and the data management module is used for managing and storing file data; the data access module is used for ensuring that data is protected in the storage and transmission processes and preventing unauthorized access and data leakage; the Ranger KMS agent module is used for EZ Key agent management.
2. A multi-environment Hadoop KMS proxy service system, comprising:
the service cluster is used for data management and key agent management;
the service cluster comprises a data access module, a data management module and a Ranger KMS proxy module, wherein the data access module is used for ensuring that data is protected in the storage and transmission processes and preventing unauthorized access and data leakage; the data management module is used for managing and storing file data, and the Ranger KMS proxy module is used for EZ Key proxy management; the Ranger KMS proxy module comprises a third-party Hadoop KMS access module and an EDEK proxy module;
the third-party Hadoop KMS is used for EZ Key management; the EZ Key agent management comprises accessing a third party Hadoop KMS through a third party Hadoop KMS access module, and enabling HTTP requests for accessing the EZ Key to be agents to the third party Hadoop KMS, and obtaining the EZ Key through the third party Hadoop KMS; and the EDEK agent module is used for agent-transmitting the EDEK or decryption request to a third-party Hadoop KMS, and the third-party Hadoop KMS returns the generated EDEK or decryption DEK.
3. The multi-environment Hadoop KMS proxy service system according to claim 2, wherein the data access module comprises a DEK acquisition module and a data file encryption and decryption module; the DEK acquisition module is used for sending the EDEK request to be decrypted to the range KMS proxy module according to the mapping relation between the EZ Key and the KMS, and receiving the decrypted DEK from the range KMS proxy module; the data file encryption and decryption module performs transparent encryption or transparent decryption operation on the data file.
4. A multi-environment Hadoop KMS proxy service system as claimed in claim 3, wherein said data management module comprises a NameNode for metadata management, maintaining file encryption status and EDEK acquisition, and a DataNode for storing actual plaintext or ciphertext data blocks.
5. The multi-environment Hadoop KMS proxy service system of claim 4, wherein said NameNode comprises an EDEK acquisition module, a metadata management module and a transparent processing module; the EDEK acquisition module is used for communicating with the Ranger KMS proxy module, sending a request for acquiring the EDEK to the Ranger KMS proxy module according to the mapping relation between the EZ Key and the KMS, and receiving the EDEK returned by the Ranger KMS proxy module; the metadata management module is used for recording the encrypted files or catalogues and metadata information such as EDEK used; the transparent processing module is used for processing the details of encryption and decryption.
6. The multi-environment Hadoop KMS proxy service system of claim 3, wherein the third party Hadoop KMS is an independent third party self-built KMS module or a third party built-in KMS module built into a third party cluster.
7. The multi-environment Hadoop KMS proxy service system of claim 6, wherein the third party self-built KMS module comprises an EZ Key access module, an EDEK generation module and an EDEK decryption module; the EZ Key access module is used for accessing the EZ Key; the EDEK generation module is used for generating EDEK; the EDEK decryption module is used for decrypting the EDEK to generate DEK.
8. The multi-environment Hadoop KMS proxy service system of claim 7, wherein the third party built-in KMS module comprises an EZ Key access module, an EDEK generation module and an EDEK decryption module; the EZ Key access module is used for accessing the EZ Key; the EDEK generation module is used for generating EDEK; the EDEK decryption module is used for decrypting the EDEK to generate DEK.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311572773.2A CN117278342B (en) | 2023-11-23 | 2023-11-23 | Multi-environment Hadoop KMS proxy service method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311572773.2A CN117278342B (en) | 2023-11-23 | 2023-11-23 | Multi-environment Hadoop KMS proxy service method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117278342A CN117278342A (en) | 2023-12-22 |
| CN117278342B true CN117278342B (en) | 2024-03-01 |
Family
ID=89201313
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311572773.2A Active CN117278342B (en) | 2023-11-23 | 2023-11-23 | Multi-environment Hadoop KMS proxy service method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117278342B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119449313B (en) * | 2025-01-10 | 2025-04-22 | 深圳市丕微科技企业有限公司 | Information security management system based on non-homogeneous digital rights and interests |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107483491A (en) * | 2017-09-19 | 2017-12-15 | 山东大学 | An access control method for distributed storage in cloud environment |
| CN108111479A (en) * | 2017-11-10 | 2018-06-01 | 中国电子科技集团公司第三十二研究所 | Key management method for transparent encryption and decryption of Hadoop distributed file system |
| CN115422570A (en) * | 2022-11-07 | 2022-12-02 | 北京数盾信息科技有限公司 | Data processing method and system for distributed storage |
| CN115688165A (en) * | 2022-09-27 | 2023-02-03 | 中国农业银行股份有限公司 | Node file processing method, device, equipment and storage medium |
-
2023
- 2023-11-23 CN CN202311572773.2A patent/CN117278342B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107483491A (en) * | 2017-09-19 | 2017-12-15 | 山东大学 | An access control method for distributed storage in cloud environment |
| CN108111479A (en) * | 2017-11-10 | 2018-06-01 | 中国电子科技集团公司第三十二研究所 | Key management method for transparent encryption and decryption of Hadoop distributed file system |
| CN115688165A (en) * | 2022-09-27 | 2023-02-03 | 中国农业银行股份有限公司 | Node file processing method, device, equipment and storage medium |
| CN115422570A (en) * | 2022-11-07 | 2022-12-02 | 北京数盾信息科技有限公司 | Data processing method and system for distributed storage |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117278342A (en) | 2023-12-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108259169B (en) | A method and system for secure file sharing based on blockchain cloud storage | |
| US20190230072A1 (en) | Securing files using per-file key encryption | |
| US8925108B2 (en) | Document access auditing | |
| US8479301B2 (en) | Offline access in a document control system | |
| US8627077B2 (en) | Transparent authentication process integration | |
| US8898482B2 (en) | Encryption system using clients and untrusted servers | |
| US8627489B2 (en) | Distributed document version control | |
| US7792300B1 (en) | Method and apparatus for re-encrypting data in a transaction-based secure storage system | |
| US9495552B2 (en) | Integrated data deduplication and encryption | |
| US10013567B2 (en) | Private and public sharing of electronic assets | |
| US20130212707A1 (en) | Document control system | |
| US20060010323A1 (en) | Method for a repository to provide access to a document, and a repository arranged in accordance with the same method | |
| US20030174842A1 (en) | Managing private keys in a free seating environment | |
| CN102687132A (en) | Trusted Extensible Markup Language for Trusted Computing and Data Services | |
| US10250385B2 (en) | Customer call logging data privacy in cloud infrastructure | |
| US7725716B2 (en) | Methods and systems for encrypting, transmitting, and storing electronic information and files | |
| CA3145851A1 (en) | Enhanced secure encryption and decryption system | |
| US11683159B2 (en) | Hybrid content protection architecture | |
| US8707034B1 (en) | Method and system for using remote headers to secure electronic files | |
| KR20210143846A (en) | encryption systems | |
| KR20210058313A (en) | Data access control method and system using attribute-based password for secure and efficient data sharing in cloud environment | |
| CN117278342B (en) | Multi-environment Hadoop KMS proxy service method and system | |
| KR100464797B1 (en) | Encryption and decryption method of electronic documents by a network key | |
| US7886147B2 (en) | Method, apparatus and computer readable medium for secure conversion of confidential files | |
| CN113553616A (en) | A trusted data security service method, device, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |