TWI846601B - Operating system and method for a fully homomorphic encryption neural network model - Google Patents

Abstract

An operating method for a fully homomorphic encrypted neural network model is provided, wherein the fully homomorphic encrypted neural network model includes a plurality of layers, and the method performed by a processor includes: for one of the plurality of layers, encrypting a plaintext input with a first encryption algorithm to generate a ciphertext vector, performing a convolution operation according to the ciphertext vector to generate a result vector, transforming the result vector into a plurality of result ciphertexts adopting a second encryption algorithm, inputting the plurality of result ciphertexts into an activation function to generate a plurality of encrypted activation values, and repacking the plurality of encrypted activation values to generate an output vector adopting the first encryption algorithm.

Description

Operation system and method of fully homomorphic encrypted neural network model

The present invention relates to fully homomorphic encryption and neural networks, and in particular to an operating system and method of a fully homomorphic encryption neural network model.

Machine Learning as a Service (MLaaS) allows users to upload data to a neural network model running on a cloud platform for reasoning. Although MLaaS is extremely convenient, since users' private data is submitted or stored in an external environment, how to maintain data privacy becomes an important issue.

Fully Homomorphic Encryption (FHE) allows direct computation on encrypted data. In other words, the neural network model does not need to use the original data during the computation process, thus ensuring the privacy of the user's data. Currently, there are two commonly used FHE schemes for encrypted neural network reasoning: CKKS and FHEW/TFHE. The CKKS scheme supports floating-point operations, so linear operations can be performed efficiently. However, it does not support non-polynomial operations. FHEW/TFHE supports lightweight bit or integer operations, and includes functional bootstrapping technology to implement non-polynomial operations. Therefore, if the above two schemes are combined, taking advantage of the linear operation advantages of CKKS and the functional bootstrapping advantages of FHEW/TFHE, it seems that it can be applied to encrypted neural network models. However, existing experimental data show that the inference accuracy of the neural network model established by simply combining these two encryption schemes decreases significantly.

In view of this, the present invention proposes an operating system and method for a fully homomorphic encrypted neural network model, thereby solving the problem of decreased reasoning accuracy of the neural network model caused by simply combining the CKKS and FHEW/TFHE schemes.

According to an embodiment of the present invention, a method for operating a fully homomorphic encrypted neural network model, wherein the fully homomorphic encrypted neural network model includes multiple layers, the method includes executing with a processor: for one of the multiple layers, encrypting the plaintext input with a first encryption algorithm to generate a ciphertext vector, performing a convolution operation on the ciphertext vector to generate a result vector, converting the result vector into multiple result ciphertexts using a second encryption algorithm, inputting the multiple result ciphertexts into an incentive function to generate multiple encrypted incentive values, and converting the multiple encrypted incentive values to generate an output vector using the first encryption algorithm.

According to an embodiment of the present invention, an operating system of a fully homomorphic encrypted neural network model includes a memory and a processor. The memory is used to store multiple instructions. The processor is electrically connected to the memory to execute the multiple instructions, which are used to perform the following operations on one of the multiple layers of the fully homomorphic encrypted neural network model: encrypting plaintext input with a first encryption algorithm to generate a ciphertext vector, performing a convolution operation on the ciphertext vector to generate a result vector, converting the result vector into multiple result ciphertexts using a second encryption algorithm, inputting the multiple result ciphertexts into an incentive function to generate multiple encrypted incentive values, and converting the multiple encrypted incentive values to generate an output vector using the first encryption algorithm.

The above description of the disclosed content and the following description of the implementation methods are used to demonstrate and explain the spirit and principle of the present invention, and provide a further explanation of the scope of the patent application of the present invention.

The detailed features and characteristics of the present invention are described in detail in the following embodiments, and the content is sufficient for any person skilled in the relevant art to understand the technical content of the present invention and implement it accordingly. According to the content disclosed in this specification, the scope of the patent application and the drawings, any person skilled in the relevant art can easily understand the concept and characteristics of the present invention. The following embodiments are to further illustrate the viewpoints of the present invention, but are not to limit the scope of the present invention by any viewpoint.

Fig. 1 is a block diagram of an operating system of a fully homomorphic encrypted neural network model according to an embodiment of the present invention. As shown in Fig. 1, the operating system 10 includes a memory 1 and a processor 3 electrically connected to the memory 1.

The memory 1 is used to store multiple instructions. In one embodiment, the memory 1 can be implemented by one or more of the following examples: dynamic random access memory (DRAM), static random access memory (SRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), flash memory and hard disk. The present invention does not limit the type and quantity of the memory 1.

The processor 3 is used to execute multiple instructions stored in the memory 1. In one embodiment, the processor 3 can be implemented by one or more of the following examples: a personal computer, a network server, a microcontroller (MCU), an application processor (AP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC) system-on-a-chip (SOC), and a deep learning accelerator. The present invention does not limit the type and quantity of the processor 3.

The fully homomorphic encrypted neural network model has multiple layers, and the multiple instructions are used to perform the process shown in Figure 2 on one of the layers. Figure 2 is a flow chart of the operation method of the fully homomorphic encrypted neural network model according to an embodiment of the present invention, including steps S1 to S5. Figure 3 is a schematic diagram of the operation method of the fully homomorphic encrypted neural network model according to an embodiment of the present invention, showing the input, output and corresponding operation of each step in Figure 2.

Step S1, encrypting the plaintext input using a first encryption algorithm to generate a ciphertext vector.

In one embodiment, the first encryption algorithm is the CKKS (Cheon-Kim-Kim-Song) algorithm, which can be found in “Cheon, J.H., Kim, A., Kim, M., Song, Y.: Homomorphic encryption for arithmetic of approximate numbers. In: ASIACRYPT. pp. 409–437. Springer (2017)”. CKKS supports arithmetic operations on floating-point vectors. In the encryption process, the plaintext input (floating-point vector) is first encoded into a plaintext polynomial and then encrypted into a ciphertext vector. Therefore, the operation of step S1 can be expressed as CKKS(m)=Enc(Ecd(m)), where m ∈ Rn represents the plaintext input, Ecd(·) represents the encoding operation, and Enc(·) represents the encryption operation.

Please note that step S1 is applicable to the first convolution layer executed in the neural network model. If it is a convolution layer in the middle of the neural network model, since the result from the previous layer is already encrypted, step S1 can be omitted and the process can be started directly from step S2.

Step S2, performing a convolution operation on the ciphertext vector to generate a result vector.

In one embodiment, the convolution operation can be expressed as fconv’(x)=W’(x)+b’, where W’ represents the convolution weight and b’ represents the bias. The operation of step S2 can be expressed as CKKS.eval(fconv’, {ctx0, ctx1, …})={ctx’0, ctx’1, …}, where {ctx0, ctx1, …} represents the ciphertext vector and {ctx’0, ctx’1, …} represents the result vector.

CKKS supports the evaluation function CKKS.eval(f, ·) which takes as input a plaintext function f and the arguments to f following f, e.g. plaintext input m or ciphertext ctx ∈ CKKS(m). The function operates by running the function in CKKS with these arguments and returns ctx ∈ CKKS(f(·)). For example: CKKS.eval(+, ctx, v) returns ctx’ ∈ CKKS(m + v).

Step S3, converting the result vector into a plurality of result ciphertexts using a second encryption algorithm.

In one embodiment, the second encryption algorithm is related to Learn with Error (LWE). The encryption operation of LWE can be expressed as LWE⌈m⌋, where ⌈·⌋ represents rounding and encoding operations, and m ∈ R is a scalar.

There are two implementations of LWE: the first is FHEW, see Ducas, L., Micciancio, D.: Fhew: bootstrapping homomorphic encryption in less than a second. In: EUROCRYPT. pp. 617–640. Springer (2015); the second is TFHE, see Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: TFHE: fast fully homomorphic encryption over the torus. Journal of Cryptology 33(1), 34–91 (2020).

The conversion operation mentioned in step S3 is applied to PEGASUS, a framework for converting between CKKS ciphertext and LWE ciphertext without decryption, see jie Lu, W., Huang, Z., Hong, C., Ma, Y., Qu, H.: PEGASUS: Bridging polynomial and non-polynomial evaluations in homomorphic encryption. In: 2021 IEEE Symposium on Security and Privacy. pp. 1057–1073. IEEE Computer Society Press (May 2021).

Therefore, step S3 can be expressed as pegasus.extract(ctx)={ctx’i}. ctx ∈ CKKS(m) represents the result vector, where m ∈ Rn. {ctx’i }∈ LWE(⌈mi⌋) represents the multiple result ciphertexts, where mi ∈ m and 0 ≤ i ＜ n.

Step S4, inputting the plurality of result ciphertexts into the incentive function to generate a plurality of encrypted incentive values.

In one embodiment, the excitation function is a rectified linear unit (ReLU). PEGASUS uses a discrete (fine-grained) look-up table (LUT) approximation to evaluate non-polynomial functions, such as ReLU. Step S4 can be expressed as pegasus.eval(f _ACT , ctx _i )=ctx' _i , where f _ACT represents the excitation function, ctx _i ∈ LWE(⌈m _i ⌋) represents the multiple result ciphertexts, and ctx' _i ∈ LWE(⌈f _ACT' (m _i )⌋) represents the multiple encrypted excitation values.

Step S5, converting the plurality of encryption incentive values into output vectors using a first encryption algorithm.

In one embodiment, step S5 can be expressed as pegasus.repack({cxt' _i }=ctx, where ctx' _i ∈ LWE(⌈m _i ⌋) represents the multiple encrypted incentive values, and ctx ∈ CKKS(m) represents the output vector.

Please note: If the fully homomorphic encrypted neural network model is simply implemented using the CKKS-FHEW/TFHE hybrid encryption inference framework and PEGASUS, its accuracy will be severely reduced.

The numerical errors accumulated during the inference of encrypted neural networks lead to a decrease in accuracy, and these numerical errors are generated by the scale-down procedure in the PEGASUS framework, which limits the numerical precision when applying functional guidance to large-domain ciphertexts.

FHEW supports functional bootstrapping. Functional bootstrapping is a type of bootstrapping used in homomorphic encryption operations to reduce error accumulation. It involves building a lookup table of a specific function to efficiently compute the result. Although it requires fewer bootstrapping iterations and less computation, it is only effective for small message domains. If a large message domain is to be supported, it will incur a lot of computational cost.

Conventional functional guidance methods can only be applied to LWE ciphertexts whose message domain is limited to the size of the lookup table, usually about 2 ^10. However, when processing LWE ciphertexts converted from CKKS ciphertexts, a larger message domain is usually required. To address this problem, PEGASUS introduces a technique for converting between CKKS and FHEW. The technique involves reducing the input ciphertext to a smaller message domain during the conversion from CKKS to FHEW, thereby being able to expand conventional functional guidance to support larger message domains. However, this expansion comes at the cost of reduced accuracy.

If the input value range of LWE functional guidance is not aligned with its predetermined input domain, the input domain will not match the input value range, or only a small number of lookup table entries will be used. Please refer to Figure 4(a) and Figure 4(b). Figure 4(a) shows an example where the predetermined input domain [-4, 4) of LWE functional guidance is larger than the input value range [-0.3, 1.7). Therefore, only three lookup table entries are used to map the input value x to the output y. When applying functional guidance, range mismatch will produce large numerical errors, which will ultimately affect the accuracy of the neural network model.

Therefore, the present invention proposes a lookup table-aware (LUT-aware) model fine-tuning method to align the range of input values with the message domain of the ciphertext. By aligning the input value with the lookup table item, as shown in Figure 4 (b), the mismatch can be alleviated. Specifically, after establishing a fully homomorphic encrypted neural network model based on the training dataset, it is necessary to adjust the model weights and incentive functions therein. Please refer to Figure 5. Figure 5 is a flow chart of a lookup table-aware (LUT-aware) model fine-tuning method according to an embodiment of the present invention. The execution process of Figure 5 is at least before step S2 "performing a convolution operation based on the ciphertext vector to generate a result vector".

Step P1, for each layer of the model, the training procedure is executed multiple times to generate multiple plaintext incentive values. Figure 6 is a flow chart of the training procedure, including step P11, performing a convolution operation on the plaintext input to generate a plaintext vector; and step P12, inputting the plaintext vector to the incentive function to generate one of the multiple plaintext incentive values.

Step P2, setting a linear mapping range according to the ranges of the plurality of plaintext stimulus values.

Since the input of the excitation function (i.e., the result ciphertext) is encrypted when the fully homomorphic encrypted neural network model is actually applied, the range of the input of the excitation function cannot be predetermined, but varies according to the input value input to the neural network. The present invention assumes that the distribution of the training data and the test data is similar. Therefore, in order to estimate the range of these inputs, in step P1, the plaintext input in the training data set is inferred using the trained model. In this process, the minimum and maximum values of the output of the excitation function at each layer are observed and recorded, respectively represented as _ai and _bi . If [a _i ,b _i ) represents the output range of the excitation function of the i-th layer, the linear mapping range is set to [-B, +B), where B=max(|ai|, |bi|), where i=1, 2, …, n. In other words, the value of +B is greater than the maximum output of the excitation function of each layer, and the value of -B is less than the minimum output of the excitation function.

Step P3, for each layer, determine the linear mapping function according to the range of the plurality of plaintext excitation values and the linear mapping range. A simple example is given below to illustrate the determination of the linear mapping function, but the numerical values in the example are not limited. Assume that a _i = 2, b _i = 102, B = 1000. Then the linear mapping function is: f(x) = (x - z) * s', where z represents the zero point and s' represents the adjustment factor. The parameters in the linear mapping function are calculated as follows:

Output range: s = b - a = 100

Zero point: z = (a + b) / 2 = 52.

Linear mapping range: S = B - (-B) = 2000

Adjustment factor: s’ = S / s = 20

A simple verification based on the above obtained linear mapping function f(x) = (x-52) * 20 is as follows:

f(a) = f(2) = (2 - 52) * 20 = -1000 = -B

f(b) = f(102) = (102 - 52) / 20 = 1000 = B

Step P4, update the weight of the convolution operation according to the linear mapping function.

Linear mapping of the input ensures that the input interval is aligned with the information domain. Incorporating the linear mapping function into the convolution weights and offset values can avoid additional computational costs or reduce memory usage during calculations. In one embodiment, step P4 can be expressed as .in represents the convolution operation after updating the weights in step S2. Represents a linear mapping function, which can be ranged to Input Linear mapping to range . represents the original convolution operation of the model built based on the training data set. The following is a simple example to illustrate how to update the weights:

Assuming that the original convolution operation is represented by y = Wx + b, and the linear mapping function is represented by f(x) = (x - z) * s’, then f(y) = (Wx + b) * s’ - z * s’ = Wx * s’ + b * s’ - z * s’ = (W * s’) * x + (b * s’ - z * s’) = W’x + b’, where the updated weight W’ = W * s’, and the updated offset value b’ = b * s’ - z * s’.

Step P5, update the incentive function according to the inverse function of the linear mapping function.

Step P5 can be expressed as ,in represents the updated incentive function, represents the original activation function established based on the training data set, represents the inverse function of the linear mapping function. Since the input of the stimulus function has been adjusted by the linear mapping function, the stimulus function itself also needs to be adjusted accordingly so that the output has the same result, that is, In this way, errors caused by the limited accuracy of functional guidance can be greatly reduced.

Table 1, where D represents the network depth and L represents the number of fully connected layers. Model D5L1 D8L1 D11L1 D7L3 Plaintext DNN Accuracy 60% 72.5% 77.5% 80% Encrypted DNN Accuracy - Not LUT-aware 47.5% 55% 62.5% 72.5% Encrypting DNN Accuracy - Using LUT-aware (Present Invention) 57.5% 70% 75% 80% Total time 881 seconds 2092 seconds 9458 seconds 2558 seconds

Please refer to Table 1. The inventors conducted a comprehensive experiment to verify the effectiveness of the LUT perception model fine-tuning method proposed in the present invention on CIFAR-10, a color dataset that is more complex than the MNIST dataset. The experimental results show that compared with the neural network model that does not use the method of the present invention, the accuracy of the application of the present invention can be improved from 7.5% (such as D7L3) to 15% (such as D8L1). In addition, the accuracy of the fine-tuned neural network model can reach the accuracy of the original neural network model at most (such as D7L3).

In summary, the operating system and method of the fully homomorphic encrypted neural network model proposed in the present invention inherit the advantages of linear operation in CKKS and the advantages of functional guidance in FHEW/TFHE. For the fully homomorphic encrypted neural network model, the CKKS scheme can be used for convolution operation, and the FHEW-TFHE functional guidance scheme can be used for non-polynomial incentive function and guidance. The present invention applies PEGASUS to convert between CKKS ciphertext and LWE ciphertext, and proposes a lookup table-aware fine-tuning method to adjust the model weights and incentive functions, thereby improving the accuracy of the fully homomorphic encrypted neural network model.

Although the present invention is disclosed as above with the aforementioned embodiments, it is not intended to limit the present invention. Any changes and modifications made within the spirit and scope of the present invention are within the scope of patent protection of the present invention. Please refer to the attached patent application for the scope of protection defined by the present invention.

10: System 1: Memory 3: Processor S1-S5, P1-P5, P11-P12: Steps

FIG1 is a block diagram of an operating system of a fully homomorphic encrypted neural network model according to an embodiment of the present invention; FIG2 is a flow chart of an operating method of a fully homomorphic encrypted neural network model according to an embodiment of the present invention; FIG3 is a schematic diagram of an operating method of a fully homomorphic encrypted neural network model according to an embodiment of the present invention; FIG4(a) and FIG4(b) are example schematic diagrams of range alignment; FIG5 is a flow chart of a lookup table-aware model fine-tuning method according to an embodiment of the present invention; and FIG6 is a detailed flow chart of the training procedure in FIG5.

S1-S5: Steps

Claims (8)

A method for operating a fully homomorphic encrypted neural network model, wherein the fully homomorphic encrypted neural network model includes multiple layers, and the method includes executing with a processor: For one of the layers, encrypt a plaintext input using a first encryption algorithm to generate a ciphertext vector; Perform a convolution operation based on the ciphertext vector to generate a result vector; Convert the result vector into multiple result ciphertexts using a second encryption algorithm; Input the result ciphertexts into an incentive function to generate multiple encrypted incentive values; and Convert the encrypted incentive values into an output vector using the first encryption algorithm. The operation method of the fully homomorphic encryption neural network model as described in claim 1 further includes: Before performing the convolution operation based on the ciphertext vector to generate the result vector, for each of the layers, a training procedure is executed multiple times to generate multiple plaintext incentive values, wherein the training procedure includes: Performing the convolution operation with the plaintext input to generate a plaintext vector; and Inputting the plaintext vector to the incentive function to generate one of the plaintext incentive values; Setting a linear mapping range based on the range of the plaintext incentive values; For each of the layers, determining a linear mapping function based on the range of the plaintext incentive values and the linear mapping range; Updating the weight of the convolution operation based on the linear mapping function; and Update the activation function according to the inverse function of the linear mapping function. An operating method of a fully homomorphic encrypted neural network model as described in claim 1, wherein the first encryption algorithm is a CKKS (Cheon-Kim-Kim-Song) algorithm, and the second encryption algorithm is related to learn with error. An operating method of a fully homomorphic encrypted neural network model as described in claim 1, wherein the excitation function is a rectified linear unit. An operating system for a fully homomorphic encrypted neural network model includes: a memory for storing a plurality of instructions; and a processor electrically connected to the memory to execute the instructions, wherein the instructions are used to perform the following operations on one of the plurality of layers of the fully homomorphic encrypted neural network model: encrypting a plaintext input with a first encryption algorithm to generate a ciphertext vector; performing a convolution operation based on the ciphertext vector to generate a result vector; converting the result vector into a plurality of result ciphertexts using a second encryption algorithm; inputting the result ciphertexts into an incentive function to generate a plurality of encrypted incentive values; and converting the encrypted incentive values into an output vector using the first encryption algorithm. The operating system of the fully homomorphic encrypted neural network model as described in claim 5, wherein the instructions further include the following operations: Before performing the convolution operation based on the ciphertext vector to generate the result vector, for each of the layers, a training procedure is executed multiple times to generate multiple plaintext incentive values, wherein the training procedure includes: Performing the convolution operation with the plaintext input to generate a plaintext vector; and Inputting the plaintext vector to the incentive function to generate one of the plaintext incentive values; Setting a linear mapping range based on the range of the plaintext incentive values; For each of the layers, determining a linear mapping function based on the range of the plaintext incentive values and the linear mapping range; Updating the weight of the convolution operation based on the linear mapping function; and Update the activation function according to the inverse function of the linear mapping function. An operating system of a fully homomorphic encrypted neural network model as described in claim 5, wherein the first encryption algorithm is a CKKS (Cheon-Kim-Kim-Song) algorithm, and the second encryption algorithm is related to learn with error. An operating system of a fully homomorphic encrypted neural network model as described in claim 5, wherein the excitation function is a rectified linear unit.

Images