CN111539008A - Image processing method and device for protecting privacy - Google Patents

Abstract

The embodiment of the specification provides an image processing method and device for protecting privacy and a model training method and device for protecting privacy. One embodiment of the image processing method includes: acquiring a mixed image corresponding to a biological characteristic image to be encrypted, wherein the mixed image is obtained by weighting and summing the biological characteristic image and a preset number of other images, the preset number of other images are used as noise images, and the sum of weights of the other images is less than a preset threshold value; and subtracting a preset image template from the mixed image to obtain an encrypted image corresponding to the biological characteristic image. The biometric image may be regarded as private data of the user. By adopting the encryption method, the protection of the private data of the user can be realized, and the obtained encrypted image can be used for subsequent model training.

Description

Image processing method and device for protecting privacy

technical field

The embodiments of this specification relate to the technical field of data security, and in particular, to an image processing method and apparatus for protecting privacy, and a model training method and apparatus for protecting privacy.

Background technique

At present, many products (such as wealth management products, payment products, security products, etc.) use a large number of biometric images approved by users, such as face images, fingerprint images, iris images and so on. In addition, a large number of biometric images are also required in the training process of the biometric recognition algorithm.

Since the biometric image is the user's private data, we need a reasonable encryption algorithm to protect the user's private data by encrypting the biometric image without affecting the encrypted biometric image for subsequent models. train.

SUMMARY OF THE INVENTION

The embodiments of this specification provide a privacy-protecting image processing method and device, and a privacy-protecting model training method and device.

In a first aspect, the embodiments of this specification provide an image processing method for protecting privacy, the method includes: acquiring a mixed image corresponding to a biometric image to be encrypted, wherein the mixed image is obtained by comparing the biometric image obtained after weighted summation with a predetermined number of other images, the predetermined number of other images are used as noise images, and the sum of their weights is less than a preset threshold; the mixed image is subtracted from the predetermined image template to obtain the The encrypted image corresponding to the biometric image.

In some embodiments, the biometric image has a corresponding label; and the method further includes generating a training sample including the encrypted image and the label.

In some embodiments, the biometric image includes any of the following images: a face image, a fingerprint image, a finger vein image, an iris image.

In some embodiments, the biometric image to be encrypted is any one image in a predetermined set of biometric images, and the predetermined number of other images are other images in the set of biometric images except the to-be-encrypted image. Images other than biometric images.

In some embodiments, the obtaining a mixed image corresponding to the biometric image to be encrypted comprises: randomly selecting a predetermined number of images in the set of biometric images other than the biometric image to be encrypted images; assign weights to the biometric images to be encrypted and the selected predetermined number of images; perform a weighted sum on the biometric images to be encrypted and the selected predetermined number of images according to the assigned weights, The mixed image is obtained.

In some embodiments, assigning weights to the biometric images to be encrypted and the selected predetermined number of images includes: acquiring a predetermined weight set; and assigning the biometric features to be encrypted according to the weight set Weights are assigned to images and a predetermined number of selected images.

In some embodiments, before the obtaining the mixed image corresponding to the biometric image to be encrypted, the method further comprises: obtaining a plurality of images; calculating an average value among the plurality of images; The value is determined for the image template.

In some embodiments, the plurality of images are images of the same class as the biometric image to be encrypted.

In a second aspect, an embodiment of this specification provides a privacy-protecting model training method, the method includes: acquiring a training sample set, wherein the training sample includes encrypted images and labels, and the encrypted images are obtained by mixing images obtained from their corresponding mixed images. It is obtained by subtracting a predetermined image template from , the mixed image is obtained by weighted summation of the corresponding biometric image to be encrypted and a predetermined number of other images, and the label is the biometric image According to the training sample set, the deep learning model to be trained is trained to obtain a biometric identification model.

In some embodiments, the biometric image to be encrypted is any one image in a predetermined set of biometric images, and the predetermined number of other images are other images in the set of biometric images except the to-be-encrypted image. Images other than biometric images.

In some embodiments, the biometric image to be encrypted includes any one of the following images: a face image, a fingerprint image, a finger vein image, and an iris image.

In some embodiments, the image template is an average across multiple images.

In some embodiments, the plurality of images are images of the same class as the biometric image to be encrypted.

In a third aspect, embodiments of this specification provide an image processing apparatus for protecting privacy, the apparatus includes: an acquisition unit configured to acquire a mixed image corresponding to a biometric image to be encrypted, wherein the mixed image is obtained by obtained after the weighted summation of the biometric image and a predetermined number of other images, the predetermined number of other images are used as noise images, and the sum of their weights is less than a preset threshold; the encryption unit is configured to A predetermined image template is subtracted from the mixed image to obtain an encrypted image corresponding to the biometric image.

In a fourth aspect, an embodiment of this specification provides a privacy-protecting model training device, the device includes: an acquisition unit configured to acquire a training sample set, wherein the training samples include encrypted images and labels, and the encrypted images are obtained by It is obtained by subtracting a predetermined image template from its corresponding mixed image, and the mixed image is obtained by performing a weighted sum of its corresponding biometric image to be encrypted and a predetermined number of other images. is the label of the biometric image; the training unit is configured to train the deep learning model to be trained according to the training sample set to obtain a biometric identification model.

In a fifth aspect, the embodiments of the present specification provide a computer-readable storage medium on which a computer program is stored, wherein, when the computer program is executed in a computer, the computer is caused to execute the method described in the first aspect and the second aspect. The method described by either implementation.

In a sixth aspect, embodiments of this specification provide a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the first aspect and the second A method as described by any implementation in an aspect.

The image processing method and device for protecting privacy provided by the above-mentioned embodiments of this specification obtain a mixed image corresponding to the biometric image to be encrypted, wherein the mixed image is obtained by weighting the biometric image and a predetermined number of other images. After the summation is obtained, a predetermined number of other images are used as noise images, and the sum of their weights is less than the preset threshold, and then the mixed image is subtracted from the predetermined image template to obtain the encrypted image corresponding to the biometric image. The encryption method can make the encrypted image undecipherable and visually difficult to identify, thus protecting the user's private data. In addition, by using the predetermined number of other images as noise images and restricting the sum of their weights to be smaller than the preset threshold, the encrypted images can be recognized algorithmically, and thus can be used for subsequent model training.

The model training method and device for protecting privacy provided by the above-mentioned embodiments of this specification are obtained by acquiring a training sample set, wherein the training samples include encrypted images and labels, and the encrypted images are obtained by subtracting a predetermined image template from its corresponding mixed image. Then, the mixed image is obtained by weighting the corresponding biometric image to be encrypted and a predetermined number of other images, and the label is the label of the biometric image, and then according to the training sample set, the image to be trained is obtained. The deep learning model is trained to obtain a biometric recognition model, and the model training based on encrypted images is realized.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present specification or in the prior art, the accompanying drawings required to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some examples or embodiments of the present specification. For those skilled in the art, other drawings can also be obtained according to the provided drawings without any creative effort. , but also can be applied to other similar scenarios according to the provided drawings. Unless obvious from the locale or otherwise specified, the same reference numbers in the figures represent the same structure or operation.

FIG. 1 is an exemplary system architecture diagram to which some embodiments of the present specification may be applied;

FIG. 2 is a flowchart of an embodiment of an image processing method for protecting privacy according to the present specification;

Fig. 3a is a schematic diagram of an encrypted face image acquisition process;

Figure 3b is another schematic diagram of an encrypted face image acquisition process;

3c is a schematic diagram of an application scenario of the image processing method for protecting privacy according to this specification;

FIG. 4 is a flowchart of still another embodiment of the image processing method for protecting privacy according to the present specification;

FIG. 5 is a flowchart of an embodiment of a model training method for protecting privacy according to the present specification;

6 is a schematic diagram of an application scenario of the model training method for protecting privacy according to this specification;

7 is a schematic structural diagram of an image processing apparatus for protecting privacy according to the present specification;

FIG. 8 is a schematic structural diagram of a model training apparatus for protecting privacy according to the present specification.

Detailed ways

The specification will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the related invention, but not to limit the invention. The described embodiments are only some of the embodiments of this specification, but not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

It should be noted that, for the convenience of description, only the parts related to the related invention are shown in the drawings. The embodiments in this specification and the features of the embodiments may be combined with each other without conflict.

As mentioned above, many products (such as wealth management products, payment products, security products, etc.) currently use a large number of biometric images approved by users. For example, existing face-swiping payment products make extensive use of user-approved face images. In addition, a large number of biometric images are also required in the training process of the biometric recognition algorithm.

Since the biometric image is the user's private data, we need a reasonable encryption algorithm to protect the user's private data by encrypting the biometric image without affecting the encrypted biometric image for subsequent models. train.

Based on this, some embodiments of this specification respectively disclose a privacy-preserving image processing method and a privacy-preserving model training method. Specifically, FIG. 1 shows an exemplary system architecture diagram suitable for use with these embodiments.

As shown in FIG. 1 , it shows an image processing end 100 and a model training end 101 . The image processing terminal 100 may be used to provide image processing services, and further, may be used to provide image encryption services. The model training terminal 101 can be used to provide model training services. Specifically, the image processing terminal 100 may encrypt the biometric image to be encrypted to obtain an encrypted image corresponding to the biometric image. Wherein, the biometric image may have a corresponding label. The encrypted image and the label can be used to compose training samples. The model training terminal 101 may obtain a training sample set composed of such training samples, and perform model training according to the training sample set to obtain a corresponding biometric identification model.

It should be noted that, the image processing terminal 100 may be implemented as a server terminal or a client terminal. The client can be a terminal device, or software installed on the terminal device. The model training terminal 101 may be implemented as a server terminal. The server can be a cloud platform, a single server or a server cluster.

It should be pointed out that when the image processing terminal 100 is implemented as a server, the image processing terminal 100 and the model training terminal 101 may be the same server or different servers, which are not specifically limited here.

It should be understood that the numbers of image processing ends and model training ends shown in FIG. 1 are only illustrative. According to implementation needs, there can be any number of image processing ends and model training ends.

The specific implementation steps of the above method will be described below with reference to specific embodiments.

Referring to FIG. 2, a flow 200 of one embodiment of a privacy-preserving image processing method is shown. The method is applied to the image processing terminal 100 shown in FIG. 1, and includes the following steps:

Step 201, obtaining a mixed image corresponding to the biometric image to be encrypted, wherein the mixed image is obtained by performing a weighted summation on the biometric image and a predetermined number of other images, and the predetermined number of other images are used as noise images. , the sum of its weights is less than the preset threshold;

Step 202, subtract a predetermined image template from the mixed image to obtain an encrypted image corresponding to the biometric image to be encrypted.

Below, step 201 and step 202 are described in detail.

For step 201, the image processing end may perform this step in response to receiving an encryption request related to the biometric image to be encrypted. The encryption request may be triggered manually or automatically, which is not specifically limited here.

For example, if the encryption request is triggered manually, the encryption request may be sent by an image manager through a terminal device, for example. If the encryption request is triggered automatically, the encryption request may be triggered by a scheduled task.

In step 201, the biometric image to be encrypted may include any of the following images: a face image, a fingerprint image, a finger vein image, an iris image, a voiceprint image, a palmprint image, and the like.

The predetermined number may be an integer greater than or equal to one. It should be understood that the predetermined number can be set according to actual requirements, which is not specifically limited here.

Weights can be numeric values in [0, 1]. The sum of the weights of the biometric image to be encrypted and the predetermined number of other images may be equal to one. In addition, the preset threshold may be a value not greater than the weight of the biometric image to be encrypted. In an optional implementation manner, the sum of the weights of the biometric image to be encrypted and a predetermined number of other images is equal to 1, the preset threshold is smaller than the weight of the biometric image to be encrypted, and the weight of the predetermined number of other images is The sum is less than the preset threshold. It should be understood that the preset threshold can be set according to actual requirements, and is not specifically limited here.

It should be noted that, by using a predetermined number of other images as noise images and limiting the sum of their weights to be smaller than a preset threshold, the mixed image can be a noise-enhanced image of the biometric image to be encrypted. The blended image looks blurry visually, but in general, outlines can be seen. Therefore, for the encrypted image obtained by subsequently subtracting the predetermined image template from the mixed image, the image can be made more blurred, but it can be recognized from the algorithm, so the encrypted image can be used for subsequent model training.

In practice, the image processing end may adopt various methods to obtain the mixed image corresponding to the biometric image to be encrypted.

For example, if a mixed image corresponding to the biometric image to be encrypted has been generated in advance, the image processing end may acquire the mixed image from a specific storage location.

For another example, if the mixed image corresponding to the biometric image to be encrypted is not pre-generated, the image processing end may perform the following mixed image acquisition steps:

Step a, from any image stored, except the biometric image to be encrypted, randomly select a predetermined number of images;

Step b, assigning weights to the biometric image to be encrypted and the selected predetermined number of images;

Step c, according to the assigned weight, weighted summation is performed on the biometric image to be encrypted and the selected predetermined number of images to obtain a mixed image.

Wherein, in step a, the selected predetermined number of images and the biometric image to be encrypted may be images of the same category or images of different categories. Since the biometric image to be encrypted is denoised by using images of the same class, the resulting mixed image can be more blurred and the noise is better than that by using images of different classes to denoise the biometric image to be encrypted. Preferably, the selected predetermined number of images and the biometric image to be encrypted are images of the same category.

In addition, the selected predetermined number of images and the biometric image to be encrypted may be derived from the same image collection, or may be derived from different image collections, which are not specifically limited herein.

In step b, as an optional implementation manner, the biometric image to be encrypted may correspond to a preset weight. For the selected predetermined number of images, weights may be randomly generated for the selected predetermined number of images based on a preset weight generation algorithm.

As another optional implementation manner, a predetermined weight set may be obtained, and according to the weight set, weights are assigned to the biometric image to be encrypted and the selected predetermined number of images. Specifically, the number of weights in the weight set is the same as the number of biometric images to be encrypted and the number of images of the selected predetermined number of images. The set of weights may include a first weight assigned to the biometric image to be encrypted, and a second weight assigned to a predetermined number of images. When assigning weights, the first weight in the weight set may be directly assigned to the biometric image to be encrypted. In addition, the second weight in the weight set may be randomly assigned to the selected predetermined number of images.

In step c, for the biometric image to be encrypted and the selected predetermined number of images, various weighted summation methods may be adopted.

As an example, according to the assigned weight, a weighted summation may be performed on the pixel features of the biometric image to be encrypted and the selected predetermined number of images.

As another example, the feature vectors extracted from the biometric image to be encrypted and the selected predetermined number of images may be obtained, and then the obtained feature vectors are weighted and summed according to the assigned weights.

As another example, image features obtained by separately processing the biometric image to be encrypted and the feature vectors of the selected predetermined number of images may be obtained, and then the obtained image features may be weighted according to the assigned weights beg for peace.

In step 202, the predetermined image template may be an average value among a plurality of images. The multiple images and the biometric image to be encrypted may be images of the same category or images of different categories, which are not specifically limited herein. Since the image template generated based on the image belonging to the same category as the biometric image to be encrypted is applied to the encryption process of the biometric image to be encrypted, the obtained encrypted image can be made to look more blurred and the encryption effect is better. Preferably, the multiple images and the biometric image to be encrypted are images of the same category.

In addition, the predetermined image template may be generated by the image processing end, or may be generated by another server, which is not specifically limited here. If the predetermined image template is generated by the image processing terminal, before step 201, the image processing terminal may perform the following image template acquisition operations: acquire multiple images; calculate the average value among the multiple images; determine the average value for image templates.

Wherein, the multiple images may be obtained by random sampling. This embodiment does not specifically limit the acquisition manner of the multiple images.

As shown in Fig. 3a, it shows a schematic diagram of the process of obtaining encrypted face images. This acquisition process is applicable to the case where the biometric image to be encrypted is a face image, the predetermined number is 1, and the image template is a face image template. Specifically, Fig. 3a shows the face image P1 to be encrypted, the noise face image P2 used for adding noise to the face image P1 to be encrypted, the weight S1 assigned to the face image P1 to be encrypted, for the noise person The weight S2 assigned by the face image P2, according to the assigned weight, the mixed image M obtained by the weighted summation of the face image P1 to be encrypted and the noise face image P2, and subtract the predetermined face image from the mixed image M. The encrypted face image obtained after the template. Wherein, S2 is smaller than the preset threshold, for example, S2 may be 0.1, and the preset threshold may be 0.15. Also, S1 may be 0.9.

As shown in Fig. 3b, it shows another schematic diagram of an encrypted face image acquisition process. This acquisition process is applicable to the case where the biometric image to be encrypted is a face image, the predetermined number is 2, and the predetermined image template is a face image template. Specifically, Fig. 3b shows the face image P1 to be encrypted, which is used for the noise face image P2 and the noise face image P3 to be added to the encrypted face image P1, and is allocated for the face image P1 to be encrypted. The weight S1, the weight S2 assigned to the noise face image P2, the weight S3 assigned to the noise face image P3, according to the assigned weight, the encrypted face image P1, the noise face image P2 and the noise face image P3 are processed. The mixed image M obtained after the weighted summation, and the encrypted face image obtained by subtracting the predetermined face image template from the mixed image M. Wherein, the sum of S2 and S3 is less than the preset threshold, for example, S2 and S3 may both be 0.1, and the preset threshold may be 0.25. Also, S1 may be 0.8.

It should be understood that the values of the weights S1 , S2 , S3 and the preset thresholds shown above are only used for exemplary description, and do not make any limitation to this specification.

In some optional implementations of this embodiment, the image processing end may store the generated encrypted image in a specific storage location. Alternatively, the generated encrypted image can be sent to the model training end (eg, the model training end 101 shown in FIG. 1 ), so that the model training end can perform model training based on the encrypted image.

In some optional implementations of this embodiment, the biometric image to be encrypted may have a corresponding label. The tag may, for example, be used to characterize the user to which the biometric image is attributed. After obtaining the encrypted image corresponding to the biometric image, the image processing end can generate a training sample including the encrypted image and the label. The training samples can be used for the training of the biometric recognition model. After that, the image processing end can store the generated training samples to a specific storage location. Alternatively, the generated training samples can be sent to the model training end, so that the model training end can perform a model training operation based on the training sample set formed by the received training samples.

Continue to refer to FIG. 3c, which is a schematic diagram of an application scenario of the image processing method for protecting privacy according to this embodiment. In this application scenario, the biometric image to be encrypted includes a face image. The predetermined number is 1. The predetermined image template is a face image template. The image processing end is a server, and the image processing end can execute the above process 200 in real time.

When the image manager needs to encrypt the user's face image, the image manager can perform the encryption operation related to the face image to be encrypted through the terminal device used by the image manager, as shown by reference numeral 301 . Afterwards, the terminal device may, in response to the encryption operation, as indicated by reference numeral 302, send an encryption request related to the face image to be encrypted to the image processing end. Then, the image processing end may, as indicated by the reference numeral 303, according to the request, use any of the above-described obtaining means for obtaining a mixed image to obtain a mixed image corresponding to the face image to be encrypted, and the mixed image is obtained by The face image and one other face image are obtained after weighted summation, the other face image is used as a noise image, and its weight is less than a preset threshold (for example, 0.15). Afterwards, the image processing end may, as indicated by reference numeral 304, subtract a predetermined face image template from the mixed image to obtain an encrypted face image corresponding to the face image to be encrypted.

Afterwards, the image processing end may perform any of the following, for example:

Store the encrypted face image in a specific storage location;

Send the encrypted face image to the model training terminal;

generating a training sample including the encrypted face image and the label of the face image to be encrypted corresponding to the encrypted face image, and storing the training sample in a specific storage location;

A training sample including the encrypted face image and the label of the face image to be encrypted corresponding to the encrypted face image is generated, and the training sample is sent to the model training end.

It should be noted that a third new face image ( For example, mixed images), in the calculation process, it is equivalent to using a quadratic linear equation. Only based on the new face image obtained by calculation, it is impossible to parse the original two face images. By using this irreversible calculation for face image encryption protection, the face image can be made more private and irreversible, and a better face recognition model can be obtained by training the encrypted face image without affecting.

It should be pointed out that those skilled in the art can obtain other types of biometric image implementations by analogy based on the content related to the face image shown in FIG. 3c, which is not listed one by one in this specification.

The privacy-preserving image processing method provided by this embodiment obtains a mixed image corresponding to the biometric image to be encrypted, wherein the mixed image is obtained by performing a weighted summation on the biometric image and a predetermined number of other images , a predetermined number of other images are used as noise images, and the sum of their weights is less than the preset threshold, and then the mixed image is subtracted from the predetermined image template to obtain the encrypted image corresponding to the biometric image. This encryption method can make The encrypted image cannot be decrypted and is visually difficult to identify, so the protection of the user's private data can be achieved. In addition, by using the predetermined number of other images as noise images and restricting the sum of their weights to be smaller than the preset threshold, the encrypted images can be recognized algorithmically, and thus can be used for subsequent model training.

With further reference to FIG. 4, a flow 400 of yet another embodiment of a privacy-preserving image processing method is shown. The method is applied to the image processing terminal 100 shown in FIG. 1, and includes the following steps:

Step 401, taking any image in the predetermined biometric image set as the biometric image to be encrypted, and randomly selecting a predetermined number of images from the images in the biometric image set except the biometric image to be encrypted;

Step 402, obtaining a predetermined weight set;

Step 403, according to the weight set, assign weights to the biometric image to be encrypted and the selected predetermined number of images;

Step 404, according to the assigned weight, weighted summation is performed on the biometric image to be encrypted and the selected predetermined number of images to obtain a mixed image;

Step 405, subtracting a predetermined image template from the mixed image to obtain an encrypted image corresponding to the biometric image to be encrypted;

Step 406, generating a training sample including the encrypted image and the label of the biometric image to be encrypted.

Specifically, in step 401, the set of biometric images may be a set formed by biometric images of the same category. The biometric image may include any of the following images: a face image, a fingerprint image, a finger vein image, an iris image, a voiceprint image, a palmprint image, and the like.

For the explanation of steps 402-406, reference may be made to the relevant description in the corresponding embodiment of FIG. 2, and details are not repeated here.

Based on the content of step 401, it can be seen that the biometric image to be encrypted and the predetermined number of other images used to generate the mixed image in this embodiment are images of the same category, and both come from the same image set, that is, A predetermined set of biometric images as described above.

It can be seen from FIG. 4 that, compared with the embodiment corresponding to FIG. 2 , the image processing method for protecting privacy provided in this embodiment highlights the steps of extending the acquisition method of the mixed image, and generating an image including an encrypted image and a pending image. Steps to encrypt the training samples for the labels of the biometric images. Therefore, the solution described in this embodiment can enrich the functions of the image processing end. In addition, by defining that the image to be encrypted for generating the mixed image and the predetermined number of other images are derived from the same set of biometric images, the resultant mixed image can be made to look more blurred, so that the resultant encrypted image can have better quality. The encryption effect can strengthen the protection of the user's private data.

With further reference to FIG. 5, a flow 500 of one embodiment of a privacy-preserving model training method is shown. The model training method is applied to the model training terminal 101 shown in FIG. 1, and includes the following steps:

Step 501: Acquire a training sample set, wherein the training samples include encrypted images and labels, the encrypted images are obtained by subtracting a predetermined image template from their corresponding mixed images, and the mixed images are obtained by deducting their corresponding mixed images to be encrypted. Obtained after the weighted summation of the biometric image and a predetermined number of other images, the label is the label of the biometric image;

Step 502 , according to the training sample set, train the deep learning model to be trained to obtain a biometric identification model.

Next, step 501 and step 502 are described in detail.

In step 501, the encrypted image in the training sample may be generated by using the image processing method for protecting privacy provided by the embodiment shown in FIG. 2 or FIG. 4 respectively. The biometric image to be encrypted corresponding to the encrypted image may be any image in a predetermined set of biometric images. The predetermined number of other images acting as noise images may be images other than the biometric image in the set of biometric images.

The biometric image to be encrypted may include any of the following images: a face image, a fingerprint image, a finger vein image, an iris image, a voiceprint image, a palmprint image, and the like.

The image template can be an average across multiple images. The multiple images and the biometric image to be encrypted may be images of the same category or images of different categories. Since the image template generated based on the image belonging to the same category as the biometric image to be encrypted is applied to the encryption process of the biometric image to be encrypted, the obtained encrypted image can be made to look more blurred and the encryption effect is better. Preferably, the multiple images and the biometric image to be encrypted are images of the same category.

It should be noted that, step 501 may be performed by the model training end in response to receiving the model training request. The model training request may be triggered manually or automatically, which is not specifically limited here.

For example, if the model training request is triggered manually, the model training request may be sent by the model manager through the terminal device. If the model training request is automatically triggered, the model training request may be automatically sent by the image processing terminal (eg, the image processing terminal 100 shown in FIG. 1 ) after executing the image processing process (eg, the above-mentioned process 200 or the above-mentioned process 400 ) .

In practice, different methods can be used to obtain the training sample set.

For example, if the training samples in the training sample set have been generated in advance, the training sample set formed by at least one training sample may be obtained from a specific storage location.

For another example, if the training samples in the training sample set are not pre-generated, the encrypted image set and the labels of the biometric images to be encrypted corresponding to the encrypted images in the encrypted image set may be obtained first. The encrypted images in the encrypted image set are generated by using the image processing method for protecting privacy described in the embodiment corresponding to FIG. 2 . Then, for any encrypted image in the encrypted image set, the encrypted image and the label of the biometric image to be encrypted corresponding to the encrypted image can be formed into a training sample. In this way, the formed set of training samples can be used as a training sample set.

In step 502, the deep learning model to be trained may be an untrained or untrained model. It should be understood that the deep learning model in this embodiment may be any type of deep learning model, including but not limited to a convolutional neural network (Convolutional Neural Network, CNN), a recurrent neural network (Recurrent Neural Network, RNN), a deep belief network ( Deep Belief Network, DBN) and so on.

When training the deep learning model to be trained, different training methods can be used.

As an example, the encrypted image included in the training samples in the training sample set can be used as input, the label corresponding to the input encrypted image can be used as output, and the deep learning model to be trained can be trained to obtain the corresponding biometric recognition model.

As another example, for the training samples in the training sample set, image features may be extracted from the encrypted images of the training samples. After that, the image features of the encrypted images included in the training samples in the training sample set can be used as input, the labels corresponding to the input image features can be used as output, and the deep learning model to be trained can be trained to obtain a corresponding biometric recognition model. It should be understood that the label corresponding to the input image feature is the label in the training sample where the encrypted image to which the image feature belongs is located.

The model training process is exemplarily described below by taking the image feature of the encrypted image included in the training sample as an input as an example.

For example, the following model training operation can be performed: the image features of the encrypted images included in each training sample in the training sample set are sequentially input into the deep learning model to be trained, and the encrypted images included in each training sample in the training sample set are obtained. The prediction results corresponding to the image features; compare the labels corresponding to the image features of the encrypted images included in each training sample in the training sample set with the prediction results to obtain the prediction accuracy of the deep learning model after this round of training; determine the prediction accuracy Whether the rate is greater than the preset accuracy threshold; if it is greater than the preset accuracy threshold, the deep learning model after this round of training is used as the biometric recognition model.

In addition, if the prediction accuracy is not greater than the preset accuracy threshold, the deep learning model after this round of training can be used as the deep learning model to be trained, and the above model training operation can be continued.

It should be noted that the encrypted image looks blurry visually. Based on the above-mentioned training sample set, the biometric identification model obtained by training can enable the biometric identification model to better identify the blurred biometric image. Therefore, the biometric identification model in this embodiment has a higher recognition ability. performance.

Therefore, when the biometric recognition model obtained by training is subsequently applied to various biometric recognition scenarios, such as but not limited to payment scenarios based on biometric recognition, account login scenarios, security inspection scenarios, etc., it can help improve recognition Efficiency and recognition accuracy.

Continue to refer to FIG. 6 , which is a schematic diagram of an application scenario of the model training method for protecting privacy according to this embodiment. In this application scenario, the biometric image to be encrypted is a face image. The encrypted image is an encrypted face image. The model training end is the server end, and the model training end can execute the above-mentioned process 500 in real time.

Also, training samples are pre-generated and stored in storage location L1. This training sample includes encrypted face images and labels. The label is the label of the face image to be encrypted corresponding to the encrypted face image. In addition, the deep learning model to be trained is stored in storage location L2.

When a face recognition model needs to be trained, the model management personnel may, as shown in reference numeral 601, perform the model training operation through the terminal device used by the model manager. Among them, the model training operation points to storage location L1 and storage location L2. Afterwards, the terminal device may send a model training request to the model training end in response to the model training operation, as indicated by reference numeral 602 . The model training request may include addresses of the storage location L1 and the storage location L2. Then, the model training end may, as shown in reference numeral 603, obtain a training sample set formed by multiple training samples from the storage location L1, and obtain the deep learning model to be trained from the storage location L2, according to the model training request. Then, the model training end may, as shown in reference numeral 604, train the deep learning model to be trained according to the training sample set to obtain a face recognition model.

In this application scenario, the encrypted face image looks blurry visually. Based on a training sample set formed by a plurality of training samples containing encrypted face images, a face recognition model is obtained by training, so that the face recognition model can better recognize the blurred face images. Therefore, the face recognition model It has high recognition performance.

Therefore, when the trained face recognition model is subsequently applied to various face recognition scenarios, including but not limited to payment scenarios based on face recognition, account login scenarios, security inspection scenarios, etc., it can help improve recognition Efficiency and recognition accuracy.

It should be pointed out that those skilled in the art can obtain other types of biometric image implementations by analogy based on the content related to the face image shown in FIG. 6 , which are not listed one by one in this specification.

In the model training method for protecting privacy provided by this embodiment, a training sample set is obtained, wherein the training samples include encrypted images and labels, and the encrypted images are obtained by subtracting a predetermined image template from the corresponding mixed images. The image is obtained by the weighted summation of the corresponding biometric image to be encrypted and a predetermined number of other images, the label is the label of the biometric image, and then the deep learning model to be trained is trained according to the training sample set , obtain the biometric recognition model, and realize the model training based on the encrypted image.

Further referring to FIG. 7 , as an implementation of the methods shown in some of the above figures, this specification provides an embodiment of an image processing apparatus for protecting privacy, and the apparatus embodiment corresponds to the method embodiment shown in FIG. 2 . The apparatus can be applied to the image processing terminal 100 shown in FIG. 1 .

As shown in FIG. 7 , the image processing apparatus 700 for protecting privacy in this embodiment may include: an acquisition unit 701 and an encryption unit 702 . The obtaining unit 701 is configured to obtain a mixed image corresponding to the biometric image to be encrypted, wherein the mixed image is obtained by performing a weighted summation on the biometric image and a predetermined number of other images, and the predetermined number of The other images act as noise images, and the sum of their weights is less than the preset threshold; the encryption unit 702 is configured to subtract a predetermined image template from the mixed image to obtain an encrypted image corresponding to the biometric image.

In this embodiment, for the specific processing of the obtaining unit 701 and the encryption unit 702 and the technical effects brought by them, reference may be made to the relevant descriptions of steps 201 and 202 in the corresponding embodiment of FIG. 2 , which are not repeated here.

In some optional implementations of this embodiment, the biometric image may have a corresponding label; and the above-mentioned apparatus 700 may further include: a generating unit (not shown in the figure) configured to generate an encrypted image and the label training samples.

In some optional implementations of this embodiment, the biometric image may include any of the following images: a face image, a fingerprint image, a finger vein image, an iris image, a voiceprint image, a palmprint image, and the like.

In some optional implementations of this embodiment, the biometric image to be encrypted is any image in a predetermined biometric image set, and the predetermined number of other images are biometric images other than the biometric image set to be encrypted in the biometric image set Images other than feature images.

In some optional implementations of this embodiment, the acquiring unit 701 may include: a selection subunit (not shown in the figure) configured to select images other than the biometric image to be encrypted from the biometric image set , randomly select a predetermined number of images; assign a subunit (not shown in the figure), which is configured to assign weights to the biometric images to be encrypted and the selected predetermined number of images; an acquisition subunit (not shown in the figure) , which is configured to perform a weighted summation of the biometric image to be encrypted and a predetermined number of images selected according to the assigned weight to obtain a mixed image.

In some optional implementations of this embodiment, the assigning subunit may be further configured to: acquire a predetermined weight set; and assign a weight to the biometric image to be encrypted and the selected predetermined number of images according to the weight set.

In some optional implementations of this embodiment, the above-mentioned apparatus 700 may further include: an image template determination unit (not shown in the figure), configured to acquire multiple images; calculate an average value among the multiple images ; this average is determined as the image template.

In some optional implementations of this embodiment, the above-mentioned multiple images and the biometric image to be encrypted are images of the same category.

In the image processing apparatus for protecting privacy provided by this embodiment, the obtaining unit obtains a mixed image corresponding to the biometric image to be encrypted, wherein the mixed image is obtained by performing weighted summation on the biometric image and a predetermined number of other images. obtained, a predetermined number of other images are used as noise images, and the sum of their weights is less than the preset threshold, and then the mixed image is subtracted by the encryption unit from the predetermined image template to obtain the encrypted image corresponding to the biometric image. The encryption method can make the encrypted image undecipherable and visually difficult to identify, thus protecting the user's private data. In addition, by using the predetermined number of other images as noise images and restricting the sum of their weights to be smaller than the preset threshold, the encrypted images can be recognized algorithmically, and thus can be used for subsequent model training.

With further reference to FIG. 8 , as an implementation of the methods shown in some of the above figures, the present specification provides an embodiment of a privacy-protecting model training apparatus. The apparatus embodiment corresponds to the method embodiment shown in FIG. 5 . The apparatus can be applied to the model training terminal 101 as shown in FIG. 1 .

As shown in FIG. 8 , the model training apparatus 800 for protecting privacy in this embodiment includes: an acquisition unit 801 and a training unit 802 . The obtaining unit 801 is configured to obtain a training sample set, wherein the training samples include encrypted images and labels, the encrypted images are obtained by subtracting a predetermined image template from their corresponding mixed images, and the mixed images are obtained by The corresponding biometric image to be encrypted and a predetermined number of other images are obtained after weighted summation, and the label is the label of the biometric image; the training unit 802 is configured to perform a deep learning model to be trained according to the training sample set. Training to obtain a biometric recognition model.

In this embodiment, for the specific processing of the acquisition unit 801 and the training unit 802 and the technical effects brought by them, reference may be made to the relevant descriptions of steps 501 and 502 in the corresponding embodiment of FIG.

In some optional implementations of this embodiment, the biometric image to be encrypted may be any image in a predetermined biometric image set, and the predetermined number of other images may be other images in the biometric image set except to be encrypted images other than biometric images.

In some optional implementations of this embodiment, the biometric image to be encrypted may include any of the following images: a face image, a fingerprint image, a finger vein image, an iris image, a voiceprint image, a palmprint image, etc. .

In some optional implementations of this embodiment, the image template may be an average value among multiple images.

In some optional implementations of this embodiment, the above-mentioned multiple images and the biometric image to be encrypted may be images of the same category.

The privacy-protecting model training device provided in this embodiment acquires a training sample set through an acquisition unit, wherein the training samples include encrypted images and labels, and the encrypted images are obtained by subtracting a predetermined image template from the corresponding mixed image. , the mixed image is obtained by the weighted summation of the corresponding biometric image to be encrypted and a predetermined number of other images, the label is the label of the biometric image, and then through the model training unit according to the training sample set, to be trained The deep learning model is trained to obtain a biometric recognition model, and the model training based on encrypted images is realized.

The embodiments of this specification also provide a computer-readable storage medium on which a computer program is stored, wherein, when the computer program is executed in a computer, the computer is made to execute the privacy-protecting images shown in the above method embodiments respectively A processing method or a privacy-preserving model training method.

Embodiments of the present specification further provide a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the protections shown in the above method embodiments are implemented respectively. Privacy-preserving image processing methods or privacy-preserving model training methods.

The embodiments of this specification also provide a computer program product, which, when executed on a data processing device, enables the data processing device to implement the privacy-preserving image processing method or the privacy-preserving model training method respectively shown in the above method embodiments.

Those skilled in the art should realize that, in the above one or more examples, the functions described in the various embodiments disclosed in this specification may be implemented by hardware, software, firmware or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium.

In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The specific embodiments described above further describe in detail the purposes, technical solutions and beneficial effects of the various embodiments disclosed in this specification. It should be understood that the above description is only for the various embodiments disclosed in this specification The specific description is only for the specific implementation, and is not intended to limit the protection scope of the multiple embodiments disclosed in this specification. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the multiple embodiments disclosed in this specification are made. , all should be included within the protection scope of the multiple embodiments disclosed in this specification.

Claims (17)

1. An image processing method for protecting privacy, comprising:

acquiring a mixed image corresponding to a biological characteristic image to be encrypted, wherein the mixed image is obtained by weighting and summing the biological characteristic image and a preset number of other images, the preset number of other images are used as noise images, and the sum of weights of the other images is smaller than a preset threshold value;

and subtracting a preset image template from the mixed image to obtain an encrypted image corresponding to the biological characteristic image.

2. The method of claim 1, wherein the biometric image has a corresponding label; and

the method further comprises the following steps:

generating a training sample comprising the encrypted image and the label.

3. The method of claim 1, wherein the biometric image comprises any one of: face image, fingerprint image, finger vein image, iris image.

4. The method according to claim 1, wherein the biometric image to be encrypted is any one of a predetermined set of biometric images, and the predetermined number of other images are images of the set of biometric images other than the biometric image to be encrypted.

5. The method of claim 4, wherein the obtaining a blended image corresponding to a biometric image to be encrypted comprises:

randomly selecting a preset number of images from the images in the biological characteristic image set except the biological characteristic image to be encrypted;

distributing weights to the biological characteristic image to be encrypted and the selected preset number of images;

and according to the distributed weight, carrying out weighted summation on the biological characteristic image to be encrypted and the selected preset number of images to obtain the mixed image.

6. The method according to claim 5, wherein the assigning weights to the biometric image to be encrypted and the selected predetermined number of images comprises:

acquiring a preset weight set;

and according to the weight set, distributing weights to the biological characteristic image to be encrypted and the selected preset number of images.

7. The method according to one of claims 1 to 6, wherein prior to said obtaining a blended image corresponding to a biometric image to be encrypted, the method further comprises:

acquiring a plurality of images;

calculating an average value between the plurality of images;

determining the average as the image template.

8. The method according to claim 7, wherein the plurality of images and the biometric image to be encrypted are images of the same category.

9. A privacy preserving model training method, comprising:

acquiring a training sample set, wherein the training sample set comprises an encrypted image and a label, the encrypted image is obtained by subtracting a preset image template from a corresponding mixed image, the mixed image is obtained by weighting and summing a corresponding biological characteristic image to be encrypted and a preset number of other images, and the label is the label of the biological characteristic image;

and training the deep learning model to be trained according to the training sample set to obtain a biological feature recognition model.

10. The method according to claim 9, wherein the biometric image to be encrypted is any one of a predetermined set of biometric images, and the predetermined number of other images are images of the set of biometric images other than the biometric image to be encrypted.

11. The method according to claim 9, wherein the biometric image to be encrypted comprises any one of: face image, fingerprint image, finger vein image, iris image.

12. The method according to one of claims 9 to 11, wherein the image template is an average value between a plurality of images.

13. The method according to claim 12, wherein the plurality of images and the biometric image to be encrypted are images of the same category.

14. An image processing apparatus that protects privacy, comprising:

an acquisition unit configured to acquire a mixed image corresponding to a biometric image to be encrypted, wherein the mixed image is obtained by weighted summation of the biometric image and a predetermined number of other images, the predetermined number of other images serving as noise images, and a sum of weights thereof is smaller than a preset threshold;

and the encryption unit is configured to subtract a predetermined image template from the mixed image to obtain an encrypted image corresponding to the biological characteristic image.

15. A privacy preserving model training apparatus comprising:

an acquisition unit configured to acquire a training sample set, wherein a training sample includes an encrypted image and a label, the encrypted image is obtained by subtracting a predetermined image template from a corresponding mixed image, the mixed image is obtained by weighted summation of a biometric image to be encrypted and a predetermined number of other images, and the label is a label of the biometric image;

and the training unit is configured to train the deep learning model to be trained according to the training sample set to obtain a biological feature recognition model.

16. A computer-readable storage medium, on which a computer program is stored, wherein the computer program, when executed in a computer, causes the computer to perform the method of any of claims 1-13.

17. A computing device comprising a memory and a processor, wherein the memory has stored therein executable code that when executed by the processor implements the method of any of claims 1-13.

Images