TWI892155B - Artificial intelligence apparatus - Google Patents

Abstract

Aspects of the present disclosure provide an apparatus that can execute an artificial intelligence (AI) model with IO changing. For example, the apparatus can include a first secured processor, a secured application embedded in the first secured processor and associated with an AI model, a secured memory configured to store an AI executable binary associated with the AI model, a second secured processor configured to execute the AI executable binary, a sub-system configured to trigger IO changing and trigger the second secured processor to execute the AI executable binary, IO meta data stored in the secured memory, an IO verifier configured to verify IO changing by determining the IO meta data, and an IO pre-fire module configured to patch the IO changing to the AI executable binary running on the second secured processor when the IO verifier determines that the IO changing matches the IO meta data.

Description

artificial intelligence devices

This invention relates to neural networks (NNs), and more specifically, to the security of always-on artificial intelligence (AI).

The background description provided herein is for the purpose of generally presenting the context of the present invention. The work of the presently mentioned inventors (to the extent that such work is described in this background section) and aspects of the description (which are not otherwise described as prior art at the time of filing) are neither explicitly nor implicitly admitted to being prior art with respect to the present invention.

Integrating machine learning (ML) capabilities into hardware pathways is a trend, and flexible and scalable designs are needed to reduce the design complexity of deep neural network (DNN) accelerator implementations.

Aspects of the present invention provide a device that can execute an artificial intelligence (AI) model under input/output (IO) changes. For example, the device may include a first security processor and a security application embedded in the first security processor. The security application may be associated with the artificial intelligence (AI) model. The device may also include a secure memory coupled to the first security processor. The secure memory may be configured to store an AI executable binary associated with the AI model. The device may also include a second security processor coupled to the secure memory. The second security processor may be configured to execute the AI executable binary stored in the secure memory. The device may also include a subsystem coupled between the first security processor and the second security processor. The subsystem can be configured to trigger IO changes and trigger the second security processor to execute the AI executable binary file stored in the secure memory. The device may also include IO metadata stored in the secure memory. The device may also include an IO verifier coupled to the subsystem and the secure memory. The IO verifier can be configured to verify the IO changes by determining the IO metadata. The device may also include an IO pre-launch module coupled to the IO verifier. The IO pre-launch module can be configured to: when the IO verifier determines that the IO changes match the IO metadata, patch the IO changes to the AI executable binary file running on the second security processor. In one embodiment, the IO verifier can be embedded in the second security processor. In another embodiment, the IO pre-issuance module may be embedded in the second security processor.

In one embodiment, the IO metadata may include an IO address range, the IO variation may include an IO address, the IO verifier may verify whether the IO address is within the IO address range, and when the IO verifier determines that the IO address is within the IO address range, the IO pre-release module may patch the IO address to the AI executable binary file running on the second secure processor. In another embodiment, the IO metadata may include a plurality of different resolutions, the IO variation may include a resolution variation, the IO verifier may verify whether the resolution variation matches any one of the different resolutions specified in the IO metadata, and when the IO verifier determines that the resolution variation matches one of the different resolutions, the IO pre-release module may patch the resolution variation to the AI executable binary file running on the second secure processor.

In one embodiment, the device may further include a secure operating system (OS) embedded in the first secure processor, the secure OS being configured to provide a trusted execution environment (TEE) in which secure applications are protected. In another embodiment, the secure memory and the second secure processor may be protected by a first firewall. In some embodiments, the subsystem may be protected by a second firewall that is different from the first firewall. In various embodiments, the first firewall may provide a higher level of security than the second firewall.

In one embodiment, the device may further include an image signal processor (ISP) coupled to the secure memory. The ISP may be configured to process images and store the processed images in the secure memory. In another embodiment, the device may further include a facial biometric pattern that is secure within the TEE. In some embodiments, the second secure processor may execute an AI executable binary to determine whether any of the processed images matches the facial biometric pattern.

In one embodiment, the first security processor may include a secure central processing unit (CPU). In another embodiment, the second security processor may include a secure deep learning accelerator (DLA). In some embodiments, the DLA may include an accelerated processing unit (APU).

Note that this summary does not specify every embodiment and/or incremental novel aspect of the present invention or the claimed invention. Rather, this summary merely provides an initial discussion of various embodiments and corresponding novelties compared to conventional techniques. For additional details and/or possible perspectives of the present invention and its embodiments, the reader is referred to the detailed description of the present invention and the corresponding figures, as discussed further below.

Ambient intelligence (AmI) (e.g., environmental sensing) is proposed to enhance the way the environment and people interact with each other. Specifically, AmI represents intelligent computing that does not require explicit input and output devices; instead, various sensors (e.g., accelerometers, global positioning systems (GPS), microphones, cameras, etc.) and processors can be embedded in everyday electronic devices (e.g., mobile phones) to collect and process contextual information using artificial intelligence (AI) techniques, for example, to interpret the state of the environment and user needs.

For example, Google's "Personal Safety" app has the ability to detect whether an individual has been in a car accident and, if so, to place an emergency call on the individual's behalf. As another example, AI and machine learning (ML) algorithms (or models) installed in cameras can recognize the face of their owner, for example, by determining whether an image captured by the camera matches the owner's facial biometric pattern.

For crash sensing to be truly effective, mobile phones need to be able to detect crashes at all times. For example, this can be done by continuously polling the accelerometer and microphone, then processing the collected data (e.g., by running always-on artificial intelligence (AI)) to determine if a crash has occurred. However, this constant on-line sensing consumes a significant amount of the phone's precious battery.

A sensor hub (or context hub) is a low-power subsystem (e.g., a processor) that can be designed to process and interpret data collected from sensors and wake up the main application processor (AP) to take action. For example, after processing and interpreting the collected data and determining that a car accident has occurred, the sensor hub can wake up the AP, and the mobile phone can call emergency services.

Figure 1 is a functional block diagram of an AmI-enabled device 100 (e.g., a mobile phone). The device 100 may include an access point (AP) 110, a low-power subsystem 120 (e.g., a sensor hub) coupled to the AP 110, a signal processor 130 (e.g., a low-power image signal processor (ISP)) coupled to the sensor hub 120, a processor 140 (e.g., an AI accelerator (e.g., a deep learning accelerator (DLA) or an accelerated processing unit (APU)) coupled to the sensor hub 120, and a memory 150 coupled to the sensor hub 120, the ISP 130, and the APU 140.

AP 110 can enable environmental sensing functions, such as always-on vision (AOV) client 111, and load AI model 122 into sensor hub 120 to offload large amounts of processed data collected from embedded sensors (e.g., a camera (not shown)) to sensor hub 120. In sensor hub 120, camera driver 123 can drive ISP 130 based on AOV client 111 to process images captured by the camera (e.g., a user's face) and send the processed images to camera input 151 of memory 150. A software development kit (SDK) 121 (e.g., an AI inference SDK) can drive the APU 140 to execute the AI model 122 on the processed image. For example, the APU 140 can use the AI executable binary file corresponding to the AI model 122 to execute the AI model 122 on the processed image sent from the camera input 151 and generate an output 152. For example, the output 152 can be a classification result related to whether the captured customer's face matches the facial biometric feature pattern of the owner's face.

In the device 100, the sensor hub 120 can provide secure computing with limited flexibility. For example, when the mobile phone is running, the sensor hub 120 can protect fixed functions and security during the secure boot phase. Environmental sensing continuously senses data, which includes user privacy such as voice, vision, surroundings, location, etc. If such data and the AI model 122 loaded into the sensor hub 122 are not well protected, they are likely to be attacked, stolen, or tampered with. In addition, the processed image on which the APU 140 executes the AI model 122 may not be captured from a camera, but may be sent from the outside by an attacker.

A firewall is a network security device that monitors all incoming and outgoing traffic and accepts, denies, or discards it based on a defined set of security rules. For example, a firewall can control network access by monitoring incoming and outgoing data packets at any Open Systems Interconnection (OSI) layer up to the application layer and allowing or stopping them based on the source and destination IP addresses, protocols, ports, and the packet's history in a state table to protect the packets from attacks, eavesdropping, or tampering. Firewalls can be hardware-based or software-based.

Figure 2 is a functional block diagram of an AmI-enabled device 200 (e.g., a mobile phone). Device 200 differs from device 100 in that, in device 200, sensor hub 120 and memory 150 are well protected (e.g., via firewall 290) (shown with a black background). Therefore, the sensed data and AI model 122 are secure, and attackers cannot send images to memory 150. However, AI model 122 needs to be restored or updated from time to time (e.g., with a new AI model 112) to continuously enhance performance or security based on device training or the internet. The AP 110 cannot restore or update the AI model 122 stored in the sensor hub 120 because the sensor hub 120 is protected by the firewall 290 and the AP 110 does not have permission to access the sensor hub 120.

3 is a functional block diagram of an AmI-enabled device 300 (e.g., a mobile phone). The device 300 may include a secure operating system (OS) 360. The secure OS 360 may provide a trusted execution environment (TEE) 393 (shown against a black background) for Android, where code and data (e.g., trusted applications (TAs)) may be protected in terms of confidentiality and integrity. The secure OS 360 may run on the same processor as Android (e.g., AP 110), but may be isolated by hardware and software from the rest of the system running a rich OS within a rich execution environment (REE).

The AI model 322 may be loaded into the TEE 393 provided by the secure OS 360, and an AI executable binary file 381 and control flow (including an AI session 327, such as an identifier (ID) of the AI model 322, and an AI executive 328) of the AI model 322 may be prepared, collectively referred to as AI preparation 361. The AI executable binary file 381 may be sent to the secure memory 380, and the AI session 327 and the AI executive 328 may be sent to the low-power subsystem 320, such as a sensor hub. A processor 340, such as an AI accelerator (such as a DLA, such as an APU), may execute the AI executable binary file 381 by determining the AI session 327 and the AI executive 328. In one embodiment, memory 380 and APU 340 are also secured (shown with a black background) (e.g., via firewall 391) to protect AI executable binary 381 from being attacked, stolen, or tampered with. In the example embodiment shown in FIG3 , sensor hub 320 is not secured because it only provides control flow for AI model 322 and does not involve any sensor data. In some embodiments, sensor hub 320 can also be secured (e.g., via a firewall). For example, this firewall may provide a lower level of security than firewall 391 because AI session 327 and AI executive 328 are less critical than AI executable binary 381.

In one embodiment, data (e.g., facial biometric pattern 363) is also secure within TEE 393 and is downloaded and stored in secure memory 380. For example, APU 340 may utilize AI executable binary 381 to execute AI model 322 on a processed image (e.g., a user's face) sent from ISP 130 (as shown in FIG1 ) and generate an output, e.g., a classification result, associated with whether the captured customer face matches the owner's face (i.e., facial biometric pattern 363).

Due to various hardware implementations (e.g., the secure memory 380 and the AI accelerator 340 of the device 300), input/output (IO) data and information related thereto (e.g., the address of the IO data) may need to be modified in order to run on the AI model 322 deployed to the AI accelerator 340. For example, in a scenario where multiple image frames are captured to improve performance, the security camera may include a ring buffer (or loop buffer) configured to serialize the captured image frames. Whenever an image frame is consumed from the annular buffer, pointers to the start and end of the image frame in the annular buffer are updated, and the address input to the AI model 322 changes. As another example, in a scenario where the AI model 322 is used to recognize patterns and includes a plurality of connected subgraphs (e.g., a feature extraction and detection subgraph and a recognition subgraph), if the APU 340 has limited capabilities, the patterns input to and detected by the feature extraction and detection subgraph may be recognized by the recognition subgraph using different (e.g., high or low) resolutions based on their sizes.

However, when IO data and/or information related thereto changes, AI executor 328 cannot modify AI executable binary file 381 because AI executable binary file 381 is protected in secure memory 380 and AI accelerator 340. For example, as shown in device 400 of FIG4 , IO pre-fire module 420 embedded in TEE 393 provided by secure OS 360 cannot patch IO changes (e.g., the address of IO 410) to AI executable binary file 381 loaded into AI accelerator 340, and AI executor 328 cannot modify AI executable binary file 381. As another example, as shown in device 500 of FIG. 5 , it includes multiple isolated virtual machines (VMs). A first VM (VM0) 501 has higher privileges than an Android system 502 and a second VM (VM1) 503. Both the Android system 502 and the second VM (VM1) 503 are connected to an AI accelerator 340. An IO pre-launch module 520 embedded in VM0 501 cannot patch IO changes (e.g., the address of IO 510) to an AI executable binary file 381 prepared by VM0 501 and loaded into the AI accelerator 340. Furthermore, the AI executor 528 of the Android system 502 and the AI executor 538 of VM1 503 cannot modify the AI executable binary file 381.

Figure 6 is a functional block diagram of an AmI-enabled device 600 (e.g., a mobile phone) according to some embodiments of the present invention. The device 600 can execute an AI model under IO changes. Compared to the device 300, the device 600 may further include IO metadata 640, an IO validator/checker 630, and an IO pre-launch module 620. In one embodiment, the IO metadata 640 may be provided by the secure OS 360, while preparing the AI executable binary file 381 and control flow (including the AI session 327 and the AI executor 328) of the AI model 322, collectively referred to as AI preparation 361, and sending it to the secure memory 380 and embedding it in the secure memory 380. In the example embodiment of FIG6 , because the secure memory 380 is protected (e.g., via a firewall 391), the IO metadata 640 can also be protected from attack, theft, or tampering. In another embodiment, the IO validator/checker 630 and the IO pre-launch module 620 can be embedded in the AI accelerator 340 and can also be protected (e.g., via a firewall 391). In one embodiment, the secure OS 360 or VM (e.g., VM0 501) can be embedded in the TEE 393. In another embodiment, the subsystem 320 can be a sensor hub or a VM (e.g., VM1 503). In the example embodiment of FIG6 , the subsystem 320 is not protected. In some embodiments, the subsystem 320 can also be protected (e.g., via a firewall). For example, a firewall may provide a lower level of security than firewall 391 because the AI session 327 and AI executor 328 are not as important as the AI executable binary 381.

In one embodiment, the IO metadata 640 may include IO address patching information and/or a valid/accessible IO (address) range. For example, the IO metadata 640 may include pointers (or addresses) to the start and end of the circular buffer of the security camera. In another embodiment, the IO validator/checker 630 may verify/check whether the IO change (e.g., IO address 610) is within the IO address range specified in the IO metadata 640, and if the IO address 610 is within the IO address range, the IO pre-release module 620 may patch the IO address 610 to the AI executable binary file 381. For example, because the subsystem 320 is not well protected in the example embodiment, the IO address 610 may be provided by a malicious entity (e.g., a hacker). In this scenario, the IO verifier/checker 630 can verify/check the IO address 610 and determine that the IO address 610 is not within the IO address range. Therefore, the IO pre-issue module 620 does not patch the unverified IO address 610 to the AI executable binary file 381 allocated to and running on the AI accelerator 340. As another example, when the IO verifier/checker 630 verifies/checks the IO address 610 and determines that the IO address 610 is within the IO address range, the IO pre-issue module 620 can patch the IO address 610 to the AI executable binary file 381 running on the AI accelerator 340. As a result, the APU 340 can apply the dynamic shape information to the AI executable binary file 381 and perform inference.

Figure 7 is a functional block diagram of an AmI-enabled device 700 (e.g., a mobile phone) according to some embodiments of the present invention. The device 700 can execute an AI model in the presence of IO changes. Compared to the device 300, the device 700 can also include IO metadata 740, a (shape) IO validator 730, and a (shape) IO pre-release module 720. In one embodiment, the IO metadata 740 can be provided while the AI executable binary file 381 is prepared, and the IO metadata 740 is sent and embedded in the secure memory 380. Because the secure memory 380 is protected (e.g., via a firewall 391), the IO metadata 740 can also be protected from being attacked, stolen, or tampered with. In another embodiment, the (shape) IO validator 730 and the (shape) I/O pre-module 720 can be embedded in the AI accelerator 340 and can also be protected (e.g., via the firewall 391).

In one embodiment, the IO metadata 740 may include a number of different resolutions, such as low resolution and high resolution. In another embodiment, the (shape) IO verifier 730 may verify whether the control 710 that triggers the resolution change matches any of the different resolutions specified in the IO metadata 740, and if the resolution change matches any of the different resolutions specified in the IO metadata 740, the (shape) IO pre-release module 720 may patch the resolution change to the AI executable binary 381. For example, because the subsystem 320 is not well protected in the example embodiment, the resolution change may be provided by a malicious entity (e.g., a hacker). In such a scenario, the (shape) IO verifier 730 may verify the resolution change and determine that the resolution change does not match any of the different resolutions. Therefore, the (shape) I/O pre-issue module 720 does not patch the unverified resolution change to the AI executable binary file 381 that is distributed and runs on the AI accelerator 340. As another example, when the (shape) IO verifier 730 verifies the resolution change and determines that the resolution change matches one of the different resolutions specified in the IO metadata 740, the (shape) I/O pre-issue module 720 may patch the resolution change to the AI executable binary file 381 that runs on the AI accelerator 340. Therefore, the APU 340 can apply the dynamic shape information to the AI executable binary file 381 and perform inference.

While aspects of the present invention have been described with reference to specific embodiments provided as examples, these examples are susceptible to alterations, modifications, and variations. Therefore, the embodiments described herein are intended to be illustrative rather than restrictive. Certain variations are possible without departing from the scope of the claims set forth below.

110: Main Application Processor (AP) 111: Always-On Vision (AOV) Client 112: New AI Models 120: Sensor Hub 121: Software Development Kit (SDK) 123: Camera Driver 130: Image Signal Processor (ISP) 141: Working Buffer 150: Memory 151: Camera Input 152: Output 320: Low-Power Subsystem (Sensor Hub) 328: AI Executive 360: Secure OS 361: AI Readiness 363: Facial Biometric Mode 380: Secure Memory 381: Binary Files 393: Trusted Execution Environment (TEE) 501: First Virtual Machine (VM0) 502: Android System 503: Second Virtual Machine (VM1) 610: I/O Address 630: I/O Verifier/Inspector 710: Control 720: (Shape) I/O Pre-Module 730: (Shape) I/O Verifier 122,322: AI Model 140,340: Accelerated Processing Unit (APU) 290,391: Firewall 410,510: I/O 528,538: AI Executor 640,740: I/O Metadata 327,527,537: AI Session 420, 520, 620: IO pre-release modules 100, 200, 300, 400, 500, 600, 700: devices

Various embodiments of the present invention, presented as examples, will be described in detail with reference to the following figures, in which like numerals refer to like elements, and in which: Figure 1 is a functional block diagram of a first ambient intelligence (AmI)-enabled device; Figure 2 is a functional block diagram of a second AmI-enabled device; Figure 3 is a functional block diagram of a third AmI-enabled device; Figure 4 is a functional block diagram of a fourth AmI-enabled device; Figure 5 is a functional block diagram of a fifth AmI-enabled device; Figure 6 is a functional block diagram of a first AmI-enabled device according to some embodiments of the present invention; and Figure 7 is a functional block diagram of a second AmI-enabled device according to some embodiments of the present invention.

100: Device

110: Main Application Processor (AP)

111: Always Online Visual (AOV) Client

112: New AI Model

120: Sensor Hub

121: Software Development Kit (SDK)

122: AI Model

123:Camera drive

130: Image Signal Processor (ISP)

140: Accelerated Processing Unit (APU)

141: Work Buffer

150: Memory

151: Camera input

152: Output

Claims (13)

An artificial intelligence device comprises: a first security processor; a security application embedded in the first security processor, the security application being associated with an artificial intelligence (AI) model; a secure memory coupled to the first security processor, the secure memory being configured to store an AI executable binary file associated with the AI model; a second security processor coupled to the secure memory, the second security processor being configured to execute the AI executable binary file stored in the secure memory; A subsystem coupled between the first and second secure processors, the subsystem configured to trigger an input/output (IO) change and trigger the second secure processor to execute the AI executable binary file stored in the secure memory; IO metadata stored in the secure memory; An IO verifier coupled to the subsystem and the secure memory, the IO verifier configured to verify the IO change using the IO metadata; and An IO pre-launch module is coupled to the IO verifier and configured to, when the IO verifier determines that the IO change matches the IO metadata, patch the IO change to the AI executable binary file running on the second secure processor. The artificial intelligence device of claim 1, wherein the IO metadata includes an IO address range, the IO variation includes an IO address, the IO verifier verifies whether the IO address is within the IO address range, and when the IO verifier determines that the IO address is within the IO address range, the IO pre-launch module patches the IO address to the AI executable binary file running on the second secure processor. The artificial intelligence device of claim 1, wherein the IO metadata includes a plurality of different resolutions, the IO change includes a resolution change, the IO verifier verifies whether the resolution change matches any one of the different resolutions specified in the IO metadata, and when the IO verifier determines that the resolution change matches one of the different resolutions, the IO pre-launch module patches the resolution change to the AI executable binary file running on the second secure processor. The artificial intelligence device of claim 1, wherein the IO verifier is embedded in the second security processor. The artificial intelligence device of claim 1, wherein the IO pre-transmission module is embedded in the second security processor. As in claim 1, the artificial intelligence device further includes a secure operating system (OS) embedded in the first secure processor, wherein the secure OS is configured to provide a trusted execution environment (TEE), within which the secure application is protected. An artificial intelligence device as claimed in claim 6, wherein the secure memory and the second secure processor are protected by a first firewall. An artificial intelligence device as claimed in claim 7, wherein the subsystem is protected by a second firewall that is different from the first firewall. An artificial intelligence device as claimed in claim 8, wherein the first firewall provides a higher level of security than the second firewall. The artificial intelligence device of claim 6, further comprising: an image signal processor (ISP), coupled to the secure memory, the ISP configured to process images and store the processed images in the secure memory, and a facial biometric pattern, the facial biometric pattern being secure within the TEE; wherein the second secure processor executes the AI executable binary file to determine whether any of the processed images matches the facial biometric pattern. The artificial intelligence device of claim 1, wherein the first security processor includes a security central processing unit (CPU). The artificial intelligence device of claim 1, wherein the second security processor includes a secure deep learning accelerator (DLA). The artificial intelligence device of claim 12, wherein the DLA includes an accelerated processing unit (APU).