HK1246905B - Behavioral malware detection using an interpreter virtual machine - Google Patents

Description

Behavioral malware detection using interpreter virtual machines

Background Art

The present invention relates to systems and methods for protecting computer systems from malware.

Malicious software (also known as malware) affects a large number of computer systems worldwide. In its many forms, such as computer viruses, worms, Trojans, unsolicited adware, ransomware, and spyware, malware poses a serious risk to millions of computer users, exposing them to loss of data and sensitive information, identity theft, and lost productivity. Malware may also display material that some users may find obscene, excessively violent, harassing, or otherwise objectionable.

Security software can be used to detect malware that infects a user's computer system and, in addition, to remove or otherwise disable such malware. Several malware detection techniques are known in the art. Some rely on matching malware agent code snippets with a library of malware-indicative signatures. Other conventional methods detect a set of malware-indicative behaviors in a malware agent.

Malware evolves rapidly, and to stay ahead of detection algorithms, security software providers typically adjust detection methods and/or parameters on a scale of minutes to hours, for example by retraining classifiers to detect new malware versions. This retraining is typically computationally expensive. Furthermore, introducing new behavioral detection algorithms or even changing existing ones typically requires extensive testing and recompiling the source code of the corresponding security software. Only then can the new software version be delivered to clients, for example as a software update. In contrast, in signature-based detection systems, new malware signatures can be easily added to the existing signature set.

Therefore, there is great interest in developing anti-malware solutions that can rapidly update algorithms and/or parameters to keep pace with the ever-changing nature of malware.

Summary of the Invention

According to one aspect, a client system includes at least one hardware processor configured to form a routine scheduler, a bytecode interpretation virtual machine, and a behavior evaluation engine. The routine scheduler is configured to select an anti-malware bytecode routine from a plurality of anti-malware bytecode routines for execution in response to detecting the occurrence of a triggering event, the anti-malware bytecode routine being selected based on the triggering event. The occurrence of the triggering event is caused by a monitored process executing within the client system. The bytecode interpretation virtual machine is configured to execute the anti-malware bytecode routine to determine whether the occurrence of the triggering event indicates malware, wherein executing the anti-malware bytecode routine includes: interpreting a set of bytecode instructions of the anti-malware bytecode routine into a native processor instruction sequence, and executing the native processor instruction sequence. The behavior evaluation engine is configured to determine whether the client system includes malware based on the result of the bytecode interpretation virtual machine executing the anti-malware bytecode routine.

According to another aspect, a computer-implemented method includes, in response to detecting an occurrence of a triggering event, selecting, from a plurality of bytecode routines, an anti-malware bytecode routine based on the triggering event, the anti-malware bytecode routine configured to determine whether the occurrence of the triggering event indicates malware. The triggering event is caused by a monitored process executing within the client system. The method further includes, in response to selecting the anti-malware bytecode routine, interpreting, in response to selecting the anti-malware bytecode routine, a set of bytecode instructions of the anti-malware bytecode routine into a native processor instruction sequence and executing the native processor instruction sequence. The method further includes, in response to executing the native processor instruction sequence, determining, in response to the at least one hardware processor of the client system, whether the client system includes malware.

According to another aspect, a non-transitory computer-readable medium stores a computer program that, when executed by at least one hardware processor of a client system, causes the client system to form a routine scheduler, a bytecode interpretation virtual machine, and a behavior evaluation engine. The routine scheduler is configured to, in response to detecting the occurrence of a triggering event, select an anti-malware bytecode routine from a plurality of anti-malware bytecode routines for execution based on the triggering event, the anti-malware bytecode routine being configured to determine whether the occurrence of the triggering event indicates malware. The triggering event is caused by a monitored process executing within the client system. The bytecode interpretation virtual machine is configured to execute the anti-malware bytecode routine, wherein executing the anti-malware bytecode routine includes interpreting a set of bytecode instructions of the anti-malware bytecode routine into a native processor instruction sequence and executing the native processor instruction sequence. The behavior evaluation engine is configured to determine whether the client system includes malware based on the result of the bytecode interpretation virtual machine executing the anti-malware bytecode routine.

According to another aspect, a non-transitory computer-readable medium stores an anti-malware bytecode routine that is executable by a bytecode interpretation virtual machine formed on a client system including at least one hardware processor, wherein executing the anti-malware bytecode routine causes the at least one hardware processor to determine whether an occurrence of a triggering event within the client system indicates malware. Executing the anti-malware bytecode routine includes interpreting a set of bytecode instructions of the anti-malware bytecode routine into a native processor instruction sequence, and executing the native processor instruction sequence. The client system further includes a routine scheduler and a behavior evaluation engine. The routine scheduler is configured to, in response to detecting the occurrence of the triggering event, select the anti-malware bytecode routine from the plurality of bytecode routines based on the triggering event. The behavior evaluation engine is configured to determine whether the client system includes malware based on a result of execution of the anti-malware bytecode routine by the bytecode interpretation virtual machine.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and advantages of the present invention will be better understood by reading the following detailed description and referring to the accompanying drawings, in which:

FIG. 1 shows an exemplary client system for protecting against computer security threats (eg, malware) according to some embodiments of the present invention.

FIG2 shows an exemplary hardware configuration of a client computer system according to some embodiments of the present invention.

FIG3 illustrates an exemplary structure of a security application according to some embodiments of the present invention.

Figure 4 illustrates an exemplary execution flow of a set of processes in an environment. Solid arrows indicate the flow without event interceptors. Dashed arrows indicate modifications to the execution flow introduced by multiple event interceptors operating according to some embodiments of the present invention.

5 shows an exemplary sequence of steps performed by a security application to configure malware detection, according to some embodiments of the invention.

FIG6 shows an exemplary byte code routine according to some embodiments of the invention.

FIG. 7 illustrates an exemplary sequence of steps performed by the routine scheduler ( FIG. 3 ), according to some embodiments of the invention.

FIG8 shows an exemplary sequence of steps performed by a routine scheduler to process an event queue according to some embodiments of the invention.

FIG. 9 shows an exemplary sequence of steps performed by the bytecode interpretation VM ( FIG. 3 ), according to some embodiments of the invention.

FIG. 10 illustrates an exemplary software update process according to some embodiments of the present invention.

DETAILED DESCRIPTION

In the following description, it should be understood that all stated connections between structures may be direct operational connections or indirect operational connections through intermediate structures. A group of elements includes one or more elements. Any statement of an element is understood to refer to at least one element. A plurality of elements includes at least two elements. Unless otherwise required, any described method steps do not necessarily have to be performed in the specific order described. A first element (e.g., data) derived from a second element encompasses a first element that is equal to the second element, as well as a first element generated by processing the second element and optionally other data. Making a determination or decision based on a parameter encompasses making a determination or decision based on the parameter and, optionally, other data. Unless otherwise specified, an indicator of some quantity/data may be the quantity/data itself, or an indicator different from the quantity/data itself. Computer security covers protecting users and equipment from accidental or unauthorized access to data and/or hardware, preventing accidental or unauthorized modification of data and/or hardware, and destruction of data and/or hardware. A computer program is a sequence of processor instructions that performs a task. The computer programs described in some embodiments of the present invention may be independent software entities or sub-entities (e.g., subroutines, libraries) of other computer programs. Unless otherwise specified, a process is an example of a computer program, such as an application or part of an operating system, and is characterized by having at least one execution thread assigned to it and a virtual memory space, wherein the contents of the corresponding virtual memory space contain executable code. The term platform is used to refer to a specific physical or virtualized hardware configuration (e.g., including a specific model or processor family). A virtual platform includes virtualized hardware, such as a virtualized processor. The term bytecode herein refers to a multi-platform encoding of a computer program, meaning that the bytecode instructions can be delivered in the same form to different platforms. In contrast to bytecode instructions, native processor instructions include instructions specific to the instruction set architecture (ISA) of the physical or virtual platform on which the corresponding native processor instructions are executed. Computer-readable media encompasses non-transitory media such as magnetic, optical, and semiconductor storage media (e.g., hard drives, optical disks, flash memory, DRAM), as well as communication links such as conductive cables and fiber optic links. According to some embodiments, the present invention provides, among other things, a computer system comprising hardware (e.g., one or more microprocessors) programmed to perform the methods described herein, and a computer-readable medium encoding instructions for performing the methods described herein.

The following description illustrates embodiments of the invention by way of example and not necessarily by way of limitation.

FIG1 shows a set of exemplary software objects executed on a client system 10 protected from computer security threats according to some embodiments of the present invention. Client system 10 may represent a computer system (e.g., an end-user computer, a corporate server, etc.). Other exemplary client systems 10 include mobile computing devices (e.g., laptops, tablets), telecommunication devices (e.g., smartphones), digital entertainment devices (TVs, game consoles, etc.), wearable computing devices (e.g., smart watches), or any other electronic device having a processor and memory and requiring computer security protection.

In some embodiments, operating system (OS) 30 includes software that provides an interface to the hardware of client system 10 and serves as a host for a set of software applications 32a-c and 40. OS 30 may include any widely available operating system, such as Windows XP or Windows XP. Applications 32a-c generally represent user software and may include, for example, word processing, image processing, database, browser, and electronic communication applications. In some embodiments, security application 40 executes concurrently with applications 32a-c and is configured to determine whether any software executing on client system 10 (including applications 32a-c and OS 30) constitutes a computer security threat. For example, application 40 may detect malware (e.g., viruses, spyware, adware, etc.) operating on client system 10. Application 40 may further be configured to remove or otherwise disable such malicious software and alert the user or system administrator of client system 10. Security application 40 may be a stand-alone program or may form part of a software suite that includes, among other things, anti-malware, anti-spam, and anti-fraud components. The operation of security application 40 is described in detail below.

FIG2 illustrates an exemplary hardware configuration of a client system 10, where the client system 10 is a computer system. A skilled artisan will appreciate that the hardware configuration of other devices, such as tablet computers, mobile phones, smart watches, and the like, may differ from the configuration illustrated, but the present description is applicable to such devices. The client system 10 comprises a set of physical devices, including a hardware processor 12, a memory unit 14, a set of input devices 16, a set of output devices 18, a set of storage devices 20, and a set of network adapters 22, all interconnected via a controller hub 24.

In some embodiments, processor 12 comprises a physical device (e.g., a microprocessor or multi-core integrated circuit formed on a semiconductor substrate) configured to perform computations and/or logical operations using a set of signals and/or data. In some embodiments, such logical operations are transmitted to processor 12 from memory unit 14 in the form of a sequence of processor instructions (e.g., machine code or other types of software). Memory unit 14 may include volatile computer-readable media (e.g., RAM) that stores data/signals accessed or generated by processor 12 in the process of executing instructions. Input devices 16 may include a computer keyboard, mouse, microphone, etc., including corresponding hardware interfaces and/or adapters that allow a user to enter data and/or instructions into client system 10. Output devices 18 may include display devices such as monitors and speakers, etc., as well as hardware interfaces/adapters such as graphics cards, allowing client system 10 to communicate data to the user. In some embodiments, input devices 16 and output devices 18 may share common hardware, such as in the case of a touchscreen device. Storage device 20 includes computer-readable media that enables non-volatile storage, reading, and writing of processor instructions and/or data. Exemplary storage devices 20 include magnetic and optical disks, as well as flash memory devices, and removable media such as CD and/or DVD disks and drives. The set of network adapters 22 enables client system 10 to connect to a network (e.g., a local area network, a wireless network, etc.) and/or other devices/computer systems. Controller hub 24 generally represents a plurality of system, peripheral, and/or chipset buses, and/or all other circuitry, that facilitates communication between processor 12 and devices 14, 16, 18, 20, and 22. For example, controller hub 22 may include a northbridge that connects processor 12 to memory 14, and/or a southbridge that connects processor 12 to devices 16, 18, 20, and 22.

In some embodiments, client system 10 is configured to operate a virtual platform (e.g., a virtual machine). Such a configuration may be set up using hardware virtualization technology (e.g., a hypervisor). The virtual platform may include virtualized hardware components, each of which at least partially emulates the functionality of a corresponding hardware component of client system 10. For example, the virtual platform may include a virtualized processor, a virtualized memory unit, and virtual input/output devices. Operating system 30, as well as application programs 32a-c and security application 40, may execute on the respective virtual platforms and interact with the virtualized hardware. When software is executed using the virtualized processor of the respective platform, the software is said to be executed on the virtual platform.

3 shows exemplary components of a security application 40 according to some embodiments of the present invention. The security application 40 includes a routine scheduler 32, a behavior evaluation engine 38, and a bytecode interpreting virtual machine (BTVM) 34 connected to the scheduler 32 and the evaluation engine 38. In some embodiments, the application 40 further includes a facilitator module 36 connected to the BTVM 34 and/or the evaluation engine 38. The routine scheduler 32 may further manage an event queue 31, as described in further detail below.

In some embodiments, the routine scheduler 32 receives a set of event notifications 42 from a set of event interceptors 28a-c installed within various software objects executing on the client system 10. Event notifications 42 may inform the scheduler 32 of various events occurring during software execution. Exemplary notification events may include, among others, the creation of a process or thread, code injection, a system call, an attempt to create a new disk file, an attempt to write to an existing disk file, an attempt to edit a system register key, and an attempt to write to a specific memory portion. Some of the notified events may be indicative of malware. Other events may not indicate a security threat on their own, but may represent a potential threat when combined with other events. In response to receiving notifications 42, some embodiments of the scheduler 32 may select a bytecode routine based on the details of notifications 42 and send the corresponding bytecode routine, along with a set of event parameters 47 for the notification event, to the bytecode interpretation VM 34 for execution. Execution of the corresponding bytecode routine may supply an evaluation indicator 50 to the behavior assessment engine 38. The engine 38 may maintain multiple such evaluation indicators (e.g., scores) and may raise an alert if at least one such indicator indicates a computer security threat. The operation of components 32, 34, and 38 is described in detail below.

To illustrate the operation of event interceptors 28a-c, FIG4 shows an exemplary execution flow for a set of software entities 60a-b according to some embodiments of the present invention. For simplicity, the entities 60a-b are selected to represent processes executed in an example operating system; similar diagrams can be presented for other operating systems, such as Linux, for example. Solid arrows represent the execution flow without event interceptors. Dashed arrows represent modifications to the flow due to the presence of event interceptors 28a-c executed according to some embodiments of the present invention.

Exemplary process 60a loads multiple dynamic link libraries (DLLs) 62a through c; in the example of FIG. 4 , DLL 62c is injected into process 60a by (possibly malicious) process 60b. When process 60a (or one of its loaded DLLs) executes an instruction that calls a system functionality (e.g., to write something to a disk file or edit a registry key), the corresponding instruction calls a user-mode API, such as KERNEL32.DLL or NTDLL.DLL. In the example of FIG. 4 , the corresponding user-mode API call is intercepted by user-level event interceptor 28a. Such interceptors can be implemented through methods such as DLL injection or hooking. Hooking is a general term in the art for methods used to intercept function calls, messages, or events passed between software components. One exemplary hooking method involves changing the entry point of a target function by inserting an instruction (in this case, event interceptor 28a) to redirect execution to a second function. Following this hooking, the second function can execute in place of the target function or before the target function. 4 , security application 40 may hook into certain functions of the KERNEL32.DLL and/or NTDLL.DLL libraries to instruct the corresponding functions to redirect execution to components of application 40. Thus, application 40 may be notified whenever process 60 a is attempting to perform a specific action identified by the hooked functions.

In a typical execution flow, a user-mode API function called by entity 60a may request a service from the kernel of the operating system. In some embodiments, such operations are performed by issuing system calls (e.g., SYSCALL and SYSENTER) on an x86 platform. In the example of FIG. 4 , such system calls are intercepted by event interceptor 28b. In some embodiments, this interception includes modifying a system call handler routine, for example, by changing a value stored in a model-specific register (MSR) of processor 12, which effectively redirects execution of the corresponding handler routine to interceptor 28b, or directly to a component of application 40. Such a technique is known in the art as MSR hooking and may allow security application 40 to be notified whenever a software entity attempts to execute certain system calls.

After the system call, control of the processor is typically transferred to the kernel of the OS 30. In some embodiments, the kernel-level event interceptor 28c is configured to intercept certain actions of the OS kernel and, therefore, determine that the evaluation process is attempting to perform certain operations, which may be indicative of malware. To intercept such actions, some embodiments may employ a set of filtering mechanisms built into and exposed by the OS 30. For example, in the OS, FltRegisterFilter may be used to intercept operations such as creating, opening, writing, and deleting files. In another example, the event interceptor 28c may use ObRegisterCallback to intercept create or copy object disposal operations, or PsSetCreateProcessNotifyRoutine to intercept the creation of a new process. In another example, CmRegisterCallbackEx may be used to intercept Windows registry operations, such as creating and setting registry keys/values. Similar event filtering mechanisms for other operating systems, such as , are known in the art. In response to detecting the occurrence of a specific event/action, the event interceptor 28 may transmit a notification 42 to the security application 40.

Figure 5 shows an exemplary sequence of steps performed by the security application 40 for establishing detection of computer security threats according to some embodiments of the present invention. For example, when the security application 40 is started, the illustrated sequence of steps may be performed. In step 102, the application 40 installs event interceptors 28a to c, for example, by hooking various functions of the OS 30, modifying entries of the dispatch table, etc. Various methods for installing such interceptors are known in the art (e.g., patching, MSR hooking, code injection, etc.). Step 104 sets up an event queue 31, which is used by some embodiments of the routine scheduler to store information about notification events and the order in which such events have occurred. In some embodiments, the event queue 31 is organized as a first-in, first-out (FIFO) queue. The exemplary queue 31 may be a simple linked, lock-free, multi-producer, multi-consumer queue.

The sequence of steps 106 to 108 accesses a routine database 26 (see, e.g., FIG3 ) and loads a set of bytecode routines from the database 26 into a portion of memory. The database 26 may reside on the storage device 20 of the client system 10 or on a computer-readable medium communicatively coupled to the client system 10.

In some embodiments, each bytecode routine stored in database 26 includes a set of bytecode instructions that, when executed by BTVM 34, causes BTVM 34 to perform specific tasks related to computer security. One exemplary bytecode routine applies a set of heuristic tests to determine whether a particular software object (or group of objects) is malicious. Another instance of a bytecode routine checks for the occurrence of a particular sequence of events (behavioral signature) on client system 10. Not all events of a sequence need be caused by the same entity. However, the occurrence of such a sequence of events may be indicative of malware. In one such instance, malicious activity is divided into groups of entities, with each member of the group performing a small portion of the malicious activity. A bytecode routine configured to associate events corresponding to such distributed activity can therefore detect an entire group of malicious entities.

Other exemplary bytecode routines perform bookkeeping and/or statistical tasks (e.g., counting events, correlating event parameters). Other bytecode routines may alter the way the software and/or users of client system 10 interact with OS 30 and/or hardware devices (e.g., a routine may prevent a software object from being written to disk or accessing network adapter 22). Each bytecode routine may embody a distinct detection method and/or heuristic. In some embodiments, each bytecode routine may be configured to detect the presence of a specific class, family, type, or variant of a malicious agent. Several distinct routines may collaborate to detect a single class, family, type, or variant of a malicious agent. In some embodiments, a single routine may participate in detecting several classes, types, families, or variants of malware.

FIG6 shows an exemplary bytecode routine 70 according to some embodiments of the present invention. Routine 70 includes a trigger indicator 72 and a bytecode segment 74. In some embodiments, trigger indicator 72 encodes one or more conditions for executing the corresponding bytecode routine. The corresponding conditions can be evaluated by routine scheduler 32 and/or BTVM 34 to determine whether the corresponding bytecode routine should be executed in response to detecting the occurrence of a specific type of event. The exemplary conditions require determining whether a specific trigger expression evaluates to true (TRUE) or false (FALSE), and the condition is determined to be satisfied when the expression is TRUE. In some embodiments, the corresponding trigger expression includes an indicator of the event type (e.g., process creation, a specific system call, etc.). The trigger expression may further include event parameters, such as a path, file name, process ID, etc. In some embodiments, the trigger expression may include variables used by BTVM 34 and/or routine scheduler 32 (see examples below).

The trigger indicator 72 may further include a flag indicating whether the corresponding routine/byte code segment should be executed in a synchronous or asynchronous manner. The term synchronous is used herein to refer to a manner of executing a byte code routine in which the process that generates the event that triggers the execution of the corresponding routine (for simplicity, considered as the triggering process herein) is suspended while the corresponding routine is executed. Some embodiments use the thread execution context of the triggering process to execute the corresponding byte code routine, which implicitly suspends the execution of the triggering process (or at least the thread of the corresponding process) for the duration of the execution of the routine. The suspension and/or resumption of the execution of the triggering process can be implemented by an interceptor of the corresponding event. For example, the interceptor 28a to c can hijack the thread of the triggering process and/or resume the execution of the triggering process only in response to a signal from the routine scheduler 32. In contrast to synchronous execution, asynchronous execution refers to a manner of executing a byte code routine in which the triggering process is allowed to continue execution, while the event that triggered the notification is inserted into an event queue for later processing. Having a synchronization flag associated with each routine or byte code section may help routine scheduler 32 determine how to handle event notifications (further details given below).

In some embodiments, bytecode segment 74 includes consecutive bytecode instructions 74a-c that instruct BTVM 34 to perform computer security tasks associated with routine 70. In some embodiments, each bytecode instruction is a native instruction for the virtual processor of BTVM 34. In an exemplary embodiment, BTVM 34 is a virtual machine, and bytecode instructions 74a-c are instructions of a language. In order for the bytecode instructions to be executed by hardware processor 12, the corresponding bytecode instructions are first interpreted (or compiled) into a sequence of native processor instructions, for example, into instructions of the instruction set architecture (ISA) of processor 12. This interpretation is performed, at least in part, by BTVM 34. When security application 40 is executed on a virtual platform, BTVM 34 may interpret each bytecode instruction into a sequence of processor instructions that are native to the virtual processor of the corresponding virtual platform.

An exemplary bytecode instruction set may include, among other things, bytecode instructions for setting values, comparing values, performing jumps, looping instructions, and instructions for calling external functions, such as the functions of helper module 36 in FIG 3. In some embodiments, each bytecode instruction includes an opcode field 76 indicating an operation and an operand field 78 indicating at least one operand.

Some bytecode routines may have multiple sections, each of which may have a different trigger indicator. In such a configuration, each section of the routine may be triggered by a different condition. In one such example, the bytecode routine 70 may be directed to detecting ransomware, i.e., malware that encrypts a user's files and demands a ransom before decrypting the corresponding files. Examples of ransomware include CryptoWall, CryptoLocker, and CTB-Locker malware, among others. This malware typically calls cryptographic APIs, enumerates the user's files, and displays a ransom message. An exemplary bytecode routine for detecting this malware may include four sections:

Section 1: Trigger:EventType=call to a cryptographic function

Output:set static variable CryptoAPICall＝TRUE

Section 2: Trigger:[EventType=enumerate files using FindFirstFile APIandCryptoAPICall=TRUE]

Do:Check file extension of enumerated files

Output:set static variable RansomDocSearch＝TRUE if enumeratedfileshave extensions PDF,DOCX,JPG,...

Section 3: Trigger:[EventType=attempt to write to a disk file andRansomDocSearch=TRUE]

Do:determine whether file is among enumerated files

Output:set RansomActivity＝TRUE and increase evaluation score by30points

Section 4: Trigger:[EventType=change of screen background andRansomActivity=TRUE]

Output:increase evaluation score by 20 points

In the above example, segment 2 will only execute if segment 1 has been executed at least once, segment 3 will only execute if segment 2 has been executed, and so on.

In some embodiments, each bytecode routine further includes an initialization section, a cleanup section, and an error section. The initialization section may include variable initialization and may be executed once when the corresponding bytecode routine is loaded into memory (e.g., at step 108 in FIG. 5 ). The cleanup section may be executed when the corresponding routine terminates. The error section may include instructions that are executed if an error occurs during the execution of any other section of the corresponding bytecode routine.

In some embodiments, in step 110 ( FIG. 5 ), security application 40 may parse the loaded bytecode routines and create an association indicator (e.g., a table or hash index) indicating the association between event types and bytecode routines triggered by corresponding event types. This association enables routine scheduler 32 to select a routine for execution in response to detecting the occurrence of certain events.

FIG7 illustrates an exemplary sequence of steps performed by the routine scheduler 32 according to some embodiments of the present invention. The sequence of steps 112 through 114 listens for event notifications from the interceptors 28a through c. In response to receiving an event notification 42, step 116 determines a set of event parameters 47 for the corresponding notification event. Such parameters may be event-specific. Some exemplary event parameters include, among other things, an identifier of the process or thread performing the notification action (e.g., a process ID), a file name, a path, a memory address, and operands of a processor instruction. The event parameters may be determined by the interceptors 28a through c and included in the event notification 42, or may be determined by the routine scheduler 32 in response to receiving the notification 42. In one example where the notification event is an attempt to create a new disk file, the event parameters may include the name of the file being created. The corresponding file name may be determined by the event interceptor and transmitted to the routine scheduler 32 as part of the notification 42. In another example, the identity of the process performing a particular action may be determined directly by the routine scheduler 32, for example by parsing a data structure used by the OS 30 to manage currently executing processes. In the OS, each process is represented as an executing process block (EPROCESS), which includes, among other things, a handle to each of the threads of the corresponding process and a unique process ID that allows the OS 30 to identify the corresponding process from multiple executing processes. Similar process/thread representations are available for other OSes, such as Linux.

The occurrence of certain events may require the synchronous execution of associated bytecode routines. Examples of such events include, in particular, events that have substantial and/or irreversible consequences for the corresponding client system, such as terminating a process, deleting a file, and deleting a registry entry. To manage synchronous execution, some embodiments of the routine scheduler 32 may maintain a list of <event, routine> tuples, each of which indicates that a corresponding routine should be executed synchronously in response to the occurrence of a corresponding event. Such tuples may be static, for example, requiring that certain routines (or sections of routines) be executed synchronously each time a corresponding trigger occurs. Static tuples may be identified, for example, by the value of a synchronization flag associated with the corresponding routine/section (see above). Other <event, routine> tuples may be situational, meaning they may only be valid in certain circumstances. For example, such tuples may be dynamically added and/or removed from the list in response to the occurrence of other events. In some embodiments, the list of <event, routine> tuples registered for synchronous processing may be based on a set of options specified by a user (e.g., an administrator) of the corresponding client system. Generally speaking, synchronous processing/execution of bytecode routines can provide greater security, but can negatively impact the user experience by slowing down some tasks. Different users may prefer different security settings. In some embodiments, each setting (e.g., desired security level) can correspond to a different component of the <event, routine> list.

In step 120, routine scheduler 32 determines whether the occurrence of the trigger event indicated by the current event notification warrants synchronous processing, for example, by determining whether there is at least one <event, routine> tuple that includes the corresponding event. The determination in step 120 may be further performed based on the type of the corresponding event, based on a synchronization flag of notification 42, or based on a trigger indicator of the bytecode routine.

In step 122, when the occurrence of the corresponding event does not require synchronous processing, the routine scheduler 32 inserts the corresponding event into the event queue 31. A further step 124 signals the interceptors 28a to c to resume execution of the process/thread that caused the corresponding event notification.

In step 126, when the occurrence of the current notification event requires synchronous processing, routine scheduler 32 selects a bytecode routine based on the corresponding event. Step 126 may include identifying a subset of bytecode routines associated with the type of the current event and evaluating the triggering condition for each such routine. Step 130 includes launching an instance of BTVM 34 instantiated with the corresponding bytecode routine. Some embodiments use the execution thread of the triggering process to execute the corresponding instance of BTVM 34. If step 126 identifies more than one bytecode routine triggered by the current event notification, the sequence of steps 128 through 130 may be re-executed for each such routine.

In some embodiments, in step 132, the routine scheduler 32 waits for a response from the behavior assessment engine 38 and/or the BTVM 34 indicating the results of the security assessment of the process that caused the notification 42. For example, the assessment may indicate whether the corresponding process is malicious. In alternative embodiments, in step 132, the scheduler 32 may wait until the BTVM 34 completes execution of the current bytecode routine. When the corresponding routine has completed execution and/or the security assessment is complete, the scheduler 32 may signal the event interceptors 28a-c to resume execution of the process that caused the event notification 42.

FIG8 illustrates an exemplary sequence of steps performed by routine scheduler 32 to process event queue 31 according to some embodiments of the present invention. Sequence steps 142 through 144 wait until queue 31 contains at least one item. Step 146 removes the event from the queue. When queue 31 is a FIFO type, events are removed in the order in which they were inserted. In step 148, routine scheduler 32 evaluates the triggering conditions based on the type and/or parameters of the event to identify a subset of bytecode routines triggered by the corresponding event. In sequence steps 150 through 152 through 154, scheduler 32 selects each such routine and sends it to BTVM 34 for execution.

FIG9 shows an exemplary sequence of steps performed by the bytecode interpretation virtual machine 34 when executing a bytecode routine. In some embodiments, the BTVM 34 comprises a software emulation of a computer system, including a virtual processor, virtual memory, and a set of virtual registers. The virtual processor of the BTVM 34 treats the bytecode instructions as if they were its native instruction set. However, because the bytecode instructions are not native to the platform on which the security application 40 is executing, executing the bytecode instructions involves interpreting each bytecode instruction into a sequence of processor instructions native to the processor of the corresponding platform. In some embodiments, the BTVM 34 comprises a code interpreter configured to interpret and execute the bytecode instructions one by one. In alternative embodiments, the BTVM 34 may comprise a just-in-time (JIT) compiler configured to compile all instructions of the bytecode routine together into a native code routine. The compiled native code routine is then executed as a whole. In some embodiments, the BTVM 34 comprises a library of functions (e.g., classes) encoded in the native instruction set of the processor 12, wherein each type of bytecode instruction corresponds to a distinct function of the library. In such cases, interpreting the bytecode instructions may include identifying the appropriate function based on the bytecode instructions and executing the corresponding function.Exemplary bytecode interpreting virtual machines include, among others, the bytecode interpreting virtual machines.

In some embodiments, the virtual memory space of the BTVM 34 is divided into several areas, each area storing variables of a different type. Some variables may be of type LOCAL; the value of such variables may be unique to the current instance of the corresponding routine or bytecode segment. Other variables may be of type STATIC (specific to a specific routine type, shared across all code segments of that specific type of routine, and retain their value across multiple instances of the same routine/bytecode segment). Other variables may be of type GLOBAL (shared across all bytecode routines and instances). Some embodiments of the BTVM 34 may also operate using variables of type PROCESS—these variables may be uniquely attached to a routine-process tuple, shared across all code segments of that specific type of routine, and retain their value across multiple instances of the same routine/bytecode segment, but different for each process. When the virtual memory of the BTVM 34 is configured to store variables of type PROCESS, such variables may be initialized once for each process monitored by the event interceptors 28a to c and erased when the corresponding process terminates.

In some embodiments, the security application 40 maintains a thread pool for use by the BTVM 34. In one example, the thread pool includes the same number of threads as the number of logical CPUs of the processor 12. All threads of the thread pool can be set to the same priority. When scheduling a byte code routine, the scheduler 32 can wait for the next available thread. Different instances of the BTVM 34 can thus run independently and simultaneously in any memory context. In some embodiments, instances of the BTVM 34 can execute out of order, that is, not necessarily in the same order in which the events that triggered them were inserted into the event queue 31.

In response to receiving a routine indicator 46 (e.g., the memory address of a bytecode routine) from the routine dispatcher 32, the BTVM 34 may access the bytecode segment 74. Without loss of generality, the steps described below describe an embodiment in which the BTVM 34 interprets and sequentially executes each bytecode instruction. Those skilled in the art will appreciate that this description is applicable to describing the operation of an embodiment using just-in-time (JIT) compilation. In the sequence of steps 166 through 168, the BTVM 34 may interpret the bytecode instructions and perform a set of operations indicated by the corresponding instructions.

Some bytecode instructions require complex functionality, such as functionality provided by functions of the OS 30. To minimize the complexity of the BTVM 34 and reduce the attack surface of the BTVM 34, some embodiments provide this functionality to the BTVM 34 via an application programming interface (API) embodied as a helper module 36 ( FIG. 3 ). Module 36 may include a set of helper functions configured to assist the BTVM 34, performing tasks such as verifying the existence of files, attempting to match the contents of memory segments with malware-indicative signatures, enumerating files, and setting assessment scores. In step 170, when the current bytecode instruction calls a helper function, the bytecode interpreting VM 34 issues an API call 48 to the helper module 36. The helper module 36 may further interface with the OS 30.

In some embodiments, facilitator module 36 may include functionality to add new events to event queue 31 in response to an API call from bytecode interpretation VM 34 (see, e.g., FIG3 ). In one such example, the first portion of a bytecode routine is processed synchronously. BTVM 34 may then cause the second portion of the corresponding routine to be executed later (asynchronously) by calling facilitator module 36 to add a new event to queue 31, the new event being selected to trigger execution of the second portion of the routine. In another example, the described mechanism may be used to execute a cascade of bytecode routines, with each routine determining execution of the next member of the cascade by instructing facilitator module 36 to insert a new event into event queue 31.

Some bytecode routines output an evaluation indicator 50 to the behavior evaluation engine 38, for example, via an API call to the engine 38 or to the facilitator module 36. The evaluation indicator 50 may include an evaluation score indicating the likelihood that a particular monitored object (e.g., a process) is malicious. Another exemplary evaluation indicator 50 includes a score increment to be added to the evaluation score maintained by the evaluation engine 38 for the corresponding object. Another exemplary indicator 50 includes a decision result (e.g., clean/malicious). In some embodiments, the sequence of steps 174 to 176 determines whether an evaluation condition is satisfied (e.g., whether a score increment has been calculated) and, if the evaluation condition is satisfied, transmits the indicator 50 to the evaluation engine 38.

Step 178 may verify whether a termination condition is satisfied, and if the termination condition is not satisfied, BTVM 34 may proceed to the next bytecode instruction of the current routine (see step 166 above). In some embodiments, step 178 may include determining whether the last bytecode instruction of the current routine/bytecode segment has been executed. Some bytecode routines may be configured to execute for a maximum time; in such cases, step 178 may determine whether the maximum allowed time period has elapsed since the current bytecode routine/bytecode segment was started for execution. Another exemplary termination condition may be satisfied whenever an error occurs during the execution of the corresponding routine/bytecode segment. When the termination condition is satisfied, step 180 may terminate the current instance of BTVM 34 and release the corresponding thread to the thread pool.

In some embodiments, the behavior assessment engine 38 maintains a centralized knowledge base of monitored software entities (e.g., processes and threads executing on the client system 10). For each such entity, the engine 38 may store a parent-child relationship indicator (e.g., to indicate a master-slave relationship, the relationship between the process injecting code and the recipient of such code injection, etc.). Some embodiments of the assessment engine 38 further store a set of assessment scores associated with each monitored entity. Each such score may be determined based on different criteria (e.g., each score is updated by a specific bytecode routine). In some embodiments, the scoring engine 38 is configured to determine whether each monitored entity poses a computer security threat based on such scores. When such a threat is identified, the engine 38 may take anti-malware actions against the corresponding malicious entity, such as stopping the execution of the corresponding software object or otherwise disabling the corresponding software object. The engine 38 may further alert the user of the client system 10 and/or the system administrator.

The exemplary systems and methods described above allow for protection of client systems from computer security threats such as malware and spyware. For each of a plurality of executable entities (e.g., processes and threads currently executing on the client system), a security application maintains a set of evaluation scores that may indicate the likelihood that the respective entity poses a threat. In some embodiments, such scores are generated in response to executing a plurality of detection routines, each of which is configured to evaluate the monitored entity according to criteria specific to each detection routine. The detection routines may implement various computer security heuristics; separate routines may correspond to different heuristics. For example, a routine may determine whether a monitored entity performs a sequence of actions indicative of malware, known in the art as a behavioral signature. Execution of a detection routine may be selectively triggered in response to detecting the occurrence of a particular triggering event.

In some embodiments of the present invention, the detection routines are expressed in bytecode form, i.e., using a custom instruction set that is not easily executed by the processor of the corresponding client system. Instead, before executing the corresponding instructions, the detection routines are executed within a bytecode interpretation virtual machine, which uses interpretation or just-in-time compilation techniques to interpret each bytecode routine into a set of processor instructions that are native to the processor of the corresponding platform.

In conventional computer security systems, detection routines are precompiled into the native code of the target platform usually and are packaged together with the rest of the components of the security application. Although this type of configuration allows the execution of the corresponding detection routine through optimization fast, it may be difficult to develop, keep up to date and be deployed to the client computer system. For example, any little change in the detection routine or the introduction of new detection routine/heuristics may need to recompile a large amount of code of the security application, and need to deploy relatively large software updates (for example, several megabytes). In addition, software failures in the update can potentially reduce the performance of the whole security application, so the update usually needs to be tested before being delivered to the client.

In contrast, the use of bytecode detection routines allows some embodiments of the present invention to significantly reduce the time to market for security solutions and simplify the software update process. Because bytecode detection routines are interpreted or compiled on the fly, they can be developed and delivered to clients separate from the main components of the security solution. New detection heuristics can be developed and implemented completely independently of existing routines. As shown in Figure 10, a software update may include only a set of bytecode routines 70, totaling as few as a few hundred bytes. To activate new detection capabilities, the corresponding bytecode routines can be introduced into the routine database 26 and then registered with the routine scheduler 32. In addition, because each detection routine is executed within an instance of the bytecode interpretation virtual machine and separately from other components of the security application, errors or software failures present within the bytecode routines are contained and are less likely to affect the operation of other software.

Some embodiments of the present invention may use widely available bytecode interpreting virtual machines (e.g., virtual machines or VMs) to implement interpretation and/or JIT compilation of the detection routines. However, to minimize architectural complexity and reduce the attack surface of bytecode interpreting virtual machines, some embodiments employ custom interpreting virtual machines with minimal functionality. When more complex functionality is called within the detection routines, some embodiments provide the required functionality through a separate helper API.

Because each instance of the bytecode interpreting virtual machine can execute within a separate, independent memory context, some embodiments of the present invention can provide the same security advantages as provided, for example, by containerization. Other advantages of the present invention include portability. While some components of the proposed security solution may need to be compiled separately for each type of client device (e.g., computer system, smartphone, etc.), the bytecode routines are platform-independent in the sense that the same detection routines (including updates) can be delivered to all platforms and client devices.

Such advantages of the present invention may come at the expense of memory overhead and processing speed. Although pre-compiled solutions are optimized for maximum performance on a particular platform, interpretation and/or JIT compilation of detection routines may make some embodiments of the present invention slower than pre-compiled security applications.

It will be apparent to those skilled in the art that the above embodiments can be modified in many ways without departing from the scope of the invention. Therefore, the scope of the invention should be determined by the appended claims and their legal equivalents.

Claims (20)

1. A client system comprising at least one hardware processor configured to form a routine scheduler, a bytecode interpretation virtual machine, and a behavior evaluation engine, wherein: The routine scheduler is configured to select an anti-malware bytecode routine from a plurality of anti-malware bytecode routines for execution in response to the detection of a triggering event, wherein the occurrence of the triggering event is caused by a monitored process executed within the client system. The bytecode interpretation virtual machine is configured to execute the anti-malware bytecode routine in response to the routine scheduler selecting the anti-malware bytecode routine to determine whether the occurrence of the triggering event indicates malware, wherein executing the anti-malware bytecode routine includes: The set of bytecode instructions of the anti-malware bytecode routine is interpreted into a sequence of instructions for the native processor, and Execute the native processor instruction sequence; and The behavior evaluation engine is configured to determine whether the client system contains malware based on the results of the anti-malware bytecode routine executed by the bytecode interpretation virtual machine. 2. The client system of claim 1, wherein the routine scheduler is further configured to respond to detecting the occurrence of the triggering event to: Determine whether the synchronization conditions are met based on the type of the triggering event; In response to determining whether the synchronization condition is met, when the synchronization condition is met, the hardware processor is configured to suspend the execution of code belonging to the monitored process until the bytecode interpretation virtual machine completes the execution of the anti-malware bytecode routine; and In response to determining whether the synchronization condition is met, if the synchronization condition is not met, the hardware processor is instructed to continue executing the code belonging to the monitored process. 3. The client system of claim 2, wherein the routine scheduler is further configured to: in response to determining whether the synchronization condition is met, insert the triggering event into an event queue for later processing when the synchronization condition is not met. 4. The client system according to claim 3, wherein the routine scheduler is further configured to: Remove the second triggering event from the event queue; In response to removing the second triggering event from the event queue, a second anti-malware bytecode routine is selected for execution, and the second anti-malware bytecode routine is selected from the plurality of bytecode routines according to the second triggering event; and In response to selecting the second anti-malware bytecode routine, the indicator of the second anti-malware bytecode routine is transmitted to the bytecode interpretation virtual machine. 5. The client system of claim 1, wherein execution of the anti-malware bytecode routine causes the hardware processor to determine whether the monitored process performs a set of actions. 6. The client system of claim 1, wherein execution of the anti-malware bytecode routine causes the hardware processor to determine whether the triggering event forms part of a sequence of malware indicative events. 7. The client system of claim 6, wherein the selected event of the event sequence is caused by another monitored process executed within the client system, the other monitored process being different from the monitored process. 8. The client system of claim 1, wherein the anti-malware bytecode routine includes encoding of trigger conditions, and wherein selecting the anti-malware bytecode routine to perform an action including determining whether the trigger conditions are met. 9. The client system of claim 1, wherein the anti-malware bytecode routine includes encoding of trigger conditions, and wherein the bytecode interpretation virtual machine is further configured to select the group of bytecode instructions for interpretation based on the result of determining whether the trigger conditions are met. 10. A computer-implemented method comprising employing at least one hardware processor of a client system to perform each of the following steps: In response to the detection of a triggering event, an anti-malware bytecode routine is selected from a plurality of bytecode routines based on the triggering event. The anti-malware bytecode routine is configured to determine whether the occurrence of the triggering event indicates malware, wherein the triggering event is caused by a monitored process executed within the client system. In response to selecting the anti-malware bytecode routine, a set of bytecode instructions of the anti-malware bytecode routine is interpreted into a sequence of native processor instructions; Execute the native processor instruction sequence; and The client system is determined to contain malware based on the result of executing the sequence of native processor instructions. 11. The method of claim 10, further comprising, in response to detecting the occurrence of the triggering event: Based on the type of the triggering event, at least one hardware processor is used to determine whether the synchronization condition is met; In response to determining whether the synchronization condition is met, when the synchronization condition is met, the at least one hardware processor is configured to suspend the execution of code belonging to the monitored process until the native processor instruction sequence has completed execution; and In response to determining whether the synchronization condition is met, if the synchronization condition is not met, the at least one hardware processor is used to continue executing the code belonging to the monitored process. 12. The method of claim 11, further comprising, in response to determining whether the synchronization condition is met, employing the at least one hardware processor to insert the triggering event into an event queue for later processing when the synchronization condition is not met. 13. The method of claim 12, further comprising: The second triggering event is removed from the event queue using at least one hardware processor; In response to removing the second triggering event from the event queue, the at least one hardware processor is used to select a second anti-malware bytecode routine for execution, and the second anti-malware bytecode routine is selected from the plurality of bytecode routines according to the second triggering event; In response to selecting the second anti-malware bytecode routine, the at least one hardware processor is employed to interpret a set of bytecode instructions of the second anti-malware bytecode routine into a second native processor instruction sequence; and The at least one hardware processor is used to execute the second native processor instruction sequence. 14. The method of claim 10, wherein executing the native processor instruction sequence causes the at least one hardware processor to determine whether the monitored process performs a set of actions. 15. The method of claim 10, wherein executing the native processor instruction sequence causes the at least one hardware processor to determine whether the triggering event forms part of a malware indicative event sequence. 16. The method of claim 15, wherein the selected events of the event sequence are caused by another monitored process executed within the client system, the other monitored process being different from the monitored process. 17. The method of claim 10, wherein the anti-malware bytecode routine includes encoding of a trigger condition, and wherein selecting the anti-malware bytecode routine to perform an action including determining whether the trigger condition is met. 18. The method of claim 10, wherein the anti-malware bytecode routine includes encoding of triggering conditions, the method further comprising: Determine whether the triggering condition is met; and In response, the group of bytecode instructions is selected for interpretation based on the result of determining whether the triggering condition is met. 19. A non-transitory computer-readable medium storing a computer program, said computer program, when executed by at least one hardware processor of a client system, causes the client system to form a routine scheduler, a bytecode interpretation virtual machine, and a behavior evaluation engine, wherein: The routine scheduler is configured to select an anti-malware bytecode routine from a plurality of anti-malware bytecode routines for execution in response to the detection of a triggering event, the anti-malware bytecode routine being configured to determine whether the occurrence of the triggering event indicates malware, wherein the triggering event is caused by a monitored process executed within the client system; The bytecode interpretation virtual machine is configured to execute the anti-malware bytecode routine in response to the routine scheduler selecting the anti-malware bytecode routine, wherein executing the anti-malware bytecode routine includes: The set of bytecode instructions of the anti-malware bytecode routine is interpreted into a sequence of instructions for the native processor, and Execute the native processor instruction sequence; and The behavior evaluation engine is configured to determine whether the client system contains malware based on the results of the anti-malware bytecode routine executed by the bytecode interpretation virtual machine. 20. A non-transitory computer-readable medium storing anti-malware bytecode routines executable by a bytecode interpretation virtual machine formed on a client system including at least one hardware processor, wherein the client system further includes a routine scheduler and a behavior evaluation engine, wherein execution of the anti-malware bytecode routines causes the at least one hardware processor to determine whether the occurrence of a triggering event within the client system indicates malware, wherein: The routine scheduler is configured to select the anti-malware bytecode routine from a plurality of bytecode routines in response to the detection of the occurrence of the triggering event; Executing the aforementioned anti-malware bytecode routine includes: In response to the routine scheduler selecting the anti-malware bytecode routine, a set of bytecode instructions of the anti-malware bytecode routine is interpreted into a sequence of native processor instructions, and Execute the native processor instruction sequence; and The behavior evaluation engine is configured to determine whether the client system contains malware based on the results of the anti-malware bytecode routine executed by the bytecode interpretation virtual machine.