CN1997973B - Processor, method, apparatus and apparatus for dynamic cache engine instructions - Google Patents

Abstract

In general, in one aspect, the disclosure describes a processor that includes an instruction store to store instructions of at least a portion of at least one program and a set of multiple engines coupled to the instruction store. The engines include an engine instruction cache and circuitry to request a subset of the at least the portion of the at least one program.

Description

Processor, method, apparatus and apparatus for dynamic cache engine instructions

RELATED APPLICATIONS REFERENCED : This application is related to the following applications filed on the same date as this application:

a. Lawyer file number P16851-"Service Engine Cache Request (service engine cache request)"

b. Lawyer case file number P16852-"Thread-Based Engine Cache Partitioning (thread-based engine cache partition)"

background

Networks enable computers and other devices to communicate. For example, a network may carry data representing video, audio, email, and the like. Typically, data sent over a network is divided into smaller messages called packets. Similarly, packets are very much like envelopes you drop in the mailbox. A packet generally includes a "payload" and a "header". The "payload" of a packet is like a letter in an envelope. The "header" of the packet is much like the information written on the envelope itself. The header may include information to help network devices process the packet appropriately. For example, the header may include an address identifying the destination of the packet.

A given packet may "hop" through many different intermediate network devices (eg, "routers," "bridges," and "switches") before reaching its destination. These intermediary devices typically perform various packet processing operations. For example, intermediary devices typically perform operations to determine how to forward a packet further toward its destination or to determine the quality of service to use in processing the packet.

As network connection speeds increase, the amount of time intermediary devices have to process packets continues to shrink. To achieve fast packet processing, many devices feature dedicated, "hardwired" designs such as application-specific integrated circuits (ASICs). However, these designs are often difficult to adapt to emerging network technologies and communication protocols.

To combine the flexibility and speed typically associated with ASICs, some networking equipment features programmable network processors. Network processors allow software engineers to quickly reprogram network processor operations.

Often, also due to ever-increasing network connection speeds, the time it takes to process packets greatly exceeds the rate at which packets arrive. Accordingly, the architecture of some network processors features multiple processing engines that process packets concurrently. For example, while one engine determines how to forward one packet, another engine determines how to forward a different one. Although the time to process a given packet may remain the same, processing multiple packets simultaneously enables the network processor to cope with a large number of incoming packets.

Brief description of the drawings

FIG. 1 is a diagram describing an instruction cache of a network processor.

Figure 2 is a diagram describing instruction operations for fetching instructions into an engine's instruction cache.

Figure 3 is a flowchart describing instruction processing performed by a network processor engine.

Figure 4 is a flowchart describing instruction caching.

Figure 5 is a diagram depicting engine circuitry for searching cached instructions.

FIG. 6 is a map of instruction cache memories allocated to different threads of a network processor engine.

Figure 7 is a network processor engine diagram.

Figure 8 is a network processor diagram.

Fig. 9 is a diagram of network equipment.

Detailed description

FIG. 1 shows a network processor 100 including a plurality of processing engines 102 . Engine 102 may be programmed to perform various packet processing operations, such as determining a packet's next hop, applying quality of service (Qos), measuring packet traffic, and the like. In the illustrated architecture, engine 102 executes program instructions 108 stored in high speed local memory 104 of engine 102 . Due to size and cost constraints, the amount of instruction memory 104 provided by the engine 102 is typically limited. To prevent the limited storage of engine memory 104 from imposing too strict constraints on the overall size and complexity of program 108, FIG. As execution of 108 proceeds, segments of larger program 108 (eg, 108b ) are dynamically downloaded to engine 102 .

In the embodiment shown in FIG. 1 , each engine 102 includes an instruction cache 104 that stores a subset of program 108 instructions. For example, instruction cache 104a of packet engine 102a holds segment 108b of program 108 . The remainder of the program 108 is stored in an instruction store 106 shared by the engines 102 .

Finally, engine 102a may need to access program segments other than segment 108b. For example, the program may branch or advance sequentially to a point within program 108 outside segment 108b. In order for the engine 102 to continue execution of the program 108, the network processor 100 will download the requested/needed segments to the cache 104a of the engine 102a. Therefore, the segments stored by the cache change dynamically as program execution progresses.

As shown in FIG. 1 , a plurality of engines 102 receive instructions to be cached from an instruction store 106 . The shared instruction store 106 may in turn cache instructions from a hierarchically higher instruction store 110, which may be internal or external to the processor. In other words, instruction stores 104, 106, and 110 may form a cache hierarchy that includes an engine's L1 instruction cache 104 and an L2 instruction cache 106 shared by different engines.

Although FIG. 1 depicts the instruction memory 106 as serving all of the engines 102 , the network processor 100 may additionally feature multiple shared memories 106 serving different groups of engines 102 . For example, one shared instruction memory 106 may store program instructions for engines #1 to #4, while the other stores program instructions for engines #5 to #8. Furthermore, although FIG. 1 depicts engine cache 104 and instruction store 106 as storing instructions of a single program 108, they may additionally store sets of instructions belonging to different programs. For example, shared instruction memory 106 may store different program instructions for each engine 102 or even different engine 102 threads.

FIG. 1 depicts instructions 108 as source code to simplify the description. The actual instructions stored by the shared memory 106 and dispatched by the engine will typically be executable instructions expressed in an instruction set provided by the engine.

Potentially, the program segments needed by engine 102 to continue program execution may be provided on an "as-needed" basis. That is, the engine 102 may continue to execute the instructions 108b stored in the instruction cache 104a until no instruction to be executed is found in the cache 104a. When this occurs, the engine 102 may signal the shared memory 106 to deliver the program segment that includes the next instruction to be executed. However, this "on demand" scenario may introduce delays into engine 102's program execution. That is, in an "on-demand" sequence, the engine 102 (or engine 102 thread) may remain idle until the required instruction is loaded. This delay may be caused not only by the operations involved in downloading the required instructions into the engine 102L1 cache 104 , but also by contention among the engines 102b - 102n accessing the shared memory 106 .

To potentially avoid this delay, FIG. 2 shows a portion of a program source code listing that includes an instruction fetch 122 that allows a program to initiate a "pre-load" of program instructions before the instruction will be required to continue execution of the program. Fetch" to the operation of the cache 104 of the engine. For example, as shown in FIG. 2, an instruction fetch 122 causes the engine 102n to issue (“1”) a request to the shared instruction store 106 for the next desired segment 108b before execution proceeds to a point in the next segment 108b. As engine 102 continues to process instructions 124 following instruction fetch 122, the requested segment is loaded into instruction cache 104n of engine 102n. In other words, the difference between the time used to retrieve ("2") the program segment and the time the engine executes the prefetch instruction 122 and the time the engine "runs out" of instructions to be executed in the currently cached program segment time overlap.

In the embodiment shown in FIG. 2, the time to fetch a program instruction is concealed by the time it takes to execute instruction 122 after the instruction is fetched. Fetch latency can also be "hide" by performing an instruction fetch after an instruction 120 (eg, a memory operation) that takes some time to complete.

The example instruction fetch shown in Figure 2 has the syntax:

Prefetch(SegmentAddress, SegmentCount)[, optional_token]

Among them, SegmentAddress identifies the starting address of the program to be fetched from the shared storage 106 and SegmentCount identifies the number of subsequent segments to fetch. Potentially, SegmentAddress can be constrained to identify the starting address of a program segment.

optional_token has syntax:

optional_token=[ctx_swap[signal],][sig_done[signal]]

The ctx_swap parameter instructs the engine 102 to switch to another engine's thread of execution until the signal indicates that the segment fetch operation is complete. The sig_done parameter also identifies a status signal that is set once the fetch operation is complete, but does not instruct the engine 102 to switch contexts.

The instruction syntax shown in FIG. 2 is exemplary only, and other instructions for fetching program instructions may feature different parameters, keywords, and feature different options. Furthermore, instructions can exist in different levels. For example, the instructions may be part of the engine's instruction set. Alternatively, the instructions may be processed by a compiler to generate source code instructions corresponding to target instructions (eg, engine-executable instructions) of the instruction fetch. Such a compiler can perform other traditional compiler operations, such as lexical analysis for grouping textual characters of source code into "tokens," syntactic analysis for grouping words into grammatical phrases, more abstract representations of Intermediate code generation for source code, optimizations, etc.

Instruction fetches can be manually inserted by the programmer during code development. For example, based on the initial classification of the packet, the remaining program flow for the packet can be known. Therefore, an instruction fetch can fetch the segments needed to process a packet after sorting. For example, a program written in a high-level language can include instructions:

switch(classify(packet. header)){

case DropPacket:{

prefetch(DropCounterInstructions);

}

case ForwardPacket{

prefetch(RountingLookupInstructions)

prefetch(PacketEnqueueInstructions);

}}

The instructions load the appropriate program segment into the instruction cache 104 of the engine 102 based on the classification of the packets.

Although programmers may manually insert instruction fetches into code, instruction fetches may also be inserted into code by software development tools, such as compilers, analyzers, profilers, and/or preprocessors. For example, code flow analysis can identify when different program segments should be loaded. For example, a compiler may insert an instruction fetch after a memory access instruction or before a set of instructions that take some time to execute.

Figure 3 shows a flowchart illustrating the operation of the engine to fetch instructions "on demand" and in response to "fetch" instructions. As shown in FIG. 3, the program counter 130 identifying the next program instruction to be executed is updated. For example, program counter 130 may be incremented to advance to the next sequential instruction address, or counter 130 may be set to some other instruction address in response to a branch instruction. As shown, the engine determines 132 whether the engine's instruction cache currently holds the instruction identified by the program counter. If not, the engine thread stalls 134 (eg, the thread needing the instruction is swapped out of the engine) until a fetch operation 136 fetches the missing instruction from shared storage.

Once the instruction to be executed is present in the engine's instruction cache, the engine may determine 140 whether the next instruction to be executed is a fetch. If so, the engine may initiate a fetch operation 142 of the requested program segment. If not, the engine can process 144 instructions as usual.

FIG. 4 shows an example architecture of the shared instruction cache 106 . For example, during network processor startup, instruction cache 106 receives an instruction ("1") to share with the engine. The shared instruction cache 106 then distributes portions of the instructions 108 to the engines as needed and/or requested.

As shown in the example architecture of FIG. 4 , two different buses 150 , 152 may connect the shared cache 106 to the engine 102 . Bus 150 carries (“2”) fetch requests to shared cache 106 . These requests may identify the program segment 108 to fetch and the engine making the request. The request can also identify whether the request is a prefetch operation or an "on demand" fetch operation. The high bandwidth bus 152 carries (“4”) the instructions in the requested program segment back to the request engine 102 . The bandwidth of bus 152 may allow shared cache 106 to transfer requested instructions to multiple engines simultaneously. For example, bus 152 may be divided into n lines that may be dynamically assigned to engines. For example, if 4 engines request segments, each may be allocated 25% of the bus bandwidth.

As shown, the shared cache 106 may queue requests as they arrive (eg, in a (first in first out) FIFO queue 154 for subsequent servicing). However, as described above, when the instruction to be executed has not been loaded into the engine's instruction cache 104, the thread stalls. Thus, servicing "on-demand" requests that cause actual stalls takes on more urgent importance than servicing "prefetch" requests that may or may not cause stalls. As shown, the shared cache 106 includes an arbiter 156 that can grant demand requests priority over prefetch requests. Arbiter 156 may comprise dedicated circuitry or may be programmable.

Arbiter 156 may prioritize demand requests in various ways. For example, the arbiter 156 may not add the demand request to the queue 154, but may otherwise indicate the request as immediate service ("3"). In order to differentiate priority between multiple "demand" requests, the arbiter 156 may also maintain a separate "demand" FIFO queue that is granted higher priority by the arbiter 156 than those in the FIFO queue 154. The priority of the request. The arbiter 156 may also immediately suspend an ongoing instruction download operation to service demand requests. In addition, the arbiter 156 may allocate a substantial portion (otherwise 100%) of the bus 152 bandwidth to pass segment instructions to engines issuing "on demand" requests.

Figure 5 shows an example architecture of the engine's instruction cache. As shown, cache storage is provided by a set of memory devices 166x that store instructions received from shared instruction memory 106 over bus 164 . A single memory element 166a may be sized to hold one program segment. As shown, each memory 166x is associated with an address decoder that receives the address of an instruction to be processed from the engine and determines whether the instruction is present in the associated memory 166 . Different decoders operate on addresses in parallel. That is, each decoder simultaneously searches its associated memory. If the instruction address is found in one of the memories 166x, then that memory 166x unit outputs 168 the requested instruction to be processed by the engine. If the instruction address is not found in any of the memories 166, then a "miss" signal 168 is generated.

As mentioned above, an engine can provide multiple threads of execution. During execution, these different threads will load different program segments into the engine's instruction cache. When the cache is full, the operation of loading a segment into the cache requires some other segments to be removed from the cache ("sacrificed"). Without some safeguard, a thread may sacrifice a segment that is currently being used by another thread. When other threads resume processing, the most recently sacrificed segment can be fetched from the shared cache 106 again. This inter-thread thrashing of the instruction cache 104 may be repeated over and over again, significantly reducing system performance as segments just loaded into the cache by one thread are prematurely sacrificed by another thread and are short-lived. time to be reloaded.

To resist such thrashing, various mechanisms can impose limits on a thread's ability to sacrifice segments. For example, FIG. 6 shows a memory map of an engine's instruction cache 104, where each engine thread is assigned an exclusive portion of the cache 104. For example, thread 0 172 is allocated memory for N program segments 172a, 172b, 172n. Instruction segments fetched for a thread may reside in allocated space in the thread's cache 104 . To prevent thrashing, logic can restrict one thread from sacrificing segments from cache partitions allocated to other threads.

For fast access to cached segments, a control and status register (CSR) associated with a thread may store the starting address of the allocated cache portion. This address may, for example, be calculated based on the number of threads (eg, allocation space start address = base address + (thread #X memory allocated per thread)). Each partition may be further divided into segments corresponding, for example, to a burst size from shared storage 106 or other transfer granularity from shared storage 106 to the engine cache. LRU (least recently used) information may be maintained in the thread's allocated cache portion for different segments. Thus, in an LRU scheme, the least recently used segment in a given thread's cache may be sacrificed first.

The map shown includes a “lock-down” portion 170 in addition to the regions divided between different threads. Instructions in locked regions can be loaded at initialization and can be protected from sacrifice. All threads can access and execute instructions stored in this area.

A memory allocation scheme such as that shown in FIG. 6 can prevent inter-thread thrashing. However, other methods can also be used. For example, an access count can be associated with the thread currently using the segment. When the count reaches zero, the segment can be sacrificed. Alternatively, cache sacrifice schemes may apply different rules. For example, the scheme may attempt to avoid sacrificing loaded segments that have not been accessed by any thread.

FIG. 7 shows an example engine 102 architecture. Engine 102 may be a Reduced Instruction Set Computing (RISC) processor customized for packet processing. For example, engine 102 may not provide floating point or integer division instructions that are typically provided by instruction sets of general-purpose processors.

The engine 102 may communicate with other network processor components (eg, shared memory) through transmit registers 192a, 192b, which buffer data to be sent/received to/from other components. Engines 102 may also communicate with other engines 102 through "neighborhood" registers 194a, 194b hardwired to the other engines.

The illustrated example engine 102 provides multiple threads of execution. To support multiple threads, the engine 102 stores a program context 182 for each thread. The context 182 may include thread state data such as a program counter. Thread arbiter 180 selects a program context 182x for a thread to execute. The program counter for the selected context is transferred to the instruction buffer 104 . Buffer 104 may initiate a program segment fetch operation when the instruction identified by the program counter is not currently cached (eg, the segment is not in a locked buffer area or in an area allocated to the currently executing thread). Otherwise, buffer 104 sends the cached instruction to instruction decode unit 186 . Potentially, instruction decode unit 190 identifies the instruction as a "fetch" instruction and initiates a fetch operation of the segment. Otherwise, the decode 190 unit may cause the instruction to be passed to an execution unit (eg, ALU) for processing, or initiate a request through the command queue 188 for a resource shared by different engines (eg, a memory controller).

The fetch control unit 184 handles fetching program segments from the shared cache 106 . For example, fetch control unit 184 may negotiate access to a shared cache request bus, issue requests, and store returned instructions into instruction cache 104 . Fetch control unit 184 may also handle the victim of previously cached instructions.

The instruction cache 104 and the decoder 186 of the engine 102 form part of an instruction processing pipeline. That is, over the course of multiple clock cycles, instructions may be loaded from cache 104′, decoder 186, and may be loaded instruction operands (e.g., from general register 196, next neighbor register 194a, transfer register 192a and local memory 198 ), and can be executed by execution data path 190 . Finally, the result of the operation may be written (eg, to general register 196, local memory 198, next neighbor register 194b, or transfer register 192b). There can be many instructions in the pipeline at the same time. That is, while one instruction is being decoded, another is being loaded from the L1 instruction cache 104 .

互联网交换网路处理器(IXP)。其他网络处理器以不同的设计为特征。FIG. 8 shows an embodiment of a network processor 200 . The network processor 200 shown is

Internet Exchange Network Processor (IXP). Other network processors feature different designs.

)，所述核心处理器210通常被编程为执行网络操作涉及到的“控制面(control plane)”任务。然而，核心处理器210还可以处理“数据面(data plane)”任务并且可以提供额外的分组处理线程。The illustrated network processor 200 features a set of packet engines 102 integrated on a single die. As noted above, a single packet engine 102 may provide multiple threads. The processor 200 may also include a core processor 210 (such as a powerful

), the core processor 210 is typically programmed to perform "control plane" tasks involved in network operations. However, core processor 210 may also handle "data plane" tasks and may provide additional packet processing threads.

As shown, network processor 200 also features an interface 202 that may communicate packets between processor 200 and other network components. For example, processor 200 may feature a switch fabric interface 202, such as a Common Switch Interface (CSIX) interface, that enables processor 200 to transmit packets to other processors or circuits connected to the fabric. Processor 200 may also feature an interface 202 , such as a System Packet Interface (SPI) interface, that enables the processor to communicate with physical layer (PHY) and/or link layer devices. Processor 200 may also include an interface 208 (eg, a Peripheral Component Interconnect (PCI) bus interface) to communicate with a host, for example. As shown, the processor 200 may also include other components shared by the engines, such as memory controllers 206, 212, hash engines, and scratch pads.

The packet processing techniques described above can be implemented in various ways on a network processor such as an IXP. For example, core processor 210 may deliver program instructions to shared instruction cache 106 during network processor boot. In addition, unlike the "two-deep" instruction cache hierarchy, for example, when the processor is characterized by a large number of engines, the processor 200 may use an N-deep instruction cache hierarchy as feature.

Figure 9 illustrates a network device that includes the techniques described above. As shown, the device features a set of line cards ("blades") interconnected by a switch fabric 310 (eg, a crossbar fabric or a shared memory switch fabric). The switch fabric can, for example, comply with CSIX or other fabric technologies, such as HyperTransport, Infiniband, Peripheral Component Interconnect Express (PCI-X), and the like.

An individual line card (eg, 300a) may include one or more physical layer (PHY) devices 302 (eg, optical, wired, and wireless PHYs) that handle communications over network connections. The PHY translates between the physical signals carried by different network media and the bits (such as "0" and "1") used by digital systems. Line card 300 may include (eg, Ethernet, Copper Optical Network (SONET), Advanced Data Link). Line card 300 may also include a framer device (e.g., Ethernet, Synchronous Optical Network (SONET), High-Level Data Link (HDLC) framer, or other "layer 2" device) 304, which may Perform operations such as error detection and/or error correction. The illustrated line card 300 may also include one or more network processors 306 using the instruction cache techniques described above. Network processor 306 is programmed to perform packet processing operations on packets received through PHY 300 and to direct the packet through switch fabric 310 to the line card providing the selected egress interface. Potentially, the network processor 306 can replace the framer device 304 to perform "layer 2" responsibilities.

Although example architectures for engines, network processors, and devices including network processors are shown in FIGS. 8 and 9, these techniques may be implemented in other engine, network processor, and device designs. Furthermore, these techniques can be used in various network devices (eg, routers, switches, bridges, hubs, traffic generators, etc.).

The term circuit as used herein includes hardwired circuits, digital circuits, analog circuits, programmable circuits, and the like. Programmable circuits can run on computer programs.

Such computer programs may be coded in a high-level procedural or object-oriented programming language. However, the programs can be implemented in assembly or machine language, if desired. Languages can be compiled or interpreted. Additionally, these techniques can be used in a variety of networking environments.

Other implementations are within the scope of the following claims.

Claims (18)

1. A processor comprising: an instruction storage storing instructions of at least a portion of at least one program; and a set of multiple multi-threaded engines coupled to the instruction store, individual ones of the multiple multi-threaded engines including engine instruction caches and circuitry responsive to fetching instructions included in the at least one program executed on at least one of the at least one program, requesting a subset of the at least a portion of instructions of the at least one program, wherein the fetching instructions includes a request to switch to a different thread, and In response to determining that a subset of the at least a portion of instructions of the at least one program is not stored in the engine instruction cache, the circuitry causes engine threads to stall until instructions for the at least one program are fetched from the instruction store. A subset of the at least a portion of instructions. 2. The processor of claim 1, wherein the engine instruction cache includes an L1 cache; and The instruction store includes an L2 cache. 3. The processor of claim 1, further comprising a second instruction store coupled to a second plurality of multithreaded engines. 4. The processor of claim 1, wherein the instruction fetch identifies a signal associated with a state of a fetch operation. 5. The processor of claim 1, wherein the instruction fetch identifies an amount of the instruction store to cache. 6. The processor of claim 5, wherein the instruction fetch identifies the quantity as a number of segments combining instructions of the program. 7. The processor of claim 1, wherein the multithreaded engine includes circuitry to select an instruction to sacrifice from the engine instruction cache. 8. The processor of claim 1, further comprising at least one of: Interfaces to switch fabrics, interfaces to media access controllers, and interfaces to physical layer devices. 9. A method for dynamically caching engine instructions comprising: In response to fetching instructions included in at least one program executing on at least one of the plurality of multi-threaded engines, requesting a subset of instructions stored by an instruction store comprised of all integrated on a single die A plurality of multi-threaded engines are shared, wherein the instruction fetch includes a request to switch to a different thread; receiving the subset of instructions at one of the plurality of multi-threaded engines requesting the subset; storing a subset of the received instructions in an instruction cache of a multithreaded engine of the plurality of multithreaded engines, and In response to determining that a subset of the at least a portion of instructions of the at least one program is not stored in the engine instruction cache, stalling engine threads until the at least a portion of the at least one program is fetched from the instruction store subset of instructions. 10. The method of claim 9, wherein the instruction store includes an L2 cache; and Wherein the instruction cache of one of the multiple multi-thread engines includes an L1 cache. 11. The method of claim 9, Wherein the instruction storage is one of a group of instruction storages, and different instruction storages of the group of instruction storages are shared by different engine groups. 12. The method of claim 9, further comprising selecting an instruction to sacrifice from the engine instruction cache. 13. The method of claim 10, further comprising executing a subset of the instructions to process packets received over the network. 14. An apparatus for dynamically caching engine instructions comprising: means for requesting, in response to fetching instructions included in at least one program executing on at least one of a plurality of multithreaded engines, a subset of instructions stored by an instruction store integrated in a single tube shared by the plurality of multi-threaded engines on the core, wherein the instruction fetch includes a request to switch to a different thread; means for receiving the subset of instructions at one of the plurality of multithreaded engines requesting the subset; means for storing a subset of the received instructions in an instruction cache of a multithreaded engine of the plurality of multithreaded engines; and for, in response to determining that the at least a subset of instructions of the at least one program is not stored in the engine instruction cache, stalling engine threads until the at least one program of the at least one program is fetched from the instruction store. means for a subset of at least a portion of the instructions. 15. A network forwarding device, comprising: exchange structural parts; A group of line cards interconnected through the switching fabric, at least one of the line card groups includes: at least one physical layer device; and at least one network processor comprising: instruction memory; A set of multi-threaded engines operatively coupled to the instruction store, individual multi-threaded engines of the set of multi-threaded engines comprising: a cache for storing instructions to be executed by the engine; and circuitry for, in response to fetching instructions included in at least one program executing on at least one of the set of multi-threaded engines, requesting a subset of instructions from the instruction store, the instructions being stored by the instruction store memory storage, the fetching instructions includes a request to switch to a different thread, and for stopping an engine thread in response to determining that a subset of the at least a portion of instructions of the at least one program is not stored in the engine instruction cache, until fetching the at least a subset of instructions of the at least one program from the instruction storage. 16. The network forwarding device of claim 15 , wherein said circuitry for requesting said subset of instructions comprises Circuitry that is invoked when an instruction to be executed is not found in the multi-threaded engine's instruction cache. 17. The network forwarding device of claim 15, wherein the circuitry for requesting the subset of instructions comprises circuitry responsive to instructions executed by the multi-threaded engine. 18. The network forwarding device of claim 15, wherein the network processor further comprises a second instruction store; and A second set of multi-threaded engines is operatively coupled to the second instruction store.

Images