CN114586019A - memory-based processor - Google Patents

Abstract

In some embodiments, an integrated circuit may include a substrate and a memory array disposed on the substrate, wherein the memory array includes a plurality of discrete memory banks. The integrated circuit can also include a processing array disposed on the substrate, wherein the processing array includes a plurality of processor sub-units, each of the plurality of processor sub-units being associated with one or more discrete memory banks among the plurality of discrete memory banks. The integrated circuit may also include a controller configured to implement at least one security measure with respect to an operation of the integrated circuit and to take one or more remedial actions if the at least one security measure is triggered.

Description

memory-based processor

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to: US Provisional Application No. 62/886,328, filed August 13, 2019; US Provisional Application No. 62/907,659, filed September 29, 2019; February 7, 2020 U.S. Provisional Application No. 62/971,912, filed; and U.S. Provisional Application No. 62/983,174, filed Feb. 28, 2020. The aforementioned applications are incorporated herein by reference in their entirety.

technical field

The present disclosure generally relates to apparatus for facilitating memory-intensive operations. In particular, the present disclosure pertains to hardware chips including processing elements coupled to dedicated memory banks. The present disclosure also relates to apparatus for improving the power efficiency and speed of memory chips. In particular, the present disclosure pertains to systems and methods for implementing partial refresh or even no refresh on memory chips. The present disclosure also pertains to sizable memory chips and dual port capabilities on memory chips.

Background technique

As both processor speed and memory size continue to increase, the significant limit to effective processing speed is the von Neumann bottleneck. Von Neumann bottlenecks are caused by throughput limitations imposed by conventional computer architectures. In particular, data transfer from memory to processor often encounters a bottleneck compared to the actual computation performed by the processor. Consequently, the number of clock cycles used to read and write to memory increases significantly with memory-intensive processing programs. These clock cycles result in lower effective processing speeds because reading and writing to memory consumes clock cycles that cannot be used to perform operations on the data. Furthermore, the computational bandwidth of a processor is typically greater than the bandwidth of the bus used by the processor to access memory.

These bottlenecks are particularly evident for memory-intensive processing programs, such as neural networks and other machine learning algorithms; database construction, index searches, and queries; and other tasks that include more read and write operations than data processing operations.

Additionally, the rapid growth in the volume and granularity of available digital data has created opportunities to develop machine learning algorithms and enabled new technologies. However, this also brings thorny challenges to the fields of databases and parallel computing. For example, the rise of social media and the Internet of Things (IoT) is generating digital data at record rates. This new data can be used to generate algorithms for a variety of purposes, ranging from new advertising technologies to more precise control methods for industrial processing programs. However, new data is difficult to store, process, analyze and dispose of.

New data resources can be huge, sometimes on the order of petabytes to zettabytes. Furthermore, the growth rate of these data resources may exceed the data processing capacity. Therefore, data scientists have turned to parallel data processing techniques to meet these challenges. In order to increase computing power and handle large amounts of data, scientists have attempted to produce systems and methods capable of parallel-intensive computing. But these existing systems and methods have not kept pace with data processing requirements, often because the technology used is limited by the technology's need for additional resources for data management, integration of segmented data, and analysis of segmented data.

To facilitate manipulation of large data sets, engineers and scientists are now looking to improve the hardware used to analyze the data. For example, new semiconductor processors or chips, such as those described herein, may be specifically designed by incorporating memory and processing functions in a single substrate fabricated in a technology more suitable for memory operations than arithmetic computing Designed for data-intensive tasks. With integrated circuits specifically designed for data-intensive tasks, it is possible to meet new data processing requirements. However, this new approach to data processing for large datasets requires addressing new issues in chip design and fabrication. For example, if a new chip designed for data-intensive tasks is fabricated using the fabrication techniques and architectures used for common chips, the new chip will have poor performance and/or unacceptable yields. Furthermore, if the new chip is designed to operate with current data handling methods, the new chip will have poor performance because current methods can limit the chip's ability to handle parallel operations.

The present disclosure describes solutions for alleviating or overcoming one or more of the problems set forth above, as well as other problems in the prior art.

SUMMARY OF THE INVENTION

In some embodiments, an integrated circuit can include a substrate and a memory array disposed on the substrate, wherein the memory array includes a plurality of discrete memory banks. The integrated circuit may also include a processing array disposed on the substrate, wherein the processing array includes a plurality of processor sub-units, each of the plurality of processor sub-units and one of the plurality of discrete memory banks or multiple discrete memory banks are associated. The integrated circuit may also include a controller configured to implement at least one safety measure with respect to an operation of the integrated circuit and to take one or more remedial actions if the at least one safety measure is triggered.

The disclosed embodiments may also include a method of protecting an integrated circuit from tampering, wherein the method includes implementing, using a controller associated with the integrated circuit, at least one security measure relative to operation of the integrated circuit and when the at least one security measure is One or more remedial actions are taken upon triggering, and wherein the integrated circuit includes: a substrate; a memory array disposed on the substrate, the memory array including a plurality of discrete memory banks; and a processing array disposed on the substrate, The processing array includes a plurality of processor subunits, each of the plurality of processor subunits being associated with one or more discrete memory banks of the plurality of discrete memory banks.

The disclosed embodiments may include an integrated circuit comprising: a substrate; a memory array disposed on the substrate, the memory array including a plurality of discrete memory banks; a processing array disposed on the substrate, the processing array including a plurality of a processor sub-unit, each of the plurality of processor sub-units is associated with one or more discrete memory banks of the plurality of discrete memory banks; and a controller configured to: implement relative to the integrated circuit at least one security measure for the operation of the ; wherein the at least one security measure includes duplicating program code in at least two different memory portions.

In some embodiments, a distributed processor memory chip is provided that includes: a substrate; a memory array disposed on the substrate; a processing array disposed on the substrate; a first communication port; and a second communication port. The memory array may include multiple discrete memory banks. The processing array may include a plurality of processor subunits, each of the plurality of processor subunits being associated with one or more discrete memory banks of a plurality of discrete memory banks. The first communication port may be configured to establish a communication connection between the distributed processor memory chip and an external entity other than another distributed processor memory chip. The second communication port may be configured to establish a communication connection between the distributed processor memory chip and the first additional distributed processor memory chip.

In some embodiments, a method of transferring data between a first distributed processor memory chip and a second distributed processor memory chip may include using a communication with the first distributed processor memory chip and the second distributed processor memory chip. A controller associated with at least one of the processor memory chips determines whether a first processor subunit of a plurality of processor subunits disposed on the first distributed processor memory chip is ready to transfer data to a processor including a second processor sub-unit in a second distributed processor memory chip; and after determining that the first processor sub-unit is ready to transfer data to the second processor sub-unit, enabling using a clock controlled by the controller The signal initiates the transfer of data from the first processor subunit to the second processor subunit.

In some embodiments, a memory cell may include: a memory array including a plurality of memory banks; at least one controller configured to control at least one aspect of a read operation with respect to the plurality of memory banks; at least one a zero value detection logic unit configured to detect multi-bit zero values stored in specific addresses of a plurality of memory banks; and wherein the at least one controller and the at least one zero value detection logic unit are configured to respond to A zero value indicator is communicated back to one or more circuits external to the memory cell upon zero value detection by the at least one zero value detection logic.

Some embodiments may include a method for detecting a zero value in a particular address of a plurality of discrete memory banks, comprising: reading data stored in an address of a plurality of discrete memory banks receiving from a circuit external to the memory cell request by the controller; in response to the received request, the zero value detection logic unit is activated by the controller to detect the zero value in the received address; and in response to the zero value detection by the zero value detection logic unit, the A zero value indicator is communicated to the circuit by the controller.

Some embodiments may include a non-transitory computer-readable medium storing a set of instructions executable by a controller of a memory unit to cause the memory unit to detect a zero value in a particular address of a plurality of discrete memory banks, the method comprising: A request to read data stored in addresses of a plurality of discrete memory banks is received from a circuit external to the memory cell; in response to the received request, a zero value detection logic unit is activated by the controller to detect a zero in the received address value; and transmitting, by the controller, a zero value indicator to the circuit in response to the zero value detection by the zero value detection logic unit.

In some embodiments, a memory cell may include: one or more memory banks; a bank controller; and an address generator; wherein the address generator is configured to convert a memory cell in a current row to be accessed in an associated memory bank The current address is provided to the bank controller, determines the predicted address of the next row to be accessed in the associated memory bank, and provides the predicted address to the bank controller before the read operation relative to the current row associated with the current address is complete .

In some embodiments, a memory cell may include: one or more memory banks, wherein each of the one or more memory banks includes a plurality of rows; a first row controller configured to control the plurality of a first subset of rows; a second row controller configured to control a second subset of the plurality of rows; a single data input to receive data to be stored in the plurality of rows; and a single data output terminal, which is used to provide data retrieved from multiple rows.

In some embodiments, a distributed processor memory chip may include: a substrate; a memory array disposed on the substrate, the memory array comprising a plurality of discrete memory banks; a processing array disposed on the substrate, the processing array Including a plurality of processor sub-units, each of the processor sub-units is associated with a corresponding dedicated memory bank of the plurality of discrete memory banks; a first plurality of buses, each of which connects the plurality of processor sub-units one of the units is connected to its corresponding dedicated memory bank; and a second plurality of buses each connecting one of the plurality of processor subunits to another of the plurality of processor subunits. At least one of the memory banks may include at least one DRAM memory pad disposed on the substrate. At least one of the processor units may include one or more logic components associated with at least one memory pad. At least one memory pad and one or more logic components may be configured to act as a cache for one or more of the plurality of processing subunits.

In some embodiments, a method of executing at least one instruction in a distributed processor memory chip may include: retrieving one or more data values from a memory array of a distributed processor memory chip; converting one or more data values Values are stored in registers formed in memory pads of the distributed processor memory chips; and one or more data values stored in the registers are accessed according to at least one instruction executed by the processor component; wherein the memory array includes an arrangement of a plurality of discrete memory banks on a substrate; wherein the processor component is a processor subunit included in a plurality of processor subunits in a processing array disposed on the substrate, wherein each of the processor subunits is associated with a corresponding dedicated memory bank of the plurality of discrete memory banks; and wherein the register is provided by a memory pad disposed on the substrate.

Some embodiments may include an apparatus comprising: a substrate; a processing unit disposed on the substrate; and a memory unit disposed on the substrate, wherein the memory unit is configured to store data to be accessed by the processing unit, and wherein the processing unit includes a memory pad configured to act as a cache for the processing unit.

The processing system is expected to process the increased amount of information at an extremely high rate. For example, the fifth generation (5G) mobile Internet is expected to receive large amounts of information streams and process these information streams at an increased rate.

The processing system may include one or more buffers and a processor. The processing operations applied by the processor may have some latency and this may require a large number of buffers. Large numbers of buffers can be expensive and/or area consuming.

Transferring large amounts of information from the buffer to the processor may require a high bandwidth connector and/or a high bandwidth bus between the buffer and the processor, which may also increase the cost and area of the processing system.

There is an increasing need to provide efficient processing systems.

The processing system is expected to process the increased amount of information at an extremely high rate. For example, the fifth generation (5G) mobile Internet is expected to receive large amounts of information streams and process these information streams at an increased rate.

The processing system may include one or more buffers and processors. The processing operations applied by the processor may have some latency and this may require a large number of buffers. Large numbers of buffers can be expensive and/or area consuming.

Transferring large amounts of information from the buffer to the processor may require a high bandwidth connector and/or a high bandwidth bus between the buffer and the processor, which may also increase the cost and area of the processing system.

There is an increasing need to provide efficient processing systems.

A disaggregated server includes multiple subsystems, each with a unique role. For example, a disaggregated server may include one or more switching subsystems, one or more computing subsystems, and one or more storage subsystems.

One or more computing subsystems and one or more storage subsystems are coupled to each other via one or more switching subsystems.

The arithmetic subsystem may include a plurality of arithmetic units.

The switching subsystem may include multiple switching units.

The storage subsystem may include multiple storage units.

The bottleneck of this disaggregated server is the bandwidth required to transfer information between subsystems.

This is especially true when performing distributed computations that require information elements to be shared among all (or at least most) computing units of different computing subsystems, such as graphics processing units.

Assume that there are N arithmetic units participating in the sharing, N is a very large integer (eg, at least 1024), and each of the N arithmetic units must send information units to (and receive from all other arithmetic units) information unit). Under these assumptions, approximately NxN transfer handlers of information units need to be executed. Bulk transfer handlers are time-consuming and energy-intensive, and will significantly limit the throughput of a disaggregated server.

There is a growing need to provide efficient disaggregated servers and efficient ways to perform distributed processing.

The database includes many entries that include multiple fields. Database processing typically involves executing one or more queries including one or more filter parameters (eg, identifying one or more related fields and one or more related field values) and also including one or more Operational parameters, the one or more operational parameters may determine the type of operation to be performed, variables or constants to be used in applying the operation, and the like.

For example, a database query may request a statistical operation (operation parameter) on all records of the database where a field has a value within a predefined range (filter parameter). For yet another example, a database query may request the deletion of records (operating parameters) that have a certain field that is less than a threshold (filtering parameters).

Large databases are usually stored in storage devices. In response to a query, the database is sent to a memory unit, typically one database section after another.

The entry of the database section is sent from the memory cell to a processor that does not belong to the same integrated circuit as the memory cell. These entries are then processed by the processor.

For each database segment of the database stored in the memory unit, processing includes the following steps: (i) selecting the records of the database segment; (ii) sending the records from the memory unit to the processor; (iii) by the processor Filtering records to determine whether the records are related; and (iv) performing one or more additional operations (summation, applying any other mathematical operations and/or statistical operations) on the related records.

The screening process ends after all records are sent to the processor and the processor determines which records are relevant.

In the case that the relevant entries of the database section are not stored in the processor, then these relevant records need to be sent to the processor for further processing after the screening phase (applying post-processing operations).

When multiple processing operations follow a single filter, then the result of each operation can be sent to the memory unit and then again to the processor.

This handler is bandwidth consuming and time consuming.

There is a growing need to provide efficient ways of performing database processing.

Word embedding is a collective term for a collection of language modeling and feature learning techniques in natural language processing (NLP), where words or phrases from a vocabulary are mapped to vectors of elements. Conceptually, it involves mathematical embedding from a space with many dimensions per word to a continuous vector space with much lower dimensions (www.wikipedia.org).

Methods for generating this map include neural networks, dimensionality reduction of word co-occurrence matrices, probabilistic models, interpretable knowledge base methods, and explicit representations in terms of the context in which the word appears.

Word and phrase embeddings have been shown to improve the performance of NLP tasks such as parsing and sentiment analysis when used as the underlying input representation.

Sentences can be segmented into words or phrases, and each segment can be represented by a vector. A statement can be represented by a matrix that includes all the vectors representing the words or phrases of the statement.

A vocabulary that maps words to vectors can be stored in a memory unit, such as a dynamic random access memory (DRAM), that can be accessed using words or phrases (or indices representing words).

These accesses can be random accesses, which reduces the throughput of the DRAM. Furthermore, these accesses can saturate the DRAM, especially when a large number of accesses are fed into the DRAM.

In particular, the words included in the sentences are usually quite random. Even when using DRAM bursts, accessing store-mapped DRAM memory will generally result in lower performance for random access, because typically during bursts, DRAM memory bank entries (in different memory banks being accessed at the same time) Only one of a small subset of entries) will store entries related to a statement.

Therefore, the throughput of DRAM memory is low and non-contiguous.

Each word or phrase of a sentence is retrieved from DRAM memory under the control of a host computer that is external to the integrated circuit of the DRAM memory and must control each word or segment representing each word based on knowledge of the location of the word Each fetch of the vector, which is a time-consuming and resource-intensive task.

Data centers and other computerized systems are expected to process and exchange increasing amounts of information at extremely high rates.

The exchange of increasing amounts of data can be a bottleneck for data centers and other computerized systems, and can cause such data centers and other computerized systems to utilize only a fraction of their capacity.

96A illustrates an example of a prior art database 12010 and a prior art server motherboard 12011. The database may include multiple servers, each server including multiple server motherboards (also denoted "CPU+memory+network"). Each server motherboard 12011 includes a CPU 12012 (such as, but not limited to, Intel's XEON) that receives traffic, is connected to a memory unit 12013 (denoted as RAM) and a plurality of database accelerators (DB accelerators) 12014 .

The DB accelerator is optional, and DB acceleration operations may be performed by the CPU 12012.

All traffic flows through the CPU, and the CPU may be coupled to the DB accelerator via a relatively limited bandwidth link, such as PCIe.

Numerous resources are dedicated to routing information units across multiple server boards.

There is an increasing need to provide efficient data centers and other computerized systems.

Artificial intelligence (AI) applications such as neural networks have grown significantly in size. To cope with the increased size of neural networks, multiple servers, each serving as AI acceleration servers (including server motherboards), are used to perform neural network processing tasks, such as, but not limited to, training. An example of a system including multiple AI acceleration servers configured in different racks is shown in FIG. 1 .

In a typical training session, a large number of images are processed simultaneously to provide a large number of values, such as losses. A large amount of value is transferred between different AI acceleration servers and results in an exceptional amount of traffic. For example, some neural network layers may be computed across multiple GPUs located in different AI acceleration servers, and may require aggregation on a network that consumes bandwidth.

The delivery of exceptional amounts of traffic requires ultra-high bandwidth, which may not be feasible or may not be cost-effective.

97A illustrates a system 12050 including subsystems, each including a switch 12051 for connecting an AI acceleration server 12052 having a server motherboard 12055 including RAM memory (RAM 12056), a central processing unit (CPU) ) 12054, a network adapter (NIC) 12053, and the CPU 12054 is connected (via the PCIe bus) to a number of AI accelerators 12057 (such as graphics processing units, AI chips (AI ASICs), FPGAs, and the like). The NICs are coupled to each other (eg, through one or more switches) by a network (using, eg, Ethernet, UDP links, and the like), and these NICs may be capable of delivering the ultra-high bandwidth required by the system.

There is a growing need to provide efficient AI computing systems.

According to other embodiments of the present disclosure, a non-transitory computer-readable storage medium may store program instructions that are executed by at least one processing device and perform any of the methods described herein.

The foregoing general description and the following detailed description are exemplary and explanatory only and do not limit the claims.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate the various disclosed embodiments. In the schema:

FIG. 1 is a schematic diagram of a central processing unit (CPU).

FIG. 2 is a schematic diagram of a graphics processing unit (GPU).

3A is a schematic diagram of one embodiment of an exemplary hardware chip consistent with the disclosed embodiments.

3B is a schematic diagram of another embodiment of an exemplary hardware chip consistent with the disclosed embodiments.

4 is a schematic diagram of a generic command executed by an exemplary hardware chip consistent with the disclosed embodiments.

5 is a schematic diagram of specialized commands executed by an exemplary hardware chip consistent with the disclosed embodiments.

6 is a schematic diagram of a processing group for use in an exemplary hardware chip consistent with the disclosed embodiments.

7A is a schematic diagram of a rectangular array of processing groups consistent with disclosed embodiments.

7B is a schematic diagram of an elliptical array of processing groups consistent with disclosed embodiments.

7C is a schematic diagram of an array of hardware chips consistent with the disclosed embodiments.

7D is a schematic diagram of another array of hardware chips consistent with the disclosed embodiments.

8 is a flowchart depicting an exemplary method for compiling a series of instructions for execution on an exemplary hardware chip consistent with the disclosed embodiments.

9 is a schematic diagram of a memory bank.

Figure 10 is a schematic diagram of a memory bank.

11 is a schematic diagram of an embodiment of an exemplary memory bank with subbank controls consistent with the disclosed embodiments.

12 is a schematic diagram of another embodiment of an exemplary memory bank with subbank controls consistent with the disclosed embodiments.

13 is a functional block diagram of an exemplary memory chip consistent with the disclosed embodiments.

14 is a functional block diagram of an exemplary set of redundant logical blocks consistent with disclosed embodiments.

15 is a functional block diagram of an exemplary logic block consistent with the disclosed embodiments.

16 is a functional block diagram of an exemplary logic block connected to a bus in accordance with the disclosed embodiments.

17 is a functional block diagram of exemplary logic blocks connected in series in accordance with the disclosed embodiments.

18 is a functional block diagram of exemplary logic blocks connected in a two-dimensional array in accordance with disclosed embodiments.

19 is a functional block diagram of exemplary logic blocks in complex connections consistent with disclosed embodiments.

20 is an exemplary flowchart illustrating a redundant block enablement process consistent with disclosed embodiments.

21 is an exemplary flow diagram illustrating an address assignment handler consistent with the disclosed embodiments.

22 is a functional block diagram of an exemplary processing device consistent with the disclosed embodiments.

23 is a functional block diagram of an exemplary processing device consistent with disclosed embodiments.

24 includes an exemplary memory configuration diagram consistent with disclosed embodiments.

25 is an exemplary flow diagram illustrating a memory configuration handler consistent with the disclosed embodiments.

26 is an exemplary flowchart illustrating a memory read processing procedure consistent with the disclosed embodiments.

27 is an exemplary flow diagram illustrating the execution of a handler consistent with the disclosed embodiments.

28 is an embodiment of a memory chip with a refresh controller consistent with the present disclosure.

29A is a refresh controller consistent with one embodiment of the present disclosure.

29B is a refresh controller consistent with another embodiment of the present disclosure.

30 is a flow diagram of one embodiment of a processing routine executed by a refresh controller consistent with the present disclosure.

31 is a flow diagram of one embodiment of a processing routine implemented by a compiler consistent with the present disclosure.

32 is a flowchart of another embodiment of a processing routine implemented by a compiler consistent with the present disclosure.

33 shows an example refresh controller configured with a stored pattern in accordance with the present disclosure.

34 is an example flow diagram of a processing routine implemented by software within a refresh controller consistent with the present disclosure.

35A shows an example wafer including a die consistent with the present disclosure.

35B shows an example memory chip connected to an input/output bus in accordance with the present disclosure.

35C shows an example wafer including memory chips arranged in rows and connected to input-output buses in accordance with the present disclosure.

35D shows two memory chips grouped and connected to an input-output bus in accordance with the present disclosure.

35E shows an example wafer consistent with the present disclosure that includes dies placed in a hexagonal lattice and connected to input-output buses.

36A-36D show various possible configurations of memory chips connected to an input/output bus consistent with the present disclosure.

37 shows an example grouping of dies sharing glue logic consistent with the present disclosure.

38A-38B show example cuts through a wafer consistent with the present disclosure.

38C shows an example arrangement of dies on a wafer consistent with the present disclosure and arrangement of input and output buses.

39 shows an example memory chip on a wafer with interconnected processor subunits consistent with the present disclosure.

40 is an example flow diagram of a process for laying out groups of memory chips from a wafer in accordance with the present disclosure.

41A is another example flow diagram of a process for laying out groups of memory chips from a wafer in accordance with the present disclosure.

41B-41C are example flowcharts of a process for determining a dicing pattern for dicing one or more groups of memory chips from a wafer, consistent with the present disclosure.

42 is an example of circuitry within a memory chip that provides dual port access along a column consistent with the present disclosure.

43 is an example of circuitry within a memory chip that provides dual port access along a row consistent with the present disclosure.

44 is an example of circuitry within a memory chip that provides dual port access along both rows and columns consistent with the present disclosure.

Figure 45A is a double read using a replicated memory array or pad.

Figure 45B is a double write using a replicated memory array or pad.

46 is an example of circuitry within a memory chip having switching elements for dual port access along a column consistent with the present disclosure.

47A is an example flow diagram of a process for providing dual port access on a single port memory array or pad in accordance with the present disclosure.

47B is an example flow diagram of another process routine for providing dual port access on a single port memory array or pad in accordance with the present disclosure.

48 is another example of circuitry within a memory chip that provides dual port access along both rows and columns consistent with the present disclosure.

49 is an example of a switching element for dual port access within a memory pad consistent with the present disclosure.

50 is an example integrated circuit having reduced cells configured to access partial words in accordance with the present disclosure.

FIG. 51 is a memory bank for use with reduced cells as described with respect to FIG. 50 .

52 is a memory bank using reduced cells integrated into PIM logic consistent with the present disclosure.

53 is a memory bank using PIM logic to enable switches for accessing partial words in accordance with the present disclosure.

54A is a memory bank with a segmented rank multiplexer for deactivating to access partial words in accordance with the present disclosure.

54B is an example flow diagram of a processing routine for partial word access in memory consistent with the present disclosure.

FIG. 55 is a conventional memory chip including multiple memory pads.

56 is an embodiment of a memory chip having a startup circuit for reducing power consumption during line disconnection consistent with the present disclosure.

57 is another embodiment of a memory chip with startup circuitry for reducing power consumption during line disconnection consistent with the present disclosure.

58 is yet another embodiment of a memory chip having startup circuitry for reducing power consumption during line disconnection consistent with the present disclosure.

FIG. 59 is yet another embodiment of a memory chip with startup circuitry for reducing power consumption during line disconnection consistent with the present disclosure.

60 is one embodiment of a memory chip having global and local word lines for reduced power consumption during line-off in accordance with the present disclosure.

61 is another embodiment of a memory chip having global and local word lines for reduced power consumption during line-off in accordance with the present disclosure.

62 is a flowchart of a processing routine for sequentially disconnecting lines in memory consistent with the present disclosure.

Figure 63 is a conventional tester for memory chips.

FIG. 64 is another conventional tester for memory chips.

65 is one embodiment of testing a memory chip using logic cells on the same substrate as the memory, consistent with the present disclosure.

66 is another embodiment of testing a memory chip using logic cells on the same substrate as the memory consistent with the present disclosure.

67 is yet another embodiment of testing a memory chip using logic cells on the same substrate as the memory consistent with the present disclosure.

68 is yet another embodiment of testing a memory chip using logic cells on the same substrate as the memory consistent with the present disclosure.

69 is another embodiment of testing a memory chip using logic cells on the same substrate as the memory consistent with the present disclosure.

70 is a flowchart of a process routine for testing memory chips consistent with the present disclosure.

71 is a flowchart of another process for testing memory chips consistent with the present disclosure.

72A is a diagrammatic representation of an integrated circuit including a memory array and a processing array in accordance with an embodiment of the present invention.

72B is a diagrammatic representation of a memory region within an integrated circuit consistent with embodiments of the present invention.

73A is a diagrammatic representation of an integrated circuit with an example configuration of a controller consistent with embodiments of the present invention.

Figure 73B is a diagrammatic representation of a configuration for concurrently executing replication models in accordance with embodiments of the present invention.

74A is a diagrammatic representation of an integrated circuit with another example configuration of a controller consistent with embodiments of the present invention.

74B is a flowchart representation of a method of protecting an integrated circuit in accordance with an illustratively disclosed embodiment.

74C is a diagrammatic representation of detection elements located at various points within a chip, according to an exemplary disclosed embodiment.

75A is a diagrammatic representation of a scalable processor memory system including multiple distributed processor memory chips consistent with an embodiment of the present invention.

75B is a diagrammatic representation of a scalable processor memory system including multiple distributed processor memory chips, consistent with an embodiment of the present invention.

Figure 75C is a diagrammatic representation of a scalable processor memory system including multiple distributed processor memory chips consistent with embodiments of the present invention.

Figure 75D is a diagrammatic representation of a dual port distributed processor memory chip consistent with an embodiment of the invention.

75E is an example timing diagram consistent with embodiments of the present invention.

76 is a diagrammatic representation of a processor memory chip having an integrated controller and interface module and constituting a scalable processor memory system in accordance with an embodiment of the present invention.

77 is a flow diagram for transferring data between processor memory chips in the scalable processor memory system shown in FIG. 75A, consistent with an embodiment of the present invention.

78A illustrates a system for detecting, at the chip level, zero values stored in one or more specific addresses of multiple memory banks implemented in a memory chip, in accordance with embodiments of the present invention.

78B illustrates a memory chip for detecting, at the memory bank level, zero values stored in one or more of particular addresses of a plurality of memory banks, consistent with embodiments of the present invention.

79 illustrates a memory bank for detecting, at the memory pad level, zero values stored in one or more of specific addresses of a plurality of memory pads, consistent with embodiments of the present invention.

80 is a flowchart illustrating an exemplary method of detecting zero values in particular addresses of multiple discrete memory banks, consistent with embodiments of the present invention.

81A illustrates a system for launching a next row associated with a memory bank based on next row prediction, consistent with embodiments of the present invention.

Figure 81B illustrates another embodiment of the system of Figure 81A consistent with embodiments of the present invention.

Figure 81C illustrates the first and second subset row controllers for each memory subset consistent with embodiments of the present invention.

Figure 81D illustrates an embodiment of next row prediction consistent with embodiments of the present invention.

Figure 81E illustrates an embodiment of a memory bank consistent with embodiments of the present invention.

Figure 81F illustrates another embodiment of a memory bank consistent with embodiments of the present invention.

Figure 82 illustrates a dual control memory bank for reducing memory row start penalties in accordance with embodiments of the present invention.

Figure 83A illustrates a first example of accessing and enabling a row of a memory bank.

Figure 83B illustrates a second example of accessing and enabling a row of a memory bank.

Figure 83C illustrates a third example of accessing and enabling a row of a memory bank.

Figure 84 provides a diagrammatic representation of a conventional CPU/register file and external memory architecture.

85A illustrates an exemplary distributed processor memory chip with memory pads serving as register files, in accordance with one embodiment.

85B illustrates an exemplary distributed processor memory chip with memory pads configured to function as register files, in accordance with another embodiment.

85C illustrates an exemplary device having a memory pad serving as a register file in accordance with another embodiment.

86 provides a flowchart representing an exemplary method for executing at least one instruction in a distributed processor memory chip consistent with the disclosed embodiments.

Figure 87A includes an example of a decomposed server;

Figure 87B is an example of distributed processing;

Figure 87C is an example of a memory/processing unit;

Figure 87D is an example of a memory/processing unit;

Figure 87E is an example of a memory/processing unit;

87F is an example of an integrated circuit including a memory/processing unit and one or more communication modules;

87G is an example of an integrated circuit including a memory/processing unit and one or more communication modules;

Figure 87H is an example of a method;

Figure 87I is an example of a method;

Figure 88A is an example of a method;

Figure 88B is an example of a method;

Figure 88C is an example of a method;

Figure 89A is an example of a memory/processing unit and vocabulary;

Figure 89B is an example of a memory/processing unit;

Figure 89C is an example of a memory/processing unit;

Figure 89D is an example of a memory/processing unit;

Figure 89E is an example of a memory/processing unit;

Figure 89F is an example of a memory/processing unit;

Figure 89G is an example of a memory/processing unit;

Figure 89H is an example of a memory/processing unit;

Figure 90A is an example of a system;

Figure 90B is an example of a system;

Figure 90C is an example of a system;

Figure 90D is an example of a system;

Figure 90E is an example of a system;

Figure 90F is an example of a method;

Figure 91A is an example of a memory and screening system, storage, and CPU;

91B is an example of a memory and processing system, storage device, and CPU;

92A is an example of a memory and processing system, storage device, and CPU;

Figure 92B is an example of a memory/processing unit;

Figure 92C is an example of a memory and screening system, storage device, and CPU;

92D is an example of a memory and processing system, storage device, and CPU;

Figure 92E is an example of a memory and processing system, storage device, and CPU;

Figure 92F is an example of a method;

Figure 92G is an example of a method;

Figure 92H is an example of a method;

Figure 92I is an example of a method;

Figure 92J is an example of a method;

Figure 92K is an example of a method;

93A is a cross-sectional view of an example of a hybrid integrated circuit;

93B is a cross-sectional view of an example of a hybrid integrated circuit;

93C is a cross-sectional view of an example of a hybrid integrated circuit;

93D is a cross-sectional view of an example of a hybrid integrated circuit;

93E is a top view of an example of a hybrid integrated circuit;

93F is a top view of an example of a hybrid integrated circuit;

93G is a top view of an example of a hybrid integrated circuit;

93H is a cross-sectional view of an example of a hybrid integrated circuit;

93I is a cross-sectional view of an example of a hybrid integrated circuit;

Figure 93J is an example of a method;

94A is an example of a storage system, one or more devices, and a computing system;

94B is an example of a storage system, one or more devices, and a computing system;

Figure 94C is an example of one or more devices and computing systems;

Figure 94D is an example of one or more devices and computing systems;

Figure 94E is an example of a database acceleration integrated circuit;

Figure 94F is an example of a database acceleration integrated circuit;

Figure 94G is an example of a database acceleration integrated circuit;

Figure 94H is an example of a database acceleration unit;

94I is an example of a group of blade and database acceleration integrated circuits;

94J is an example of a group of database acceleration integrated circuits;

94K is an example of a group of database acceleration integrated circuits;

94L is an example of a group of database acceleration integrated circuits;

94M is an example of a group of database acceleration integrated circuits;

Figure 94N is an example of a system;

Figure 940 is an example of a system;

Figure 94P is an example of a method;

Figure 95A is an example of a method;

Figure 95B is an example of a method;

Figure 95C is an example of a method;

Figure 96A is an example of a prior art system;

Figure 96B is an example of a system;

Figure 96C is an example of a database accelerator board;

Figure 96D is an example of a portion of a system;

Figure 97A is an example of a prior art system;

Figure 97B is an example of a system; and

Figure 97C is an example of an AI network adapter.

Detailed ways

The following detailed description refers to the accompanying drawings. Wherever convenient, the same reference numbers are used in the drawings and the following description to refer to the same or like parts. While several illustrative embodiments are described herein, modifications, adaptations, and other implementations are possible. For example, components illustrated in the figures may be replaced, added, or modified, and the illustrative methods described herein may be modified by substituting, reordering, removing steps, or adding steps to the disclosed methods. Accordingly, the following detailed description is not limited to the disclosed embodiments and examples. Rather, the proper scope is defined by the appended claims.

processor architecture

As used throughout this disclosure, the term "hardware chip" refers to a semiconductor wafer (such as silicon or the like) on which is formed one or more circuit elements (such as transistors, capacitors, resistors, and/or the like) thing). The circuit elements may form processing elements or memory elements. "Processing element" refers to one or more circuit elements that collectively perform at least one logical function, such as an arithmetic function, logic gates, other Boolean operations, or the like. A processing element may be a general-purpose processing element (such as a configurable plurality of transistors) or a special-purpose processing element (such as a specific logic gate or circuit elements designed to perform a specific logic function). A "memory element" refers to one or more circuit elements that can be used to store data. A "memory element" may also be referred to as a "memory cell." The memory elements may be dynamic (so that electrical refresh is required to maintain data storage), static (so that data persists for at least a period of time after a loss of power), or non-volatile memory.

The processing elements can be joined to form a processor subunit. A "processor sub-unit" may thus comprise the smallest grouping of processing elements that can execute at least one task or instruction (eg, belonging to a processor instruction set). For example, a subunit may include one or more general-purpose processing elements configured to collectively execute instructions, one or more general-purpose processing elements paired with one or more special-purpose processing elements configured to execute instructions in a complementary manner, or its analogs. The processor subunits may be arranged in an array on a substrate (eg, a wafer). Although an "array" may comprise a rectangular shape, any arrangement of subunits in an array may be formed on a substrate.

Memory elements can be bonded to form memory banks. For example, a memory bank may include one or more lines of memory elements linked along at least one wire (or other conductive connection). Additionally, the memory elements may be linked in the other direction along at least one additional wire. For example, memory elements may be arranged along word lines and bit lines, as explained below. Although a memory bank may contain lines, any arrangement of elements in a bank may be used to form a bank on a substrate. Additionally, one or more groups may be electrically coupled to at least one memory controller to form a memory array. Although a memory array may contain a rectangular arrangement of banks, any arrangement of banks in an array may be formed on a substrate.

As used further throughout this disclosure, a "bus" refers to any communication connection between components of a substrate. For example, wires or wires (forming electrical connections), optical fibers (forming optical connections), or any other connections that carry out communication between elements may be referred to as "buses."

Conventional processors pair general-purpose logic circuits with shared memory. Shared memory may store both a set of instructions for execution by logic circuits and data used for execution of the set of instructions and resulting from execution of the set of instructions. As described below, some conventional processors use cache systems to reduce latency when performing fetches from shared memory; however, conventional cache systems remain shared. Conventional processors include central processing units (CPUs), graphics processing units (GPUs), various application-specific integrated circuits (ASICs), or the like. Figure 1 shows an example of a CPU, and Figure 2 shows an example of a GPU.

As shown in FIG. 1, CPU 100 may include processing unit 110, which may include one or more processor subunits, such as processor subunit 120a and processor subunit 120b. Although not depicted in FIG. 1, each processor sub-unit may include multiple processing elements. Additionally, processing unit 110 may include one or more levels of on-chip cache. Such cache elements are typically formed on the same semiconductor die as processing unit 110, rather than being connected to processor subunits 120a and 120b via one or more buses formed in a substrate containing processor subunits 120a and 120b. 120b and the cache element. For first-level (L1) and second-level (L2) caches in conventional processors, arrangements directly on the same die rather than connected via a bus are common. Instead, in earlier processors, the L2 cache was shared among processor subunits using a backside bus between the subunit and the L2 cache. The backside bus is generally larger than the front side bus described below. Thus, because the cache is to be shared by all processor subunits on a die, cache 130 may be formed on the same die as processor subunits 120a and 120b or communicatively coupled via one or more backside buses Connected to the processor sub-units 120a and 120b. In both embodiments that do not have a bus (eg, the cache is formed directly on the die) and embodiments that use a backside bus, the cache is shared among the processor subunits of the CPU.

Additionally, the processing unit 110 is in communication with shared memory 140a and memory 140b. For example, memories 140a and 140b may represent memory banks that share dynamic random access memory (DRAM). Although depicted as having two memory banks, most conventional memory chips include between eight and sixteen memory banks. Thus, the processor sub-units 120a and 120b may use the shared memory 140a and 140b to store data, which is then manipulated by the processor sub-units 120a and 120b. However, this arrangement causes the bus between the memories 140a and 140b and the processing unit 110 to become a bottleneck when the clock speed of the processing unit 110 exceeds the data transfer speed of the bus. For conventional processors, this is often the case, resulting in an effective processing speed that is lower than the specified processing speed based on clock rate and transistor count.

Similar flaws exist in GPUs, as shown in Figure 2. GPU 200 may include a processing unit 210, which may include one or more processor subunits (eg, subunits 220a, 220b, 220c, 220d, 220e, 220f, 220g, 220h, 220i, 220j, 220k, 220l, 220m, 220n, 220o and 220p). Additionally, processing unit 210 may include one or more levels of on-chip cache and/or register files. Such cache elements are typically formed on the same semiconductor die as processing unit 210 . In fact, in the embodiment of FIG. 2, the cache 210 and the processing unit 210 are formed on the same die and are shared among all the processor sub-units, while the caches 230a, 230b, 230c and 230d are respectively formed on the processor sub-units on a subset of units and dedicated to this processor subunit.

Additionally, the processing unit 210 is in communication with shared memories 250a, 250b, 250c, and 250d. For example, memories 250a, 250b, 250c, and 250d may represent memory banks that share DRAM. Accordingly, the processor subunit of the processing unit 210 may use the shared memories 250a, 250b, 250c, and 250d to store data, which is then manipulated by the processor subunit. However, this arrangement causes the bus between the memories 250a, 250b, 250c, and 250d and the processing unit 210 to become a bottleneck, similar to that described above with respect to the CPU.

Overview of the disclosed hardware chip

FIG. 3A is a schematic diagram depicting an embodiment of an exemplary hardware chip 300 . Hardware chip 300 may include distributed processors designed to alleviate the bottlenecks described above with respect to CPUs, GPUs, and other conventional processors. A distributed processor may include multiple processor subunits that are spatially distributed on a single substrate. Furthermore, as explained above, in the distributed processor of the present disclosure, the corresponding memory banks are also spatially distributed on the substrate. In some embodiments, a distributed processor may be associated with a set of instructions, and each of the processor subunits of the distributed processor may be responsible for performing one or more tasks included in the set of instructions.

As depicted in Figure 3A, hardware chip 300 may include multiple processor subunits, eg, logic and control subunits 320a, 320b, 320c, 320d, 320e, 320f, 320g, and 320h. As further depicted in Figure 3A, each processor sub-unit may have an instance of dedicated memory. For example, logic and control subunit 320a is operably connected to dedicated memory instance 330a, logic and control subunit 320b is operably connected to dedicated memory instance 330b, logic and control subunit 320c is operably connected to dedicated memory instance 330c, The logic and control subunit 320d is operably connected to the special purpose memory instance 330d, the logic and control subunit 320e is operably connected to the special purpose memory instance 330e, the logic and control subunit 320f is operably connected to the special purpose memory instance 330f, the logic and Control subunit 320g is operably connected to dedicated memory instance 330g, and logic and control subunit 320h is operably connected to dedicated memory instance 330h.

Although FIG. 3A depicts each memory instance as a single memory bank, hardware chip 300 may include two or more memory banks as dedicated memory instances for processor sub-units on hardware chip 300 . Additionally, although FIG. 3A depicts each processor sub-unit as including both logic components and controls for dedicated memory banks, hardware chip 300 may use controls for memory banks that are at least partially related to The logical components are separated. Furthermore, as depicted in Figure 3A, two or more processor sub-units and their corresponding memory components may be grouped into, for example, processing groups 310a, 310b, 310c, and 310d. A "processing group" may represent a spatial distinction on a substrate on which the hardware chip 300 is formed. Thus, a processing group may include other controls for the memory banks in the group, eg, controls 340a, 340b, 340c, and 340d. Additionally or alternatively, a "processing group" may represent a logical grouping for the purpose of compiling code for execution on hardware chip 300 . Thus, a compiler for hardware chip 300 (described further below) may divide entire sets of instructions among processing groups on hardware chip 300 .

In addition, host 350 may provide instructions, data, and other inputs to hardware chip 300 and read outputs from the hardware chip. Thus, a set of instructions may all be executed on a single die, such as the die hosting the hardware chip 300 . In fact, the only communication outside the die may include the loading of instructions to the hardware chip 300 , any input sent to the hardware chip 300 , and any output read from the hardware chip 300 . Thus, all computation and memory operations can be performed on the die (on the hardware chip 300 ) because the processor subunit of the hardware chip 300 communicates with the dedicated memory banks of the hardware chip 300 .

3B is a schematic diagram of an embodiment of another exemplary hardware chip 300'. Although depicted as an alternative to hardware chip 300, the architecture depicted in Figure 3B may be combined, at least in part, with the architecture depicted in Figure 3A.

As depicted in FIG. 3B, hardware chip 300' may include multiple processor subunits, eg, processor subunits 350a, 350b, 350c, and 350d. As further depicted in Figure 3B, each processor sub-unit may have multiple dedicated memory instances. For example, processor subunit 350a is operably connected to special-purpose memory instances 330a and 330b, processor sub-unit 350b is operably connected to special-purpose memory instances 330c and 330d, and processor sub-unit 350c is operably connected to special-purpose memory instance 330e and 330f, and processor sub-unit 350d is operably connected to dedicated memory instances 330g and 330h. Furthermore, as depicted in Figure 3B, processor subunits and their corresponding memory components may be grouped into processing groups 310a, 310b, 310c, and 310d, for example. As explained above, a "processing group" may represent a spatial distinction on a substrate on which the hardware chip 300' is formed and/or a logical grouping for the purpose of compiling code for execution on the hardware chip 300'.

As further depicted in Figure 3B, the processor subunits may communicate with each other via a bus. For example, as shown in Figure 3B, processor subunit 350a may communicate with processor subunit 350b via bus 360a, with processor subunit 350c via bus 360c, and with processor subunit 350d via bus 360f. Similarly, processor subunit 350b may communicate with processor subunit 350a (as described above) via bus 360a, with processor subunit 350c via bus 360e, and with processor subunit 350d via bus 360d. Additionally, processor subunit 350c may communicate with processor subunit 350a (as described above) via bus 360c, with processor subunit 350b (as described above) via bus 360e, and with processor subunit 350b via bus 360b Unit 350d communicates. Accordingly, processor subunit 350d may communicate with processor subunit 350a (as described above) via bus 360f, with processor subunit 350b (as described above) via bus 360d, and with processor subunit 350b via bus 360b Unit 350c communicates (as described above). Those skilled in the art will understand that fewer buses than those depicted in Figure 3B may be used. For example, bus 360e may be eliminated so that communications between processor subunits 350b and 350c pass through processor subunits 350a and/or 350d. Similarly, bus 360f may be eliminated so that communications between processor subunit 350a and processor subunit 350d pass through processor subunit 350b or 350c.

Furthermore, those skilled in the art will understand that architectures other than those depicted in Figures 3A and 3B may be used. For example, an array of processing groups each having a single processor subunit and a memory instance can be arranged on a substrate. The processor subunit may additionally or alternatively form part of the controller for the corresponding dedicated memory bank, part of the controller for the corresponding dedicated memory memory pad, or the like.

Given the architectures described above, hardware chips 300 and 300' can significantly improve the efficiency of memory-intensive tasks compared to conventional architectures. For example, database operations and artificial intelligence algorithms (such as neural networks) are examples of memory-intensive tasks for which conventional architectures are less efficient than hardware chips 300 and 300'. Therefore, the hardware chips 300 and 300' may be referred to as database accelerator processors and/or artificial intelligence accelerator processors.

Configure the disclosed hardware chip

The hardware chip architecture described above can be configured for code execution. For example, each processor subunit may execute code (defining a set of instructions) separately from other processor subunits in the hardware chip. Thus, instead of relying on the operating system to manage multi-threading or using multi-tasking (which is concurrent rather than parallel), the hardware chips of the present disclosure may allow processor sub-units to operate in full parallel.

In addition to the fully parallel implementation described above, at least some of the instructions assigned to each processor subunit may overlap. For example, multiple processor subunits on a distributed processor may execute overlapping instructions as an implementation of, for example, an operating system or other management software, while simultaneously executing non-overlapping instructions to perform parallel tasks within the context of the operating system or other management software.

FIG. 4 depicts an exemplary processing procedure 400 by processing group 410 for executing generic commands. For example, processing group 410 may comprise a portion of a hardware chip of the present disclosure (eg, hardware chip 300, hardware chip 300', or the like).

As depicted in FIG. 4 , commands may be sent to processor sub-unit 430 paired with dedicated memory instance 420 . An external host (eg, host 350) may send the command to processing group 410 for execution. Alternatively, host 350 may have sent an instruction set including the command for storage in memory instance 420 so that processor sub-unit 430 may retrieve the command from memory instance 420 and execute the retrieved command. Accordingly, the command may be executed by processing element 440, which is a general-purpose processing element configurable to execute the received command. Additionally, processing group 410 may include controls 460 for memory instance 420 . As depicted in FIG. 4, control 460 may perform any reads and/or writes to memory instance 420 required by processing element 440 in executing the received command. After executing the command, processing group 410 may output the result of the command to, for example, an external host or to a different processing group on the same hardware chip.

In some embodiments, the processor sub-unit 430 may also include an address generator 450, as depicted in FIG. 4 . An "address generator" may include a plurality of processing elements configured to determine addresses in one or more memory banks for performing reads and writes, and may also process data at the determined addresses Perform an operation (eg, addition, subtraction, multiplication, or the like). For example, the address generator 450 may determine the address for any reads or writes to memory. In one example, the address generator 450 may improve efficiency by overwriting the read value with a new value determined based on the command when the read value is no longer needed. Additionally or alternatively, address generator 450 may select an available address for storing results from command execution. This may allow the result readout to be scheduled for a subsequent clock cycle, which is convenient for an external host. In another example, address generator 450 may determine addresses for reads and writes during multi-cycle computations, such as vector or matrix multiply-accumulate computations. Thus, address generator 450 may maintain or calculate memory addresses for reading data and writing intermediate results of multi-cycle computations so that processor sub-unit 430 may continue processing without having to store these memory addresses.

FIG. 5 depicts an exemplary processing procedure 500 by processing group 510 for executing specialized commands. For example, processing group 510 may comprise a portion of a hardware chip of the present disclosure (eg, hardware chip 300, hardware chip 300', or the like).

As depicted in FIG. 5 , specialized commands (eg, multiply-accumulate commands) may be sent to processing element 530 paired with specialized memory instance 520 . An external host (eg, host 350) may send the command to processing element 530 for execution. Thus, the command may be executed by processing element 530, which is a specialized processing element configurable to execute a particular command (including received commands), under a given signal from the host. Alternatively, processing element 530 may retrieve commands from memory instance 520 for execution. Thus, in the example of FIG. 5 , processing element 530 is a multiply-accumulate (MAC) circuit configured to execute MAC commands received from an external host or retrieved from memory instance 520 . After executing the command, processing group 410 may output the result of the command to, for example, an external host or to a different processing group on the same hardware chip. Although depicted with respect to a single command and a single result, multiple commands may be received or retrieved and executed, and multiple results may be combined on processing group 510 prior to output.

Although depicted in FIG. 5 as MAC circuitry, additional or alternative specialized circuitry may be included in processing group 510 . For example, the MAX read command (which returns the maximum value of the vector), the MAX0 read command (also known as a common function of rectifiers, which returns the entire vector and returns the maximum value of 0), or its analog.

Although depicted separately, the general processing group 410 of FIG. 4 and the specialized processing group 510 of FIG. 5 may be combined. For example, a general-purpose processor sub-unit may be coupled to one or more specialized processor sub-units to form a processor sub-unit. Thus, a general-purpose processor subunit may be used for all instructions that are not executable by one or more specialized processor subunits.

Those skilled in the art will understand that neural network implementation and other memory-intensive tasks may be handled by specialized logic circuits. For example, database queries, packet inspection, string comparisons, and other functions may improve efficiency if performed by the hardware chips described herein.

Memory-Based Architecture for Distributed Processing

On a hardware chip consistent with the present disclosure, a dedicated bus may transfer data between processor subunits on the chip and/or between the processor subunits and their corresponding dedicated memory banks. Using a dedicated bus can reduce arbitration costs because competing requests are impossible or easy to avoid using software rather than hardware.

FIG. 6 schematically depicts a schematic diagram of a processing group 600 . Processing group 600 may be used in a hardware chip (eg, hardware chip 300, hardware chip 300', or the like). The processor subunit 610 may be connected to the memory 620 via the bus 630 . Memory 620 may include random-accessible memory (RAM) elements that store data and code for execution by processor sub-unit 610 . In some embodiments, memory 620 may be an N-way memory (where N is a number equal to or greater than 1, which implies the number of segments in memory 620 that are interleaved). Because processor sub-unit 610 is coupled to memory 620 dedicated to processor sub-unit 610 via bus 630, N can be kept relatively small without compromising execution performance. This represents an improvement over conventional multi-way register files or caches, where lower N generally results in lower performance, and higher N generally results in large area and power penalties.

The size of memory 620, the number of lanes, and the width of bus 630 may be adjusted to meet the requirements of the task and application implementation of the system using processing group 600, for example, depending on the size of the data involved in one or more tasks. Memory element 620 may comprise one or more types of memory known in the art, eg, volatile memory (such as RAM, DRAM, SRAM, phase-change RAM (PRAM), magnetoresistive RAM (MRAM), Resistive RAM (ReRAM or the like) or non-volatile memory such as flash memory or ROM. According to some embodiments, a portion of memory element 620 may include a first memory type, while another portion may include another memory type. For example, the code region of memory element 620 may include ROM elements, while the data region of memory element 620 may include DRAM elements. Another example of this split is to store the weights of the neural network in flash memory and the data for computation in DRAM.

The processor sub-unit 610 includes a processing element 640, which may include a processor. The processor may be pipelined or non-pipelined, a custom reduced instruction set computing (RISC) element or other processing scheme, implemented in any commercial integrated circuit (IC) known in the art (such as ARM, ARC, RISCV, etc.), as understood by those skilled in the art. Processing element 640 may include a controller, which in some embodiments includes an arithmetic logic unit (ALU) or other controller.

According to some embodiments of the present disclosure, the processing element 640 executing the received or stored code may comprise a general-purpose processing element, and thus be flexible and capable of performing a wide variety of processing operations. When comparing the power consumed during the performance of a particular operation, the non-dedicated circuitry typically consumes more power than the particular operation dedicated circuitry. Thus, when performing certain complex arithmetic calculations, processing element 640 may consume more power and perform less efficiently than dedicated hardware. Thus, according to some embodiments, the controller of processing element 640 may be designed to perform specific operations (eg, add or "move" operations).

In an embodiment of the present disclosure, certain operations may be performed by one or more accelerators 650 . Each accelerator may be dedicated and programmed to perform specific computations (such as multiplication, floating-point vector operations, or the like). By using accelerators, the average power consumed per computation per processor subunit can be reduced, and computational throughput thereafter increased. Accelerator 650 may be selected according to the application for which the system is designed to be implemented (eg, executing neural networks, executing database queries, or the like). Accelerator 650 may be configured by processing element 640 and may operate in tandem with the processing element for reducing power consumption and accelerating computation and computation. Accelerators may additionally or alternatively be used to transfer data between the memory of processing group 600, such as intelligent direct memory access (DMA) peripherals, and MUX/DEMUX/input/output ports (eg, MUX 650 and DEMUX 660) .

Accelerator 650 may be configured to perform various functions. For example, an accelerator can be configured to perform 16-bit floating point computations or 8-bit integer computations commonly used in neural networks. Another example of accelerator functionality is the 32-bit floating point computations commonly used during the training phase of neural networks. Yet another example of accelerator functionality is query processing, such as for query processing in a database. In some embodiments, accelerator 650 may include specialized processing elements to perform these functions and/or may be configured such that it may be modified according to configuration data stored on memory element 620 .

Accelerator 650 may additionally or alternatively implement a configurable script processing list of memory moves to time the movement of data to/from memory 620 or to/from other accelerators and/or input/output. Therefore, as explained further below, all data movement within the hardware chips using processing group 600 may use software synchronization rather than hardware synchronization. For example, accelerators in one processing group (eg, group 600) may transfer data from its input to its accelerator every ten cycles, and then output data on the next cycle, thereby streaming information from the memory of the processing group to another memory.

As further depicted in FIG. 6, in some embodiments, processing group 600 may also include at least one input multiplexer (MUX) 660 connected to its input ports and at least one output DEMUX 670 connected to its output ports. These MUX/DEMUXs may be controlled by control signals (not shown) from the processing element 640 and/or from one of the accelerators 650 according to the current instruction being made by the processing element 640 and/or by an accelerator in the accelerators 650 determined by the operation performed. In some scenarios, processing group 600 may be required to transfer data from its input port to its output port (according to predefined instructions from its code memory). Thus, in addition to each of the DEMUX/MUX being connected to processing element 640 and accelerator 650, one or more of the input MUXs (eg, MUX 660 ) may also be directly connected to output DEMUXs (eg, MUX 660 ) via one or more buses DEMUX 670).

The processing group 600 of Figure 6 may be arranged in an array to form a distributed processor, eg, as depicted in Figure 7A. Processing groups may be disposed on substrate 710 to form an array. In some embodiments, the substrate 710 may comprise a semiconductor substrate such as silicon. Additionally or alternatively, the substrate 710 may comprise a circuit board, such as a flexible circuit board.

As depicted in FIG. 7A, substrate 710 may include multiple process groups, such as process group 600, disposed thereon. Thus, substrate 710 includes a memory array that includes a plurality of groups, such as groups 720a, 720b, 720c, 720d, 720e, 720f, 720g, and 720h. Further, substrate 710 includes a processing array, which may include a plurality of processor subunits, such as subunits 730a, 730b, 730c, 730d, 730e, 730f, 730g, and 730h.

Furthermore, as explained above, each processing group may include a processor sub-unit and one or more corresponding memory banks dedicated to that processor sub-unit. Thus, as depicted in Figure 7A, each subunit is associated with a corresponding dedicated memory bank, eg, processor subunit 730a is associated with memory bank 720a, processor subunit 730b is associated with memory bank 720b, processing Processor subunit 730c is associated with memory bank 720c, processor subunit 730d is associated with memory bank 720d, processor subunit 730e is associated with memory bank 720e, processor subunit 730f is associated with memory bank 720f, and processor subunit 730f is associated with memory bank 720f. Unit 730g is associated with memory bank 720g, and processor sub-unit 730h is associated with memory bank 720h.

To allow each processor subunit to communicate with its corresponding dedicated memory bank, substrate 710 may include a first plurality of buses connecting one of the processor subunits to its corresponding dedicated memory bank. Thus, bus 740a connects processor subunit 730a to memory bank 720a, bus 740b connects processor subunit 730b to memory bank 720b, bus 740c connects processor subunit 730c to memory bank 720c, and bus 740d connects processor subunit 730c to memory bank 720c. unit 730d connects to memory bank 720d, bus 740e connects processor subunit 730e to memory bank 720e, bus 740f connects processor subunit 730f to memory bank 720f, and bus 740g connects processor subunit 730g to memory bank 720g, And bus 740h connects processor subunit 730h to memory bank 720h. Furthermore, to allow each processor subunit to communicate with other processor subunits, the substrate 710 may include a second plurality of buses connecting one of the processor subunits to another of the processor subunits. In the example of Figure 7A, bus 750a connects processor subunit 730a to processor subunit 750e, bus 750b connects processor subunit 730a to processor subunit 750b, and bus 750c connects processor subunit 730b to the processor subunit 750b processor subunit 750f, bus 750d connects processor subunit 730b to processor subunit 750c, bus 750e connects processor subunit 730c to processor subunit 750g, and bus 750f connects processor subunit 730c to processor subunit 750g Unit 750d, bus 750g connects processor subunit 730d to processor subunit 750h, bus 750h connects processor subunit 730h to processor subunit 750g, bus 750i connects processor subunit 730g to processor subunit 750g , and bus 750j connects processor subunit 730f to processor subunit 750e.

Thus, in the example arrangement shown in Figure 7A, a plurality of logical processor sub-units are arranged in at least one row and at least one column. The second plurality of buses connect each processor subunit to at least one adjacent processor subunit in the same row and to at least one adjacent processor subunit in the same column. Figure 7A may be referred to as a "partial block connection".

The configuration shown in Figure 7A can be modified to form "full block connections." A full block connection includes additional buses connecting diagonal processor subunits. For example, the second plurality of buses may include between processor subunit 730a and processor subunit 730f, between processor subunit 730b and processor subunit 730e, and between processor subunit 730b and processor subunit 730g There are additional buses between, between processor subunit 730c and processor subunit 730f, between processor subunit 730c and processor subunit 730h, and between processor subunit 730d and processor subunit 730g.

Full block connections can be used for convolution computations, where data and results stored in nearby processor subunits are used. For example, during convolutional image processing, each processor subunit may receive a block of an image (such as a pixel or group of pixels). To compute the convolution result, each processor subunit may obtain data from all eight adjacent processor subunits, each of which has received a corresponding block. In a partial block connection, data from a diagonally adjacent processor subunit may be passed via other adjacent processor subunits connected to that processor subunit. Therefore, the distributed processor on the chip can be an artificial intelligence accelerator processor.

In certain embodiments of convolution computations, the NxM image may be partitioned across multiple processor subunits. Each processor sub-unit may perform convolution with an AxB filter on its corresponding block. In order to perform filtering on one or more pixels on the boundaries between blocks, each processor sub-unit may require data from adjacent processor sub-units with data that includes pixels on the same boundary piece. Thus, the code generated for each processor subunit configures that subunit to compute convolutions and fetch from the second plurality of buses whenever data from adjacent subunits is needed. Corresponding commands to output data to the second plurality of buses are provided to the subunit to ensure proper timing of the required data transfers.

The partial block connections of FIG. 7A can be modified to N partial block connections. In this modification, the second plurality of buses may further connect each processor subunit to the processing in the four directions (ie, up, down, left, and right) along which the bus of FIG. 7A extends. processor subunits within a threshold distance of the processor subunits (eg, within n processor subunits). A similar modification can be made to the full block connections (to produce N full block connections), such that the second plurality of buses further connect each processor subunit to the bus of FIG. 7A in directions other than two diagonal directions. Processor subunits that are within a threshold distance of that processor subunit (eg, within n processor subunits) in four directions along.

Other arrangements are also possible. For example, in the arrangement shown in Figure 7B, bus 750a connects processor subunit 730a to processor subunit 730d, bus 750b connects processor subunit 730a to processor subunit 730b, and bus 750c connects processor subunit 730b Unit 730b is connected to processor subunit 730c, and bus 750d connects processor subunit 730c to processor subunit 730d. Thus, in the example arrangement shown in Figure 7B, the plurality of processor subunits are arranged in a star pattern. The second plurality of buses connect each processor subunit to at least one adjacent processor subunit within the star pattern.

Other arrangements (not shown) are also possible. For example, a neighbor connection arrangement may be used such that multiple processor subunits are arranged in one or more lines (eg, similar to the situation depicted in Figure 7A). In a neighbor-connected arrangement, a second plurality of buses connects each processor subunit to the processor subunit on the left in the same line, the processor subunit in the same line on the right, the processor subunit in the same line Both the left and right processor subunits, and so on.

In another embodiment, an N linear connection arrangement may be used. In an N linear connection arrangement, the second plurality of buses connects each processor subunit to processor subunits within a threshold distance of that processor subunit (eg, within n processor subunits). The N-linear connection arrangement can be used with line arrays (described above), rectangular arrays (depicted in Figure 7A), elliptical arrays (depicted in Figure 7B), or any other geometric array.

In yet another embodiment, an N logarithmic connection arrangement may be used. In an N log-connected arrangement, a second plurality of buses connect each processor subunit to a processor subunit within a threshold distance of the power of two (eg, within ²ⁿ processor subunits) of the processor subunit. processor sub-unit. The N-log connection arrangement can be used with line arrays (described above), rectangular arrays (depicted in Figure 7A), elliptical arrays (depicted in Figure 7B), or any other geometric array.

Any of the connection schemes described above can be combined for use in the same hardware chip. For example, a full block connection can be used in one zone and a partial block connection in another zone. In another embodiment, an N linear connection arrangement may be used in one region and N full block connections in another region.

Instead of or in addition to a dedicated bus between processor subunits of a memory chip, one or more shared buses may be used to interconnect all processor subunits (or subunits of processor subunits) of a distributed processor. set). Conflicts on the shared bus can still be avoided by using code executed by the processor subunit to time data transfers on the shared bus, as explained further below. Instead of or in addition to a shared bus, a configurable bus may also be used to dynamically connect processor sub-units to form groups of processor units connected to separate buses. For example, a configurable bus may include transistors or other mechanisms that may be controlled by a processor subunit to direct data transfers to selected processor subunits.

In both Figures 7A and 7B, the plurality of processor subunits of the processing array are spatially distributed among the plurality of discrete memory banks of the memory array. In other alternative embodiments (not shown), multiple processor subunits may be clustered in one or more regions of the substrate, and multiple memory banks may be clustered in one or more other regions of the substrate. In some embodiments, a combination of spatial distribution and aggregation (not shown) may be used. For example, one region of the substrate may include a cluster of processor subunits, another region of the substrate may include a cluster of memory banks, and yet another region of the substrate may include processing arrays distributed among the memory banks.

Those skilled in the art will recognize that arraying processor groups 600 on a substrate is not an exclusive embodiment. For example, each processor sub-unit may be associated with at least two dedicated memory banks. Accordingly, processing groups 310a, 310b, 310c, and 310d of FIG. 3B may be used in place of or in combination with processing group 600 to form processing arrays and memory arrays. Other processing groups (not shown) including, for example, three, four, or more than four dedicated memory banks may be used.

Each of the plurality of processor sub-units may be configured to execute software code associated with a particular application program independently of other processor sub-units included in the plurality of processor sub-units. For example, as explained below, multiple sub-series of instructions may be grouped into machine code and provided to each processor sub-unit for execution.

In some embodiments, each dedicated memory bank includes at least one dynamic random access memory (DRAM). Alternatively, the memory bank may contain a mix of memory types such as static random access memory (SRAM), DRAM, flash memory, or the like.

In conventional processors, data sharing between processor subunits is typically performed through shared memory. Shared memory typically requires most of the chip area and/or implements a bus managed by additional hardware such as an arbiter. As described above, this bus creates a bottleneck. Additionally, shared memory, which may be off-chip, typically includes cache coherency mechanisms and more complex caches (eg, L1 cache, L2 cache, and shared DRAM) to provide accurate and up-to-date data to processor subunits. As explained further below, the dedicated bus depicted in Figures 7A and 7B allows a hardware chip without hardware management, such as an arbiter. Furthermore, the use of dedicated memory as depicted in Figures 7A and 7B allows for the elimination of complex cache layers and coherency mechanisms.

Instead, to allow each processor subunit to access data computed by the other processor subunits and/or stored in memory banks dedicated to the other processor subunits, a bus is provided whose timing is used by each processor subunit. Code executed individually by processor subunits is executed dynamically. This situation allows the elimination of most, if not all, bus management hardware as conventionally used. In addition, direct transfers on these buses replace complex caching mechanisms to reduce latency during memory reads and writes.

memory-based processing array

As depicted in Figures 7A and 7B, the memory chips of the present disclosure can operate independently. Alternatively, the memory chips of the present disclosure may be combined with one or more additional devices such as memory devices (eg, one or more DRAM banks), system-on-chip, field-programmable gate arrays (FPGA), or other processing and/or memory chips The integrated circuits are operably connected. In these embodiments, tasks in a series of instructions executed by the architecture may be divided between the processor sub-unit of the memory chip and any processor sub-unit of the additional integrated circuit (eg, by compiling a program, as described below). describe). For example, other integrated circuits may include a host (eg, host 350 of FIG. 3A ) that inputs instructions and/or data to and receives output from the memory chip.

To interconnect the memory chips of the present disclosure with one or more additional integrated circuits, the memory chips may include a memory interface, such as compliance with any of the Joint Electron Device Engineering Council (JEDEC) standards or variants thereof memory interface. One or more additional integrated circuits may then be connected to the memory interface. Thus, if the one or more additional integrated circuits are connected to multiple memory chips of the present disclosure, data may be shared among the memory chips via the one or more additional integrated circuits. Additionally or alternatively, the one or more additional integrated circuits may include a bus to connect to a bus on the memory chip of the present disclosure such that the one or more additional integrated circuits may execute code in tandem with the memory chip of the present disclosure . In these embodiments, the one or more additional integrated circuits further facilitate distributed processing, even though the additional integrated circuits may be on different substrates from the memory chips of the present disclosure.

Furthermore, the memory chips of the present disclosure may be arranged in an array to form an array of distributed processors. For example, one or more buses may connect memory chip 770a to additional memory chips 770b, as depicted in Figure 7C. In the embodiment of FIG. 7C, memory chip 770a includes processor subunits and one or more corresponding memory banks dedicated to each processor subunit, eg, processor subunit 730a is associated with memory bank 720a, processing Processor subunit 730b is associated with memory bank 720b, processor subunit 730e is associated with memory bank 720c, and processor subunit 730f is associated with memory bank 720d. A bus connects each processor subunit to its corresponding memory bank. Thus, bus 740a connects processor subunit 730a to memory bank 720a, bus 740b connects processor subunit 730b to memory bank 720b, bus 740c connects processor subunit 730e to memory bank 720c, and bus 740d connects processor subunit 730b to memory bank 720c. Subunit 730f is connected to memory bank 720d. Additionally, bus 750a connects processor subunit 730a to processor subunit 750e, bus 750b connects processor subunit 730a to processor subunit 750b, bus 750c connects processor subunit 730b to processor subunit 750f, And bus 750d connects processor subunit 730e to processor subunit 750f. For example, as described above, other arrangements of memory chips 770a may be used.

Similarly, memory chip 770b includes processor subunits and one or more corresponding memory banks dedicated to each processor subunit, eg, processor subunit 730c is associated with memory bank 720e, processor subunit 730d is associated with Memory bank 720f is associated, processor sub-unit 730g is associated with memory bank 720g, and processor sub-unit 730h is associated with memory bank 720h. A bus connects each processor subunit to its corresponding memory bank. Thus, bus 740e connects processor subunit 730c to memory bank 720e, bus 740f connects processor subunit 730d to memory bank 720f, bus 740g connects processor subunit 730g to memory bank 720g, and bus 740h connects processor subunit 730d to memory bank 720g Subunit 730h is connected to memory bank 720h. Additionally, bus 750g connects processor subunit 730c to processor subunit 750g, bus 750h connects processor subunit 730d to processor subunit 750h, bus 750i connects processor subunit 730c to processor subunit 750d, And bus 750j connects processor sub-unit 730g to processor sub-unit 750h. For example, as described above, other arrangements of memory chips 770b may be used.

The processor subunits of memory chips 770a and 770b may be connected using one or more buses. Thus, in the embodiment of FIG. 7C, bus 750e may connect processor subunit 730b of memory chip 770a with processor subunit 730c of memory chip 770b, and bus 750f may connect processor subunit 730f of memory chip 770a with The processor subunit 730c of the memory 770b is connected. For example, bus 750e may be used as an input bus to memory chip 770b (and thus as an output bus of memory chip 770a), while bus 750f may be used as an input bus to memory chip 770a (and thus as an output of memory chip 770b) bus), or vice versa. Alternatively, both buses 750e and 750f may function as bidirectional buses between memory chips 770a and 770b.

Buses 750e and 750f may include direct wires or may be interleaved on high-speed connections in order to reduce the pins used for the inter-chip interface between memory chip 770a and integrated circuit 770b. Furthermore, any of the connection arrangements described above for use in the memory chip itself may be used to connect the memory chip to one or more additional integrated circuits. For example, memory chips 770a and 770b may be connected using full block or partial block connections rather than using only two buses as shown in Figure 7C.

Thus, although depicted using buses 750e and 750f, architecture 760 may include fewer or additional buses. For example, a single bus between processor subunits 730b and 730c or between processor subunits 730f and 730c may be used. Alternatively, additional buses may be used, eg, between processor subunits 730b and 730d, between processor subunits 730f and 730d, or the like.

Furthermore, although depicted as using a single memory chip and an additional integrated circuit, multiple memory chips may be connected using a bus as explained above. For example, as depicted in the embodiment of Figure 7C, memory chips 770a, 770b, 770c, and 770d are connected in an array. Similar to the memory chips described above, each memory chip includes a processor subunit and a dedicated memory bank. Therefore, descriptions of these components are not repeated here.

In the embodiment of FIG. 7C, memory chips 770a, 770b, 770c, and 770d are connected in a loop. Thus, bus 750a connects memory chips 770a and 770d, bus 750c connects memory chips 770a and 770b, bus 750e connects memory chips 770b and 770c, and bus 750g connects memory chips 770c and 770d. While memory chips 770a, 770b, 770c, and 770d may be connected using full block connections, partial block connections, or other connection arrangements, the embodiment of Figure 7C allows fewer pin connections between memory chips 770a, 770b, 770c, and 770d .

relatively large memory

Embodiments of the present disclosure may use dedicated memory that is relatively large in size compared to the shared memory of conventional processors. Using dedicated memory rather than shared memory allows continued efficiency gains without tapering as memory increases. This allows memory-intensive tasks such as neural network processing and database queries to be performed more efficiently than in conventional processors, where the efficiency gains of increasing shared memory taper off due to the von Neumann bottleneck.

For example, in a distributed processor of the present disclosure, a memory array disposed on a substrate of the distributed processor may include a plurality of discrete memory banks. Each of the discrete memory banks may have a capacity greater than one megabyte; and a processing array disposed on the substrate, the processing array including a plurality of processor subunits. As explained above, each of the processor sub-units may be associated with a corresponding dedicated memory bank of a plurality of discrete memory banks. In some embodiments, multiple processor sub-units may be spatially distributed among multiple discrete memory banks within the memory array. By using at least one megabyte of dedicated memory for a large CPU or GPU instead of several megabytes of shared cache, the distributed processor of the present disclosure achieves the von Neumann bottleneck in conventional systems due to the CPU and GPU impossible to achieve efficiency.

Different memories can be used as dedicated memories. For example, each dedicated memory bank may include at least one DRAM bank. Alternatively, each dedicated memory bank may include at least one static random access memory bank. In other embodiments, different types of memory may be combined on a single hardware chip.

As explained above, each dedicated memory may be at least one megabyte. Thus, each dedicated memory bank may be the same size, or at least two memory banks of the plurality of memory banks may be of different sizes.

Furthermore, as described above, the distributed processor may include: a first plurality of buses each connecting one of the plurality of processor subunits to a corresponding dedicated memory bank; and a second plurality of buses, which Each connects one of the plurality of processor subunits to another of the plurality of processor subunits.

Synchronization using software

As explained above, the hardware chip of the present disclosure may manage data transfer using software rather than hardware. Specifically, because the timing of transfers on the bus, reads and writes to memory, and calculations by the processor subunits are set by the subseries of instructions executed by the processor subunits, the hardware chip of the present disclosure can Execute code to prevent collisions on the bus. Therefore, the hardware chip of the present disclosure can avoid hardware mechanisms conventionally used to manage data transfer, such as an on-chip network controller, a packet parser and packet transmitter between processor sub-units, a bus arbiter, a Avoid arbitrating multiple buses, or the like).

If the hardware chips of the present disclosure routinely transfer data, connecting the N processor sub-units using a bus would require bus arbitration or a wide MUX controlled by an arbiter. Instead, as described above, embodiments of the present disclosure may use a bus between processor subunits that are merely wires, optical cables, or the like, where the processor subunits execute code individually to avoid conflict. Accordingly, embodiments of the present disclosure may save space on the substrate as well as material cost and efficiency losses (eg, power and time consumption due to arbitration). The efficiency and space gains are even greater compared to other architectures using first-in-first-out (FIFO) controllers and/or letter boxes.

Furthermore, as explained above, in addition to one or more processing elements, each processor sub-unit may also include one or more accelerators. In some embodiments, the accelerator may read and write from the bus rather than from the processing elements. In these embodiments, additional efficiency may be gained by allowing the accelerator to transfer data during the same cycle in which the processing element performs one or more computations. However, these embodiments require additional materials for the accelerator. For example, additional transistors may be required for making accelerators.

The code may also take into account the internal behavior of the processor subunit (eg, including processing elements and/or accelerators that form part of the processor subunit), including timing and latency. For example, a compiler (as described below) may perform preprocessing that considers timing and latency when generating the sub-series of instructions that control the transfer of data.

In one embodiment, multiple processor sub-units may be tasked with computing a neural network layer containing multiple neurons that are all connected to a previous layer of a larger multiple of neurons. Assuming that the data of the previous layer is spread evenly among the multiple processor sub-units, one way to perform this calculation is to configure each processor sub-unit to transfer the data of the previous layer to the main bus in turn, and Each processor subunit then multiplies this data by the weight of the corresponding neuron implemented by the subunit. Because each processor subunit computes more than one neuron, each processor subunit will transmit data from the previous layer a number of times equal to the number of neurons. Therefore, the code for each processor subunit is not the same as the code for the other processor subunits because the subunits will transmit at different times.

In some embodiments, a distributed processor may include a substrate (eg, a semiconductor substrate such as silicon and/or a circuit board such as a flexible circuit board) having a memory array disposed on the substrate, the memory The array includes a plurality of discrete memory banks; and a processing array disposed on the substrate, the processing array including a plurality of processor subunits, as depicted, for example, in Figures 7A and 7B. As explained above, each of the processor sub-units may be associated with a corresponding dedicated memory bank of a plurality of discrete memory banks. Furthermore, as depicted in, eg, Figures 7A and 7B, a distributed processor may also include multiple buses, each of the multiple buses connecting one of the multiple processor sub-units to one of the multiple processor sub-units at least the other.

As explained above, multiple buses can be controlled by software. Thus, multiple buses may not contain sequential hardware logic components, such that data transfers between processor subunits and across corresponding ones of the multiple buses are not controlled by sequential hardware logic components. In one embodiment, multiple buses may not contain a bus arbiter, such that data transfers between processor subunits and across corresponding ones of the multiple buses are not controlled by the bus arbiter.

In some embodiments, as depicted in, eg, Figures 7A and 7B, the distributed processor may also include a second plurality of buses connecting one of the plurality of processor sub-units to a corresponding dedicated memory bank. Similar to the plurality of buses described above, the second plurality of buses may be free of sequential hardware logic components such that data transfers between processor subunits and corresponding dedicated memory banks are not controlled by sequential hardware logic components. In one embodiment, the second plurality of buses may not contain a bus arbiter, such that data transfers between the processor subunits and the corresponding dedicated memory banks are not controlled by the bus arbiter.

As used herein, the phrase "without" does not necessarily imply the absolute absence of components such as sequential hardware logic components (eg, bus arbiters, arbitration trees, FIFO controllers, mailboxes, or the like). These components may still be included in hardware chips that are described as "without" these components. Rather, the phrase "excluding" refers to the function of a hardware chip; that is, a hardware chip "excluding" sequential hardware logic elements controls the timing of its data transfers without using the sequential hardware logic elements included therein, if any. For example, a hardware chip executes code that includes a sub-series of instructions that control data transfers between processor subunits of the hardware chip, even though the hardware chip includes sequential hardware logic components as a precaution against conflicts due to errors in the executed code The same is true for supplementary preventive measures.

As explained above, the plurality of buses may include at least one of wires or optical fibers between corresponding ones of the plurality of processor subunits. Thus, in one embodiment, a distributed processor without sequential hardware logic components may include only wires or fibers, but no bus arbiters, arbitration trees, FIFO controllers, mailboxes, or the like.

In some embodiments, the plurality of processor subunits are configured to transfer data across at least one of the plurality of buses in accordance with code executed by the plurality of processor subunits. Thus, as explained below, a compiler may organize sub-series of instructions, each sub-series containing code to be executed by a uniprocessor subunit. The subseries of instructions may instruct the processor subunits when to transfer data onto one of the buses and when to retrieve data from the bus. When the sub-series executes in tandem across distributed processors, the timing of transfers between processor sub-units can be controlled by the instructions included in the sub-series to transfer and retrieve. Accordingly, the code specifies the timing of data transfers across at least one of the plurality of buses. The compiler may generate code to be executed by the uniprocessor subunit. Additionally, a compiler may generate code to be executed by a group of processor subunits. In some cases, the compiler may treat all processor sub-units collectively as if the processor sub-unit is one superprocessor (eg, a distributed processor), and the compiler may generate a superprocessor for the superprocessor defined by it /Code executed by distributed processors.

As explained above and depicted in Figures 7A and 7B, multiple processor sub-units may be spatially distributed among multiple discrete memory banks within a memory array. Alternatively, multiple processor subunits may be grouped in one or more regions of the substrate, and multiple memory banks may be grouped in one or more other regions of the substrate. In some embodiments, a combination of spatial distribution and aggregation may be used, as explained above.

In some embodiments, a distributed processor may include a substrate (eg, a semiconductor substrate including silicon and/or a circuit board such as a flexible circuit board) having a memory array disposed thereon, the memory array including Multiple discrete memory banks. Also disposed on the substrate is a processing array that includes a plurality of processor subunits, as depicted, for example, in Figures 7A and 7B. As explained above, each of the processor sub-units may be associated with a corresponding dedicated memory bank of a plurality of discrete memory banks. Furthermore, as depicted in, eg, Figures 7A and 7B, the distributed processor may also include multiple buses, each of which connects one of the multiple processor sub-units to one of the multiple discrete memory banks The corresponding dedicated memory bank.

As explained above, multiple buses can be controlled by software. Thus, multiple buses may not contain sequential hardware logic components such that data transfers between processor subunits and corresponding dedicated discrete memory banks of the multiple discrete memory banks and across corresponding ones of the multiple busses are not subject to sequential hardware Logic component control. In one embodiment, multiple buses may not contain a bus arbiter, such that data transfers between processor subunits and across corresponding ones of the multiple buses are not controlled by the bus arbiter.

In some embodiments, as depicted in, eg, Figures 7A and 7B, the distributed processor may also include a second plurality of buses connecting one of the plurality of processor sub-units to the plurality of processes at least one other of the device subunits. Similar to the plurality of buses described above, the second plurality of buses may be free of sequential hardware logic components such that data transfers between processor subunits and corresponding dedicated memory banks are not controlled by sequential hardware logic components. In one embodiment, the second plurality of buses may not contain a bus arbiter, such that data transfers between the processor subunits and the corresponding dedicated memory banks are not controlled by the bus arbiter.

In some embodiments, a distributed processor may use a combination of software timing components and hardware timing components. For example, a distributed processor may include a substrate (eg, a semiconductor substrate including silicon and/or a circuit board such as a flexible circuit board) upon which a memory array is disposed, the memory array including a plurality of discrete memory banks . Also disposed on the substrate is a processing array that includes a plurality of processor subunits, as depicted, for example, in Figures 7A and 7B. As explained above, each of the processor sub-units may be associated with a corresponding dedicated memory bank of a plurality of discrete memory banks. Furthermore, as depicted in, eg, Figures 7A and 7B, a distributed processor may also include multiple buses, each of the multiple buses connecting one of the multiple processor sub-units to one of the multiple processor sub-units at least the other. Furthermore, as explained above, the plurality of processor sub-units may be configured to execute software that controls the timing of data transfers across the plurality of buses to avoid conflicts with data transfers on at least one of the plurality of buses. In this embodiment, software may control the timing of data transfers, but the transfers themselves may be controlled, at least in part, by one or more hardware components.

In these embodiments, the distributed processor may also include a second plurality of buses connecting one of the plurality of processor subunits to a corresponding dedicated memory bank. Similar to the plurality of buses described above, the plurality of processor subunits may be configured to execute software that controls the timing of data transfers across the second plurality of buses to avoid interference with the second plurality of buses A data transfer conflict on at least one of the . In this embodiment, as explained above, software may control the timing of data transfers, but the transfers themselves may be controlled, at least in part, by one or more hardware components.

code division

As explained above, the hardware chip of the present disclosure may execute code in parallel across processor subunits included on a substrate forming the hardware chip. In addition, the hardware chip of the present disclosure may perform multitasking. For example, the hardware chips of the present disclosure may perform regional multitasking, where one group of processor subunits of the hardware chip performs one task (eg, audio processing) while another group of processor subunits of the hardware chip performs Another task (eg image processing). In another embodiment, the hardware chip of the present disclosure may perform sequential multitasking, wherein one or more processor sub-units of the hardware chip perform one task during a first time period and another during a second time period Task. A combination of regional multitasking and sequential multitasking may also be used, such that one task may be assigned to a first group of processor subunits during a first time period, while another task may be assigned to a first group of processor subunits during the first time period. A second group of processor subunits, thereafter, a third task may be assigned to the processor subunits included in the first group and the second group during the second time period.

To organize machine code for execution on a memory chip of the present disclosure, the machine code may be divided among processor subunits of the memory chip. For example, a processor on a memory chip may include a substrate and a plurality of processor subunits disposed on the substrate. The memory chip may also include a corresponding plurality of memory banks disposed on the substrate, each of the plurality of processor subunits connected to at least one that is not shared by any other processor subunits of the plurality of processor subunits A dedicated memory bank. Each processor subunit on the memory chip can be configured to execute a series of instructions independently of the other processor subunits. Each series of instructions may be executed by configuring one or more general processing elements of a processor subunit according to the code that defines the series of instructions and/or starting according to the sequences provided in the code defining the series of instructions One or more special processing elements (eg, one or more accelerators) of a processor subunit.

Thus, each series of instructions may define a series of tasks to be performed by the uniprocessor subunit. A single task may contain instructions within an instruction set defined by the architecture of one or more processing elements in a processor subunit. For example, the processor subunit may include specific registers, and a single task may push data onto registers, extract data from registers, perform arithmetic functions on data within registers, perform logical operations on data within registers, or the like . Additionally, processor subunits may be configured for any number of operands, such as 0-operand processor subunits (also known as "stackers"), 1-operand processor subunits (also known as accumulators) , a 2-operand processor subunit such as a RISC, a 3-operand processor subunit such as a complex instruction set computer (CISC), or the like. In another embodiment, the processor subunit may include one or more accelerators, and a single task may activate an accelerator to perform a specific function, such as a MAC function, MAX function, MAX-0 function, or the like.

The series of instructions may also include tasks for reading and writing to dedicated memory banks of the memory chip. For example, a task may include writing a piece of data to a memory bank of a processor subunit dedicated to performing the task, reading a piece of data from a memory bank of the processor subunit dedicated to performing the task, or the like. In some embodiments, reading and writing may be performed by the processor subunit in tandem with the controller of the memory bank. For example, the processor subunit may perform a read or write task by sending control signals to the controller to perform the read or write. In some embodiments, the control signal may include specific addresses for reading and writing. Alternatively, the processor subunit may listen to the memory controller to select addresses available for reading and writing.

Additionally or alternatively, reading and writing may be performed by one or more accelerators in tandem with the controller of the memory bank. For example, the accelerator may generate control signals for the memory controller, similar to how processor subunits generate control signals, as described above.

In any of the embodiments described above, an address generator may also be used to direct reads and writes to specific addresses of a memory bank. For example, the address generator may include processing elements configured to generate memory addresses for reading and writing. The address generator may be configured to generate addresses for increased efficiency, eg by writing a result of a later calculation to the same address as a result of a previous calculation that is no longer needed. Thus, the address generator may generate an address for the memory controller in response to a command from the processor subunit (eg, from a processing element included therein or from one or more accelerators therein) or in tandem with the processor subunit. control signal. Additionally or alternatively, the address generator may generate addresses based on some configuration or register, such as generating a nested loop structure to iterate over certain addresses in memory in a pattern.

In some embodiments, each series of instructions may contain a set of machine code that defines a corresponding series of tasks. Thus, the series of tasks described above can be encapsulated within machine code containing the series of instructions. In some embodiments, as explained below with respect to FIG. 8 , the series of tasks may be defined by a compiler that is configured to distribute the higher-order series of tasks as multiple series of tasks among multiple logic circuits. For example, a compiler may generate multiple series of tasks based on the higher-order series of tasks such that processor subunits executing the corresponding series of tasks in tandem perform the same functions as outlined by the higher-order series of tasks .

As explained further below, the higher-order series of tasks may comprise a set of instructions written in a human-readable programming language. Correspondingly, the series of tasks for each processor subunit may include a lower-order series of tasks, each of the tasks including a set of instructions written in machine code.

As explained above with respect to Figures 7A and 7B, the memory chip may also include a plurality of buses, each bus connecting one of the plurality of processor sub-units to at least another of the plurality of processor sub-units. Furthermore, as explained above, the transfer of data on the multiple buses may be controlled using software. Thus, data transfers across at least one of the plurality of buses may be predefined by the series of instructions included in a processor subunit connected to at least one of the plurality of buses. Thus, one of the tasks included in the series of instructions may include outputting data to or fetching data from one of the buses. These tasks may be performed by the processing elements of the processor subunit or by one or more accelerators included in the processor subunit. In the latter embodiment, the processor subunits may perform computations or send control signals to corresponding memory banks in the same cycle during which the accelerator fetches data from or places data on one of the buses. one on.

In one embodiment, the series of instructions included in a processor subunit connected to at least one of the plurality of buses may include an issue task that includes an issue for the processor subunit connected to at least one of the plurality of buses A command to write data to at least one of the plurality of buses. Additionally or alternatively, the series of instructions included in a processor subunit connected to at least one of the plurality of buses may include a receive task that includes a function for the processor subunit connected to at least one of the plurality of buses. command to read data from at least one of the multiple buses.

In addition to or instead of distributing code among processor subunits, data may be divided among memory banks of a memory chip. For example, as explained above, a distributed processor on a memory chip may include multiple processor sub-units disposed on the memory chip and multiple memory banks disposed on the memory chip. Each of the plurality of memory banks may be configured to store data independent of data stored in others of the plurality of memory banks, and one of the plurality of processor sub-units may be connected to at least one of the plurality of memory banks Dedicated memory bank. For example, each processor subunit may access one or more memory controllers dedicated to one or more corresponding memory banks of that processor subunit, and other processor subunits may not have access to these corresponding one or more memory banks memory controller. Thus, the data stored in each memory bank may be unique to a dedicated processor subunit. Furthermore, the data stored in each memory bank can be independent of the memory stored in the other memory banks, since no memory controller can be shared among the memory banks.

In some embodiments, as described below with respect to FIG. 8, the data stored in each of the plurality of memory banks may be defined by a compiler that is configured to distribute the data among the plurality of memory banks. Furthermore, the compiler may be configured to distribute data defined in higher-order series of tasks among multiple memory banks using multiple lower-order tasks distributed among corresponding processor subunits.

As explained further below, the higher-order series of tasks may comprise a set of instructions written in a human-readable programming language. Correspondingly, the series of tasks for each processor subunit may include a lower-order series of tasks, each of the tasks including a set of instructions written in machine code.

As explained above with respect to Figures 7A and 7B, the memory chip may also include multiple buses, each bus connecting one of the multiple processor subunits to one or more corresponding dedicated memory banks of the multiple memory banks. Furthermore, as explained above, the transfer of data on the multiple buses may be controlled using software. Thus, the transfer of data across a particular bus of the plurality of buses may be controlled by a corresponding processor sub-unit connected to the particular bus of the plurality of buses. Thus, one of the tasks included in the series of instructions may include outputting data to or fetching data from one of the buses. As explained above, these tasks may be performed by (i) the processing elements of the processor subunit or (ii) one or more accelerators included in the processor subunit. In the latter embodiment, a processor subunit may perform computations or use a bus connecting the processor subunit to other processor subunits in the same cycle during which the accelerator is connected from one or more corresponding fetches data or places data on one of the buses of the dedicated memory bank.

Thus, in one embodiment, the series of instructions included in a processor subunit connected to at least one of the plurality of buses may include issue tasks. The send task may include a command for a processor subunit connected to at least one of the plurality of buses to write data to at least one of the plurality of buses for storage in one or more corresponding dedicated memory banks. Additionally or alternatively, the series of instructions included in a processor subunit coupled to at least one of the plurality of buses may include receive tasks. The receive task may include a command for a processor subunit connected to at least one of the plurality of buses to read data from at least one of the plurality of buses for storage in one or more corresponding dedicated memory banks. Accordingly, transmit tasks and receive tasks in these embodiments may include control signals that are sent along at least one of a plurality of buses to one or more memory controllers in one or more corresponding dedicated memory banks. Furthermore, send and receive tasks may be performed by one portion of the processing subunit (eg, by one or more different accelerators of the processing subunit) concurrently with computations or other tasks performed by another portion of the processing subunit (eg, by one or more different accelerators of the processing subunit). One or more accelerators of the processing subunit) execute. Embodiments of such concurrent execution may include MAC relay commands, where receive, multiply and transmit are performed in tandem.

In addition to distributing data among memory banks, specific portions of data may also be replicated across different memory banks. For example, as explained above, a distributed processor on a memory chip may include multiple processor sub-units disposed on the memory chip and multiple memory banks disposed on the memory chip. Each of the plurality of processor sub-units can be connected to at least one dedicated memory bank of the plurality of memory banks, and each memory bank of the plurality of memory banks can be configured to store independently of other memory banks stored in the plurality of memory banks. the data in the data. Furthermore, at least some of the data stored in a particular memory bank of the plurality of memory banks may include replicas of data stored in at least another memory bank of the plurality of memory banks. For example, numbers, strings, or other types of data used in the series of instructions may be stored in multiple memory banks dedicated to different processor subunits, rather than being transferred from one memory bank to other processors in the memory chip subunit.

In one embodiment, parallel string matching may use data replication as described above. For example, multiple strings can be compared to the same string. Conventional processors may sequentially compare each of the plurality of strings to the same string. On the hardware chip of the present disclosure, the same character string can be replicated across memory banks, so that the processor subunits can compare separate character strings of the plurality of character strings with the replicated character string in parallel.

In some embodiments, as described below with respect to FIG. 8, at least some of the data replicated across a particular memory bank of the plurality of memory banks and at least another memory bank of the plurality of memory banks is defined by a compiler, the compiler Configured to replicate data across storage groups. Furthermore, the compiler may be configured to copy at least some data using a plurality of lower order tasks distributed among corresponding processor subunits.

Replication of data may be suitable for specific tasks that reuse the same portion of data across different computations. By duplicating these portions of data, different computations can be distributed among the processor subunits of the memory chip for parallel execution, and each processor subunit can store the portion of the data in and from a dedicated memory bank The stored portion is accessed (rather than pushing and fetching that portion of the data across the bus connecting the processor subunits). In one embodiment, at least some of the data replicated across a particular one of the plurality of memory banks and at least another memory bank of the plurality of memory banks may include the weights of the neural network. In this embodiment, each node in the neural network may be defined by at least one processor subunit among a plurality of processor subunits. For example, each node may contain machine code executed by at least one processor subunit that defines the node. In this embodiment, the replication of weights may allow each processor subunit to execute machine code to at least partially implement the corresponding node, while only accessing one or more dedicated memory banks (rather than executing data with other processor subunits) transmission). Because the timing of reads and writes to dedicated memory banks is independent of other processor subunits, and the timing of data transfers between processor subunits requires timing synchronization (eg, using software, as explained above), the Duplicating memory to avoid data transfers between processor subunits can further improve overall execution efficiency.

As explained above with respect to Figures 7A and 7B, the memory chip may also include multiple buses, each bus connecting one of the multiple processor subunits to one or more corresponding dedicated memory banks of the multiple memory banks. Furthermore, as explained above, the transfer of data on the multiple buses may be controlled using software. Thus, the transfer of data across a particular bus of the plurality of buses may be controlled by a corresponding processor sub-unit connected to the particular bus of the plurality of buses. Thus, one of the tasks included in the series of instructions may include outputting data to or fetching data from one of the buses. As explained above, these tasks may be performed by (i) the processing elements of the processor subunit or (ii) one or more accelerators included in the processor subunit. As explained further above, these tasks may include send tasks and/or receive tasks that include control signals that are sent along at least one of the plurality of buses to one or more of the one or more corresponding dedicated memory banks memory controller.

Figure 8 depicts a flow diagram of a method 800 for compiling a series of instructions for execution on an exemplary memory chip of the present disclosure, eg, as depicted in Figures 7A and 7B. Method 800 may be implemented by any conventional processor, whether general purpose or special purpose.

The method 800 may be performed as part of a computer program forming a compiled program. As used herein, a "compiler" refers to a compilation of higher-level languages (eg, procedural languages such as C, FORTRAN, BASIC, or the like; object-oriented languages such as Java, C++, Pascal, Python, or the like) ; etc.) into a lower-level language (eg, composite code, object code, machine code, or the like). A compiler may allow a human to program a series of instructions in a human-readable language, and then convert the human-readable language into a machine-executable language.

At step 810, the processor may assign tasks associated with the series of instructions to different ones of the processor subunits. For example, the series of instructions may be divided into subgroups to be executed in parallel across processor subunits. In one embodiment, a neural network may be divided into its nodes, and one or more nodes may be assigned to separate processor subunits. In this embodiment, each subgroup may include multiple nodes connected across different layers. Thus, a processor subunit may implement a node from a first layer of a neural network, a node from a second layer connected to a node from the first layer implemented by the same processor subunit, and the like. By assigning nodes based on their connections, data transfers between processor subunits can be reduced, which can lead to increased efficiency, as explained above.

As explained above as depicted in Figures 7A and 7B, the processor sub-units may be spatially distributed among multiple memory banks disposed on a memory chip. Thus, the assignment of tasks may be, at least in part, a spatial as well as a logical division.

At step 820, the processor may generate tasks to transfer data between pairs of processor subunits of the memory chip, each pair of processor subunits being connected by a bus. For example, as explained above, this data transfer may be controlled using software. Thus, the processor subunit can be configured to push data on the bus and fetch data on the bus at synchronous times. The resulting tasks may thus include tasks for performing this synchronous push and pull of data.

As explained above, step 820 may include preprocessing to take into account the internal behavior of the processor sub-units, including timing and latency. For example, the processor may use known times and delays of processor subunits (eg, time to push data to the bus, time to pull data from the bus, delay between computation and push or pull, or the like) ) to ensure that the resulting tasks are synchronized. Thus, data transfers involving at least one push by one or more processor subunits and at least one fetch by one or more processor subunits can occur simultaneously without Delays are caused by timing differences, delays in processor subunits, or the like.

At step 830, the processor may group the assigned and generated tasks into groups of sub-series instructions. For example, the sub-series of instructions may each contain a sequence of tasks for execution by a uniprocessor sub-unit. Thus, each of the multiple groups of sub-series instructions may correspond to a different processor sub-unit of the multiple processor sub-units. Thus, steps 810, 820 and 830 may result in dividing the series of instructions into groups of sub-series instructions. As explained above, step 820 may ensure synchronization of any data transfers between the different groups.

At step 840, the processor may generate machine code corresponding to each of the plurality of groups of the sub-series of instructions. For example, higher-order code representing a sub-series of instructions may be converted into lower-order code, such as machine code, executable by corresponding processor sub-units.

At step 850, the processor may assign the generated machine code corresponding to each of the plurality of groups of the subseries of instructions to a corresponding processor subunit of the plurality of processor subunits according to the partitioning. For example, the processor may tag each sub-series of instructions with the identifier of the corresponding processor sub-unit. Thus, when the sub-series of instructions are uploaded to the memory chip for execution (eg, by the host 350 of Figure 3A), each sub-series may be configured with a correct processor sub-unit.

In some embodiments, assigning tasks associated with the series of instructions to different ones of the processor subunits may depend, at least in part, on two or more of the processor subunits on the memory chip the spatial proximity between them. For example, as explained above, efficiency may be improved by reducing the number of data transfers between processor subunits. Thus, the processor can minimize data transfers that move data across more than two of the processor subunits. Thus, the processor may use the known layout of the memory chip in conjunction with one or more optimization algorithms, such as a greedy algorithm, to assign sub-series to processor sub-units in a manner that maximizes (at least regionally) contiguous transfers ) and minimize transfers (at least regionally) to non-adjacent processor subunits.

The method 800 may include further optimization for the memory chips of the present disclosure. For example, the processor may group the data associated with the series of instructions based on the partitioning and assign the data to the memory banks according to the grouping. Thus, the memory bank may hold data for the sub-series of instructions assigned to each processor subunit to which each memory bank is dedicated.

In some embodiments, grouping the data may include determining at least a portion of the data to be replicated in two or more of the memory banks. For example, as explained above, some data may be used across more than one sub-series of instructions. This data may be replicated across memory banks dedicated to multiple processor sub-units assigned different sub-series of instructions. This optimization can further reduce data transfers across processor subunits.

The output of method 800 may be input to a memory chip of the present disclosure for execution. For example, a memory chip may include multiple processor subunits and corresponding multiple memory banks, each processor subunit connected to at least one memory bank dedicated to the processor subunit, and the processor of the memory chip The subunit may be configured to execute the machine code generated by method 800 . As explained above with respect to FIG. 3A, the host 350 may input the machine code generated by the method 800 to the processor subunit for execution.

Subgroups and Subcontrollers

In a conventional memory bank, the controller is placed at the bank level. Each set includes a plurality of pads, which are generally arranged in a rectangular fashion, but may be arranged in any geometric shape. Each pad includes a plurality of memory cells, which are also typically arranged in a rectangular fashion, but may be arranged in any geometric shape. Each cell can store a single bit of data (eg, depending on whether the cell is held at a high voltage or a low voltage).

Embodiments of this conventional architecture are depicted in FIGS. 9 and 10 . As shown in FIG. 9, at the group level, a plurality of pads (eg, pads 930-1, 930-2, 940-1, and 940-2) may form a group 900. As shown in FIG. In a conventional rectangular organization, group 900 may be controlled across global word lines (eg, word line 950 ) and global bit lines (eg, bit line 960 ). Thus, row decoder 910 can select the correct word line based on incoming control signals (eg, a request to read from an address, a request to write to an address, or the like), and the global sense amplifier 920 (and/or A global column decoder, not shown in Figure 9) can select the correct bit line based on this control signal. Amplifier 920 may also amplify any voltage levels from the selected group during read operations. Although depicted as using row decoders for initial selection and performing upscaling along columns, groups may additionally or alternatively use column decoders for initial selection and perform upscaling along rows.

FIG. 10 depicts an embodiment of a pad 1000 . For example, pad 1000 may form part of a memory bank such as bank 900 of FIG. 9 . As depicted in FIG. 10, a plurality of cells (eg, cells 1030-1, 1030-2, and 1030-3) may form pad 1000. Each cell may include a capacitor, transistor, or other circuitry that stores at least one data bit. For example, a cell may include a capacitor or may include a flip-flop that is charged to represent a "1" and discharged to represent a "0", the flip-flop having a first state representing a "1" and representing The second state of "0". A conventional pad may contain, for example, 512 bits by 512 bits. In embodiments where pad 1000 forms part of an MRAM, ReRAM, or the like, a cell may include transistors, resistors, capacitors, or other mechanisms for isolating ions or portions of the material storing at least one data bit. For example, a cell may contain electrolyte ions having a first state representing a "1" and a second state representing a "0", a portion of a chalcogenide glass, or the like.

As further depicted in FIG. 10, in a conventional rectangular organization, pad 1000 may be controlled across regional word lines (eg, word line 1040) and regional bit lines (eg, bit line 1050). Thus, word line drivers (eg, word line drivers 1020-1, 1020-2, . (eg, request to read from address, request to write to address, refresh signal) controls the selected word line to perform a read, write, or refresh. Additionally, regional sense amplifiers (eg, regional amplifiers 1010-1, 1010-2, ..., 1010-x) and/or regional column decoders (not shown in Figure 10) can control selected bit lines to perform reads , write or refresh. The area sense amplifier can also amplify any voltage level from the selected cell during a read operation. Although depicted as using word line drivers for initial selection and amplification along columns, pads may alternatively use bit line drivers for initial selection and amplification along rows.

As explained above, a large number of pads are replicated to form memory banks. Memory groups can be grouped to form memory chips. For example, a memory chip may contain eight to thirty-two memory banks. Thus, pairing a processor subunit with a memory bank on a conventional memory chip can yield only eight to thirty-two processor subunits. Accordingly, embodiments of the present disclosure may include memory chips with additional subgroup hierarchies. These memory chips of the present disclosure can then include processor subunits with memory subunits serving as dedicated memory banks paired with processor subunits to allow a larger number of subprocessors, which can then enable in-memory computing High parallelism and performance.

In some embodiments of the present disclosure, the global row decoders and global sense amplifiers of group 900 may be replaced with subgroup controllers. Thus, the controller of the memory bank may direct the control signal to the appropriate sub-bank controller instead of sending the control signal to the global row decoder and global sense amplifier of the memory bank. Steering may be dynamically controlled or may be hardwired (eg, via one or more logic gates). In some embodiments, fuses may be used to indicate whether the controller of each subgroup or pad blocks or passes control signals to the appropriate subgroup or pad. In these embodiments, a fuse may thus be used to deactivate the faulty subgroup.

In one of these embodiments, a memory chip may include a plurality of memory banks, each memory bank having a set of controllers and a plurality of memory banks, each memory bank having a bank of row decoders and A subset of column decoders to allow reading and writing of locations on the memory subset. Each subset may include multiple memory pads, each memory pad having multiple memory cells and may have internal regional row decoders, column decoders, and/or regional sense amplifiers. The subgroup row decoder and the subgroup column decoder can process read and write requests from the bank controller or from the subgroup processor subunits for in-memory computations on the subgroup memory, as described below . Additionally, each memory bank may further have a controller configured to decide to process read and write requests from the bank controller and/or to forward read and write requests to the next level (eg, the next level of row decoder and column decoder on the pad), or block the request, eg, to allow internal processing elements or processor subunits to access memory. In some embodiments, the set of controllers may be synchronized to the system clock. However, the subgroup controller may not be synchronized to the system clock.

As explained above, the use of subgroups may allow a larger number of processor subunits to be included in a memory chip than would be the case if the processor subunits were paired with the memory banks of a conventional chip. Thus, each subgroup may further have a processor subunit that uses the subgroup as dedicated memory. As explained above, the processor subunit may include a RISC, CISC, or other general purpose processing subunit and/or may include one or more accelerators. Additionally, the processor sub-unit may include an address generator, as explained above. In any of the embodiments described above, each processor sub-unit may be configured to use row decoders and column decoders dedicated to the sub-group of that processor sub-unit without using a group controller to access the subgroup. The processor sub-units associated with the subgroups may also handle memory pads (including the decoders and memory redundancy mechanisms described below) and/or determine whether to forward and thus handle from the upper level (eg, group level or memory level) read or write requests.

In some embodiments, the subgroup controller may also include a register that stores the state of the subgroup. Thus, if the subgroup controller receives a control signal from the memory controller when the register indicates that the subgroup is in use, the subgroup controller may return an error. In embodiments where each subgroup also includes a processor subunit, the register may indicate an error if the processor subunit in the subgroup is accessing memory that conflicts with an external request from a memory controller.

Figure 11 shows an embodiment of another embodiment of a memory bank using a bank controller. In the embodiment of FIG. 11, bank 1100 has row decoders 1110, column decoders 1120, and multiple memory banks (eg, bank 1170a) with bank controllers (eg, controllers 1130a, 1130b, and 1130c). , 1170b and 1170c). The subgroup controller can include address resolvers (eg, resolvers 1140a, 1140b, and 1140c) that can determine whether to pass the request to one or more subgroups controlled by the subgroup controller.

The subset controller may also include one or more logic circuits (eg, logic 1150a, 1150b, and 1150c). For example, logic circuitry including one or more processing elements may allow one or more operations to be performed, such as flushing cells in a subgroup, clearing cells in a subgroup, or the like, without requiring a processing request from outside of group 1100 . Alternatively, the logic circuit may include a processor sub-unit, as explained above, such that the processor sub-unit has as corresponding dedicated memory any sub-group controlled by the sub-group controller. 11, logic 1150a may have subset 1170a as the corresponding dedicated memory, logic 1150b may have subset 1170b as the corresponding dedicated memory, and logic 1150c may have subset 1170c as the corresponding dedicated memory. In any of the embodiments described above, the logic circuit may have a bus to a subgroup, eg, bus 1131a, 1131b, or 1131c. As further depicted in FIG. 11, the subgroup controllers may each include multiple decoders, such as subgroup row decoders and subgroup column decoders, to allow processing elements or processor subunits or higher orders to issue commands The memory controller reads and writes addresses on the memory bank. For example, subgroup controller 1130a includes decoders 1160a, 1160b, and 1160c, subgroup controller 1130b includes decoders 1160d, 1160e, and 1160f, and subgroup controller 1130c includes decoders 1160g, 1160h, and 1160i. Based on the request from the group row decoder 1110, the subgroup controller may select a word line using a decoder included in the subgroup controller. The described system may allow a subset of processing elements or processor subunits to access memory without interrupting other groups and even other subsets, thereby allowing each subset of processor subunits to communicate with other subsets of processor subunits. The cells perform memory computations in parallel.

Additionally, each subset may include multiple memory pads, each memory pad having multiple memory cells. For example, subgroup 1170a includes pads 1190a-1, 1190a-2, ..., 1190a-x; subgroup 1170b includes pads 1190b-1, 1190b-2, ..., 1190b-x; and subgroup 1170c includes pads 1190c- 1, 1190c-2, ..., 1190c-3. As further depicted in FIG. 11, each subset may include at least one decoder. For example, subset 1170a includes decoder 1180a, subset 1170b includes decoder 1180b, and subset 1170c includes decoder 1180c. Thus, the group column decoder 1120 may select a global bit line (eg, bit line 1121a or 1121b) based on an external request, while the subgroup selected by the group row decoder 1110 may use its column decoder based on the A region bit line (eg, bit line 1181a or 1181b) is selected according to the region request of the logic circuit. Thus, each processor sub-unit can be configured to use the row and column decoders of the subset to access the subset dedicated to that processor sub-unit without using the row and column decoders. Thus, each processor subunit can access the corresponding subgroup without interrupting the other subgroups. Additionally, when a request for a subgroup is outside the processor subunit, the subgroup decoder may reflect the accessed data to the group decoder. Alternatively, in embodiments where each subset has only one row of memory pads, the regional bit lines may be the bit lines of the pads rather than the bit lines of the subset.

A combination of the following embodiments may be used: an embodiment using a subset row decoder and a subset column decoder; and the embodiment depicted in FIG. 11 . For example, group row decoders can be eliminated, but group column decoders are retained and regional bit lines are used.

FIG. 12 shows an embodiment of an embodiment of a memory subset 1200 with multiple pads. For example, subgroup 1200 may represent a portion of subgroup 1100 of FIG. 11 or may represent an alternate implementation of a memory bank. In the embodiment of FIG. 12, subset 1200 includes a plurality of pads (eg, pads 1240a and 1240b). Additionally, each pad may include multiple cells. For example, pad 1240a includes cells 1260a-1, 1260a-2, ..., 1260a-x, and pad 1240b includes cells 1260b-1, 1260b-2, ..., 1260b-x.

Each pad may be assigned a range of addresses to be assigned to the memory cells of the pad. These addresses can be configured at production time so that the pads can be moved around and so that faulty pads can be deactivated and left unused (eg, using one or more fuses, as explained further below).

Subgroup 1200 receives read and write requests from memory controller 1210 . Although not depicted in Figure 12, requests from memory controller 1210 may be filtered by the controller of subgroup 1200 and directed to the appropriate pads of subgroup 1200 for address resolution. Alternatively, at least a portion (eg, higher bits) of the requested address from memory controller 1210 may be transferred to all pads of subgroup 1200 (eg, pads 1240a and 1240b ), such that only if the assigned address range of the pads includes When the address is specified in the command, each pad can process the full address and the request associated with that address. Similar to the subgroup guidance described above, pad determination may be dynamically controlled or may be hard-wired. In some embodiments, fuses may be used to determine the address range of each pad to also allow for deactivation of faulty pads by assigning invalid address ranges. The pads may additionally or alternatively be deactivated by other common methods or the connection of fuses.

In any of the embodiments described above, each pad of the subset may include a row decoder (eg, row decoder 1230a or 1230b) for selecting a word line in the pad. In some embodiments, each pad may also include fuses and comparators (eg, 1220a and 1220b). As described above, comparators may allow each pad to determine whether to process an incoming request, and fuses may allow each pad to be deactivated in the event of a fault. Alternatively, groups and/or subgroups of row decoders may be used instead of row decoders in each pad.

Furthermore, in any of the embodiments described above, a column decoder (eg, column decoder 1250a or 1250b) included in the appropriate pad may select a regional bit line (eg, bit line 1251 or 1253) . The local bit lines can be connected to the global bit lines of the memory bank. In embodiments where a subgroup has its own local bit line, the cell's local bit line may be further connected to the subgroup's local bit line. Thus, the column decoders (and/or sense amplifiers) of the cell may be passed through, followed by the column decoders (and/or sense amplifiers) of the subsets (in the embodiments) and then read the data in the selected cells via the group's column decoders (and/or sense amplifiers).

The pads 1200 may be replicated and arrayed to form memory groups (or memory subgroups). For example, a memory chip of the present disclosure may include multiple memory banks, each memory bank having a plurality of memory subsets, and each memory bank having a Subgroup Controller. Furthermore, each memory subset may include multiple memory pads, each memory pad having multiple memory cells and having a pad row decoder and a pad column decoder (eg, as depicted in Figure 12). The pad row decoder and the pad column decoder can handle read and write requests from the subgroup controller. For example, the pad decoder may receive all requests and decide (eg, using a comparator) whether to process the request based on a known address range for each pad, or the pad decoder may be based on the subgroup (or group) controller's response to the pads Select to only receive requests within a known address range.

Controller data transfer

In addition to using processing subunits to share data, any of the memory chips of the present disclosure may also use a memory controller (or subgroup controller or pad controller) to share data. For example, a memory chip of the present disclosure may include multiple memory banks (eg, SRAM banks, DRAM banks, or the like), each memory bank having a set of controllers, a row of decoders, and a column of decoders to allow the locations on the memory bank for reading and writing; and a plurality of buses connecting each controller of the plurality of bank controllers to at least one other controller of the plurality of bank controllers. The plurality of buses may be similar to the buses connecting the processing subunits as described above, but the plurality of buses connect the set of controllers directly rather than via the processing subunits. Furthermore, although described as connecting a group controller, the bus may additionally or alternatively connect a group controller and/or a pad controller.

In some embodiments, the plurality of buses may be accessed without interrupting data transfers on the main bus of the memory bank connected to the one or more processor subunits. Thus, a memory bank (or sub-bank) can transfer data to the corresponding processor sub-units in the same clock cycle as data is transferred to or from a different memory bank (or sub-bank) Or transfer data from the corresponding processor subunit. In embodiments where each controller is connected to multiple other controllers, the controller may be configurable to select another of the other controllers for sending or receiving data. In some embodiments, each controller may be connected to at least one adjacent controller (eg, pairs of spatially adjacent controllers may be connected to each other).

Redundant logic in memory circuits

The present disclosure generally relates to memory chips having primary logic for on-chip data processing. The memory chip can include redundant logic portions that can replace defective primary logic portions to improve chip manufacturing yield. Thus, the chip may include on-chip components that allow the logic blocks in the memory chip to be configured based on individual testing of the logic portion. This feature of the chip can improve yield because memory chips with larger areas dedicated to logic are more prone to manufacturing failures. For example, DRAM memory chips with large redundant logic sections can be prone to manufacturing problems that reduce yield. However, implementing redundant logic sections can result in improved yield and reliability because the implementation enables manufacturers or users of DRAM memory chips to switch all logic sections on or off while maintaining high parallelism. It should be noted that here and throughout this disclosure, embodiments of certain memory types, such as DRAM, may be identified in order to facilitate explanation of the disclosed embodiments. It should be understood, however, that the identified memory types are not intended to be limiting in these cases. Rather, memory types such as DRAM, flash memory, SRAM, ReRAM, PRAM, MRAM, ROM, or any other memory may be used with the disclosed embodiments, even if a relatively The same is true for few embodiments.

13 is a functional block diagram of an exemplary memory chip 1300 consistent with the disclosed embodiments. The memory chip 1300 may be implemented as a DRAM memory chip. The memory chip 1300 may also be implemented as any type of volatile or non-volatile memory, such as flash memory, SRAM, ReRAM, PRAM, and/or MRAM, among others. The memory chip 1300 may include a substrate 1301 having disposed therein an address manager 1302, a memory array 1304 including a plurality of memory banks 1304(a,a) to 1304(z,z), memory logic 1306, business logic 1308, and redundancy. I Business Logic 1310. Memory logic 1306 and business logic 1308 may constitute a primary logic block, while redundant business logic 1310 may constitute a redundant block. Additionally, the memory chip 1300 may include configuration switches, which may include a deactivation switch 1312 and an activation switch 1314 . The deactivation switch 1312 and the activation switch 1314 may also be disposed in the substrate 1301 . In this application, memory logic 1306, business logic 1308, and redundant business logic 1310 may also be collectively referred to as "logical blocks."

Address manager 1302 may include row and column decoders or other types of memory aids. Alternatively or additionally, the address manager 1302 may comprise a microcontroller or processing unit.

In some embodiments, as shown in FIG. 13 , a memory chip 1300 can include a single memory array 1304 that can arrange a plurality of memory blocks on a substrate 1301 in a two-dimensional array. However, in other embodiments, the memory chip 1300 may include multiple memory arrays 1304, and each of the memory arrays 1304 may arrange memory blocks in different configurations. For example, memory blocks (also referred to as memory banks) in at least one of the memory arrays may be configured in a radial distribution to facilitate routing between address manager 1302 or memory logic 1306 to memory blocks.

Business logic 1308 may be used to perform in-memory computations for applications independent of the logic used to manage the memory itself. For example, business logic 1308 may implement AI-related functions, such as floating point, integer, or MAC operations used as initiating functions. Additionally, business logic 1308 may implement database related functions such as min, max, sorting, counting, and others. Memory logic 1306 may perform tasks related to memory management including, but not limited to, read, write, and refresh operations. Thus, business logic may be added at one or more of the group level, the pad level, or the pad group level. Business logic 1308 may have one or more address outputs and one or more data inputs/outputs. For example, business logic 1308 may be addressed via row\column lines to address manager 1302. However, in some embodiments, logic blocks may additionally or alternatively be addressed via data input\output.

Redundant business logic 1310 may be a reproduction of business logic 1308 . Additionally, redundant business logic 1310 may be connected to deactivation switch 1312 and/or activation switch 1314, which may include small fuses\antifuses, and used to logically disable or enable one of the instances (eg, a preset connected instance) and enable one of the other logical blocks (eg, the default disconnected instance). In some embodiments, as further described with respect to FIG. 15 , the redundancy of blocks may be regional within a logical block such as business logic 1308 .

In some embodiments, logic blocks in memory chip 1300 may be connected to a subset of memory array 1304 through dedicated buses. For example, the set of memory logic 1306, business logic 1308, and redundant business logic 1310 may be connected to the first row of memory blocks in memory array 1304 (ie, memory blocks 1304(a,a) through 1304(a,z) )). A dedicated bus may allow the associated logical block to quickly access the data of the memory block without requiring the opening of communication lines via, for example, the address manager 1302.

Each of the plurality of primary logical blocks may be connected to at least one of the plurality of memory banks 1304 . Additionally, redundant blocks, such as redundant business block 1310, may be connected to at least one of memory instances 1304(a,a)-1304(z,z). A redundant block may replicate at least one of multiple primary logical blocks, such as memory logic 1306 or business logic 1308 . The deactivation switch 1312 may be connected to at least one of the plurality of primary logical blocks, and the activation switch 1314 may be connected to at least one of the plurality of redundant blocks.

In these embodiments, upon detection of a failure associated with one of the plurality of primary logic blocks (memory logic 1306 and/or business logic 1308 ), the deactivation switch 1312 may be configured to disable the plurality of primary logic blocks The one in the logical block. Meanwhile, the enable switch 1314 may be configured to enable a redundant block of the plurality of redundant blocks that reproduces one of the plurality of primary logical blocks, such as the redundant logical block 1310 .

In addition, activation switch 1314 and deactivation switch 1312, which may be collectively referred to as "configuration switches," may include external inputs to configure the state of the switches. For example, enable switch 1314 may be configured such that an enable signal in the external input generates a closed switch condition, while deactivation switch 1312 may be configured such that a deactivate signal in the external input generates an open switch condition. In some embodiments, all configuration switches in 1300 may be deactivated by default and become activated or enabled after a test indicates that the associated logic block is functional and a signal is applied to an external input. Alternatively, in some cases, all configuration switches in 1300 may be enabled by default, and may be deactivated or deactivated after testing indicates that the associated logic block is inoperative and the deactivation signal is applied to the external input.

Regardless of whether the configuration switch is initially enabled or disabled, upon detection of a failure associated with the associated logic block, the configuration switch may disable the associated logic block. With the configuration switch initially enabled, the state of the configuration switch may be changed to disabled in order to disable the associated logic block. In the event that the configuration switch is initially deactivated, the state of the configuration switch may remain in its deactivated state in order to deactivate the associated logical block. For example, the results of an operability test may indicate that a logic block does not operate or that the logic block cannot operate within certain specifications. Under these conditions, logical blocks may be disabled, possibly without their corresponding configuration switches enabled.

In some embodiments, a configuration switch may be connected to two or more logic blocks, and may be configured to select between different logic blocks. For example, a configuration switch may be connected to both business logic block 1308 and redundant logic block 1310. A configuration switch can enable redundant logic blocks 1310 while disabling business logic 1308.

Alternatively or additionally, at least one of the plurality of primary logic blocks (memory logic 1306 and/or business logic 1308 ) may be connected to a plurality of memory banks or subsets of memory instances 1304 through a first dedicated connection. Next, at least one redundant block of the plurality of redundant blocks that reproduces at least one of the plurality of primary logic blocks, such as redundant business logic 1310, may be connected to the same plurality by a second dedicated connection A subset of memory banks or instances 1304.

Furthermore, memory logic 1306 may have different functions and capabilities than business logic 1308 . For example, while memory logic 1306 may be designed to implement read and write operations in memory bank 1304, business logic 1308 may be designed to perform in-memory computations. Thus, if business logic 1308 includes a first block of business logic and business logic 1308 includes a second block of business logic (eg, redundant business logic 1310 ), it is possible to disconnect and reconnect redundant business logic 1308 to the defective business logic 1308 Business logic 1310 is such that no capability is lost.

In some embodiments, configuration switches (including deactivation switch 1312 and activation switch 1314 ) may be implemented with fuses, antifuses, or programmable devices (including one-time programmable devices) or other forms of non-volatile memory .

14 is a functional block diagram of an exemplary redundant logical block set 1400 consistent with disclosed embodiments. In some embodiments, the set of redundant logical blocks 1400 may be disposed in the substrate 1301 . Redundant logic block set 1400 may include at least one of business logic 1308 and redundant business logic 1310 connected to switches 1312 and 1314, respectively. Additionally, business logic 1308 and redundant business logic 1310 may be connected to address bus 1402 and data bus 1404 .

In some embodiments, as shown in FIG. 14, switches 1312 and 1314 may connect logic blocks to clock nodes. In this way, the configuration switch can engage or disengage the logic block from the clock signal to effectively activate or deactivate the logic block. However, in other embodiments, switches 1312 and 1314 may connect logical blocks to other nodes for activation or deactivation. For example, a configuration switch may connect the logic block to a voltage supply node (eg, VCC) or to a ground node (eg, GND) or a clock signal. In this way, a logic block can be enabled or disabled by a configuration switch because the configuration switch can create an open circuit or cut off power to the logic block.

In some embodiments, as shown in FIG. 14, the address bus 1402 and the data bus 1404 may be in opposite sides of a logic block connected in parallel to each of the buses. In this manner, routing of different on-chip components may be facilitated through logical block set 1400 .

In some embodiments, each of the plurality of deactivation switches 1312 couples at least one of the plurality of primary logic blocks to the clock node, and each of the plurality of activation switches 1314 may couple the plurality of redundant blocks At least one of the blocks is coupled to a clock node to allow connecting/disconnecting the clock as a simple enable/disable mechanism.

The redundant business logic 1310 of the redundant logical block set 1400 allows the designer to select blocks worth replicating based on area and routing. For example, chip designers may choose larger blocks to replicate because larger blocks can be more prone to errors. Therefore, chip designers may decide to replicate large logic blocks. On the other hand, designers may prefer to copy smaller logical blocks because smaller logical blocks are easy to copy without significant space loss. Furthermore, using the configuration in FIG. 14, the designer can easily select replicated logical blocks depending on the statistics of errors for each region.

15 is a functional block diagram of an exemplary logic block 1500 consistent with the disclosed embodiments. The logic block may be business logic 1308 and/or redundant business logic 1310. However, in other embodiments, the example logic blocks may describe memory logic 1306 or other components of memory chip 1300 .

Logic block 1500 presents yet another embodiment of using logical redundancy within a small processor pipeline. Logic block 1500 may include registers 1508 , fetch circuits 1504 , decoders 1506 , and write-back circuits 1518 . Additionally, logical block 1500 may include computing unit 1510 and replicate computing unit 1512 . However, in other embodiments, logic block 1500 may include other units that do not include a controller pipeline, but include discrete processing elements that include the required business logic.

Computing unit 1510 and replica computing unit 1512 may include digital circuits capable of performing digital computations. For example, compute unit 1510 and replicate compute unit 1512 may include arithmetic logic units (ALUs) to perform arithmetic and bit-wise operations on binary numbers. Alternatively, compute unit 1510 and replicate compute unit 1512 may include floating point units (FPUs) that operate on floating point numbers. Furthermore, in some embodiments, the computation unit 1510 and the replicate computation unit 1512 may implement database related functions, such as min, max, count and compare operations, and others.

In some embodiments, as shown in FIG. 15 , computing unit 1510 and replica computing unit 1512 may be connected to switching circuits 1514 and 1516 . When activated, the switch circuit can enable or disable the computing unit.

In logical block 1500, a duplicate computing unit 1512 may replicate the computing unit 1510. Furthermore, in some embodiments, registers 1508 , fetch circuits 1504 , decoders 1506 , and write-back circuits 1518 (collectively referred to as local logic units) may be smaller in size than compute unit 1510 . Because larger elements are more prone to problems during manufacture, a designer may decide to duplicate larger cells (such as compute unit 1510) rather than copy smaller cells (such as regional logic cells). However, depending on historical yield and error rates, the designer may also choose to copy regional logic cells in addition to or instead of copying a large cell (or an entire block). For example, computation unit 1510 may be larger than register 1508, fetch circuit 1504, decoder 1506, and write-back circuit 1518, and thus be more prone to errors. Instead of duplicating other elements in logical block 1500 or the entire block, the designer may choose to duplicate computing unit 1510 .

The logic block 1500 may include a plurality of regional configuration switches, each of the plurality of regional configuration switches being connected to at least one of the computing unit 1510 or at least one of the replica computing units 1512 . When a failure in compute unit 1510 is detected, the regional configuration switch may be configured to disable compute unit 1510 and enable duplicate compute unit 1512.

16 shows a functional block diagram of an exemplary logic block connected to a bus in accordance with the disclosed embodiments. In some embodiments, logic blocks 1602 (which may represent memory logic 1306, business logic 1308, or redundant business logic 1310) may be independent of each other, may be connected via a bus, and may be External start. For example, the memory chip 1300 may include many logical blocks, each logical block having an ID number. However, in other embodiments, logic block 1602 may represent a larger unit comprised of several (one or more) of memory logic 1306 , business logic 1308 , or redundant business logic 1310 .

In some embodiments, each of logical blocks 1602 may be redundant with other logical blocks 1602 . This full redundancy that all blocks can operate as primary or redundant blocks can improve manufacturing yields because designers can disconnect faulty cells while maintaining the functionality of the entire chip. For example, a designer may be able to disable logic regions that are error-prone but maintain similar computing power because all replicated blocks can be connected to the same address bus and data bus. For example, the initial number of logical blocks 1602 may be greater than the target capacity. Thus, disabling some logical blocks 1602 will not affect the target capacity.

Buses connected to the logic blocks may include address bus 1614 , command lines 1616 and data lines 1618 . As shown in FIG. 16, each of the logic blocks may be connected independently of each line in the bus. However, in some embodiments, logic blocks 1602 may be connected in a hierarchical structure to facilitate routing. For example, each line in the bus may be connected to a multiplexer that routes the line to a different logic block 1602.

In some embodiments, to allow external access without knowledge of the internal chip structure (which may change due to enabling and disabling cells), each of the logic blocks may include a blow ID, such as a blow ID 1604 . Fuse identification 1604 may include an array of ID-determining switches (eg, fuses), and may be connected to management circuitry. For example, the circuit breaker 1604 may be connected to the address manager 1302. Alternatively, the blown flag 1604 may be connected to a higher memory address location. In these embodiments, the fuse flag 1604 may be configurable for a specific address. For example, the fuse identification 1604 may include a programmable non-volatile device that determines the final ID based on instructions received from the management circuit.

A distributed processor on a memory chip can be designed with the configuration depicted in FIG. 16 . A test program executed as a BIST at chip wake-up or at factory test can assign run ID numbers to blocks of a number of primary logic blocks (memory logic 1306 and business logic 1308) that pass the test protocol. The test program may also assign invalid ID numbers to blocks of multiple primary logical blocks that fail the test protocol. The test program may also assign run ID numbers to blocks (redundant logic block 1310) of the plurality of redundant blocks that pass the test protocol. Because redundant blocks replace failed primary logical blocks, blocks in multiple redundant blocks assigned operational ID numbers may be equal to or greater than blocks in multiple primary logical blocks assigned illegal ID numbers block, thereby deactivating the block. In addition, each of the plurality of primary logical blocks and each of the plurality of redundant blocks may include at least one fuse indicator 1604 . Additionally, as shown in FIG. 16, the bus connecting logic block 1602 may include command lines, data lines, and address lines.

However, in other embodiments, all logic blocks 1602 connected to the bus will initially be disabled and have no ID numbers. Tested one by one, each good logical block will get a running ID number, and the logical blocks that are not working will retain the invalid ID, which will disable the blocks. In this way, redundant logic blocks can improve manufacturing yield by replacing blocks that are known to be defective during the test process.

An address bus 1614 may couple management circuitry to each of the plurality of memory banks, each of the plurality of primary logical blocks, and each of the plurality of redundant blocks. These connections allow management circuitry to assign invalid addresses to one of multiple primary logical blocks and to assign valid addresses to multiple redundant logical blocks upon detection of a failure associated with a primary logical block, such as business logic 1308 one of the blocks.

For example, as shown in FIG. 16A, an invalid ID is configured to all logical blocks 1602(a)-1602(c) (eg, address 0xFFF). After testing, logic blocks 1602(a) and 1602(c) were verified to be functional, while logic block 1602(b) was not functional. In FIG. 16A, unshaded logical blocks may represent logical blocks that successfully passed functional testing, while shaded logical blocks may represent logical blocks that failed functional testing. Thus, the test program changes invalid IDs to valid IDs for functional logical blocks, while retaining invalid IDs for non-functional logical blocks. As an example, in FIG. 16A, the addresses of logical blocks 1602(a) and 1602(c) are changed from 0xFFF to 0x001 and 0x002, respectively. In contrast, the address of the logical block 1602(b) is still the illegal address 0xFFF. In some embodiments, the ID is changed by programming the corresponding fuse identification 1604.

Different results from the tests of logic block 1602 may result in different configurations. For example, as shown in Figure 16B, address manager 1302 may initially assign invalid IDs to all logical blocks 1602 (ie, 0xFFF). However, the test results may indicate that both logic blocks 1602(a) and 1602(b) are functional. Under these circumstances, testing of logic block 1602(c) may not be necessary since memory chip 1300 may only require two logic blocks. Therefore, in order to minimize testing resources, logical blocks may only be tested according to the minimum number of functional logical blocks required by the product definition of 1300, leaving other logical blocks untested. 16B also shows unshaded logic blocks representing tested logic blocks that passed functional testing and shaded logic blocks representing untested logic blocks.

In these embodiments, a production tester (external or internal, automated or manual) or controller that performs BIST at startup can change the invalid ID to a run ID for the tested logic block that is functioning, but Untested logical blocks retain illegal IDs. As an example, in FIG. 16B, the addresses of logical blocks 1602(a) and 1602(b) are changed from 0xFFF to 0x001 and 0x002, respectively. In contrast, the address of the untested logical block 1602(c) is still the illegal address 0xFFF.

17 is a functional block diagram of exemplary cells 1702 and 1712 connected in series in accordance with disclosed embodiments. Figure 17 may represent the entire system or chip. Alternatively, Figure 17 may represent a block in a chip containing other active blocks.

Cells 1702 and 1712 may represent complete cells that include multiple logical blocks, such as memory logic 1306 and/or business logic 1308 . In these embodiments, units 1702 and 1712 may also include elements required to perform operations, such as address manager 1302 . However, in other embodiments, units 1702 and 1712 may represent logical units such as business logic 1308 or redundant business logic 1310.

Figure 17 presents an embodiment in which units 1702 and 1712 may need to communicate between themselves. Under such conditions, cells 1702 and 1712 may be connected in series. However, non-working units can break the continuity between logical blocks. Thus, the connection between cells may include bypass options when cells need to be deactivated due to a defect. The bypass option can also be part of the bypass unit itself.

In Figure 17, cells can be connected in series (eg, 1702(a) to 1702(c)), and a failed cell (eg, 1702(b)) can be bypassed if it is defective. The unit may further be connected in parallel with the switching circuit. For example, in some embodiments, cells 1702 and 1712 may be connected with switch circuits 1722 and 1728, as depicted in FIG. 17 . In the embodiment depicted in Figure 17, cell 1702(b) is defective. For example, cell 1702(b) fails the circuit functionality test. Thus, cell 1702(b) may be disabled using, for example, activation switch 1314 (not shown in FIG. 17 ), and/or switch circuit 1722(b) may be activated to bypass cell 1702(b) and maintain between logic blocks connectivity.

Thus, when multiple primary cells are connected in series, each of the multiple cells can be connected in parallel with a parallel switch. Upon detection of a fault associated with one of the plurality of cells, a parallel switch connected to the one of the plurality of cells may be activated to connect both of the plurality of cells.

In other embodiments, as shown in FIG. 17, the switch circuit 1728 may include one or more sample points that will cause one or more cyclic delays to maintain synchronization between different lines of cells. When a cell is disabled, shorting of connections between adjacent logic blocks may create synchronization errors with other computations. For example, if a task requires data from both lines A and B, and each of A and B is carried by a separate series of cells, then deactivating a cell will result in further data management between lines that will be required desynchronization. To prevent desynchronization, sample circuit 1730 may simulate the delay caused by disabled cell 1712(b). However, in some embodiments, the parallel switch may include an antifuse instead of the sampling circuit 1730 .

18 is a functional block diagram of exemplary cells connected in a two-dimensional array in accordance with disclosed embodiments. Figure 18 may represent the entire system or chip. Alternatively, Figure 18 may represent a block in a chip containing other active blocks.

Unit 1806 may represent an autonomous unit that includes multiple logical blocks, such as memory logic 1306 and/or business logic 1308 . However, in other embodiments, unit 1806 may represent a logical unit such as business logic 1308 . Where convenient, the discussion of FIG. 18 may refer to elements identified in FIG. 13 (eg, memory chip 1300 ) and discussed above.

As shown in Figure 18, cells may be arranged in a two-dimensional array, wherein cells 1806 (which may include or represent one or more of memory logic 1306, business logic 1308, or redundant business logic 1310) are connected via switch boxes 1808 and Boxes 1810 are interconnected. Furthermore, to control the configuration of the two-dimensional array, the two-dimensional array may include I/O blocks 1804 in the perimeter of the two-dimensional array.

Connection box 1810 can be a programmable and reconfigurable device that can respond to signals input from I/O block 1804 . For example, a junction box may include multiple input pins from unit 1806 and may also be connected to switch box 1808. Alternatively, connection box 1810 may include a group of switches that connect pins of programmable logic cells to routing tracks, and switch box 1808 may include a group of switches that connect different tracks.

In certain embodiments, connection box 1810 and switch box 1808 may be implemented by configuration switches such as switches 1312 and 1314 . In these embodiments, junction box 1810 and switch box 1808 may be configured by a production tester or a BIST executed at chip startup.

In some embodiments, the junction box 1810 and the switch box 1808 may be configured after the circuit functionality of the unit 1806 is tested. In these embodiments, I/O block 1804 may be used to send test signals to unit 1806 . Depending on the test results, I/O block 1804 may send programming signals that configure junction box 1810 and switch box 1808 in a manner that disables cells 1806 that fail the test protocol and enables cells 1806 that pass the test protocol.

In these embodiments, the plurality of primary logic blocks and the plurality of redundant blocks may be disposed on the substrate in a two-dimensional grid. Thus, each of the plurality of primary cells 1806 and each of the plurality of redundant blocks, such as redundant business logic 1310, may be interconnected with switch boxes 1808, and the input blocks may be placed in a two-dimensional grid of in the perimeter of each line and each column.

19 is a functional block diagram of exemplary cells in complex connections consistent with disclosed embodiments. Figure 19 may represent the entire system. Alternatively, Figure 19 may represent a block in a chip containing other active blocks.

The complex connection of Figure 19 includes cells 1902(a)-1902(f) and configuration switches 1904(a)-1904(f). Unit 1902 may represent an autonomous unit that includes multiple logical blocks such as memory logic 1306 and/or business logic 1308 . However, in other embodiments, unit 1902 may represent a logical unit such as memory logic 1306 , business logic 1308 , or redundant business logic 1310 . Configuring switches 1904 may include deactivating any of switch 1312 and enabling switch 1314 .

As shown in Figure 19, the complex connection may include cells 1902 in two planes. For example, a complex connection may include two separate substrates separated in the z-axis. Alternatively or additionally, the cells 1902 may be arranged in both surfaces of the substrate. For example, for the purpose of reducing the area of the memory chip 1300, the substrate 1301 may be arranged in two overlapping surfaces and connected with configuration switches 1904 arranged in three dimensions. Configuration switches may include deactivating switch 1312 and/or enabling switch 1314 .

The first plane of the substrate may include the "main" cell 1902. These blocks can be enabled by default. In these embodiments, the second plane may include "redundant" cells 1902 . These units can be disabled by default.

In some embodiments, the configuration switch 1904 may include an antifuse. Thus, after testing cell 1902, a block can be connected in a block of active cells by switching some antifuses to "always on" and disabling selected cells 1902, even if the cell is in a different plane in this way. In the embodiment presented in FIG. 19, one of the "main" cells (cell 1902(e)) is inactive. Figure 19 may represent inactive or untested blocks as shaded blocks, while tested or active blocks may be unshaded. Accordingly, configuration switch 1904 is configured to cause one of the logical blocks in a different plane (eg, cell 1902(f)) to become active. In this way, even if one of the main logic blocks is defective, the memory chip still functions by replacing the spare logic cells.

Figure 19 additionally shows that one of the cells 1902 in the second plane (ie, 1902(c)) is not tested or enabled because the main logic block is functional. For example, in FIG. 19, two main units 1902(a) and 1902(d) pass the functional test. Therefore, unit 1902(c) is not tested or enabled. Thus, Figure 19 shows the ability to specifically select logic blocks that become active depending on the test results.

In some embodiments, as shown in Figure 19, not all cells 1902 in the first plane may have corresponding spare or redundant blocks. However, in other embodiments, all units may be redundant with each other for full redundancy, wherein all units are primary or redundant. Furthermore, while some implementations may follow the star network topology depicted in Figure 19, other implementations may use parallel connections, series connections, and/or coupling different elements in parallel or in series with configuration switches.

20 is an exemplary flow diagram illustrating a redundant block enablement process 2000 consistent with the disclosed embodiments. Enable handler 2000 may be implemented for memory chips 1300 and particularly for DRAM memory chips. In some embodiments, process 2000 may include the steps of: testing at least one circuit functionality of each of a plurality of logic blocks on a substrate of a memory chip; identifying faults in the plurality of primary logic blocks based on the test results logic blocks; testing at least one circuit functionality of at least one redundant or additional logic block on a substrate of a memory chip; deactivating at least one failed logic block by applying an external signal to a de-enable switch; and by applying the An external signal is applied to an enable switch to enable the at least one redundant block, the enable switch is connected to the at least one redundant block and is disposed on the substrate of the memory chip. The following description of FIG. 20 further details each step of the process 2000.

Process 2000 may include testing multiple logical blocks such as business block 1308 (step 2002) and multiple redundant blocks (eg, redundant business block 1310). Testing may be performed prior to packaging using, for example, a probing station for on-wafer testing. However, step 2000 may also be performed after packaging.

Testing in step 2002 may include applying a limited sequence of test signals to each logic block in memory chip 1300 or a subset of logic blocks in memory chip 1300 . The test signal may include requests for computations that are expected to result in 0 or 1. In other embodiments, the test signal may request to read a specific address in a memory bank or write to a specific memory bank.

Testing techniques may be implemented in step 2002 to test the response of the logic block under iterative processing procedures. For example, the testing may involve testing a logical block by transmitting an instruction to write data into a memory bank and then verifying the integrity of the written data. In some embodiments, the test may include utilizing an inverse data repetition algorithm.

In an alternate embodiment, the testing of step 2002 may include running a model of the logic block to generate a target memory image based on a set of test instructions. Then, the same sequence of instructions can be executed on the logic blocks in the memory chip, and the results can be recorded. The simulated residual memory image can also be compared to the image obtained from the self-test, and any mismatches can be flagged as faults.

Alternatively, in step 2002, the test may include shadow modeling in which a diagnosis may result, but not necessarily predict an outcome. Instead, testing using shadow modeling can be performed on both the memory chip and the simulation in parallel. For example, when a logic block in a memory chip completes an instruction or task, the emulation can be signaled to execute the same instruction. Once the logic block in the memory chip completes the instruction, the architectural states of the two models can be compared. If there is a mismatch, a fault is indicated.

In some embodiments, all logic blocks (including, for example, each of memory logic 1306 , business logic 1308 , or redundant business logic 1310 ) may be tested in step 2002 . However, in other embodiments, only a subset of logic blocks may be tested in different test rounds. For example, in a first test pass, only memory logic 1306 and associated blocks may be tested. In the second round, only the business logic 1308 and associated blocks may be tested. In the third round, depending on the results of the previous two rounds, logic blocks associated with redundant business logic 1310 may be tested.

Process 2000 may continue to step 2004 . In step 2004, a failed logical block can be identified, and a failed redundant block can also be identified. For example, logic blocks that fail the tests of step 2002 may be identified in step 2004 as failed blocks. However, in other embodiments, only certain failed logical blocks may initially be identified. For example, in some embodiments, only logical blocks associated with business logic 1308 may be identified, and a failed redundant block may only be identified if a failed redundant block is required to replace a failed logical block. Additionally, identifying the failed block may include writing identification information of the identified failed block on a memory bank or non-volatile memory.

In step 2006, the failed logical block may be disabled. For example, using a configuration circuit, a failed logic block can be disabled by disconnecting it from the clock, ground, and/or power nodes. Alternatively, a failed logical block may be deactivated by configuring the junction box in an arrangement that avoids the logical block. Additionally, in other embodiments, the failed logical block may be disabled by receiving an invalid address from the address manager 1302.

In step 2008, redundant blocks that replicate the failed logical block can be identified. In order to support the same capabilities of the memory chip even if some logical blocks have failed, in step 2008, redundant blocks of available and replicable failed logical blocks can be identified. For example, if the logical block performing the multiplication of the vector is determined to be faulty, in step 2008, the address manager 1302 or the on-chip controller may identify an available redundant logical block that also performs the multiplication of the vector.

In step 2010, the redundant blocks identified in step 2008 may be enabled. In contrast to the deactivation operation of step 2006, in step 2010, the identified redundant block may be enabled by connecting it to a clock, ground, and/or power node. Alternatively, the identified redundant blocks may be enabled by configuring the connection box in an arrangement that connects the identified redundant blocks. Additionally, in other embodiments, the identified redundant blocks may be enabled by receiving a run address at test program run time.

21 is an exemplary flow diagram illustrating an address assignment handler 2100 consistent with the disclosed embodiments. Address assignment handler 2100 may be implemented for memory chips 1300 and particularly for DRAM memory chips. As described with respect to FIG. 16, in some embodiments, logic blocks in memory chip 1300 may be connected to a data bus and have address identifications. Process 2100 describes an address assignment method that disables failed logical blocks and enables logical blocks that pass the test. The steps described in handler 2100 will be described as being performed by a production tester or a BIST executed at chip startup; however, other components of memory chip 1300 and/or external devices may also execute one or more of handler 2100 step.

In step 2102, the tester may disable all logical blocks and redundant blocks by assigning an invalid identification to each logical block at the chip level.

In step 2104, the tester may execute a test protocol for the logic block. For example, the tester may perform the test method described in step 2002 for one or more of the logic blocks in the memory chip 1300 .

In step 2106, depending on the results of the test in step 2104, the tester may determine whether the logic block is defective. If the logical block is free of defects (step 2106: NO), the address manager may, in step 2108, assign a run ID to the tested logical block. If the logical block is defective (step 2106: Yes), the address manager 1302 may reserve an invalid ID for the defective logical block in step 2110.

In step 2112, the address manager 1302 may select a redundant logical block that replicates the defective logical block. In some embodiments, a redundant logical block replicating a defective logical block may have the same components and connections as the defective logical block. However, in other embodiments, redundant logical blocks may have different components and/or connections than defective logical blocks, yet capable of performing equivalent operations. For example, if the defective logical block is designed to perform multiplication of vectors, the selected redundant logical block will be able to perform the multiplication of vectors even if the selected redundant logical block does not have the same architecture as the defective cell .

In step 2114, the address manager 1302 may test for redundant blocks. For example, the tester may apply the testing techniques applied in step 2104 to the identified redundant blocks.

In step 2116, based on the results of the test in step 2114, the tester may determine whether the redundant block is defective. In step 2118, if the redundant block is free of defects (step 2116: NO), the tester may assign a run ID to the identified redundant block. In some embodiments, process 2100 may return to step 2104 after step 2118 to generate an iterative loop that tests all logic blocks in the memory chip.

If the tester determines that the redundant block is defective (step 2116: Yes), then in step 2120, the tester may determine whether additional redundant blocks are available. For example, the tester may query the memory bank for information about available redundant logical blocks. If the redundant logical block is available (step 2120: Yes), the tester may return to step 2112 and identify a new redundant logical block that reproduces the defective logical block. If the redundant logic block is not available (step 2120: NO), then in step 2122, the tester may generate an error signal. The error signal may include information of defective logical blocks and defective redundant blocks.

coupled memory bank

Embodiments disclosed in this disclosure also include distributed high performance processors. The processor may include a memory controller that interfaces the memory banks and the processing unit. The processor may be configurable to expedite the delivery of data to the processing unit for computation. For example, if the processing unit requires two data instances to perform a task, the memory controller may be configured such that the communication lines independently provide access to information from the two data instances. The disclosed memory architecture attempts to minimize the hardware requirements associated with complex cache and complex register file schemes. Typically, a processor chip includes a cache hierarchy that allows the core to work directly with registers. However, cache operations require considerable die area and consume additional power. The disclosed memory architecture avoids the use of cache hierarchies by adding logical components in memory.

The disclosed architecture also enables strategic (or even optimal) placement of data in memory banks. Even if the memory bank has a single port and high latency, the disclosed memory architecture can achieve high performance and avoid memory access bottlenecks by strategically locating data in different blocks of the memory bank. With the goal of providing a continuous stream of data to the processing unit, the compile optimization step can determine how the data should be stored in the memory bank for a specific or general task. Next, the memory controller that interfaces the processing unit and the memory bank can be configured to grant access to a particular processing unit when the particular processing unit needs data to perform an operation.

Configuration of the memory chips may be performed by a processing unit (eg, a configuration manager) or an external interface. This configuration can also be written by a compiler or other SW tool. Furthermore, the configuration of the memory controller may be based on the available ports in the memory bank and the organization of data in the memory bank. Thus, the disclosed architecture can provide a constant stream of data or simultaneous information from different memory blocks to the processing unit. In this way, in-memory computing tasks can be processed quickly by avoiding latency bottlenecks or cache requirements.

Furthermore, the data stored in the memory chips can be arranged based on the compilation optimization steps. Compilation may allow building handlers in which processors efficiently assign tasks to processing units without the delays associated with memory delays. The compilation may be performed by a compiler and transmitted to a host computer connected to an external interface in the baseboard. Often, high latency and/or a small number of ports for certain access patterns will result in a data bottleneck for the processing units that require the data. However, the disclosed compilation may locate data in a memory bank in a manner that enables a processing unit to continuously receive data even with unfavorable memory types.

Additionally, in some embodiments, the configuration manager may signal the required processing units based on the computations required by the task. Different processing units or logic blocks in a chip may have specialized hardware or architectures for different tasks. Thus, depending on the task to be performed, a processing unit or group of processing units may be selected to perform the task. The memory controller on the substrate may be configurable to route data or authorize access according to the selection of processing subunits to improve data transfer speed. For example, based on compilation optimizations and memory architecture, when a processing unit is required to perform a task, access to a memory bank may be granted to the processing unit.

Additionally, the chip architecture may include on-chip components that facilitate the transfer of data by reducing the time required to access data in a memory bank. Accordingly, the present disclosure describes a chip architecture along with compilation optimization steps for a high performance processor capable of performing specific or general tasks using a simple memory instance. Memory instances may have high random access latencies and/or a small number of ports, such as those used in DRAM devices or other memory-oriented technologies, but the disclosed architecture may be implemented by enabling sequential (or almost continuous) data flow to overcome these shortcomings.

In this application, simultaneous communication may refer to communication within one clock cycle. Alternatively, simultaneous communication may refer to sending information within a predetermined amount of time. For example, simultaneous communication may refer to communication within a few nanoseconds.

22 provides a functional block diagram of an exemplary processing device consistent with the disclosed embodiments. 22A shows a first embodiment of a processing apparatus 2200 in which a memory controller 2210 connects a first memory bank 2202 and a second memory bank 2204 using a multiplexer. The memory controller 2210 may also be connected to at least a configuration manager 2212, a logic block 2214, and a plurality of accelerators 2216(a)-2216(n). 22B shows a second embodiment of processing apparatus 2200 in which memory controller 2210 connects memory blocks 2202 and 2204 using a bus that connects memory controller 2210 with at least one configuration manager 2212, a logical block 2214, and a plurality of Accelerators 2216(a) through 2216(n). Furthermore, the host 2230 may be external to the processing device 2200 and connected to the processing device via, for example, an external interface.

Memory blocks 2202 and 2204 may include DRAM pads or groups of pads, DRAM banks, MRAM\PRAM\RERAM\SRAM cells, flash memory pads, or other memory technologies. Memory blocks 2202 and 2204 may alternatively include non-volatile memory, flash memory devices, resistive random access memory (ReRAM) devices, or magnetoresistive random access memory (MRAM) devices.

Memory blocks 2202 and 2204 may additionally include a plurality of memory cells arranged in rows and columns between a plurality of word lines (not shown) and a plurality of bit lines (not shown). The gates of each row of memory cells may be connected to respective ones of a plurality of word lines. Each column of memory cells may be connected to a respective one of a plurality of bit lines.

In other embodiments, memory regions, including memory blocks 2202 and 2204, are constructed from simple memory instances. In this application, the term "memory instance" is used interchangeably with the term "memory block." A memory instance (or block) can have undesirable characteristics. For example, the memory may be only single port memory and may have high random access latency. Alternatively or additionally, memory may be inaccessible during column and line changes and face data access issues related to, for example, capacitor charging and/or circuitry setup. However, the architecture presented in Figure 22 still facilitates parallel processing in memory devices by allowing dedicated connections between memory instances and processing units and arranging data in a manner that takes into account the characteristics of the blocks.

In some device architectures, a memory instance may include several ports to facilitate parallel operation. However, in these embodiments, the chip can still achieve improved performance when the data is compiled and organized based on the chip architecture. For example, compilers can improve the efficiency of accesses in memory regions by providing instructions and organizing data placement so that memory regions can be easily accessed even when single-ported memory is used.

Furthermore, memory blocks 2202 and 2204 may be multiple types of memory in a single chip. For example, memory blocks 2202 and 2204 may be eFlash and eDRAM. Additionally, a memory block may include DRAM with an instance of ROM.

Memory controller 2210 may include logic to handle memory accesses and pass results back to the rest of the module. For example, memory controller 2210 may include an address manager and selection devices, such as multiplexers, to route data between memory blocks and processing units or to authorize access to memory blocks. Alternatively, memory controller 2210 may include a double data rate (DDR) memory controller to drive DDR SDRAM, where data is transferred on the rising and falling edges of the system's memory clock.

Also, the memory controller 2210 may constitute a dual-channel memory controller. The incorporation of dual channel memory may facilitate control of parallel access lines by the memory controller 2210. The parallel access lines can be configured to have the same length to facilitate data synchronization when multiple lines are used in combination. Alternatively or additionally, the parallel access lines may allow access to multiple memory ports of the memory bank.

In some embodiments, the processing device 2200 may include one or more multiplexers connectable to the processing unit. The processing unit may include a configuration manager 2212, a logic block 2214, and an accelerator 2216, which may be directed to the multiplexer. Additionally, the memory controller 2210 can include at least one data input from the plurality of memory banks or blocks 2202 and 2204, and at least one data output connected to each of the plurality of processing units. With this configuration, the memory controller 2210 can simultaneously receive data from the memory banks or memory blocks 2202 and 2204 via the two data inputs and simultaneously transmit the received data to at least one selected processing unit via the two data outputs . However, in some embodiments, at least one data input and at least one data output may be implemented in a single port to allow read or write only operations. In these embodiments, a single port may be implemented as a data bus including data lines, address lines, and command lines.

A memory controller 2210 may be connected to each of the plurality of memory banks 2202 and 2204, and may also be connected to a processing unit via, for example, a selection switch. The processing units on the substrate, including the configuration manager 2212, the logic blocks 2214, and the accelerators 2216, may also be independently connected to the memory controller 2210. In some embodiments, configuration manager 2212 may receive indications of tasks to perform and, in response, configure memory controller 2210, accelerator 2216, and/or logic block 2214 according to a configuration stored in memory or supplied externally . Alternatively, the memory controller 2210 may be configured by an external interface. The task may require at least one computation available to select at least one selected processing unit from the plurality of processing units. Alternatively or additionally, the selection may be based, at least in part, on the ability of the selected processing unit to perform at least one computation. In response, the memory controller 2210 may grant access to the memory banks or route data between the at least one selected processing unit and the at least two memory banks using a dedicated bus and/or with pipelined memory accesses.

In some embodiments, a first memory bank 2202 of the at least two memory banks can be arranged on a first side of the plurality of processing units; and a second memory bank 2204 of the at least two memory banks can be arranged on all on a second side of the plurality of processing units opposite the first side. Additionally, selected processing units (eg, accelerators 2216(n)) to perform tasks may be configured to access the second memory during clock cycles when the communication lines to the first memory bank or first memory bank 2202 are open Group 2204. Alternatively, the selected processing unit may be configured to transfer data to the second memory bank 2204 during a clock cycle in which the communication line to the first memory bank 2202 is open.

In some embodiments, the memory controller 2210 may be implemented as a separate element, as shown in FIG. 22 . However, in other embodiments, the memory controller 2210 may be embedded in a memory region or may be positioned along the accelerators 2216(a)-2216(n).

The processing area in processing device 2200 may include configuration manager 2212, logic block 2214, and accelerators 2216(a)-2216(n). Accelerator 2216 may include a number of processing circuits with predefined functions and may be defined by a particular application. For example, the accelerator may be a vector multiply-accumulate (MAC) unit or a direct memory access (DMA) unit that handles memory movement between modules. The accelerator 2216 may also be able to calculate its own address and request data from or write data to the memory controller 2210. For example, configuration manager 2212 can signal at least one of accelerators 2216 the accelerator-accessible memory bank. Next, the accelerator 2216 may configure the memory controller 2210 to deliver data or grant access to the accelerator itself. In addition, accelerator 2216 may include at least one arithmetic logic unit, at least one vector processing logic unit, at least one string comparison logic unit, at least one register, and at least one direct memory access element.

Configuration manager 2212 may include digital processing circuitry to configure accelerators 2216 and to direct execution of tasks. For example, the configuration manager 2212 may be connected to the memory controller 2210 and each of the plurality of accelerators 2216. The configuration manager 2212 may have its own dedicated memory to hold the configuration of the accelerator 2216. Configuration manager 2212 may use memory banks to obtain commands and configurations via memory controller 2210. Alternatively, configuration manager 2212 may be programmed via an external interface. In some embodiments, configuration manager 2212 may be implemented with an on-chip reduced instruction set computer (RISC) or an on-chip complex CPU with its own cache hierarchy. In some embodiments, configuration manager 2212 may also be omitted, and the accelerator may be configured via an external interface.

Processing device 2200 may also include an external interface (not shown). The external interface allows access to the memory from the upper level (the memory bank controller, which receives commands from the external host 2230 or the on-chip main processor), or from the external host 2230 or the on-chip main processor. This external interface may allow programming of configuration manager 2212 and accelerator 2216 by writing configuration or code to memory via memory controller 2210 for later use by configuration manager 2212 or units 2214 and 2216 themselves. However, the external interface can also program the processing unit directly without routing through the memory controller 2210. Where configuration manager 2212 is a microcontroller, configuration manager 2212 may allow code to be loaded from main memory to controller area memory via an external interface. The memory controller 2210 may be configured to interrupt a task in response to receiving a request from an external interface.

The external interface may include a plurality of connectors associated with the logic circuit that provide a glue-free interface to various elements on the processing device. The external interface may include: a data I/O input terminal for data reading and an output terminal for data writing; an external address output terminal; an external CE0 chip selection pin; a low-active chip selector; a byte enable connection pin; wait state pin for memory cycling; write enable pin; output enable valid pin; and read write enable pin. Therefore, the external interface has the necessary inputs and outputs to control the processing program and obtain information from the processing device. For example, the external interface may conform to the JEDEC DDR standard. Alternatively or additionally, the external interface may conform to other standards, such as SPI\OSPI or UART.

In some embodiments, the external interface can be disposed on the chip substrate and can be connected to the external host 2230 . An external host can access the memory blocks 2202 and 2204, the memory controller 2210, and the processing unit via the external interface. Alternatively or additionally, external host 2230 may read and write memory, or may signal configuration manager 2212 via read and write commands to perform operations, such as starting a handler and/or stopping a handler. Additionally, the external host 2230 may configure the accelerator 2216 directly. In some embodiments, the external host 2230 can directly perform read/write operations on the memory banks 2202 and 2204.

In some embodiments, configuration manager 2212 and accelerator 2216 may be configured to use a direct bus to connect device regions and memory regions depending on the target task. For example, a subset of accelerators 2216 may be connected to memory instance 2204 when the subset of accelerators is capable of performing computations required for task execution. By doing this separation, it is possible to ensure that dedicated accelerators get the bandwidth (BW) required by memory blocks 2202 and 2204. In addition, this configuration with a dedicated bus can allow large memory to be split into smaller instances or blocks, since connecting a memory instance to the memory controller 2210 allows fast memory even with high row latency Fetch data from different memories. To achieve parallelization of connections, memory controller 2210 may connect to each of the memory instances with a data bus, address bus, and/or control bus.

The above-described inclusion of the memory controller 2210 may eliminate the requirement for a cache hierarchy or complex register file in the processing device. Although cache hierarchies may be added for added capabilities, the architecture in processing device processing device 2200 may allow designers to add enough memory blocks or instances based on processing operations and manage the instances accordingly without a cache hierarchy . For example, the architecture in processing device processing device 2200 may eliminate the need for a cache hierarchy by implementing pipelined memory accesses. In pipelined memory access, a processing unit may receive a continuous stream of data in each cycle where certain data lines may be opened (or enabled) while other data lines receive or transmit data. Continuous data flow using separate communication lines may allow for improved execution speed and minimal latency due to line changes.

Furthermore, the disclosed architecture in Figure 22 enables pipelined memory access, potentially organizing data in a small number of memory blocks and saving power consumption caused by line switching. For example, in some embodiments, the compiler may communicate to the host 2230 the organization of data in memory banks or a method to organize data in memory banks to facilitate accessing data during a given task. Next, the configuration manager 2212 can define which memory banks and in some cases which ports of the memory banks are accessible by the accelerator. This synchronization between the location of data in the memory bank and the method of data access improves computational tasks by feeding the data to the accelerator with minimal latency. For example, in embodiments where configuration manager 2212 includes a RISC\CPU, the method may be implemented with offline software (SW), and configuration manager 2212 may then be programmed to perform the method. The method can be developed in any language executable by a RISC/CPU computer and can be executed on any platform. Inputs to the method may include the configuration of the memory behind the memory controller and the data itself, as well as the pattern of memory accesses. Furthermore, the method may be implemented in an embodiment-specific language or machine language, and may also simply be a series of configuration values in binary or literal representation.

As discussed above, in some embodiments, the compiler may provide instructions to host 2230 for organizing data in memory banks 2202 and 2204 in preparation for pipelined memory access. The pipelined memory access may generally include the steps of: receiving a plurality of addresses for a plurality of memory banks or memory banks 2202 and 2204; accessing the plurality of memory banks using independent data lines according to the received addresses; via a first A communication line supplies data from a first address to at least one of the plurality of processing units and is open to a second communication line at a second address in a first memory bank of the plurality of memory banks, the The second address is in a second memory bank 2204 of the plurality of memory banks; and within a second clock cycle, data from the second address is supplied into the plurality of processing units via the second communication line The at least one of the first lines is open to the third communication line of the third address in the first memory bank. In some embodiments, the pipelined memory access may be performed with two memory banks connected to a single port. In these embodiments, the memory controller 2210 may hide the two memory blocks behind a single port, but utilize a pipelined memory access method to transfer the data to the processing unit.

In some embodiments, the compiler may execute on the host 2230, after which the task is performed. In these embodiments, the compiler may be able to determine the configuration of the data flow based on the architecture of the memory device, since this configuration will be known to the compiler.

In other embodiments, if the configuration of memory blocks 2204 and 2202 is unknown at offline time, the pipelined method may be performed on host 2230, which may arrange data in the memory blocks before starting computation. For example, host 2230 may write data directly into memory blocks 2204 and 2202. In these embodiments, processing units such as configuration manager 2212 and memory controller 2210 may not have information about the required hardware until runtime. Next, it may be necessary to delay the selection of accelerator 2216 until the task begins to run. In these cases, the processing unit or memory controller 2210 can randomly select the accelerator 2216 and generate a test data access pattern that can be modified as the task is performed.

However, when the tasks are known in advance, the compiler may organize data and instructions in memory banks for the host 2230 to provide to processing units such as the configuration manager 2212 to set signal connections that minimize access latency . For example, in some cases the accelerator 2216 may require n words simultaneously. However, each memory instance supports fetching only m words at a time, where "m" and "n" are integers and m<n. Thus, the compiler can place the required data across different memory instances or blocks to facilitate data access. Additionally, to avoid line error leakage delays, where processing device 2200 includes multiple memory memories, the host may split data in different lines of different memory instances. The partitioning of data may allow access to the next line of data in the next instance, while still using data from the current instance.

For example, accelerator 2216(a) may be configured to multiply two vectors. Each of the vectors may be stored in separate memory blocks, such as memory blocks 2202 and 2204, and each vector may include multiple words. Therefore, it may be necessary to access two memory banks and retrieve multiple words in order to complete a task that requires the accelerator 2216(a) to do the multiplication. However, in some embodiments, a memory bank only allows access to one word per clock cycle. For example, a memory block may have a single port. In these cases, in order to speed up data transfer during operation, the compiler may organize the words that make up the vector in different memory blocks to allow parallel and/or simultaneous reads of the words. In these cases, the compiler may store words in memory banks with dedicated lines. For example, if each vector includes two words and the memory controller has direct access to four memory banks, the compiler can arrange the data in four memory banks, transferring one word per memory bank and speeding up the data deliver. Furthermore, in an embodiment, when the memory controller 2210 may have more than a single connection to each memory bank, the compiler may instruct the configuration manager 2212 (or other processing unit) to access port-specific ports. In this manner, processing device 2200 may perform pipelined memory accesses to continuously provide data to processing units by simultaneously loading words in some lines and transferring data in other lines. Therefore, this pipelined memory access avoidance avoids latency issues.

23 is a functional block diagram of an exemplary processing device 2300 consistent with the disclosed embodiments. This functional block diagram shows a simplified processing device 2300 showing a single accelerator in the form of a MAC unit 2302, configuration manager 2304 (equivalent or similar to configuration manager 2212), memory controller 2306 (equivalent or similar to memory control 2210) and a plurality of memory blocks 2308(a)-2308(d).

In some embodiments, MAC unit 2302 may be a specific accelerator for processing a specific task. As an example, the processing device 2300 may be tasked with 2D convolution. Next, configuration manager 2304 can signal accelerators with appropriate hardware to perform computations associated with the tasks. For example, MAC unit 2302 may have four internal increment counters (logical adders and registers to manage the four loops required for convolution calculations) and a multiply-accumulate unit. Configuration manager 2304 can signal MAC unit 2302 to process incoming data and perform tasks. Configuration manager 2304 may transmit instructions to MAC unit 2302 to perform the task. In these cases, the MAC unit 2302 may iterate over the calculated address, multiply the numbers, and accumulate them to internal registers.

In some embodiments, configuration manager 2304 may configure accelerators while memory controller 2306 authorizes access to block 2308 and MAC unit 2302 using a dedicated bus. However, in other embodiments, the memory controller 2306 may configure the accelerator directly based on instructions received from the configuration manager 2304 or an external interface. Alternatively or additionally, the configuration manager 2304 may preload several configurations and allow the accelerator to run iteratively at different addresses with different sizes. In these embodiments, configuration manager 2304 may include a cache that stores commands that are then transmitted to at least one of a plurality of processing units, such as accelerators 2216 . However, in other embodiments, configuration manager 2304 may not include a cache.

In some embodiments, the configuration manager 2304 or the memory controller 2306 may receive an address that needs to be accessed for a task. The configuration manager 2304 or the memory controller 2306 may check the registers to determine whether the address is already in the loaded line to one of the memory banks 2308. If in the loaded line, the memory controller 2306 may read the word from the memory block 2308 and pass the word to the MAC unit 2302. If the address is not in the loaded line, the configuration manager 2304 may request that the memory controller 2306 may load the line and signal the MAC unit 2302 to delay until the loaded line is retrieved.

In some embodiments, as shown in FIG. 23, the memory controller 2306 may include two inputs that form two independent addresses. However, if more than two addresses should be accessed at the same time, and these addresses are in a single memory bank (eg, addresses are only in memory bank 2308(a)), then memory controller 2306 or configuration manager 2304 may cause exceptions. Alternatively, configuration manager 2304 may return an invalid data signal when both addresses are only accessible via a single line. In other embodiments, the unit may delay handler execution until it is possible to retrieve all required data. This can reduce overall performance. However, the compiler may be able to find configuration and data placement that will prevent the delay.

In some embodiments, the compiler may generate a configuration or set of instructions for processing device 2300 that may configure configuration manager 2304 and memory controller 2306 and accelerator 2302 to handle the need for access from a single memory block Multiple addresses but the memory block has one port. For example, the compiler may rearrange the data in the memory block 2308 so that multiple lines in the memory block 2308 can be accessed by the processing unit.

In addition, the memory controller 2306 may also operate on more than one input at the same time. For example, the memory controller 2306 may allow access to one of the memory blocks 2308 via one port and supply data when a request for a different memory block is received in the other input. Thus, this operation may result in the accelerator 2216 tasked with the exemplary 2D convolution receiving data from the dedicated communication line of the associated memory bank.

Additionally or alternatively, the memory controller 2306 or logic block may maintain a refresh counter for each memory block 2308 and handle the refresh of all lines. Having this counter allows the memory controller 2306 to insert refresh cycles between the device's dead access times.

Additionally, the memory controller 2306 may be configurable to perform pipelined memory access to receive addresses and open lines in memory blocks before supplying data. This pipelined memory access can provide data to processing units without interrupting or delaying clock cycles. For example, although the memory controller 2306 or one of the logic blocks is accessing data using the right line in Figure 23, the memory controller or logic block may be transferring data in the left line. These methods will be explained in more detail with respect to FIG. 26 .

In response to the required data, the processing device 2300 may use a multiplexer and/or other switching device to select which devices to serve to perform a given task. For example, configuration manager 2304 may configure the multiplexer so that at least two data lines reach MAC unit 2302. In this way, tasks that require data from multiple addresses, such as 2D convolution, can be performed faster because vectors or words that require multiplication during convolution can arrive at the processing unit simultaneously in a single clock. This method of data transfer may allow a processing unit such as accelerator 2216 to output results quickly.

In some embodiments, configuration manager 2304 may be configurable to execute handlers based on the priority of tasks. For example, the configuration manager 2304 may be configured to cause the running handler to complete without any interruption. In this case, configuration manager 2304 may provide instructions or configurations for the task to accelerator 2216, causing the accelerator to run uninterrupted and switching the multitasker only when the task is complete. However, in other embodiments, configuration manager 2304 may interrupt tasks and reconfigure data routing when it receives priority tasks, such as requests from external interfaces. However, where memory banks 2308 are sufficient, memory controller 2306 may be configurable to route data to or grant access to processing units using dedicated lines that do not have to be changed until the task is complete. Furthermore, in some embodiments, all devices may be connected to the entity of configuration manager 2304 through a bus, and the devices may manage access between the devices themselves and the bus (eg, using the same logic as a multiplexer). Thus, the memory controller 2306 may be directly connected to several memory instances or memory blocks.

Alternatively, the memory controller 2306 may connect directly to the memory sub-instance. In some embodiments, each memory instance or block may be constructed from a sub-instance (eg, a DRAM may be constructed of pads with independent data lines arranged in multiple sub-blocks). Additionally, examples may include at least one of a DRAM pad, DRAM, bank, flash memory pad, or SRAM pad, or any other type of memory. Next, memory controller 2306 may include dedicated lines to directly address sub-instances to minimize latency during pipelined memory accesses.

In some embodiments, memory controller 2306 may also maintain logic required for a particular memory instance (such as row\column decoders, refresh logic, etc.), and memory block 2308 may handle its own logic. Thus, memory block 2308 can obtain an address and generate a command to return\write data.

24 depicts an exemplary memory configuration diagram consistent with disclosed embodiments. In some embodiments, a compiler generating code or configuration for processing device 2200 may execute a method to configure loading from memory blocks 2202 and 2204 by pre-arranging data in each block. For example, a compiler may pre-arrange data such that each word required by a task is associated with a line of memory instance or memory bank. But for tasks that require more memory blocks than are available in processing device 2200, the compiler may implement methods that fit data into more than one memory location per memory block. The compiler can also store the data sequentially and evaluate the latency of each memory block to avoid line-fault-miss latency. In some embodiments, the host may be part of the processing unit, such as configuration manager 2212, but in other embodiments, the compiler host may be connected to processing device 2200 via an external interface. In these embodiments, the host may execute a compilation function, such as that described for the compiler.

In some embodiments, the configuration manager 2212 may be a CPU or a microcontroller (uC). In these embodiments, configuration manager 2212 may have to access memory to obtain commands or instructions placed in memory. Certain compilers can generate code and place that code in memory in a way that allows sequential commands to be stored in the same memory line and across several memory banks to allow pipelined memory access to also fetched commands . In these embodiments, configuration manager 2212 and memory controller 2210 may be able to avoid row latency in linear execution by facilitating pipelined memory accesses.

The previous state of linear execution of a program describes a method for a compiler to identify and place instructions to allow pipelined memory execution. However, other software structures may be more complex and would require the compiler to recognize other software structures and act accordingly. For example, in situations where a task requires looping and branching, the compiler can place all looping code in a single line so that a single line can loop without line-open delays. Next, memory controller 2210 may not need to change lines during execution.

In some embodiments, configuration manager 2212 may include an internal cache or small memory. The internal cache may store commands executed by the configuration manager 2212 to handle branches and loops. For example, commands in the internal cache may include instructions to configure accelerators for accessing memory blocks.

25 is an exemplary flow diagram illustrating a possible memory configuration handler 2500 consistent with the disclosed embodiments. In the context of facilitating the description of the memory configuration handler 2500, reference may be made to the identifiers of the elements depicted in FIG. 22 and described above. In some embodiments, handler 2500 may be executed by a compiler that provides instructions to a host connected via an external interface. In other embodiments, handler 2500 may be executed by a component of processing device 2200, such as configuration manager 2212.

In general, the process 2500 can include: determining the number of words required simultaneously to perform a task; determining the number of words that can be simultaneously accessed from each of the plurality of memory banks; and when the number of words required simultaneously is greater than When the number of words that can be accessed at the same time, the number of words required at the same time is divided among the multiple memory banks. Furthermore, dividing the number of words required at the same time may include performing a cyclic organization of words and sequentially assigning one word per memory bank.

More specifically, processing routine 2500 may begin at step 2502, where a compiler may receive a task specification. The specification includes required calculations and/or priority levels.

In step 2504, the compiler may identify an accelerator or group of accelerators that can execute the task. Alternatively, a compiler may generate instructions so a processing unit (such as configuration manager 2212) may identify an accelerator to perform the task. For example, using desired compute configuration manager 2212 can identify accelerators in the group of accelerators 2216 that can handle the task.

In step 2506, the compiler may determine the number of words that need to be accessed simultaneously in order to perform the task. For example, the multiplication of two vectors requires access to at least two vectors, and the compiler may therefore decide that the vector words must be simultaneously accessed to perform the operation.

In step 2508, the compiler may determine the number of cycles necessary to perform the task. For example, if the task requires four convolution operations with incidental results, the compiler may determine that at least 4 loops will be necessary to perform the task.

In step 2510, the compiler may place words that require simultaneous access in different memory banks. In this way, the memory controller 2210 can be configured to open lines to different memory instances and access the required memory blocks within one clock cycle without requiring any cache data.

In step 2512, the compiler places sequentially accessed words in the same memory bank. For example, where four loops of operations are required, the compiler may generate instructions to write the desired words into a single memory bank in sequential loops to avoid changing lines between different memory banks during execution .

In step 2514, the compiler generates instructions for programming a processing unit such as configuration manager 2212. The instruction may specify conditions for operating a switching device (such as a multiplexer) or configuring a data bus. Through these instructions, the configuration manager 2212 can configure the memory controller 2210 to use dedicated communication lines to route data from the memory block to the processing unit or to authorize access to the memory block according to the task.

26 is an exemplary flow diagram illustrating a memory read handler 2600 consistent with the disclosed embodiments. In the context of facilitating the description of the memory read handler 2600, reference may be made to the identifiers of the elements depicted in FIG. 22 and described above. In some embodiments, process 2600 may be implemented by memory controller 2210, as described below. However, in other embodiments, handler 2600 may be implemented by other elements in processing device 2200, such as configuration manager 2212.

In step 2602, the memory controller 2210, configuration manager 2212, or other processing unit may receive an indication to post data from the memory bank or to authorize access to the memory bank. Requests can specify addresses and memory blocks.

In some embodiments, the request may be received via the data bus specifying the read command in line 2218 and the address specified in line 2220. In other embodiments, the request may be received via a demultiplexer connected to the memory controller 2210.

In step 2604, the configuration manager 2212, host, or other processing unit may query the internal registers. The internal registers may include information about open lines to memory banks, open addresses, open memory banks, and/or upcoming tasks. Based on the information in the internal registers, it can be determined whether there is an open line to the memory bank and/or whether the memory block received the request in step 2602 . Alternatively or additionally, the memory controller 2210 may query the internal register directly.

If the internal register indicates that the memory bank is not loaded into the open line (step 2606: NO), then process 2600 may continue to step 2616 and the line may be loaded into the memory bank associated with the received address. Additionally, memory controller 2210 or a processing unit such as configuration manager 2212 may in step 2616 signal a delay to the element requesting information from the memory address. For example, if the accelerator 2216 is requesting memory information located in an occupied memory block, in step 2618, the memory controller 2210 may send a delay signal to the accelerator. In step 2620, configuration manager 2212 or memory controller 2210 may update internal registers to indicate lines that have been opened to new memory banks or new memory blocks.

If the internal register indicates that the memory bank is loaded into an open line (step 2606 : Yes), then process 2600 may continue to step 2608 . In step 2608, it may be determined whether the line loaded with the memory bank is being used for a different address. If the line is being used for a different address (step 2608: YES), then this would indicate that there are two instances in a single block, and therefore, both instances cannot be accessed at the same time. Accordingly, an error or exemption signal may be sent in step 2616 to the element requesting information from the memory address. But if the line is not being used for a different address (step 2608: NO), then the line for that address can be opened and the data retrieved from the target memory bank, and proceed to step 2614 to transfer the data to the requesting address from the memory address element of information.

Using the processing program 2600, the processing device 2200 can establish a direct connection between a processing unit and a memory block or memory instance that contains the information needed to perform a task. This organization of data will enable reading information from organized vectors in different memory instances, as well as allowing information to be retrieved from different memory blocks simultaneously when a device requests multiple of these addresses.

FIG. 27 is an exemplary flow diagram illustrating an execution process 2700 consistent with the disclosed embodiments. Reference may be made to the identifiers of the elements depicted in FIG. 22 and described above in the context of facilitating the description of the execution process 2700.

In step 2702, a compiler or a local unit such as configuration manager 2212 may receive an indication of tasks that need to be performed. The task may include a single operation (eg, multiplication) or a more complex operation (eg, convolution between matrices). The task may also indicate required calculations.

In step 2704, the compiler or configuration manager 2212 may determine the number of words required to perform the task simultaneously. For example, the configuration compiler may determine that two words are needed at the same time to perform multiplication between vectors. In another embodiment (2D convolution task), configuration manager 2212 may determine that convolution between matrices requires "n" by "m" words, where "n" and "m" are matrix dimensions. Additionally, in step 2704, the configuration manager 2212 may also determine the number of cycles necessary to perform the task.

In step 2706, depending on the determination in step 2704, the compiler may write words that require simultaneous access into multiple memory banks disposed on the substrate. For example, when the number of words that can be accessed simultaneously from one of the multiple memory banks is less than the number of words required at the same time, the compiler may organize the data in the multiple memory banks to facilitate one clock memory Take the different desired words. Additionally, when the configuration manager 2212 or the compiler determines the number of cycles necessary to perform a task, the compiler may write the required words into a single one of the multiple memory banks in sequential loops to prevent the memory bank Toggle between lines.

In step 2708, the memory controller 2210 may be configured to read or authorize storage of at least one first word from a first memory bank of the plurality of memory banks or banks using the first memory line Pick.

In step 2170, a processing unit (eg, one of accelerators 2216) may process the task using the at least one first word.

In step 2712, the memory controller 2210 may be configured to open a second memory line in the second memory bank. For example, based on a task and using a pipelined memory access method, the memory controller 2210 may be configured to open a second memory line in the second memory bank in which the information required by the task was written in step 2706. In some embodiments, the second memory line may be opened when the task in step 2170 is about to complete. For example, if a task requires 100 clocks, the second memory line can be opened in the 90th clock.

In some embodiments, steps 2708-2712 may be performed within one line access cycle.

In step 2714, the memory controller 2210 may be configured to authorize access to data from at least one second word of the second memory bank using the second memory line opened in step 2710.

In step 2176, a processing unit (eg, one of accelerators 2216) may process the task using at least the second word.

In step 2718, the memory controller 2210 may be configured to open the second memory line in the first memory bank. For example, based on the task and using a pipelined memory access method, the memory controller 2210 may be configured to open a second memory line to the first memory bank. In some embodiments, the second memory line to the first block may be opened when the task in step 2176 is about to complete.

In some embodiments, steps 2714-2718 may be performed within one line access cycle.

In step 2720, the memory controller 2210 may use the second memory line in the first memory group or the first line in the third memory group and continue in a different memory group from the first memory line in the plurality of memory groups or blocks. A memory bank reads at least one third word or grants access to the at least one third word.

Some memory chips, such as dynamic random access memory (DRAM) chips, use refresh to avoid loss of stored data (eg, using capacitance) due to voltage decay in the chip's capacitors or other electrical components. For example, in DRAM, each cell must be refreshed from time to time (based on specific processing procedures and designs) to restore the charge in the capacitors so that data is not lost or corrupted. As the memory capacity of DRAM chips increases, the amount of time required to refresh the memory becomes significant. During the period of time a line of memory is being refreshed, the bank containing the line being refreshed cannot be accessed. This can lead to reduced performance. Additionally, the power associated with the refresh handler can also be significant. Previous efforts have been made to reduce the rate at which refresh is performed to reduce the adverse effects associated with refreshing memory, but most of these efforts have focused on the physical layer of DRAM.

Flushing is analogous to reading and writing rows back to memory. Using this principle and focusing on patterns of accessing memory, embodiments of the present disclosure include software and hardware techniques and modifications to memory chips to use less power for refresh and reduce the amount of time during which memory is refreshed. For example, as an overview, some embodiments may use hardware and/or software to track line access timing and skip most recently accessed lines within a refresh cycle (eg, based on timing thresholds). In another embodiment, some embodiments may rely on software executed by the memory chip's refresh controller to assign reads and writes such that access to the memory is non-random. Therefore, software can control refresh more precisely to avoid wasting refresh cycles and/or lines. These techniques can be used alone or in combination with a compiler that encodes commands for refreshing the controller and machine code for the processor, so that access to memory is also non-random. Using any combination of these techniques and configurations described in detail below, the disclosed embodiments may reduce memory refresh power requirements and/or improve system performance by reducing the amount of time during which memory cells are refreshed.

28 depicts an embodiment memory chip 2800 with a refresh controller 2803 consistent with the present disclosure. For example, memory chip 2800 may include multiple memory banks (eg, memory bank 2801a and the like) on a substrate. In the embodiment of Figure 28, the substrate includes four memory banks, each having four lines. A line may refer to one or more memory banks of memory chip 2800 or within any other collection of memory cells within memory chip 2800 (such as a portion of a memory bank or an entire row along a memory bank or a group of memory banks) word line.

In other embodiments, the substrate may include any number of memory banks, and each memory bank may include any number of lines. Some memory banks may include the same number of lines (as shown in Figure 28), while other memory banks may include different numbers of lines. As further depicted in FIG. 28, the memory chip 2800 may include a controller 2805 for receiving inputs to the memory chip 2800 and transmitting outputs from the memory chip 2800 (eg, as described above in "Partitioning of Codes").

In some embodiments, the plurality of memory banks may include dynamic random access memory (DRAM). However, the plurality of memory banks may include any volatile memory that stores data that needs to be refreshed periodically.

As will be discussed in more detail below, embodiments disclosed in the present disclosure may use a counter or resistor-capacitor circuit to time refresh cycles. For example, a counter or timer can be used to count the time since the last complete refresh cycle, and then when the counter reaches its target value, another counter can be used to iterate over all rows. Embodiments of the present disclosure may additionally track accesses to sectors of the memory chip 2800 and reduce the required refresh power. For example, although not depicted in Figure 28, the memory chip 2800 may also include a data store configured to store access information indicative of an access operation of one or more sectors in the plurality of memory banks . For example, the one or more segments may comprise any portion of a line, column, or any other grouping of memory cells within memory chip 2800. In a particular embodiment, the one or more sections may comprise at least one row of memory structures within the plurality of memory banks. The refresh controller 2803 can be configured to perform a refresh operation for the one or more sectors based at least in part on the stored access information.

For example, the data store may include one or more registers, static random access memory (SRAM) associated with a segment of memory chip 2800 (eg, a line, column, or any other grouping of memory cells within memory chip 2800 ). SRAM) cell, or its analogs. Additionally, the data store may be configured to store bits that indicate whether the associated segment was accessed in one or more previous cycles. A "bit" may include any data structure that stores at least one bit, such as a register, SRAM cell, non-volatile memory, or the like. Furthermore, a bit may be set by setting the corresponding switch (or switching element, such as a transistor) of the data structure to on (which may be equivalent to "1" or "true"). Additionally or alternatively, a bit may be changed by modifying any other property within the data structure (such as charging a floating gate of flash memory, modifying the state of one or more flip-flops in SRAM, or the like) in order to set a "1" Write to this data structure (or any other value indicating the setting of the bits) to set. If a bit is determined to be set as part of the memory controller's refresh operation, the refresh controller 2803 may skip the refresh cycle for the associated sector and clear the registers associated with its portion.

In another embodiment, the data store may include one or more non-volatiles associated with a segment of the memory chip 2800 (eg, a line, column, or any other grouping of memory cells within the memory chip 2800 ) Memory (eg, flash memory or the like). The non-volatile memory may be configured to store bits that indicate whether the associated segment was accessed in one or more previous cycles.

Some embodiments may additionally or alternatively add a timestamp register on each row or row group (or other section of the memory chip 2800) that holds the last time the line was accessed within the current refresh cycle. This means that on every row access, the refresh controller can update the row timestamp register. Thus, when the next refresh occurs (eg, at the end of a refresh cycle), the refresh controller can compare the stored timestamps and if the associated segment was previously within a certain period of time (eg, at the time as applied to the stored time) within a certain threshold of the stamp), the refresh controller may skip to the next segment. This avoids the system consuming refresh power on recently accessed segments. In addition, the refresh controller can continue to track accesses to ensure that each segment is accessed or refreshed on the next cycle.

Thus, in yet another embodiment, a data store may include one or more registers associated with a segment of memory chip 2800 (eg, a line, column, or any other grouping of memory cells within memory chip 2800) or non-volatile memory. Rather than using bits to indicate whether the associated segment has been accessed, the register or non-volatile memory may be configured to store a timestamp or other information indicating the most recent access to the associated segment. In this embodiment, the refresh controller 2803 may be based on the amount of time between a timestamp stored in an associated register or memory and the current time (eg, from a timer, as explained below in FIGS. 29A and 29B ) Whether a predetermined threshold (eg, 8ms, 16ms, 32ms, 64ms, or the like) is exceeded is determined to refresh or access the associated segment.

Thus, the predetermined threshold may comprise the amount of time of a refresh cycle that ensures that the associated segment is refreshed (if not accessed) at least once in each refresh cycle. Alternatively, the predetermined threshold may comprise an amount of time that is shorter than the amount of time required for the refresh cycle (eg, to ensure that any required refresh or access signals can reach the associated section before the refresh cycle is complete). For example, the predetermined time may include 7ms for a memory chip with an 8ms refresh period, such that if a segment has not been accessed within 7ms, the refresh controller will send a refresh or memory that arrives at the segment at the end of the 8ms refresh period Take the signal. In some embodiments, the predetermined threshold may depend on the size of the associated segment. For example, for smaller sections of memory chip 2800, the predetermined threshold may be smaller.

Although described above with respect to memory chips, the refresh controllers of the present disclosure may also be used in distributed processor architectures, such as those described above and in sections throughout this disclosure. One embodiment of such an architecture is depicted in Figure 7A. In these embodiments, the same substrate as memory chip 2800 may include multiple processing groups disposed thereon, eg, as depicted in Figure 7A. As explained above with respect to FIG. 3A, a "processing group" may refer to two or more processor sub-units and their corresponding memory groups on a substrate. The group may represent a spatial distribution and/or logical grouping on the substrate for the purpose of compiling code for execution on the memory chip 2800 . Thus, the substrate can include a memory array that includes multiple groups, such as group 2801a shown in FIG. 28 and others. Additionally, the substrate can include a processing array that can include a plurality of processor subunits (such as subunits 730a, 730b, 730c, 730d, 730e, 730f, 730g, and 730h shown in Figure 7A).

As explained further above with respect to FIG. 7A, each processing group may include a processor sub-unit and one or more corresponding memory banks dedicated to that processor sub-unit. Furthermore, to allow each processor subunit to communicate with its corresponding dedicated memory bank, the baseboard may include a first plurality of buses connecting one of the processor subunits to its corresponding dedicated memory bank.

In these embodiments, as shown in FIG. 7A , the substrate may include elements to connect each processor subunit to at least one other processor subunit (eg, adjacent subunits in the same row, a second plurality of buses adjacent to the processor subunit, or any other processor subunit on the substrate). The first plurality of buses and/or the second plurality of buses may not contain sequential hardware logic components such that data transfers between processor subunits and across corresponding ones of the plurality of buses are not controlled by sequential hardware logic components , as explained above in the "Using Software Synchronization" section.

In embodiments where the same substrate as memory chip 2800 may include multiple processing groups disposed thereon (eg, as depicted in FIG. 7A ), the processor subunit may also include an address generator (eg, as depicted in FIG. 7A ) address generator 450 depicted in 4). Additionally, each processing group may include a processor sub-unit and one or more corresponding memory banks dedicated to the processor sub-unit. Accordingly, each of the address generators may be associated with a corresponding dedicated memory bank of the plurality of memory banks. Additionally, the substrate may include a plurality of buses, each bus connecting one of the plurality of address generators to its corresponding dedicated memory bank.

FIG. 29A depicts an embodiment refresh controller 2900 consistent with the present disclosure. The refresh controller 2900 may be incorporated into a memory chip of the present disclosure, such as the memory chip 2800 of FIG. 28 . As depicted in FIG. 29A, the refresh controller 2900 may include a timer 2901, which may include an on-chip oscillator or any other sequential circuit for the refresh controller 2900. In the configuration depicted in Figure 29A, timer 2901 may trigger a refresh cycle periodically (eg, every 8ms, 16ms, 32ms, 64ms, or the like). A refresh cycle may use row counters 2903 to cycle through all rows of the corresponding memory chip, and adders 2901 in conjunction with valid bits 2905 to generate a refresh signal for each row. As shown in Figure 29A, bit 2905 may be fixed to 1 ("true") to ensure that each row is refreshed during a cycle.

In an embodiment of the present disclosure, the refresh controller 2900 may include data storage. As described above, the data store may include one or more registers or nonvolatile registers associated with segments of memory chip 2800 (eg, lines, columns, or any other grouping of memory cells within memory chip 2800 ) Sexual memory. The register or non-volatile memory may be configured to store a timestamp or other information indicating the most recent access to the associated segment.

The refresh controller 2900 may use the stored information to skip the refresh of sectors of the memory chip 2900 . For example, if the information indicates that a segment has been refreshed during one or more previous refresh cycles, the refresh controller 2900 may skip the segment in the current refresh cycle. In another embodiment, if the difference between the timestamp stored for a segment and the current time is below a threshold, the refresh controller 2900 may skip the segment in the current refresh cycle. Refresh controller 2900 may further continue to track access and refresh of segments of memory chip 2800 through multiple refresh cycles. For example, refresh controller 2900 may use timer 2901 to update the stored timestamp. In these embodiments, the refresh controller 2900 may be configured to use the output of the timer to clear the access information stored in the data store after a threshold time interval. For example, in embodiments where the data store stores a timestamp of the most recent access or refresh of the associated segment, refresh controller 2900 may update the new time each time an access command or refresh signal is sent to that segment. The stamps are stored in the data storage. If the data store stores bits rather than time stamps, then timer 2901 can be configured to clear bits that are set to last longer than a threshold time period. For example, in embodiments where data store storage indicates that the associated section was accessed in one or more previous cycles, refresh controller 2900 may clear the data store whenever timer 2901 triggers a new refresh cycle bit (eg, set it to 0), this new refresh cycle goes through a critical number of cycles (eg, one, two, or the like) after the associated bit is set (eg, set to 1) cycle.

The refresh controller 2900 may cooperate with other hardware of the memory chip 2800 to track access to segments of the memory chip 2800 . For example, memory chips use sense amplifiers to perform read operations (eg, as shown above in Figures 9 and 10). The sense amplifier may include a plurality of transistors configured to sense low power signals from segments of the memory chip 2800 that store data in one or more memory cells, and to convert small voltages The swing is amplified to higher voltage levels so that the data can be interpreted by logic such as an external CPU or GPU or an integrated processor sub-unit as explained above. Although not depicted in Figure 29A, refresh controller 2900 may further communicate with a sense amplifier configured to access one or more segments and change the state of at least one bit register. For example, when a sense amplifier accesses one or more segments, it may set a bit associated with that segment (eg, set to 1), which indicates that the associated segment has been access. In embodiments where the data store stores a timestamp of the most recent access or refresh of the associated segment, when the sense amp accesses one or more segments, it can trigger the writing of the timestamp from timer 2901 to registers, memory, or other elements that contain data storage.

In any of the embodiments described above, refresh controller 2900 may be integrated with a memory controller for multiple memory banks. For example, similar to the embodiment depicted in FIG. 3A , refresh controller 2900 may be incorporated into logic and control subunits associated with memory banks or other sections of memory chip 2800 .

Figure 29B depicts another embodiment refresh controller 2900' consistent with the present disclosure. The refresh controller 2900 ′ may be incorporated into a memory chip of the present disclosure, such as the memory chip 2800 of FIG. 28 . Similar to refresh controller 2900 , refresh controller 2900 ′ includes timer 2901 , line counter 2903 , valid bits 2905 and adder 2907 . Additionally, refresh controller 2900' may include data storage 2909. As shown in FIG. 29B, data store 2909 may include one or more registers or non-volatile registers associated with a segment of memory chip 2800 (eg, a line, column, or any other grouping of memory cells within memory chip 2800). Volatile memory, and the state within the data store can be configured to change in response to one or more segments being accessed (eg, through sense amplifiers and/or other elements of the refresh controller 2900', as above Described). Accordingly, the refresh controller 2900' may be configured to skip the refresh of one or more sectors based on the state within the data store. For example, the refresh controller 2900' may skip the associated region if the state associated with the segment is enabled (eg, set to 1 by turning on, changing a property to store a "1", or the like) A refresh cycle for a segment and clears the state associated with its part. The state may be stored by at least a one-bit register or any other memory structure configured to store at least one bit of data.

To ensure that sectors of the memory chip are refreshed or accessed during each refresh cycle, refresh controller 2900' may reset or otherwise clear the state to trigger a refresh signal during the next refresh cycle. In some embodiments, after a segment is skipped, the refresh controller 2900' may clear the associated state to ensure that the segment is refreshed on the next refresh cycle. In other embodiments, the refresh controller 2900' may be configured to reset the state within the data store after a threshold time interval. For example, whenever the timer 2901 exceeds a threshold time since the associated state has been set (eg, set to 1 by turning on, changing a property to store a "1", or the like), the controller 2900' is refreshed The state in the data store can be cleared (eg, set to 0). In some embodiments, refresh controller 2900' may use a critical number of refresh cycles (eg, one, two, or the like) or use a critical number of clock cycles (eg, two, four, or the like) rather than a threshold time.

In other embodiments, the state may include the timestamp of the most recent refresh or access of the associated segment, such that if the time between the timestamp and the current time (eg, timer 2901 from FIGS. 29A and 29B ) is amount exceeds a predetermined threshold (eg, 8ms, 16ms, 32ms, 64ms, or the like), the refresh controller 2900' may send an access command or refresh signal to the associated segment and update the timestamp associated with its portion ( For example, use timer 2901). Additionally or alternatively, if the refresh time indicator indicates that the last refresh time is within a predetermined time threshold, the refresh controller 2900' may be configured to skip a refresh operation with respect to one or more sectors in the plurality of memory banks. In these embodiments, after skipping a refresh operation with respect to one or more segments, the refresh controller 2900' may be configured to alter the stored refresh time indicators associated with the one or more segments, Such that during the next cycle of operation, the one or more sections will be refreshed. For example, as described above, refresh controller 2900' may use timer 2901 to update the stored refresh time indicator.

Accordingly, the data store may include a time stamp register configured to store a refresh time indicator indicating when one or more sectors in the plurality of memory banks were last refreshed. Additionally, the refresh controller 2900' may use the output of the timer to clear the access information stored in the data store after a threshold time interval.

In any of the embodiments described above, an access to one or more sectors may include a write operation associated with the one or more sectors. Additionally or alternatively, accessing the one or more sectors may include a read operation associated with the one or more sectors.

Additionally, as depicted in FIG. 29B, refresh controller 2900' may include row counter 2903 and adder 2907 configured to assist in updating data store 2909 based at least in part on the state within the data store. Data store 2909 may contain bit tables associated with multiple memory banks. For example, the bit table may contain an array of switches (or switching elements, such as transistors) or registers (eg, SRAM or the like) configured to hold bits for the associated segment. Additionally or alternatively, the data store 2909 may store timestamps associated with multiple memory banks.

Additionally, the refresh controller 2900' may include a refresh gate 2911 configured to control whether a refresh of one or more sectors is performed based on corresponding values stored in the bit table. For example, refresh gate 2911 may include a logic gate (such as an "and gate") that indicates that the associated segment was refreshed or accessed during one or more previous clock cycles at the corresponding state of data store 2909 deasserts the refresh signal from row counter 2903. In other embodiments, refresh gate 2911 may include a microprocessor or other circuit configured to operate at corresponding times from data store 2909 The stamp indicates that the refresh signal from row counter 2903 is invalidated if the associated segment is refreshed or accessed within a predetermined threshold time value.

Figure 30 is a flow diagram of an embodiment of a process 3000 for partial refresh in a memory chip (eg, memory chip 2800 of Figure 28) that may be executed by a refresh controller consistent with the present disclosure, such as the refresh control of Figure 29A 2900 or refresh controller 2900' of FIG. 29B.

At step 3010, the refresh controller may access information indicative of access operations for one or more sectors in the plurality of memory banks. For example, as explained above with respect to FIGS. 29A and 29B , the refresh controller may include a data store that is associated with a segment of the memory chip 2800 (eg, a line, column, or any line of memory cells within the memory chip 2800 ) other packets) are associated and are configured to store timestamps or other information indicating the most recent access to the associated segment.

At step 3020, the refresh controller may generate refresh and/or access commands based at least in part on the accessed information. For example, as explained above with respect to Figures 29A and 29B, if the accessed information indicates that the last refresh or access time is within a predetermined time threshold and/or if the accessed information indicates that the last refresh or access occurred one or more previous During a clock cycle, the refresh controller may then skip refresh operations with respect to one or more banks in the plurality of memory banks. Additionally or alternatively, the refresh controller may be based on whether the accessed information indicates that the last refresh or access time exceeded a predetermined threshold and/or whether the accessed information indicates that the last refresh or access did not occur during one or more previous clock cycles , which generates views to refresh or access the associated section.

At step 3030, the refresh controller may modify the stored refresh time indicators associated with one or more segments so that during the next cycle of operation, the one or more segments will be refreshed. For example, after skipping a refresh operation with respect to one or more sectors, the refresh controller may alter information indicative of access operations for the one or more sectors such that during the next clock cycle, the one or more sectors will be refreshed. multiple sections. Therefore, after skipping the refresh cycle, the refresh controller may clear the state of the segment (eg, set to 0). Additionally or alternatively, the refresh controller may set the state (eg, set to 1) of the segment that is refreshed and/or accessed during the current cycle. In embodiments where the information indicative of an access operation for one or more segments includes a time stamp, the refresh controller may update any stored time stamps associated with segments that were refreshed and/or accessed during the current cycle .

Method 3000 may also include additional steps. For example, in addition to or in lieu of step 3030, a sense amplifier may access one or more segments and may change information associated with the one or more segments. Additionally or alternatively, the sense amplifier can signal the refresh controller when an access has occurred so that the refresh controller can update information associated with one or more segments. As explained above, a sense amplifier may include a plurality of transistors configured to sense low power signals from a segment of a memory chip that stores data in one or more memory cells, and to store data in one or more memory cells. Small voltage swings are amplified to higher voltage levels so that the data can be interpreted by logic such as an external CPU or GPU or an integrated processor sub-unit as explained above. In this embodiment, whenever a sense amplifier accesses one or more segments, it may set (eg, set to 1) a bit associated with the segment that indicates that the associated segment is previous Accessed in a cycle. In embodiments where the information indicative of the access operation of one or more sectors includes a timestamp, each time the sense amplifier accesses one or more sectors, it can trigger a timer to be sent from the refresh controller. Timestamps are written to the data store to update any stored timestamps associated with the segment.

FIG. 31 is a flowchart of an embodiment of a processing procedure 3100 for determining refresh of a memory chip (eg, the memory chip 2800 of FIG. 28 ). The handler 3100 may be implemented within a compiler consistent with the present disclosure. As explained above, a "compiler" refers to converting a higher-level language (eg, a procedural language, such as C, FORTRAN, BASIC, or the like; an object-oriented language, such as Java, C++, Pascal, Python, or the like; etc.) into a lower-level language (eg, composite code, object code, machine code, or the like). A compiler may allow a human to program a series of instructions in a human-readable language, and then convert the human-readable language into a machine-executable language. A compiler may contain software instructions that are executed by one or more processors.

At step 3110, one or more processors may receive higher order computer code. For example, the higher-level computer code may be encoded in one or more files on memory (eg, non-volatile memory such as a hard disk drive or the like, volatile memory such as DRAM, or the like) or received via a network (eg, the Internet or the like). Additionally or alternatively, the higher-level computer code may be received from a user (eg, using an input device such as a keyboard).

At step 3120, the one or more processors may identify a plurality of memory sectors distributed over a plurality of memory banks associated with the memory chip to be accessed by the higher level computer code. For example, one or more processors may access data structures that define a plurality of memory banks and a corresponding structure of a memory chip. One or more processors can access data structures from memory (eg, non-volatile memory such as a hard drive or the like, volatile memory such as DRAM, or the like), or via a network (eg, Internet or the like) to receive the data structure. In these embodiments, the data structures are included in one or more libraries accessible by the compiler to permit the compiler to generate instructions for the particular memory chip to be accessed.

At step 3130, one or a processor may evaluate higher-level computer code to identify multiple memory read commands occurring within multiple memory access cycles. For example, one or more processors may identify each operation within higher level computer code that requires one or more read commands to read from memory and/or one or more write commands to write to memory. These instructions may include variable initialization, variable reassignment, logical operations on variables, input and output operations, or the like.

At step 3140, the one or more processors may cause the distribution of data associated with the plurality of memory access commands across each of the plurality of memory sectors such that during each of the plurality of memory access cycles Each of the plurality of memory banks is accessed. For example, one or more processors may customize the data structure of the memory chip's structure to identify memory segments, and then assign variables from higher-level code to each of the memory segments such that at each refresh cycle ( It may include accessing (eg, via writing or reading) each memory segment at least once during a specified number of clock cycles). In this embodiment, one or more processors may access information indicating how many clock cycles each line of higher-order code requires in order to assign variables from lines of higher-order code such that at a particular number of clock cycles Each memory segment is accessed (eg, via writing or reading) at least once during the period.

In another embodiment, one or more processors may first generate machine code or other lower-order code from higher-order code. The one or more processors may then assign variables from the lower order code to each of the memory segments such that during each refresh cycle (which may include a certain number of clock cycles) accesses (eg, via writes) input or read) at least once per memory segment. In this embodiment, each line of lower order code may require a single clock cycle.

In any of the embodiments presented above, one or more processors may further assign logical operations or other commands using temporary outputs to each of the memory segments. These temporary outputs can also generate read and/or write commands so that the assigned memory segment is accessed during this refresh cycle even if the specified variable has not been assigned to the memory segment.

Method 3100 may also include additional steps. For example, in embodiments where variables are assigned prior to compilation, one or more processors may generate machine code or other lower-order code from higher-order code. In addition, one or more processors may transmit compiled code for execution by memory chips and corresponding logic circuits. The logic circuit may include conventional circuitry such as a GPU or CPU, or may include a processing group on the same substrate as the memory chip, eg, as depicted in Figure 7A. Thus, as described above, the substrate may include a memory array that includes a plurality of groups, such as group 2801a shown in FIG. 28 and others. Additionally, the substrate can include a processing array that can include a plurality of processor subunits (such as subunits 730a, 730b, 730c, 730d, 730e, 730f, 730g, and 730h shown in Figure 7A).

FIG. 32 is a flowchart of another embodiment of a processing routine 3200 for determining refresh of a memory chip (eg, memory chip 2800 of FIG. 28 ). The handler 3200 may be implemented within a compiler consistent with the present disclosure. Process 3200 may be executed by one or more processors executing software instructions including a compiler. The processing procedure 3200 may be implemented separately or in combination with the processing procedure 3100 of FIG. 31 .

At step 3210, similar to step 3110, one or more processors may receive higher order computer code. At step 3220, similar to step 3210, one or more processors may identify a plurality of memory sectors distributed over a plurality of memory banks associated with the memory chip to be accessed by the higher level computer code.

At step 3230, the one or more processors may evaluate higher-level computer code to identify a plurality of memory read commands each involving one or more of the plurality of memory sectors. For example, one or more processors may identify each operation within higher level computer code that requires one or more read commands to read from memory and/or one or more write commands to write to memory. These instructions may include variable initialization, variable reassignment, logical operations on variables, input and output operations, or the like.

In some embodiments, one or more processors may simulate the execution of higher order code using logic circuits and multiple memory segments. For example, the simulation may include a line-by-line step-through of higher order code, similar to the case of a debugger or other instruction set simulator (ISS). The emulation may further maintain internal variables representing addresses of multiple memory segments, similar to how a debugger may maintain internal variables representing registers of a processor.

At step 3240, the one or more processors may track, based on the analysis of the memory access commands, and for each memory segment of the plurality of memory segments, the accumulated amount since the last access to the memory segment amount of time. For example, using the simulations described above, one or more processors may determine whether each access (eg, read or write) to one or more addresses within each of the plurality of memory segments length of time between. The length of time can be measured in absolute time, clock cycles, or refresh cycles (eg, as determined by the known refresh rate of the memory chip).

At step 3250, in response to a determination that the amount of time elapsed since the last access of any particular memory segment will exceed a predetermined threshold, the one or more processors may be configured to cause access to the particular memory segment At least one of a memory refresh command or a memory access command of the is introduced into the higher-level computer code. For example, one or more processors may include refresh commands for execution by a refresh controller (eg, refresh controller 2900 of Figure 29A or refresh controller 2900' of Figure 29B). In embodiments where the logic circuits are not embedded on the same substrate as the memory chips, one or more processors may generate refresh commands for sending to the memory chips separate from lower-level code for sending to the logic circuits.

Additionally or alternatively, the one or more processors may include access commands for execution by the memory controller (which may be separate from or incorporated into the refresh controller). The access command may include a dummy command configured to trigger a read operation to the memory segment, but not cause the logic circuit to perform any other operations on the read or written variables from the memory segment.

In some embodiments, the compiler may include a combination of steps from handler 3100 and steps from handler 3200 . For example, the compiler may assign variables according to step 3140 and then run the simulation described above according to step 3250 to add any additional memory refresh commands or memory access commands. This combination may allow the compiler to distribute variables across as many memory segments as possible, and generate flush or access commands for any memory segment that cannot be accessed within a predetermined threshold amount of time. In another combined embodiment, the compiler may simulate the code according to step 3230 and assign variables according to step 3140 based on the simulation indicating any memory segments that will not be accessed for a predetermined threshold amount of time. In some embodiments, this combination may also include step 3250 to allow the compiler to generate refresh or access commands for any memory segments that cannot be accessed within a predetermined threshold amount of time, even after assignments according to step 3140 are complete .

The refresh controller of the present disclosure may allow software executed by logic circuits (whether conventional logic circuits such as CPUs and GPUs or processing groups on the same substrate as memory chips, eg, as depicted in FIG. 7A ) to disable Automatic refresh performed by the refresh controller, and alternatively controlled refresh via executed software. Accordingly, some embodiments of the present disclosure may provide software with known access patterns to a memory chip (eg, if a compiler can access data structures that define multiple memory banks and a corresponding structure of the memory chip). In these embodiments, the post-compile optimizer may disable automatic flushing and manually set flush control only for sections of the memory chip that have not been accessed for a threshold amount of time. Thus, similar to step 3250 described above but after compilation, the post-compile optimizer may generate flush commands to ensure that each memory segment is accessed or flushed using a predetermined threshold amount of time.

Another embodiment of reducing refresh cycles may include using a predefined pattern of accesses to memory chips. For example, some embodiments may generate access patterns for refresh beyond conventional linear line refresh if software executed by logic circuitry can control its access patterns for memory chips. For example, if the controller determines that the software executed by the logic circuit regularly accesses every other line of memory, the refresh controller of the present disclosure can use an access pattern that is not refreshed every other line in order to speed up the memory chip and reduce power usage .

An embodiment of such a refresh controller is shown in FIG. 33 . 33 depicts an embodiment refresh controller 3300 configured with stored patterns consistent with the present disclosure. The refresh controller 3300 may be incorporated into a memory chip of the present disclosure, such as the memory chip 2800 of FIG. 28 , for example, having a plurality of memory banks and a plurality of memory sectors included in each of the plurality of memory banks.

The refresh controller 3300 includes a timer 3301 (similar to the timer 2901 of FIGS. 29A and 29B ), a row counter 3303 (similar to the row counter 2903 of FIGS. 29A and 29B ), and an adder 3305 (similar to the row counter 2903 of FIGS. 29A and 29B ) adder 2907). Additionally, refresh controller 3300 includes data storage 3307 . Unlike data storage area 2909 of FIG. 29B, data storage 3307 may store at least one memory refresh pattern to be implemented to refresh a plurality of memory sectors included in each of a plurality of memory banks . For example, as depicted in FIG. 33, data store 3307 may include Li (eg, in the embodiment of FIG. 33, L1, L2, L3, and L4) and Hi of rows and/or columns that define segments in a memory bank (eg, in the embodiment of Figure 33, H1, H2, H3, and H4). Additionally, each section may be associated with an Inci variable (eg, in the embodiment of Figure 33, Inc1, Inc2, Inc3, and Inc4) that defines how the row associated with the section is incremented (eg, whether to access or refresh every row, whether to access or refresh every other row, or the like). Thus, as shown in FIG. 33, the refresh pattern may include a table that includes a plurality of memory sector identifiers assigned by software to identify those in a particular memory bank that are being refreshed The range of memory sectors that need to be refreshed during the cycle, and the range of memory sectors in that particular memory bank that do not need to be refreshed during the refresh cycle.

Thus, data store 3308 may define software executed by logic circuits (whether conventional logic circuits such as CPUs and GPUs or processing groups on the same substrate as memory chips, eg, as depicted in FIG. 7A ) that can be selected to Refresh pattern for use. The memory refresh pattern is software configurable to identify which of the plurality of memory banks in a particular memory bank need to be refreshed during a refresh cycle and which of the plurality of memory banks in a particular memory bank are in that refresh cycle No need to refresh during this period. Accordingly, the refresh controller 3300 may refresh some or all of the rows within the defined segment that have not been accessed during the current cycle according to Inci. Refresh controller 3300 may skip other rows of the defined section that are set to be accessed during the current cycle.

In embodiments where the data store 3308 of the refresh controller 3300 includes multiple memory refresh patterns, each memory refresh pattern may represent a different refresh pattern for refreshing multiple ones included in each of the multiple memory banks memory segment. The memory refresh pattern may be selectable for use on multiple memory sectors. Accordingly, refresh controller 3300 may be configured to allow selection of which of a plurality of memory refresh patterns to implement during a particular refresh cycle. For example, software executed by logic circuits (whether conventional logic circuits such as CPUs and GPUs or processing groups on the same substrate as memory chips, eg, as depicted in FIG. 7A ) may select different memory refresh patterns for use in Used during one or more different refresh cycles. Alternatively, software executed by logic circuitry may select a memory refresh pattern for use throughout some or all of the different refresh cycles.

The memory refresh pattern may be encoded using one or more variables stored in data store 3308. For example, in embodiments where multiple memory banks are arranged in rows, each memory bank identifier may be configured to identify a particular location within a row of memory at which a memory refresh should begin or end. For example, in addition to Li and Hi, one or more additional variables may also define which parts of which rows are within a section as defined by Li and Hi.

34 is a flowchart of an embodiment of a processing routine 3400 for determining refresh of a memory chip (eg, memory chip 2800 of FIG. 28). Process 3100 may be implemented by software within a refresh controller consistent with the present disclosure (eg, refresh controller 3300 of Figure 33).

At step 3410, the refresh controller may store at least one memory refresh pattern to be implemented to refresh the plurality of memory banks included in each of the plurality of memory banks. For example, as explained above with respect to FIG. 33, the refresh pattern may include a table including a plurality of memory sector identifiers assigned by software to identify the The range of memory banks that need to be refreshed during a refresh cycle, and the range of memory banks that do not need to be refreshed during the refresh cycle in that particular memory bank.

In some embodiments, the at least one refresh pattern may be encoded onto the refresh controller during manufacture (eg, onto a read-only memory associated with or at least accessible by the refresh controller). Accordingly, the refresh controller may access the at least one memory refresh pattern, but not store the at least one memory refresh pattern.

At steps 3420 and 3430, the refresh controller may use software to identify which of the plurality of memory banks in a particular memory bank need to be refreshed during a refresh cycle and which of the plurality of memory banks in the particular memory bank are No refresh is required during this refresh cycle. For example, as explained above with respect to FIG. 33, software executed by logic circuits (whether conventional logic circuits such as CPUs and GPUs or processing groups on the same substrate as memory chips, eg, as depicted in FIG. 7A), may Select at least one memory refresh pattern. Additionally, the refresh controller may access the selected at least one memory refresh pattern to generate corresponding refresh signals during each refresh cycle. The refresh controller may refresh some or all portions of the defined segments that are not accessed during the current cycle according to the at least one memory refresh pattern, and may skip defined defined segments that are set to be accessed during the current cycle other parts of the section.

At step 3440, the refresh controller may generate a corresponding refresh command. For example, as depicted in FIG. 33 , adder 3305 may include logic configured to de-assert the refresh signal for a particular sector that is not refreshed according to at least one memory refresh pattern in data store 3307 . Additionally or alternatively, a microprocessor (not shown in FIG. 33 ) may generate specific refresh signals based on which sectors are to be refreshed according to at least one memory refresh pattern in data store 3307 .

Method 3400 may also include additional steps. For example, in an implementation where at least one memory refresh pattern is configured to change for every, two, or other number of refresh cycles (eg, moving from L1, H1, and Inc1 to L2, H2, and Inc2, as shown in FIG. 33) For example, the refresh controller may access different portions of the data store according to steps 3430 and 3440 for the next determination of the refresh signal. Similarly, if software executed by logic circuits (whether conventional logic circuits such as CPUs and GPUs or processing groups on the same substrate as memory chips, eg, as depicted in FIG. 7A ) selects new data from the data store The memory refresh pattern is used in one or more future refresh cycles, then the refresh controller may access different portions of the data store according to steps 3430 and 3440 for the next determination of the refresh signal.

When designing a memory chip and targeting a certain capacity of memory, changing the memory capacity to a larger or smaller size may require redesign of the product and redesign of the entire reticle set. Often, product design is performed in parallel with market research, and in some cases, product design is completed before market research is available. Therefore, there may be a disconnect between product design and the actual needs of the market. The present disclosure proposes a way to flexibly provide memory chips with memory capacities that meet market demands. The design method may include designing the die together with appropriate interconnecting circuitry on the wafer such that memory chips, which may contain one or more dies, may be selectively diced from the wafer in order to provide production from a single wafer with sizes of variable size. Opportunities for memory chips of variable memory capacity.

The present disclosure relates to systems and methods for fabricating memory chips by dicing memory chips from a wafer. The method can be used to produce memory chips of selectable size from wafers. An embodiment embodiment of wafer 3501 containing die 3503 is shown in FIG. 35A. Wafer 3501 may be made of semiconductor material (eg, silicon (Si), silicon germanium (SiGe), silicon on insulator (SOI), gallium nitride (GaN), aluminum nitride (AlN), aluminum gallium nitride (AlGaN), nitrogen Boronide (BN), Gallium Arsenide (GaAs), Aluminum Gallium Arsenide (AlGaAs), Indium Nitride (InN), combinations of the above, and the like). Die 3503 may include any suitable circuit elements (eg, transistors, capacitors, resistors, and/or the like), which may include any suitable semiconductor, dielectric, or metallic components. Die 3503 may be formed of a semiconductor material that may or may not be the same material as wafer 3501 . In addition to die 3503, wafer 3501 may also include other structures and/or circuitry. In some embodiments, one or more coupling circuits may be provided and couple one or more of the dies together. In one example embodiment, this coupling circuit may include a bus shared by two or more dies 3503 . Additionally, the coupling circuit may include one or more logic circuits designed to control circuitry associated with die 3503 and/or to direct information to/from die 3503 . In some cases, the coupling circuit may include memory access management logic. This logic may translate logical memory addresses into physical addresses associated with die 3503. It should be noted that, as used herein, the term fabrication may collectively refer to any of the steps used to build the disclosed wafers, dies, and/or chips. For example, fabrication may refer to the simultaneous placement and formation of various dies (and any other circuitry) included on a wafer. Fabrication may also refer to dicing memory chips of selectable sizes from a wafer to include one die in some cases, or multiple dies in other cases. Of course, the term fabrication is not intended to be limited to these embodiments, but may include other aspects associated with the production of any or all of the disclosed memory chips and intermediate structures.

Die 3503 or a group of dies can be used to fabricate memory chips. The memory chips may include distributed processors, as described in other sections of this disclosure. As shown in Figure 35B, die 3503 may include a substrate 3507 and a memory array disposed on the substrate. The memory array may include one or more memory cells, such as memory banks 3511A-3511D designed to store data. In various embodiments, the memory banks may include semiconductor-based circuit elements such as transistors, capacitors, and the like. In one embodiment, a memory bank may include multiple rows and columns of storage cells. In some cases, this memory bank may have a capacity greater than one megabyte. The memory bank may include dynamic or static access memory.

Die 3503 may also include a processing array disposed on the substrate, the processing array including a plurality of processor subunits 3515A-3515D, as shown in Figure 35B. As described above, each memory bank may include dedicated processor subunits connected by a dedicated bus. For example, processor subunit 3515A is associated with memory bank 3511A via bus or connection 3512. It should be understood that various connections between memory banks 3511A-3511D and processor subunits 3515A-3515D are possible, and that only some illustrative connections are shown in Figure 35B. In an embodiment embodiment, the processor sub-units may perform read/write operations to the associated memory banks, and may further perform refresh operations or any other suitable operations with respect to the memory stored in the various memory banks .

As mentioned, die 3503 may include a first group of buses configured to connect processor subunits with their corresponding memory banks. An embodiment bus may include a set of wires or conductors that connect electrical components and allow data and addresses to be transferred to and from each memory bank and its associated processor subunits. unit transmits data. In one embodiment, connector 3512 may serve as a dedicated bus for connecting processor subunit 3515A to memory bank 3511A. Die 3503 may include groups of such buses, each bus connecting a processor subunit to a corresponding dedicated memory bank. Additionally, die 3503 may include another group of buses, each bus connecting processor subunits (eg, subunits 3515A-3515D) to each other. For example, such a bus may include connections 3516A-3516D. In various embodiments, data for memory banks 3511A-3511D may be delivered via input-output bus 3530. In one embodiment, the I/O bus 3530 may carry data-related information, as well as command-related information for controlling the operation of the memory cells of the die 3503 . The data information may include data for data stored in the memory banks, data read from the memory banks, processing from one or more of the processor sub-units based on operations performed relative to the data stored in the corresponding memory banks Results, command-related information, various codes, etc.

Under various conditions, the data and commands transmitted by the input output bus 3530 may be controlled by the input output (IO) controller 3521 . In one embodiment, the IO controller 3521 may control the flow of data from the bus 3530 to and from the processor sub-units 3515A-3515D. The IO controller 3521 can determine from which of the processor sub-units 3515A-3515D the information is to be retrieved. In various embodiments, the IO controller 3521 may include a fuse 3554 configured to deactivate the IO controller 3521. Fuse 3554 may be used if multiple dies are combined together to form a larger memory chip (also known as a multi-die memory chip, as an alternative to a single-die memory chip containing only one die). The multi-die memory chip can then use one of the IO controllers that form one of the die cells of the multi-die memory chip, while using fuses corresponding to the other IO controllers associated with the other die cells to deactivate other IO controllers.

As mentioned, each memory chip or pre-die or group of dies may include a distributed processor associated with the corresponding memory bank. In some embodiments, these distributed processors may be arranged in a processing array disposed on the same substrate as multiple memory banks. Additionally, the processing array may include one or more logic sections each including an address generator (also referred to as an address generator unit (AGU)). In some cases, the address generator may be part of at least one processor subunit. The address generator may generate the memory addresses required to fetch data from one or more memory banks associated with the memory chip. Address generation calculations may involve integer arithmetic operations such as addition, subtraction, modulo operations, or bit shifting. The address generator can be configured to operate on multiple operands at once. Additionally, multiple address generators can perform more than one address calculation operation simultaneously. In various embodiments, address generators may be associated with corresponding memory banks. The address generators can be connected to their corresponding memory banks by means of corresponding bus lines.

In various embodiments, memory chips of selectable size may be formed from wafer 3501 by selectively dicing different regions of the wafer. As mentioned, the wafer may include a group of dies 3503 that includes two or more dies (eg, 2, 3, 4, 5) included on the wafer , 10 or more dies) any group. As will be discussed further below, in some cases a single memory chip may be formed by dicing a portion of a wafer that includes only one of a group of dies. Under these conditions, the resulting memory chip will include memory cells associated with one die. However, in other cases, sized memory chips may be formed to include more than one die. These memory chips may be formed by dicing regions of the wafer that include two or more dies in a group of dies included on the wafer. Under these conditions, the dies together with the coupling circuits that couple the dies together provide a multi-die memory chip. Some additional circuit elements may also be grounded on-board between chips, such as clock elements, data buses, or any suitable logic circuit.

In some cases, at least one controller associated with a group of dies may be configured to control the group of dies to operate as a single memory chip (eg, a multi-memory cell memory chip). The controller may include one or more circuits that manage the flow of data to and from the memory chip. The memory controller may be part of the memory chip, or it may be part of a separate chip not directly related to the memory chip. In an embodiment, the controller may be configured to facilitate read and write requests or other commands associated with the distributed processors of the memory chip, and may be configured to control any other suitable aspect of the memory chip (eg, , refresh memory chips, interact with distributed processors, etc.). In some cases, the controller may be part of die 3503 , and in other cases, the controller may be disposed adjacent to die 3503 . In various embodiments, the controller may also include at least one memory controller for at least one of the memory cells included on the memory chip. In some cases, the protocol used to access information on a memory chip may be independent of replication logic and memory cells (eg, memory banks) that may exist on the memory chip. The protocol can be configured with different IDs or address ranges for adequate access to data on the memory chip. Examples of chips with this protocol may include chips with a Joint Electronic Device Engineering Council (JEDEC) Double Data Rate (DDR) controller, where different memory banks may have different address ranges, Serial Peripheral Interface (SPI) connections, where Different memory cells (eg, memory banks) have different identifications (IDs), and the like.

In various embodiments, multiple regions may be cut from the wafer, wherein each region includes one or more dies. In some cases, each partition can be used to build a multi-die memory chip. In other cases, each region to be cut from the wafer may include a single die to provide a single die memory chip. In some cases, two or more of the regions may have the same shape and have the same number of dies coupled to the coupling circuit in the same manner. Alternatively, in some embodiments, a first group of regions may be used to form a first type of memory chip, and a second group of regions may be used to form a second type of memory chip. For example, as shown in Figure 35C, wafer 3501 may include region 3505, which may include a single die, and second region 3504, which may include a group of two dies. When region 3505 is cut from wafer 3501, a single die memory chip will be provided. When region 3504 is diced from wafer 3501, a multi-die memory chip will be provided. The groups shown in FIG. 35C are illustrative only, and various other regions and groups of die may be cut from wafer 3501.

In various embodiments, dies may be formed on wafer 3501 such that they are arranged along one or more rows of the wafer, as shown, for example, in Figure 35C. The dies may share an I/O bus 3530 corresponding to one or more rows. In one embodiment, various dicing shapes may be used to cut groups of dies from wafer 3501, where shared input and output may not be included when cutting a group of dies that may be used to form memory chips At least a portion of bus 3530 (eg, only a portion of input-output bus 3530 may be included as part of a memory chip formed to include a group of dies).

As previously discussed, when multiple dies (eg, dies 3506A and 3506B, as shown in FIG. 35C ) are used to form memory chip 3517, one IO controller corresponds to one of the multiple dies May be enabled and configured to control data flow to all processor subunits of dies 3506A and 3506B. For example, Figure 35D shows memory dies 3506A and 3506B combined to form memory chip 3517, which includes memory banks 3511A-3511H, processor subunits 3515A-3515H, IO controllers 3521A and 3521B, and fuses 3554A and 3554A. 3554B. It should be noted that the memory chip 3517 corresponds to the region 3517 of the wafer 3501 before the memory chip 3517 is removed from the wafer. In other words, as used here and elsewhere in this disclosure, once diced from wafer 3501, regions 3504, 3505, 3517, etc. of wafer 3501 will yield memory chips 3504, 3505, 3517, etc. Additionally, fuses are also referred to herein as deactivation elements. In one embodiment, fuse 3554B may be used to deactivate IO controller 3521B, and IO controller 3521A may be used to control data flow to all memory banks 3511A-3511H by communicating data to processor subunits 3515A-3515H. In one embodiment, the IO controller 3521A may connect to the various processor subunits using any suitable connections. In some embodiments, as described further below, the processor subunits 3515A-3515H may be interconnected, and the IO controller 3521A may be configured to control communication to the processor subunits 3515A-3515H that form the processing logic of the memory chip 3517. data flow.

In one embodiment, IO controllers such as controllers 3521A and 3521B and corresponding fuses 3554A and 3554B may be formed on wafer 3501 along with forming memory banks 3511A-3511H and processor subunits 3515A-3515H. In various embodiments, when memory chip 3517 is formed, one of the fuses (eg, fuse 3554B) can be activated such that dies 3506A and 3506B are configured to form memory chip 3517 that functions as a single chip and Controlled by a single I/O controller (eg, controller 3521A). In one embodiment, activating the fuse may include applying a current to trigger the fuse. In various embodiments, when more than one die is used to form a memory chip, all but one IO controller may be deactivated via corresponding fuses.

In various embodiments, as shown in FIG. 35C, multiple dies are formed on wafer 3501 together with the same set of input-output buses and/or control buses. An embodiment of an I/O bus 3530 is shown in Figure 35C. In one embodiment, one of the I/O buses (eg, I/O bus 3530) may be connected to multiple dies. FIG. 35C shows an embodiment of an input-output bus 3530 through which proximity dies 3506A and 3506B pass. The configuration of dies 3506A and 3506B and input output bus 3530 as shown in FIG. 35C is merely illustrative, and various other configurations may be used. For example, FIG. 35E illustrates die 3540 formed on wafer 3501 and arranged in a hexagonal form. A memory chip 3532 including four dies 3540 can be diced from wafer 3501 . In one embodiment, memory chip 3532 may include a portion of input-output bus 3530 connected to the four dies through suitable bus lines (eg, line 3533, as shown in FIG. 35E). To route information to the appropriate memory cells of memory chip 3532, memory chip 3532 may include input/output controllers 3542A and 3542B placed at branch points of output bus 3530. Controllers 3542A and 3542B may receive command data via input-output bus 3530 and select branches of bus 3530 for transferring information to the appropriate memory cells. For example, if the command data includes read/write information from/to memory cells associated with die 3546, then controller 3542A may receive the command request and transfer the data to branch 3531A of bus 3530, as shown in Figure 35D Shown, while controller 3542B can receive the command request and transmit data to branch 3531B. Figure 35E indicates the various cuts that can be made in different regions, where the cut lines are represented by dashed lines.

In one embodiment, a group of dies and interconnect circuitry can be designed to be included in a memory chip 3506 as shown in Figure 36A. This embodiment may include processor subunits (for in-memory processing) that may be configured to communicate with each other. For example, each die to be included in memory chip 3506 may include various memory cells such as memory banks 3511A-3511D, processor sub-units 3515A-3515D, and IO controllers 3521 and 3522. The IO controllers 3521 and 3522 can be connected to the input and output bus 3530 in parallel. IO controller 3521 may have fuse 3554 and IO controller 3522 may have fuse 3555. In one embodiment, the processor subunits 3515A-3515D may be connected by means of a bus 3613, for example. In some cases, one of the IO controllers may be disabled using a corresponding fuse. For example, fuse 3555 can be used to disable IO controller 3522, and IO controller 3521 can control data flow into memory banks 3511A-3511D via processor subunits 3515A-3515D, which are connected to each other via bus 3613 .

The configuration of memory cells as shown in FIG. 36A is merely illustrative, and various other configurations may be formed by dicing different regions of wafer 3501 . For example, FIG. 36B shows a configuration with three domains 3601-3603 containing memory cells and connected to an input-output bus 3530. In one embodiment, domains 3601-3603 are connected to I/O bus 3530 using IO control modules 3521-3523 that can be deactivated by corresponding fuses 3554-3556. Another embodiment of an embodiment of arranging domains containing memory cells is shown in FIG. 36C , where bus lines 3611 , 3612 and 3613 are used to connect three domains 3601 , 3602 and 3603 to an input-output bus 3530 . 36D shows another embodiment of memory chips 3506A-3506D connected to input output buses 3530A and 3530B via IO controllers 3521-3524. In one embodiment, the IO controller may be deactivated using corresponding fuse elements 3554-3557, as shown in Figure 36D.

37 shows various groups of dies 3503, such as group 3713 and group 3715, which may include one or more dies 3503. In one embodiment, in addition to forming die 3503 on wafer 3501 , wafer 3501 may also contain logic circuits 3711 referred to as glue logic 3711 . Glue logic 3711 may take up some space on wafer 3501 to result in a lower number of dies being fabricated per wafer 3501 compared to the number of dies that could have been fabricated without glue logic 3711 . However, the presence of glue logic 3711 may allow multiple dies to be configured together to function as a single memory chip. For example, glue logic can connect multiple dies without having to change configurations and without specifying areas within any of the dies themselves for circuitry that is only used to connect the dies together. In various embodiments, glue logic 3711 provides an interface to other memory controllers so that a multi-die memory chip acts as a single memory chip. Glue logic 3711 may be diced together with dies of the same group (eg, as shown by group 3713). Alternatively, if only one die is required for the memory chip, eg, as for group 3715, the glue logic may not be cut. For example, glue logic can be selectively eliminated where collaboration between different dies is not required. In Figure 37, various cuts of different regions can be made, eg, as shown by the dashed regions. In various embodiments, as shown in FIG. 37, for every two dies 3506, one glue logic element 3711 may be placed on the wafer. In some cases, one glue logic element 3711 may be used to form any suitable number of dies 3506 of a group of dies. Glue logic 3711 may be configured to connect to all dies from a group of dies. In various embodiments, the die connected to glue logic 3711 may be configured to form a multi-die memory chip, and may be configured to form a separate single-die memory chip when the die is not connected to glue logic 3711 . In various embodiments, dies connected to glue logic 3711 and designed to function together may be diced from wafer 3501 as a group and may include glue logic 3711 , eg, as indicated by group 3713 . Dies not connected to glue logic 3711 may be diced from wafer 3501 without including glue logic 3711, eg, as indicated by group 3715, to form single-die memory chips.

In some embodiments, during the fabrication of multi-die memory chips from wafer 3501, one or more cutting shapes (eg, shapes forming groups 3713, 3715) may be determined for use in producing desired in the multi-die memory chips gather. In some cases, as shown by group 3715, the cut shape may not include glue logic 3711.

In various embodiments, glue logic 3711 may be a controller for controlling multiple memory cells of a multi-die memory chip. In some cases, glue logic 3711 may include parameters that are modifiable by various other controllers. For example, coupling circuitry for a multi-die memory chip may include circuitry for configuring parameters of glue logic 3711 or parameters of a memory controller (eg, processor subunits 3515A-3515D, as shown, eg, in FIG. 35B ) . Glue logic 3711 can be configured to perform a variety of tasks. For example, logic 3711 may be configured to determine which dies may need addressing. In some cases, logic 3711 may be used to synchronize multiple memory cells. In various embodiments, logic 3711 may be configured to control various memory cells such that the memory cells operate as a single chip. In some cases, amplifiers may be added between the input output bus (eg, bus 3530 as shown in FIG. 35C ) and the processor subunits 3515A-3515D to amplify the data signals from the bus 3530.

In various embodiments, cutting complex shapes from wafer 3501 may be technically difficult/expensive, and simpler cutting methods may be employed, limited by die alignment on wafer 3501 . For example, Figure 38A shows die 3506 aligned to form a rectangular grid. In one embodiment, vertical dicing 3803 and horizontal dicing 3801 across the entire wafer 3501 may be performed to separate a diced group of dies. In one embodiment, vertical dicing 3803 and horizontal dicing 3801 can produce groups containing a selected number of dies. For example, dicing 3803 and 3801 may result in a region containing a single die (eg, region 3811A), a region containing two die (eg, region 3811B), and a region containing four die (eg, region 3811C). The regions formed by cuts 3801 and 3803 are merely illustrative, and any other suitable regions may be formed. In various embodiments, various dicing may be performed depending on the die alignment. For example, if the dies are arranged in a triangular grid, as shown in Figure 38B, scribe lines such as lines 3802, 3804, and 3806 can be used to make a multi-die memory chip. For example, some regions may include six dies, five dies, four dies, three dies, two dies, one die, any other suitable number of dies.

38C shows bus lines 3530 arranged in a triangular grid, with dies 3503 aligned at the centers of the triangles formed by the intersection of bus lines 3530. Die 3503 may be connected to all adjacent bus lines via bus lines 3820 . By dicing a region (eg, region 3822, as shown in Figure 38C) containing two or more adjacent dies, at least one bus line (eg, line 3824) remains within region 3822, and bus line 3824 is available To supply data and commands to a multi-die memory chip formed using area 3822.

39 shows various connections that may be formed between processor subunits 3515A-3515P to allow groups of memory cells to function as a single memory chip. For example, group 3901 of various memory cells may include connections 3905 between processor sub-unit 3515B and sub-unit 3515E. Connector 3905 can be used as a bus line for transferring data and commands from subunit 3515B to subunit 3515E, which can be used to control respective memory banks 3511E. In various embodiments, connections between processor subunits may be implemented during die formation on wafer 3501 . In some cases, additional connections may be fabricated during the packaging stage of a memory chip formed from several dies.

As shown in FIG. 39, the processor subunits 3515A-3515P may be connected to each other using various buses (eg, connections 3905). Connectors 3905 may not contain sequential hardware logic components, such that data transfers between processor subunits and across connectors 3905 may not be controlled by sequential hardware logic components. In various embodiments, the busses connecting the processor subunits 3515A-3515P may be placed on wafer 3501 prior to fabrication of various circuits on wafer 3501.

In various embodiments, processor subunits (eg, subunits 3515A-3515P) may be interconnected. For example, subunits 3515A-3515P may be connected by a suitable bus (eg, connector 3905). Connector 3905 can connect any of subunits 3515A-3515P with any other of subunits 3515A-3515P. In one embodiment, the connected subunits may be on the same die (eg, subunits 3515A and 3515B), and in other cases, the connected subunits may be on different dies (eg, subunits 3515B and 3515E) ). Connector 3905 may include a dedicated bus for connector units and may be configured to efficiently transfer data between subunits 3515A-3515P.

Various aspects of the present disclosure relate to methods for producing selectable sized memory chips from wafers. In one embodiment, a size selectable memory chip may be formed from one or more dies. As mentioned previously, the dies may be arranged along one or more rows, as shown, for example, in Figure 35C. In some cases, at least one shared I/O bus corresponding to one or more rows may be arranged on wafer 3501 . For example, bus 3530 may be arranged as shown in Figure 35C. In various embodiments, the bus 3530 can be electrically connected to memory cells of at least two of the dies, and the connected dies can be used to form a multi-die memory chip. In one embodiment, one or more controllers (eg, input output controllers 3521 and 3522, as shown in FIG. 35B ) may be configured to control the control of at least two dies used to form a multi-die memory chip. memory unit. In various embodiments, a die with memory cells connected to bus 3530, which has a corresponding portion of a shared input-output bus (eg, bus 3530, as shown in FIG. 35B), can be diced from the wafer, The shared I/O bus communicates information to at least one controller (eg, controllers 3521, 3522) to configure the controller to control the memory cells of the connected die for use as a single chip.

In some cases, memory cells located on wafer 3501 may be tested prior to fabricating memory chips by dicing regions of wafer 3501 . This test can be performed using at least one shared input-output bus (eg, bus 3530, as shown in Figure 35C). When a memory cell passes the test, the memory chip may be formed from a group of dies containing the memory cell. Memory cells that fail the test can be discarded and not used in the manufacture of memory chips.

40 shows one embodiment of a process 4000 for building a memory chip from a group of dies. At step 4011 of process sequence 4000 , dies may be placed on semiconductor wafer 3501 . At step 4015, dies may be fabricated on wafer 3501 using any suitable method. For example, the die may be fabricated by etching the wafer 3501, depositing various dielectric, metal or semiconductor layers, further etching the deposited layers, and the like. For example, multiple layers can be deposited and etched. In various embodiments, the layers may be n-type doped or P-type doped using any suitable doping element. For example, the semiconductor layer may be n-doped with phosphorus and p-doped with boron. As shown in FIG. 35A , dies 3503 may be separated from each other by spaces that may be used to cut dies 3503 from wafer 3501 . For example, the dies 3503 may be separated from each other by spacers, wherein the width of the spacers may be selected to allow wafer dicing in the spacers.

At step 4017, die 3503 may be diced from wafer 3501 using any suitable method. In one embodiment, the die 3503 may be cut off using a laser. In one embodiment, wafer 3501 may be scribed first, followed by mechanical dicing. Alternatively, a mechanical scribing saw can be used. In some cases, an implicit stripe handler may be used. During dicing, once the die has been cut, the wafer 3501 can be mounted on a dicing tape for holding the die. In various embodiments, large cuts may be made, eg, as shown by cuts 3801 and 3803 in Figure 38A or cuts 3802, 3804, or 3806 in Figure 38B. Once dies 3503 have been cut individually or in groups, as shown, for example, by group 3504 in Figure 35C, dies 3503 can be packaged. Packaging of the die may include forming contacts to the die 3503 , depositing a protective layer over the contacts, attaching thermal management devices (eg, heat sinks), and encapsulating the die 3503 . In various embodiments, depending on how many dies are selected to form a memory chip, an appropriate configuration of contacts and busses may be used. In one embodiment, some of the contacts between the different dies forming the memory chip may be fabricated during packaging of the memory chip.

FIG. 41A shows one embodiment of a process 4100 for fabricating a memory chip containing multiple dies. Step 4011 of processing procedure 4100 may be the same as step 4011 of processing procedure 4000 . At step 4111 , as shown in FIG. 37 , glue logic 3711 may be disposed on wafer 3501 . Glue logic 3711 may be any suitable logic for controlling the operation of die 3506, as shown in FIG. As previously described, the presence of glue logic 3711 may allow multiple dies to function as a single memory chip. Glue logic 3711 may provide an interface to other memory controllers so that memory chips formed from multiple dies function as a single memory chip.

At step 4113 of process routine 4100 , buses (eg, input and output buses and control buses) may be placed on wafer 3501 . The bus may be arranged such that it connects with various dies and logic circuits such as glue logic 3711 . In some cases, a bus may connect memory cells. For example, a bus may be configured to connect processor subunits of different dies. At step 4115, the die, glue logic, and busses may be fabricated using any suitable method. For example, logic elements may be fabricated by etching wafer 3501, depositing various dielectric, metal or semiconductor layers, further etching the deposited layers, and the like. The bus can be fabricated using, for example, metal evaporation.

At step 4140, a cut shape may be used to cut groups of dies connected to a single glue logic 3711, as shown in, eg, FIG. 37 . The dicing shape may be determined using the memory requirements of a memory chip containing multiple dies 3503. For example, FIG. 41B shows handler 4101, which can be a variation of handler 4100, in which steps 4117 and 4119 can be placed before step 4140 of handler 4100. At step 4117, the system for dicing wafer 3501 may receive instructions describing the requirements for the memory chips. For example, requirements may include forming a memory chip including four dies 3503 . In some cases, at step 4119, the program software may determine a periodic pattern for a group of dies and glue logic 3711. For example, a periodic pattern may include two glue logic 3711 elements and four dies 3503, where every two dies are connected to one glue logic 3711. Alternatively, at step 4119, the pattern may be provided by the designer of the memory chip.

In some cases, the pattern may be selected to maximize the yield of memory chips formed from wafer 3501. In one embodiment, the memory cells of die 3503 can be tested to identify dies with failed memory cells (such dies are referred to as failed dies), and based on the location of the failed dies, can be identified that contain a passing test A group of dies 3503 of memory cells and an appropriate dicing pattern can be determined. For example, if a large number of dies 3503 fail at the edge of the wafer 3501 , a dicing pattern can be determined to avoid the dies at the edge of the wafer 3501 . Other steps of process 4101 such as steps 4011 , 4111 , 4113 , 4115 and 4140 may be the same as the same numbered steps of process 4100 .

41C shows an embodiment of a handler 4102 that may be a variation of handler 4101. Steps 4011, 4111, 4113, 4115, and 4140 of handler 4102 may be identical to the same numbered steps of handler 4101, step 4131 of handler 4102 may be substituted for step 4117 of handler 4101, and step 4133 of handler 4102 may be substituted for processing Step 4119 of procedure 4101. At step 4131, the system for dicing wafer 3501 may receive instructions describing requirements for the first set of memory chips and the second set of memory chips. For example, the requirements may include: forming a first set of memory chips having memory chips consisting of four dies 3503 ; and forming a second set of memory chips having memory chips consisting of two dies 3503 . In some cases, more than two sets of memory chips may need to be formed from wafer 3501 . For example, the third set of memory chips may include memory chips consisting of only one die 3503 . In some cases, at step 4133, the program software may determine a periodic pattern for a group of dies and glue logic 3711 for forming the memory chips of each memory chip set. For example, the first set of memory chips may include memory chips containing two glue logic 3711 and four dies 3503 , where every two dies are connected to one glue logic 3711 . In various embodiments, glue logic cells 3711 for the same memory chip can be linked together to function as a single glue logic. For example, during the manufacture of glue logic 3711, appropriate bus lines linking glue logic cells 3711 to each other may be formed.

The second set of memory chips may include memory chips containing one glue logic 3711 and two dies 3503 , wherein the die 3503 is connected to the glue logic 3711 . In some cases, when the third set of memory chips is selected and when the third set of memory chips includes memory chips composed of a single die 3503, these memory chips may not require glue logic 3711.

An important characteristic when designing a memory chip or a memory instance within a chip is the number of words that can be accessed simultaneously during a single clock cycle. For reading and/or writing, the more addresses that can be accessed simultaneously (eg, addresses along rows, also called words or word lines, and columns, also called bits or bit lines), the more quick. While there has been some activity in developing memories that include multiplexed ports that allow simultaneous access to multiple addresses, such as for building register files, cash, or shared memory, most instances use larger sizes and support multiple address to access the memory pad. However, DRAM chips typically include a single bit line and a single row line connected to each capacitor of each memory cell. Accordingly, embodiments of the present disclosure seek to provide multi-port access to existing DRAM chips without modifying this conventional single-port memory structure of the DRAM array.

Embodiments of the present disclosure may use memory to clock memory instances or chips at twice the speed of logic circuits. Any logic circuit using memory may thus "correspond" to memory and any of its components. Thus, embodiments of the present disclosure can fetch or write two addresses in two memory array clock cycles, which are equivalent to a single processing clock cycle for the logic circuit. The logic circuit may include circuitry such as a controller, accelerator, GPU, or CPU, or may include a processing group on the same substrate as the memory chip, eg, as depicted in Figure 7A. As explained above with respect to FIG. 3A, a "processing group" may refer to two or more processor sub-units and their corresponding memory groups on a substrate. The group may represent a spatial distribution and/or logical grouping on the substrate for the purpose of compiling code for execution on the memory chip 2800 . Thus, as described above with respect to FIG. 7A, a substrate with memory chips may include a memory array having multiple groups, such as group 2801a shown in FIG. 28 and others. In addition, the substrate may also include a processing array, which may include a plurality of processor subunits (such as subunits 730a, 730b, 730c, 730d, 730e, 730f, 730g, and 730h shown in Figure 7A).

Thus, embodiments of the present disclosure can retrieve data from the array in each of two consecutive memory cycles to handle two addresses for each logic cycle and provide two results to the logic, just like a single port memory array For dual-port memory chips in general. The additional clocking may allow the memory chips of the present disclosure to function as if the single-port memory array were a dual-port memory instance, a triple-port memory instance, a quad-port memory instance port, or any other multi-port memory instance.

42 depicts an embodiment of a circuit system 4200 that provides dual port access along a column of memory chips in which the circuit system 4200 is used consistent with the present disclosure. The embodiment depicted in Figure 42 may use one memory array 4201 with two column multiplexers ("mux") 4205a and 4205b to access two words on the same row during the same clock cycle for the logic circuit . For example, during a memory clock cycle, RowAddrA is used in row decoder 4203 and ColAddrA is used in multiplexer 4205a to buffer data from memory cells with address (RowAddrA, ColAddrA). During the same memory clock cycle, ColAddrB is used in the multiplexer 4205b to buffer data from the memory cell with address (RowAddrA, ColAddrB). Thus, circuitry 4200 may allow dual port access to data (eg, DataA and DataB) stored on memory cells at two different addresses along the same row or word line. Thus, two addresses can share a row such that the row decoder 4203 enables the same word line for both fetches. Furthermore, the embodiment depicted in Figure 42 may use a column multiplexer so that both addresses may be accessed during the same memory clock cycle.

Similarly, Figure 43 depicts one embodiment of a circuit system 4300 that provides dual port access along a row of memory chips in which the circuit system 4300 is used consistent with the present disclosure. The embodiment depicted in Figure 43 may use one memory with row decoders 4303 coupled with a multiplexer ("mux") to access two words on the same column during the same clock cycle for the logic circuit Array 4301. For example, on the first memory clock cycle of two memory clock cycles, RowAddrA is used in row decoder 4303 and ColAddrA is used in column multiplexer 4305 to buffer memory cells with address (RowAddrA, ColAddrA) data (eg, to the "buffer word" buffer of Figure 43). On the second memory clock cycle of the two memory clock cycles, RowAddrB is used in row decoder 4303 and ColAddrA is used in column multiplexer 4305 to buffer data from the memory cell with address (RowAddrB, ColAddrA). Thus, circuitry 4300 may allow dual-port access to data (eg, DataA and DataB) stored on memory cells at two different addresses along the same column or bit line. Thus, two addresses may share a row such that a column decoder (which may be separate or combined with one or more column multiplexers, as depicted in Figure 43) enables the same bit line for both fetches. The embodiment depicted in FIG. 43 may use two memory clock cycles, since row decoder 4303 may require one memory clock cycle to activate each word line. Thus, a memory chip using circuitry 4300 can function as a dual port memory if clocked at least twice as fast as the corresponding logic circuit.

Thus, as explained above, Figure 43 may retrieve DataA and DataB during two memory clock cycles faster than the clock cycle for the corresponding logic circuit. For example, a row decoder (eg, row decoder 4303 of FIG. 43 ) and a column decoder (which may be separate or combined with one or more column multiplexers, as depicted in FIG. 43 ) may be configured in at least two Clocking is performed at a rate that is twice the rate at which the corresponding logic circuit generates the two addresses. For example, a clock circuit (not shown in FIG. 43 ) for the circuit system 4300 may clock the circuit system 4300 at a rate that is at least twice the rate at which the corresponding logic circuit generates the two addresses.

The embodiment of Figure 42 and the embodiment of Figure 43 may be used separately or in combination. Thus, circuitry that provides dual-port functionality on a single-port memory array or pad (eg, circuitry 4200 or 4300) may include multiple memory banks arranged along at least one row and at least one column. The plurality of memory banks are depicted as memory array 4201 in FIG. 42 and as memory array 4301 in FIG. 43 . This embodiment may further use at least one row multiplexer (as depicted in Figure 43) or at least one column multiplexer (as depicted in Figure 43) configured to receive two addresses for reading or writing during a single clock cycle depicted in Figure 42). Furthermore, this embodiment may use row decoders (eg, row decoder 4203 of FIG. 42 and row decoder 4303 of FIG. 43 ) and column decoders (which may be separate or combined with one or more column multiplexers, such as 42 and 43) to read from or write to two addresses. For example, the row decoder and the column decoder may retrieve a first of the two addresses from the at least one row multiplexer or the at least one column multiplexer during a first cycle and decode the word line corresponding to the first address and bit lines. Additionally, the row decoder and the column decoder may retrieve a second one of the two addresses from the at least one row multiplexer or the at least one column multiplexer during the second cycle and decode the word line corresponding to the second address and bit lines. The retrieval may each include enabling a word line corresponding to the address using a row decoder and enabling a bit line corresponding to the address on the enabled word line using a column decoder.

Although described above with respect to retrieval, the embodiments of Figures 42 and 43, whether implemented separately or in combination, may include write commands. For example, during the first cycle, the row decoder and the column decoder may write the first data retrieved from the at least one row multiplexer or the at least one column multiplexer to a first address of the two addresses. For example, during the second cycle, the row decoder and the column decoder may write the second data retrieved from the at least one row multiplexer or the at least one column multiplexer to a second address of the two addresses.

FIG. 42 shows an embodiment of this processing procedure when the first address and the second address share a word line address, and FIG. 43 shows an embodiment of this processing procedure when the first address and the second address share a row address. As described further below with respect to FIG. 47, the same processing procedure may be implemented when the first address and the second address do not share either word line addresses or row addresses.

Thus, while the above embodiments provide dual port access along at least one of rows or columns, additional embodiments may provide dual port access along both rows and columns. 44 depicts an embodiment of a circuit system 4400 that provides dual port access along both rows and columns of a memory chip in which the circuit system 4400 is used consistent with the present disclosure. Thus, circuitry 4700 may represent a combination of circuitry 4200 of FIG. 42 and circuitry 4300 of FIG. 43 .

The embodiment depicted in FIG. 44 may use one memory array 4401 with row decoders 4403 coupled to a multiplexer ("mux") to access two rows during the same clock cycle for the logic circuit. Furthermore, the embodiment depicted in Figure 44 may use a memory array 4401 having a column decoder (or multiplexer) 4405 coupled with a multiplexer ("mux") to access both columns during the same clock cycle. For example, on the first memory clock cycle of two memory clock cycles, RowAddrA is used in row decoder 4403 and ColAddrA is used in column multiplexer 4405 to buffer memory cells with address (RowAddrA, ColAddrA) data (eg, to the "buffer word" buffer of Figure 44). On the second memory clock cycle of the two memory clock cycles, RowAddrB is used in row decoder 4403 and ColAddrB is used in column multiplexer 4405 to buffer data from the memory cell with address (RowAddrB, ColAddrB). Thus, circuitry 4400 may allow dual port access to data (eg, DataA and DataB) stored on memory cells at two different addresses. Embodiments as depicted in Figure 44 may use additional buffers because row decoder 4403 may require one memory clock cycle to activate each word line. Thus, a memory chip using circuitry 4400 can function as a dual port memory if clocked at least twice as fast as the corresponding logic circuit.

Although not depicted in Figure 44, the circuitry 4400 may also include the additional circuitry of Figure 46 (described further below) along rows or word lines and/or similar additional circuitry along columns or bit lines. Accordingly, the circuitry 4400 may activate the corresponding circuitry (eg, by opening one or more switching elements, such as one or more of the switching elements 4613a, 4613b of FIG. 46, and the like) to initiate the opening including the address section (eg, by connecting a voltage or allowing current to flow to a disconnected section). Thus, when an element of a circuit system (such as a wire or the like) includes a location identifying an address and/or when an element of the circuit system (such as a switch element) controls the supply or voltage to the memory cell identified by the address and / or current, the circuit system can "correspond". Circuitry 4400 may then use row decoder 4403 and column multiplexer 4405 to decode corresponding word lines and bit lines to retrieve data from or write data to addresses located in the activated disconnected portion.

As further depicted in FIG. 44, circuitry 4400 may further use at least one row multiplexer (depicted as separate from row decoder 4403) configured to receive two addresses for reading or writing during a single clock cycle , but may be incorporated therein) and/or at least one column multiplexer (eg, depicted as separate from, but may be incorporated into, column multiplexer 4405). Thus, embodiments may use row decoders (eg, row decoder 4403) and column decoders (which may be separate or combined with column multiplexer 4405) to read from or write to two addresses. For example, the row and column decoders may retrieve a first of two addresses from at least one row multiplexer or at least one column multiplexer during a memory clock cycle and decode the word line corresponding to the first address and bit lines. Furthermore, the row decoder and the column decoder can retrieve a second one of the two addresses from the at least one row multiplexer or the at least one column multiplexer during the same memory cycle and decode the word line corresponding to the second address and bit lines.

45A and 45B depict existing replication techniques for providing dual port functionality on a single port memory array or pad. As shown in Figure 45A, dual port reads can be provided by keeping copies of data synchronized across a memory array or pad. Thus, reads can be performed from both replicas of the memory instance, as depicted in Figure 45A. Furthermore, as shown in Figure 45B, dual port writes can be provided by duplicating all writes across the memory array or pad. For example, a memory chip may need to use the logic circuitry of the memory chip to repeatedly send write commands, one for each copy of the data. Alternatively, in some embodiments, as shown in FIG. 45A, additional circuitry may allow logic using the memory instance to send a single write command that is automatically replicated by the additional circuitry to span the memory array or pad. Make a replica of the written data so that the replicas are kept in sync. The embodiments of Figures 42, 43, and 44 may access two bit lines in a single memory clock cycle (eg, as depicted in Figure 42) by using a multiplexer and/or by faster than corresponding logic circuits Clocking the memory (eg, as depicted in Figures 43 and 44) and providing additional multiplexers to handle additional addresses rather than copying all data in the memory reduces redundancy from these existing replication techniques.

In addition to the faster clocking and/or additional multiplexers described above, embodiments of the present disclosure may also use circuitry that disconnects bit lines and/or word lines at points within the memory array. These embodiments may allow multiple simultaneous accesses to the array as long as the row and column decoder accesses are not coupled to different locations disconnecting the same portion of the circuitry. For example, locations with different word lines and bit lines can be accessed simultaneously because disconnecting the circuitry allows row and column decoding to access different addresses without electrical interference. During the design of a memory chip, the granularity of the disconnected regions within the memory array can be weighed against the additional area required to disconnect the circuitry.

The architecture for implementing this simultaneous access is depicted in FIG. 46 . Specifically, Figure 46 depicts an embodiment of a circuit system 4600 that provides dual port functionality on a single port memory array or pad. As depicted in FIG. 46, circuitry 4600 may include a plurality of memory pads (eg, memory pad 4609a, pad 4609b, and the like) arranged along at least one row and at least one column. The layout of circuitry 4600 also includes a plurality of word lines, such as word lines 4611a and 4611b corresponding to rows, and bit lines 4615a and 4615b corresponding to columns.

The embodiment depicted in Figure 46 includes twelve memory pads, each memory pad having two lines and eight columns. In other embodiments, the substrate may include any number of memory pads, and each memory pad may include any number of lines and any number of columns. Some memory pads may include the same number of lines and columns (as shown in Figure 46), while other memory pads may include different numbers of lines and/or columns.

Although not depicted in Figure 46, circuitry 4600 may further use at least one row multiplexing configured to receive two (or three or any more) addresses for reading or writing during a single clock cycle or at least one column multiplexer (eg, column multiplexers 4603a and/or 4603b). Furthermore, embodiments may use row decoders (eg, row decoders 4601a and/or 4601b) and column decoders (which may be separate or combined with column multiplexers 4603a and/or 4603b) to extract data from two (or more than two) addresses are read or written to two (or more than two) addresses. For example, the row and column decoders may retrieve a first of two addresses from at least one row multiplexer or at least one column multiplexer during a memory clock cycle and decode the word line corresponding to the first address and bit lines. Furthermore, the row decoder and the column decoder can retrieve a second one of the two addresses from the at least one row multiplexer or the at least one column multiplexer during the same memory cycle and decode the word line corresponding to the second address and bit lines. As explained above, the two addresses can be performed during the same memory clock cycle as long as the two addresses are in different locations that are not coupled to the same portion of disconnected circuitry (eg, switching elements such as 4613a, 4613b, and the like) access. Additionally, the circuitry 4600 may concurrently access the first two addresses during the first memory clock cycle, and then concurrently access the next two addresses during the second memory clock cycle. In these embodiments, a memory chip using circuitry 4600 can function as a quad-port memory if clocked at least twice as fast as the corresponding logic circuit.

Figure 46 also includes at least one row circuit and at least one column circuit configured to function as switches. For example, corresponding switching elements such as 4613a, 4613b and the like may comprise transistors or any other electrical element configured to allow or stop current flow and/or connect or disconnect voltages and connections such as 4613a, 4613b and the like word line or bit line of the switching element of the object. Accordingly, corresponding switching elements may divide the circuit system 4600 into disconnected portions. Although depicted as including a single row with each row including sixteen columns, the disconnected regions within circuitry 4600 may include different levels of granularity depending on the design of circuitry 4600 .

Circuitry 4600 can use a controller (eg, row controls 4607) to enable corresponding ones of at least one row circuit and at least one column circuit to enable corresponding open regions during the address operations described above. For example, circuitry 4600 may transmit one or more control signals to close corresponding ones of switching elements (eg, switching elements 4613a, 4613b, and the like). In embodiments where switching elements 4613a, 4613b, and the like, comprise transistors, the control signal may comprise a voltage that turns off the transistors.

Depending on the open region including the address, more than one of the switching elements may be activated by the circuitry 4600 . For example, to reach an address within memory pad 4609b of Figure 46, the switching elements that allow access to memory pad 4609a and the switching elements that allow access to memory pad 4609b must be turned off. Row controls 4607 can determine which switching elements to activate in order to retrieve specific addresses within circuitry 4600 based on specific addresses.

46 shows an embodiment of circuitry 4600 for dividing word lines of a memory array (eg, including memory pad 4609a, pad 4609b, and the like). However, other embodiments may use similar circuitry (eg, switching elements that divide the memory chip 4600 into disconnected regions) to divide the bit lines of the memory array. Thus, the architecture of the circuitry 4600 can be used in dual column access (as depicted in FIG. 42 or FIG. 44) as well as dual row access (as depicted in FIG. 43 or FIG. 44).

The processing procedure for multi-cycle access to a memory array or pad is depicted in Figure 47A. Specifically, FIG. 47A is a flowchart of an embodiment of a process 4700 for providing dual-port access on a single-port memory array or pad (eg, using the circuitry 4300 of FIG. 43 or the circuitry 4400 of FIG. 44 ), Process 4700 may be performed using row and column decoders consistent with the present disclosure, such as row decoders 4303 or 4403 of FIG. 43 or FIG. 44, respectively, and column decoders (which may be separate from one or more column multiplexers) or a combination, such as column multiplexers 4305 or 4405 depicted in Figure 43 or Figure 44, respectively).

At step 4710, during a first memory clock cycle, the circuitry may use at least one row multiplexer and at least one column multiplexer to decode word lines and bit lines corresponding to a first of the two addresses. For example, at least one row decoder may enable word lines, and at least one column multiplexer may amplify voltages from memory cells along the enabled word lines and corresponding to the first address. The amplified voltage may be provided to a logic circuit using a memory chip including this circuitry, or buffered according to step 4720 described below. The logic circuit may include circuitry such as a GPU or CPU, or may include a processing group on the same substrate as the memory chip, eg, as depicted in Figure 7A.

Although described above as read operations, method 4700 may similarly handle write operations. For example, at least one row decoder can activate a word line, and at least one column multiplexer can apply a voltage to a memory cell along the activated word line and corresponding to a first address to write new data to the memory cells. In some embodiments, the circuitry may provide an acknowledgement of the write to logic circuitry using a memory chip that includes the circuitry, or buffer the acknowledgement according to step 4720 below.

At step 4720, the circuitry may buffer the fetched data for the first address. For example, as depicted in Figures 43 and 44, the buffer may allow the circuitry to fetch the second of the two addresses (as described below in step 4730) and return the results of both fetches back together. The buffers may include registers, SRAM, non-volatile memory, or any other data storage device.

At step 4730, during a second memory clock cycle, the circuitry may use at least one row multiplexer and at least one column multiplexer to decode word lines and bit lines corresponding to a second of the two addresses. For example, at least one row decoder may activate a word line, and at least one column multiplexer may amplify voltages from memory cells along the activated word line and corresponding to the second address. The amplified voltage may be provided to logic circuits using the memory chip including the circuitry, either individually or together with the buffered voltage from step 4720, for example. The logic circuit may include circuitry such as a GPU or CPU, or may include a processing group on the same substrate as the memory chip, eg, as depicted in Figure 7A.

Although described above as read operations, method 4700 may similarly handle write operations. For example, at least one row decoder can activate a word line, and at least one column multiplexer can apply voltages to memory cells along the activated word line and corresponding to the second address to write new data to the memory cells. In some embodiments, the circuitry may provide an acknowledgement of the write to logic circuitry using the memory chip that includes the circuitry, either individually or in conjunction with buffered voltages from step 4720, for example.

At step 4740, the circuitry may output the second address and the fetched data of the buffered first address. For example, as depicted in Figures 43 and 44, the circuitry may return the results of the two retrievals (eg, from steps 4710 and 4730) together. The circuitry can communicate results back to logic using the memory chip that includes the circuitry, which can include circuitry such as a GPU or CPU, or can include a processing group on the same substrate as the memory chip, eg, As depicted in Figure 7A.

Although described with reference to multiple cycles, if two addresses share a word line, as depicted in FIG. 42, method 4700 may allow a single cycle access to both addresses. For example, steps 4710 and 4730 may be performed during the same memory clock cycle because multiple column multiplexers may decode different bit lines on the same word line during the same memory clock cycle. In these embodiments, buffering step 4720 may be skipped.

The processing procedure for simultaneous access (eg, using the circuitry 4600 described above) is depicted in Figure 47B. Thus, although shown sequentially, the steps of Figure 47B may all be performed during the same memory clock cycle, and at least some of the steps (eg, steps 4760 and 4780 or steps 4770 and 4790) may be performed concurrently. Specifically, Figure 47B is a flowchart of an embodiment of a process 4750 for providing dual-port access on a single-port memory array or pad (eg, using the circuitry 4200 of Figure 42 or the circuitry 4600 of Figure 46), Process 4750 may be performed using row and column decoders consistent with the present disclosure, such as row decoder 4203 or row decoders 4601a and 4601b of FIG. 42 or FIG. 46, respectively, and column decoders (which may be combined with one or more Column multiplexers are separated or combined, such as column multiplexers 4205a and 4205b or column multiplexers 4603a and 4306b depicted in Figure 42 or Figure 46, respectively).

At step 4760, during a memory clock cycle, the circuitry may enable corresponding ones of the at least one row circuit and the at least one column circuit based on the first of the two addresses. For example, the circuitry may transmit one or more control signals to close corresponding ones of switching elements including at least one row circuit and at least one column circuit. Thus, the circuitry can access the corresponding open region including the first of the two addresses.

At step 4770, during the memory clock cycle, the circuitry may use at least one row multiplexer and at least one column multiplexer to decode word lines and bit lines corresponding to the first address. For example, at least one row decoder may enable word lines, and at least one column multiplexer may amplify voltages from memory cells along the enabled word lines and corresponding to the first address. The amplified voltage may be provided to logic circuits using memory chips including the circuitry. For example, as described above, the logic circuit may include circuitry such as a GPU or CPU, or may include a processing group on the same substrate as the memory chip, eg, as depicted in Figure 7A.

Although described above as read operations, method 4500 may similarly handle write operations. For example, at least one row decoder can activate a word line, and at least one column multiplexer can apply a voltage to a memory cell along the activated word line and corresponding to a first address to write new data to the memory cells. In some embodiments, the circuitry may provide an acknowledgement of the write to logic circuitry using a memory chip that includes the circuitry.

At step 4780, during the same cycle, the circuitry may enable corresponding ones of the at least one row circuit and the at least one column circuit based on the second of the two addresses. For example, the circuitry may transmit one or more control signals to close corresponding ones of switching elements including at least one row circuit and at least one column circuit. Thus, the circuitry can access the corresponding open region including the second of the two addresses.

At step 4790, during the same cycle, the circuitry may use at least one row multiplexer and at least one column multiplexer to decode word lines and bit lines corresponding to the second address. For example, at least one row decoder may activate a word line, and at least one column multiplexer may amplify voltages from memory cells along the activated word line and corresponding to the second address. The amplified voltage may be provided to logic circuits using memory chips including the circuitry. For example, as described above, the logic circuit may comprise conventional circuitry such as a GPU or CPU, or may comprise a processing group on the same substrate as the memory chip, eg, as depicted in Figure 7A.

Although described above as read operations, method 4500 may similarly handle write operations. For example, at least one row decoder can activate a word line, and at least one column multiplexer can apply voltages to memory cells along the activated word line and corresponding to the second address to write new data to the memory cells. In some embodiments, the circuitry may provide an acknowledgement of the write to logic circuitry using a memory chip that includes the circuitry.

Although described with reference to a single cycle, method 4500 may allow if two addresses are in open regions that share word lines or bit lines (or otherwise share switching elements in at least one row circuit and at least one column circuit). Multiple round-robin access to two addresses. For example, steps 4760 and 4770 may be performed during a first memory clock cycle in which a first row decoder and a first column multiplexer may decode word lines and bit lines corresponding to a first address , and steps 4780 and 4790 may be performed during a second memory clock cycle in which the second row decoder and second column multiplexer may decode word lines and bit lines corresponding to the second address .

Another embodiment of an architecture for dual port access along both rows and columns is depicted in FIG. 48 . Specifically, Figure 48 depicts one embodiment of a circuit system 4800 that provides dual port access along both rows and columns using multiple row decoders in conjunction with multiple column multiplexers. In Figure 48, row decoder 4801a can access the first word line, and column multiplexer 4803a can decode data from one or more memory cells along the first word line, while row decoder 4801b can store A second word line is taken, and column multiplexer 4803b can decode data from one or more memory cells along the second word line.

As described with respect to Figure 47B, this access may be concurrent during one memory clock cycle. Thus, similar to the architecture of FIG. 46, the architecture of FIG. 48 (including the memory pad described below in FIG. 49) may allow multiple addresses to be accessed in the same clock cycle. For example, the architecture of Figure 48 may include any number of row decoders and any number of column multiplexers such that a number of addresses corresponding to the number of row decoders and column multiplexers can all be accessed within a single memory clock cycle.

In other embodiments, this access may occur sequentially along two memory clock cycles. By clocking the memory chip 4800 faster than the corresponding logic circuit, two memory clock cycles may be equivalent to one clock cycle of the logic circuit using the memory. For example, as described above, the logic circuit may comprise conventional circuitry such as a GPU or CPU, or may comprise a processing group on the same substrate as the memory chip, eg, as depicted in Figure 7A.

Other embodiments may allow simultaneous access. For example, as described with respect to FIG. 42, multiple column decoders (which may include column multiplexers, such as 4803a and 4803b, as shown in FIG. 48) may read data along the same word line during a single memory clock cycle Multiple bit lines. Additionally or alternatively, as described with respect to FIG. 46, the circuitry 4800 may incorporate additional circuitry such that this access may be simultaneous. For example, row decoder 4801a can access the first word line, and column multiplexer 4803a can decode data from memory cells along the first word line during the same memory clock cycle in which the row Decoder 4801b accesses the second word line, and column multiplexer 4803b decodes data from memory cells along the second word line.

The architecture of FIG. 48 can be used with modified memory pads forming memory banks, as shown in FIG. 49 . In Figure 49, each memory cell is accessed by two word lines and two bit lines (depicted as capacitors similar to DRAM, but may also include data arranged in a manner similar to SRAM or any other memory cell transistors). Thus, the memory pad 4900 of Figure 49 allows simultaneous access to two different bits, or even the same bit, by two different logic circuits. However, the embodiment of Figure 49 uses modifications to the memory pads rather than implementing a dual port solution on standard DRAM memory pads that are wired for single port access as in the above embodiments.

Although described as having two ports, any of the embodiments described above may be extended to more than two ports. For example, the embodiments of Figures 42, 46, 48, and 49 may include additional column multiplexers or row multiplexers, respectively, to provide access to additional columns or rows during a single clock cycle. As another example, the embodiments of Figures 43 and 44 may include additional row decoders and/or column multiplexers to provide access to additional rows or columns, respectively, during a single clock cycle.

Variable Word Length Access in Memory

As used above and further below, the term "coupled" may include direct connections, indirect connections, electrical communication, and the like.

Furthermore, terms such as "first," "second," and the like, are used to distinguish between elements or method steps having the same or similar names or headings, and do not necessarily indicate a spatial or temporal order.

Typically, a memory chip may include a memory bank. The memory banks may be coupled to row and column decoders that are configured to select particular words (or other fixed-size units of data) to be read or written. Each memory bank may include memory cells to store data cells, to amplify voltages from memory cells selected by row and column decoders, and any other suitable circuitry.

Each memory bank usually has a specific I/O width. For example, the I/O width can contain words.

While some processing routines performed by logic circuits using memory chips may benefit from the use of extremely long words, some other processing routines may require only a portion of the word.

In practice, in-memory computing units, such as processor sub-units disposed on the same substrate as memory chips, eg, as depicted and described in Figure 7A, frequently perform memory access operations that require only a portion of the word.

To reduce the latency associated with accessing an entire word when only a portion is used, embodiments of the present disclosure may provide methods and systems for fetching only one or more portions of a word, thereby reducing the time required to transmit the word. Part of the associated data loss is not required and power saving in the memory device is allowed.

Furthermore, embodiments of the present disclosure may also reduce memory chips from other entities, such as logic circuits, whether separate, such as CPUs and GPUs, or included on the same substrate as memory chips, such as that depicted and described in FIG. 7A . The power consumption of the interaction between the processor sub-unit), the other entity accessing the memory chip, which may only receive or write part of the word.

A memory access command (eg, from a logic circuit using the memory) may include an address in the memory. For example, the address may include a column address and a row address, or may be transferred to a column address and a row address, such as by a memory controller of the memory.

In many volatile memories such as DRAM, the column address is sent (eg, directly by logic circuitry or using a memory controller) to a row decoder, which activates an entire row (also called a word line) and loads the All bit lines included in the row.

The row address identifies the bit line on the enabled row, which is forwarded outside the memory bank that includes the bit line and passed to the next level of circuitry. For example, the next level of circuitry may include the I/O bus of the memory chip. In embodiments using in-memory processing, the next level of circuitry may include the processor subunit of the memory chip (eg, as depicted in Figure 7A).

Accordingly, the memory chip described below may be included in any of FIGS. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22, or 23. One of the memory chips described in, or otherwise included in, the memory chips.

The memory chip may be fabricated with a first fabrication process optimized for memory cells rather than logic cells. For example, memory cells fabricated by the first fabrication process may exhibit critical dimensions that are smaller than the critical dimensions of logic circuits fabricated by the first fabrication process (eg, more than 2 times, 3 times, 4 times, 5 times smaller, 6 times, 7 times, 8 times, 9 times, 10 times and the like). For example, the first fabrication process may include an analog fabrication process, a DRAM fabrication process, and the like.

This memory chip may include integrated circuits, which may include memory cells. The memory cells may include memory cells, output ports, and read circuitry. In some embodiments, the memory unit may also include a processing unit, such as a processor sub-unit as described above.

For example, the read circuitry may include a reduction unit and a first group of in-memory read paths for outputting up to a first number of bits via an output port. The output port may be connected to off-chip logic (such as an accelerator, CPU, GPU, or the like) or to an on-chip processor subunit, as described above.

In some embodiments, the processing unit may comprise a reduction unit, may be part of a reduction unit, may be different from a reduction unit, or may otherwise include a reduction unit.

In-memory read paths may be included in integrated circuits (eg, may be in memory cells), and may include any circuits and/or chains configured to read from and/or write to memory cells road. For example, in-memory read paths may include sense amplifiers, conductors coupled to memory cells, multiplexers, and the like.

The processing unit may be configured to send a read request to the memory unit to read a second number of bits from the memory unit. Additionally or alternatively, the read request may originate from off-chip logic (such as an accelerator, CPU, GPU, or the like).

The reduction unit may be configured to assist in reducing power consumption associated with access requests, eg, by using any of the partial word accesses described herein.

The reduction unit may be configured to control the in-memory read path based on the first number of bits and the second number of bits during a read operation triggered by the read request. For example, the control signal from the reduction unit may affect the memory consumption of the read paths to reduce the energy consumption of the memory read paths unrelated to the requested second number of bits. For example, the reduction unit may be configured to control unrelated memory read paths when the second number is less than the first number.

As explained above, the integrated circuit may be included in any of FIGS. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22, or 23. Among the memory chips described in one, the memory chip may be included or otherwise included.

The uncorrelated in-memory read paths may be related to uncorrelated bits of the first number of bits, such as bits of the first number of bits that are not included in the second number of bits.

50 illustrates an integrated circuit 5000 in an embodiment of the present disclosure, which includes: memory cells 5001-5008 in a memory cell array 5050; an output port 5020, which includes bits 5021-5028; read circuitry 5040, which Including memory read paths 5011 to 5018; and reduction unit 5030.

When the second number of bits are read using the corresponding memory read path, irrelevant bits in the first number of bits may correspond to bits that should not be read (eg, bits not included in the second number of bits) ).

During a read operation, the reduction unit 5030 may be configured to initiate memory read paths corresponding to the second number of bits, such that the initiated memory read paths may be configured to deliver the second number of bits. In these embodiments, only memory read paths corresponding to the second number of bits may be enabled.

During a read operation, the reduction unit 5030 can be configured to cut off at least a portion of each unrelated memory read path. For example, uncorrelated memory read paths may correspond to uncorrelated bits of the first number of bits.

It should be noted that instead of cutting off at least a portion of the irrelevant memory paths, the reduction unit 5030 may instead ensure that the irrelevant memory paths are not enabled.

Additionally or alternatively, reduce unit 5030 may be configured to maintain unrelated memory read paths in a low power mode during read operations. For example, a low power mode may include supplying voltages or currents, respectively, to unrelated memory paths that are lower than normal operating voltages or currents.

The reduction unit 5030 may be further configured to control the bit lines of the unrelated memory read paths.

Thus, the reduction unit 5030 may be configured to load the bit lines of the relevant memory read paths and maintain the bit lines of the unrelated memory read paths in a low power mode. For example, only the bit lines of the associated memory read path may be loaded.

Additionally or alternatively, the reduction unit 5030 may be configured to load the bit lines of the relevant memory read paths while maintaining the bit lines of the unrelated memory read paths as deactivated.

In some embodiments, the reduction unit 5030 may be configured to utilize portions of the memory read paths that are associated during read operations, and maintain a portion of each unassociated memory read path in a low power mode, where the Parts differ from bit lines.

As explained above, memory chips may use sense amplifiers to amplify voltages from memory cells included in the memory chips. Accordingly, the reduction unit 5030 can be configured to utilize portions of the memory read paths that are relevant during a read operation and maintain sense amplifiers associated with at least some of the unrelated memory read paths low in power mode.

In these embodiments, the reduction unit 5030 may be configured to utilize a portion of the memory read paths that are relevant during a read operation, and maintain one or more sense amplifiers associated with all unrelated memory read paths in low power mode.

Additionally or alternatively, the reduction unit 5030 may be configured to utilize portions of the associated memory read paths during read operations, and will be after one or more sense amplifiers associated with the unassociated memory read paths ( For example, portions of memory read paths that are spatially and/or temporally unrelated are maintained in a low power mode.

In any of the embodiments described above, the memory unit may comprise a column multiplexer (not shown).

In these embodiments, the reduction unit 5030 may be coupled between the column multiplexer and the output port.

Additionally or alternatively, the reduction unit 5030 may be embedded in a column multiplexer.

Additionally or alternatively, the reduction unit 5030 may be coupled between the memory cells and the column multiplexer.

Reduction unit 5030 may include reduction subunits that may be independently controllable. For example, different reduced subcells may be associated with different columns of memory cells.

Although described above with respect to read operations and read circuitry, the above embodiments may be similarly applied to write operations and write circuitry.

For example, integrated circuits in accordance with the present disclosure may include memory cells that include memory cells, output ports, and write circuitry. In some embodiments, the memory unit may also include a processing unit, such as a processor sub-unit as described above. The write circuitry may include a reduction unit and a first group of in-memory write paths for outputting up to a first number of bits via an output port. The processing unit may be configured to send a write request to the memory unit to write the second number of bits from the memory unit. Additionally or alternatively, the write request may originate from off-chip logic (such as an accelerator, CPU, GPU, or the like). Reduction unit 5030 may be configured to control the memory write path based on the first number of bits and the second number of bits during a write operation triggered by the write request.

51 illustrates a memory bank 5100 that includes memory cells addressed using column addresses and row addresses (eg, from an on-chip processor sub-unit or off-chip logic such as accelerators, CPUs, GPUs, or the like) Array 5111. As shown in Figure 51, memory cells are fed to bit lines (vertical) and word lines (horizontal, many word lines omitted for simplicity). Additionally, row decoders 5112 may be fed with column addresses (eg, from an on-chip processor subunit, off-chip logic, or a memory controller not shown in Figure 51), and column multiplexers 5113 may be fed with row addresses (eg, from an on-chip processor sub-unit, off-chip logic, or a memory controller not shown in FIG. 51), and the column multiplexer 5113 can receive, via the output bus 5115, outputs from up to an entire line and up to a word output. In FIG. 51, the output bus 5115 of the column multiplexer 5113 is coupled to the main I/O bus 5114. In other embodiments, output bus 5115 may be coupled to a processor subunit of a memory chip (eg, as depicted in Figure 7A) that sends column and row addresses. For simplicity, the grouping of memory banks into memory pads is not shown.

FIG. 52 illustrates memory bank 5101. In FIG. 52, the memory bank is also illustrated as including processing in memory (PIM) logic 5116 having an input coupled to an output bus 5115. PIM logic 5116 may generate addresses (eg, including column and row addresses) and output the addresses via PIM address bus 5118 to access memory banks. PIM logic 5116 is an embodiment of a reduced unit (eg, unit 5030) that also includes processing units. PIM logic 5016 may control other circuits not shown in FIG. 52 that assist in power reduction. PIM logic 5116 may further control the memory paths of the memory cells including memory bank 5101 .

As explained above, in some cases the word length (eg, the number of bit lines selected for a transfer) may be large.

Under these conditions, each word used for reading and/or writing may be associated with a memory path that may consume power during various phases of the read and/or write operation, such as:

a. Loading the bit line - in order to load the bit line to the desired value (from the capacitor on the bit line in a read cycle, or a new value to be written to the capacitor in a write cycle), the memory on the memory needs to be disabled sense amplifiers at the ends of the array and ensure that the capacitors holding the data do not discharge or charge (otherwise the data stored on them would be destroyed); and

b. Move the data from the sense amps to the rest of the chip via the column multiplexer that selects the bit line (to the I/O bus that carries the data in and out of the chip or to the embedded logic that will use the data , such as a processor subunit on the same substrate as the memory).

To achieve power savings, the integrated circuits of the present disclosure may determine some portion of a word to be irrelevant at row enable time and then send a disable signal to one or more sense amplifiers for the irrelevant portion of the word .

53 illustrates a memory cell 5102 that includes a memory cell array 5111, a row decoder 5112, a column multiplexer 5113 coupled to an output bus 5115, and PIM logic 5116.

The memory cell 5102 also includes a switch 5201 that enables or disables the passage of bits to the column multiplexer 5113 . Switch 5201 may include an analog switch, a transistor configured to function as a switch, configured to control the supply or voltage and/or current flow to portions of memory cell 5102 . A sense amplifier (not shown) may be located at the end of the memory cell array, eg, before switch 5201 (in space and/or in time).

Switch 5201 may be controlled by an enable signal sent from PIM logic 5116 via bus 5117. When open, the switch is configured to disconnect the sense amplifier (not shown) of the memory cell 5102, and thus not discharge or charge the bit line disconnected from the sense amplifier.

Switch 5201 and PIM logic 5116 may form a reduction unit (eg, reduction unit 5030).

In yet another embodiment, the PIM logic 5116 may send the enable signal to the sense amplifier (eg, when the sense amplifier has an enable input) instead of the switch 5201 .

The bit line may additionally or alternatively be broken at other points, eg, not at the end of the bit line and after the sense amplifier. For example, the bit lines can be broken before entering the array 5111.

In these embodiments, power may also be saved on data transfers from sense amplifiers and forwarding hardware, such as output bus 5115.

Other embodiments, which may save less power, but may be easier to implement, focus on saving the power of the column multiplexer 5113 and shifting the losses from the column multiplexer 5113 to the next level of circuitry. For example, as explained above, the next level of circuitry may include the memory chip's I/O bus (such as bus 5115). In embodiments using in-memory processing, the next level of circuitry may additionally or alternatively include a processor sub-unit of the memory chip (such as PIM logic 5116).

54A illustrates column multiplexer 5113 segmented into segments 5202. Each segment 5202 of the column multiplexer 5113 can be individually enabled or disabled by enable and/or disable signals sent from the PIM logic 5116 via the bus 5119. The column multiplexer 5113 may also be fed by the address column bus 5118.

The embodiment of FIG. 54A may provide better control over different portions of the output from the column multiplexer 5113.

It should be noted that control of different memory paths may have different resolutions, eg ranging from one-bit resolution to multi-bit resolution. The former may be more efficient in the sense of power saving. The latter implementation may be simpler and require fewer control signals.

54B illustrates an embodiment of method 5130. For example, method 5130 may be implemented using any of the memory cells described above with respect to Figures 50, 51, 52, 53, or 54A.

Method 5130 may include steps 5132 and 5134.

Step 5132 may include sending, through a processing unit of the integrated circuit (eg, PIM logic 5116), an access request to a memory cell of the integrated circuit to read the second number of bits from the memory cell. The memory cells may include memory cells (eg, memory cells of array 5111), output ports (eg, output bus 5115), and read/write circuitry, which may include reduced cells (eg, reduction unit 5030) and a first group of memory read/write paths for outputting and/or inputting up to a first number of bits via output ports.

Access requests may include read requests and/or write requests.

Memory input/output paths may include memory read paths, memory write paths, and/or paths for both read and write.

Step 5134 may include responding to the access request.

For example, step 5134 may include controlling a memory read/write path based on the first number of bits and the second number of bits by a reduction unit (eg, unit 5030) during an access operation triggered by the access request.

Step 5134 may also include any of the following operations and/or any combination of any of the following operations. Any of the operations listed below may be performed during responding to an access request, but may also be performed before and/or after responding to an access request.

Accordingly, step 5134 may include at least one of the following operations:

a. controlling uncorrelated memory read paths when the second number is less than the first number, wherein the uncorrelated memory read paths are associated with bits of the first number of bits that are not included in the second number of bits;

b. Initiating an associated memory read path during a read operation, wherein the associated memory read path is configured to deliver the second number of bits;

c. severing at least a portion of each of the unrelated memory read paths during a read operation;

d. maintaining unrelated memory read paths in a low power mode during read operations;

e. Bit lines that control unrelated memory read paths;

f. Load the bit lines of the relevant memory read paths and maintain the bit lines of the unrelated memory read paths in a low power mode;

g. Load the bit lines of the relevant memory read paths while maintaining the bit lines of the unrelated memory read paths as deactivated;

h. Utilize a portion of an associated memory read path during a read operation and maintain a portion of each unassociated memory read path in a low power mode, wherein the portion is distinct from a bit line;

i. Utilize portions of the relevant memory read paths and maintain the sense amplifiers for at least some of the unrelated memory read paths in a low power mode during a read operation;

j. Utilize portions of the associated memory read paths and maintain the sense amplifiers of at least some of the unassociated memory read paths in a low power mode during a read operation; and

k. Utilize portions of the relevant memory read paths during read operations and maintain the unrelated memory read paths after the sense amplifiers of the unrelated memory read paths in a low power mode.

A low power mode or an idle mode may include a mode in which the power consumption of the memory access path is lower than the power consumption of the memory access path when the memory access path is used for access operations. In some embodiments, the low power mode may even involve cutting off memory access paths. The low power mode may additionally or alternatively include not enabling memory access paths.

It should be noted that the power reduction that occurs during the bit line phase may require that the memory access path dependencies or irrelevance should be known before opening the word lines. Power reductions that occur elsewhere (eg, in a column multiplexer) may instead allow the memory access paths to be coherent or irrelevant on each access.

Fast and low power startup and fast access memory

DRAM and other memory types, such as SRAM, flash memory, or the like, are often constructed from memory banks that are typically constructed to allow row and column access schemes.

Figure 55 illustrates an embodiment of a memory chip 5140 that includes multiple memory pads and associated logic (such as row and column decoders, depicted in Figure 55 as RD and COL, respectively). In the embodiment of Figure 55, the pads are grouped into groups and have wordlines and bitlines through them. The memory pads and associated logic are labeled 5141 , 5142 , 5143 , 5144 , 5145 , and 5146 in FIG. 55 and share at least one bus 5147 .

The memory chip 5140 may be included as illustrated in any of FIGS. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22, or 23 The memory chip may include the memory chip or otherwise contain the memory chip.

For example, in DRAM, the overhead associated with starting a new line (eg, preparing a new line for access) is significant. Once a line is activated (also called open), the data within that line is available for faster access. In DRAM, this access may occur in a random fashion.

Two issues associated with starting a new line are power and time:

c. The power will rise due to the current surge caused by accessing all capacitors on the line in common and having to load the line (eg, when opening a line with only a few memory banks, the power can reach several amps); and

d. The time delay problem is mainly associated with the time it takes to load the row (word) lines and then the bit (column) lines.

Some embodiments of the present disclosure may include systems and methods to reduce peak power consumption and reduce line start-up time during start-up of a line. Some embodiments may reduce these power and time costs at the expense of at least some degree of full random access within a line.

For example, in one embodiment, a memory cell may include a first memory pad, a second memory pad, and an enable unit configured to enable a first group of memory cells included in the first memory pad, and The second group of memory cells included in the second memory pad is not activated. The first group of memory cells and the second group of memory cells may both belong to a single row of the memory cells.

Alternatively, the enabling unit may be configured to enable the second group of memory cells included in the second memory pad without enabling the first group of memory cells.

In some embodiments, the activation unit may be configured to activate the second group of memory cells after initiating the first group of memory cells.

For example, the activation unit may be configured to activate the second group of memory cells after expiration of a delay period that begins after activation of the first group of memory cells has completed.

Additionally or alternatively, the enable unit may be configured to enable a second group of memory cells based on the value of a signal generated on a first word line segment coupled to the first group of memory cells of.

In any of the embodiments described above, the activation cell may include an intermediate circuit disposed between the first word line segment and the second word line segment. In these embodiments, a first wordline segment may be coupled to a first memory cell and a second wordline segment may be coupled to a second memory cell. Non-limiting examples of intermediate circuits include switches, flip-flops, buffers, inverters, and the like, some of which are described throughout FIGS. 56-61 .

In some embodiments, the second memory cell may be coupled to the second word line segment. In these embodiments, the second word line segment may be coupled to a bypass word line path through at least the first memory pad. An example of such a bypass path is illustrated in FIG. 61 .

The enable unit may include a control unit configured to control voltages (and/or currents) to the first group of memory cells and to the memory cells based on enable signals from word lines associated with a single row Supply of the second group.

In another embodiment, a memory cell may include a first memory pad, a second memory pad, and an enable unit configured to supply an enable signal to a first group of memory cells of the first memory pad and delay The enable signal is supplied to the second group of memory cells of the second memory pad at least until the enablement of the first group of memory cells has been completed. The first group of memory cells and the second group of memory cells may belong to a single row of the memory cells.

For example, the activation unit may comprise a delay unit which may be configured to delay supply of the activation signal.

Additionally or alternatively, the enable unit may include a comparator that may be configured to receive an enable signal at an input thereof and to control the delay unit based on at least one characteristic of the enable signal.

In another embodiment, a memory cell may include a first memory pad, a second memory pad, and an isolation unit that may be configured to: during an initial startup period in which the first memory cell of the first memory pad is enabled isolating the first memory cell from a second memory cell of a second memory pad; and coupling the first memory cell to the two memory cells after the initial startup period. The first memory cell and the second memory cell may belong to a single row of memory cells.

In the following embodiments, modifications to the memory pads themselves may not be required. In some examples, embodiments may rely on minor modifications to memory banks.

The following figures depict a mechanism for shortening word signals added to a memory bank, thereby splitting the word line into shorter sections.

In the following figures, various memory bank components are omitted for clarity.

Figures 56-61 illustrate portions of a memory bank (labeled 5140(1), 5140(2), 5140(3), 5140(4), 5140(5), and 5149(6), respectively) that include groups grouped in Row decoders 5112 within different groups and multiple memory pads such as 5150(1), 5150(2), 5150(3), 5150(4), 5150(5), 5150(6), 5151(1 ), 5151(2), 5151(3), 5151(4), 5151(5), 5151(6), 5152(1), 5152(2), 5152(3), 5152(4), 5152(5 ) and 5152(6)).

Memory pads arranged in rows may include different groups.

56-59 and 61 illustrate nine groups of memory pads, where each group includes a pair of memory pads. Any number of groups may be used, each group having any number of memory pads.

Memory pads 5150(1), 5150(2), 5150(3), 5150(4), 5150(5), and 5150(6) are arranged in rows, sharing multiple memory lines, and divided into three groups: first upper group, which includes memory pads 5150(1) and 5150(2); a second upper group, which includes memory pads 5150(3) and 5150(4); and a third upper group, which includes memory pad 5150( 5) and 5150(6).

Similarly, memory pads 5151(1), 5151(2), 5151(3), 5151(4), 5151(5), and 5151(6) are arranged in rows, sharing multiple memory lines and divided into three groups: A middle group that includes memory pads 5151(1) and 5151(2); a second middle group that includes memory pads 5151(3) and 5151(4); and a third middle group that includes memory pads 5151(5) and 5151(6).

Additionally, memory pads 5152(1), 5152(2), 5152(3), 5152(4), 5152(5), and 5152(6) are arranged in rows, sharing multiple memory lines and grouped into three groups: A lower group, which includes memory pads 5152(1) and 5152(2); a second lower group, which includes memory pads 5152(3) and 5152(4); and a third lower group, which includes memory Pads 5152(5) and 5152(6). Any number of memory pads can be arranged in rows and share memory lines, and can be divided into any number of groups.

For example, the number of memory pads per group may be one, two, or may exceed two.

As explained above, the enable circuit can be configured to enable one group of memory pads without enabling another group of memory pads that share the same memory line or at least are coupled to different memory line segments with the same line address.

56 to 61 illustrate different examples of start-up circuits. In some embodiments, at least a portion of an enable circuit, such as an intermediate circuit, may be located between groups of memory pads to allow one group of memory pads to be enabled without enabling another group of memory pads of the same row.

56 illustrates intermediate circuits, such as delay or isolation circuits 5153(1)-5153(3), as positioned between different lines of the first upper group of memory and different lines of the second upper group of memory pads.

56 also illustrates intermediate circuits, such as delay or isolation circuits 5154(1)-5154(3), as positioned between different lines of the second upper group of memory and different lines of the third upper group of memory pads. Additionally, some delay or isolation circuits are positioned between groups formed by intermediate groups of memory pads. In addition, some delay or isolation circuits are positioned between the groups formed by the lower group of memory pads.

The delay or isolation circuit can delay or stop the propagation of word line signals from row decoder 5112 along a row to another group.

57 illustrates an intermediate circuit, such as a delay or isolation circuit, including flip-flops, such as 5155(1)-5155(3) and 5156(1)-5156(3).

When an enable signal is injected into a word line, one of the first pad groups (depending on the word line) is enabled, while the other groups along the word line remain de-enabled. Additional groups can be activated on the next clock cycle. For example, a second group of the other groups can be activated on the next clock cycle, and a third group of the other groups can be activated after another clock cycle.

The flip-flops may include D-type flip-flops or any other type of flip-flops. For simplicity, the clock fed to the D-type flip-flop is omitted from the diagram.

Thus, access to the first group can use power to charge only a portion of the word lines associated with the first group, which is faster and requires less current than charging the entire word line.

More than one flip-flop can be used between groups of memory pads, thereby increasing the delay between open sections. Additionally or alternatively, embodiments may use slower clocks to increase latency.

Furthermore, the activated group may still contain the group from the previous line value used. For example, the method may allow a new line segment to be started while still accessing data from a previous line, thereby reducing losses associated with starting a new line.

Thus, some embodiments may have a first group enabled and allow other groups of previously enabled lines to remain active, where the signals of the bit lines do not interfere with each other.

Additionally, some embodiments may include switches and control signals. This control signal can be controlled by the bank controller or by adding flip-flops between the control signals (eg, to produce the same timing effects that the mechanisms described above have).

Figure 58 illustrates an intermediate circuit, such as a delay or isolation circuit, that is a switch (such as 5157(1)-5157(3) and 5158(1)-5158(3)) and is located in one of a group of another group between. A set of switches positioned between groups can be controlled by dedicated control signals. In Figure 58, control signals may be sent by row control unit 5160(1) and delayed by a sequence of one or more delay units (eg, units 5160(2) and 5160(3)) between different sets of switches.

59 illustrates an intermediate circuit, such as a delay or isolation circuit, which is a sequence of inverter gates or buffers such as 5159(1)-5159(3) and 5159'1(0-5159'(3)) and positioned between memory pad groups.

Instead of switches, buffers can be used between groups of memory pads. The buffer may not allow voltage drop between switches along the word line, an effect that sometimes occurs when a single transistor structure is used.

Other embodiments may allow more random access and still provide very low startup power and time by using regions added to the memory bank.

Figure 60 illustrates an embodiment to illustrate the use of global word lines (such as 5152(1)-5152(8)) positioned proximate memory pads. These wordlines may or may not pass through the memory pad and are coupled to wordlines within the memory pad via intermediate circuits such as switches, such as 5157(1)-5157(8). This switch controls which memory pad will be activated and allows the memory controller to activate only the relevant line portion at each point in time. Unlike the embodiments described above that use sequential activation of line segments, the embodiment of FIG. 60 may provide better control.

Enable signals such as row portion enable signals 5170(1) and 7150(2) may originate from logic not shown, such as a memory controller.

61 illustrates a global word line 5180 through the memory pad and forming a bypass path for word line signals that may not need to be routed outside the pad. Thus, the embodiment shown in Figure 61 can reduce the area of a memory bank at the expense of some memory density.

In Figure 61, the global world line may pass uninterrupted through the memory pad and may not be connected to memory cells. The local word line segment can be controlled by one of the switches and connected to the memory cells in the pad.

When groups of memory pads provide substantial partitioning of word lines, groups of memory may actually support full random access.

Another embodiment for slowing the spread of enable signals along word lines may also save some wiring and logic, using switches and/or other buffering or isolation circuits between memory pads instead of using dedicated enable signals and dedicated lines to Delivery enable signal.

For example, comparators can be used to control switches or other buffering or isolation circuits. When the level of the signal on the word line segment monitored by the comparator reaches a certain level, the comparator may activate a switch or other buffering or isolation circuit. For example, a certain level may indicate that the previous word line segment is fully loaded.

62 illustrates a method 5190 for operating a memory cell. For example, method 5130 may be implemented using any of the memory banks described above with respect to FIGS. 56-61 .

Method 5190 may include steps 5192 and 5194.

Step 5192 may include enabling, by the enabling unit, a first group of memory cells included in a first memory pad of the memory unit without enabling a second group of memory cells included in a second memory pad of the memory unit . The first group of memory cells and the second group of memory cells may both belong to a single row of the memory cells.

Step 5194 may include activating the second group of memory cells by the activation unit, eg, after step 5192.

may be activated at the time of activation of the first group of memory cells, after the first group of memory cells is fully activated, after the expiration of a delay period initiated after activation of the first group of memory cells has completed, at Step 5194 is performed after the first group of memory cells is deactivated and in similar circumstances.

The delay period may be fixed or adjustable. For example, the duration of the delay period may be based on the expected access pattern of the memory cells, or may be set independently of the expected access pattern. The delay period can range between less than one millisecond and more than one second.

In some embodiments, step 5194 may be initiated based on the value of a signal generated on a first word line segment coupled to a first group of memory cells. For example, when the value of the signal exceeds a first threshold, it may indicate that the first group of memory cells is fully activated.

Either of steps 5192 and 5194 may involve the use of an intermediate circuit (eg, an intermediate circuit that activates the cell) disposed between the first word line segment and the second word line segment. The first word line segment can be coupled to the first memory cell and the second word line segment can be coupled to the second memory cell.

Embodiments of intermediate circuits are described throughout FIGS. 56 to 61 .

Steps 5192 and 5194 may also include controlling, by the control unit, the supply of enable signals from word lines associated with a single row to the first group of memory cells and the second group of memory cells.

Use memory parallelism to speed up test time and use vectors to test logic in memory

Some embodiments of the present disclosure may use an in-chip test unit to speed up testing.

In general, memory chip testing requires a large amount of test time. Reducing test time reduces production costs and also allows more tests to be performed, resulting in more reliable products.

63 and 64 illustrate a tester 5200 and a chip (or wafer of chips) 5210. Tester 5200 may include software to manage testing. The tester 5200 can run a different sequence of data to all the memories 5210, and then read the sequence back to identify where the failed bits of the memory 5210 are located. Once identified, the tester 5200 may issue a command to repair the bit, and if the problem can be repaired, the tester 5200 may declare the memory 5210 to pass. In other cases, some chips may be declared to fail.

Tester 5200 can write a test sequence and then read back the data to compare it to expected results.

64 shows a test system with a tester 5200 and a complete wafer 5202 of chips (such as 5210) being tested in parallel. For example, tester 5200 may be connected to each of the chips by a wire bus.

As shown in Figure 64, the tester 5200 must read and write all memory chips several times, and this data must be passed through the external chip interface.

In addition, it may be beneficial to test both the logic and memory banks of an integrated circuit, for example, using programmable configuration information, which may be provided using regular I/O operations.

The test can also benefit from the presence of test cells within the integrated circuit.

The test unit can belong to an integrated circuit and can analyze the test results and find, for example, in logic (eg, a processor subunit as depicted and described in FIG. 7A ) and/or memory (eg, across multiple memory banks) Fault.

Memory testers are generally very simple and exchange test patterns with integrated circuits according to a simple format. For example, there may be a write vector that includes pairs of the address of the memory entry to be written and the value to be written to the memory entry. There may also be a read vector that includes the address of the memory entry to be read. At least some of the addresses in the write vector may be the same as at least some addresses in the read vector. At least some other addresses of the write vector may be different from at least some other addresses of the read vector. When programmed, the memory tester may also receive an expected result vector, which may include the address of the memory entry to be read and the expected value to be read. The memory tester can compare the expected value with its read value.

According to an embodiment, the logic (eg, processor subunits) of an integrated circuit (memory with or without an integrated circuit) can be tested by a memory tester using the same protocol/format. For example, some of the values written into the vector may be commands to be executed by the logic of the integrated circuit (and may, for example, involve computation and/or memory access). The memory tester can be programmed with a read vector and an expected result vector, which can include memory entry addresses, some of which store the calculated expected value. Thus, a memory tester can be used to test logic as well as memory. Memory testers are generally simpler and less expensive than logic testers, and the proposed method allows complex logic tests to be performed using simple memory testers.

In some embodiments, the logic within the memory can use only vectors (or other data structures) without using more complex mechanisms common in logic testing (such as, for example, communicating with a controller via an interface to tell the logic which circuit to test) to enable testing of the logic within the memory.

Instead of using a test unit, the memory controller may be configured to receive instructions to access memory entries included in the configuration information, and to execute the access instructions and output the results.

Any of the integrated circuits illustrated in FIGS. 65-69 may perform testing, even in the absence of test cells or in the presence of inability to perform testing.

Embodiments of the present disclosure may include methods and systems that use memory parallelism and internal chip bandwidth to accelerate and improve test time.

The method and system can be based on the memory chip testing itself (running the test with respect to the tester, reading the test results, and analyzing the results), saving the results and ultimately allowing the tester to read the results (and, when needed, program the memory chip back, e.g. activation of redundant mechanisms). This testing may include testing memory or testing memory banks and logic (in the case of computational memory with active logic portions to be tested, such as the conditions described above in FIG. 7).

In one embodiment, the method may include reading and writing data within the chip such that external bandwidth does not limit testing.

In embodiments where the memory chip includes processor subunits, each processor subunit may be programmed with test code or configuration.

In embodiments where the memory chip has a processor sub-unit that cannot execute the test code or does not have a processor sub-unit but has a memory controller, the memory controller can then be configured to read and write patterns (eg, externally programmed to the controller) and flag the location of the failure (eg, write a value to a memory entry, read the entry, and receive a value different from the written value) for further analysis.

It should be noted that testing a memory may require testing a large number of bits, eg, testing each bit of the memory and verifying that the bit under test functions. In addition, memory tests can sometimes be repeated under different voltage and temperature conditions.

For some defects, one or more redundancy mechanisms may be activated (eg, by programming flash memory or OTP or blowing fuses). In addition, logic and analog circuits (eg, controllers, regulators, I/Os) of the memory chip may also have to be tested.

In one embodiment, an integrated circuit may include a substrate, a memory array disposed on the substrate, a processing array disposed on the substrate, and an interface disposed on the substrate.

The integrated circuits described herein may be included in any of FIGS. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22, or 23 Among the memory chips described in the above, the memory chip may be included, or the memory chip may be included in other ways.

65-69 illustrate various integrated circuits 5210 and testers 5200.

The integrated circuit is illustrated as including a memory bank 5212, a chip interface 5211 (such as an I/O controller 5214 and a bus 5213 shared by the memory bank), and a logic unit (hereinafter "logic") 5215. 66 illustrates a fuse interface 5216 and a bus 5217 coupled to the fuse interface and different memory banks.

Figures 65-70 also illustrate various steps in the test handler, such as:

a. Write test sequence 5221 (FIG. 65, FIG. 67, FIG. 68, and FIG. 69);

b. Read back test results 5222 (Figure 67, Figure 68 and Figure 69);

c. Write the expected result sequence 5223 (Figure 65);

d. Read the faulty address to repair 5224 (FIG. 66); and

e. Program fuse 5225 (FIG. 66).

Each memory bank may be coupled to and/or controlled by its own logic unit 5215. However, as described above, any memory bank assignment to logic unit 5215 may be provided. Thus, the number of logic cells 5215 may differ from the number of memory banks, the logic cells may control more than a single memory bank or a portion of a memory bank, and the like.

Logic unit 5215 may include one or more test units. 65 illustrates a test unit (TU) 5218 within logic 5215. TUs may be included in all or some of the logic units 5212. It should be noted that the test unit may be separate from the logic unit or integrated with the logic unit.

Figure 65 also illustrates a test pattern generator (designated GEN) 5219 within the TU 5218.

The test pattern generator may be included in all or some of the test cells. For simplicity, test pattern generators and test cells are not illustrated in Figures 66-70, but may be included in these embodiments.

The memory array may include multiple memory banks. Additionally, the processing array may include multiple test cells. The plurality of test units may be configured to test the plurality of memory banks to provide test results. The interface may be configured to output information indicative of the test results to a device external to the integrated circuit.

The plurality of test cells may include at least one test pattern generator configured to generate at least one test pattern for testing one or more of the plurality of memory banks. In some embodiments, as explained above, each of the plurality of test cells may include a test pattern generator configured to generate a test pattern for use in the plurality of test cells A specific test unit is used to test at least one of the plurality of memory banks. As indicated above, Figure 65 illustrates a test pattern generator (GEN) 5219 within a test cell. One or more or even all logic cells may include a test pattern generator.

The at least one test pattern generator may be configured to receive instructions from the interface for generating the at least one test pattern. A test pattern may include a memory entry that should be accessed (eg, read and/or written) during testing and/or a value to be written to the entry, and the like.

The interface may be configured to receive configuration information including instructions for generating at least one test pattern from an external unit, which may be external to the integrated circuit.

The at least one test pattern generator may be configured to read configuration information from the memory array, the configuration information including instructions for generating the at least one test pattern.

In some embodiments, the configuration information may include a vector.

The interface may be configured to receive configuration information from a device that may be external to the integrated circuit, the configuration information may include instructions that may be at least one test pattern.

For example, at least one test pattern may include memory array entries to be accessed during testing of the memory array.

The at least one test pattern may further include input data to be written to memory array entries accessed during testing of the memory array.

Additionally or alternatively, the at least one test pattern may further include input data to be written to memory array entries accessed during testing of the memory array, and data to be read from memory array entries accessed during testing of the memory array The expected value of the output data.

In some embodiments, the plurality of test cells may be configured to retrieve test instructions from the memory array that, once executed by the plurality of test cells, cause the plurality of test cells to test the memory array.

For example, the test instructions may be included in configuration information.

The configuration information may include expected results of testing of the memory array.

Additionally or alternatively, the configuration information may include values of output data to be read from memory array entries accessed during testing of the memory array.

Additionally or alternatively, the configuration information may include a vector.

In some embodiments, the plurality of test cells may be configured to retrieve from the memory array test instructions that, once executed by the plurality of test cells, cause the plurality of test cells to test the memory array and test the processing array .

For example, the test instructions may be included in configuration information.

The configuration information may include vectors.

Additionally or alternatively, the configuration information may include expected results of testing of the memory array and processing array.

In some embodiments, as described above, the plurality of test cells may lack a test pattern generator for generating test patterns used during testing of the plurality of memory banks.

In these embodiments, at least two of the plurality of test cells may be configured to test at least two of the plurality of memory banks in parallel.

Alternatively, at least two of the plurality of test units may be configured to test at least two of the plurality of memory banks in series.

In some embodiments, the information indicative of the test result may include an identifier of the failed memory array entry.

In some embodiments, the interface may be configured to retrieve partial test results obtained by multiple test circuits multiple times during testing of the memory array.

In some embodiments, the integrated circuit may include an error correction unit configured to correct at least one error detected during testing of the memory array. For example, the error correction unit may be configured to repair memory errors using any suitable technique, eg, by disabling some memory words and replacing those words with redundant words.

In any of the embodiments described above, the integrated circuit may be a memory chip.

For example, the integrated circuit may include a distributed processor, wherein the processing array may include multiple subunits of the distributed processor, as depicted in Figure 7A.

In these embodiments, each of the processor sub-units may be associated with a corresponding dedicated memory bank of the plurality of memory banks.

In any of the embodiments described above, the information indicative of the test results may be indicative of the status of at least one memory bank. The state of a memory bank may be provided at one or more granularities: per memory word, per group of entries, or per full memory bank.

65-66 illustrate the four steps in the tester testing phase.

In a first step, the tester writes (5221) the test sequence and the logic cells of the group write data to their memory. The logic may also be complex enough to receive commands from the tester and generate sequences by itself (as explained below).

In the second step, the tester writes (5223) the expected result to the memory under test, and the logic unit compares the expected result with the data read from its memory bank, saving a list of errors. Writing of the expected result can be simplified if the logic is complex enough to produce the sequence of the expected result by itself (as explained below).

In a third step, the tester reads (5224) the fault address from the logic unit.

In a fourth step, the tester takes action on the result (5225) and the error can be recovered. For example, a tester can be connected to a specific interface to program fuses in memory, but any other mechanism that allows programming of error correction mechanisms in memory can also be used.

In these embodiments, the memory tester may use the vectors to test the memory.

For example, each vector can be constructed from an input series and an output series.

The input series can include pairs of addresses and data written to memory (in many embodiments, this series can be modeled as a formula that allows a program, such as a program executed by a logic unit, to generate when needed).

In some embodiments, the test pattern generator can generate such vectors.

It should be noted that a vector is one example of a data structure, but some embodiments may use other data structures. This data structure is compatible with other test data structures generated by testers located outside the integrated circuit.

The output series may include address and data pairs containing the expected data to be read back from memory (in some embodiments, the series may additionally or alternatively be generated by the program at runtime, such as by a logic unit).

Memory testing typically involves performing a list of vectors, each vector writing data to memory according to the input series, and then reading the data back according to the output series and comparing the data to its expected data.

In the event of a mismatch, the memory can be classified as failed, or if the memory includes a mechanism for redundancy, the redundant mechanism can be activated so that the vector is tested again on the activated redundant mechanism.

In embodiments where the memory includes a processor sub-unit (as described above with respect to Figure 7A) or contains many memory controllers, the entire test may be handled by a group of logic units. Thus, the memory controller or processor subunit may perform the test.

The memory controller can be programmed from the tester, and the test results can be saved in the controller itself to be read by the tester later.

To configure and test the operation of the logic cells, the tester can configure the logic cells for memory access and verify that the results can be read through the memory access.

For example, the input vector may contain the programming sequence for the logic cell, and the output vector may contain the expected result of this test. For example, if a logic unit such as a processor subunit includes a multiplier or adder configured to perform computations on two addresses in memory, the input vector may include a set of commands to write data to memory and to the adder / A set of commands for multiplier logic. As long as the adder/multiplier result can be read back into the output vector, the result can be sent to the tester.

The testing may also include loading a logic configuration from memory and sending logic outputs to memory.

In embodiments where the logic unit loads its configuration from memory (eg, if the logic is a memory controller), the logic unit may run code from the memory itself.

Thus, the input vector may include a program for the logic unit, and the program itself may test the various circuits in the logic unit.

Thus, testing may not be limited to receiving vectors in a format used by external testers.

If the command loaded into the logic unit instructs the logic unit to write the results back into the memory bank, the tester can read the results and compare the results to the expected output series.

For example, a vector written to memory may be or may include a test program for the logic unit (eg, the test may assume the memory is valid, but even if the memory is invalid, the written test program will not work and the test will fail, which is acceptable results because the chip is invalid anyway) and/or how the logic unit runs the code and writes the results back to memory. Since all testing of logic cells can be done through memory (eg, writing logic test inputs to memory and writing test results back to the memory), a tester can run simple vector tests with input sequences and expected output sequences.

Logic configurations and results can be accessed as read and/or write commands.

Figure 68 illustrates a tester 5200 sending a write test sequence 5221 as a vector.

Portions of the vector include test code 5232 split between memory banks 5212 coupled to logic 5215 of the processing array.

Each logic 5215 can execute code 5232 stored in its associated memory bank, and the execution can include accessing one or more memory banks, performing calculations, and storing results (eg, test results 5231 ) in memory banks 5212 .

Test results may be sent back by tester 5200 (eg, read back results 5222).

This may allow logic 5215 to be controlled by commands received by I/O controller 5214.

In Figure 68, the I/O controller 5214 is connected to memory banks and logic. In other embodiments, logic may be connected between the I/O controller 5214 and the memory banks.

70 illustrates a method 5300 for testing a memory bank. For example, method 5300 may be implemented using any of the memory banks described above with respect to FIGS. 65-69.

Method 5300 may include steps 5302, 5310, and 5320. Step 5302 may include receiving a request to test a memory bank of the integrated circuit. The integrated circuit may include a substrate, a memory array disposed on the substrate and including memory banks, a processing array disposed on the substrate, and an interface disposed on the substrate. The processing array may include multiple test cells, as described above.

In some embodiments, the request may include configuration information, one or more vectors, commands, and the like.

In these embodiments, the configuration information may include expected results of testing of the memory array, instructions, data, values of output data to be read from memory array entries accessed during testing of the memory array, test patterns, and the like analog.

The test pattern may include at least one of: (i) memory array entries to be accessed during testing of the memory array, (ii) memory array entries to be written to during testing of the memory array Input data, or (iii) the expected value of output data to be read from memory array entries accessed during testing of the memory array.

Step 5302 may include at least one of and/or may be followed by at least one of:

a. receiving, through the at least one test pattern generator, an instruction to generate at least one test pattern from an interface;

b. receiving configuration information through the interface and from an external unit external to the integrated circuit, the configuration information including instructions for generating at least one test pattern;

c. reading configuration information from the memory array by the at least one test pattern generator, the configuration information including instructions for generating the at least one test pattern;

d. receiving configuration information through the interface and from an external unit external to the integrated circuit, the configuration information containing instructions for at least one test pattern;

e. Passing and retrieving from the memory array test instructions that, once executed by the plurality of test cells, cause the plurality of test cells to test the memory array; and

f. Receiving, through and from the memory array, test instructions that, once executed by the plurality of test cells, cause the plurality of test cells to test the memory array and test the processing array.

Step 5302 may be followed by step 5310. Step 5310 may include testing the plurality of memory banks through the plurality of test units and in response to requests to provide test results.

Method 5300 may also include receiving, via the interface, partial test results obtained by the plurality of test circuits during testing of the memory array.

Step 5310 may include at least one of and/or may be followed by at least one of:

a. Generate test patterns by one or more test pattern generators (eg, included in one, some, or all of the plurality of test cells) for use by the one or more test cells to test at least one of the plurality of memory banks One;

b. Test at least two of the plurality of memory banks in parallel with at least two of the plurality of test units;

c. serially testing at least two of the plurality of memory banks with at least two of the plurality of test units;

d. Write the value to the memory entry, read the memory entry and compare the result; and

e. Correct at least one error detected during testing of the memory array by the error correction unit.

Step 5310 may be followed by step 5320. Step 5320 may include outputting information indicative of the test results through an interface and external to the integrated circuit.

The information indicative of the test result may include an identifier of the failed memory array entry. Time can be saved by not sending read data for each memory entry.

Additionally or alternatively, the information indicative of the test results may be indicative of the status of at least one memory bank.

Thus, in some embodiments, this information indicative of the test results may be much smaller than the total size of data cells written to or read from the memory bank during the test, and may be possible without test cell assistance The input data sent from the tester testing the memory is much smaller.

The integrated circuit under test may include a memory chip and/or a distributed processor as illustrated in any of the previous figures. For example, the integrated circuits described herein may be included in FIGS. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22, or 23. The memory chip may be included in any of the described memory chips, or otherwise included.

71 illustrates an embodiment of a method 5350 for testing a memory bank of an integrated circuit. For example, method 5350 may be implemented using any of the memory banks described above with respect to Figures 65-69.

Method 5350 may include steps 5352, 5355, and 5358. Step 5352 may include receiving configuration information including instructions through an interface of the integrated circuit. An integrated circuit that includes an interface may also include a substrate, a memory array including a memory bank and disposed on the substrate, a processing array disposed on the substrate, and an interface disposed on the substrate.

The configuration information may include expected results of testing of the memory array, instructions, data, values of output data to be read from memory array entries accessed during testing of the memory array, test patterns, and the like.

Additionally or alternatively, the configuration information may include the instruction, the address of the memory entry to which the instruction is written, input data, and may also include the address of the memory entry to receive output values computed during execution of the instruction.

The test pattern may include at least one of: (i) memory array entries to be accessed during testing of the memory array, (ii) memory array entries to be written to during testing of the memory array Input data, or (iii) the expected value of output data to be read from memory array entries accessed during testing of the memory array.

Step 5352 may be followed by step 5355. Step 5355 may include executing instructions through the processing array by accessing the memory array, performing computational operations, and providing results.

Step 5355 may be followed by step 5358. Step 5358 may include outputting information indicative of the result through an interface and external to the integrated circuit.

Network (cyber) security and tamper detection technology

Memory chips and/or processors may be targeted by malicious actors and may be subject to various types of cyber attacks. In some cases, such attacks may attempt to alter data and/or code stored in one or more memory resources. Cyberattacks can be especially problematic relative to trained neural networks or other types of artificial intelligence (AI) models that depend on large amounts of data stored in memory. If the stored data is manipulated or even obscured, this manipulation can be harmful. For example, autonomous vehicle systems that rely on data-intensive AI models to identify other vehicles, pedestrians, etc. may incorrectly assess the host vehicle's environment if the data on which they rely is corrupted or obscured. As a result, accidents may occur. As AI models become more commonplace across a wide range of technologies, cyberattacks on the data associated with such models can cause significant damage.

In other situations, a cyber attack may involve one or more actors tampering or attempting to tamper with operating parameters associated with processors or other types of integrated circuit-based logic elements. For example, processors are typically designed to operate within certain operating specifications. A cyberattack involving tampering may attempt to alter one or more of the operating parameters of a processor, memory unit, or other circuit so that the processor, memory unit, or other circuit exceeds its designed operating specifications (e.g., clock speed, bandwidth specification, temperature limits, operating rates, etc.). This tampering can cause the target hardware to fail.

Conventional techniques for defending against network attacks may include computer programs (eg, anti-virus or anti-malware software) operating at the processor level. Other techniques may include the use of software-based firewalls associated with routers or other hardware. While these techniques can be used to combat cyberattacks using software programs that execute outside of the memory cells, there remains a need for additional or alternative techniques for efficiently protecting the data stored in the memory cells, especially where the accuracy and availability of the data is critical to factors such as where the operation of memory-intensive applications such as neural networks is critical. Embodiments of the present invention may provide various integrated circuit designs including memory that are resistant to cyberattacks on memory.

Capture sensitive information and commands to the integrated circuit in a secure manner (eg, during the boot process when the interface to the outside of the chip/integrated circuit is not yet functional) and then maintain the sensitive information and commands within the integrated circuit It is not exposed outside the integrated circuit, which increases the security of sensitive information and commands. CPUs and other types of processing units are vulnerable to cyber attacks, especially when those CPUs/processing units operate with external memory. The disclosed embodiments that include distributed processor subunits disposed on memory chips in a memory array that includes multiple memories may be less vulnerable to cyber attacks and tampering (eg, because processing occurs within the memory chips) Group. Inclusion of any combination of the disclosed security measures discussed in greater detail below may further reduce the susceptibility of the disclosed embodiments to cyber attack and/or tampering.

Figure 72A is a diagrammatic representation of an integrated circuit 7200 including a memory array and a processing array in accordance with an embodiment of the invention. For example, integrated circuit 7200 may include any of the distributed processor architectures (and features) on a memory chip described in the preceding sections and throughout this disclosure. The memory array and processing array can be formed on a common substrate, and in some disclosed embodiments, the integrated circuit 7200 can constitute a memory chip. For example, as discussed above, integrated circuit 7200 may include a memory chip that includes a plurality of memory banks and a plurality of processor subunits spatially distributed on the memory chip, wherein each of the plurality of memory banks One is associated with a dedicated one or more of the plurality of processor subunits. In some cases, each processor sub-unit may be dedicated to one or more memory banks.

In some embodiments, the memory array may include a plurality of discrete memory banks 7210_1 , 7210_2 . . . 7210_J1 , 7210_Jn, as shown in FIG. 72A . According to embodiments of the invention, memory array 7210 may include one or more types of memory including, for example, volatile memory such as RAM, DRAM, SRAM, phase change RAM (PRAM), magnetoresistive RAM (MRAM), Resistive RAM (ReRAM or the like) or non-volatile memory such as Flash or ROM. According to some embodiments of the present invention, the memory banks 7210_1 to 7210_Jn may include a plurality of MOS memory structures.

As mentioned above, the processing array may include multiple processor subunits 7220_1 to 7220_K. In some embodiments, each of the processor sub-units 7220_1 - 7220_K may be associated with one or more discrete memory banks among the plurality of discrete memory banks 7210_1 - 7210_Jn. Although the example embodiment of Figure 72A illustrates that each processor subunit is associated with two discrete memory banks 7210, it should be appreciated that each processor subunit may be associated with any number of discrete dedicated memory banks. And vice versa, each memory bank may be associated with any number of processor subunits. According to embodiments of the invention, the number of discrete memory banks included in the memory array of integrated circuit 7200 may be equal to, less than, or greater than the number of processor subunits included in the processing array of integrated circuit 7200 .

The integrated circuit 7200 may further include a plurality of first buses 7260 consistent with embodiments of the invention (and as described in the above sections). Each bus 7260 may connect processor sub-unit 7220_k to a corresponding dedicated memory bank 7210_j. According to some embodiments of the present invention, the integrated circuit 7200 may further include a plurality of second buses 7261 . Each bus 7261 may connect processor sub-unit 7220_k to another processor sub-unit 7220_k+1. As shown in FIG. 72A, a plurality of processor sub-units 7220_1-7220_K may be connected to each other via a bus 7261. Although FIG. 72A illustrates the plurality of processor sub-units 7220_1 through 7220_K forming a loop as being connected in series via a bus 7261, it should be understood that the processor units 7220 may be connected in any other manner. For example, in some cases a particular processor sub-unit may not be connected to other processor sub-units via bus 7261. In other cases, a particular processor sub-unit may be connected to only one other processor sub-unit, and in still other cases, a particular processor sub-unit may be connected to two or more than two via one or more buses 7261 Other processor subunits (eg, forming series connections, parallel connections, branch connections, etc.). It should be noted that the embodiments of integrated circuit 7200 described herein are exemplary only. In some cases, integrated circuit 7200 may have different internal components and connections, and in other cases, one or more of the internal components and described connections may be omitted (eg, depending on the needs of a particular application).

Referring back to FIG. 72A, integrated circuit 7200 may include one or more structures for implementing at least one security measure relative to integrated circuit 7200. In some cases, such a structure may be configured to detect network attacks that manipulate or obscure (or attempt to manipulate or obscure) data stored in one or more of the memory banks. In other cases, such structures may be configured to detect tampering with operating parameters associated with integrated circuit 7200 or tampering with one or more hardware elements that directly or indirectly affect one or more operations associated with integrated circuit 7200 ( whether included within the integrated circuit 7200 or external to the integrated circuit 7200).

In some cases, the controller 7240 may be included in the integrated circuit 7200. The controller 7240 may be connected via one or more buses 7250 to, for example, one or more of the processor subunits 7220_1 . . . 7220_k. The controller 7240 may also be connected to one or more of the memory banks 7210_1 . . . 7210_Jn. Although the example embodiment of FIG. 72A shows one controller 7240, it should be understood that the controller 7240 may include multiple processor elements and/or logic circuits. In the disclosed embodiments, the controller 7240 may be configured to implement at least one security measure with respect to at least one operation of the integrated circuit 7200 . Additionally, in the disclosed embodiments, if at least one safety measure is triggered, the controller 7240 may be configured to take (or cause) one or more remedial actions.

According to some embodiments of the invention, at least one security measure may include a controller-implemented handler for locking access to certain aspects of the integrated circuit 7200 . Access locking involves having the controller prevent access (read and/or write) to certain areas of memory from outside the chip. Access control may be applied at address resolution, fraction of memory bank resolution, memory bank resolution, and the like. In some cases, one or more physical locations in memory associated with integrated circuit 7200 may be locked (eg, one or more memory banks of integrated circuit 7200 or any portion of one or more of the memory banks). In some embodiments, controller 7240 may lock access to certain portions of integrated circuit 7200 associated with the execution of artificial intelligence models (or other types of software-based systems). For example, in some embodiments, the controller 7240 may lock access to the weights of the neural network model stored in memory associated with the integrated circuit 7200. It should be noted that a software program (ie, a model) may include three components, including: input data for the program, code data for the program, and output data for executing the program. This component can also be adapted for neural network models. During operation of this model, input data may be generated and fed into the model, and executing the model may generate output data for reading. However, the program code and data values (eg, predetermined model weights, etc.) associated with executing the model using the received input data may remain fixed.

As described herein, a lock may refer to an operation of a controller that does not allow read or write operations originating from outside the chip/integrated circuit with respect to certain regions of the memory, for example. The I/O-accessible controller of the chip/integrated circuit can lock not only the entire memory bank, but also any range of memory addresses within the memory bank, from a single memory address to an address range including all addresses of the available memory bank (or any address range in between).

Because memory locations associated with receiving input data and storing output data are associated with changing values and interactions with components external to integrated circuit 7200 (eg, components supplying input data or receiving output data), locking access to those memory locations Access may be impractical in some situations. On the other hand, restricting access to memory locations associated with model program code and fixed data values can be effective against certain types of network attacks. Thus, in some embodiments, memory associated with program code and data values may be locked (eg, not used to write/receive input data and memory used to read/provide output data) as a security measure. Restricting access may include locking certain memory locations so that changes cannot be made to certain program code and/or data values (eg, those associated with an execution model based on received input data). Additionally, memory regions associated with intermediate data (eg, data generated during execution of the model) may also be locked against external access. Thus, while various operational logic (whether on-board or external to integrated circuit 7200) may provide data to or from memory locations associated with receiving input data or retrieving generated output data Data is received, but the operational logic will not be able to access or modify memory locations storing program code and data values associated with program execution based on the received input data.

In addition to locking memory locations on integrated circuit 7200 to provide security measures, certain arithmetic logic elements (and the memory areas they access) that are configured to execute program code associated with a particular program or model may also be restricted by access to implement other security measures. In some cases, relative to the operational logic (and its associated memory regions) located on the integrated circuit 7200 (eg, operational memory (eg, memory that includes computational capabilities, such as the distribution on a memory chip disclosed herein) type processors), etc.) implement this access constraint. Operational logic associated with any execution of code stored in the locked memory portion of integrated circuit 7200 or any access to data values stored in the locked memory portion of integrated circuit 7200 may also be locked/restricted ( and associated memory locations) regardless of whether the arithmetic logic resides on the integrated circuit 7200 board. Restricting access to the computational logic responsible for executing the program/model may further ensure that code and data values associated with operations on received input data remain protected from manipulation, shadowing, and the like.

Controller-implemented security measures may be implemented in any suitable manner, including locking or restricting access to hardware-based regions associated with certain portions of the memory array of integrated circuit 7200. In some embodiments, this locking may be implemented by adding or supplying commands to the controller 7240 that are configured to cause the controller 7240 to lock certain portions of memory. In some embodiments, the portion of hardware-based memory to be locked may be specified by a particular memory address (eg, the address associated with any memory element of memory banks 7210_1 . . . 7210_J2, etc.). In some embodiments, locked regions of memory may remain fixed during program or model execution. In other cases, the lock zone may be configurable. That is, in some cases, commands may be supplied to the controller 7240 such that during execution of a program or model, the locked region may be changed. For example, certain memory locations may be added to a locked area of memory at certain times. Or at certain times, certain memory locations (eg, previously locked memory locations) may be excluded from the locked area of memory.

Locking of certain memory locations may be accomplished in any suitable manner. In some cases, records of locked memory locations (eg, files, databases, data structures, etc. that store and identify locked memory addresses) may be accessible by controller 7240 so that controller 7240 can determine whether a certain memory request is related to Lock memory location related. In some cases, the controller 7240 maintains a database of lock addresses to use to control access to certain memory locations. In other cases, the controller may have a table or set of one or more registers configurable up to a lock, and may include identifying memory locations to be locked (eg, memory access to these memory locations from outside the chip should be restricted) ) is a fixed predetermined value. For example, when a memory access is requested, the controller 7240 can compare the memory address associated with the memory access request to the locked memory address. If it is determined that the memory address associated with the memory access request is within the list of locked memory addresses, the memory access request (eg, read or write operation) may be denied.

As discussed above, at least one security measure may include locking access to certain memory portions of the memory array 7210 that are not used for receiving input data or for providing access to generated output data. In some cases, the portion of memory within the locked area may be adjusted. For example, the locked memory portion can be unlocked, and the unlocked memory portion can be locked. Any suitable method may be used to partially unlock the locked memory. For example, the security measures implemented may include requiring a complex password for unlocking one or more portions of a locked memory area.

The implemented security measures may be triggered upon detection of any action against the implemented security measures. For example, an attempted access (whether a read or write request) to a locked memory portion can trigger a security measure. Additionally, if a complex password entered (eg, an attempt to partially unlock the locked memory) does not match a predetermined complex password, security measures may be triggered. In some cases, security measures may be triggered if the correct complex password is not provided within an allowable threshold number of complex password entry attempts (eg, 1, 2, 3, etc.).

The memory section may be locked at any suitable time. For example, in some cases, portions of memory may be locked at various times during program execution. In other cases, portions of memory may be locked after startup or before program/model execution. For example, the memory address to be locked may be determined and identified in conjunction with programming of the program/model program code or after generating and storing data to be accessed by the program/model. Thereby, vulnerabilities to attacks on the memory array 7210 may be reduced or eliminated during times at or after the start of program/model execution, after data to be used by the program/model has been generated and stored, and the like.

Unlocking of the locked memory may be accomplished by any suitable method or at any suitable time. As described above, the locked memory portion may be unlocked upon receipt of the correct complex code or password or the like. In other cases, locked memory can be unlocked by rebooting (either by command or by powering off and on) or deleting the entire memory array 7210. Additionally or alternatively, a release command sequence may be implemented to unlock one or more memory portions.

According to embodiments of the present invention and as described above, the controller 7240 may be configured to control traffic to and from the integrated circuit 7200 , particularly traffic from sources external to the integrated circuit 7200 . For example, as shown in Figure 72A, traffic between components external to integrated circuit 7200 and components internal to integrated circuit 7200 (eg, memory array 7210 or processor subunit 7220) may be controlled by controller 7240 . This traffic may pass through the controller 7240 or one or more buses controlled or monitored by the controller 7240 (eg, 7250, 7260, or 7261).

According to some embodiments of the invention, the integrated circuit 7200 may receive unchangeable data (eg, fixed data; eg, model weights, coefficients, etc.) and certain commands (eg, code; eg, identify the portion of memory to be locked) during the power-on handler ). Here, unchangeable data may refer to data that remains fixed during execution of a program or model and may remain unchanged until subsequent power-on processing of the program. During program execution, integrated circuit 7200 may interact with changeable data, which may include input data to be processed and/or output data produced by processing associated with integrated circuit 7200 . As discussed above, access to memory array 7210 or processing array 7220 may be restricted during program or model execution. For example, access may be limited to certain portions of the memory array 7210 or to certain processor sub-units associated with the processing of incoming input data to be written or with Interaction of incoming input data to be written, or processing with respect to generated output data to be read, or interaction with generated output data to be read. During execution of a program or model, portions of memory containing immutable data may be locked and thereby rendered inaccessible. In some embodiments, unchangeable data and/or commands associated with the portion of memory to be locked may be included in any suitable data structure. Such data and/or commands may be made available to controller 7240, for example, via one or more configuration files that may be accessed during or after the power-on sequence.

Referring back to FIG. 72A , the integrated circuit 7200 may further include a communication port 7230. As shown in FIG. 72A, a controller 7240 may be coupled between a communication port 7230 and a bus 7250, which is shared among the processing sub-units 7220_1-7220_K. In some embodiments, communication port 7230 may be indirectly or directly coupled to a host computer 7270 associated with host memory 7280, which may include, for example, non-volatile memory. In some embodiments, the host computer 7270 may retrieve changeable data 7281 (eg, input data to be used during execution of a program or model), unchangeable data 7282 and/or commands 7283 from its associated host memory 7280 . Changeable data 7181, unchangeable data 7282, and commands 7283 may be uploaded to controller 7240 via 7230 host computer 7270 during the power-on process.

72B is a diagrammatic representation of a memory region within an integrated circuit consistent with embodiments of the present invention. As shown, FIG. 72B depicts an example of a data structure included in host memory 7280.

Referring now to FIG. 73A, another example of an integrated circuit consistent with embodiments of the present invention. As shown in FIG. 73A, the controller 7240 may include a network attack detector 7241 and a response module 7242. In some embodiments of the invention, the controller 7240 may be configured to store or access the access control rules 7243. According to some embodiments of the invention, the access control rules 7243 may be included in a configuration file accessible by the controller 7240. In some embodiments, the access control rules 7243 may be uploaded to the controller 7240 during the boot process. Access control rules 7243 may include information prompting access rules associated with any of: changeable data 7281, unchangeable data 7282, and commands 7283 and their corresponding memory locations. As explained above, the access control rules 7243 or profiles may include information identifying certain memory addresses within the memory array 7210. In some embodiments, controller 7240 may be configured to provide locking mechanisms and/or functions that lock various addresses of memory array 7210, such as addresses used to store commands or unchangeable data.

Controller 7240 may be configured to enforce access control rules 7243, eg, to prevent unauthorized entities from altering immutable data or commands. In some embodiments, reads of unchangeable data or commands may be prohibited according to access control rules 7243. According to some embodiments of the invention, the controller 7240 may be configured to determine whether an access attempt has been made to certain commands or at least a portion of the immutable data. Controller 7240 (eg, including network attack detector 7241 ) can compare memory addresses associated with access requests to memory addresses for unchangeable data and commands to detect whether one or more memory locations have been locked An unauthorized access attempt was made. In this manner, for example, the cyber attack detector 7241 of the controller 7240 may be configured to determine whether a suspected cyber attack has occurred, such as altering one or more commands or altering or masking non-accessible data associated with one or more locked memory portions. A request to change data. The response module 7242 can be configured to determine how to respond to and/or implement a response to a detected cyber attack. For example, in some conditions, in response to detecting an attack on data or commands in one or more locked memory locations, the response module 7242 of the controller 7240 can implement or cause to implement a response, which can include, for example, stopping One or more operations, such as memory access operations associated with the detected attack. Responding to a detected attack may also include stopping one or more operations associated with the execution of the program or model, returning a warning or other indicator of the attempted attack, asserting the line to the host, or deleting entire memory, etc.

In addition to locking portions of memory, other techniques for defending against network attacks may also be implemented to provide the described security measures associated with integrated circuit 7200. For example, in some embodiments, controller 7240 may be configured to replicate programs or models within different memory locations and processor subunits associated with integrated circuit 7200. In this way, programs/models and replicas of programs/models can be executed independently, and the results of execution of independent programs/models can be compared. For example, a program/model may be replicated in the two memory banks 7210 and executed by different processor sub-units 7220 in the integrated circuit 7200. In other embodiments, the program/model may be replicated in two different integrated circuits 7200. In either case, the results of the program/model executions can be compared to determine if there are any differences between the replicated program/model executions. Detected differences in execution results (eg, intermediate execution results, final execution results, etc.) may indicate a cyber attack that has altered one or more aspects of the program/model or its associated data. In some embodiments, different memory banks 7210 and processor subunits 7220 may be assigned to perform the two replication models based on the same input data. In some embodiments, intermediate results may be compared during execution of two replicated models based on the same input data, and if there is a mismatch between the two intermediate results at the same stage, execution may be temporarily suspended as a potential remedial action. The integrated circuit can also compare the results in the case where the processor subunits of the same integrated circuit execute two replicated models. This can be done without informing any entity external to the integrated circuit about the execution of the two replication models. In other words, entities external to the chip are unaware that the replication model is running in parallel on the integrated circuit.

Figure 73B is a diagrammatic representation of a configuration for concurrently executing replication models in accordance with embodiments of the present invention.

Although a single program/model replication is described as one example for detecting possible cyber attacks, any number of replications (eg, 1, 2, 3, or more than 3) may be used to detect possible cyber attacks . As the number of replications and independent program/model executions increases, the confidence in the detection of a network attack can also increase. A larger number of replications can also reduce the potential success rate of a network attack, as it may be more difficult for an attacker to affect multiple program/model replicators. The number of program or model replicators can be determined at runtime to further increase the difficulty for a cyber attacker to successfully influence program or model execution.

In some embodiments, the replication models may differ in one or more aspects that differ from each other. In this example, the code associated with the two programs/models can be made different from each other, but the programs/models can be designed so that both return the same output. At least in this way, the two programs/models can be considered replicas of each other. For example, two neural network models may have different orderings of neurons in a layer relative to each other. However, despite this change in the model program code, both neural network models can return the same output. Duplicating programs/models in this manner can make it more difficult for a cyber attacker to identify these valid replicas of the program or model to be cracked, and as a result, duplicating a model/program can not only provide a way to provide redundancy to minimize the impact of a cyber attack , and can enhance detection of cyber attacks (eg, by highlighting tampering or unauthorized access where a cyber attacker changes a program/model or its data, but fails to make corresponding changes to the program/model replicator).

In many cases, replicators/models (especially including replicators/models that exhibit code differences) may be designed such that their outputs do not match exactly, but instead constitute soft values (eg, approximately the same output values), rather than exact values Fixed value. In such an embodiment, the output results from two or more active program/model replicas may be compared (eg, using dedicated modules or by a host processor) to determine their output results (whether intermediate results) or the final result) is within a predetermined range. The difference in the output soft values not exceeding a predetermined threshold or range may be considered evidence of no tampering, unauthorized access, or the like. On the other hand, if the difference in the output soft values exceeds a predetermined threshold or range, this difference may be considered evidence that a network attack in the form of tampering, unauthorized access to memory, or the like has occurred. Under these conditions, duplicate program/model security measures will be triggered and one or more remedial actions may be taken (eg, stop execution of the program or model, shut down one or more operations of integrated circuit 7200, in a safe mode with limited functionality operation, along with many other actions).

Security measures associated with integrated circuit 7200 may also involve quantitative analysis of data associated with executing or executed programs or models. For example, in some embodiments, the controller 7240 may be configured to calculate one or more checksums/hashes/cyclic redundancy checks ( CRC)/parity value. The calculated value may be compared to one or more predetermined values. If there is a discrepancy between the compared values, this discrepancy can be interpreted as evidence of tampering with data stored in at least a portion of the memory array 7210. In some embodiments, checksum/hash/CRC/check bit values may be calculated for all memory locations associated with memory array 7210 to identify changes in data. In this example, the entire memory (or bank of memory) in question may be read by, for example, the host computer 7270 or a processor associated with the integrated circuit 7200, for use in calculating the checksum/hash/CRC/check bit value. In other cases, checksum/hash/CRC/check bit values may be computed for a predetermined subset of memory locations associated with memory array 7210 to identify changes in data associated with the subset of memory locations . In some embodiments, the controller 7240 may be configured to calculate a checksum/hash/CRC/check bit value associated with a predetermined data path (eg, associated with a memory access pattern), and The calculated values may be compared to each other or to predetermined values to determine whether tampering or another form of network attack has occurred.

By protecting one or more predetermined values within integrated circuit 7200 or in locations accessible to integrated circuit 7200 (eg, expected checksum/hash/CRC/check bit value, expected difference in intermediate or final output results) values, expected difference ranges associated with certain values, etc.), can make the integrated circuit 7200 even more secure against cyber attacks. For example, in some embodiments, one or more predetermined values may be stored in registers of memory array 7210 and may be used during or after each run of the model (eg, by a controller of integrated circuit 7200 ) 7240) Evaluate intermediate or final output results, checksums, etc. In some cases, the "save last result data" command may be used to update register values to compute predetermined values on the fly, and the computed values may be saved in a register or another memory location. In this way, valid output values can be used to update predetermined values for comparison after each program or model execution or partial execution. This technique may increase the difficulty a cyber attacker may experience when attempting to modify or otherwise tamper with one or more predetermined reference values designed to expose the cyber attacker's activity.

In operation, a CRC calculator may be used to track memory accesses. For example, such calculation circuits may be disposed at the memory bank level, in a processor sub-unit, or at the controller, where each calculation circuit may be configured to accumulate to a CRC calculator as each memory access is made.

Referring now to FIG. 74A, a diagrammatic representation of another embodiment of an integrated circuit 7200 is provided. In the example embodiment represented by FIG. 74A, the controller 7240 may include a tamper detector 7245 and a response module 7246. Similar to the other disclosed embodiments, the tamper detector 7245 may be configured to detect evidence of potential tamper attempts. According to some embodiments of the invention, security measures associated with integrated circuit 7200 and implemented by controller 7240 may include, for example, comparing actual program/model operating patterns to predetermined/allowed operating patterns. If, in one or more aspects, the actual program/model operating pattern differs from the predetermined/allowed operating pattern, a safety measure may be triggered. And if a safety measure is triggered, the response module 7246 of the controller 7240 can be configured to implement one or more remedial measures in response.

74C is a diagrammatic representation of detection elements that may be located at various points within a chip, according to an exemplary disclosed embodiment. As described above, detection of network attacks and tampering can be performed using detection elements located at various points within the chip, as shown, for example, in Figure 74C. For example, a certain code may be associated with an expected number of processing events within a certain period of time. The detector shown in FIG. 74C may count the number of events (monitored by the event counter) experienced by the system during a certain period of time (monitored by the time counter). If the number of events exceeds some predetermined threshold (eg, the number of expected events during a predefined time period), tampering may be prompted. Such detectors may be included at various points in the system to monitor various types of events, as shown in Figure 74C.

More specifically, in some embodiments, controller 7240 may be configured to store or access expected program/model operation patterns 7244. For example, in some cases, the operational pattern may be represented as a curve 7247 that suggests allowed loads per time pattern and prohibited or illegal loads per time pattern. A tamper attempt may cause the memory array 7210 or the processing array 7220 to operate outside certain operating specifications. This can cause the memory array 7210 or the processing array 7220 to heat up or fail, and can enable changes to data or code associated with the memory array 7210 or the processing array 7220. This change can result in operating patterns that exceed the allowed operating patterns as suggested by curve 7247.

According to some embodiments of the invention, the controller 7240 may be configured to monitor the operational patterns associated with the memory array 7210 or the processing array 7220. The operation pattern may be associated with the number of access requests, types of access requests, timing of access requests, and the like. The controller 7240 may be further configured to detect tampering attacks if the operational pattern is different from the allowable operational pattern.

It should be noted that the disclosed embodiments can be used to defend not only against network attacks, but also against non-malicious errors in operation. For example, the disclosed embodiments are also effective in protecting systems such as integrated circuit 7200 from errors caused by environmental factors such as temperature or voltage changes or levels, especially at such levels beyond those used for integrated circuit 7200 operating specifications.

In response to detecting a suspected cyber attack (eg, in response to triggered security measures), any suitable remedial action may be implemented. For example, remedial actions may include ceasing one or more operations associated with program/model execution, operating one or more components associated with integrated circuit 7200 in a safe mode, Components are locked to additional input or access, etc.

74B provides a flowchart representation of a method 7450 of protecting an integrated circuit from tampering in accordance with an illustratively disclosed embodiment. For example, step 7452 may include implementing at least one security measure with respect to operation of the integrated circuit using a controller associated with the integrated circuit. At step 7454, if at least one security measure is triggered, one or more remedial actions may be taken. The integrated circuit includes: a substrate; a memory array disposed on the substrate, the memory array including a plurality of discrete memory banks; and a processing array disposed on the substrate, the processing array including a plurality of processor subunits, the plurality of processing Each of the memory subunits is associated with one or more discrete memory banks among the plurality of discrete memory banks.

In some embodiments, the disclosed security measures may be implemented in multiple memory chips, and at least one or more of the disclosed security mechanisms may be implemented for each memory chip/integrated circuit. In some cases, each memory chip/integrated circuit may implement the same security measures, but in some cases, different memory chips/integrated circuits may implement different security measures (eg, when different security measures may be more suitable for when a certain type of operation is associated with a particular integrated circuit). In some embodiments, more than one security measure may be implemented by a particular controller of the integrated circuit. For example, a particular integrated circuit may implement any number or type of disclosed security measures. Additionally, a particular integrated circuit controller may be configured to implement a number of different remedial actions in response to triggered safety measures.

It should also be noted that two or more of the above security mechanisms may be combined to improve security against network attacks or tampering attacks. Additionally, security measures may be implemented across different integrated circuits, and the integrated circuits may coordinate security measure implementation. For example, model replication can be performed within one memory chip or can be performed across different memory chips. In this example, results from one memory chip or results from two or more memory chips can be compared to detect potential network attacks or tampering attacks. In some embodiments, replication security measures applied across multiple integrated circuits may include one or more of the following: disclosed access locking mechanisms, hash protection mechanisms, model replication, program/model execution patterns analysis, or any combination of these or other disclosed embodiments.

Multiport processor subunit in DRAM

As described above, the disclosed embodiments may include a distributed processor memory chip that includes an array of processor subunits and an array of memory banks, wherein each of the processor subunits may be dedicated to a memory bank at least one of the arrays. As discussed in the following sections, distributed processor memory chips can serve as the basis for scalable systems. That is, in some cases, a distributed processor memory chip may include one or more communication ports configured to transfer data from one distributed processor memory chip to another distributed processor memory chip. In this manner, any desired number of distributed processor memory chips may be linked together (eg, in series, in parallel, in loops, or any combination thereof) to form a scalable array of distributed processor memory chips. Such an array may provide a flexible solution for efficiently performing memory-intensive operations and for scaling computing resources associated with the performance of memory-intensive operations. Because distributed processor memory chips may include clocks with different timing patterns, the disclosed embodiments include methods to accurately control data between distributed processor memory chips even in the presence of clock timing differences Transmission characteristics. Such embodiments may enable efficient data sharing among different distributed processor memory chips.

75A is a diagrammatic representation of a scalable processor memory system including multiple distributed processor memory chips consistent with an embodiment of the present invention. According to an embodiment of the present invention, a scalable processor memory system may include a plurality of distributed processor memory chips, such as a first distributed processor memory chip 7500, a second distributed processor memory chip 7500', and a third distributed processor memory chip 7500'. The processor memory chip 7500". Each of the first distributed processor memory chip 7500, the second distributed processor memory chip 7500', and the third distributed processor memory chip 7500" may be included and described in this disclosure Any of the configurations and/or features associated with any of the embodiments in a distributed processor.

In some embodiments, each of the first distributed processor memory chip 7500 , the second distributed processor memory chip 7500 ′, and the third distributed processor memory chip 7500 ″ may be similar to that shown in diagram 7200 75A, a first distributed processor memory chip 7500 may include a memory array 7510, a processing array 7520, and a controller 7540. The memory array 7510, processing array 7520, and controller 7540 may be similar It is configured in the memory array 7210, the processing array 7220 and the controller 7240 in FIG. 72A.

According to an embodiment of the present invention, the first distributed processor memory chip 7500 may include a first communication port 7530 . In some embodiments, the first communication port 7530 may be configured to communicate with one or more external entities. For example, communication port 7530 may be configured to establish communication between distributed processor memory chip 7500 and an external entity other than another distributed processor memory chip, such as distributed processor memory chips 7500' and 7500" For example, the communication port 7530 may be coupled indirectly or directly to a host computer (eg, as illustrated in Figure 72A) or any other computing device, communication module, or the like.

According to an embodiment of the invention, the first distributed processor memory chip 7500 may further include one or more additional communication ports configured to communicate with other distributed processor memory chips, eg, 7500' or 7500". In some implementations For example, the one or more additional communication ports may include a second communication port 7531 and a third communication port 7532, as shown in Figure 75A. The second communication port 7531 may be configured to communicate with the second distributed processor memory chip 7500 'communication and establishes a communication connection between the first distributed processor memory chip 7500 and the second distributed processor memory chip 7500'. Similarly, the third communication port 7532 may be configured to communicate with the third distributed processor The memory chip 7500' communicates and establishes a communication connection between the first distributed processor memory chip 7500 and the third distributed processor memory chip 7500". In some embodiments, the first distributed processor memory chip 7500 (and any of the memory chips disclosed herein) may include multiple communication ports, including any suitable number of communication ports (eg, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 50, 100, 1000, etc.).

In some embodiments, the first communication port, the second communication port, and the third communication port are associated with corresponding buses. The corresponding bus may be a bus common to each of the first communication port, the second communication port, and the third communication port. In some embodiments, corresponding buses associated with each of the first communication port, the second communication port, and the third communication port are connected to a plurality of discrete memory banks. In some embodiments, the first communication port is connected to at least one of a main bus inside the memory chip or at least one processor sub-unit included in the memory chip. In some embodiments, the second communication port is connected to at least one of a bus internal to the memory chip or at least one processor sub-unit included in the memory chip.

While the configuration of the disclosed distributed processor memory chip is explained with respect to the first distributed processor memory chip 7500, it should be noted that the second processor memory chip 7500' and the third processor memory chip 7500" may be similar to A first distributed processor memory chip 7500. For example, a second distributed processor memory chip 7500' may also include a memory array 7510', a processing array 7520', a controller 7540', and/or multiple communication ports , such as ports 7530', 7531', and 7532'. Similarly, a third distributed processor memory chip 7500" may include a memory array 7510", a processing array 7520", a controller 7540", and/or multiple communication ports, such as Ports 7530", 7531" and 7532". In some embodiments, the second communication port 7531' and the third communication port 7532' of the second distributed processor memory chip 7500' may be configured to communicate with the third distributed processor memory chip 7500" and the first distribution, respectively Similarly, the second communication port 7531" and the third communication port 7532" of the third distributed processor memory chip 7500" may be configured to communicate with the first distributed processor memory chip 7500, respectively. and the second distributed processor memory chip 7500'. This configuration similarity among distributed processor memory chips may facilitate scaling of computing systems based on the disclosed distributed processor memory chips. Additionally, the disclosed arrangement and configuration of communication ports associated with each distributed processor memory chip may enable flexible arrangement of arrays of distributed processor memory chips (eg, including series connections, parallel connections, ring connections, star connections form connection or network connection, etc.).

According to embodiments of the present invention, distributed processor memory chips such as the first through third distributed processor memory chips 7500, 7500' and 7500" may communicate with each other via a bus 7533. In some embodiments, the bus 7533 may connect Two communication ports of two different distributed processor memory chips. For example, a second communication port 7531 of a first processor memory chip 7500 can be connected via bus 7533 to a third communication port of a second processor memory chip 7500' Communication port 7532'. According to embodiments of the present invention, distributed processor memory chips such as the first to third distributed processor memory chips 7500, 7500' and 7500" can also communicate with external entities ( For example, host computer) communication. For example, the first communication port 7530 of the first distributed processor memory chip 7500 may be connected to one or more external entities via the bus 7534. Distributed processor memory chips can be connected to each other in various ways. In some cases, distributed processor memory chips may exhibit series connectivity, with each distributed processor memory chip connected to a pair of adjacent distributed processor memory chips. In other cases, distributed processor memory chips may exhibit a higher degree of connectivity, with at least one distributed processor memory chip connected to two or more other distributed processor memory chips. In some cases, all distributed processor memory chips within the plurality of memory chips may be connected to all other distributed processor memory chips within the plurality of memory chips.

As shown in Figure 75A, bus 7533 (or any other bus associated with the embodiment of Figure 75A) may be unidirectional. Although Figure 75A illustrates bus 7533 as being unidirectional with some data transfer flow (as suggested by the arrows shown in Figure 75A), bus 7533 (or any other bus in Figure 75A) may be implemented as a bidirectional bus. According to some embodiments of the present invention, a bus connected between two distributed processor memory chips may be configured to have a higher communication speed than a communication speed of a bus connected between the distributed processor memory chips and an external entity . In some embodiments, communication between distributed processor memory chips and external entities may occur during limited periods of time, such as during preparation for execution (loading of program code, input data, weighting data, etc. Occurs during a period in which results and the like resulting from the execution of the network model are output to the host computer. During execution of one or more programs associated with the distributed processors of chips 7500, 7500' and 7500" (eg, during memory-intensive operations associated with artificial intelligence applications, etc.), the distributed processors Communication between memory chips may occur via buses 7533, 7533', etc. In some embodiments, communication between distributed processor memory chips and external entities occurs compared to communication between two processor memory chips The frequency may be lower. Depending on the communication requirements and the embodiment, the bus between the distributed processor memory chips and the external entity can be configured to have a communication speed equal to, greater than, or less than the communication speed of the bus between the distributed processor memory chips speed.

In some embodiments, as represented by Figure 75A, multiple distributed processor memory chips, such as first through third distributed processor memory chips 7500, 7500' and 7500", may be configured to communicate with each other. As mentioned Thus, this capability can facilitate the assembly of scalable distributed processor memory chip systems. For example, memory arrays 7510, 7510' and 7510" from first through third processor memory chips 7500, 7500' and 7500" and The processing arrays 7520, 7520', and 7520" may be considered to actually belong to a single distributed processor memory chip when linked by a communication channel, such as the bus shown in Figure 75A.

Communications between multiple distributed processor memory chips and/or communications between distributed processor memory chips and one or more external entities may be managed in any suitable manner in accordance with embodiments of the present invention. In some embodiments, these communications may be managed by processing resources such as processing array 7520 in distributed processor memory chip 7500. In some other embodiments, distributed processor memory chips such as controllers 7540, 7540', 7540", etc., such as to relieve the computational load imposed by communication management on the processing resources provided by the array of distributed processors, The controller may be configured to manage communications between distributed processor memory chips and/or between distributed processor memory chips and one or more external entities. For example, with respect to other distributed processors The memory chips, each controller 7540, 7540' and 7540" of the first to third processor memory chips 7500, 7500' and 7500" may be configured to manage communications related to their corresponding distributed processor memory chips. In some In an embodiment, the controllers 7540, 7540' and 7540" may be configured to control these communications via corresponding communication ports such as ports 7531, 7531', 7531", 7532, 7532' and 7532".

Controllers 7540, 7540' and 7540" may also be configured to manage communications between distributed processor memory chips while taking into account timing differences that may exist between distributed processor memory chips. For example, distributed processing A processor memory chip (eg, 7500) may be fed by an internal clock that may be different relative to the clock of other distributed processor memory chips (eg, 7500' and 7500"). Accordingly, in some embodiments, controller 7540 may be configured to implement one or more strategies for accounting for different clock timing patterns among distributed processor memory chips, and by taking into account distributed processor memory chips between possible time skew to manage communication between distributed processor memory chips.

For example, in some embodiments, the controller 7540 of the first distributed processor memory chip 7500 may be configured to enable the transfer of data from the first distributed processor memory chip 7500 to the second under certain conditions Processor memory chip 7500'. In some cases, the controller 7540 may inhibit data transfer if one or more processor sub-units of the first distributed processor memory chip 7500 are not ready to transfer data. Alternatively or additionally, the controller 7540 may inhibit data transfer if the receive processor subunit of the second distributed processor memory chip 7500' is not ready to receive data. In some cases, the controller 7540 may initiate data transfer from the transmit processor sub-unit (eg, in chip 7500) after determining that the transmit processor sub-unit is ready to transmit data and the receive processor sub-unit is ready to receive data to the receive processor subunit (eg, in chip 7500'). In other embodiments, the controller 7540 may initiate a data transfer based solely on whether the transmit processor subunit is ready to send data, especially where data may be buffered in the controller 7540 or 7540', eg, until the receive processor subunit is ready When receiving the transmitted data.

According to an embodiment of the invention, the controller 7540 may be configured to determine whether one or more other timing constraints are satisfied in order to enable data transfer. Such time constraints may be related to the time difference from the transmit time of the transmit processor sub-unit to the receive time in the receive processor sub-unit, from external entities (eg, host computers) to processing data. Access requests, refresh operations performed on memory resources (eg, memory arrays) associated with transmit or receive processor subunits, and others.

75E is an example timing diagram consistent with embodiments of the present invention. Figure 75E illustrates the following example.

In some embodiments, controller 7540 and other controllers associated with distributed processor memory chips may be configured to use clock enable signals to manage data transfers between chips. For example, the processing array 7520 may be fed by a clock. In some embodiments, whether one or more processor sub-units are responsive to a supplied clock signal may be controlled, eg, by controller 7540 using a clock enable signal (eg, shown in Figure 75A as "to CE"). Each processor sub-unit, eg, 7520_1 through 7520_K, can execute program code, and the program code may include communication commands. According to some embodiments of the present invention, the controller 7540 may control the timing of communication commands by controlling the clock enable signals to the processor subunits 7520_1 to 7520_K. For example, according to some embodiments, when the sending processor sub-unit (eg, in the first processor memory chip 7500) is programmed to transmit data at a certain cycle (eg, the 1000th clock cycle) and the receiving processor The subunit (eg, in the second processor memory chip 7500') is programmed to receive data at a certain cycle (eg, the 1000th clock cycle), the controller 7540 of the first processor memory chip 7500 and the second The controller 7540' of the processor memory chip 7500' may not allow the data transfer until both the transmitting processor sub-unit and the receiving processor sub-unit are ready to perform the data transfer. For example, the controller 7540 may "suppress" data transfers from the transmit processor subunit by supplying a certain clock enable signal (eg, logic low) to the transmit processor subunit in the chip 7500, the clock enable signal The transmit processor subunit may be prevented from transmitting data in response to the received clock signal. A certain clock enable signal can "freeze" the entire distributed processor memory chip or any portion of the distributed processor memory chip. On the other hand, the controller 7540 can cause the transmit processor subunit to initiate a data transfer by supplying an opposite clock enable signal (eg, logic high) to the transmit processor subunit that causes the transmit processor subunit to Respond to the received clock signal. Similar operations may be controlled using a clock enable signal issued by controller 7540', eg, reception or non-reception by a receive processor sub-unit in chip 7500'.

In some embodiments, the clock enable signal may be sent to all processor subunits (eg, 7520_1 through 7520_K) in a processor memory chip (eg, 7500). In general, clock enable signals may have the effect of causing processor subunits to respond to their respective clock signals or to ignore those clock signals. For example, in some cases, when the clock enable signal is high (depending on the conventions of a particular application), the processor sub-unit may respond to its clock signal and may execute one or more instructions according to its clock signal timing. On the other hand, when the clock enable signal is low, the processor subunit is prevented from responding to its clock signal so that it does not execute instructions in response to the clock timing. In other words, the processor subunit may ignore the received clock signal when the clock enable signal is low.

Returning to the example of FIG. 75A, any of the controllers 7540, 7540' or 7540" may be configured to use a clock enable signal to cause one or more processor subunits in the respective array to respond to the received clock The signals respond or do not respond to control the operation of the respective distributed processor memory chips. In some embodiments, the controller 7540, 7540' or 7540" may be configured to selectively advance program code execution, such as herein When code is related to or includes data transfer operations and their timing. In some embodiments, the controller 7540, 7540' or 7540" may be configured to use a clock enable signal to control communication between two different distributed processor memory chips via communication ports 7531, 7531', 7531", 7532, 7532' and 7532", etc., the timing of data transfers. In some embodiments, the controller 7540, 7540' or 7540" may be configured to use a clock enable signal to control two different distributed processors Time of data reception between memory chips via any of communication ports 7531, 7531', 7531", 7532, 7532' and 7532", etc.

In some embodiments, the timing of data transfers between two different distributed processor memory chips may be configured based on a compilation optimization step. Compilation can allow for the construction of handlers in which tasks can be efficiently assigned to processing subunits without being affected by propagation delays on buses connecting two different processor memory chips. Compilation can be performed by a compiler program in the host computer, or transferred to the host computer. Often, transfer delays on the bus between two different processor memory chips will cause a data bottleneck for the processing subunits that require the data. The disclosed compilation may schedule data transfers in a manner that enables a processing unit to continuously receive data even with unfavorable transfer delays on the bus.

Although the embodiment of Figure 75A includes three ports for each distributed processor memory chip (7500', 7500", 7500"'), any number of ports may be included in the distributed processor memory in accordance with the disclosed embodiments in the chip. For example, in some cases, distributed processor memory chips may include more or fewer ports. In the embodiment of Figure 75B, each distributed processor memory chip (eg, 7500A-7500I) may be configured with multiple ports. These ports may be substantially the same as each other or may be different. In the example shown, each distributed processor memory chip includes five ports, including one host communication port 7570 and four chip ports 7572. The host communication port 7570 can be configured to communicate between any of the distributed processors in the array (as shown in FIG. 75B ) and a host computer remotely located, for example, with respect to the array of distributed processor memory chips (via bus 7534). Chip port 7572 may be configured to enable communication between distributed processor memory chips via bus 7535 .

Any number of distributed processor memory chips may be connected to each other. In the example shown in Figure 75B including four chip ports per distributed processor, an array may be implemented in which each distributed processor memory chip is connected to two or more other distributed processors processor memory chips, and in some cases some chips may be connected to four other distributed processor memory chips. Including more chip ports in a distributed processor memory chip enables more interconnectivity between distributed processor memory chips.

Additionally, although distributed processor memory chips 7500A-7500I are shown in FIG. 75B as having two different types of communication ports 7570 and 7572, in some cases a single type of communication port may be included in each distributed processing in the memory chip. In other cases, more than two different types of communication ports may be included in one or more of the distributed processor memory chips. In the example of FIG. 75C, each of the distributed processor memory chips 7500A'-7500C' includes two (or more than two) communication ports 7570 of the same type. In this embodiment, the communication port 7570 may be configured to enable communication with an external entity, such as a host computer, via the bus 7534, and may also be configured to enable communication between distributed processor memory chips (eg, via the bus 7535) , between distributed processor memory chips 7500B' and 7500C').

In some embodiments, ports provided on one or more distributed processor memory chips may be used to provide access to more than one host. For example, in the embodiment shown in FIG. 75D, the distributed processor memory chip includes two or more ports 7570. Port 7570 may constitute a host port, a chip port, or a combination of host port and chip port. In the example shown, two ports 7570 and 7570' may enable two different hosts (eg, host computers or computing elements or other types of logic units) to access distributed processor memory chips via buses 7534 and 7534' 7500A. This embodiment may enable two (or more than two) different host computers to access the distributed processor memory chip 7500A. However, in other embodiments, both buses 7534 and 7534' may be connected to the same host entity, such as where the host entity requires additional bandwidth or to one of the processor subunits/memory banks of distributed processor memory chip 7500A or multiple concurrent accesses.

In some cases, as shown in Figure 75D, more than one controller 7540 and 7540' may be used to control access to the distributed processor subunits/memory banks of distributed processor memory chip 7500A. In other cases, a single controller may be used to handle communications from one or more external host entities.

Additionally, one or more buses within the distributed processor memory chip 7500A may enable parallel access to the distributed processor subunits/memory banks of the distributed processor memory chip 7500A. For example, the distributed processor memory chip 7500A may include a first bus 7580 and a second bus 7580' that enable access to, for example, distributed processor subunits 7520_1 to 7520_6 and their corresponding dedicated memory banks 7510_1 to 7510_6 parallel access. This configuration may allow simultaneous access to two different locations in the distributed processor memory chip 7500A. Additionally, without using all ports at the same time, the ports may share hardware resources (eg, a common bus and/or a common controller) within the distributed processor memory chip 7500A, and may constitute multiplexing (mux) to IO of this hardware.

In some embodiments, some of the arithmetic units (eg, processor sub-units 7520_1 to 7520_6) may be connected to additional ports (7570') or controllers, while others are not connected to additional ports or controllers. However, data from arithmetic units not connected to additional ports 7570' may pass through an internal grid of connections to arithmetic units connected to port 7570'. In this way, communication can be performed at both ports 7570 and 7570' simultaneously without adding an additional bus.

Although the communication ports (eg, 7530-7532) and the controller (eg, 7540) have been illustrated as separate components, it should be understood that the communication ports and the controller (or any other component) may be implemented as integrated in accordance with embodiments of the present invention unit. 76 provides a diagrammatic representation of a distributed processor memory chip 7600 with integrated controller and interface modules, consistent with embodiments of the present invention. As shown in FIG. 76, the processor memory chip 7600 may be implemented with an integrated controller and interface module 7547 configured to perform the functions of the controller 7540 and communication ports 7530, 7531 and 7532 in FIG. As shown in FIG. 76, a controller and interface module 7547 is configured to communicate with external entities, such as one or more distributed processor memory chips, via interfaces 7548_1 through 7548_N similar to communication ports (eg, 7530, 7531, and 7532) communication with multiple different entities, etc. The controller and interface module 7547 may be further configured to control communications between the distributed processor memory chips or between the distributed processor memory chips 7600 and an external entity such as a host computer. In some embodiments, controller and interface module 7547 may include communication interfaces 7548_1 to 7548_N configured to communicate in parallel with one or more other distributed processor memory chips and with external entities such as host computers, communication modules, etc. .

77 provides a flowchart representing a process for transferring data between distributed processor memory chips in the scalable processor memory system shown in FIG. 75, consistent with an embodiment of the present invention. For illustrative purposes, a flow for transferring data will be described with reference to FIG. 75 and assume that data is transferred from a first processor memory chip 7500 to a second processor memory chip 7500'.

At step S7710, a data transfer request may be received. It should be noted, however, and as described above, that in some embodiments, a data transfer request may not be necessary. For example, in some cases, the timing of data transfers may be predetermined (eg, by specific software code). In this situation, the data transfer can continue without a separate data transfer request. Step S7710 may be performed by, for example, the controller 7540 and others. In some embodiments, the data transfer request may include a request to transfer data from one processor subunit of the first distributed processor memory chip 7500 to another processor subunit of the second distributed processor memory chip 7500' .

At step S7720, the data transfer timing may be determined. As mentioned, the data transfer timing may be predetermined and may depend on the order of execution of particular software programs. Step S7720 may be performed by, for example, the controller 7540 and others. In some embodiments, data transfer timing may be determined by considering (1) whether the transmit processor subunit is ready to transmit data and/or (2) whether the receive processor subunit is ready to receive data. Whether one or more other timing constraints are satisfied to enable this data transfer may also be considered in accordance with embodiments of the present invention. One or more time constraints may be related to: the time difference from the transmit time of the transmit processor subunit to the receive time at the receive processor subunit, the Access requests for data, refresh operations performed on memory resources (eg, memory arrays) associated with transmit or receive processor subunits, and the like. According to an embodiment of the invention, the processing subunit may be fed by a clock. In some embodiments, the clock supplied to the processing subunit may be controlled, eg, using a clock enable signal. According to some embodiments of the present invention, the controller 7540 may control the timing of communication commands by controlling the clock enable signals to the processor sub-units 7520_1 to 7520_K.

At step S7730, data transfer may be performed based on the data transfer timing determined at step S7720. Step S7730 may be performed by, for example, the controller 7540 and others. For example, the transmit processor subunit of the first processor memory chip 7500 may transmit data to the receive processor subunit of the second processor memory chip 7500' according to the data transfer timing determined at step S7720.

The disclosed architecture is applicable to a variety of applications. For example, in some cases, the above architecture may facilitate sharing data among different distributed processor memory chips, such as weights or neuron values or partial neuron values associated with neural networks, especially large neural networks. Additionally, data from multiple different distributed processor memory chips may be required in certain operations such as SUM, AVG, and the like. In this case, the disclosed architecture may facilitate sharing this data from multiple distributed processor memory chips. Still further, for example, the disclosed architecture may facilitate sharing of records among distributed processor memory chips to support splicing operations for queries.

It should also be noted that although the above embodiments have been described with respect to distributed processor memory chips, the same principles and techniques may be applied, for example, to conventional memory chips that do not include distributed processor subunits. For example, in some cases, multiple memory chips may be combined together into a multi-port memory chip to form an array of memory chips that do not even have an array of processor subunits. In another embodiment, multiple memory chips can be grouped together to form an array of connected memories, effectively providing the host with one larger memory containing multiple memory chips.

The port's internal connection may be to the main bus or to one of the internal processor subunits included in the processing array.

In-memory zero detection

Some embodiments of the present invention relate to memory cells for detecting zero values stored in one or more specific addresses of a plurality of memory banks. This zero value detection feature of the disclosed memory cells may be useful for reducing power consumption of computing systems, and additionally or alternatively, may also reduce the processing time required for retrieving zero values from memory. This feature may be especially relevant in systems where a large amount of data read is actually 0-valued and also used for computational operations, such as multiplication\addition\subtraction\ and more operations, for which self- Memory fetching of zero values may not be necessary (eg, the product of a zero value and any other value is zero), and the arithmetic circuit may use the fact that one of the operands is zero and compute the result more efficiently in time and energy. In such cases, the detection of the presence of zero values may be used in place of memory accesses and retrieving zero values from memory.

Throughout this section, the disclosed embodiments are described with respect to read functionality. It should be noted, however, that the disclosed architecture and techniques are equally applicable to zero-valued write operations, or other specific predetermined non-zero-valued operations where other values may occur more often.

In the disclosed embodiments, instead of retrieving a zero value from memory, when this value is detected at a particular address, the memory cell may communicate a zero value indicator back to one or more circuits external to the memory cell (eg, one or more processors, CPUs, etc.) located outside the memory unit. A zero value is a multi-bit zero-value zero (eg, a zero-value byte, a zero-value word, a multi-bit zero value less than one byte, greater than one byte, and the like). The zero value indicator is a 1-bit signal that hints at a zero value stored in memory, so it is beneficial to transmit a 1-bit zero value for an indication signal rather than transmitting n data bits stored in memory. The transmitted zero hints can reduce the energy consumption for transmission to 1/n and can speed up operations, such as where inputs are computed by the weights of neurons, convolutions, applying cores to input data, and interfacing with trained neurons. Multiplication is involved in many other computations associated with networking, artificial intelligence, and a wide range of other types of operations. To provide this functionality, the disclosed memory cells can include one or more zero-value detection logic cells that can detect the presence of a zero value in a particular location in the memory, preventing fetching A zero value (eg, via a read command) and causes the zero value indicator to be communicated instead to circuitry external to the memory cell (eg, using one or more control lines of the memory, one or more associated with the memory cell bus, etc.). Zero detection may be performed at the memory pad level, at the bank level, at the subgroup level, at the chip level, and so on.

It should be noted that while the disclosed embodiments are described with respect to delivering a zero indicator to a location external to the memory chip, the disclosed embodiments and features may also provide significant benefits in processing systems that may be internal to the memory chip. For example, in embodiments such as the distributed processor memory chips disclosed herein, processing may be performed on data in various memory banks by corresponding processor subunits. In many situations, such as neural network execution or data analysis, where the associated data may include many zeros, the disclosed techniques may speed up processing and/or reduce processing associated with processing performed by processor sub-units in distributed processor memory chips of power consumption.

78A illustrates a system 7800 for detecting, at the chip level, zero values stored in one or more specific addresses of a plurality of memory banks implemented in a memory chip 7810, in accordance with an embodiment of the invention. System 7800 may include memory chip 7810 and host 7820. The memory chip 7810 may include multiple control units and each control unit may have a dedicated memory bank. For example, the control unit may be operably connected to a dedicated memory bank.

In some cases, such as with respect to the distributed processor memory chips disclosed herein, processing within the memory chips may involve memory accesses (whether reads or writes) that are included in the space processor sub-units distributed among an array of memory banks. Even in the case of processing inside the memory chip, the disclosed techniques of detecting zero values associated with read or write commands may allow internal processor units or subunits to forego transmitting actual zero values. Indeed, in response to zero value detection and zero value indicator transmission (eg, to one or more internal processing sub-units), distributed processor memory chips can save zero data values that would otherwise have been used to transmit zero data values within the memory chip energy of.

In another example, each of memory chip 7810 and host 7820 may include input/output (IO) to enable communication between memory chip 7810 and host 7820. Each IO may be coupled with a zero value indicator line 7830A and a bus 7840A. The zero-value indicator line 7830A may transmit a zero-value indicator from the memory chip 7810 to the host 7820, where the zero-value indicator may include a zero value stored in a particular address of a memory bank requested by the host 7820 after detection of a zero value. 1-bit signal generated by memory chip 7810. Upon receiving the zero-value indicator via the zero-value indicator line 7830A, the host 7820 may perform one or more predefined actions associated with the zero-value indicator. For example, if the host 7820 requests the memory chip 7810 to retrieve operands for multiplication, the host 7820 can compute the multiplication more efficiently because the host 7820 will acknowledge from the received zero-value indicator (no actual memory is received) value) one of the operands is zero. Host 7820 may also provide instructions, data, and other inputs to memory chip 7810 via bus 7840, and read outputs from memory chip 7810. After receiving the communication from the host 7820, the memory chip 7810 can retrieve the data associated with the received communication and transmit the retrieved data to the host 7820 via the bus 7840.

In some embodiments, the host may send a zero value indicator to the memory chip instead of a zero data value. In this way, a memory chip (eg, a controller disposed on the memory chip) can store or refresh zero values in memory without having to receive zero data values. This update may occur based on receipt of a zero-value indicator (eg, as part of a write command).

78B illustrates a memory chip 7810 for detecting, at the memory bank level, zero values stored in one or more specific addresses of a plurality of memory banks 7811A-7811B, in accordance with an embodiment of the invention. The memory chip 7810 may include a plurality of memory banks 7811A-7811B and an IO bus 7812. Although FIG. 78B depicts two memory banks 7811A-7811B implemented in memory chip 7810, memory chip 7810 may include any number of memory banks.

IO bus 7812 can be configured to transfer data to/from an external chip (eg, host 7820 in Figure 78A) via bus 7840B. Bus 7840B may function similarly to bus 7840A in Figure 78A. IO 7812 may also transmit a zero-value indicator via zero-value indicator line 7830B, which may function similarly to zero-value indicator line 7830A in Figure 78A. IO bus 7812 may also be configured to communicate with memory banks 7811A-7811B via internal zero-value indicator line 7831 and bus 7841. IO bus 7812 may transfer received data from external chips to one of memory banks 7811A-7811B. For example, IO bus 7812 may transmit data via bus 7841 including instructions to read data stored in a particular address of memory bank 7811A. A multiplexer may be included between IO bus 7812 and memory banks 7811A-7811B, and may be connected by internal zero indicator line 7831 and bus 7841A. The multiplexer may be configured to transfer received data from the IO bus 7812 to a particular memory bank, and may be further configured to transfer received data or a received zero value indicator from the particular memory bank to the IO bus 7812.

In some cases, the host entity may only be configured to receive regular data transmissions, and may not be equipped to interpret or respond to the disclosed zero-value indicator. In this case, the disclosed embodiments (eg, controller/chip IO, etc.) can regenerate a zero value on the data line to the host IO in place of the zero value indicator signal, and thus can save data transfer power inside the chip .

Each of the memory banks 7811A-7811B may include a control unit. The control unit may detect a zero value stored in the requested address of the memory bank. After detecting the stored zero value, the control unit may generate a zero value indicator and transmit the generated zero value indicator to the IO bus 7812 via the internal zero value indicator line 7831, where the zero value indicator is indicated via the zero value value The hook line 7830B is further transmitted to the external chip.

79 illustrates a memory bank 7911 for detecting, at the memory pad level, zero values stored in one or more of specific addresses of a plurality of memory pads, in accordance with an embodiment of the present invention. In some embodiments, memory bank 7911 can be organized into memory pads 7912A-7912B, each of which can be independently controlled and independently accessed. Memory bank 7911 may include memory pad controllers 7913A-7913B, which may include zero value detection logic units 7914A-7914B. Each of the memory pad controllers 7913A-7913B may allow reading and writing to locations on the memory pads 7912A-7912B. Memory bank 7911 may further include read disable components, regional sense amplifiers 7915A-7915B, and/or global sense amplifier 7916.

Each of memory pads 7912A-7912B may include multiple memory cells. Each of the plurality of memory cells can store one bit of binary information. For example, any of the memory cells may individually store a zero value. If all memory cells in a particular memory pad store zero values, then the zero values may be associated with the entire memory pad.

Each of the memory pad controllers 7913A-7913B can be configured to access the dedicated memory pads, and to read or write data stored in the dedicated memory pads.

In some embodiments, zero value detection logic unit 7914A or 7914B may be implemented in memory bank 7911. One or more zero-value detection logic units 7914A-7914B may be associated with memory banks, memory sub-banks, memory pads, and sets of one or more memory cells. The zero value detection logic unit 7914A or 7914B may detect that a particular address (eg, memory pad 7912A or 7912B) is requested to store a zero value. This detection can be performed in a number of ways.

A first method may include using a digital comparator relative to zero. The digital comparator may be configured to take two numbers as inputs in binary form, and to determine whether the first number (the captured data) is equal to the second number (zero). If the digital comparator determines that the two numbers are equal, the zero value detection logic unit may generate a zero value indicator. The zero value indicator can be a 1-bit signal and can cause the data bits to be sent to amplifiers (eg, regional sense amplifiers 7915A-7915B), transmitters, and The buffer is deactivated. The zero value indicator may be further transmitted to the global sense amplifier 7916 via the zero value indicator line 7931A or 7931B, but in some cases, the global sense amplifier may be bypassed.

A second method for zero detection may include using an analog comparator. Analog comparators can also function similarly to digital comparators, except that the voltages of the two analog inputs are used for comparison. For example, all bits can be sensed, and the comparator can act as a logical OR (OR) function between the signals.

A third method for zero detection may include using transmit signals from regional sense amplifiers 7915A-7915B into a global sense amplifier 7916, where the global sense amplifier 7916 is configured to sense whether any of the inputs are is high (non-zero) and this logic signal is used to control the next stage of the amplifier. Regional sense amplifiers 7915A-7915B and global sense amplifier 7916 may include multiple transistors configured to sense low power signals from multiple memory banks, and the amplifiers amplify small voltage swings to higher voltages. The high voltage level enables data stored in multiple memory banks to be interpreted by at least one controller, such as memory pad controller 7913A or 7913B. For example, memory cells may be arranged on memory bank 7911 in rows and columns. Each line can be attached to each memory cell in the row. The lines running along the row are called word lines, which are activated by selectively applying voltages to the word lines. The lines running along the columns are called bit lines, and two such complementary bit lines can be attached to sense amplifiers at the edges of the memory array. The number of sense amplifiers may correspond to the number of bit lines (columns) on the memory bank 7911. To read a bit from a particular memory cell, a word line along a row of cells is turned on, enabling all memory cells in that row. The stored value (0 or 1) from each cell is then available on the bit line associated with the particular cell. At the ends of the two complementary bit lines, sense amplifiers can amplify small voltages to normal logic levels. The bits from the desired cell can then be latched from the cell's sense amplifier into a buffer and placed on the output bus.

A fourth method for zero value detection may include: if the value is 0, use an extra bit for each word saved to memory and stored at write time, and use the extra bit when reading data to know Whether the data is zero. This method avoids writing all zeros to memory, thus saving more energy.

As described above and throughout this disclosure, some embodiments may include a memory unit, such as memory unit 7800, that includes a plurality of processor sub-units. These processor subunits may be spatially distributed on a single substrate (eg, a substrate such as a memory chip of memory cell 7800). Furthermore, each of the plurality of processor sub-units may be dedicated to a corresponding memory bank among the plurality of memory banks of memory unit 7800. And these memory banks dedicated to the corresponding processor sub-units can also be spatially distributed on the substrate. In some embodiments, memory unit 7800 may be associated with a particular task (eg, performing one or more operations associated with running a neural network, etc.), and each of the processor sub-units of memory unit 7800 may be responsible for perform part of this task. For example, each processor sub-unit may be equipped with instructions that may include data manipulation and memory operations, arithmetic and logical operations, and the like. In some cases, zero-value detection logic may be configured to provide a zero-value indicator to one or more of the described processor sub-units that are spatially distributed over memory unit 7800.

Referring now to FIG. 80, a flowchart illustrating an exemplary method 8000 of detecting zero values stored in particular addresses of a plurality of memory banks in accordance with embodiments of the present invention. Method 8000 may be performed by a memory chip (eg, memory chip 7810 of Figure 78B). In particular, a controller of a memory cell (eg, controller 7913A of FIG. 79 ) and a zero value detection logic unit (eg, zero value detection logic unit 7914A) may perform method 8000 .

In step 8010, a read or write operation may be initiated by any suitable technique. In some cases, the controller may receive a request to read data stored in a particular address of a plurality of discrete memory banks (eg, the memory banks depicted in FIG. 78). The controller may be configured to control at least one aspect of read/write operations with respect to the plurality of discrete memory banks.

In step 8020, one or more zero value detection circuits may be used to detect the presence of a zero value associated with a read or write command. For example, a zero value detection logic unit (eg, zero value detection logic unit 7830 of Figure 78) may detect a zero value associated with a particular address associated with a read or write.

In step 8030, the controller may transmit a zero value indicator to one or more circuits external to the memory cell in response to the zero value detection by the zero value detection logic unit in step 8020. For example, the zero value detection logic can detect that the requested address stores a zero value, and can transmit a zero value hint to outside the memory chip (or within the memory chip, such as in the disclosed distributed processor memory chip) An entity (eg, one or more circuits) comprising processor subunits distributed among an array of memory banks. If the zero value associated with the read or write command is not detected, the controller may transmit a data value instead of a zero value indicator. In some embodiments, the one or more circuits that are passed back to the zero value indicator may be internal to the memory cell.

Although the disclosed embodiments have been described with respect to zero value detection, the same principles and techniques will apply to detecting other memory values (eg, 1, etc.). In some cases, in addition to a zero value indicator, the detection logic may also return one or more indicators of other values (eg, 1, etc.) associated with the read or write command, and these indicators may Returned/transmitted if any value corresponding to the value indicator is detected. In some cases, these values may be adjusted by the user (eg, by updating one or more registers). Such an update may be particularly useful in situations where it is possible to know properties about the dataset and to know (eg, to the user) that some values may be more prevalent in the data than others. In this case, one, two, three, or more than three value indicators may be associated with the most prevalent data associated with the dataset.

Compensate for DRAM startup penalty

In some types of memory (eg, DRAM), memory cells may be arranged in arrays within memory banks, and memory cells may be accessed and retrieved (reads) included in memory cells at a time for one row of memory cells in the array. The value in the element. This read process may involve first opening (enabling) a line (or row) of memory cells to make available the data values stored by the memory cells. Next, the values of memory cells in open rows can be sensed simultaneously, and column addresses can be used to cycle through individual memory cell values or groups of memory cell values (ie, words), and each memory cell The cell value is connected to the external data bus for reading the memory cell value. These handlers are time consuming. In some cases, opening a memory bank for reading may require 32 cycles of computation time, and reading a value from an open bank may require another 32 cycles. Significant latency can result if the next row to be read is only opened after the read operation of the currently open row is completed. In this example, during the 32 cycles required to open the next row, no data is read, and reading each row effectively requires a total of 64 cycles rather than just 32 cycles to traverse the row data. Conventional memory systems do not allow a second bank in the same bank to be opened while the first bank is being read or written. To save latent time, the next row to be opened may therefore be in a different group o in a special group for dual row access, as discussed in further detail below. Before opening the next row, the current row can all be sampled to a flipflop or latch, and when the next row can be opened, all processing is done on the flipflop\latch. If the next prediction is in the same group (and none of the above situations exist), latent times may not be avoided and the system may need to wait. These mechanisms are relevant both to standard memory and in particular to memory processing devices.

The disclosed embodiments may reduce this latency by, for example, predicting the next memory bank to be opened before the read operation of the currently open memory bank has completed. That is, if the next row to be opened can be predicted, the processing for opening the next row can be started before the read operation of the current row has completed. Depending on when the next row prediction is made in the process, the latency associated with opening the next row may be reduced from 32 cycles (in the specific example described above) to less than 32 cycles. In one particular example, if the next row opening is predicted 20 cycles ahead, the extra latency is only 12 cycles. In another example, if the next row is predicted to open 32 cycles ahead, there is no latent time at all. As a result, instead of requiring a total of 64 cycles to open and read each row serially, the effective time to read each row can be reduced by opening the next row while the current row is being read.

The following mechanisms may require the current row and the predicted row to be in the same group, but they can also be used if there is such a group that can support both starting and working on a row.

In the disclosed embodiments, next row prediction may be performed using various techniques (discussed in more detail below). For example, the next row prediction can be based on pattern recognition, based on a predetermined row access schedule, based on the output of an artificial intelligence model (eg, a trained neural network to analyze row access and make predictions of the next row to be opened), or Based on any other suitable forecasting technique. In some embodiments, 100% successful prediction can be achieved by using a delayed address generator or formula or other methods as described below. Predicting may include building a system with the ability to sufficiently predict the next row to be opened before the next row needs to be accessed. In some cases, next-row prediction may be performed by a next-row predictor, which may be implemented in various ways. For example, a predicted address generator to generate a current address for reading and/or writing a memory row. The entity that generates addresses for accessing memory (read or write) may be based on any logic circuit or controller\CPU executing software instructions. The predicted address generator may include a pattern learning model that observes accessed rows, identifies and accesses (eg, sequential accesses, accesses for every second row, accesses for every third row accesses, etc.) associated with one or more patterns and estimate the next row to be accessed based on the observed patterns. In other examples, the predicted address generator may include applying a formula/algorithm to predict the next row of cells to be accessed. In still other embodiments, the predictive address generator may include a trained neural network based on, for example, the current address row being accessed, the last 2, 3, 4, or more than 4 accessed input of an address/row etc. to output the predicted next row to be accessed (including one or more addresses associated with the predicted row). Using any of the described predicted address generators to predict the next memory rank to be accessed can significantly reduce latency associated with memory accesses. The predicted address/row generator described is applicable in any system involving accessing memory to retrieve data. In some cases, the described predictive address/row generator and associated techniques for predicting next memory bank accesses may be particularly suitable in systems that execute artificial intelligence models, since AI models can be combined with can facilitate next row predictions associated with a repeating memory access pattern.

81A illustrates a system 8100 for starting a next row associated with a memory bank 8180 based on next row prediction, consistent with embodiments of the invention. System 8100 may include current and predicted address generator 8192, bank controller 8191, and memory banks 8180A-8180B. An address generator may be an entity that generates addresses for accessing memory banks 8180A-8180B, and may be based on any logic circuit, controller, or microprocessor executing a software program. Bank controller 8191 may be configured to access the current row of memory bank 8180A (eg, using the current row identifier generated by address generator 8192). The bank controller 8191 may also be configured to enable the predicted next row to be accessed within the memory bank 8180B based on the predicted row identifier generated by the address generator 8192. The following examples describe two groups. In other instances, more groups may be used. In some embodiments, there may be memory banks that allow access to more than one row at a time (as discussed below), and thus the same process may be performed on a single bank. As described above, the initiation of the predicted next row to be accessed may begin before the completion of a read operation performed relative to the current row being accessed. Thus, in some cases, the address generator 8192 can predict the next row to be accessed, and can use an identifier (eg, one or more of the predicted next row) at any time before the access to the current row has completed. address) to the group controller 8191. This timing may allow the group controller to initiate activation of the predicted next row at any point in time while the current row is being accessed and before the access to the current row is complete. In some cases, the bank controller 8291 may initiate the activation of the memory bank 8180 at the same time (or within a few clock cycles) that the activation of the current row to be accessed is complete and/or a read operation relative to the current row has begun. The predicted start of the next line.

In some embodiments, the operation relative to the current row associated with the current address may be a read or write operation. In some embodiments, the current row and the next row may be in the same memory bank. In some embodiments, the same memory bank may allow the next row to be accessed while the current row is being accessed. The current row and the next row can be in different memory banks. In some embodiments, the memory unit may include a processor configured to generate the current address and the predicted address. In some embodiments, the memory unit may comprise a distributed processor. A distributed processor may include multiple processor subunits of the processing array that are spatially distributed among multiple discrete memory banks of the memory array. In some embodiments, the predicted address may be generated by a series of flip-flops that sample the delayed generated address. The delay may be configurable via a multiplexer that selects between flip-flops that store sampled addresses.

It should be noted that the predicted next row may become the current row to be accessed after confirming that the predicted next row is actually the next row requested to be accessed by executing software (eg, after completing a read operation relative to the current row) . In the disclosed embodiments, because the handler for initiating the predicted next row may be initiated before the current row read operation is completed, after confirming that the predicted next row is the correct next row to be accessed, it may be possible to Fully or partially start the next line to be accessed. This can significantly reduce the latency associated with platoon activation. A power reduction can be achieved if the next row is activated such that the activation ends before or at the same time as the read of the current row ends.

The current and predicted address generator 8192 may include a row configured to identify a row to be accessed in the memory bank 8180 (eg, based on program execution) and predict the next row to be accessed (eg, based on an observed pattern in row access, Any suitable logic components, arithmetic units, memory units, algorithms, trained models, etc. based on predetermined patterns (n+1, n+2, etc.). For example, in some embodiments, the current and predicted address generator 8192 may include a counter 8192A, a current address generator 8192B, and a predicted address generator 8192C. The current address generator 8192B may be configured to generate the current address of the current row to be accessed in the memory bank 8180 based on the output of the counter 8192A, eg, based on a request from the arithmetic unit. The address associated with the current row to be accessed may be provided to the group controller 8191. The predicted address generator 8192C may be configured to determine the pending memory bank 8180 based on the output of the counter 8192A, based on a predetermined access pattern (eg, in conjunction with the counter 8192A), or based on the output of a trained neural network or other type of pattern prediction algorithm. The predicted address of the next row to be fetched, the pattern prediction algorithm observes the row access and predicts the next row to be accessed based, for example, on the pattern associated with the observed row access. The address generator 8192 may provide the predicted next row address from the predicted address generator 8192C to the bank controller 8191.

In some embodiments, the current address generator 8192B and the predicted address generator 8192C may be implemented internal or external to the system 8100. External hosts may also be implemented external to system 8100 and further connected to system 8100. For example, the current address generator 8192B may be software at an external host executing the program, and the predicted address generator 8192C may be implemented either internal to the system 8100 or external to the system 8100 to avoid any latency.

As mentioned, the predicted next row address may be determined using a trained neural network that predicts the next row to be accessed based on an input that may include one or more previously accessed row addresses. A trained neural network or other type of model may operate within the logic associated with the predicted address generator 8192C. In some cases, trained neural networks, etc., may be performed by one or more arithmetic units external to, but in communication with, predictive address generator 8192C.

In some embodiments, the predicted address generator 8192C may comprise a replica or a substantial replica of the current address generator 8192B. Additionally, the timing of the operations of the current address generator 8192B and the predicted address generator 8192C may be fixed or adjustable relative to each other. For example, in some cases, predictive address generator 8192C may be configured to issue address identifiers associated with the next row to be accessed at a fixed time (eg, a fixed number of clocks) relative to current address generator 8192B loop) output the address identifier associated with the predicted next row. In some cases, before or after the start of the current row to be accessed begins, before or after the start of a read operation associated with the current row to be accessed, or before or after the start of the current row being accessed The predicted next row identifier can be generated any time before the read operation is complete. In some cases, the predicted next row identifier may be generated at the same time as the initiation of the current row to be accessed begins or at the beginning of a read operation associated with the current row to be accessed.

In other cases, the time between the generation of the predicted next row identifier and the initiation of the current row to be accessed or the initiation of a read operation associated with the current row may be adjustable. For example, under some conditions, this time may be extended or shortened during operation of memory cell 8100 based on values associated with one or more operating parameters. In some cases, the current temperature (or any other parameter value) associated with the memory cell or another component of the computing system may cause the current address generator 8192B and the predicted address generator 8192C to change their relative operating timings. In embodiments where in memory processing, the prediction mechanism may be part of the logic.

The current and predicted address generator 8192 can generate the confidence associated with the predicted next row to access the decision. This confidence level, which may be determined by the prediction address generator 8192C as part of the prediction handler, may be used to determine, for example, whether during a read operation of the current row (ie, before the current row read operation has completed and pending storage) fetched before the identification of the next line has been confirmed) to start the predicted start of the next line. For example, in some cases, the confidence associated with the predicted next row to be accessed may be compared to a threshold level. If the confidence level falls below a threshold level, for example, the memory unit 8100 may forego starting the predicted next row. On the other hand, if the confidence level exceeds a threshold level, memory cell 8100 may initiate activation of the predicted next row in memory bank 8180.

The mechanism to test the confidence of the predicted next row relative to a threshold level and the subsequent initiation or non-initiation of the predicted initiation of the next row can be implemented in any suitable manner. Under some conditions, for example, if the confidence associated with the predicted next row falls below a threshold, the predicted address generator 8192C may forego outputting its predicted next row result to downstream logic components. Alternatively, in this case, the current and predicted address generator 8192 may suppress the predicted next row identifier from the group controller 8191, or the group controller (or another logic unit) may be equipped to use the predicted next row The confidence level to determine whether to start the predicted next row before the read operation associated with the current row being read is completed.

The confidence associated with the predicted next row may be generated in any suitable manner. In some situations, such as where the predicted next row is identified based on a predetermined known access pattern, the predicted address generator 8192C may generate a high confidence level or may forgo generating a confidence level entirely given the predetermined pattern of row accesses. On the other hand, one or more algorithms are executed at the predicted address generator 8192C to monitor row accesses and output predicted rows based on patterns computed relative to the monitored row accesses, or at one or more trained neural Where the network or other model is configured to output a predicted next row based on input including the most recent row access, the confidence of the predicted next row may be determined based on any relevant parameters. For example, in some cases, the confidence may depend on whether one or more previous next row predictions proved accurate (eg, past performance indicators). Confidence may also be based on one or more characteristics of the input to the algorithm/model. For example, an input that includes an actual row access that follows a pattern may result in a higher confidence level than an actual row access that exhibits less patterning. And in some cases, for example, the resulting confidence may be low relative to the stream detection randomness relative to the input including the most recent row access. Additionally, in the event that randomness is detected, the next row prediction process may be completely aborted, one or more of the components of the memory unit 8100 may ignore the next row prediction, or any other action may be taken to forego initiating the predicted next row prediction. one line.

In some cases, a feedback mechanism may be included with respect to the operation of memory 8100 . For example, periodically or even after each next row prediction, the accuracy of the prediction address generator 8192C in predicting the actual next row to be accessed may be determined. In some cases, if there is an error in predicting the next row to be accessed (or after a predetermined number of errors), the next row prediction operation of the predicted address generator 8192C may be temporarily suspended. In other cases, the predicted address generator 8192C may include a learning element such that one or more aspects of its prediction operation may be adjusted based on received feedback regarding its accuracy in predicting the next row to be accessed. This capability can improve the operation of the predictive address generator 8192C so that the address generator 8192C can adapt to changing access patterns and the like.

In some embodiments, the timing of the generation of the predicted next row and/or the activation of the predicted next row may depend on the overall operation of the memory cell 8100 . For example, after power-up or after resetting the memory cell 8100, predicting the next row to be accessed (or forwarding the predicted next row to the bank controller 8191) may be temporarily suspended (eg, for a predetermined amount of time or clock cycles) , until a predetermined number of row accesses/reads have been completed, until the confidence of the predicted next row exceeds a predetermined threshold, or based on any other suitable criteria).

81B illustrates another configuration of a memory cell 8100 in accordance with an illustratively disclosed embodiment. In system 8100B of FIG. 81B, cache 8193 may be associated with group controller 8191. For example, the cache 8193 can be configured to store one or more lines of data after they are accessed, and prevent the lines of data from needing to be activated again. Thus, cache 8193 may enable bank controller 8191 to access row data from cache 8193 instead of accessing memory bank 8180. For example, cache 8193 may store the last X lines of data (or any other cache saving strategy), and bank controller 8191 may fill cache 8193 according to the predicted lines. Furthermore, if the predicted line is already in cache 8193, the predicted line does not need to be opened again, and the set controller (or a cache controller implemented in cache 8193) can protect the predicted line from being swapped. Cache 8193 may provide several benefits. First, since cache 8193 loads lines into cache 8193 and the set controller can access cache 8193 to fetch line data, no special set or more than one set is required for next line prediction. Second, reading and writing to cache 8193 saves energy because the physical distance from bank controller 8191 to cache 8193 is less than the physical distance from bank controller 8191 to memory bank 8180. Third, the latency caused by cache 8193 is generally lower compared to memory bank 8180 because cache 8193 is smaller and closer to controller 8191. In some cases, when the predicted next row is enabled in the memory bank 8180 by the bank controller 8191, the identifier of the predicted next row generated by the predicted address generator may be stored in the cache 8193, for example. Based on program execution, etc., the current address generator 8192B may identify the actual next row in the memory bank 8191 to be accessed. The identifier associated with the actual next row to be accessed may be compared to the identifier of the predicted next row stored in cache 8193. If the actual next row to be accessed is the same as the predicted next row to be accessed, then the group controller 8191 may begin a read operation relative to the actual next row to be accessed after the start of the actual next row to be accessed has completed (which may fully or partially started due to the next line of prediction handlers). On the other hand, if the actual next row to be accessed (as determined by the current address generator 8192B) does not match the predicted next row identifier stored in the cache 8193, then there will be no comparison to the fully or partially enabled predicted The next row starts the read operation, but the system will start the actual next row to be accessed.

dual boot group

As discussed, it is valuable to describe several mechanisms that allow building groups that can start a row while another row is still being processed. Several embodiments may be provided for groups that enable additional rows while another row is being accessed. Although the embodiment describes only two lines of activation, it should be understood that it is applicable to more lines. In the first proposed embodiment, a bank of memory may be divided into sub-banks of memory, and the described embodiments may be used to perform a read operation relative to a bank in one bank, while enabling a predicted or all of a bank in another bank The next line is required. For example, as shown in FIG. 81C, memory bank 8180 may be configured to include multiple memory sub-banks 8181. Additionally, the bank controller 8191 associated with the memory bank 8180 may include a plurality of subgroup controllers associated with corresponding subgroups. A first subgroup controller of the plurality of subgroup controllers may be configured to enable access to data included in a current row of a first subgroup of the plurality of subgroups, while a second subgroup controller of the plurality of subgroups The group controller may initiate the next row in a second subgroup of the plurality of subgroups. Only one column decoder may be used when accessing words in only one subset at a time. Two groups can be tied to the same output bus to appear as a single group. The new single group input can also be a single address and an extra row address for opening the next row.

81C illustrates the first and second subset row controllers (8183A, 8183B) for each memory subset 8181. Memory bank 8180 may include multiple sub-banks 8181, as shown in Figure 81C. Additionally, the group controller 8191 may include a plurality of subgroup controllers 8183A-8183B, each associated with a corresponding subgroup 8181 . A first subgroup controller 8183A of the plurality of subgroup controllers can be configured to enable access to data included in the current row of the first portion in subgroup 8181, while a second subgroup controller 8183B can activate the subgroup The next line in the second part of the 8181.

Because activating a row directly adjacent to the row being accessed may distort the accessed row and/or corrupt the data being read from the accessed row, the disclosed embodiments may, for example, be configured such that the to-be-activated row The predicted next row may be separated by at least two rows from the current row of data being accessed in the first subset. In some embodiments, the rows to be activated may be separated by at least one pad, such that activation may be performed in different pads. The second subgroup controller may be configured such that data included in the current row of the second subgroup is accessed, while the first subgroup controller activates the next row in the first subgroup. The activated next row of the first subset may be separated by at least two rows from the current row of data being accessed in the second subset.

This predefined distance between the row being read/accessed and the row being enabled can be determined by, for example, hardware coupling different parts of the memory bank to different row decoders, and software can maintain this predefined distance to avoid corrupt data. The spacing between the current row may be more than two columns (eg, it may be 3 rows, 4 rows, 5 rows, and even more than 5 rows). The distance may vary over time, for example based on an assessment of the distortion introduced in the stored data. Distortion can be assessed in various ways, such as by calculating signal-to-noise ratio, error rate, error codes needed to repair the distortion, and the like. If the two rows are far enough apart and the two group controllers are implemented on the same group, two rows can actually be enabled. The new architecture (implementing two controllers on the same group) prevents opening up multiple rows in the same pad.

Figure 81D illustrates an embodiment of next row prediction consistent with embodiments of the present invention. Embodiments may include additional pipelines of flip-flops (address registers A to C). The pipeline can be implemented with any number of flip-flops (stages) to initiate and delay overall execution after the address generator by the delay required to use the delayed address, then prediction can be the new address generated (at the beginning of the pipeline, at the address register C) and the current address is the end of the pipeline. In this embodiment, there is no need to duplicate the address generator. A selector (multiplexer shown in Figure 81D) can be added to configure the delay, while the address register provides the delay.

Figure 81E illustrates an embodiment of a memory bank consistent with embodiments of the present invention. A memory bank can be implemented such that if a newly started rank is far enough from the current rank, starting a new rank will not destroy the current rank. As shown in Figure 81E, a memory bank may include additional memory pads (black) between every two rows of pads. Thus, a control unit (such as a row decoder) may activate multiple rows separated by a pad.

In some embodiments, the memory unit may be configured to receive the first address at predetermined times for processing and to receive the second address for activation and access.

Figure 81F illustrates another embodiment of a memory bank consistent with embodiments of the present invention. A memory bank can be implemented such that if a newly started rank is far enough from the current rank, starting a new rank will not destroy the current rank. The embodiment depicted in Figure 81F may allow the row decoder to open up rows n and n+1 by ensuring that all even rows are implemented at the upper half of the memory bank and all odd rows are implemented at the lower half of the memory bank. Implementations may allow access to consecutive rows that are always far enough apart.

According to the disclosed embodiments, a dual control memory bank may allow different portions of a single memory bank to be accessed and activated, even when the dual control memory bank is configured to output one data unit at a time. For example, as described, dual control may enable a memory bank to access a first row when a second row is activated (eg, a predicted next row or a predetermined next row to be accessed).

82 illustrates a dual control memory bank 8280 for reducing memory row start penalties (eg, latency) in accordance with an embodiment of the present invention. Dual control memory bank 8280 may include inputs including data input (DIN) 8290, row address (ROW) 8291, column address (COLUMN) 8292, first command input (COMMAND_1) 8293, and second command input (COMMAND_2) 8294. Memory bank 8280 may include data output (Dout) 8295.

It is assumed that addresses can include row addresses and column addresses, and that there are two row decoders. Other configurations of addresses may be provided, the number of row decoders may exceed two, and there may be more than a single column decoder.

A row address (ROW) 8291 may identify the row associated with a command such as a start command. Since a row can be followed by reading from or writing to that row, then when the row is opened (after it is started), it may not be necessary to send a row for writing to or reading from an open row address.

The first command input (COMMAND_1) 8293 may be used to send a command, such as, but not limited to, a start command, to the row accessed by the first row decoder. A second command (COMMAND_2) input 8294 may be used to send a command, such as, but not limited to, a start command, to the row accessed by the second row decoder.

A data input (DIN) 8290 can be used to feed data when performing write operations.

Since the entire row cannot be read at once, individual row sectors can be read sequentially, and the column address (COLUMN) 8292 can indicate which sector (which columns) of the row is to be read. For simplicity of explanation, it may be assumed that there are 2Q segments and that the column input has Q bits; Q is a positive integer greater than one.

Dual control memory bank 8280 may operate with or without the address prediction described above with respect to Figures 81A-81B. Of course, to reduce operating latency, dual control memory banks may operate with address prediction in accordance with the disclosed embodiments.

83A, 83B, and 83C illustrate an example of accessing and enabling a row of memory bank 8180. As mentioned above, assume that in one example, 32 cycles (segments) are required for both reading a line and starting a line. Additionally, in order to reduce the startup penalty (having a length denoted as delta), it may be beneficial to know in advance (at least delta before the next row needs to be accessed) that the next row should be opened. In some cases, the difference may be equal to four cycles. Each memory bank depicted in Figures 83A, 83B, and 83C may include two or more subgroups within which, in some embodiments, at any given time Only one row can be opened. In some cases, even rows may be associated with a first subset, and odd rows may be associated with a second subset. In this example, use of the disclosed predictive addressing embodiments may enable a certain memory to be initiated before the end of a read operation relative to a row of another memory subgroup is reached (a delay period before the end is reached). Start of a row of a subgroup. In this way, sequential memory accesses (eg, a predefined memory access sequence, where rows 1, 2, 3, 4, 5, 6, 7, 8 . . . are to be read, and rows 1, 3, . , 5, . . . etc. are associated with a first memory subgroup and rows 2, 4, 6, .

83A may illustrate states for accessing memory rows included in two different memory subgroups. In the state shown in Figure 83A:

a. Row A may be accessible by the first row decoder. The first sector (the leftmost sector marked in grey) can be accessed after the first row of decoder starts row A.

b. Row B may be accessible by a second row decoder. In these states shown in Figure 83A, row B is turned off and not yet started.

The state illustrated in Figure 83A may be preceded by sending a start command and the address of row A to the first row decoder.

83B illustrates a state for accessing row B after row A is accessed. According to this example: Row A may be accessible by the first row decoder. In the state shown in Figure 83B, the first row decoder starts row A and has accessed all but the four rightmost sectors (the four not marked in gray). Because the delta (four white segments in row A) equals four cycles, the bank controller can enable the second row decoder to enable row B before accessing the rightmost segment in row A. In some cases, enabling row B may be responsive to a predetermined access pattern (eg, sequential row access, with odd rows designated in a first subset and even rows designated in a second subset). In other cases, enabling row B may be responsive to any of the row prediction techniques described above. The group controller may enable the second row decoder to pre-enable row B so that when row B is accessed, row B is already activated (opened) rather than waiting to be activated to open row B.

The state illustrated in Figure 83B may be preceded by the following operations:

a. Send the start command and the address of line A to the first line decoder.

b. Write or read the first twenty-eight sectors of row A.

c. After a read or write operation of the twenty-eight sectors of the row, send a start command relative to the address of row B to the second row decoder.

In some embodiments, even-numbered columns are located in half of one or more memory banks. In some embodiments, odd-numbered columns are located in half of one or more memory banks.

In some embodiments, an additional row of redundant pads is placed between each of the two pad rows to establish a distance for enabling activation. In some embodiments, multiple rows in close proximity to each other may not be activated at the same time.

83C may illustrate a state for accessing row C (eg, the next odd row included in the first subset) after row A is accessed. As shown in Figure 83C, row B may be accessible by the second row decoder. As shown, the second row decoder has enabled row B and has accessed all but the four rightmost sectors (the four remaining sectors not marked in gray). Because the delta is equal to four cycles in this example, the group controller may enable the first row decoder to start row C before accessing the rightmost sector in row B. The bank controller may enable the first row decoder to pre-enable row C so that when row C is accessed, row C is already enabled rather than waiting to be enabled. Operating in this manner may reduce or completely eliminate latency associated with memory read operations.

memory pad as register file

In computer architecture, processor registers constitute storage locations that are quickly accessible by a computer processor (eg, a central processing unit (CPU)). Registers typically include the memory location closest to the processor core (L0). Registers provide the fastest way to access certain types of data. A computer may have several types of registers, each classified according to the type of information it stores or based on the type of instructions that operate on information in a certain type of register. For example, a computer may include: data registers, which store numerical information, operands, intermediate results, and configuration; address registers, which store address information used by instructions to access main memory; general purpose registers, which store data and address information both; and the status register; and other registers. A register file includes a logical grouping of registers available to a computer processing unit.

In many cases, a computer's register file is located within a processing unit (eg, a CPU) and implemented by logic transistors. However, in the disclosed embodiments, the arithmetic processing unit may not reside in a conventional CPU. Rather, such processing elements (eg, processor subunits) may be spatially distributed (as described in the above sections) within a memory chip as a processing array. Each processor sub-unit may be associated with one or more corresponding and dedicated memory units (eg, memory banks). Through this architecture, each processor sub-unit can be located spatially near one or more memory elements that store data on which a particular processor sub-unit is to operate. As described herein, this architecture can significantly speed up operations in certain memory-intensive operations by, for example, eliminating memory access bottlenecks experienced by typical CPU and external memory architectures.

However, the distributed processor memory chip architecture described herein may still utilize register files, which include various types of registers for operating on data from memory elements dedicated to corresponding processor subunits. However, since the processor sub-units may be distributed among the memory elements of the memory chip, it is possible to combine one or more memory elements that may benefit from the The same process) is added in the corresponding processor sub-unit to act as a register file or cache for the corresponding processor sub-unit, rather than acting as the main memory storage.

This architecture can provide several advantages. For example, since a register file is part of a corresponding processor sub-unit, a processor sub-unit may be located spatially near the associated register file. This configuration can significantly increase operational efficiency. Traditional register files are implemented by logic transistors. For example, each bit of a conventional register file is made of about 12 logic transistors, and thus a 16-bit register file is made of 192 logic transistors. This register file may require a large number of logic components to access the logic transistors, and thus may take up a large amount of space. The register files of the disclosed embodiments may require significantly less space than register files implemented by logic transistors. This size reduction can be achieved by implementing the register file of the disclosed embodiments using memory pads that include memory cells by processes optimized for fabricating memory structures rather than logic structures to manufacture. The size reduction may also allow for larger register files or caches.

In some embodiments, distributed processor memory chips may be provided. A distributed processor memory chip may include: a substrate; a memory array disposed on the substrate and including a plurality of discrete memory banks; and a processing array disposed on the substrate and including a plurality of processor subunits. Each of the processor sub-units may be associated with a corresponding dedicated memory bank of a plurality of discrete memory banks. The distributed processor memory chip may also include a first plurality of buses and a second plurality of buses. Each of the first plurality of buses may connect one of the plurality of processor subunits to its corresponding dedicated memory bank. Each of the second plurality of buses may connect one of the plurality of processor subunits to another of the plurality of processor subunits. In some cases, the second plurality of buses may connect one or more of the plurality of processor subunits to two or more other processor subunits of the plurality of processor subunits. One or more of the processor subunits may also include at least one memory pad disposed on the substrate. At least one memory pad may be configured to function as at least one register of a register file for one or more of the plurality of processing subunits.

In some cases, a register file may be associated with one or more logical components to enable a memory pad to function as one or more registers of the register file. For example, such logic components may include switches, amplifiers, inverters, sense amplifiers, and others. In instances where the register file is implemented by a dynamic random access memory (DRAM) pad, logic components may be included to perform refresh operations to prevent loss of stored data. Such logical components may include row and column multiplexers ("mux"). Additionally, the register file implemented by the DRAM pads may include redundancy mechanisms to combat yield degradation.

84 illustrates a conventional computer architecture 8400 including a CPU 8402 and external memory 8406. During operation, values from memory 8406 may be loaded into registers associated with register file 8504 included in CPU 8402.

85A illustrates an exemplary distributed processor memory chip 8500a consistent with disclosed embodiments. In contrast to the architecture of Figure 84, a distributed processor memory chip 8500a includes memory components and processor components disposed on the same substrate. That is, chip 8500a may include a memory array and a processing array including a plurality of processor subunits each associated with one or more dedicated memory banks included in the memory array. In the architecture of Figure 85, the registers used by the processor subunits are provided by one or more memory pads disposed on the same substrate on which the memory array and processing array are formed.

As depicted in FIG. 85A, a distributed processor memory chip 8500a may be formed by a plurality of processing groups 8510a, 8510b, and 8510c disposed on a substrate 8502. More specifically, the distributed processor memory chip 8500a may include a memory array 8520 and a processing array 8530 disposed on a substrate 8502. Memory array 8520 may include multiple memory banks, such as memory banks 8520a, 8520b, and 8520c. Processing array 8530 may include multiple processor subunits, such as processor subunits 8530a, 8530b, and 8530c.

Additionally, each of processing groups 8510a, 8510b, and 8510c may include a processor subunit and one or more corresponding memory banks dedicated to that processor subunit. In the embodiment depicted in Figure 85A, each of the processor subunits 8530a, 8530b, and 8530c may be associated with a corresponding dedicated memory bank 8520a, 8520b, or 8520c. That is, processor sub-unit 8530a may be associated with memory bank 8520a; processor sub-unit 8530b may be associated with memory bank 8520b; and processor sub-unit 8530c may be associated with memory bank 8520c.

To allow each processor sub-unit to communicate with its corresponding dedicated memory bank, the distributed processor memory chip 8500a may include a first plurality of buses 8540a connecting one of the processor sub-units to its corresponding dedicated memory bank, 8540b and 8540c. In the embodiment depicted in Figure 85A, bus 8540a may connect processor subunit 8530a to memory bank 8520a; bus 8540b may connect processor subunit 8530b to memory bank 8520b; and bus 8540c may connect processor subunit 8540c 8530c is connected to memory bank 8520c.

Furthermore, to allow each processor subunit to communicate with other processor subunits, the distributed processor memory chip 8500a may include a second plurality of connecting one of the processor subunits to at least one other processor subunit Buses 8550a and 8550b. In the embodiment depicted in Figure 85, bus 8550a may connect processor subunit 8530a to processor subunit 8530b, and bus 8550b may connect processor subunit 8530a to processor subunit 8550b, and so on.

Each of discrete memory banks 8520a, 8520b, and 8520c may include multiple memory pads. In the embodiment depicted in Figure 84, memory bank 8520a may include memory pads 8522a, 8524a, and 8526a; memory bank 8520b may include memory pads 8522b, 8524b, and 8526b; and memory bank 8520c may include memory pads 8522c, 8524c, and 8526c . As previously disclosed with respect to FIG. 10, a memory pad may include a plurality of memory cells, and each cell may include a capacitor, transistor, or other circuitry that stores at least one bit of data. A conventional memory pad may include, for example, 512 bits by 512 bits, although the embodiments disclosed herein are not so limited.

At least one of the processor subunits 8530a, 8530b, and 8530c may include at least one memory pad, such as memory pads 8532a, 8532b, and 8532c, configured to serve as a register file for the corresponding processor subunits 8530a, 8530b, and 8530c. That is, at least one memory pad 8532a, 8532b, and 8532c provides at least one register of a register file used by one or more of processor subunits 8530a, 8530b, and 8530c. A register file can include one or more registers. In the embodiment depicted in Figure 85A, the memory pad 8532a in the processor sub-unit 8530a may serve as a function for the processor sub-unit 8530a (and/or any other processor sub-unit included in the distributed processor memory chip 8500a) unit) of the register file (also referred to as "register file 8532a"); the memory pad 8532b in the processor sub-unit 8530b may serve as a register file for the processor sub-unit 8530b; and the memory pad in the processor sub-unit 8530c The 8532c can act as a register file for the processor sub-unit 8530c.

At least one of processor sub-units 8530a, 8530b, and 8530c may also include at least one logical component, such as logical components 8534a, 8534b, and 8534c. Each logical component 8534a, 8534b, or 8534c may be configured such that the corresponding memory pad 8532a, 8532b, or 8532c can function as a register file for the corresponding processor subunit 8530a, 8530b, or 8530c.

In some embodiments, at least one memory pad may be disposed on the substrate, and the at least one memory pad may contain at least one of at least one redundancy register configured to provide for one or more of the plurality of processor subunits Redundant memory bits. In some embodiments, at least one of the processor sub-units may include a mechanism to stop the current task and trigger a memory refresh operation at certain times to refresh the memory pad.

85B illustrates an exemplary distributed processor memory chip 8500b consistent with disclosed embodiments. The memory chip 8500b illustrated in Figure 85B is substantially the same as the memory chip 8500 illustrated in Figure 85A, except that the memory pads 8532a, 8532b, and 8532c in Figure 85B are not included in the corresponding processor subunits 8530a, 8530b, and 8530c . In fact, memory pads 8532a, 8532b, and 8532c in Figure 85B are disposed outside but spatially close to the corresponding processor subunits 8530a, 8530b, and 8530c. In this way, the memory pads 8532a, 8532b, and 8532c can still serve as register files for the corresponding processor subunits 8530a, 8530b, and 8530c.

Figure 85C illustrates a device 8500c consistent with the disclosed embodiments. Device 8500c includes substrate 8560 , first memory bank 8570 , second memory bank 8572 , and processing unit 8580 . The first memory group 8570 , the second memory group 8572 and the processing unit 8580 are disposed on the substrate 8560 . The processing unit 8580 includes a processor 8584 and a register file 8582 implemented by a memory pad. During operation of processing unit 8580, processor 8584 may access register file 8582 to read or write data.

Distributed processor memory chips 8500a, 8500b or device 8500c may provide a variety of functions based on processor subunit accesses to registers provided by memory pads. For example, in some embodiments, a distributed processor memory chip 8500a or 8500b may include a processor subunit that acts as an accelerator coupled to memory, allowing it to use more memory bandwidth. In the embodiment depicted in Figure 85A, the processor subunit 8530a may function as an accelerator (also referred to as "accelerator 8530a"). The accelerator 8530a may use a memory pad 8532a disposed in the accelerator 8530a to provide one or more registers of a register file. Alternatively, in the embodiment depicted in Figure 85B, the accelerator 8530a may use a memory pad 8532a disposed outside the accelerator 8530a as a register file. Still further, accelerator 8530a may use any of memory pads 8522b, 8524b, and 8526b in memory bank 8520b or any of memory pads 8522c, 8524c, and 8526c in memory bank 8520c to provide one or more registers .

The disclosed embodiments may be particularly suitable for certain types of image processing, neural networks, database analysis, compression and decompression, and many more applications. For example, in the embodiment of Figure 85A or Figure 85B, the memory pad may provide one or more registers for the register file of one or more processor subunits included on the same chip as the memory pad. One or more registers may be used to store data frequently accessed by processor subunits. For example, during convolutional image processing, the convolution accelerator may reuse the same coefficients over the entire image held in memory. The proposed implementation for this convolution accelerator may store all of these coefficients in a "closed" register file within one or more registers included in the registers dedicated to one or more processor subsystems. Within the unit's memory pad, the one or more processor subunits are located on the same chip as the register file memory pad. This architecture can place registers (and stored coefficient values) in close proximity to processor subunits that operate on the coefficient values. Because the register file implemented by the memory pad can act as a spatially compact, efficient cache, significantly lower penalties for data transfers and lower latencies for accesses can be achieved.

In another example, the disclosed embodiments can include accelerators that can input words into registers provided by memory pads. The accelerator can handle registers as circular buffers to multiply vectors in a single cycle. For example, in the device 8500c illustrated in Figure 85C, the processor 8584 in the processing unit 8580 acts as an accelerator that uses a register file 8582 implemented by a memory pad as a circular buffer to store data A1, A2, A3 . . . . The first memory group 8570 stores data B1 , B2 , B3 . . . to be multiplied by data A1 , A2 , A3 . . . The second memory group 8572 stores the multiplication results C1, C2, C3 . . . That is, Ci=Ai×Bi. If no register file exists in the processing unit 8580, the processor 8584 would require more memory bandwidth and more cycles to read data A1, A2, A3... and data B1, from an external memory bank such as memory bank 8570 or 8572 B2, B3...both, this can create significant delays. On the other hand, in the present embodiment, the data A1 , A2 , A3 . . . are stored in the register file 8582 formed in the processing unit 8580 . Therefore, the processor 8584 will only need to read the data B1, B2, B3 . . . from the external memory bank 8570. Therefore, memory bandwidth can be significantly reduced.

In memory handlers, memory pads typically allow unidirectional access (ie, a single access). In unidirectional access, there is one port to the memory. As a result, only one access operation, such as a read or a write, to a particular address can be performed at a time. However, bidirectional access may be a valid option if the memory pad itself allows bidirectional access. In bidirectional access, two different addresses can be accessed at a time. The method of accessing the memory pad can be determined based on area and requirements. In some cases, register files implemented by memory pads may allow four-way access if connected to a processor that needs to read two sources and has one destination register. In some cases, when the register file is implemented by DRAM pads to store configuration or cache data, the register file may only allow one-way access. Standard CPUs may include multidirectional access pads, while unidirectional access pads may be better for DRAM applications.

When the controller or accelerator is designed in such a way that only a single access to the registers is required (in the few possible cases), memory pad-implemented registers may be used instead of traditional register files. In a single access, only one word can be accessed at a time. For example, a processing unit may access two words from two register files at a time. Each of the two register files may be implemented with memory pads (eg, DRAM pads) that allow only a single access.

In most technologies, the memory pad IP, which is an enclosed block (IP) obtained from the manufacturer, will be accompanied by wiring, such as word and row lines, in place for row and column access. But the memory pad IP does not include wrap-around logic components. Accordingly, a register file implemented by a memory pad disclosed in embodiments of the present invention may include logic components. The size of the memory pad can be chosen based on the desired size of the register file.

Certain challenges may arise when using memory pads to provide registers of a register file, and these challenges may depend on the particular memory technology used to form the memory pad. For example, in memory production, not all memory cells fabricated may operate properly after production. This is a known problem, especially if there is a high density of SRAM or DRAM on the chip. To address this problem in memory technology, one or more redundancy mechanisms may be used in order to maintain yield at a reasonable level. In the disclosed embodiments, the redundancy mechanism may not be as important as in normal memory applications because the number of memory instances (eg, memory banks) used to provide the registers of the register file can be relatively small. On the other hand, the same production issues that affect memory functionality can also affect whether a particular memory pad can function properly when providing one or more registers. As a result, redundant elements may be included in the disclosed embodiments. For example, at least one redundant memory pad may be disposed on the substrate of the distributed processor memory chips. At least one redundant memory pad may be configured to provide at least one redundant register for one or more of the plurality of processor subunits. In another example, the pads can be larger than desired (eg, 620x620 instead of 512x512), and the redundancy mechanism can be built into regions of the memory pads outside the 512x512 region or its equivalent.

Another challenge can be related to timing. The timing of loading words and bit lines is usually determined by the size of the memory. Since the register file can be implemented by a relatively small single memory pad (eg, 512x512 bits), the time required to load a word from the memory pad will be small, and timing may be sufficient to run fairly quickly compared to logic.

Refresh - Some memory types such as DRAM require periodic refresh. Refreshing can be performed while pausing the processor or accelerator. For small memory pads, the refresh time may be a fraction of the time. Therefore, even if the system stops for a short period of time, the gain obtained by using the memory pad as a register is worth the downtime in terms of overall performance. In one embodiment, the processing unit may include a counter that counts backwards from a predefined number. When the counter reaches "0", the processing unit may stop the current task being executed by the processor (eg, the accelerator) and trigger a refresh operation that refreshes the memory pad row by row. When the refresh operation is complete, the processor can resume its tasks, and the counter can be reset to count backwards from a predefined number.

86 provides a flowchart 8600 representing an exemplary method for executing at least one instruction in a distributed processor memory chip consistent with the disclosed embodiments. For example, at step 8602, at least one data value can be retrieved from a memory array on a substrate of a distributed processor memory chip. At step 8604, the retrieved data values may be stored in registers provided by memory pads of a memory array on the substrate of the distributed processor memory chip. At step 8606, a processor element, such as one or more of the distributed processor sub-units on the distributed processor memory chip board, may operate on the stored data values from the memory pad registers.

Here and throughout, it should be understood that all references to a register file shall refer equally to a cache, as a register file may be the lowest level cache.

processing bottlenecks

The terms "first", "second", "third" and the like are only used to distinguish different terms. These terms may not suggest the order and/or timing and/or importance of the components. For example, the first handler can be preceded by the second handler, and the like.

The term "coupled" can mean directly connected and/or indirectly connected.

The terms "memory/processing," "memory and processing," and "memory processing" are used interchangeably.

Various methods, computer readable media, memory/processing units, and/or systems may be provided that may be memory/processing units.

A memory/processing unit is a hardware unit with memory and processing capabilities.

The memory/processing unit may be a memory processing integrated circuit, may be included in a memory processing integrated circuit, or may include one or more memory processing integrated circuits.

The memory/processing unit may be a distributed processor as described in PCT Patent Application Publication WO2019025892.

The memory/processing unit may comprise a distributed processor as described in PCT Patent Application Publication WO2019025892.

The memory/processing unit may belong to a distributed processor as described in PCT Patent Application Publication WO2019025892.

The memory/processing unit may be a memory chip as described in PCT Patent Application Publication WO2019025892.

The memory/processing unit may comprise a memory chip as described in PCT Patent Application Publication WO2019025892.

The memory/processing unit may be a distributed processor as described in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may belong to a distributed processor as described in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may be a memory chip as described in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may comprise a memory chip as described in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may belong to a memory chip as described in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may be an integrated circuit that uses inter-wafer bonding and multiple conductors to connect to each other.

Any reference to distributed processor memory chips, distributed memory processing integrated circuits, memory chips, distributed processors may be implemented as a pair of integrated circuits connected to each other by inter-wafer bonding and a plurality of conductors.

Memory/processing units can be fabricated by a first fabrication process that is better suited for memory cells than logic cells. Therefore, the first manufacturing process can be regarded as a manufacturing process of the memory class. A memory cell may include one of a plurality of transistors. A logic cell may include one or more transistors. The first fabrication process can be applied to fabricate the memory bank. A logic cell may include one or more transistors that together perform logic functions, and may be used as a basic building block for larger logic circuits. A memory cell may include one or more transistors that together perform memory functions, and may be used as a basic building block for larger logic circuits. Corresponding logical cells may implement the same logical function.

The memory/processing unit may be different from any of the processors, processing integrated circuits and/or processing units by being better suited to logical cells than memory cells Manufactured by the second manufacturing process of Yuan. Therefore, the first manufacturing process can be regarded as a logical class of manufacturing processes. The second manufacturing process can be used to manufacture central processing units, graphics processing units, and the like.

A memory/processing unit may be better suited to perform less arithmetically intensive operations than a processor, processing integrated circuit, and/or processing unit.

For example, memory cells fabricated by the first manufacturing process may exhibit more than and even greatly exceeds (eg, more than 2 times, 3 times, 4 times, 5 times, 9 times, 7 times, 8 times, 9 times, 10 times times and the like) the critical dimension of the critical dimension of the logic circuit fabricated by the first fabrication process.

The first manufacturing process may be an analog manufacturing process, the first manufacturing process may be a DRAM manufacturing process, and the like.

The size of the logic cells fabricated by the first fabrication process may exceed the size of the corresponding logic cells fabricated by the second fabrication process by at least two times. The corresponding logical calls may have the same functionality as the logical cells fabricated by the first manufacturing process.

The second manufacturing process may be a digital manufacturing process.

The second fabrication process may be any of Complementary Metal Oxide Semiconductor (CMOS), Bipolar, Bipolar CMOS (BiCOMS), Double Diffused Metal Oxide Semiconductor (DMOS), Silicon on Oxide fabrication process, and the like.

The memory/processing unit may include multiple processor sub-units.

The processor sub-units of the one or more memory/processing units may operate independently of each other and/or may cooperate with each other and/or perform distributed processing. Distributed processing may be performed in various manners, eg, in a flat manner or in a hierarchical manner.

A flat approach may involve having processor subunits perform the same operations (and may or may not output processing results between the processor subunits).

A hierarchical approach may involve performing sequences of processing operations at different levels, with processing operations at one level occurring after processing operations at another level. Processor subunits may be assigned (dynamically or statically) to different layers and participate in hierarchical processing.

Distributed processing may also involve other units, such as controllers of memory/processing units and/or units not belonging to the memory/processing units.

The terms logic and processor subunit are used interchangeably.

Any processing mentioned in this application may be performed in any manner (distributed and/or non-distributed and the like).

In the following applications, various references and/or incorporation by reference are made to PCT Patent Application Publication WO2019025892 and PCT Patent Application No. PCT/IB2019/001005 (September 9, 2019). PCT Patent Application Publication WO2019025892 and/or PCT Patent Application No. PCT/IB2019/001005 provide non-limiting examples of various methods, systems, processors, memory chips, and the like. Other methods, systems, processors may be provided.

A processing system (system) may be provided in which the processor is preceded by one or more memories/processing units, each memory and processing unit (memory/processing unit) having processing resources and storage resources.

A processor may request or instruct one or more memory/processing units to perform various processing tasks. The execution of various processing tasks may reduce the burden on the processor, reduce latency, and in some cases reduce the overall information bandwidth between one or more memory/processing units and the processor, and the like.

The processor may provide instructions and/or requests at different granularities, eg, the processor may issue instructions for certain processing resources or may issue higher order instructions for memory/processing units without specifying any processing resources.

A memory/processing unit may manage its processing and/or memory resources in any manner (dynamic, static, distributed, centralized, offline, online, and the like). The management of resources may be performed autonomously, under the control of the processor, after the processor is configured, and the like.

For example, a task may be divided into subtasks that may require one or more processing resources and/or memory resources of one or more memory/processing units to execute or one or more instructions. Each processing resource may be configured to execute (eg, independently or not independently) at least one instruction. See, eg, execution of sub-series of instructions by processing resources of a processor sub-unit such as PCT Patent Application Publication WO2019025892.

Allocation of memory resources may also be provided at least to entities other than one or more memory/processing units, such as a direct access memory (DMA) unit that may be coupled to the one or more memory/processing units.

The compiler may prepare configuration files for each type of task performed by the memory/processing unit. Profiles include memory allocations and processing resource allocations associated with task types. A configuration file may include instructions executable by different processing resources and/or may define memory allocations.

For example, a configuration file related to the task of matrix multiplication (multiplying matrix A by matrix B, A*B=C) may suggest where to store the elements of matrix A, where to store the elements of matrix B, where where elements of matrix C are stored, where intermediate results produced during matrix multiplication are stored, and may include instructions for processing resources used to perform any mathematical operations related to matrix multiplication. A configuration file is an instance of a data structure and can provide other data structures.

Matrix multiplication may be performed in any manner by one or more memory/processing units.

One or more memory/processing units may multiply matrix A by vector V. This can be done in any way. For example, this may involve maintaining a row or column of a matrix per processing resource (maintaining a different row of rows per different processing resource), and looping (between different processing resources) the final result of a multiplication of a row or column of a matrix and a vector (during the first iteration), and looping over the final result of the previous multiplication (during the second to last iteration).

Assume that matrix A is a 4x4 matrix, vector V is a 1x4 vector, and there are four processing resources. Under this assumption, the first row of matrix A is stored at the first processor subunit, the second row of matrix A is stored at the second processor subunit, and the third row of matrix A is stored at the third processing resource , and matrix A is stored at the fourth processor subunit in the fourth row. Begin the multiplication by: sending the first through fourth elements of vector V to the first through fourth processing resources; and multiplying the first through fourth elements of vector V by a different vector of A to provide a first intermediate result . The multiplication continues by looping over the first intermediate result by sending, by each processing resource, the first intermediate result computed by the first processing resource to its neighboring processing resource. Each processing resource multiplies the first multiplication result by the vector to provide the second multiplication result. This process is repeated many times until the multiplication of matrix A and vector V ends.

90A is an example of a system 10900 that includes one or more memory/processing units (collectively denoted 10910) and a processor 10920. The processor 10920 may send the request or instruction (via link 10931) to one or more memory/processing units 10920, which in turn complete (or selectively complete) the request and/or instruction and The results are sent (via link 10932) to processor 10920, as described above. Processor 10920 may further process the results to provide (via link 10933) one or more outputs.

One or more memory/processing units may include J (J is a positive integer) memory resources 10912(1,1) to 10912(1,J) and K (K is a positive integer) processing resources 10911(1,1) to 10911(1,K).

J may be equal to K or may be different from K.

The processing resources 10911(1,1) to 10911(1,K) may be, for example, processing groups or processor sub-units, as described in PCT Patent Application Publication WO2019025892.

Memory resources 10912(1,1) to 10912(1,J) may be memory instances, memory pads, memory banks, as described in PCT Patent Application Publication WO2019025892.

There may be any connectivity and/or any functional relationship between any of the resources (memory or processing) of one or more memory/processing units.

Figure 90B is an example of a memory/processing unit 10910(1).

In FIG. 90B, K (K is a positive integer) processing resources 10911(1,1) to 10911(1,K) form a loop because the processing resources are connected to each other in series (see link 10915). Each processing resource is also coupled to its own pair of dedicated memory resources (eg, processing resource 10911(1) is coupled to memory resources 10912(1) and 10912(2), and processing resource 10911(K) is coupled to Memory resources 10912(J-1) and 10912(J)). The processing resources can be connected to each other in any other way. The number of memory resources allocated for each processing resource may be different from two. An example of connectivity between different resources is described in PCT Patent Application Publication WO2019025892.

90C is an example of a system 10901 of N (N is a positive integer) memory/processing units 10910(1)-10910(N) and processors 10920. Processor 10920 may send requests or instructions (via links 10931(1) to 10931(N)) to memory/processing units 10920(1) to 10910(N), which in turn fulfill requests and/or instructs and sends the results (via links 10932(1) to 3232(N)) to processor 10920, as described above. Processor 10920 may further process the results to provide (via link 10933) one or more outputs.

90D is an example of a system 10902 that includes N (N is a positive integer) memory/processing units 10910(1)-10910(N) and a processor 10920. Figure 90D illustrates the preprocessor 10909 prior to the memory/processing units 10910(1)-10910(N). The preprocessor may perform various preprocessing operations, such as frame extraction, header detection, and the like.

90E is an example of a system 10903 including one or more memory/processing units 10910 and a processor 10920. 90E illustrates a preprocessor 10909 preceding one or more memory/processing units 10910 and a DMA controller 10908.

90F illustrates a method 10800 for distributed processing of at least one stream of information.

Method 10800 may begin at step 10810 of receiving, by one or more memory processing integrated circuits, at least one stream of information via a first communication channel; wherein each memory processing integrated unit includes a controller, a plurality of processor sub-units, and a plurality of memory unit.

Step 10810 may be followed by steps 10820 and 10830.

Step 10820 may include buffering the stream of information by one or more memory processing integrated circuits.

Step 10830 may include performing, by one or more memory processing integrated circuits, a first processing operation on the at least one stream of information to provide a first processing result.

Step 10830 may involve compression or decompression.

Therefore, the total size of the information stream may exceed the total size of the first processing result. The total size of the information stream may reflect the amount of information received during a period of given duration. The total size of the first processing result may reflect the amount of the first processing result output during any period of the same given duration.

Alternatively, the total size of the information stream (any other information entities mentioned in this specification) is smaller than the total size of the first processing result. In this situation, compression is obtained.

Step 10830 may be followed by step 10840 of sending the first processing result to one or more processing integrated circuits.

One or more memory processing integrated circuits may be fabricated by a memory class of fabrication process.

One or more memory processing integrated circuits may be fabricated by a logic class of fabrication process.

In a memory processing integrated unit, each of the memory units may be coupled to a processor subunit.

Step 10840 may be followed by step 10850 of performing a second processing operation on the first processing result by one or more processing integrated circuits to provide a second processing result.

Step 10820 and/or step 10830 may be instructed by one or more processing integrated circuits, may be requested by one or more processing integrated circuits, may be integrated by one or more processing integrated circuits in one or more memory processing The circuits are configured for execution, or may be executed independently without the intervention of one or more processing integrated circuits.

The first processing operation may have a lower arithmetic intensity than the second processing operation.

Step 10830 and/or step 10850 may be at least one of: (a) cellular network processing operations; (b) other network related processing operations (processing of networks other than cellular networks); (c) database processing operations; (d) database analytical processing operations; (e) artificial intelligence processing operations; or any other processing operations.

Decomposed system memory/processing unit and method for distributed processing

A disaggregated system, a method for distributed processing, a processing/memory unit, a method for operating a disaggregated system, a method for operating a processing/memory unit, and a computer-readable medium, the computer-readable medium, may be provided. Instructions for performing any of the methods are transiently and stored. A disaggregated system assigns different subsystems to perform different functions. For example, storage may be implemented primarily in one or more storage subsystems, and operations may be performed primarily in one or more storage subsystems.

A disaggregated system may be a disaggregated server, one or more disaggregated servers, and/or may be different from one or more servers.

A disaggregated system may include one or more switching subsystems, one or more computing subsystems, one or more storage subsystems, and one or more processing/memory subsystems.

One or more processing/memory subsystems, one or more computing subsystems, and one or more storage subsystems are coupled to each other via one or more switching subsystems.

One or more processing/memory subsystems may be included in one or more subsystems of the disaggregated system.

87A illustrates various examples of decomposed systems.

Any number of subsystems of any type may be provided. The exploded system may include one or more additional subsystems of the type not included in FIG. 87A, may include fewer types of subsystems, and the like.

The decomposed system 7101 includes two storage subsystems 7130 , an arithmetic subsystem 7120 , a switching subsystem 7140 , and a processing/memory subsystem 7110 .

The decomposed system 7102 includes two storage subsystems 7130 , an arithmetic subsystem 7120 , a switching subsystem 7140 , a processing/memory subsystem 7110 , and an accelerator subsystem 7150 .

The decomposed system 7103 includes two storage subsystems 7130 , an arithmetic subsystem 7120 , and a switch subsystem 7140 including a processing/memory subsystem 7110 .

The decomposed system 7104 includes two storage subsystems 7130 , an arithmetic subsystem 7120 , a switch subsystem 7140 including a processing/memory subsystem 7110 , and an accelerator subsystem 7150 .

Including processing/memory subsystem 7110 in switching subsystem 7140 can reduce traffic within disaggregated systems 7101 and 7102, can reduce switching latency, and the like.

The different subsystems of the disaggregated system can communicate with each other using various communication protocols. It has been found that using the Ethernet and even the Ethernet RDMA communication protocol can increase throughput, and possibly even reduce the complexity of various control and/storage operations associated with the exchange of information units between components of a disaggregated system.

A disaggregated system can perform distributed processing by allowing the processing/memory subsystem to participate in computations, especially by performing memory-intensive computations.

For example, given that N arithmetic units should share information units among them (all shared), then (a) N information units may be sent to one or more processing/memory units of one or more processing/memory subsystems , (b) one or more processing/memory units may perform computations that require all sharing, and (c) send N updated information units to N arithmetic units. This would require about N transfer operations.

For example, Figure 87B illustrates the distributed process of updating a model of a neural network that includes weights assigned to nodes of the neural network.

Each of the N arithmetic units PU( 1 ) 7120( 1 ) through PU(N) 7120(N) may belong to an arithmetic subsystem 7120 of any of the decomposed systems 7101 , 7102 , 7103 , and 7104 .

The N arithmetic units compute N part model updates (updated N different parts) 7121(1)-7121(N) and send them (via switch subsystem 7140) to processing/memory subsystem 7110.

Processing/memory subsystem 7110 computes the updated model 7122 and sends (via exchange subsystem 7140) the updated model to N arithmetic units PU(1) 7120(1) through PU(N) 7120(N).

Figures 87C, 87D, and 87E illustrate examples of memory/processing units 7011, 7012, and 7013, respectively, and Figures 87F and 87G illustrate integrated circuits 7014 and 7015, which include memory/processing units 9010 such as Ethernet modules and Ethernet One or more communication modules of the RDMA module 22 .

The memory/processing unit includes a controller 9020, an internal bus 9021, and pairs of logic 9030 and memory banks 9040. The controller is configured to operate as or couplable to the communication module.

Connectivity between controller 9020 and pairs of logic 9030 and memory banks 9040 may be implemented in other ways. Memory banks and logic can be configured in other ways (not in pairs).

One or more memory/processing units 9010 of the processing/memory subsystem 7110 can process (using different logic and fetching different parts of the model in parallel from different memory banks) model updates in parallel, and benefit from massive memory resources, memory banks The extremely high bandwidth of the connection to the logic enables these computations to be performed in an efficient manner.

Memory/processing units 7011, 7012 and 7013 of Figures 87C-87E and integrated circuits 7014 and 7015 of Figures 87C-87E include one or more communication modules, such as Ethernet module 7023 (in Figures 87C-87G) and Ethernet RDMA module 7022 (in Figures 87E and 87G).

Having such RDMA and/or Ethernet modules (either within the memory/processing unit or within the same integrated circuit as the memory/processing unit) greatly speeds up communication between the different elements of the disaggregated system, and in the case of RDMA, Greatly simplifies communication between different elements of a disaggregated system.

It should be noted that a memory/processing unit including RDMA and/or Ethernet modules may be beneficial in other environments, even when the memory/processing unit is not included in a disaggregated system.

It should also be noted that RDMA and/or Ethernet modules may be allocated for each group of memory/processing units, eg, for cost reduction reasons.

It should be noted that memory/processing units, groups of memory/processing units, and even processing/memory subsystems may include other communication ports, such as PCIe communication ports.

Using RDMA and/or Ethernet modules can be cost effective because the need to connect the memory/processing unit to a bridge to a network integrated circuit (NIC) that can have an Ethernet port can be eliminated.

The use of RDMA and/or Ethernet modules can make Ethernet (or Ethernet RDMA) native to the memory/processing unit.

It should be noted that an Ethernet network is only an example of a local area network (LAN) protocol. PCIe is just an example of another communication protocol that can be used over longer distances than Ethernet.

87H illustrates a method 7000 for distributed processing.

Method 7000 may include one or more processing iterations.

Processing iterations may be performed by one or more memory processing integrated circuits of the disaggregated system.

Processing iterations may be performed by one or more processing integrated circuits of the disaggregated system.

Processing iterations performed by more memory processing integrated circuits may be followed by processing iterations performed by one or more processing integrated circuits.

Processing iterations performed by more memory processing integrated circuits may precede processing iterations performed by one or more processing integrated circuits.

Yet another processing iteration may be performed by other circuits of the decomposed system. For example, the one or more preprocessing circuits may perform any type of preprocessing, including preparing units of information for processing iterations performed by one or more memory processing integrated circuits.

Method 7000 may include a step 7020 of receiving a unit of information by one or more memory processing integrated circuits of the disaggregated system.

Each memory processing integrated unit may include a controller, multiple processor sub-units, and multiple memory units.

One or more memory processing integrated circuits may be fabricated by a memory class of fabrication process.

The information unit may convey part of the model of the neural network.

The information element may deliver partial results of at least one database query.

The information element may deliver partial results of at least one aggregated database query.

Step 7020 may include receiving information units from one or more storage subsystems of the disaggregated system.

Step 7020 may include receiving information units from one or more computing subsystems of the disaggregated system, which may include a plurality of processing integrated circuits fabricated by a logical class of manufacturing processes.

Step 7020 may be followed by step 7030 of performing processing operations on the information elements by one or more memory processing integrated circuits to provide processing results.

The total size of the information elements may exceed, may be equal to, or may be less than the total size of the processing result.

Step 7030 may be followed by step 7040 of outputting the processing results by one or more memory processing integrated circuits.

Step 7040 may include outputting the processing results to one or more computing subsystems of the disaggregated system, which may include multiple processing integrated circuits fabricated by a logical class of manufacturing processes.

Step 7040 may include outputting the processing results to one or more storage subsystems of the disaggregated system.

Information units may be sent from different groups of processing units of multiple processing integrated circuits, and may be different parts of intermediate results of a processing program executed in a distributed fashion by multiple processing integrated circuits. The group of processing units may include at least one processing integrated circuit.

Step 7030 may include processing the information unit to provide the results of the overall processing procedure.

Step 7040 may include sending the results of the entire processing procedure to each of the plurality of processing integrated circuits.

The different parts of the intermediate result may be different parts of the updated neural network model, and wherein the result of the entire process is the updated neural network model.

Step 7040 may include sending the updated neural network model to each of a plurality of processing integrated circuits.

Step 7040 may be followed by step 7050 of performing, by the plurality of processing integrated circuits, another process based at least in part on the processing results to the plurality of processing integrated circuits.

Step 7040 may include outputting the processing results using the switching subunits of the decomposed system.

Step 7020 may include receiving information units, which are preprocessed information units.

Figure 87I illustrates a method 7001 for distributed processing.

Method 7001 differs from method 7000 in that it includes step 7010 of preprocessing information by a plurality of processing integrated circuits to provide preprocessed information units.

Step 7010 may be followed by steps 7010, 7020, 7030, and 7040.

Database Analytics Acceleration

An apparatus, method, and computer-readable medium are provided that store instructions for performing at least screening by a screening unit belonging to the same integrated circuit as the memory unit, wherein the filter may prompt Which entries are relevant to a database query. An arbiter or any other flow control manager can send relevant entries to the processor and not send irrelevant entries to the processor, thus saving almost most of the traffic to and from the processor.

See, eg, FIG. 91A, which shows a processor (CPU 9240), an integrated circuit including memory and a screening system 9220. Memory and screening system 9220 may include screening unit 9224 coupled to memory unit entries 9222 and one or more arbiters, such as arbiter 9229 for sending relevant entries to the processor. Any arbitration handler may be applied. There may be any relationship between the number of entries, the number of filter units and the number of arbiters.

The arbiter can be replaced by any unit capable of controlling the flow of information, such as communication interfaces, flow controllers, and the like.

Reference screening, which is based on one or more correlation/screening criteria.

Correlations can be set for each database query, and correlations can be prompted in any manner, for example a memory unit can store a correlation flag 9224' that indicates which entry is relevant. There is also a storage device 9210 that stores K database segments 9220(k), where k ranges between 1 and K. It should be noted that the entire database can be stored in the memory unit and not in the storage device (this solution is also referred to as a volatile memory-stored database).

A memory cell entry may be too small to store the entire database, and thus can receive one segment at a time.

The filtering unit may perform filtering operations, such as comparing the value of a field with a threshold, comparing the value of a field with a predefined value, determining whether the value of a field is within a predefined range, and the like.

Thus, the screening unit can perform known database screening operations and can be a compact and inexpensive circuit.

The final result of the filtering operation (eg, the contents of the relevant database entry) 9101 is sent to the CPU 9420 for processing.

The memory and screening system 9220 may be replaced by a memory and processing system as illustrated in Figure 91B.

The memory and processing system 9229 includes a processing unit 9225 coupled to the memory unit entry 9222. Processing unit 9225 may perform filtering operations and may participate, at least in part, in performing one or more additional operations on related records.

A processing unit may be customized to perform particular operations and/or may be a programmable unit configured to perform multiple operations. For example, a processing unit may be a pipelined processing unit, may include an ALU, may include multiple ALUs, and the like.

Processing unit 9225 may perform all of the one or more additional operations.

Alternatively, a portion of the one or more additional operations is performed by a processing unit, and the processor (CPU 9240) may perform another portion of the one or more additional operations.

The final result of the processing operation (eg, a partial response to a database query 9102, or a complete response 9103) is sent to the CPU 9420.

Part of the response requires further processing.

92A illustrates a memory/processing system 9228 including a memory/processing unit 9227 configured to perform screening and additional processing.

The memory/processing system 9228 implements the processing and memory units of FIG. 91 with the memory/processing unit 9227.

The role of the processor may include controlling a processing unit, performing at least a portion of one or more additional operations, and the like.

The combination of memory entry and processing unit may be implemented, at least in part, by one or more memory/processing units.

92B illustrates an example memory/processing unit 9010.

The memory/processing unit 9010 includes a controller 9020, an internal bus 9021, and pairs of logic 9030 and memory banks 9040. The controller is configured to operate as or couplable to the communication module.

Connectivity between controller 9020 and pairs of logic 9030 and memory banks 9040 may be implemented in other ways. Memory banks and logic can be configured in other ways (not in pairs). Multiple memory banks may be coupled to and/or managed by a single logic.

The memory/processing system receives database query 9100 via interface 9211. The interface 9211 can be a bus, port, input/output interface, and the like.

It should be noted that responses to database queries may be generated by at least one of (or a combination of one or more of): one or more memories/processing systems, one or more memories and processing systems , one or more memory and screening systems, one or more processors external to these systems, and the like.

It should be noted that responses to database queries may be generated by at least one of (or a combination of one or more of): one or more filtering units, one or more memory/processing units, one or more or more processing units, one or more other processors (such as one or more other CPUs), and the like.

Any processing procedure may include finding the relevant database entry, and processing the relevant database entry. Processing may be performed by one or more processing entities.

The processing entity may be at least one of: a processing unit of a memory and processing system (eg, processing unit 9225 of memory and processing system 9229), a processor sub-unit (or logic) of a memory/processing unit, another A processor (eg, CPU 9240 of Figures 91A, 91B, and 74), and the like.

The processing involved in generating a response to a database query may result from any one or a combination of the following:

a. Processing unit 9225 of memory and processing system 9229.

b. Processing unit 9225 of different memory and processing system 9229.

c. The processor sub-unit (or logic 9030) of one or more memory/processing units 9227 of the memory/processing system 9228.

d. The processor sub-unit (or logic 9030) of the memory/processing unit 9227 of the different memory/processing system 9228.

e. A controller of one or more memory/processing units 9227 of the memory/processing system 9228.

f. Controllers of one or more memory/processing units 9227 of different memory/processing systems 9228.

Thus, the processing involved in responding to a database query may result from a combination or sub-combination of (a) one or more controllers of one or more memory/processing units, (b) a memory processing system's One or more processing units, (c) one or more processor sub-units of one or more memory/processing units, and (d) one or more other processors, and the like.

Processing performed by more than one processing entity may be referred to as distributed processing.

It should be noted that screening may be performed by screening entities in one or more screening units and/or one or more processing units and/or one or more processor sub-units. In this sense, processing units and/or processor sub-units that perform screening operations may be referred to as screening units.

The processing entity can be a screening entity or can be different from the screening entity.

A processing entity may perform processing operations that are considered relevant database entries by another filtering entity.

Processing entities can also perform filtering operations.

Responses to database queries may utilize one or more filtering entities and one or more processing entities.

One or more screening entities and one or more processing entities may belong to the same system (eg, memory/processing system 9228, memory and processing system 9229, memory and screening system 9220) or belong to different systems.

The memory/processing unit may include multiple processor sub-units. The processor subunits may operate independently of each other, may cooperate in part with each other, may participate in distributed processing, and the like.

92C illustrates a number of memory and screening systems 9220, a number of other processors (such as a CPU 9240), and a storage device 9210.

A plurality of memory and screening systems 9220 may participate (simultaneously or not simultaneously) in the screening of one or more database entries based on one or more screening criteria within one of a plurality of database queries.

92D illustrates a plurality of memory and processing systems 9229, a plurality of other processors (such as CPU 9240), and storage device 9210.

A plurality of memory and processing systems 9229 may participate (simultaneously or not) in the screening and at least part of the processing involved in responding to one of the plurality of database queries.

92F illustrates multiple memory/processing systems 9228, multiple other processors (such as CPU 9240), and storage device 9210.

Multiple memory/processing systems 9228 may participate (simultaneously or not) in the filtering and at least part of the processing involved in responding to one of the multiple database queries.

Figure 92G illustrates a method 9300 method for database analysis acceleration.

The method 9300 may begin at step 9310 of receiving, by the memory processing integrated circuit, a database query including at least one relevance criterion that prompts a database entry in the database that is relevant to the database query.

The database entries in the database related to the database query may not be the database entries of the database, but may be one, some or all of the database entries of the database.

A memory processing integrated circuit may include a controller, multiple processor sub-units, and multiple memory units.

Step 9310 may be followed by step 9320 of determining, by the memory processing integrated circuit and based on at least one correlation criterion, a group of related database entries stored in the memory processing integrated circuit.

Step 9320 may be followed by sending the group of related database entries to one or more processing entities for further processing without substantially sending unrelated data entries stored in the memory processing integrated circuit to the one or more processing entities Entity step 9330.

The phrase "without substantially sending" means not sending at all (during responding to a database query) or sending a small number of irrelevant entries. Not much can mean at most 1, 2, 3, 4, 5, 9, 7, 8, 9, 10 percent, or send any amount that does not significantly affect bandwidth.

Step 9330 may be followed by a step 9340 of processing groups of related database entries to provide responses to database queries.

Figure 92H illustrates a method 9301 for database analysis acceleration.

It is assumed that the filtering and overall processing required to respond to database queries is performed by the memory processing integrated circuit.

The method 9301 may begin at step 9310 of receiving, by the memory processing integrated circuit, a database query including at least one relevance criterion that prompts a database entry in the database that is relevant to the database query.

Step 9310 may be followed by step 9320 of determining, by the memory processing integrated circuit and based on at least one correlation criterion, a group of related database entries stored in the memory processing integrated circuit.

Step 9320 may be followed by sending the group of related database entries to one or more processing entities of the memory processing integrated circuit for full processing without substantially sending unrelated data entries stored in the memory processing integrated circuit to the memory processing integrated circuit. Step 9331 of one or more processing entities.

Step 9331 may be followed by step 9341 of fully processing the group of related database entries to provide a response to the database query.

Step 9341 may be followed by step 9351 of outputting a response to the database query from the memory processing integrated circuit.

Figure 92I illustrates a method 9302 for database analysis acceleration.

It is assumed that only a portion of the filtering and processing required to respond to database queries is performed by the memory processing integrated circuit. The memory processing integrated circuit will output partial results, which will be processed by one or more other processing entities external to the memory processing integrated circuit.

The method 9301 can begin at step 9310 of receiving, by the memory processing integrated circuit, a database query including at least one relevance criterion that prompts a database entry in the database that is relevant to the database query.

Step 9310 may be followed by step 9320 of determining, by the memory processing integrated circuit and based on at least one correlation criterion, a group of related database entries stored in the memory processing integrated circuit.

Step 9320 may be followed by sending the group of related database entries to one or more processing entities of the memory processing integrated circuit for partial processing without substantially sending unrelated data entries stored in the memory processing integrated circuit to the memory processing integrated circuit. Step 9332 of one or more processing entities.

Step 9332 may be followed by a step 9342 of partially processing the group of related database entries to provide an intermediate response to the database query.

Step 9342 may be followed by step 9352 of outputting an intermediate response to the database query from the memory processing integrated circuit.

Step 9352 may be followed by step 9390 of further processing the intermediate response to provide a response to the database.

Figure 92J illustrates a method 9303 for database analysis acceleration.

It is assumed that the memory processing integrated circuit performs the filtering of the relevant database entries, but does not perform the processing of the relevant database entries. The memory processing integrated circuit will output a group of related database entries to be fully processed by one or more other processing entities external to the memory processing integrated circuit.

The method 9301 can begin at step 9310 of receiving, by the memory processing integrated circuit, a database query including at least one relevance criterion that prompts a database entry in the database that is relevant to the database query.

Step 9310 may be followed by step 9320 of determining, by the memory processing integrated circuit and based on at least one correlation criterion, a group of related database entries stored in the memory processing integrated circuit.

Step 9320 may be followed by sending groups of related database entries to one or more processing entities external to the memory processing integrated circuit without substantially sending unrelated data entries stored in the memory processing integrated circuit to the one or more processing entities. Step 9333 of multiple processing entities.

Step 9333 may be followed by step 9391 of fully processing the intermediate response to provide a response to the database.

Figure 92K illustrates a method 9304 for database analysis acceleration.

The method 9303 may begin at step 9315 of receiving, by the integrated circuit, a database query, the database query including at least one relevance criterion prompting database entries in the database that are relevant to the database query; wherein the integrated circuit includes a controller, a screening unit, and a plurality of memory unit.

Step 9315 may be followed by a step 9325 of determining, by a screening unit and based on at least one relevance criterion, a group of related database entries stored in the integrated circuit.

Step 9325 may be followed by sending the group of related database entries to one or more processing entities external to the integrated circuit for further processing without substantially sending unrelated data entries stored in the integrated circuit to one or more processing entities. Step 9335 of a processing entity.

Step 9335 may be followed by step 9391.

Figure 92L illustrates a method 9305 for database analysis acceleration.

The method 9305 may begin at step 9314 of receiving, by the integrated circuit, a database query, the database query including at least one relevance criterion prompting a database entry in the database that is relevant to the database query; wherein the integrated circuit includes a controller, a screening unit, and a plurality of memories unit.

Step 9314 may be followed by step 9324 of determining, by the processing unit, a group of related database entries stored in the integrated circuit based on at least one correlation criterion.

Step 9324 may be followed by step 9334 of processing the group of related database entries by the processing unit without processing the unrelated data entries stored in the integrated circuit by the processing unit to provide the processing result.

Step 9334 may be followed by step 9344 of outputting the processing results from the integrated circuit.

In any of methods 9300, 9301, 9302, 9304, and 9305, the memory processing integrated circuit outputs an output. The output may be a related database entry, one or more intermediate results, or a group of one or more (complete) results.

The output may be preceded by one or more relevant database entries and/or one or more results (full or intermediate) retrieved from a screening entity and/or a processing entity of the memory processing integrated circuit.

The retrieval may be controlled in one or more ways and may be controlled by an arbiter and/or one or more controllers of the memory processing integrated circuit.

Exporting and/or retrieving may include one or more parameters that control the ingesting and/or exporting. The parameters may include acquisition timing, acquisition rate, acquisition source, bandwidth, order or acquisition, output timing, output rate, output source, bandwidth, order or output, type of acquisition method, type of arbitration method, and its similar.

Export and/or capture executable flow control handlers.

Exporting and/or retrieving (eg, applying a flow control handler) may be in response to an indicator output from one or more processing entities of completion of processing of the database entry for the group. An indicator may indicate whether the intermediate result is ready to be retrieved by the self-processing entity.

The output may include an attempt to match the bandwidth used during the output to the maximum allowable bandwidth on the link coupling the memory processing integrated circuit to the requestor unit. The link may be a link to a recipient of the output of the memory processing integrated circuit. The maximum allowable bandwidth may be specified by the capacity and/or availability of the link, the capacity and/or availability of the recipient of the output content, and the like.

Outputting may include attempting to output the output in an optimal or sub-optimal manner.

The output of the output content may include an attempt to maintain fluctuations in the output traffic rate below a threshold.

Any of methods 9300, 9301, 9302, and 9305 may include generating, by one or more processing entities, a processing status indicator that may inform progress of further processing of the group of related database entries.

When the processing included in any of the above-mentioned methods is performed by more than a single processing entity, then the processing may be considered distributed processing because the processing is performed in a distributed fashion.

As hinted above, processing can be performed in a hierarchical fashion or in a planar fashion.

Any of methods 9300-9305 may be performed by multiple systems that may respond to one or more database queries simultaneously or sequentially.

word embedding

As mentioned above, word embedding is an umbrella term for a collection of language modeling and feature learning techniques in natural language processing (NLP), where words or phrases from a vocabulary are mapped to vectors of elements. Conceptually, word embeddings involve mathematical embeddings from a space with many dimensions per word to a continuous vector space with much lower dimensions.

These vectors can be mathematically processed. For example, vectors belonging to a matrix can be summed to provide a summed vector.

For yet another example, the covariance of the matrix (of the sentence) may be calculated. This can include multiplying the matrix by its transpose.

The memory/processing unit may store the vocabulary. In particular, portions of the vocabulary may be stored in multiple memory banks of the memory/processing unit.

Thus, the memory/processing unit may be accessed using access information (such as a retrieval key) for the set of words that will represent the phrase's phrase, such that representations will be retrieved from at least some of the memory/processing unit's memory bank A vector of words for the phrase of the sentence.

Different memory banks of the memory/processing unit may store different parts of the vocabulary and may be accessed in parallel (depending on the distribution of the indexes of the statement). Prediction reduces penalties even when more than a single row of memory banks needs to be accessed sequentially.

The allocation of vocabulary words between different memory banks of memory/processing units can be optimized, or the allocation can be highly beneficial in the sense that the chance of parallel access per statement to different memory banks of memory/processing units can be increased. The assignment can be learned per user, per general group or per population.

In addition, the memory/processing unit may also be used to perform at least some of the processing operations (by virtue of its logic) and thereby reduce the bandwidth required from the bus external to the memory/processing unit, allowing multiple computations to be performed in an efficient manner (even in parallel). operations (multiple processors using memory/processing units in parallel).

Memory groups can be logically associated.

At least a portion of the processing operations may be performed by one or more additional processors, such as vector processors, including but not limited to vector adders.

The memory/processing units may include one or more additional processors that may be assigned to some or all of the memory banks (logical pairs).

Thus, a single additional processor may be assigned to all or some of the memory banks (logical pairs). For yet another example, the additional processors may be configured in a hierarchical manner, such that additional processors at a certain level process output from additional processors at lower levels.

It should be noted that processing operations may be performed without the use of any additional processors, but may be performed by the logic of the memory/processing unit.

89A, 89B, 89C, 89D, 89E, 89F, and 89G illustrate examples of memory/processing units 9010, 9011, 9012, 9013, 9014, 9015, and 9019, respectively. The memory/processing unit 9010 includes a controller 9020, an internal bus 9021, and pairs of logic 9030 and memory banks 9040.

It should be noted that logic 9030 and memory bank 9040 may be coupled to the controller and/or to each other in other ways, for example, multiple buses may be provided between the controller and logic, logic may be configured in multiple layers, a single logic Can be shared by multiple memory banks (see, eg, Figure 89E), and the like.

The length of the pages of each memory bank within the memory/processing unit 9010 may be defined in any way, for example, it may be small enough and the number of memory banks may be large enough to enable a large number of vectors to be output in parallel without interfering with irrelevant information. A lot of bits are wasted.

Logic 9020 may include full ALUs, partial ALUs, memory controllers, partial memory controllers, and the like. A partial ALU (memory controller) unit can perform only a portion of the functions that can be performed by a full ALU (memory controller). Any logic or sub-processors described in this application may include full ALUs, partial ALUs, memory controllers, partial memory controllers, and the like.

Connectivity between controller 9020 and pairs of logic 9030 and memory banks 9040 may be implemented in other ways. Memory banks and logic can be configured in other ways (not in pairs).

Memory/processing unit 9010 may not have additional vectors, and processing of vectors (from memory banks) is performed by logic 9030.

89B illustrates an additional processor, such as a vector processor 9050 coupled to the internal bus 9021.

89C illustrates an additional processor, such as a vector processor 9050 coupled to the internal bus 9021. One or more additional processors perform (alone or in conjunction with logic) the processing operations.

89D also illustrates the host 9018 coupled to the memory/processing unit 9010 via the bus 9022.

89D also illustrates a vocabulary 9070 that maps words/phrases 9072 to vectors 9073. The memory/processing unit is accessed using retrieval keys 9071, each retrieval key representing a previously recognized word or phrase. The host 9018 sends a plurality of retrieval keys 9071 representing the statement to the memory/processing unit, and the memory/processing unit can output a vector 9070 or the final result of the processing operation applied by the vector associated with the statement. Words/phrases are generally not stored in memory/processing unit 9010.

Memory controller functionality for controlling memory banks may be included (individually or partially) in logic, may be included (individually or partially) in controller 9020, and/or may be included (individually or partially) in memory/ In one or more memory controllers (not shown) within the processing unit 9010.

The memory/processing unit may be configured to maximize throughput of vectors/results sent to the host 9018, or may be applied to control internal memory/processing unit traffic and/or control the memory/processing unit and the host computer (or memory/processing unit). Any handler for business between any other entity outside the cell).

Various logics 9030 are coupled to memory banks 9040 of memory/processing units and can perform mathematical operations (preferably in parallel) on the vectors to produce processed vectors. One logic 9030 can send the vector to another logic (see, eg, line 38 of Figure 89G), and the other logic can apply mathematical operations on the received vector and its computed vector. Logic may be configured in layers, and logic at one level may process vectors or intermediate results (produced by applying mathematical operations) from logic at a previous level. These logics can form trees (binary, ternary, and the like).

When the total size of the processed vectors exceeds the total size of the result, then a reduction in output bandwidth (outside the memory/processing unit) is obtained. For example, when K vectors are summed by the memory/processing unit to provide a single output vector, then a K:1 reduction in bandwidth is obtained.

The controller 9020 can be configured to open multiple memory banks in parallel by broadcasting addresses of different vectors to be accessed.

The controller may be configured to control fetching of different values from the plurality of memory banks (or from any intermediate buffers or storage circuits that store different vectors, see buffer 9033 of FIG. 89D ) based at least in part on the order of the words or phrases in the statement. order of vectors.

The controller 9020 can be configured to manage the fetching of the different vectors based on one or more parameters related to outputting the vectors external to the memory/processing unit 9010, for example, the rate at which the different vectors are fetched from the memory bank can be set to be substantially Equal to the allowable rate at which different vectors are output from the memory/processing unit 9010.

The controller can output different vectors outside the memory/processing unit 9010 by applying any traffic shaping handler. For example, the controller 9020 may aim to output the different vectors at a rate as close as possible to the maximum rate allowed by the host computer or the link coupling the memory/processing unit 9010 to the host computer. For yet another example, the controller may output different vectors while minimizing, or at least substantially reducing, fluctuations in traffic rates over time.

Controller 9020 belongs to the same integrated circuit as memory bank 9040 and logic 9030, and thus can easily receive fetch status for different vectors from different logic/memory banks (eg, is the vector ready, is the vector ready but from the same memory group is capturing or about to capture feedback from another vector), and the like. Feedback can be provided in any way: via dedicated control lines, via shared control lines. One or more status bits and the like are used (see status line 9039 of Figure 89F).

The controller 9020 can independently control the capture and output of different vectors, and thus can reduce the involvement of the host computer. Alternatively, the host computer may be unaware of the management capabilities of the controller and may continue to send detailed instructions, and in this case the memory/processing unit 9010 may ignore the detailed instructions, may hide the management capabilities of the controller, and the like. The above mentioned solutions can be used based on a protocol that can be managed by the host computer.

It has been found that it is extremely beneficial (in terms of energy) to perform processing operations in the memory/processing unit even when these operations consume more power than processing operations in the host and even when these operations are The same is true when transfer operations between memory/processing units consume more power. For example, assuming that the vector is large enough, the energy consumption of transmitting a data unit is 4pJ, and the energy consumption of the processing operation of the data unit (by the host) is 0.1pJ, then the energy consumption of processing the data unit by the memory/processing unit is low At 5pJ, it is more efficient to process data units by the memory/processing unit.

Each vector (of the matrix representing the statement) may be represented by a sequence of words (or other multi-bit segments). For simplicity of explanation, multiple bit segments are assumed to be words.

Additional power savings may be obtained when the vector includes zero-valued words. Instead of outputting the entire zero-valued word, a zero-valued flag that is shorter than the word (eg, bits) (even delivered via a dedicated control line) may be output instead of the entire word. Flags can be assigned to other values (eg, words of value 1).

88A illustrates a method 9400 for embedding, or rather, a method for extracting feature vector related information. The feature vector related information may include the feature vector and/or the result of processing the feature vector.

The method 9400 may begin with a step 9410 of receiving, by a memory processing integrated circuit, fetch information for retrieving a plurality of requested feature vectors, which may be mapped to a plurality of sentence segments.

A memory processing unit may include a controller, multiple processor sub-units, and multiple memory units. Each of the memory units may be coupled to a processor sub-unit.

Step 9410 may be followed by step 9420 of retrieving the plurality of requested feature vectors from at least some of the plurality of memory cells.

The fetching may include concurrently requesting two or more memory cells for the requested feature vector stored in the two or more memory cells.

The request is performed based on a known mapping between the statement segments and the locations of the feature vectors mapped to the statement segments.

This map may be uploaded during the power-on handler of the memory processing integrated circuit.

It may be beneficial to fetch as many requested feature vectors as possible at once, but this depends on where the requested feature vectors are stored and on the number of different memory cells.

If more than one requested feature vector is stored in the same memory bank, predictive fetching may be applied for reducing penalties associated with fetching information from the memory bank. Various methods for reducing penalties are described in various sections of this application.

Fetching may include applying predictive fetching of at least some of the set of requested feature vectors stored in a single memory unit.

The requested feature vectors can be distributed among the memory cells in an optimal manner.

The requested feature vectors may be distributed among the memory cells based on the expected retrieval pattern.

The retrieval of the plurality of requested feature vectors may be performed according to a certain order. For example, according to the order of statement sections in one or more statements.

The retrieval of the plurality of requested feature vectors may be performed at least partially out of order; and wherein the retrieval may further include reordering the plurality of requested feature vectors.

The retrieval of the plurality of requested features may include buffering the plurality of requested feature vectors before the plurality of requested feature vectors can be read by the controller.

The retrieval of the plurality of requested features may include generating buffer status indicators that indicate when one or more buffers associated with the plurality of memory cells are storing the one or more requested feature vectors.

The method may include delivering the buffer status indicator via a dedicated control line.

A dedicated control line can be assigned per memory cell.

The buffer status indicators may be status bits stored in one or more buffers.

The method may include conveying the buffer status indicator via one or more shared control lines.

Step 9420 may be followed by step 9430 of processing the plurality of requested feature vectors to provide processing results.

Additionally or alternatively, step 9420 may be followed by a step 9440 of outputting from the memory the integrated circuit output, which may include at least one of: (a) the requested feature vector; and (b) processing the requested feature vector the result of. At least one of (a) the requested feature vector and (b) the result of processing the requested feature vector is also referred to as feature vector related information.

When step 9430 is performed, then step 9440 may include outputting (at least) the result of processing the requested feature vector.

When step 9430 is skipped, then step 9440 includes outputting the requested feature vector and may not include outputting the result of processing the requested feature vector.

Figure 88B illustrates a method 9401 for embedding.

The output is assumed to include the requested feature vector, but not the result of processing the requested feature vector.

Method 9401 may begin at step 9410 of receiving, by a memory processing integrated circuit, fetch information for retrieving a plurality of requested feature vectors, which may be mapped to a plurality of sentence segments.

Step 9410 may be followed by step 9420 of retrieving the plurality of requested feature vectors from at least some of the plurality of memory cells.

Step 9420 may be followed by step 9431 of outputting from the memory processing integrated circuit an output that includes the requested feature vector but does not include the result of processing the requested feature vector.

Figure 88C illustrates a method 9402 for embedding.

The output is assumed to include the result of processing the requested feature vector.

The method 9402 may begin with a step 9410 of receiving, by the memory processing integrated circuit, fetch information for retrieving a plurality of requested feature vectors, which may be mapped to a plurality of sentence segments.

Step 9410 may be followed by step 9420 of retrieving the plurality of requested feature vectors from at least some of the plurality of memory cells.

Step 9420 may be followed by step 9430 of processing the plurality of requested feature vectors to provide processing results.

Step 9430 may be followed by a step 9442 of processing the output of the integrated circuit output from the memory, which may include processing the results of the requested feature vector.

The output of the output may include applying business shaping to the output.

The output of the output may include an attempt to match the bandwidth used during the output with the maximum allowable bandwidth on the link coupling the memory processing integrated circuit to the requestor unit.

The output of the output may include an attempt to maintain fluctuations in the output traffic rate below a threshold.

Any of the steps of capturing and exporting may be performed under the control of the host and/or independently or in part by the controller.

The host can send fetch commands with different granularities, from generally sending fetch information regardless of the location of the requested feature vector within multiple memory cells, to sending detailed information based on the location of the requested feature vector within multiple memory cells. Fetch information.

The host may control (or attempt to control) the timing of the different fetch operations within the memory processing integrated circuit, but may be independent of timing.

The controller can be controlled at various levels by the host, and can even ignore detailed commands from the host, and independently control at least capture and/or output.

The processing of the requested feature vector may be performed by at least one of (a combination of one or more of): one or more memories/processing units, and external to the one or more memories/processing units one or more processors, and the like.

It should be noted that the processing of the requested feature vector may be performed by at least one of the following (a combination of one or more of the following): one or more processor sub-units, a controller, one or more vectors A processor, and one or more memory/processing units external to the one or more memory/processing units.

The processing of the requested feature vector may be performed by any one or a combination of the following, and may result from any one or a combination of the following:

a. The processor sub-unit (or logic 9030) of the memory/processing unit.

b. Processor sub-unit (or logic 9030) of multiple memory/processing units.

c. The controller of the memory/processing unit.

d. A controller for multiple memory/processing units.

e. One or more vector processors of the memory/processing unit.

f. One or more vector processors, multiple memory/processing units.

Accordingly, processing of the requested feature vector may be performed by any combination or subcombination of: (a) one or more controllers of one or more memory/processing units; (b) one or more memory/processing units (c) one or more vector processors of one or more memory/processing units; and (d) one or more other processors external to one or more memory/processing units processor.

Processing performed by more than one processing entity may be referred to as distributed processing.

The memory/processing unit may include multiple processor sub-units. The processor subunits may operate independently of each other, may cooperate in part with each other, may participate in distributed processing, and the like.

Processing may be performed in a planar fashion, where all processor sub-units perform the same operations (and may or may not output processing results between them).

Processing may be performed in a hierarchical fashion, where processing involves a sequence of processing operations at different levels, with processing operations at one level following processing operations at another level. Processor subunits may be assigned (dynamically or statically) to different layers and participate in hierarchical processing.

Any processing of the requested feature vectors may be performed by more than one processing entity (processor subunits, controllers, vector processors, other processors), distributed processing may be performed in any manner (either flat, hierarchical or otherwise). For example, the processor sub-unit can output its processing results to the controller, which can further process the results. One or more other processors external to the one or more memory/processing units may further process the output of the memory processing integrated circuit.

It should be noted that the retrieval information may also include information for retrieving requested feature vectors that do not map to sentence segments. These feature vectors can be mapped to one or more persons, devices, or any other entities that can be associated with sentence segments. For example, the user of the device that sensed the sentence segment, the device that sensed the segment, the user identified as the source of the sentence segment, the website accessed when the sentence was generated, the location where the sentence was captured, and the like .

Methods 9400, 9401, and 9402, mutatis mutandis, may be applicable to processing and/or requested retrieving vectors that do not map to sentence segments.

Non-limiting examples of processing of feature vectors may include summing, weighted summing, averaging, subtracting, or applying any other mathematical function.

mixing device

As both processor speed and memory size continue to increase, a significant limit to effective processing speed is a von Neumann bottleneck. Von Neumann bottlenecks are caused by throughput limitations imposed by traditional computer architectures. In particular, the transfer of data from memory to the processor (outside the logic die such as external DRAM memory) often encounters a bottleneck compared to the actual operations performed by the processor. Consequently, the number of clock cycles used to read and write memory increases significantly with memory-intensive processes. These clock cycles result in lower effective processing speeds because reading and writing the memory consumes clock cycles that cannot be used to perform operations on the data. In addition, the operational bandwidth of the processor is typically greater than the bandwidth of the bus used by the processor to access the memory.

These bottlenecks are particularly evident for: memory-intensive processing programs, such as neural networks and other machine learning algorithms; database construction, index searches, and queries; and other tasks that include more read and write operations than data processing operations.

The present disclosure describes solutions for alleviating or overcoming one or more of the problems set forth above, as well as other problems in the prior art.

A hybrid device for memory intensive processing can be provided that can include a base die, a plurality of processors, at least a first memory resource of another die, and a second memory resource of at least one other die .

The base die and the at least one other die are connected to each other by wafer-on-wafer bonding.

A plurality of processors are configured to perform processing operations and retrieve the retrieved information stored in the first memory resource.

The second memory resource is configured to send additional information from the second memory resource to the first memory resource.

The total bandwidth of the first path between the base die and the at least one other die exceeds the total bandwidth of the second path between the at least one other die and the at least one other die, and the storage capacity of the first memory resource is the th Part of the storage capacity of two memory resources.

The second memory resource is a high bandwidth memory (HBM) resource.

At least one other die is a stack of high bandwidth memory (HBM) chips.

At least some of the second memory resources may belong to another die that is connected to the base die by connectivity other than inter-wafer bonding.

At least some of the second memory resources belong to another die that is connected to the other die by a different connectivity than the inter-wafer bonding.

The first memory resource and the second memory resource are caches of different levels.

The first memory resource is located between the base die and the second memory resource.

The first memory resource is located to one side of the second memory resource.

Another die is configured to perform additional processing, wherein the other die includes a plurality of processor subunits and a first memory resource.

Each processor subunit is coupled to a unique portion of the first memory resource allocated to the processor subunit.

The only part of the first memory resource is at least one memory bank.

The plurality of processors are the plurality of processor sub-units included in the first memory resource of the memory processing chip.

The base die includes a plurality of processors, where the plurality of processors are a plurality of processor subunits coupled to a first memory resource via conductors formed using wafer-to-wafer bonding.

Each processor subunit is coupled to a unique portion of the first memory resource allocated to the processor subunit.

Hybrid integrated circuits can be provided that can utilize wafer-on-wafer (WOW) connectivity to couple at least a portion of the base die to second memory resources included in one or more other dies and connected using a different connectivity than WOW connectivity. An example of the second memory resource may be a high bandwidth memory (HBM) memory resource. In the various figures, a second memory resource is included in a stack of HBM memory cells, which may be coupled to a controller using through-silicon (TSV) connectivity. The controller may be included in the base die or coupled (eg, via microbumps) to at least a portion of the base die.

The base die can be a logic die, but can be a memory/processing unit.

WOW connectivity is used to couple one or more portions of a base die to one or more portions of another die (WOW connected die), which may be a memory die or memory/processing unit. WOW connectivity is very high throughput connectivity.

Stacks of high bandwidth memory (HBM) chips can be coupled to the base die (direct or via WOW connected die) and can provide high throughput connections and very broad memory resources.

The WOW connected die can be coupled between the stack of HBM chips and the base die to form a HBM memory chip stack with TSV connectivity and the WOW connected die at its bottom.

A stack of HBM chips with TSV connectivity and WOW-connected dies at the bottom can provide a multi-layer memory hierarchy, where the WOW-connected dies can be used as lower-level memory accessible by the base die (eg, 3-level high-speed cache), where fetches and/or prefetch operations from higher-level HBM memory stacks populate WOW-connected dies.

The HBM memory chips can be HBM DRAM chips, but any other memory technology can be used.

Using WOW connectivity in combination with HMB chips enables the provision of multi-layer memory structures that can include multiple memory layers that can provide different trade-offs between bandwidth and memory density.

The proposed solution can act as an additional new memory tier between traditional DRAM memory/HBM to the logic die's internal cache, enabling more bandwidth and better management and reuse on the DRAM side.

This can provide a new memory hierarchy on the DRAM side that better manages memory reads in a fast manner.

93A to 93I illustrate hybrid integrated circuits 11011' to 11019', respectively.

Figure 93A illustrates a HBM DRAM stack with TSV connectivity and microbumps at the lowest level (collectively denoted 11030) that includes a first memory controller coupled to each other and to the base die using TSVs (11039) Stack of 11031 HDM DRAM memory chips 11032.

Figure 93A also illustrates a wafer (collectively denoted 11040) having at least memory resources and coupled using WOW technology, the wafer including a base coupled to a DRAM wafer (11021) via one or more WOW interlayers (11023) Second memory controller 11022 of die 11019. The one or more WOW interlayers may be made of different materials, but may be different from the pad connectivity and/or may be different from the TSV connectivity.

Conductors 11022' through one or more WOW interlayers electrically couple the DRAM die to components of the base die.

Base die 11019 is coupled to interposer 11018, which in turn is coupled to package substrate 11017 using microbumps. The package substrate has an array of micro bumps at its lower surface.

Microbumps can be replaced by other connectivity. The interposer 11018 and the package substrate 11017 may be replaced by other layers.

The first and/or second memory controllers (11031 and 11032, respectively) may be located (at least in part) outside the base die 11019, such as in the DRAM wafer, between the DRAM wafer and the base die, HBM memory Between the stack of cells and the base die, and the like.

The first and/or second memory controllers (11031 and 11032, respectively) may belong to the same controller or may belong to different controllers.

One or more of the HBM memory units may include logic as well as memory, and may or may include a memory/processing unit.

The first and second memory controllers are coupled to each other by a plurality of buses 11016 for transferring information between the first memory resource and the second memory resource. Figure 93A also illustrates a bus 11014 from the second memory controller to components of the base die (eg, multiple processors). Figure 93A further illustrates the bus 11015 from the first memory controller to the components of the base die (eg, multiple processors, as shown in Figure 93C).

93B illustrates a hybrid integrated circuit 11012 that differs from the hybrid integrated circuit 11011 of FIG. 93A by having a memory/processing unit 11021' instead of a DRAM die 11021.

93C illustrates a hybrid integrated circuit 11013 that differs from the hybrid integrated circuit 11011 of FIG. 93A by having a stack of HBM memory chips with TSV connectivity and WOW connected dies (collectively denoted 11040 ) at their bottom , the die includes the DRAM die 11021 between the stack of HBM memory cells and the base die 11018 .

The DRAM die 11021 is coupled to the first memory controller 11031 of the base die 11019 using WOW technology (see WOW interlayer 11023). One or more of the HBM memory dies 11032 may include logic as well as memory, and may or may include a memory/processing unit.

The lowermost DRAM die (denoted as DEAM die 11021 in Figure 93C) may be an HBM memory die or may be different from the HBM die. The lowermost DRAM die (DRAM die 11021) may be replaced by memory/processing unit 11021', as illustrated by hybrid integrated circuit 11014 of Figure 93D.

Figures 93E-93G illustrate hybrid integrated circuits 11015, 11016, and 11016', respectively, with base die 11019 coupled to a HBM DRAM stack (11020) with TSV connectivity and microbumps at the lowest level and with at least memory resources And multiple instances of wafers (11030) coupled using WOW technology, and/or multiple instances of HBM memory chip stacks (11040) coupled to die with TSV connectivity and WOW-connected dies on the bottom .

93H illustrates a hybrid integrated circuit 11014' that differs from the hybrid integrated circuit 11014 of FIG. 93D in that it illustrates a memory cell 53, a L2 cache (L2 cache 52), a plurality of processors 11051. A plurality of processors 11051 are coupled to L2 cache 11052 and may be fed with coefficients and/or data stored in memory unit 11053 and L2 cache 11052.

Any of the hybrid integrated circuits mentioned above can be used for artificial intelligence (AI) processing, which is bandwidth intensive.

When coupled to a memory controller using WOW technology, the memory/processing unit 11021' of Figures 93D and 93H can perform AI computations and can receive data and coefficients from the HBM DRAM stack and/or from WOW connected dies at very high rates both.

Any memory/processing unit may include distributed memory arrays and processor arrays. A distributed memory and processor array may include multiple memory banks and multiple processors. Multiple processors may form a processing array.

93C, 93D, and 93H and assume that a hybrid integrated circuit (11013, 11014, or 11014') is required to perform general matrix-vector multiplications (GEMV), which include computing matrix-vector products. This type of computation is bandwidth intensive because there is no re-use of the extracted matrices. Therefore, the entire matrix only needs to be retrieved and used once.

GEMV may be part of a sequence of mathematical operations involving (i) multiplying a first matrix (A) by a first vector (V1) to provide a first intermediate vector, applying a first nonlinear operation (NLO1) to the first intermediate vector to provide a first intermediate result; (ii) multiplying the second matrix (B) by the first intermediate result to a second intermediate vector, applying a second nonlinear operation (NLO2) to the second intermediate vector to provide a second intermediate result, and so on (until the Nth intermediate result is received, N may exceed 2).

Assuming each matrix is large (eg, 1Gb), the computation will require 1Tbs of computing power and 1Tbs of bandwidth/throughput. Operations and calculations can be performed in parallel.

Assume that the GEMV calculation exhibits N=4 and has the form: result=NLO4(D*(NLO3(C*(NLO2(B*(NLO1(A*V1))))))).

Also assuming that DRAM die 11021 (or memory/processing unit 11021') does not have sufficient memory resources to store A, B, C, and D simultaneously, at least some of these matrices will be stored in HDM DRAM die 11032.

A base die is assumed to be a logic die that includes computational units such as, but not limited to, processors, arithmetic logic units, and the like.

When the first die calculates A*V1, the first memory controller 11031 retrieves missing portions of other matrices from one or more HBM DRAM dies 11032 for subsequent calculations.

93H and assume (a) DRAM die 11021 has 2TBs bandwidth and 512Mb capacity, (b) HBM DRAM die 11032 has 0.2TBs bandwidth and 8Gb capacity, and (c) L2 cache 11052 has 6Ts bandwidth and 10Mb capacity SRAM.

Matrix multiplication involves reusing the data, segmenting a large matrix into multiple segments (eg, 5Mb segments to fit in an L2 cache that can be used in a double buffer configuration) and dividing the extracted first matrix segment Multiply by the segments of the second matrix (one second matrix segment followed by another second matrix segment).

When multiplying the first matrix segment by the second matrix segment, another second matrix segment is fetched from the DRAM die 11021 (of the memory processing unit 11021') to the L2 cache.

Assuming that the matrices are each 1Gb, DRAM die 11021 or memory/processing unit 11021' is fed with matrix segments from HBM DRAM die 11032 when fetching and computing are performed.

The DRAM die 11021 or memory/processing unit 11021' gathers the matrix segments, and the matrix segments are then fed to the base die 11019 via the WOW interlayer (11023).

The memory/processing unit 11021' may reduce the amount of information sent to the base die 11019 via the WOW intermediate layer (11023) by performing calculations and sending the results rather than sending intermediate values calculated to provide the results. When multiple (Q) intermediate values are processed to provide a result, then the compression ratio may be Q to 1.

93I illustrates an example of a memory processing unit 11019' implemented using WOW technology. The logic unit 9030 (which may be a processor sub-unit), the controller 9020 and the bus 9021 are located in one chip 111061, the memory banks 9040 allocated to different logic units are located in the second chip 11062, and the first and second chips use the The conductors 11012' of the WOW junction 11061 are connected to each other, which WOW junction may include one or more WOW interlayers.

93J is an example of a method 11100 for memory intensive processing. Memory intensive means that processing requires or is associated with high bandwidth memory consumption.

Method 11100 may begin at steps 11110, 11120, and 11130.

Step 11110 includes performing processing operations by a plurality of processor hybrid devices including a base die, a first memory resource of at least one other die, and a second memory resource of at least one other die; wherein the base die and At least one other die is connected to each other by wafer-on-wafer bonding.

Step 11120 includes retrieving, by the plurality of processors, the retrieved information stored in the first memory resource.

Step 11130 can include sending additional information from the second memory resource to the first memory resource, wherein the total bandwidth of the first path between the base die and the at least one other die exceeds the at least one other die and the at least one other die The total bandwidth of the second path between the particles, and wherein the storage capacity of the first memory resource is a part of the storage capacity of the second memory resource.

The method 11100 may also include the step 11140 of performing additional processing by another die including the plurality of processor subunits and the first memory resource.

Each processor subunit may be coupled to a unique portion of the first memory resource allocated to the processor subunit.

The only part of the first memory resource is at least one memory bank.

Steps 11110, 11120, 11130, and 11140 may be performed simultaneously, in a partially overlapping manner, and the like.

The second memory resource may be a high bandwidth memory (HBM) memory resource or may be different from the HBM memory resource.

At least one other die is a stack of high bandwidth memory (HBM) memory chips.

Communication chip

The database includes many entries that include multiple fields. Database processing typically involves executing one or more queries including one or more filter parameters (eg, identifying one or more related fields and one or more related field values) and also including one or more Operational parameters, the one or more operational parameters may determine the type of operation to be performed, variables or constants to be used in applying the operation, and the like. Data processing may include database analysis or other database processing procedures.

For example, a database query may request a statistical operation (operation parameter) on all records of the database where a field has a value within a predefined range (filter parameter). For yet another example, a database query may request the deletion of records (operating parameters) that have a certain field that is less than a threshold (filtering parameters).

Large databases are usually stored in storage devices. In response to a query, the database is sent to a memory unit, typically one database section after another.

The entry of the database section is sent from the memory cell to a processor that does not belong to the same integrated circuit as the memory cell. These entries are then processed by the processor.

For each database segment of the database stored in the memory unit, processing includes the following steps: (i) selecting the records of the database segment; (ii) sending the records from the memory unit to the processor; (iii) by the processor Filtering records to determine whether the records are related; and (iv) performing one or more additional operations (summation, applying any other mathematical operations and/or statistical operations) on the related records.

The screening process ends after all records are sent to the processor and the processor determines which records are relevant.

In the case that the relevant entries of the database section are not stored in the processor, then these relevant records need to be sent to the processor for further processing after the screening phase (applying post-processing operations).

When multiple processing operations follow a single filter, then the result of each operation can be sent to the memory unit and then again to the processor.

This handler is bandwidth consuming and time consuming.

There is a growing need to provide efficient ways of performing database processing.

An apparatus may be provided that may include a database acceleration integrated circuit.

An apparatus may be provided that may include one or more groups of database acceleration integrated circuits that may be configured between database acceleration integrated circuits in the one or more groups of database acceleration integrated circuits Exchange information and/or accelerate results (accelerating the final result of processing by the integrated circuit by means of a database).

A group of database acceleration ICs can be connected to the same printed circuit board.

A group of database acceleration integrated circuits may belong to modular units of a computerized system.

Different groups of database acceleration integrated circuits can be connected to different printed circuit boards.

Different groups of database acceleration integrated circuits may belong to different modular units of the computerized system.

The apparatus may be configured to accelerate the execution of distributed processing by integrated circuits with one or more groups of databases.

The apparatus may be configured to use at least one switch for exchanging at least one of (a) information and (b) database acceleration results between database acceleration integrated circuits of different groups of one or more groups .

The apparatus may be configured to accelerate the execution of distributed processing by some of the integrated circuits by means of a database of some of the one or more groups.

The apparatus may be configured to execute distributed processing of first and second data structures, wherein the combined size of the first and second data structures exceeds the storage capacity of the plurality of memory processing integrated circuits.

The apparatus may be configured to execute a distributed processing program by performing a plurality of iterations of: (a) performing a new assignment of different pairs of the first data structure portion and the second data structure portion to different database acceleration integrated circuits; and (b) deal with different pairs.

94A and 9B illustrate examples of storage system 11560, computer system 11150, and one or more devices 11520 for database acceleration. One or more devices 11520 for database acceleration may monitor communications between storage system 11560 and computer system 11150 (either by listening or by being positioned between computer system 11150 and storage system 11560) in various ways.

Storage system 11560 may include many (eg, more than 20, 50, 100, 100, and the like) storage units (such as disks or raids of disks), and may, for example, store more than 100 terabytes information. Computing system 11510 can be a large computer system and can include tens, hundreds, and even thousands of processing units.

The computing system 11510 may include a plurality of computing nodes 11512 controlled by the manager 11511 .

A computing node may control or otherwise interact with one or more devices 11520 for database acceleration.

One or more means 11520 for database acceleration may include one or more database acceleration integrated circuits (see, eg, database acceleration integrated circuits 11530 of FIGS. 94A and 94B ) and memory resources 11550. A memory resource may belong to one or more chips dedicated to memory but may belong to a memory/processing unit.

94C and 94D illustrate an example of a computer system 11150 and one or more devices 11520 for database acceleration.

One or more database acceleration integrated circuits of one or more devices 11520 for database acceleration may be controlled by a management unit 11513, which may be located within the computer system (see Figure 94C) or at one or more of the means for database acceleration within device 11520 (FIG. 94D).

94E illustrates an apparatus 11520 for database acceleration that includes a database acceleration integrated circuit 11530 and a plurality of memory processing integrated circuits 1151. Each memory processing integrated circuit may include a controller, multiple processor sub-units, and multiple memory units.

Database acceleration integrated circuit 11530 is illustrated as including network communication interface 11531 , first processing unit 11532 , memory controller 11533 , database acceleration unit 11535 , interconnect 11536 , and management unit 11513 .

The network communication interface (11531) may be configured to receive (eg, via the first port 11531(1) of the network communication interface) bulk information from the bulk storage unit. Each storage unit can output information at rates in excess of tens and even hundreds of megabytes per second, and data transfer speeds are expected to increase over time (eg, doubling every 2-3 years). The number (large number) of stored data units may exceed 10, 50, 100, 200 and even more. Large amounts of information can exceed tens, hundreds of gigabytes per second, and can even be in the range of terabytes per second and gigabytes per second.

The first processing unit 11532 may be configured to perform first processing (preprocessing) on the bulk information to provide first processed information.

The memory controller 11533 may be configured to send the first processed information to the plurality of memory processing integrated circuits via the high throughput interface 11534.

The plurality of memory processing integrated circuits 11551 may be configured to perform second processing (processing) on at least a portion of the first processed information by the plurality of memory processing integrated circuits to provide second processed information.

The memory controller 11533 may be configured to retrieve the retrieved information from the plurality of memory processing integrated circuits. The retrieved information may include at least one of: (a) at least a portion of the first processed information; and (b) at least a portion of the second processed information.

The database acceleration unit 11535 may be configured to perform database processing operations on the retrieved information to provide database accelerated results.

The database acceleration integrated circuit may be configured to output database acceleration results, eg, via one or more second ports 11531(2) of the network communication interface.

94E also illustrates a management unit 11513 that is configured to manage at least one of: retrieval of the retrieved information, first processing (preprocessing), second processing (processing), and third processing (database processing). The management unit 11513 may be located outside the database acceleration integrated circuit.

The management unit may be configured to perform the management based on the execution plan. The execution plan may be generated by the management unit, or may be generated by an entity external to the database acceleration integrated circuit. The execution plan may include at least one of: (a) instructions to be executed by the various components of the database acceleration integrated circuit, (b) data and/or coefficients required to implement the execution plan, (c) instructions and / or memory allocation for data.

The management unit may be configured to perform management by allocating at least some of: (a) network communication network interface resources, (b) decompression unit resources, (c) memory controller resources, (d) multiple memory processing integrated circuit resources, and (e) database acceleration unit resources.

As illustrated in Figures 94E and 94G, the network communication network interface may include different types of network communication ports.

Different types of network communication ports may include storage interface protocol ports (eg, SATA ports, ATA ports, ISCSI ports, network file systems, Fibre Channel ports) and common network storage interface protocol ports (eg, Ethernet ATA, Ethernet Fibre Channel , NVME, Roce, and others).

Different types of network communication ports may include storage interface protocol ports and PCIe ports.

Figure 94F includes dashed lines that illustrate the flow of bulk information, first processed information, retrieved information, and database acceleration results. 94F illustrates database acceleration integrated circuit 11530 as coupled to a plurality of memory resources 11550. A number of memory resources 11550 may not belong to a memory processing integrated circuit.

The apparatus 11520 for database acceleration can be configured to perform multiple tasks simultaneously by the database acceleration integrated circuit 11530, because the network communication interface 11531 can receive multiple information streams (simultaneously), and the first processing unit 11532 can simultaneously The plurality of information units perform the first processing, the memory controller 11533 can send the plurality of first processed information units to the plurality of memory processing integrated circuits 11551 at the same time, and the database acceleration unit 11535 can simultaneously process the plurality of retrieved information units.

The means 11520 for database acceleration may be configured to perform at least one of fetching, first processing, sending, and third processing by a computing node of a large computing system based on an execution plan sent to the database acceleration integrated circuit.

The means 11520 for database acceleration can be configured to manage at least one of fetching, first processing, sending, and third processing in a manner that substantially optimizes utilization of the database acceleration integrated circuit. This optimization takes into account latency, throughput, and any other timing or storage or processing considerations, and attempts to keep all components along the flow path busy and bottleneck-free.

The database acceleration integrated circuit may be configured to output database acceleration results, eg, via one or more second ports 11531(2) of the network communication interface.

The means 11520 for database acceleration can be configured to substantially optimize the bandwidth of traffic exchanged over the network communication network interface.

The means 11520 for database acceleration can be configured to substantially prevent bottlenecks from forming in at least one of fetching, first processing, sending, and third processing in a manner that substantially optimizes utilization of the database acceleration integrated circuit.

The means 11520 for database acceleration may be configured to allocate resources of the database acceleration integrated circuit according to temporal I/O bandwidth.

94G illustrates an apparatus 11520 for database acceleration that includes a database acceleration integrated circuit 11530 and a plurality of memory processing integrated circuits 1151. 94G also illustrates the various units coupled to database acceleration integrated circuit 11530: remote RAM 11546, Ethernet memory DIMM 11547, storage system 11560, local storage unit 11561, and non-volatile memory (NVM) 11563 (the non-volatile memory 11563). The memory may be an express NVM unit (NVME).

Database acceleration integrated circuit 11530 is illustrated as including Ethernet port 11531(1), RDMA unit 11545, serial expansion port 11531(15), SATA controller 11540, PCIe port 11531(9), first processing unit 11532, memory control 11533, database acceleration unit 11535, interconnect 11536, management unit 11513, cryptographic engine 11537 for performing cryptographic operations, and second order static random access memory (L2 SRAM) 11538.

The database acceleration unit is illustrated as including a DMA engine 11549, a third-level (L3) memory 11548, and a database acceleration sub-unit 11547. The database acceleration subunit 11547 may be a configurable unit.

Ethernet port 11531(1), RDMA unit 11545, serial expansion port 11531(15), SATA controller 11540, PCIe port 11531(9) may be considered part of network communication interface 11531.

Remote RAM 11546, Ethernet DIMM 11547, storage system 11560 are coupled to Ethernet port 11531(1), which in turn is coupled to RDMA unit 11545.

The local storage unit 11561 is coupled to the SATA controller 11540 .

PCIe port 11531(9) is coupled to NVM 11563. PCIe ports can also be used to exchange commands, eg for management purposes.

94H is an example of the database acceleration unit 11535.

Database acceleration unit 11535 can be configured to concurrently execute database processing instructions by database processing sub-unit 11573 , which can include a group of database accelerator sub-units that share a shared memory unit 11575 .

Different combinations of database acceleration subunits 11535 may be dynamically linked to each other (via configurable links or interconnects 11576) to provide the execution pipeline required to perform database processing operations that may include multiple instructions.

Each database processing subunit may be configured to execute a particular type of database processing instruction (eg, filter, merge, accumulate, and the like).

FIG. 94H also illustrates a separate database processing unit 11572 coupled to cache 11571. A database processing unit 11572 and a cache 11571 may also be provided in place of or in addition to the reconfigurable array 11574 of the DB accelerator.

The apparatus may facilitate scale-in and/or scale-out, thus enabling multiple database acceleration integrated circuits 11530 (and their associated memory resources 11550 or their associated multiple memory processing integrated circuits 11551 ) to, for example, by participating in The distributed processing of database operations complements each other.

94I illustrates a modular unit, such as a blade 11580, that includes two database acceleration integrated circuits 11530 (and their associated memory resources 11550). The blade may include one, two, or more than two memory processing integrated circuits 11551 and their associated memory resources 11550.

The blade may also include one or more non-volatile memory units, an Ethernet switch, a PCIe switch, and an Ethernet switch.

Multiple blades can communicate with each other using any communication method, communication protocol, and connectivity.

94I illustrates four database acceleration integrated circuits 11530 (and their associated memory resources 11550) fully connected to each other, each database acceleration integrated circuit 11530 being connected to all three other database acceleration integrated circuits 11530. Connectivity can be achieved using any communication protocol, such as by using the Ethernet RDMA protocol.

Figure 94I also illustrates database acceleration integrated circuit 11530 connected to its associated memory resource 11550 and unit 11531 including RAM memory and Ethernet ports.

94J, 94K, 94L, and 94M illustrate four groups 11580 of database acceleration integrated circuits, each group including four database acceleration integrated circuits 11530 (fully connected to each other) and their associated memory resources 11550. The different groups are connected to each other via switch 11590.

The number of groups can be two, three or more than four. The number of database acceleration integrated circuits per group may be two, three, or more than four. The number of groups may be the same as (or may be different from) the number of database acceleration integrated circuits per group.

Figure 94K illustrates two tables, A and B, which are too large (eg, 1 trillion bytes) to be efficiently joined at one time.

The table is actually segmented into shards and the join operation is applied to the pair comprising the shard of Table A and the shard of Table B.

The group of database acceleration integrated circuits can handle sharding in various ways.

For example, a device may be configured to execute distributed processing by:

g. Allocate different first data structure portions (slices of Table A, eg, first to sixteenth slices A0 to A15) to one or more groups of different database acceleration integrated circuits.

h. Perform multiple iterations of: (i) Newly assign different second data structure parts (shards of Table B, eg first through sixteenth slices B0 to B15) to one or more groups different database acceleration integrated circuits of the set; and (ii) processing the first and second data structure portions by the database acceleration integrated circuit.

The apparatus may be configured to perform the new assignment of the next iteration in a manner that overlaps at least part of the time with the processing of the current iteration.

The apparatus may be configured to perform the new allocation by exchanging portions of the second data structure between different database acceleration integrated circuits.

The exchange may be performed in a manner that overlaps at least part of the time with the handler.

The device may be configured to perform the new allocation by exchanging the second data structure portion between the different groups of database acceleration integrated circuits; and once the exchange has been completed, between the different groups of the database acceleration integrated circuits. exchange the second data structure portion.

In FIG. 94K, four cycles of some of the joining operations, eg, with reference to the upper left database acceleration integrated circuit 11530 of the upper left group, are shown, the four cycles include computing Join(A0, B0), Join(A0, B3), Join(A0, B2) and Join(A0, B1). During these four cycles, A0 remains at the same database acceleration integrated circuit 11530, while the slices of matrix B (B0, Bl, B2, and B3) are rotated among members of the same group of database acceleration integrated circuits 11530.

In Figure 94L, the slices of the second matrix are rotated between different groups, (a) slices B0, B1, B2 and B3 (previously processed by the upper left group) are sent from the upper left group to the lower left group, (b) send shards B4, B5, B6 and B7 (previously processed by the lower left group) from the lower left group to the upper right group, (c) send shards B8, B9, B10 and B11 (previously processed by the top right group) is sent from the top right group to the bottom right group, and (d) slices B12, B13, B14 and B15 (previously processed by the bottom right group) are sent from the bottom right group to the upper left group.

94N is an example of a system that includes multiple blades 11580, SATA controller 11540, local storage unit 11561, NVME 11563, PCIe switch 11601, Ethernet storage DIMM 11547, and Ethernet port 11531(4).

Blade 11580 may be coupled to each of PCIE switch 11601 , Ethernet port 11531 , and SATA controller 11540 .

Figure 94O illustrates two systems 11621 and 11622.

System 11621 may include one or more devices 11520 for database acceleration, switching system 11611, storage system 11612, and computing system 11613. Switching system 11611 provides connectivity between one or more devices 11520, storage system 11612, and computing system 11613 for database acceleration.

System 11622 may include a storage system and one or more devices 11615 for database acceleration, switching system 11611 and computing system 11613. The switching system 11611 provides connectivity between the storage system and one or more devices 11615 and computing system 11613 for database acceleration.

95A illustrates a method 11200 for database acceleration.

The method 11200 may begin with a step 11210 of retrieving mass information from mass storage units via a database acceleration integrated circuit's network communication network interface.

Connecting to a large number of storage units (eg, using multiple different buses) enables the network communication network interface to receive large amounts of information, even when a single storage unit has limited throughput.

Step 11210 may be followed by a first processing of the bulk information to provide first processed information. The first processing may include buffering, extracting information from the payload, removing headers, decompressing, compressing, decrypting, filtering database queries, or performing any other processing operation. The first process may also be limited to buffering.

Step 11210 may be followed by step 11220 of accelerating the memory controller of the integrated circuit by a database and sending the first processed information to a plurality of memory processing integrated circuits via a high-throughput interface, wherein each memory processing integrated circuit may include a controller , multiple processor sub-units, and multiple memory units. The memory processing integrated circuit may be a memory/processing unit or a distributed processor or memory chip, as described in any other section of this patent application.

Step 11220 may be followed by step 11230 of performing a second processing of at least a portion of the first processed information by a plurality of memory processing integrated circuits to provide second processed information.

Step 11230 may include accelerating the integrated circuit to perform multiple tasks concurrently with the database.

Step 11230 may include concurrent execution of database processing instructions by database processing subunits, wherein the database acceleration unit may comprise a group of database accelerator subunits that share a shared memory unit.

Step 11230 may be followed by step 11240 of retrieving the retrieved information from the plurality of memory processing integrated circuits by the memory controller of the database acceleration integrated circuit, wherein the retrieved information may include at least one of: ( a) at least a portion of the first processed information; and (b) at least a portion of the second processed information.

Step 11240 may be followed by step 11250 of performing database processing operations on the retrieved information by the database acceleration unit of the database acceleration integrated circuit to provide database acceleration results.

Step 11250 may include allocating resources of the database acceleration integrated circuit based on time I/O bandwidth.

Step 11250 may be followed by step 11260 of outputting database acceleration results.

Step 11260 may include dynamically linking database processing subunits to provide the execution pipeline required to perform database processing operations that may include multiple instructions.

Step 11260 may include outputting and retrieving database acceleration results to local storage.

It should be noted that steps 11210, 11220, 11230, 11240, 11250, and 11260 or any other steps of method 11100 may be performed in a pipelined fashion. These steps may be performed simultaneously or in an order different from that mentioned above.

For example, step 1120 may be followed by step 11250 such that the first processed information is further processed by the database acceleration unit.

For yet another example, the first processed information may be sent to multiple memory processing integrated circuits, and then sent (not processed by multiple memory processing integrated circuits) to a database acceleration unit.

For yet another example, the first processed information and/or the second processed information may be output from the database acceleration integrated circuit without database processing by the database acceleration unit.

The method may include, by a computing node of the large computing system, performing at least one of the following operations: fetching, first processing, sending, and third processing based on the execution plan sent to the database acceleration integrated circuit.

The method can include managing at least one of fetching, first processing, sending, and third processing in a manner that substantially optimizes utilization of the database acceleration integrated circuit.

The method may include substantially optimizing the bandwidth of traffic exchanged over the network communication network interface.

The method may include substantially preventing bottlenecks from forming in at least one of fetching, first processing, sending, and third processing in a manner that substantially optimizes utilization of the database acceleration integrated circuit.

Method 11200 may also include at least one of the following steps:

Step 11270 may include managing at least one of retrieval, first processing, sending, and third processing by a management unit of the database acceleration integrated circuit.

The management may be performed based on an execution plan generated by a management unit of the database acceleration integrated circuit.

The management may be performed based on an execution plan received by a management unit of the database acceleration integrated circuit rather than generated by the management unit.

The managing may include allocating at least some of: (a) network communication network interface resources, (b) decompression unit resources, (c) memory controller resources, (d) multiple memory processing integrated circuit resources, and (e) database acceleration unit resources.

Step 11271 may include controlling at least one of at least one of fetching, first processing, sending, and third processing by a computing node of a large computing system.

Step 11272 may include managing at least one of fetching, first processing, sending, and third processing by a management unit external to the database acceleration integrated circuit.

95B illustrates a method 11300 for operating a group of database acceleration integrated circuits.

Method 11300 may begin with step 11310 of performing database acceleration operations by a database acceleration integrated circuit. Step 11310 may include performing one or more steps of method 11200.

The method 11300 may also include the step 11320 of exchanging at least one of (a) information and (b) database acceleration results between the database acceleration integrated circuits of the one or more groups of database acceleration integrated circuits.

The combination of steps 11310 and 11320 may amount to accelerating the integrated circuit's execution of distributed processing by means of one or more groups of databases.

The exchange may be performed using one or more groups of databases to accelerate the network communication network interface of the integrated circuit.

Swapping can be performed through multiple groups, which can be connected to each other by star connections.

Step 11320 can include using at least one switch for exchanging at least one of: (a) information; and (b) between database acceleration integrated circuits of different ones of the one or more groups Database speeds up results.

Step 11310 may include step 11311 of accelerating some of the integrated circuits to perform distributed processing by means of a database of some of the one or more groups.

Step 11311 may include performing distributed processing of the first and second data structures, wherein the combined size of the first and second data structures exceeds the storage capacity of the plurality of memory processing integrated circuits.

Execution of the distributed processing may include performing multiple iterations of: (a) performing a new assignment of different pairs of the first data structure portion and the second data structure portion to different database acceleration integrated circuits; and (b) processing the different right.

Execution of distributed processing may include performing database join operations.

Step 11310 may comprise the step 11312 of (a) assigning different first data structure portions to different database acceleration integrated circuits of one or more groups; and (b) performing multiple iterations of: assigning different first data structure portions Step 11314 of newly assigning the data structure parts to different database acceleration integrated circuits of one or more groups; and step 11316 of processing the first and second data structure parts by the database acceleration integrated circuit.

Step 11314 may be performed in a manner that overlaps at least part of the time with processing of the current iteration.

Step 11314 may include exchanging portions of the second data structure between different database acceleration integrated circuits.

Step 11320 may be performed in a manner that overlaps at least partially with step 11310 in time.

Step 11314 may include exchanging the second data structure portion between the different database acceleration integrated circuits of the group; and once the exchange has been completed, exchanging the second data structure portion between the different groups of the database acceleration integrated circuit.

Figure 95C illustrates a method 11350 for database acceleration.

The method 11350 may include the step 11352 of retrieving mass information from mass storage units via a network communication network interface of the database acceleration integrated circuit.

Step 11352 may be followed by step 11354 of first processing the bulk information to provide first processed information.

Step 11352 may be followed by step 11354 of accelerating the memory controller of the integrated circuit by the database and sending the first processed information to the plurality of memory resources via the high throughput interface.

Step 11354 may be followed by step 11356 of retrieving the retrieved information from the plurality of memory resources.

Step 11356 may be followed by step 11358 of performing database processing operations on the retrieved information by the database acceleration unit of the database acceleration integrated circuit to provide database acceleration results.

Step 11358 may be followed by step 11359 of outputting database acceleration results.

The method may also include the step 11355 of subjecting the first processed information to a second process to provide second processed information. The second processing is performed by a plurality of processors located in one or more memory processing integrated circuits that further include a plurality of memory resources. Step 11355 follows step 11354 and precedes step 11356.

The total size of the second processed information may be smaller than the total size of the first processed information.

The total size of the first processed information may be less than the total size of the bulk information.

The first process may include filtering database entries. Thus, database entries that are not relevant to a query are filtered out before any other processing is performed and/or even before they are stored in multiple memory resources, thereby saving bandwidth, storage resources, and other processing resources.

The second process may include filtering database entries. Filtering can be applied when the filter conditions can be complex (including multiple conditions) and may require receiving multiple database entry fields before the filter can proceed. For example, when searching for people who are (a) over a certain age and like bananas and (b) over another age and like apples.

database

The following examples can refer to the database. The database may be a data center, may be part of a data center, or may not be part of a data center.

The database may be coupled to multiple users via one or more networks. The database may be a cloud database.

A database may be provided that includes one or more management units and a plurality of database accelerator boards including one or more memory/processing units.

96B illustrates a database 12020 that includes a management unit 12021 and a plurality of DB accelerator boards 12022, each of which includes a communication/management processor (processor 12024) and a plurality of memory/processing units 12026.

The processor 12024 may support various communication protocols such as, but not limited to, PCIe, ROCE-like protocols, and the like.

Database commands may be executed by the memory/processing unit 12026, and the processor may route traffic between the memory/processing units 12026, between the different DB accelerator boards 12022, and with the management unit 12021.

The use of multiple memory/processing units 12026 can significantly speed up the execution of database commands and avoid communication bottlenecks, especially when large internal memory banks are included.

96C illustrates a DB accelerator board 12022 that includes a processor 12024 and a plurality of memory/processing units 12026. Processor 12024 includes a number of communication specific components, such as DDR controller 12033 for communicating with memory/processing unit 12026, RDMA engine 12031, DB query database engine 12034, and the like. A DDR controller is an example of a communication controller, and an RDMA engine is an example of any communication engine.

A method for operating the system (or any portion of an operating system) of any of Figures 96B, 96C, and 96D may be provided.

It should be noted that database acceleration integrated circuit 11530 may be associated with multiple memory resources that are not included in multiple memory processing integrated circuits or otherwise not associated with processing units. In this case, the processing is mainly, and even only, performed by the database acceleration integrated circuit.

Figure 94P illustrates a method 11700 for database acceleration.

The method 11700 can include the step 11710 of retrieving information from a storage unit via a network communication interface of the database acceleration integrated circuit.

Step 11710 may be followed by step 11720 of first processing the amount of information to provide first processed information.

Step 11720 may be followed by step 11730 of accelerating the memory controller of the integrated circuit by the database and sending the first processed information to the plurality of memory resources via the throughput interface.

Step 11730 may be followed by step 11740 of retrieving information from a plurality of memory resources.

Step 11740 may be followed by step 11750 of performing database processing operations on the retrieved information by the database acceleration unit of the database acceleration integrated circuit to provide database acceleration results.

Step 11750 may be followed by step 11760 of outputting database acceleration results.

The first process and/or the second process may include filtering database entries to determine which database entries should be further processed.

The second process involves filtering database entries.

Hybrid system

The memory/processing unit may be efficient in performing computations that may be memory intensive and/or bottleneck related to fetch operations. Processing-oriented (and less memory-oriented) processor units (such as, but not limited to, graphics processing units, central processing units) may be more efficient when bottlenecks are associated with computational operations.

A hybrid system may include both one or more processor units and one or more memory/processing units that are fully or partially connectable to each other.

Memory/processing units (MPUs) can be fabricated by a first manufacturing process that is better suited for memory cells than logic cells. For example, memory cells fabricated by the first fabrication process may exhibit smaller and even much smaller critical dimensions (eg, more than 2 times, 3 times, 4 times smaller) than logic circuits fabricated by the first fabrication process , 5 times, 6 times, 7 times, 8 times, 9 times, 10 times, and the like). For example, the first fabrication process may be an analog fabrication process, the first fabrication process may be a DRAM fabrication process, and the like.

The processor may be fabricated by a second fabrication process that is better suited for logic. For example, the critical dimension of the logic circuit fabricated by the second fabrication process may be smaller and even much smaller than the critical dimension of the logic circuit fabricated by the first fabrication process. As yet another example, the critical dimensions of logic circuits fabricated by the second fabrication process may be smaller and even much smaller than the critical dimensions of memory cells fabricated by the first fabrication process. For example, the second fabrication process may be an analog fabrication process, the second fabrication process may be a CMOS fabrication process, and the like.

Tasks can be distributed among different units in a static or dynamic manner by taking into account the benefits of each unit and any penalties associated with transferring data between units.

For example, memory-intensive processing programs may be assigned to memory/processing units, while processing-intensive memory-light processing may be assigned to processing units.

A processor may request or instruct one or more memory/processing units to perform various processing tasks. The execution of various processing tasks may reduce the burden on the processor, reduce latency, and in some cases reduce the overall information bandwidth between one or more memory/processing units and the processor, and the like.

The processor may provide instructions and/or requests at different granularities, eg, the processor may issue instructions for certain processing resources or may issue higher order instructions for memory/processing units without specifying any processing resources.

96D is an example of a hybrid system 12040 that includes one or more memory/processing units (MPUs) 12043 and processors 12042. The processor 12042 may send the request or instruction to one or more MPUs 12043, which in turn complete (or selectively complete) the request and/or instruction and send the result to the processor 12042, as described above .

The processor 12042 can further process the results to provide one or more outputs.

Each MPU includes memory resources, processing resources (such as compact microcontroller 12044 ), and cache 12049 . Microcontrollers may have limited computing power (eg, may primarily include multiply-accumulate units).

The microcontroller 12044 may apply processing programs for in-memory acceleration purposes, or may be the CPU or the entire full-DB processing engine or a subset thereof.

MPU 12043 may include a microprocessor and packet processing unit that may be connected in mesh/ring/or other topologies for fast inter-group communication.

There may be more than one DDR controller for fast inter-DIMM communication.

The goals of in-memory packet processors are to reduce BW, data movement, power consumption, and increase performance. Using an in-memory packet processor will result in a significant increase in performance/TCO compared to standard solutions.

It should be noted that the snap-in is optional.

Each MPU can operate as an artificial intelligence (AI) memory/processing unit because it can perform AI computations and only pass the results back to the processor, thereby reducing traffic, especially when the MPU receives and stores for multiple There is no need to receive coefficients from an external chip every time a portion of the neural network is used in a computation of the neural network coefficients to process new data.

The MPU can determine when the coefficients are zero, and inform the processor that multiplications involving zero-valued coefficients need not be performed.

It should be noted that the first process and the second process may include filtering database entries.

The MPU may be any of the memory processing units described in this specification, in any of PCT patent application WO2019025862 and PCT patent application No. PCT/IB2019/001005.

AI computing systems (and systems executable by the systems) may be provided in which network adapters have AI processing capabilities and are configured to perform some AI processing tasks in order to reduce the amount of traffic to be sent over a network coupled to multiple AI acceleration servers.

For example, in some inference systems, the input is a network (eg, multiple streams of IP cameras connected to an AI server). In this case, utilizing RDMA+AI on the processing and network connection unit can reduce the load on the CPU and PCIe bus and provide processing to the processing and network connection unit, rather than by the GPU that is not included in the processing and network connection unit. deal with.

For example, instead of computing and sending the initial results to the target AI acceleration server (applying one or more AI processing operations), the processing and networking unit may perform preprocessing that reduces the amount of values sent to the target AI acceleration server . A target AI computing server is an AI computing server that is assigned to perform computations on values provided by other AI acceleration servers. This reduces the bandwidth of traffic exchanged between AI acceleration servers and also reduces the load on the target AI acceleration server.

Target AI acceleration servers can be allocated dynamically or statically by using load balancing or other allocation algorithms. There may be more than a single target AI acceleration server.

For example, if the target AI acceleration server adds multiple losses, the processing and networking unit may add the losses incurred by its AI acceleration server and send the sum of the losses to the target AI acceleration server, thereby reducing bandwidth. The same benefits can be obtained when performing other preprocessing operations such as derivative calculations and aggregation and the like.

97B illustrates a system 12060 including subsystems, each including a switch 12061 for connecting AI processing and network connection units 12063 with server motherboards 12064 to each other. The server motherboard includes one or more AI processing and network connection units 12063 with network capability and AI processing capability. AI processing and networking unit 12063 may include one or more NICs and ALUs or other computing circuits for performing preprocessing.

The AI processing and network connection unit 12063 may be a chip, or may include more than a single chip. It may be beneficial to have the AI processing and networking unit 12063 being a single chip.

AI processing and networking unit 12063 may include (only or primarily) processing resources. The AI processing and networking unit 12063 may include in-memory arithmetic circuits, or may not include in-memory arithmetic circuits, or may not include mass-memory arithmetic circuits.

The AI processing and networking unit 12063 may be an integrated circuit, may include more than a single integrated circuit, may be part of an integrated circuit, and the like.

The AI processing and network connection unit 12063 can transport (see, eg, FIG. 97C ) traffic between the AI acceleration server including the AI processing and network connection unit 12063 and other AI acceleration servers (eg, by using methods such as DDR channels, network channels and/or or a communication port for a PCIe lane). The AI processing and network connection unit 12063 can also be coupled to external memory such as DDR memory. The processing and networking unit may include memory and/or may include a memory/processing unit.

In Figure 97C, AI processing and networking unit 12063 is illustrated as including local DDR connections, DDR channels, AI accelerators, RAM memory, encryption/decryption engines, PCIe switches, PCIe interfaces, multiple core processing arrays, fast network connections and the like.

A method for operating the system (or any portion of an operating system) of any of Figures 97B and 97C may be provided.

Any combination of any steps of any of the methods mentioned in this application may be provided.

Any combination of units, integrated circuits, memory resources, logic, processing subunits, controllers, components mentioned in this application may be provided.

Any reference to "comprising" and/or "comprising" may apply to "consists of", "consisting essentially of" mutatis mutandis.

The previous description has been presented for purposes of illustration. The preceding description is not exhaustive and is not limited to the precise form or embodiments disclosed. Modifications and adaptations will be apparent to those skilled in the art from consideration of this specification and practice of the disclosed embodiments. Additionally, although aspects of the disclosed embodiments are described as being stored in memory, those skilled in the art will appreciate that these aspects may also be stored on other types of computer-readable media, such as secondary storage devices such as hard disks or CD ROMs , or other forms of RAM or ROM, USB media, DVD, Blu-ray, 4K Ultra HD Blu-ray, or other optical drive media.

Computer programs based on the written description and the disclosed methods are within the skill of an experienced developer. Various programs or program modules may be created using any of the techniques known to those skilled in the art or may be designed in conjunction with existing software. For example, program sections or program modules may be available or by means of .Net Framework, .Net Compact Framework (and related languages such as Visual Basic, C, etc.), Java, C++, Objective-C, HTML, HTML/AJAX combination, XML or Include the HTML of the Java applet to design.

Furthermore, although illustrative embodiments have been described herein, those skilled in the art will appreciate having equivalent components, modifications, omissions, combinations (eg, combinations across aspects of the various embodiments), adaptations and/or based on this disclosure The scope of any and all embodiments of modification. The limitations in the claims are to be construed broadly based on the language used in the claims, and are not limited to the embodiments described in this specification or during the prosecution of this application. The examples should be construed as non-exclusive. Furthermore, the steps of the disclosed methods may be modified in any way, including by reordering steps and/or inserting or deleting steps. Accordingly, the specification and examples are to be regarded as illustrative only, with the true scope and spirit being indicated by the appended claims along with their full scope of equivalents.

Claims (368)

1. An integrated circuit comprising: substrate; a memory array disposed on the substrate, the memory array comprising a plurality of discrete memory banks; a processing array disposed on the substrate, the processing array including a plurality of processor subunits, each of the plurality of processor subunits and one or more of the plurality of discrete memory banks Discrete memory bank associations; and Controller, which is configured as: At least one security measure is implemented with respect to the operation of the integrated circuit. 2. The integrated circuit of claim 1, wherein the controller is configured to take one or more remedial actions if the at least one safety measure is triggered. 3. The integrated circuit of claim 1, wherein the controller is configured to implement at least one security measure in at least one memory location. 4. The integrated circuit of claim 2, wherein the data includes weight data for the neural network model. 5. The integrated circuit of claim 1, wherein the controller is configured to implement at least one security measure, the at least one security measure comprising locking out one of the memory arrays not used for input data or output data operations or access to multiple memory sections. 6. The integrated circuit of claim 1, wherein the controller is configured to implement at least one security measure, the at least one security measure comprising locking only a subset of the memory array. 7. The integrated circuit of claim 6, wherein the subset of the array is designated by a particular memory address. 8. The integrated circuit of claim 6, wherein the subset of the memory array is configurable. 9. The integrated circuit of claim 1, wherein the controller is configured to implement at least one security measure, the at least one security measure comprising controlling traffic to or from the integrated circuit. 10. The integrated circuit of claim 1, wherein the controller is configured to implement at least one security measure, the at least one security measure including uploading changeable data, code, or fixed data. 11. The integrated circuit of claim 1, wherein the uploading of the changeable data, code, or fixed data occurs during a power-on handler. 12. The integrated circuit of claim 1, wherein the controller is configured to implement at least one security measure, the at least one security measure comprising, during a power-on handler, uploading a configuration file, the configuration file identifying the pending A specific memory address of at least a portion of the memory array that is locked upon completion of the power-on handler. 13. The integrated circuit of claim 1, wherein the controller is further configured to require a complex password to unlock access to memory portions of the memory array associated with one or more memory addresses . 14. The integrated circuit of claim 1, wherein the at least one security measure is triggered upon detection of an attempted access to at least one locked memory address. 15. The integrated circuit of claim 1, wherein the controller is configured to implement at least one security measure, the at least one security measure comprising: computing a checksum, hash, CRC (Cyclic Redundancy Check) or check bit computed relative to at least a portion of the memory array; and The calculated checksum, hash, CRC or check digit is compared to a predetermined value. 16. The integrated circuit of claim 15, wherein the controller is configured to determine, as part of the at least one security measure, whether the calculated checksum, hash, CRC or check bit matches the predetermined value. 17. The integrated circuit of claim 1, wherein the at least one security measure includes duplicating program code in at least two different memory portions. 18. The integrated circuit of claim 17, wherein the at least one security measure includes determining whether output results of executing the program code in the at least two different memory portions are different. 19. The integrated circuit of claim 18, wherein the output results comprise intermediate or final output results. 20. The integrated circuit of claim 17, wherein the at least two different memory portions are included within the integrated circuit. 21. The integrated circuit of claim 1, wherein the at least one security measure includes determining whether an operating pattern is different from one or more predetermined operating patterns. 22. The integrated circuit of claim 2, wherein the one or more remedial actions include ceasing to perform operations. 23. A method of protecting an integrated circuit against tampering, the method comprising: At least one security measure is implemented with respect to operation of the integrated circuit using a controller associated with the integrated circuit; wherein the integrated circuit includes: substrate; a memory array disposed on the substrate, the memory array including a plurality of discrete memory banks; and a processing array disposed on the substrate, the processing array including a plurality of processor subunits, each of the plurality of processor subunits and one or more of the plurality of discrete memory banks Discrete memory banks are associated. 24. The method of claim 23, further comprising taking one or more remedial actions if the at least one security measure is triggered. 25. An integrated circuit comprising: substrate; a memory array disposed on the substrate, the memory array comprising a plurality of discrete memory banks; a processing array disposed on the substrate, the processing array including a plurality of processor subunits, each of the plurality of processor subunits and one or more of the plurality of discrete memory banks Discrete memory bank associations; and Controller, which is configured as: At least one security measure is implemented with respect to operation of the integrated circuit; wherein the at least one security measure includes duplicating program code in at least two different memory portions. 26. An integrated circuit comprising: substrate; a memory array disposed on the substrate, the memory array comprising a plurality of discrete memory banks; a processing array disposed on the substrate, the processing array including a plurality of processor subunits, each of the plurality of processor subunits and one or more of the plurality of discrete memory banks Discrete memory bank associations; and A controller configured to implement at least one security measure with respect to operation of the integrated circuit. 27. The integrated circuit of claim 26, wherein the controller is further configured to take one or more remedial actions if the at least one security measure is triggered. 28. A distributed processor memory chip comprising: substrate; a memory array disposed on the substrate, the memory array comprising a plurality of discrete memory banks; a processing array disposed on the substrate, the processing array including a plurality of processor subunits, each of the plurality of processor subunits and one or more of the plurality of discrete memory banks Discrete memory bank associations; and a first communication port configured to establish a communication connection between the distributed processor memory chip and an external entity other than another distributed processor memory chip; and A second communication port configured to establish a communication connection between the distributed processor memory chip and the first additional distributed processor memory chip. 29. The distributed processor memory chip of claim 28, further comprising a third communication port configured to interface between the distributed processor memory chip and a second additional distributed processor A communication connection is established between the memory chips. 30. The distributed processor memory chip of claim 29, further comprising a controller configured to communicate via the first communication port, the second communication port, the third communication port At least one of them controls the communication. 31. The distributed processor memory chip of claim 29, wherein each of the first communication port, the second communication port, and the third communication port is associated with a corresponding bus. 32. The distributed processing memory chip of claim 31, wherein the corresponding bus is a bus common to each of the first communication port, the second communication port, and the third communication port . 33. The distributed processing memory chip of claim 31, wherein The corresponding buses associated with each of the first communication port, the second communication port, and the third communication port are connected to the plurality of discrete memory banks. 34. The distributed processor memory chip of claim 31, wherein at least one bus associated with the first communication port, the second communication port, and the third communication port is unidirectional. 35. The distributed processor memory chip of claim 31, wherein at least one bus associated with the first communication port, the second communication port, and the third communication port is bidirectional. 36. The distributed processor memory chip of claim 30, wherein the controller is configured to schedule data transfers between the distributed processor memory chip and the first additional distributed processor memory chip , causing the receiving processor sub-unit of the first additional distributed processor memory chip to execute its associated program code based on the data transmission and during a time period when the data transmission is received. 37. The distributed processor memory chip of claim 30, wherein the controller is configured to send a clock enable signal to at least one of the plurality of processor subunits of the distributed processor memory chip One, to control one or more operational aspects of the at least one of the plurality of processor sub-units. 38. The distributed processor memory chip of claim 37, wherein the controller is configured to control the clock enable signal sent to the at least one of the plurality of processor subunits to control the timing of one or more communication commands associated with the at least one of the plurality of processor subunits. 39. The distributed processor memory chip of claim 30, wherein the controller is configured to selectively initiate processing by one of the plurality of processor subunits on the distributed processor memory chip One or more executable program codes. 40. The distributed processor memory chip of claim 30, wherein the controller is configured to use a clock enable signal to control communication from one or more of the plurality of processor subunits to the second Timing of data transmission for at least one of the communication port and the third communication port. 41. The distributed processor memory chip of claim 28, wherein a communication speed associated with the first communication port is lower than a communication speed associated with the second communication port. 42. The distributed processor memory chip of claim 30, wherein the controller is configured to determine whether a first processor sub-unit of the plurality of processor sub-units is ready to transfer data to a a second processor subunit in the first additional distributed processor memory chip and uses a clock after determining that the first processor subunit is ready to transfer the data to the second processor subunit An enable signal initiates the transfer of the data from the first processor subunit to the second processor subunit. 43. The distributed processor memory chip of claim 42, wherein the controller is further configured to determine whether the second processor sub-unit is ready to receive the data, and after determining the second processing The clock enable signal is used to initiate the transfer of the data from the first processor subunit to the second processor subunit after the processor subunit is ready to receive the data. 44. The distributed processor memory chip of claim 42, wherein the controller is further configured to determine whether the second processor sub-unit is ready to receive the data and buffer the data included in the transfer the data until after a determination that the second processor sub-unit of the first additional distributed processor memory chip is ready to receive the data. 45. A method of transferring data between a first distributed processor memory chip and a second distributed processor memory chip, the method comprising: using a controller associated with at least one of the first distributed processor memory chip and the second distributed processor memory chip to determine a plurality of disposed on the first distributed processor memory chip whether a first processor subunit of the processor subunits is ready to transfer data to a second processor subunit included in the second distributed processor memory chip; and A clock enable signal controlled by the controller is used to initiate processing of the data from the first processor subunit after determining that the first processor subunit is ready to transfer the data to the second processor subunit transfer from the processor subunit to the second processor subunit. 46. The method of claim 45, further comprising: determining, using the controller, whether the second processor sub-unit is ready to receive the data; and The clock enable signal is used after determining that the second processor subunit is ready to receive the data to initiate the transfer of the data from the first processor subunit to the second processor subunit send. 47. The method of claim 45, further comprising: determining, using the controller, whether the second processor sub-unit is ready to receive the data, and buffering the data included in the transfer until the first additional distributed processor memory chip's After a determination that the second processor sub-unit is ready to receive the data. 48. A memory chip comprising: substrate; a memory array disposed on the substrate, the memory array including a plurality of discrete memory banks; and a first communication port configured to establish a communication connection between the memory chip and an external entity other than another memory chip; and A second communication port configured to establish a communication connection between the memory chip and the first additional memory chip. 49. The memory chip of claim 48, wherein the first communication port is connected to at least one of a main bus internal to the memory chip or at least one processor sub-unit included in the memory chip. 50. The memory chip of claim 48, wherein the second communication port is connected to at least one of a main bus internal to the memory chip or at least one processor sub-unit included in the memory chip. 51. A memory cell comprising: a memory array that includes a plurality of memory banks; at least one controller configured to control at least one aspect of read operations relative to the plurality of memory banks; at least one zero value detection logic unit configured to detect multi-bit zero values associated with data stored in particular addresses of the plurality of memory banks; and wherein the at least one controller is configured to communicate a zero value indicator back to one or more circuits in response to a zero value detection by the at least one zero value detection logic unit. 52. The memory cell of claim 51, wherein the one or more circuits that return the zero-value indicator are external to the memory cell. 53. The memory cell of claim 51, wherein the one or more circuits that return the zero-value indicator are internal to the memory cell. 54. The memory cell of claim 51, wherein the memory cell further comprises at least one read disable element configured to detect logic cells at the at least one zero value A read command associated with the particular address is interrupted upon detection of a zero value associated with the particular address. 55. The memory cell of claim 51, wherein the at least one controller is configured to send the zero value indicator to the one or more circuits rather than sending a zero stored in the particular address value data. 56. The memory cell of claim 51, wherein the size of the zero value indicator is less than the size of zero data. 57. The memory cell of claim 51, wherein the energy consumed by the first handler comprising the following operations is less than the energy consumed by sending zero-valued data to the one or more circuits: (a) detecting measuring the zero value; (b) generating the zero value indicator; and (c) sending the zero value indicator to the one or more circuits. 58. The memory cell of claim 57, wherein the energy consumed by the first handler is less than the energy consumed by sending the zero-valued data to the one or more circuits half. 59. The memory cell of claim 51, wherein the memory cell further comprises at least one sense amplifier configured to perform zero detection at the at least one zero detection unit At least one of the plurality of memory banks is then blocked from starting. 60. The memory cell of claim 59, wherein the at least one sense amplifier comprises a plurality of transistors configured to sense low power signals from the plurality of memory banks, and the At least one sense amplifier amplifies small voltage swings to higher voltage levels so that data stored in the plurality of memory banks can be interpreted by the at least one controller. 61. The memory cell of claim 51, wherein each of the plurality of memory banks is further organized into subgroups, the at least one controller comprises a subgroup controller, and wherein the at least one zero value The detection logic unit includes zero value detection logic associated with the subset. 62. The memory cell of claim 61, wherein the memory cell further comprises at least one read disable element comprising a read disable element associated with each of the subsets sense amplifier. 63. The memory unit of claim 51, further comprising a plurality of processor sub-units spatially distributed within the memory unit, wherein each of the plurality of processor sub-units is associated with the A dedicated at least one of the plurality of memory banks is associated, and wherein each of the plurality of processor sub-units is configured to access and operate on data stored in the corresponding memory bank. 64. The memory unit of claim 63, wherein the one or more circuits comprise one or more of the processor subunits. 65. The memory unit of claim 63, wherein each of the plurality of processor sub-units is connected to two or more of the plurality of processor sub-units by one or more buses Two other processor subunits. 66. The memory cell of claim 51, further comprising a plurality of buses. 67. The memory unit of claim 66, wherein the plurality of buses are configured to transfer data between the plurality of memory banks. 68. The memory cell of claim 67, wherein at least one of the plurality of buses is configured to communicate the zero-value indicator to the one or more circuits. 69. A method for detecting zero values in specific addresses of a plurality of memory banks, comprising: receiving, from circuitry external to the memory unit, a request to read data stored at addresses of a plurality of memory banks; in response to the received request, enabling a zero value detection logic unit to detect, by the controller, a zero value in the received address; and A zero value indicator is communicated to the circuit by the controller in response to the zero value detection by the zero value detection logic unit. 70. The method of claim 69, further comprising configuring, by the controller, a read disable element to interrupt when the zero value detection logic unit detects a zero value associated with a requested address a read command associated with the requested address. 71. The method of claim 69, further comprising configuring, by the controller, a sense amplifier to block at least one of the plurality of memory banks when the zero value detection unit detects a zero value starter. 72. A non-transitory computer-readable medium storing a set of instructions executable by a controller of a memory unit to cause the memory unit to detect a zero value in a particular address of a plurality of memory banks, method Include: receiving, from circuitry external to the memory unit, a request to read data stored at addresses of a plurality of memory banks; in response to the received request, enabling a zero value detection logic unit to detect, by the controller, a zero value in the received address; and A zero value indicator is communicated to the circuit by the controller in response to the zero value detection by the zero value detection logic unit. 73. The non-transitory computer-readable medium of claim 72, wherein the method further comprises configuring, by the controller, a read disable element to detect at the zero value detection logic unit and all A read command associated with the requested address is interrupted when the zero value associated with the requested address is interrupted. 74. The non-transitory computer-readable medium of claim 72, wherein the method further comprises configuring, by the controller, a sense amplifier to prevent all of the zero values when the zero value detection unit detects a zero value activation of at least one of the plurality of memory banks. 75. An integrated circuit comprising: a memory unit comprising a plurality of memory banks, at least one controller configured to control at least one aspect of a read operation relative to the plurality of memory banks, and configured to detect and store in the plurality of memory banks at least one zero value detection logic unit of the multi-bit zero value associated with the data in the specific address; a processing unit configured to send a read request to the memory unit to the memory for reading data from the memory unit; and wherein the at least one controller and the at least one zero-value detection logic are configured to communicate a zero-value indicator back to one or more of the zero-value indicators in response to a zero-value detection by the at least one zero-value detection logic a circuit. 76. A memory cell comprising: a memory array that includes a plurality of memory banks; at least one controller configured to control at least one aspect of read operations relative to the plurality of memory banks; at least one detection logic unit configured to detect predetermined multi-bit values associated with data stored in particular addresses of the plurality of memory banks; and wherein the at least one controller is configured to communicate a value indicator back to one or more circuits in response to detection of the predetermined multi-bit value by the at least one detection logic. 77. The memory cell of claim 76, wherein the predetermined multi-bit value is selectable by a user. 78. A memory cell comprising: a memory array that includes a plurality of memory banks; at least one controller configured to control at least one aspect of write operations with respect to the plurality of memory banks; at least one detection logic unit configured to detect a predetermined multi-bit value associated with data to be written to a particular address of the plurality of memory banks; and wherein the at least one controller is configured to provide a value indicator to one or more circuits in response to detection of the predetermined multi-bit value by the at least one detection logic. 79. A distributed processor memory chip comprising: substrate; a memory array including a plurality of memory banks disposed on the substrate; a plurality of processor subunits disposed on the substrate; at least one controller configured to control at least one aspect of read operations relative to the plurality of memory banks; at least one detection logic unit configured to detect predetermined multi-bit values associated with data stored in particular addresses of the plurality of memory banks; and wherein the at least one controller is configured to return a value indicator to one of the plurality of processor sub-units in response to a detection of the predetermined multi-bit value by the at least one detection logic or multiple. 80. A memory cell comprising: one or more memory banks; group controller; and address generator; where the address generator is configured as: providing a current address of a current row to be accessed in an associated one of the one or more memory banks to the bank controller; determining the predicted address of the next row to be accessed in the associated memory bank; and The predicted address is provided to the bank controller prior to completion of operations relative to the current row associated with the current address. 81. The memory cell of claim 80, wherein the operation relative to the current row associated with the current address is a read operation or a write operation. 82. The memory cell of claim 80, wherein the current row and the next row are in the same memory bank. 83. The memory cell of claim 82, wherein the same memory bank allows access to the next row while the current row is being accessed. 84. The memory cell of claim 80, wherein the current row and the next row are in different memory banks. 85. The memory unit of claim 80, a distributed processor, wherein the distributed processor comprises a plurality of processor subunits of a processing array, the plurality of processor subunits of the processing array being spatially distributed in multiple discrete memory banks of the memory array. 86. The memory cell of claim 80, wherein the bank controller is configured to access the current row and start the next row prior to completion of the operation relative to the current row. 87. The memory cell of claim 80, wherein each of the one or more memory banks includes at least a first subset and a second subset, and wherein the one or more memory banks are the same as in the one or more memory banks The group controllers associated with each of the include a first subgroup controller associated with the first subgroup and a second group controller associated with the second subgroup. 88. The memory cell of claim 87, wherein a first subset controller is configured to enable access to data included in a current row of the first subset, and a second subset controller enables all . the next row in the second subgroup. 89. The memory cell of claim 88, wherein an activated next row of the second subset is separated by at least two rows from the current row of data being accessed in the first subset. 90. The memory cell of claim 87, wherein the second subset controller is configured such that data included in a current row of the second subset is accessed while the first subset controls The processor starts the next row in the first subgroup. 91. The memory cell of claim 90, wherein an activated next row of the first subset is separated by at least two rows from the current row of data being accessed in the second subset. 92. The memory cell of claim 80, wherein the predicted address is determined using a trained neural network. 93. The memory cell of claim 80, wherein the predicted address is determined based on the determined bank access pattern. 94. The memory cell of claim 80, wherein the address generator comprises a first address generator configured to generate the current address and a second address generator configured to generate the predicted address. 95. The memory cell of claim 94, wherein the second address generator is configured to calculate the predicted address within a predetermined period of time after the current address generator has generated the current address. 96. The memory cell of claim 95, wherein the predetermined period of time is adjustable. 97. The memory cell of claim 96, wherein the predetermined period of time is adjusted based on a value of at least one operating parameter associated with the memory cell. 98. The memory cell of claim 97, wherein the at least one operating parameter comprises a temperature of the memory cell. 99. The memory cell of claim 80, wherein the address generator is further configured to generate a confidence level associated with the predicted address, and cause the confidence level to fall below a predetermined threshold if the confidence level falls below a predetermined threshold. The group controller relinquishes access to the next row at the predicted address. 100. The memory cell of claim 80, wherein the predicted address is generated by a series of flip-flops that sample the delayed generated address. 101. The memory cell of claim 100, wherein the delay is configurable via a multiplexer that selects between flip-flops that store sampled addresses. 102. The memory cell of claim 80, wherein the bank controller is configured to ignore predicted addresses received from the address generator during a predetermined period of time following a reset of the memory cell. 103. The memory cell of claim 80, wherein the address generator is configured to forego providing the predicted address to a random pattern in row access relative to the associated memory bank after detecting a random pattern. the group controller. 104. A memory cell comprising: one or more memory banks, wherein each of the one or more memory banks comprises: multiple lines; a first row controller configured to control a first subset of the plurality of rows; a second row controller configured to control a second subset of the plurality of rows; a single data input for receiving data to be stored in the plurality of rows; and a single data output to provide data retrieved from the plurality of rows. 105. The memory unit of claim 104, wherein the memory unit is configured to receive a first address at predetermined times for processing and to receive a second address for activation and access. 106. The memory cell of claim 104, wherein the first subset of the plurality of rows consists of even-numbered rows. 107. The memory cell of claim 106, wherein even numbered rows are located in half of the one or more memory banks. 108. The memory cell of claim 106, wherein odd-numbered rows are located in half of the one or more memory banks. 109. The memory cell of claim 104, wherein the second subset of the plurality of rows consists of odd-numbered rows. 110. The memory cell of claim 104, wherein the first subset of the plurality of rows is contained in a first subset of a memory bank adjacent to the first subset containing the plurality of A second subset of the memory banks of the second subset of rows. 111. The memory cell of claim 104, wherein the first row controller is configured to cause access to data included in a row of the first subset of the plurality of rows, and The second row controller activates a row of the second subset of the plurality of rows. 112. The memory cell of claim 111, wherein an activated row in the second subset of the plurality of rows is the same as the first subset of the plurality of rows of data being accessed Lines are separated by at least two lines. 113. The memory cell of claim 104, wherein the second row controller is configured to cause access to data included in a row of the second subset of the plurality of rows, and The first row controller activates a row of the second subset of the plurality of rows. 114. The memory cell of claim 113, wherein an enabled row in the first subset of the plurality of rows and a difference in data being accessed in the second subset of the plurality of rows Lines are separated by at least two lines. 115. The memory cell of claim 104, wherein each of the one or more memory banks includes a column input for receiving a column identification indicating a portion of a row to be accessed symbol. 116. The memory cell of claim 104, wherein an additional row of redundant pads is placed between each of the two rows of pads to create a distance for enabling activation. 117. The memory cell of claim 104, wherein rows in close proximity to each other may not be activated at the same time. 118. A distributed processor on a memory chip comprising: substrate; a memory array disposed on the substrate, the memory array comprising a plurality of discrete memory banks; a processing array disposed on the substrate, the processing array including a plurality of processor subunits, each of the processor subunits associated with a corresponding dedicated memory bank of the plurality of discrete memory banks and at least one memory pad disposed on the substrate, wherein the at least one memory pad is configured to function as at least one register of a register file for one or more of the plurality of processor subunits. 119. The memory chip of claim 118, wherein the at least one memory pad is included in at least one of the plurality of processor subunits of the processing array. 120. The memory chip of claim 118, wherein the register file is configured as a data register file. 121. The memory chip of claim 118, wherein the register file is configured as an address register file. 122. The memory chip of claim 118, wherein the at least one memory pad is configured to provide at least one register of a register file for one or more of the plurality of processor subunits to store pending memory data accessed by one or more of the plurality of processor subunits. 123. The memory chip of claim 118, wherein the at least one memory pad is configured to provide at least one register of a register file for one or more of the plurality of processing subunits, wherein the register The at least one register of the file is configured to store coefficients used by the plurality of processor subunits during execution of convolution accelerator operations by the plurality of processor subunits. 124. The memory chip of claim 118, wherein the at least one memory pad is a DRAM memory pad. 125. The memory chip of claim 118, wherein the at least one memory pad is configured to communicate via unidirectional access. 126. The memory chip of claim 118, wherein the at least one memory pad allows bidirectional access. 127. The memory chip of claim 1, further comprising at least one redundant memory pad disposed on the substrate, wherein the at least one redundant memory pad is configured to provide for the plurality of processors At least one redundancy register of one or more of the subunits. 128. The memory chip of claim 118, further comprising at least one memory pad disposed on the substrate, wherein the at least one memory pad contains elements configured to be provided in the plurality of processor subunits at least one redundant memory bit of one or more of at least one redundant register. 129. The memory chip of claim 118, further comprising: a first plurality of buses, each of the first plurality of buses connecting one of the plurality of processor sub-units to a corresponding dedicated memory bank; and A second plurality of buses, each of the second plurality of buses connecting one of the plurality of processor subunits to another of the plurality of processor subunits. 130. The memory chip of claim 118, wherein at least one of the processor subunits includes a counter configured to count backwards from a predefined number, and after the counter reaches a zero value, the The at least one of the processor sub-units is configured to stop the current task and trigger a memory refresh operation. 131. The memory chip of claim 118, wherein at least one of the processor sub-units includes a mechanism to stop a current task and trigger a memory refresh operation at a specific time to refresh the memory pad. 132. The memory chip of claim 118, wherein the register file is configured to function as a cache. 133. A method of executing at least one instruction in a distributed processor memory chip, the method comprising: retrieving one or more data values from a memory array of the distributed processor memory chips; storing the one or more data values in a register formed in a memory pad of the distributed processor memory chip; and Accessing the one or more data values stored in the register in accordance with at least one instruction executed by a processor element; wherein the memory array includes a plurality of discrete memory banks disposed on a substrate; wherein the processor element is a processor subunit included in a plurality of processor subunits in a processing array disposed on the substrate, wherein each of the processor subunits is associated with the plurality of Corresponding dedicated memory banks in the discrete memory banks are associated; and wherein the registers are provided by memory pads disposed on the substrate. 134. The method of claim 133, wherein the processor element is configured to function as an accelerator, and the method further comprises: accessing first data stored in the register; accessing second data from the memory array; An operation is performed on the first data and the second data. 135. The method of claim 133, wherein at least one memory pad includes a plurality of word lines and bit lines, and the method further comprises: Timing of loading the word lines and bit lines is determined, the timing being determined by the size of the memory pad. 136. The method of claim 133, further comprising: The registers are refreshed periodically. 137. The method of claim 12, wherein the memory pads comprise DRAM memory pads. 138. The method of claim 133, wherein the memory pad is included in at least one of the plurality of discrete memory banks of the memory array. 139. An apparatus comprising: substrate; a processing unit disposed on the substrate; and a memory unit disposed on the substrate, wherein the memory unit is configured to store data to be accessed by the processing unit, and wherein the processing unit includes a memory pad configured to act as a cache for the processing unit. 140. A method for distributed processing of at least one stream of information, the method comprising: The at least one information stream is received by one or more memory processing integrated circuits via a first communication channel; wherein each memory processing integrated circuit includes a controller, a plurality of processor sub-units, and a plurality of memory units; buffering, by the one or more memory processing integrated circuits, the at least one stream of information; performing, by the one or more memory processing integrated circuits, a first processing operation on the at least one stream of information to provide a first processing result; sending the first processing result to a processing integrated circuit; and performing, by the one or more memory processing integrated circuits, a second processing operation on the first processing result to provide a second processing result; wherein the size of the logical cells of the one or more memory processing integrated circuits is smaller than the size of the logical cells of the processing integrated circuits. 141. The method of claim 140, wherein each of the plurality of memory units is coupled to at least one of the plurality of processor sub-units. 142. The method of claim 140, wherein a total size of information elements of the at least one information stream received during a particular time duration exceeds the total size of a first processing result output during the particular time duration. 143. The method of claim 140, wherein a total size of the at least one information stream is less than a total size of the first processing result. 144. The method of claim 140, wherein the memory class manufacturing process is a DRAM manufacturing process. 145. The method of claim 140, wherein the processing integrated circuit is fabricated by a memory class of fabrication process; and Wherein the processing integrated circuit is manufactured by a logic class of manufacturing process. 146. The method of claim 140, wherein a logical cell of the one or more memory processing integrated circuits is at least twice the size of a corresponding logical cell of the processing integrated circuit. 147. The method of claim 140, wherein a critical dimension of a logical cell of the one or more memory processing integrated circuits is at least twice the critical dimension of a corresponding logical cell of the processing integrated circuit. 148. The method of claim 140, wherein a critical dimension of a memory cell of the one or more memory processing integrated circuits is at least twice the critical dimension of a corresponding logic cell of the processing integrated circuit. 149. The method of claim 140, comprising requesting, by the processing integrated circuit, the one or more memory processing integrated circuits to perform the first processing operation. 150. The method of claim 140, comprising instructing, by the processing integrated circuit, the one or more memory processing integrated circuits to perform the first processing operation. 151. The method of claim 140, comprising configuring, by the processing integrated circuit, the one or more memory processing integrated circuits to perform the first processing operation. 152. The method of claim 140, comprising performing the first processing operation by the one or more memory processing integrated circuits without the processing integrated circuits intervening. 153. The method of claim 140, wherein the first processing operation is less computationally complex than the second processing operation. 154. The method of claim 140, wherein the overall throughput of the first processing operation exceeds the overall throughput of the second processing operation. 155. The method of claim 140, wherein the at least one stream of information comprises one or more streams of preprocessed information. 156. The method of claim 157, wherein the one or more preprocessed information streams are data extracted from a network transport unit. 157. The method of claim 140, wherein a portion of the first processing operation is performed by one of the plurality of processor subunits and another portion of the first processing operation is performed by the plurality of The other of the processor subunits executes. 158. The method of claim 140, wherein the first processing operation and the second processing operation comprise cellular network processing operations. 159. The method of claim 140, wherein the first processing operation and the second processing operation comprise database processing operations. 160. The method of claim 140, wherein the first processing operation and the second processing operation comprise database analysis processing operations. 161. The method of claim 140, wherein the first processing operation and the second processing operation comprise artificial intelligence processing operations. 162. A method for distributed processing, the method comprising: Information units are received by one or more memory processing integrated circuits of a disaggregated system, the disaggregated system including one or more arithmetic subsystems separate from one or more storage subsystems; wherein the one or more memories Each of the processing integrated circuits includes a controller, a plurality of processor sub-units, and a plurality of memory units; wherein the one or more computing subsystems comprise a plurality of processing integrated circuits; wherein the size of logical cells of the one or more memory processing integrated circuits is at least twice the size of corresponding logical cells of the plurality of processing integrated circuits; performing, by the one or more memory processing integrated circuits, processing operations on the information units to provide processing results; and The processing results are output from the one or more memory processing integrated circuits. 163. The method of claim 162, comprising outputting the processing results to the one or more arithmetic subsystems of the factorized system. 164. The method of claim 162, comprising receiving the information unit from the one or more storage subsystems of the disaggregated system. 165. The method of claim 162, comprising outputting the processing results to the one or more storage subsystems of the disaggregated system. 166. The method of claim 162, comprising receiving the information unit from the one or more arithmetic subsystems of the factorized system. 167. The method of claim 166, wherein the information units sent from different groups of processing units of the plurality of processing integrated circuits comprise different portions of intermediate results of processing programs executed by the plurality of processing integrated circuits , wherein the group of processing units includes at least one processing integrated circuit. 168. The method of claim 167, comprising outputting results of an entire processing procedure by the one or more memory processing integrated circuits. 169. The method of claim 168, comprising sending the result of the entire processing procedure to each of the plurality of processing integrated circuits. 170. The method of claim 168, wherein the different portions of the intermediate results are different portions of an updated neural network model, and wherein the result of the entire processing procedure is the updated neural network model . 171. The method of claim 168, comprising sending the updated neural network model to each of the plurality of processing integrated circuits. 172. The method of claim 162, comprising outputting the processing result using a switching subunit of the disaggregated system. 173. The method of claim 162, wherein the one or more memory processing integrated circuits are included in a memory processing subsystem of the disaggregated system. 174. The method of claim 162, wherein at least one of the one or more memory processing integrated circuits is included in one or more computing subsystems of the disaggregated system. 175. The method of claim 162, wherein at least one of the one or more memory processing integrated circuits is included in one or more memory subsystems of the disaggregated system. 176. The method of claim 162, wherein at least one of the following is true: (a) receiving the information unit from at least one of the plurality of processing integrated circuits; and (b) converting The processing results are sent to one or more memory processing integrated circuits of the plurality of processing integrated circuits. 177. The method of claim 176, wherein a critical dimension of logical cells of the one or more memory processing integrated circuits exceeds a critical dimension of corresponding logical cells of the plurality of processing integrated circuits by at least two times. 178. The method of claim 176, wherein a critical dimension of memory cells of the one or more memory processing integrated circuits exceeds a critical dimension of corresponding logic cells of the plurality of processing integrated circuits by at least two times. 179. The method of claim 162, wherein the information units comprise preprocessed information units. 180. The method of claim 179, comprising preprocessing the information unit by the plurality of processing integrated circuits to provide the preprocessed information unit. 181. The method of claim 162, wherein the information unit conveys part of a model of a neural network. 182. The method of claim 162, wherein the information element conveys partial results of at least one database query. 183. The method of claim 162, wherein the information element conveys partial results of at least one aggregated database query. 184. A method for database analysis acceleration, the method comprising: receiving, by the memory processing integrated circuit, a database query, the database query including at least one relevance criterion that prompts a database entry in a database related to the database query; wherein the memory processing integrated circuit includes a controller, a plurality of processor sub-units, and a plurality of memory units; determining, by the memory processing integrated circuit and based on the at least one correlation criterion, a group of related database entries stored in the memory processing integrated circuit; and sending the group of related database entries to one or more processing entities for further processing without substantially sending unrelated database entries stored in the memory processing integrated circuit to the one or more processing entities entity; wherein the unrelated database entry is different from the related database entry. 185. The method of claim 184, wherein the one or more processing entities are included in the plurality of processor subunits of the memory processing integrated circuit. 186. The method of claim 185, comprising further processing, by the memory processing integrated circuit, the group of related database entries to complete a response to the database query. 187. The method of claim 186, comprising outputting the response to the database query from the memory processing integrated circuit. 188. The method of claim 187, wherein the output comprises an application flow control handler. 189. The method of claim 188, wherein the application of the flow control handler is responsive to an indicator output from the one or more processing entities, the indicator relating to one or more of the groups Completion of processing of multiple database entries. 190. The method of claim 185, comprising further processing, by the memory processing integrated circuit, the group of related database entries to provide an intermediate response to the database query. 191. The method of claim 190, comprising outputting the intermediate response to the database query from the memory processing integrated circuit. 192. The method of claim 191, wherein the output comprises an application flow control handler. 193. The method of claim 192, wherein the application of the flow control handler is responsive to an indicator output from the one or more processing entities, the indicator pertaining to a database entry for the group part of the processing is complete. 194. The method of claim 185, comprising generating, by the one or more processing entities, a processing status indicator that prompts the further processing of the group of related database entries progress. 195. The method of claim 185, comprising using the memory processing integrated circuit to further process the group of related database entries. 196. The method of claim 195, wherein the processing is performed by the plurality of processor subunits. 197. The method of claim 195, wherein the processing comprises computing an intermediate result by one of the plurality of processing sub-units, sending the intermediate result to one of the plurality of processing sub-units The other and additional computations are performed by the other processing sub-unit. 198. The method of claim 195, wherein the processing is performed by the controller. 199. The method of claim 195, wherein the processing is performed by the plurality of processor subunits and the controller. 200. The method of claim 184, wherein the one or more processing entities are external to the memory processing integrated circuit. 201. The method of claim 200, comprising outputting the group of related database entities from the memory processing integrated circuit. 202. The method of claim 201, wherein the output comprises an application flow control handler. 203. The method of claim 202, wherein the application of the flow control handler is responsive to an indicator output from the one or more processing entities, the indicator relating to a relationship with the one or more processing entities Handles dependencies of database entries associated with entities. 204. The method of claim 184, wherein the plurality of processor subunits comprise complete arithmetic logic units. 205. The method of claim 184, wherein the plurality of processor subunits comprise portions of arithmetic logic units. 206. The method of claim 184, wherein the plurality of processor subunits comprise memory controllers. 207. The method of claim 184, wherein the plurality of processor subunits comprise portions of a memory controller. 208. The method of claim 184, comprising outputting at least one of: (i) the group of related database entries, (ii) a response to the database query, and (iii) a pair of Intermediate responses to the database query. 209. The method of claim 212, wherein the output comprises application business shaping. 210. The method of claim 212, wherein the outputting comprises attempting to match a bandwidth used during the outputting with a maximum allowable bandwidth on a link coupling the memory processing integrated circuit to Requester unit. 211. The method of claim 212, wherein the output of the output comprises maintaining fluctuations in output traffic rates below a threshold. 212. The method of claim 184, wherein the one or more processing entities comprise a plurality of processing entities, wherein at least one of the plurality of processing entities belongs to the memory processing integrated circuit, and the plurality of processing entities At least another of the processing entities does not belong to the memory processing integrated circuit. 213. The method of claim 184, wherein the one or more processing entities belong to another memory processing integrated circuit. 214. A method for database analysis acceleration, the method comprising: receiving, by a plurality of memory processing integrated circuits, a database query including at least one relevance criterion that prompts a database entry in a database related to the database query; wherein each of the plurality of memory processing integrated circuits comprising a controller, a plurality of processor sub-units and a plurality of memory units; determining, by each of the plurality of memory processing integrated circuits and based on the at least one correlation criterion, a group of related database entries stored in the memory processing integrated circuit; and by each of the plurality of memory processing integrated circuits sending the group of related database entries stored in the memory processing integrated circuit to one or more processing entities for further processing, in essence Unrelated database entries stored in the memory processing integrated circuit are not sent to the one or more processing entities; wherein the unrelated database entries are different from the related database entries. 215. A method for database analysis acceleration, the method comprising: receiving, by an integrated circuit, a database query, the database query including at least one relevance criterion prompting a database entry in a database related to the database query; wherein the integrated circuit includes a controller, a screening unit, and a plurality of memory units; determining, by the screening unit and based on the at least one correlation criterion, a group of related database entries stored in the integrated circuit; and sending the group of related database entries to one or more processing entities external to the integrated circuit for further processing without substantially sending unrelated database entries stored in the integrated circuit to the integrated circuit One or more processing entities. 216. A method for database analysis acceleration, the method comprising: receiving, by the integrated circuit, a database query, the database query including at least one relevance criterion that prompts a database entry in a database related to the database query; wherein the integrated circuit includes a controller, a processing unit and a plurality of memory units; determining, by the processing unit and based on the at least one correlation criterion, a group of related database entries stored in the integrated circuit; processing the group of related database entries by the processing unit and not processing unrelated data entries stored in the integrated circuit by the processing unit to provide processing results; wherein the unrelated database entries are different from the relevant database entry; and The processing result is output from the integrated circuit. 217. A method for extracting feature vector related information, the method comprising: Fetch information is received by a memory processing integrated circuit for retrieval of a plurality of requested feature vectors mapped to a plurality of sentence segments; wherein the memory processing integrated circuit includes a controller, a plurality of processor sub-units, and a plurality of memory units, each of the memory units coupled to a processor sub-unit; retrieving the plurality of requested feature vectors from at least some of the plurality of memory cells; wherein the retrieving includes concurrently requesting storage in the two or more memory cells from the two or more memory cells the requested feature vector in the memory cells; and An output including at least one of: (a) the requested feature vector; and (b) the result of processing the requested feature vector is output from the memory processing integrated circuit. 218. The method of claim 217, wherein the output comprises the requested feature vector. 219. The method of claim 217, wherein the output comprises the result of the processing of the requested feature vector. 220. The method of claim 219, wherein the processing is performed by the plurality of processor subunits. 221. The method of claim 220, wherein the processing comprises sending the requested feature vector from one processing sub-unit to another processing sub-unit. 222. The method of claim 220, wherein the processing comprises computing an intermediate result by one processing subunit, sending the intermediate result to another processing subunit and computing another by the other processing subunit. An intermediate result or processing result. 223. The method of claim 219, wherein the processing is performed by the controller. 224. The method of claim 219, wherein the processing is performed by the plurality of processor subunits and the controller. 225. The method of claim 219, wherein the processing is performed by a vector processor of the memory processing integrated circuit. 226. The method of claim 217, wherein the controller is configured to concurrently request the requested feature based on a known mapping between statement segments and locations of feature vectors mapped to the statement segments vector. 227. The method of claim 11, wherein the mapping is uploaded during a power-on handler of the memory processing integrated circuit. 228. The method of claim 217, wherein the controller is configured to manage the retrieval of the plurality of requested feature vectors. 229. The method of claim 217, wherein the plurality of statement segments have a particular order, and wherein the outputting of the requested feature vector is performed according to the particular order. 230. The method of claim 229, wherein the retrieving of the plurality of requested feature vectors is performed according to the particular order. 231. The method of claim 229, wherein the retrieving of the plurality of requested feature vectors is performed at least partially out-of-order; and wherein the retrieving further comprises re-processing the plurality of requested feature vectors sort. 232. The method of claim 217, wherein the retrieving of the plurality of requested features comprises buffering the plurality of requested features before the plurality of requested feature vectors are read by the controller vector. 233. The method of claim 232, wherein the retrieving of the plurality of requested features comprises generating a buffer status indicator, the buffer status indicator indicating one or more associated with the plurality of memory cells When multiple buffers store one or more requested feature vectors. 234. The method of claim 233, comprising conveying the buffer status indicator via a dedicated control line. 235. The method of claim 234, wherein one dedicated control line is allocated per memory cell. 236. The method of claim 234, wherein the buffer status indicator comprises one or more status bits stored in one or more of the buffers. 237. The method of claim 234, comprising conveying the buffer status indicator via one or more shared control lines. 238. The method of claim 217, wherein the capture information is included in one or more capture commands at a first resolution, the first resolution representing a specified number of bits. 239. The method of claim 238, comprising managing, via the controller, the acquisition at a higher resolution, the higher resolution representing a number of bits below the particular number of bits. 240. The method of claim 238, wherein the controller is configured to manage the acquisition according to eigenvector resolution. 241. The method of claim 238, comprising independently managing, by the controller, the fetching. 242. The method of claim 217, wherein the plurality of processor subunits comprise complete arithmetic logic units. 243. The method of claim 217, wherein the plurality of processor subunits comprise portions of arithmetic logic units. 244. The method of claim 217, wherein the plurality of processor subunits comprise memory controllers. 245. The method of claim 217, wherein the plurality of processor subunits comprise portions of memory controllers. 246. The method of claim 217, wherein the output of the output comprises applying business shaping to the output. 247. The method of claim 217, wherein the output of the output comprises matching a bandwidth used during the output to a maximum allowable bandwidth on a link that couples the memory processing integrated circuit Coupled to the requestor unit. 248. The method of claim 217, wherein the outputting of the output comprises maintaining fluctuations in output traffic rates below a predetermined threshold. 249. The method of claim 217, wherein the fetching comprises applying predictive fetching to at least some of the requested feature vectors from a set of requested feature vectors stored in a single memory unit. 250. The method of claim 217, wherein the requested feature vector is distributed among the memory cells. 251. The method of claim 217, wherein the requested feature vector is distributed among the memory cells based on an expected retrieval pattern. 252. A method for memory-intensive processing, the method comprising: Processing operations are performed by a plurality of processors included in a hybrid device including a base die, a first memory resource associated with at least one second die, and a memory resource associated with at least one third die a second memory resource; wherein the base die and the at least one second die are connected to each other by inter-wafer bonding; retrieving information stored in the first memory resource using the plurality of processors; and sending additional information from the second memory resource to the first memory resource, wherein the total bandwidth of the first path between the base die and the at least one second die exceeds the at least one second The total bandwidth of the second path between the die and the at least one third die, and wherein the storage capacity of the first memory resource is less than the storage capacity of the second memory resource. 253. The method of claim 252, wherein the second memory resource comprises a high bandwidth memory (HBM) resource. 254. The method of claim 252, wherein the at least one third die comprises a stack of high bandwidth memory (HBM) chips. 255. The method of claim 252, wherein at least some of the second memory resources belong to a third one of at least one third die, the third die not using inter-wafer bonding connected to the base die. 256. The method of claim 252, wherein at least some of the second memory resources belong to a third one of at least one third die, the third die not using inter-wafer bonding is connected to a second one of the at least one second die. 257. The method of claim 252, wherein the first memory resource and the second memory resource comprise different levels of cache. 258. The method of claim 252, wherein the first memory resource is located between the base die and the second memory resource. 259. The method of claim 252, wherein the first memory resource is not located on top of the second memory resource. 260. The method of claim 252, comprising performing additional processing by a second one of at least one second die, the second die comprising a plurality of processor subunits and the first memory resource. 261. The method of claim 260, wherein at least one processor subunit is coupled to a dedicated portion of the first memory resource allocated to the processor subunit. 262. The method of claim 261, wherein the dedicated portion of the first memory resource comprises at least one memory bank. 263. The method of claim 252, wherein the plurality of processors belong to a memory processing chip that also includes the first memory resource. 264. The method of claim 252, wherein the base die includes the plurality of processors, wherein the plurality of processors include conductors coupled to the first processor via conductors formed using the wafer-to-wafer bonding A plurality of processor subunits of a memory resource. 265. The method of claim 264, wherein each processor subunit is coupled to a dedicated portion of the first memory resource allocated to the processor subunit. 266. A hybrid device for memory-intensive processing, the hybrid device comprising: base grain; multiple processors; a first memory resource of at least one second die; a second memory resource of at least one third die; wherein the base die and the at least one second die are connected to each other by inter-wafer bonding; wherein the plurality of processors are configured to perform processing operations and retrieve information stored in the first memory resource; and wherein the second memory resource is configured to send additional information from the second memory resource to the first memory resource; wherein the total bandwidth of the first paths between the base die and the at least one second die exceeds the total bandwidth of the second paths between the at least one second die and the at least one third die bandwidth, and The storage capacity of the first memory resource is smaller than the storage capacity of the second memory resource. 267. The hybrid device of claim 266, wherein the second memory resource comprises a high bandwidth memory (HBM) resource. 268. The hybrid device of claim 266, wherein the at least one third die comprises a stack of high bandwidth memory (HBM) memory chips. 269. The hybrid device of claim 266, wherein at least some of the second memory resources belong to a third one of the at least one third die that is not using a wafer connected to the base die in the case of an indirect bond. 270. The hybrid device of claim 266, wherein at least some of the second memory resources belong to a third one of the at least one third die that is not using a wafer is connected to a second one of the at least one second die in the case of an indirect bond. 271. The hybrid device of claim 266, wherein the first memory resource and the second memory resource comprise different levels of cache. 272. The hybrid device of claim 266, wherein the first memory resource is located between the base die and the second memory resource. 273. The hybrid device of claim 266, wherein the first memory resource is located to one side of the second memory resource. 274. The hybrid device of claim 266, wherein a second one of the at least one second die is configured to perform additional processing, wherein the second die comprises a plurality of processor subunits and all the first memory resource. 275. The hybrid device of claim 274, wherein each processor subunit is coupled to a dedicated portion of the first memory resource allocated to the processor subunit. 276. The hybrid device of claim 275, wherein the dedicated portion of the first memory resource comprises at least one memory bank. 277. The hybrid device of claim 266, wherein the plurality of processors comprise a plurality of processor sub-units of a memory processing chip that also includes the first memory resource. 278. The hybrid device of claim 266, wherein the base die includes the plurality of processors, wherein the plurality of processors include conductors coupled to the plurality of processors via conductors formed using the wafer-to-wafer bonding a plurality of processor subunits of the first memory resource. 279. The hybrid device of claim 278, wherein each processor subunit is coupled to a dedicated portion of the first memory resource allocated to the processor subunit. 280. A method for database acceleration, the method comprising: Retrieve a certain amount of information from the storage unit through the network communication interface of the database acceleration integrated circuit; first processing the amount of information to provide first processed information; A memory controller of an integrated circuit is accelerated using the database and the first processed information is sent via an interface to a plurality of memory processing integrated circuits, wherein each memory processing integrated circuit includes a controller, a plurality of processor sub-units, and a plurality of memory unit, performing second processing on at least a portion of the first processed information using the plurality of memory processing integrated circuits to provide second processed information; retrieving information from the plurality of memory processing integrated circuits by the memory controller of the database acceleration integrated circuit, wherein the retrieved information includes at least one of: (a) the first processed processing at least a portion of the information; and (b) at least a portion of the second processed information; performing database processing operations on the retrieved information using a database acceleration unit of the database acceleration integrated circuit to provide database acceleration results; and The database acceleration results are output. 281. The method of claim 280, comprising managing at least one of the fetching, first processing, sending, and processing using a management unit of the database acceleration integrated circuit. 282. The method of claim 281, wherein the managing is performed based on an execution plan generated by the management unit of the database acceleration integrated circuit. 283. The method of claim 281, wherein the managing is performed based on an execution plan received by the management unit of the database acceleration integrated circuit and not generated by the management unit. 284. The method of claim 281, wherein the managing comprises allocating at least one of: (a) network communication network interface resources; (b) decompression unit resources; (c) memory controllers resources; (d) multiple memory processing integrated circuit resources; and (e) database acceleration unit resources. 285. The method of claim 280, wherein the network communication interface comprises two or more different types of network communication ports. 286. The method of claim 285, wherein the two or more different types of network communication ports comprise a storage interface protocol port and a common network protocol storage interface port. 287. The method of claim 285, wherein the two or more different types of network communication ports comprise a storage interface protocol port and an ethernet protocol storage interface port. 288. The method of claim 285, wherein the two or more different types of network communication ports comprise storage interface protocol ports and PCIe ports. 289. The method of claim 280, comprising a management unit comprising a computing node of a computing system and controlled by a manager of the computing system. 290. The method of claim 280, comprising controlling, by a computing node of a computing system, at least one of the at least one of the fetching, first processing, sending, and third processing. 291. The method of claim 280, comprising using the database to accelerate integrated circuit execution of multiple tasks simultaneously. 292. The method of claim 280, comprising managing at least one of the fetching, first processing, sending, and third processing using a management unit external to the database acceleration integrated circuit. 293. The method of claim 280, wherein the database acceleration integrated circuit belongs to a computing system. 294. The method of claim 280, wherein the database acceleration integrated circuit is not part of a computing system. 295. The method of claim 280, comprising performing, by a computing node of a computing system, of the fetching, first processing, sending, and third processing based on an execution plan sent to the database acceleration integrated circuit at least one. 296. The method of claim 280, wherein the execution of the database processing operations comprises concurrent execution of database processing instructions by database processing subunits, wherein the database acceleration units comprise database accelerator subunits that share a shared memory unit. group of units. 297. The method of claim 296, wherein each database processing subunit is configured to execute a particular type of database processing instruction. 298. The method of claim 297, comprising dynamically linking database processing subunits to provide an execution pipeline for executing database processing operations comprising a plurality of instructions. 299. The method of claim 280, wherein the performing of the database processing operation comprises allocating resources of the database acceleration integrated circuit according to temporal I/O bandwidth. 300. The method of claim 280, comprising outputting the database acceleration results to local storage and retrieving the database acceleration results from the local storage. 301. The method of claim 280, wherein the network communication interface comprises an RDMA unit. 302. The method of claim 280, comprising exchanging information between database acceleration integrated circuits of one or more groups of database acceleration integrated circuits. 303. The method of claim 280, comprising exchanging database acceleration results between database acceleration integrated circuits of one or more groups of database acceleration integrated circuits. 304. The method of claim 280, comprising exchanging at least one of: (a) information; and (b) between database acceleration integrated circuits of one or more groups of database acceleration integrated circuits ) database to speed up results. 305. The method of claim 304, wherein the database acceleration integrated circuits of the group are connected to a common printed circuit board. 306. The method of claim 304, wherein the database acceleration integrated circuits of the group belong to modular units of the computerized system. 307. The method of claim 304, wherein different groups of database acceleration integrated circuits are connected to different printed circuit boards. 308. The method of claim 304, wherein different groups of database acceleration integrated circuits belong to different modular units of the computerized system. 309. The method of claim 304, comprising accelerating integrated circuit execution of distributed processing by the database of the one or more groups. 310. The method of claim 304, wherein the exchanging is performed using a network communication interface of the database acceleration integrated circuit of one or more groups. 311. The method of claim 304, wherein the exchange is performed on a plurality of groups connected to each other by star connections. 312. The method of claim 304, comprising using at least one switch for switching at least one of the following between database acceleration integrated circuits of different groups of the one or more groups : (a) information and (b) database accelerated results. 313. The method of claim 304, comprising accelerating at least some of the integrated circuits to perform distributed processing by the database of the one or more groups. 314. The method of claim 304, comprising performing distributed processing of first and second data structures, wherein the combined size of the first and second data structures exceeds storage of the plurality of memory processing integrated circuits ability. 315. The method of claim 314, wherein the performing of the distributed processing comprises performing multiple iterations of: (a) performing different pairs of a first data structure portion and a second data structure portion to Different databases speed up new assignments of integrated circuits; and (b) process the different pairs. 316. The method of claim 315, wherein the performing of the distributed processing comprises a database join operation. 317. The method of claim 315, wherein the performing of the distributed processing comprises: assigning different first data structure portions to different database acceleration integrated circuits of the one or more groups; and Perform multiple iterations of: newly assigning a different second data structure portion to a different database acceleration integrated circuit of the one or more groups, and Processing of the first and second data structure portions by the integrated circuit is accelerated by the database. 318. The method of claim 317, wherein the new assignment of the next iteration is performed in a manner that overlaps at least partially in time with the processing of the current iteration. 319. The method of claim 317, wherein the new allocation comprises exchanging a second data structure portion between the different database acceleration integrated circuits. 320. The method of claim 319, wherein the exchanging is performed in a manner that overlaps at least a portion of the time with the processing. 321. The method of claim 317, wherein the new allocation comprises exchanging a second data structure portion between the different database acceleration integrated circuits of a group; and once the exchange has been completed, integrating at the database acceleration Parts of the second data structure are exchanged between different groups of circuits. 322. The method of claim 280, wherein the database acceleration integrated circuit is included in a blade, the blade comprising a plurality of database acceleration integrated circuits, one or more non-volatile memory cells, an Ethernet switch, PCIe switches and Ethernet switches, and the plurality of memory processing integrated circuits. 323. An apparatus for database acceleration, the apparatus comprising: database acceleration integrated circuits; and a plurality of memory processing integrated circuits; wherein each memory processing integrated circuit includes a controller, a plurality of processor sub-units, and a plurality of memory units; wherein the network communication interface of the database acceleration integrated circuit is configured to receive information from the storage unit; wherein the database acceleration integrated circuit is configured to perform a first process on an amount of information to provide first processed information; wherein the memory controller of the database acceleration integrated circuit is configured to send the first processed information to the plurality of memory processing integrated circuits via an interface; wherein the plurality of memory processing integrated circuits are configured to perform second processing, by the plurality of memory processing integrated circuits, on at least a portion of the first processed information to provide second processed information; wherein the memory controller of the database acceleration integrated circuit is configured to retrieve information from the plurality of memory processing integrated circuits, wherein the retrieved information includes at least one of: (a) the first at least a portion of once processed information; and (b) at least a portion of said second processed information; wherein the database acceleration unit of the database acceleration integrated circuit of the database acceleration integrated circuit is configured to perform database processing operations on the retrieved information to provide database acceleration results; and wherein the database acceleration integrated circuit is configured to output the database acceleration result. 324. The device of claim 323, configured to manage at least one of the retrieval, first processing, and second processing of the retrieved information using a management unit of the database acceleration integrated circuit . 325. The apparatus of claim 324, wherein the management unit is configured to manage based on an execution plan generated by the management unit of the database acceleration integrated circuit. 326. The apparatus of claim 324, wherein the management unit is configured to manage based on an execution plan received by the management unit of the database acceleration integrated circuit and not generated by the management unit. 327. The apparatus of claim 324, wherein the management unit is configured to manage by allocating one or more of: (a) network communication network interface resources; (b) a decompression unit resources; (c) memory controller resources; (d) multiple memory processing integrated circuit resources; and (e) database acceleration unit resources. 328. The device of claim 323, wherein the network communication interface comprises different types of network communication ports. 329. The device of claim 328, wherein the different types of network communication ports comprise storage interface protocol ports and common network protocol storage interface ports. 330. The device of claim 328, wherein the different types of network communication ports include a storage interface protocol port and an ethernet protocol storage interface port. 331. The device of claim 328, wherein the different types of network communication ports comprise storage interface protocol ports and PCIe ports. 332. The device of claim 323, wherein the device is coupled to a management unit, the management unit comprising a computing node of a computing system and controlled by a manager of the computing system. 333. The apparatus of claim 323, configured to be controlled by a computing node of a computing system. 334. The apparatus of claim 323, configured to accelerate an integrated circuit by the database to perform multiple tasks simultaneously. 335. The apparatus of claim 323, wherein the database acceleration integrated circuit belongs to a computing system. 336. The apparatus of claim 323, wherein the database acceleration integrated circuit is not part of a computing system. 337. The apparatus of claim 323, configured to perform the fetching, first processing, sending, and third processing by a computing node of a computing system based on an execution plan sent to the database acceleration integrated circuit at least one of the. 338. The apparatus of claim 323, wherein the database acceleration unit is configured to concurrently execute database processing instructions by database processing subunits, wherein the database acceleration unit comprises a database accelerator subunit that shares a shared memory unit. group. 339. The apparatus of claim 338, wherein each database processing subunit is configured to execute a particular type of database processing instruction. 340. The apparatus of claim 339, wherein the apparatus is configured to dynamically link database processing subunits to provide an execution pipeline for performing database processing operations comprising a plurality of instructions. 341. The apparatus of claim 323, wherein the apparatus is configured to allocate resources of the database acceleration integrated circuit according to temporal I/O bandwidth. 342. The device of claim 323, wherein the device comprises local storage accessible by the database acceleration integrated circuit. 343. The device of claim 323, wherein the network communication interface comprises an RDMA unit. 344. The apparatus of claim 323, wherein the apparatus comprises one or more groups of database acceleration integrated circuits configured to operate at the one or more groups of database acceleration integrated circuits A database of groups accelerates the exchange of information between integrated circuits. 345. The apparatus of claim 323, wherein the apparatus comprises one or more groups of database acceleration integrated circuits configured to operate at the one or more groups of database acceleration integrated circuits The group's database accelerates the exchange of accelerated results between integrated circuits. 346. The apparatus of claim 323, wherein the apparatus comprises one or more groups of database acceleration integrated circuits configured to operate at the one or more groups of database acceleration integrated circuits The database-accelerated integrated circuits of the set perform at least one of: (a) information; and (b) database-accelerated results. 347. The apparatus of claim 346, wherein the database acceleration integrated circuits of the group are connected to the same printed circuit board. 348. The apparatus of claim 346, wherein the database acceleration integrated circuits of the group belong to modular units of the computerized system. 349. The apparatus of claim 346, wherein different groups of database acceleration integrated circuits are connected to different printed circuit boards. 350. The apparatus of claim 346, wherein different groups of database acceleration integrated circuits belong to different modular units of the computerized system. 351. The apparatus of claim 346, wherein the exchanging is performed using a network communication interface of the database acceleration integrated circuit of one or more groups. 352. The device of claim 346, wherein the exchange is performed on a plurality of groups connected to each other by star connections. 353. The apparatus of claim 346, wherein the apparatus is configured to use at least one switch for switching of the following between database acceleration integrated circuits of different ones of the one or more groups At least one of: (a) information; and (b) database accelerated results. 354. The apparatus of claim 346, wherein the apparatus is configured to accelerate some of the integrated circuits to perform distributed processing by the database of some of the one or more groups. 355. The apparatus of claim 346, wherein the apparatus is configured to perform distributed processing using first and second data structures, wherein the combined size of the first and second data structures exceeds the plurality of memories Handles the memory capacity of integrated circuits. 356. The apparatus of claim 355, wherein the apparatus is configured to perform the distributed processing by performing a plurality of iterations of: (a) performing a first data structure portion and a second data structure Partially different pairs to different databases accelerate new assignment of integrated circuits; and (b) processing the different pairs. 357. The apparatus of claim 355, wherein the distributed processing comprises a database join operation. 358. The apparatus of claim 355, wherein the apparatus is configured to perform the distributed processing by: assigning different first data structure portions to different database acceleration integrated circuits of the one or more groups; and Perform multiple iterations of: newly assigning a different second data structure portion to a different database acceleration integrated circuit of the one or more groups, and Processing of the first and second data structure portions by the integrated circuit is accelerated by the database. 359. The apparatus of claim 358, wherein the apparatus is configured to perform the new assignment of the next iteration in a manner that overlaps at least a portion of the time with processing of the current iteration. 360. The apparatus of claim 358, wherein the apparatus is configured to perform the new allocation by exchanging portions of a second data structure between the different database acceleration integrated circuits. 361. The device of claim 360, wherein the exchange is performed by the database acceleration integrated circuit in a manner that overlaps at least a portion of the time with the processing. 362. The apparatus of claim 358, wherein the apparatus is configured to perform the new allocation by: exchanging a second data structure portion between the different database acceleration integrated circuits of a group; and once After the exchange is complete, the second data structure portion is exchanged between the different groups of database acceleration integrated circuits. 363. The apparatus of claim 323, wherein the database acceleration integrated circuit is included in a blade, the blade comprising a plurality of database acceleration integrated circuits, one or more non-volatile memory units, an Ethernet switch, PCIe switches and Ethernet switches, and the plurality of memory processing integrated circuits. 364. A method for database acceleration, the method comprising: retrieving information from the storage unit through the network communication interface of the database acceleration integrated circuit; performing a first process on an amount of information to provide first processed information; Accelerating, by the database, a memory controller of an integrated circuit and sending the first processed information to a plurality of memory resources via an interface; retrieving information from the plurality of memory resources; performing database processing operations on the retrieved information by a database acceleration unit of the database acceleration integrated circuit to provide database acceleration results; and The database acceleration results are output. 365. The method of claim 364, further comprising processing the first processed information to provide second processed information, wherein the processing of the first processed information is performed by a plurality of processors, the The plurality of processors are located in one or more memory processing integrated circuits that further include the plurality of memory resources. 366. The method of claim 364, wherein the first process comprises filtering database entries. 367. The method of claim 364, wherein the second process comprises filtering database entries. 368. The method of claim 364, wherein the first and second processes comprise filtering database entries.

Images