[go: up one dir, main page]

US20230133604A1 - METHOD FOR CONTROLLING BW SLA IN NVMe oF ETHERNET SSD STORAGE SYSTEM - Google Patents

METHOD FOR CONTROLLING BW SLA IN NVMe oF ETHERNET SSD STORAGE SYSTEM Download PDF

Info

Publication number
US20230133604A1
US20230133604A1 US18/092,432 US202318092432A US2023133604A1 US 20230133604 A1 US20230133604 A1 US 20230133604A1 US 202318092432 A US202318092432 A US 202318092432A US 2023133604 A1 US2023133604 A1 US 2023133604A1
Authority
US
United States
Prior art keywords
storage device
attached storage
bandwidth
network
network attached
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US18/092,432
Inventor
Ramdas P. Kachare
Sompong Paul Olarig
Son T. PHAM
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Priority to US18/092,432 priority Critical patent/US20230133604A1/en
Publication of US20230133604A1 publication Critical patent/US20230133604A1/en
Assigned to SAMSUNG ELECTRONICS CO., LTD. reassignment SAMSUNG ELECTRONICS CO., LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: KACHARE, RAMDAS P., OLARIG, SOMPONG PAUL, PHAM, SON T.
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061Improving I/O performance
    • G06F3/0611Improving I/O performance in relation to response time
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38Information transfer, e.g. on bus
    • G06F13/40Bus structure
    • G06F13/4004Coupling between buses
    • G06F13/4022Coupling between buses using switching circuits, e.g. switching matrix, connection or expansion network
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38Information transfer, e.g. on bus
    • G06F13/42Bus transfer protocol, e.g. handshake; Synchronisation
    • G06F13/4282Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38Information transfer, e.g. on bus
    • G06F13/42Bus transfer protocol, e.g. handshake; Synchronisation
    • G06F13/4282Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
    • G06F13/4295Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus using an embedded synchronisation
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061Improving I/O performance
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629Configuration or reconfiguration of storage systems
    • G06F3/0635Configuration or reconfiguration of storage systems by changing the path, e.g. traffic rerouting, path reconfiguration
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0683Plurality of storage devices
    • G06F3/0688Non-volatile semiconductor memory arrays
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/40Bus networks
    • H04L12/40052High-speed IEEE 1394 serial bus
    • H04L12/40065Bandwidth and channel allocation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/0893Assignment of logical groups to network elements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/0896Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/50Network service management, e.g. ensuring proper service fulfilment according to agreements
    • H04L41/5003Managing SLA; Interaction between SLA and QoS
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/70Admission control; Resource allocation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/35Switches specially adapted for specific applications
    • H04L49/351Switches specially adapted for specific applications for local area network [LAN], e.g. Ethernet switches
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2213/00Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F2213/0024Peripheral component interconnect [PCI]
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2213/00Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F2213/0026PCI express
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/0894Policy-based network configuration management
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Definitions

  • the subject matter disclosed herein generally relates to storage systems, and more particularly, to a system and a method for managing bandwidth Service Level Agreements (SLAs) associated with Ethernet Solid-State Drive systems.
  • SLAs Service Level Agreements
  • Ethernet SSDs allow one or more remote server hosts to be connected to an eSSD through an Ethernet network in a remote Direct Attached Storage (rDAS) mode.
  • rDAS remote Direct Attached Storage
  • Conventional eSSD storage systems do not easily allow a bandwidth SLA to be applied to a particular eSSD in a storage system having a plurality of eSSDs.
  • An example embodiment provides an eSSD system that may include at least one eSSD, an Ethernet switch and a controller, such as a baseboard management controller.
  • the Ethernet switch may be coupled the at least one eSSD; and the controller may be coupled to the at least one eSSD and the Ethernet switch.
  • the controller may control the Ethernet switch to provide a predetermined bandwidth to the at least one eSSD in which the predetermined bandwidth may be based on bandwidth information for the at least one eSSD stored in a policy table of the controller.
  • each predetermined bandwidth for the at least one eSSD may include a predetermined ingress bandwidth and a predetermined egress bandwidth for the at least one eSSD.
  • the predetermined ingress bandwidth may be different from the predetermined egress bandwidth.
  • the predetermined bandwidth may be based on a service level associated with the at least one eSSD.
  • the predetermined bandwidth may be adaptively based on operating parameters of the at least one eSSD.
  • An example embodiment may provide a method to control bandwidth for an Ethernet solid-state drive (eSSD) system that may include receiving at a controller bandwidth information for at least one eSSD of a plurality of eSSDs; storing at the controller the received bandwidth information for the at least one eSSD; and configuring by the controller a bandwidth capacity of an Ethernet switch for the at least one eSSD in which the Ethernet switch may be coupled to the at least one eSSD.
  • the bandwidth capacity for the at least one eSSD may include a predetermined ingress bandwidth and a predetermined egress bandwidth for the at least one eSSD.
  • the predetermined ingress bandwidth may be different from the predetermined egress bandwidth.
  • the bandwidth capacity may be based on a service level associated with the at least one eSSD.
  • the bandwidth capacity may be adaptively based on operating parameters of the at least one eSSD.
  • FIG. 1 depicts an eSSD control platform according to the subject matter disclosed herein;
  • FIGS. 2 A- 2 C respectively depict example policy tables for managing and/or enforcing Ethernet ingress and egress bandwidth according to the subject matter disclosed herein;
  • FIG. 3 depicts a flow diagram of an example embodiment of a process for initializing SLA policy enforcement according to the subject matter disclosed herein;
  • FIG. 4 depicts a flow diagram of an example embodiment of a process for monitoring the status of eSSDs and modifying a policy table based on the status of eSSDs according to the subject matter disclosed herein.
  • first,” “second,” etc., as used herein, are used as labels for nouns that they precede, and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless explicitly defined as such.
  • same reference numerals may be used across two or more figures to refer to parts, components, blocks, circuits, units, or modules having the same or similar functionality. Such usage is, however, for simplicity of illustration and ease of discussion only; it does not imply that the construction or architectural details of such components or units are the same across all embodiments or such commonly-referenced parts/modules are the only way to implement the teachings of particular embodiments disclosed herein.
  • module refers to any combination of software, firmware and/or hardware configured to provide the functionality described herein in connection with a module.
  • software as applied to any implementation described herein, may be embodied as a software package, code and/or instruction set or instructions.
  • hardware as applied to any implementation described herein, may include, for example, singly or in any combination, hardwired circuitry, programmable circuitry, state-machine circuitry, and/or firmware that stores instructions executed by programmable circuitry.
  • the modules may, collectively or individually, be embodied as software, firmware and/or hardware that forms part of a larger system, such as, but not limited to, an integrated circuit (IC), system on-chip (SoC) and so forth.
  • IC integrated circuit
  • SoC system on-chip
  • a baseboard management controller (BMC) device in the eSSD control platform may maintain various bandwidth policies for one or more eSSDs connected to the control platform.
  • the BMC device may program, or configure, an Ethernet Switch in the control platform 100 to rate limit uplink ports connected to a host and/or downlink ports connected to the eSSDs.
  • the rate-limiting configurations may be different for an ingress direction (host writes) and for an egress direction (host reads) for each separate eSSD.
  • the rate-limiting configuration of the Ethernet switch may be set in response to commands and/or bandwidth information received from an administrator at a management server; in response to the BMC device enforcing one or more policies contained within the BMC device; or in response to a current health or status of one or more of the eSSDs in the system.
  • bandwidth information may include information relating to an ingress and/or an egress bandwidth that is to be applied to an individual eSSD, a group of eSSDs and/or an entire pool of eSSDs coupled to a control platform.
  • the BMC device has visibility of all of the SSDs that are present in the system and may monitor their respective parameters and operational status, so intelligent and complex policies may be implemented and monitored that are optimal for an individual eSSD, a subset of all of the eSSDs connected to the system, and/or all of the eSSDs connected to the system. Accordingly, the BMC device does not need to manage multiple proprietary mechanisms of the different SSDs that may be used in a system because the BMC device enforces the SLAs of the system by programming, or configuring, the Ethernet switch, thereby making the SLA bandwidth management independent of the eSSDs in the system.
  • the BMC device also includes a communication path to a storage management server from which a storage administrator may effectively manage the SLAs of a system.
  • FIG. 1 depicts an eSSD control platform 100 according to the subject matter disclosed herein.
  • the eSSD control platform 100 may include an Ethernet switch 101 , a BMC device 102 , a Peripheral Component Interconnect Express (PCIe) switch 103 , a mid-plane 104 , and one or more eSSDs 105 .
  • PCIe Peripheral Component Interconnect Express
  • FIG. 1 depicts only one Ethernet switch 101 and only one PCIe switch 103 are depicted in FIG. 1 as being part of the eSSD control platform 100 , it should be understood that multiple Ethernet switches 101 and multiple PCIe switches 103 may be part of the Ethernet SSD control platform 100 .
  • the components forming the eSSD control platform 100 may be embodied as separate components or as separate modules. As an alternative, two or more of the components forming the eSSD control platform 100 may be integral with each other.
  • the eSSD control platform 100 may be physically embodied as a chassis, or as a rack, in which one or more eSSDs 105 may be locally arranged with respect to the control platform 100 . One or more additional eSSDs 105 may also be remotely located with respect to the control platform 100 .
  • the control platform 100 may include 24 eSSDs 105 .
  • the eSSD control platform 100 may include 48 eSSDs 105 .
  • the eSSD control platform 100 may include any number of eSSDs 105 .
  • the Ethernet switch 101 may include a plurality of uplink Ethernet ports 106 , of which only one up-link port 106 is depicted in FIG. 1 .
  • the uplink ports 106 may connect the Ethernet switch 101 to one or more remote hosts 150 , of which only one host 150 is depicted.
  • the Ethernet switch 101 also may include a plurality of downlink Ethernet ports 107 , of which only one port 107 is depicted in FIG. 1 .
  • the downlink ports 107 may connect the Ethernet switch 101 through the mid-plane 104 to Ethernet ports 108 on individual eSSDs 105 .
  • each eSSD 105 may include an Ethernet port 108 .
  • the Ethernet switch 101 may perform rate limiting on the uplink Ethernet ports 106 and/or the downlink Ethernet ports 107 .
  • Rate limiting may have three components: packet or frame queueing, scheduling, and flow and congestion control.
  • a Weighted Fair Queueing (WFQ) rate-limiting ⁇ component may be used for implementing the queueing sub-systems.
  • Some examples of a scheduling rate-limiting component include a Weighted Round Robin (WRR) technique and a Deficit Round Robin (DRR) technique.
  • WRR Weighted Round Robin
  • DRR Deficit Round Robin
  • a flow and congestion control rate-limiting component may be implemented using, for example, a Tail Drop technique and/or a Weighted Random Early Discard (WRED) technique.
  • WRED Weighted Random Early Discard
  • an early congestion notification technique may be used to flow control the traffic.
  • PFC Priority Flow Control
  • the BMC device 102 may be located in a main switch board that is part of the Ethernet SSD control platform 100 . In another embodiment, the BMC device 102 and the Ethernet switch 101 may be integral with each other.
  • the BMC device 102 may be configured to provide management functions, such as discovery, configuration, operational status, and health monitoring of each respective eSSDs 105 .
  • the BMC device 102 may include one or more policy tables 109 that may contain information relating to the management functions of, but not limited to discovery, configuration, operational status, and health monitoring of each respective eSSDs 105 .
  • the BMC device 102 may use the policy tables 109 to manage and/or enforce one or more SLAs that have been associated with the eSSD control platform 100 .
  • one or more of the SLAs may relate to specific bandwidth policies that the BMC device 102 may enforce.
  • a bandwidth policy may relate to an individual eSSD 105 .
  • a bandwidth policy may relate to a group of eSSDs 105 that form a subset of all of the eSSDs 105 that are connected to the eSSD control platform 100 .
  • a bandwidth policy may relate to all of the eSSDs 105 that are connected to the eSSD control platform 100 .
  • Ethernet SSD control platform 100 there may be three communication paths between the BMC device 102 and the eSSDs 105 that may be used for obtaining information relating to the management functions provided by the BMC device 102 .
  • a first communication path may be over an Ethernet network 110 through the Ethernet switch 101 .
  • a second communication path may be over a PCIe bus 111 through the PCIe switch 103 and the mid-plane 104 .
  • a third path may be through a System Management Bus (SMBus) 112 that is connected between the BMC device 102 and the eSSDs 105 .
  • SMBus System Management Bus
  • the BMC device 102 may also have a management port 113 through which a management server 160 that is operated by an administrator (a user) may communicate with and control the BMC device 102 .
  • the BMC device 102 may receive bandwidth information for one or more eSSDs 105 that may be used for forming one or more bandwidth policies.
  • the management server 160 may be located in proximity to or remotely from the eSSD control platform 100 .
  • the uplink port 106 of the Ethernet switch 101 that connects to the remote host 150 may be a high-bandwidth link.
  • the uplink ports 106 of the Ethernet switch 101 may include multiple 40 Gbit/s and/or 100 Gbit/s links.
  • the individual eSSDs 105 typically may have a 10 Gbit/s and/or a 25 Gbit/s Ethernet port 108 .
  • the individual eSSDs 105 in the eSSD control platform 100 may be oversubscribed for the higher bandwidth of an uplink port 106 of the Ethernet switch 102 . Such an oversubscription of bandwidth is possible because generally not all eSSDs 105 are active at the same time.
  • the amount of Ethernet bandwidth consumed by a particular eSSD 105 depends upon the workload submitted to that eSSD by the remote host 150 .
  • the workload submitted by a remote host 150 may use a protocol that is based on the Non-Volatile Memory Express over Fabrics (NVMe over Fabrics) specification to send read/write IO commands to the eSSDs 105 .
  • NVMe over Fabrics Non-Volatile Memory Express over Fabrics
  • the amount of bandwidth consumed by an eSSD 105 at any time depends upon the number of I/O commands that have been submitted and the data transfer size of those commands.
  • For host write commands an eSSD 105 transfers user data from the remote host 150 to the local media of an eSSD 105 and, in so doing, consumes Ethernet ingress bandwidth.
  • an eSSD 105 transfers user data from the local media of the eSSD 105 to the remote host 150 , thus consuming Ethernet egress bandwidth.
  • Each eSSD 150 may include a predetermined number of command submission queues (SQ).
  • An eSSD 105 may perform arbitration to serve the queues. After selecting a SQ, the eSSD 105 fetches the next command for execution from the selected SQ. As part of the command execution, the eSSD 105 performs the specified data transfer to or from the remote host 150 . Once the data transfer has been completed, the eSSD 105 may send a completion message to the remote host 150 . At that point, the eSSD 105 returns to the SQ arbitration, and so on. The eSSD 105 continues to cycle through a command execution loop that arbitrates the SQs as long as there are pending commands in the SQs.
  • the BMC device 102 may use one or more policy tables 109 to manage and/or enforce one or more SLAs that have been associated with the eSSD control platform 100 .
  • An SLA may relate to specific bandwidth policies that are applied to one or more of the eSSDs 105 in the control platform 100 .
  • a bandwidth policy may be based on one or more aspects, such as, but not limited to, temporal aspects (time of day, day of week, etc.), eSSD operating parameters, eSSD power consumption, Flash-media technology, a user subscription rate, a cost of utility rate (i.e., weekday vs. holiday or weekend), and/or a promotional arrangement of a service provider.
  • VPD Vital Product Data
  • NVMe-MI NVMe Management Interface
  • FIGS. 2 A- 2 C respectively depict example policy tables for managing and/or enforcing Ethernet ingress and egress bandwidth (B/W) according to the subject matter disclosed herein.
  • FIG. 2 A depicts an example policy table 109 a for Ethernet ingress and egress B/W SLAs for a plurality of eSSDs in which the Ethernet ingress and egress B/W SLAs are identical for each of the eSSDs in the table.
  • each eSSD has an ingress B/W SLA of 5 Gbps and an egress B/W SLA of 10 Gbps.
  • FIG. 1 depicts an example policy table 109 a for Ethernet ingress and egress B/W SLAs for a plurality of eSSDs in which the Ethernet ingress and egress B/W SLAs are identical for each of the eSSDs in the table.
  • each eSSD has an ingress B/W SLA of 5 Gbps and an egress B/W SLA of
  • FIG. 2 B depicts another example policy table 109 b for Ethernet ingress and egress B/W SLAs for a plurality of eSSDs in which the Ethernet ingress and egress B/W SLAs are different for each of the eSSDs in the table.
  • FIG. 2 C depicts yet another example policy table 109 c for Ethernet ingress and egress B/W SLAs for a plurality of eSSDs in which the Ethernet ingress and egress B/W SLAs are identical for different groups of the eSSDs in the table.
  • eSSD- 0 and eSSD- 1 as a first group both have an ingress B/W SLA of 5 Gbps and an egress B/W SLA of 5 Gbps; whereas eSSD- 2 through eSSD-n as a second group each have an ingress B/W SLA of 5 Gbps and an egress B/W SLA of 10 Gbps.
  • FIG. 3 depicts a flow diagram of an example embodiment of a process 300 for initializing SLA policy enforcement according to the subject matter disclosed herein. That is, the process 300 may be a process that a BMC performs at power up or at system reset to identify eSSDs in the system and to initially configure one or more SLAs.
  • the process begins.
  • the BMC determines whether all eSSDs in the system have been identified. If not, flow continues to 303 where the BMC reads VPD through an NVMe-MI of an eSSD.
  • the VPD may include, but is not limited to, a common header, product information, multirecord information, internal use information, a chassis information area and a board specific area.
  • the BMC identifies the eSSD based on the VPD.
  • the BMC locally stores the eSSD parameters for the eSSD. Flow returns to 302 until all eSSDs in the platform have been identified and VPD corresponding to each eSSD has been locally stored.
  • Some reasons for a policy change at system power up or at system reset may be because one or more new eSSDs may be added to the plurality of eSSDs 105 , one or more eSSDs previously existing the plurality of eSSDs 105 may be removed, and or one or more eSSDs previously existing the plurality of eSSDs 105 may have become non-operational. Events such as, but not limited to these events may cause different policies to become active. Additionally, eSSD characteristics, such as Bit Error Rate (BER), operating temperature, storage capacity, etc., may change over time and thereby cause a need for a policy change.
  • BER Bit Error Rate
  • one or more eSSDs 105 may be reassigned to different hosts and/or applications over time, which may result in a need for policy change. Still another example that may cause a policy change may involve multiple policies that are applied at different times, e.g., daytime vs nighttime or weekday vs weekend.
  • FIG. 4 depicts a flow diagram of an example embodiment of a process 400 for monitoring the status of eSSDs and modifying a policy table based on the status of eSSDs according to the subject matter disclosed herein.
  • the process begins at 401 .
  • Flow continues to 402 where the BMC determines whether the (health) status of all eSSDs in the system has been scanned. If not, flow continues to 403 where the (health) status of a first eSSD that has not been scanned is scanned using, for example, a NVMe-MI health status poll.
  • the BMC reads the SMART/Health Information log of the eSSD.
  • the BMC locally stores the current health status of the eSSD. Flow returns to 402 .
  • the health status parameters of an eSSD that may be monitored by the BMC may include, but are not limited to a data structure within the eSSD, an operating temperature, media integrity (i.e., BER), remaining life, available spare capacity, total capacity, used capacity, amount of data read/written since a predetermined time, a number of host commands responded to since a predetermined time, drive writes per day (DWPD), eSSD controller busy time, and any proprietary information.
  • policies change may be determined to be needed at 406 may include that one or more new eSSDs may be added (i.e., hot swapped) to the plurality of eSSDs 105 , one or more eSSDs previously existing the plurality of eSSDs 105 may be removed (i.e., hot swapped), and or one or more eSSDs previously existing the plurality of eSSDs 105 may have become non-operational. Events such as, but not limited to these events may cause different policies to become active. Additionally, as mentioned eSSD characteristics, such as BER, operating temperature, storage capacity, etc., may change over time and thereby cause a need for a policy change.
  • eSSD characteristics such as BER, operating temperature, storage capacity, etc.
  • one or more eSSDs 105 may be reassigned to different hosts and/or applications over time, which may result in a need for policy change. Still another example that may cause a policy change at 406 may involve multiple policies that are applied at different times, e.g., daytime vs nighttime or weekday vs weekend.
  • Ethernet switch it is determined based on the change of health status for the eSSD and the contents of the policy table relating to the health status of the eSSD whether the Ethernet switch should be reconfigured. If not, flow returns to 402 . If the Ethernet switch should be reconfigured, flow continues to 409 where the BMC reprograms the Ethernet switch to an updated configuration.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Mathematical Physics (AREA)
  • Computer Hardware Design (AREA)
  • Small-Scale Networks (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Debugging And Monitoring (AREA)

Abstract

An Ethernet solid-state drive (eSSD) system includes a plurality of eSSDs, an Ethernet switch and a baseboard management controller. The Ethernet switch is coupled to each of the eSSDs, and the baseboard management controller is coupled to the each of the eSSDs and to the Ethernet switch. The baseboard management controller controls the Ethernet switch to provide to each eSSD a corresponding predetermined bandwidth that is based on bandwidth information for the eSSD that is stored in a policy table of the baseboard management controller. The at least one predetermined bandwidth may include a predetermined ingress bandwidth and a predetermined egress bandwidth for the corresponding eSSD. The at least one predetermined bandwidth may be based on a service level associated with the corresponding eSSD, and may be adaptively based on operating parameters of the corresponding eSSD.

Description

    REFERENCE TO RELATED APPLICATIONS
  • This patent application is a continuation of U.S. patent application Ser. No. 15/487,431, filed Apr. 13, 2017, which is incorporated herein by reference in its entirety and which claims the priority benefit under 35 U.S.C. § 119(e) of U.S. Provisional Patent Application Ser. No. 62/462,901, filed Feb. 23, 2017, the disclosure of which is incorporated herein by reference in its entirety.
  • TECHNICAL FIELD
  • The subject matter disclosed herein generally relates to storage systems, and more particularly, to a system and a method for managing bandwidth Service Level Agreements (SLAs) associated with Ethernet Solid-State Drive systems.
  • BACKGROUND
  • Ethernet SSDs (eSSDs) allow one or more remote server hosts to be connected to an eSSD through an Ethernet network in a remote Direct Attached Storage (rDAS) mode. Conventional eSSD storage systems, however, do not easily allow a bandwidth SLA to be applied to a particular eSSD in a storage system having a plurality of eSSDs.
  • SUMMARY
  • An example embodiment provides an eSSD system that may include at least one eSSD, an Ethernet switch and a controller, such as a baseboard management controller. The Ethernet switch may be coupled the at least one eSSD; and the controller may be coupled to the at least one eSSD and the Ethernet switch. The controller may control the Ethernet switch to provide a predetermined bandwidth to the at least one eSSD in which the predetermined bandwidth may be based on bandwidth information for the at least one eSSD stored in a policy table of the controller. In one embodiment, each predetermined bandwidth for the at least one eSSD may include a predetermined ingress bandwidth and a predetermined egress bandwidth for the at least one eSSD. In one embodiment, the predetermined ingress bandwidth may be different from the predetermined egress bandwidth. In another embodiment, the predetermined bandwidth may be based on a service level associated with the at least one eSSD. In still another embodiment, the predetermined bandwidth may be adaptively based on operating parameters of the at least one eSSD.
  • An example embodiment may provide a method to control bandwidth for an Ethernet solid-state drive (eSSD) system that may include receiving at a controller bandwidth information for at least one eSSD of a plurality of eSSDs; storing at the controller the received bandwidth information for the at least one eSSD; and configuring by the controller a bandwidth capacity of an Ethernet switch for the at least one eSSD in which the Ethernet switch may be coupled to the at least one eSSD. In one embodiment, the bandwidth capacity for the at least one eSSD may include a predetermined ingress bandwidth and a predetermined egress bandwidth for the at least one eSSD. In one embodiment, the predetermined ingress bandwidth may be different from the predetermined egress bandwidth. In one embodiment, the bandwidth capacity may be based on a service level associated with the at least one eSSD. In another embodiment, the bandwidth capacity may be adaptively based on operating parameters of the at least one eSSD.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • In the following section, the aspects of the subject matter disclosed herein will be described with reference to exemplary embodiments illustrated in the figures, in which:
  • FIG. 1 depicts an eSSD control platform according to the subject matter disclosed herein;
  • FIGS. 2A-2C respectively depict example policy tables for managing and/or enforcing Ethernet ingress and egress bandwidth according to the subject matter disclosed herein;
  • FIG. 3 depicts a flow diagram of an example embodiment of a process for initializing SLA policy enforcement according to the subject matter disclosed herein; and
  • FIG. 4 depicts a flow diagram of an example embodiment of a process for monitoring the status of eSSDs and modifying a policy table based on the status of eSSDs according to the subject matter disclosed herein.
  • DETAILED DESCRIPTION
  • In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the disclosure. It will be understood, however, by those skilled in the art that the disclosed aspects may be practiced without these specific details. In other instances, well-known methods, procedures, components and circuits have not been described in detail not to obscure the subject matter disclosed herein.
  • Reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment disclosed herein. Thus, the appearances of the phrases “in one embodiment” or “in an embodiment” or “according to one embodiment” (or other phrases having similar import) in various places throughout this specification may not be necessarily all referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In this regard, as used herein, the word “exemplary” means “serving as an example, instance, or illustration.” Any embodiment described herein as “exemplary” is not to be construed as necessarily preferred or advantageous over other embodiments. Also, depending on the context of discussion herein, a singular term may include the corresponding plural forms and a plural term may include the corresponding singular form. It is further noted that various figures (including component diagrams) shown and discussed herein are for illustrative purpose only, and are not drawn to scale. Similarly, various waveforms and timing diagrams are shown for illustrative purpose only. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity. Further, if considered appropriate, reference numerals have been repeated among the figures to indicate corresponding and/or analogous elements.
  • The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting of the claimed subject matter. As used herein, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The terms “first,” “second,” etc., as used herein, are used as labels for nouns that they precede, and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless explicitly defined as such. Furthermore, the same reference numerals may be used across two or more figures to refer to parts, components, blocks, circuits, units, or modules having the same or similar functionality. Such usage is, however, for simplicity of illustration and ease of discussion only; it does not imply that the construction or architectural details of such components or units are the same across all embodiments or such commonly-referenced parts/modules are the only way to implement the teachings of particular embodiments disclosed herein.
  • Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this subject matter belongs. For example, the term “mod” as used herein means “modulo.” It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
  • As used herein, the term “module” refers to any combination of software, firmware and/or hardware configured to provide the functionality described herein in connection with a module. The term “software,” as applied to any implementation described herein, may be embodied as a software package, code and/or instruction set or instructions. The term “hardware,” as applied to any implementation described herein, may include, for example, singly or in any combination, hardwired circuitry, programmable circuitry, state-machine circuitry, and/or firmware that stores instructions executed by programmable circuitry. The modules may, collectively or individually, be embodied as software, firmware and/or hardware that forms part of a larger system, such as, but not limited to, an integrated circuit (IC), system on-chip (SoC) and so forth.
  • The subject matter disclosed herein provides a system and a technique to enforce one or more bandwidth SLAs by an eSSD control platform in a non-intrusive manner from the point of view of eSSDs in the system. A baseboard management controller (BMC) device in the eSSD control platform may maintain various bandwidth policies for one or more eSSDs connected to the control platform. The BMC device may program, or configure, an Ethernet Switch in the control platform 100 to rate limit uplink ports connected to a host and/or downlink ports connected to the eSSDs. The rate-limiting configurations may be different for an ingress direction (host writes) and for an egress direction (host reads) for each separate eSSD. The rate-limiting configuration of the Ethernet switch may be set in response to commands and/or bandwidth information received from an administrator at a management server; in response to the BMC device enforcing one or more policies contained within the BMC device; or in response to a current health or status of one or more of the eSSDs in the system. As used herein, the term “bandwidth information” may include information relating to an ingress and/or an egress bandwidth that is to be applied to an individual eSSD, a group of eSSDs and/or an entire pool of eSSDs coupled to a control platform.
  • The BMC device has visibility of all of the SSDs that are present in the system and may monitor their respective parameters and operational status, so intelligent and complex policies may be implemented and monitored that are optimal for an individual eSSD, a subset of all of the eSSDs connected to the system, and/or all of the eSSDs connected to the system. Accordingly, the BMC device does not need to manage multiple proprietary mechanisms of the different SSDs that may be used in a system because the BMC device enforces the SLAs of the system by programming, or configuring, the Ethernet switch, thereby making the SLA bandwidth management independent of the eSSDs in the system. The BMC device also includes a communication path to a storage management server from which a storage administrator may effectively manage the SLAs of a system.
  • FIG. 1 depicts an eSSD control platform 100 according to the subject matter disclosed herein. In one embodiment, the eSSD control platform 100 may include an Ethernet switch 101, a BMC device 102, a Peripheral Component Interconnect Express (PCIe) switch 103, a mid-plane 104, and one or more eSSDs 105. Although only one Ethernet switch 101 and only one PCIe switch 103 are depicted in FIG. 1 as being part of the eSSD control platform 100, it should be understood that multiple Ethernet switches 101 and multiple PCIe switches 103 may be part of the Ethernet SSD control platform 100. Additionally, the components forming the eSSD control platform 100 may be embodied as separate components or as separate modules. As an alternative, two or more of the components forming the eSSD control platform 100 may be integral with each other.
  • The eSSD control platform 100 may be physically embodied as a chassis, or as a rack, in which one or more eSSDs 105 may be locally arranged with respect to the control platform 100. One or more additional eSSDs 105 may also be remotely located with respect to the control platform 100. In one embodiment, the control platform 100 may include 24 eSSDs 105. In another embodiment, the eSSD control platform 100 may include 48 eSSDs 105. In still another embodiment, the eSSD control platform 100 may include any number of eSSDs 105.
  • The Ethernet switch 101 may include a plurality of uplink Ethernet ports 106, of which only one up-link port 106 is depicted in FIG. 1 . The uplink ports 106 may connect the Ethernet switch 101 to one or more remote hosts 150, of which only one host 150 is depicted. The Ethernet switch 101 also may include a plurality of downlink Ethernet ports 107, of which only one port 107 is depicted in FIG. 1 . The downlink ports 107 may connect the Ethernet switch 101 through the mid-plane 104 to Ethernet ports 108 on individual eSSDs 105. In one embodiment, each eSSD 105 may include an Ethernet port 108. The Ethernet switch 101 may perform rate limiting on the uplink Ethernet ports 106 and/or the downlink Ethernet ports 107. Rate limiting may have three components: packet or frame queueing, scheduling, and flow and congestion control. A Weighted Fair Queueing (WFQ) rate-limiting\component may be used for implementing the queueing sub-systems. Some examples of a scheduling rate-limiting component include a Weighted Round Robin (WRR) technique and a Deficit Round Robin (DRR) technique. A flow and congestion control rate-limiting component may be implemented using, for example, a Tail Drop technique and/or a Weighted Random Early Discard (WRED) technique. In some embodiments, an early congestion notification technique may be used to flow control the traffic. For Ethernet links, pause frames and Priority Flow Control (PFC) standard techniques may be used.
  • In one embodiment, the BMC device 102 may be located in a main switch board that is part of the Ethernet SSD control platform 100. In another embodiment, the BMC device 102 and the Ethernet switch 101 may be integral with each other.
  • The BMC device 102 may be configured to provide management functions, such as discovery, configuration, operational status, and health monitoring of each respective eSSDs 105. In one embodiment, the BMC device 102 may include one or more policy tables 109 that may contain information relating to the management functions of, but not limited to discovery, configuration, operational status, and health monitoring of each respective eSSDs 105. The BMC device 102 may use the policy tables 109 to manage and/or enforce one or more SLAs that have been associated with the eSSD control platform 100. In one embodiment, one or more of the SLAs may relate to specific bandwidth policies that the BMC device 102 may enforce. In one embodiment, a bandwidth policy may relate to an individual eSSD 105. In another embodiment, a bandwidth policy may relate to a group of eSSDs 105 that form a subset of all of the eSSDs 105 that are connected to the eSSD control platform 100. In still another embodiment, a bandwidth policy may relate to all of the eSSDs 105 that are connected to the eSSD control platform 100.
  • In one embodiment of the Ethernet SSD control platform 100, there may be three communication paths between the BMC device 102 and the eSSDs 105 that may be used for obtaining information relating to the management functions provided by the BMC device 102. A first communication path may be over an Ethernet network 110 through the Ethernet switch 101. A second communication path may be over a PCIe bus 111 through the PCIe switch 103 and the mid-plane 104. A third path may be through a System Management Bus (SMBus) 112 that is connected between the BMC device 102 and the eSSDs 105. The BMC device 102 may also have a management port 113 through which a management server 160 that is operated by an administrator (a user) may communicate with and control the BMC device 102. The BMC device 102 may receive bandwidth information for one or more eSSDs 105 that may be used for forming one or more bandwidth policies. The management server 160 may be located in proximity to or remotely from the eSSD control platform 100.
  • The uplink port 106 of the Ethernet switch 101 that connects to the remote host 150 may be a high-bandwidth link. In one embodiment, the uplink ports 106 of the Ethernet switch 101 may include multiple 40 Gbit/s and/or 100 Gbit/s links. The individual eSSDs 105 typically may have a 10 Gbit/s and/or a 25 Gbit/s Ethernet port 108. The individual eSSDs 105 in the eSSD control platform 100 may be oversubscribed for the higher bandwidth of an uplink port 106 of the Ethernet switch 102. Such an oversubscription of bandwidth is possible because generally not all eSSDs 105 are active at the same time. The amount of Ethernet bandwidth consumed by a particular eSSD 105 depends upon the workload submitted to that eSSD by the remote host 150.
  • In one embodiment, the workload submitted by a remote host 150 may use a protocol that is based on the Non-Volatile Memory Express over Fabrics (NVMe over Fabrics) specification to send read/write IO commands to the eSSDs 105. Accordingly, the amount of bandwidth consumed by an eSSD 105 at any time depends upon the number of I/O commands that have been submitted and the data transfer size of those commands. For host write commands, an eSSD 105 transfers user data from the remote host 150 to the local media of an eSSD 105 and, in so doing, consumes Ethernet ingress bandwidth. For host read commands, an eSSD 105 transfers user data from the local media of the eSSD 105 to the remote host 150, thus consuming Ethernet egress bandwidth.
  • Each eSSD 150 may include a predetermined number of command submission queues (SQ). An eSSD 105 may perform arbitration to serve the queues. After selecting a SQ, the eSSD 105 fetches the next command for execution from the selected SQ. As part of the command execution, the eSSD 105 performs the specified data transfer to or from the remote host 150. Once the data transfer has been completed, the eSSD 105 may send a completion message to the remote host 150. At that point, the eSSD 105 returns to the SQ arbitration, and so on. The eSSD 105 continues to cycle through a command execution loop that arbitrates the SQs as long as there are pending commands in the SQs.
  • The BMC device 102 may use one or more policy tables 109 to manage and/or enforce one or more SLAs that have been associated with the eSSD control platform 100. An SLA may relate to specific bandwidth policies that are applied to one or more of the eSSDs 105 in the control platform 100. A bandwidth policy may be based on one or more aspects, such as, but not limited to, temporal aspects (time of day, day of week, etc.), eSSD operating parameters, eSSD power consumption, Flash-media technology, a user subscription rate, a cost of utility rate (i.e., weekday vs. holiday or weekend), and/or a promotional arrangement of a service provider. Information obtained from an eSSD by a Vital Product Data (VPD) read may be stored locally at the BMC device 102 and used to adaptively control a bandwidth SLA. In one embodiment, the status and/or parameters of the eSSDs 105 may be read by the BMC device 102 using a NVMe Management Interface (NVMe-MI) based protocol running over the SMBus and/or PCIe interface.
  • FIGS. 2A-2C respectively depict example policy tables for managing and/or enforcing Ethernet ingress and egress bandwidth (B/W) according to the subject matter disclosed herein. FIG. 2A depicts an example policy table 109 a for Ethernet ingress and egress B/W SLAs for a plurality of eSSDs in which the Ethernet ingress and egress B/W SLAs are identical for each of the eSSDs in the table. In particular, each eSSD has an ingress B/W SLA of 5 Gbps and an egress B/W SLA of 10 Gbps. FIG. 2B depicts another example policy table 109 b for Ethernet ingress and egress B/W SLAs for a plurality of eSSDs in which the Ethernet ingress and egress B/W SLAs are different for each of the eSSDs in the table. FIG. 2C depicts yet another example policy table 109 c for Ethernet ingress and egress B/W SLAs for a plurality of eSSDs in which the Ethernet ingress and egress B/W SLAs are identical for different groups of the eSSDs in the table. In particular, eSSD-0 and eSSD-1 as a first group both have an ingress B/W SLA of 5 Gbps and an egress B/W SLA of 5 Gbps; whereas eSSD-2 through eSSD-n as a second group each have an ingress B/W SLA of 5 Gbps and an egress B/W SLA of 10 Gbps.
  • FIG. 3 depicts a flow diagram of an example embodiment of a process 300 for initializing SLA policy enforcement according to the subject matter disclosed herein. That is, the process 300 may be a process that a BMC performs at power up or at system reset to identify eSSDs in the system and to initially configure one or more SLAs. At 301, the process begins. At 302, the BMC determines whether all eSSDs in the system have been identified. If not, flow continues to 303 where the BMC reads VPD through an NVMe-MI of an eSSD. The VPD may include, but is not limited to, a common header, product information, multirecord information, internal use information, a chassis information area and a board specific area. At 304, the BMC identifies the eSSD based on the VPD. At 305, the BMC locally stores the eSSD parameters for the eSSD. Flow returns to 302 until all eSSDs in the platform have been identified and VPD corresponding to each eSSD has been locally stored.
  • If, at 302, all eSSDs in the platform have been identified, flow continues to 306 where the BMC determines whether an SLA policy is to be added, modified or deleted from a policy management table. If an SLA policy is to be added, flow continues to 307 and to 310 where a policy is added to a policy management table. If an SLA policy is to be modified, flow continues to 308 and to 310 wherein policy is modified in a policy management table. If an SLA policy is to be deleted, flow continues to 309 and to 310 where a policy is deleted from an SLA policy table. Some reasons for a policy change at system power up or at system reset may be because one or more new eSSDs may be added to the plurality of eSSDs 105, one or more eSSDs previously existing the plurality of eSSDs 105 may be removed, and or one or more eSSDs previously existing the plurality of eSSDs 105 may have become non-operational. Events such as, but not limited to these events may cause different policies to become active. Additionally, eSSD characteristics, such as Bit Error Rate (BER), operating temperature, storage capacity, etc., may change over time and thereby cause a need for a policy change. Further, one or more eSSDs 105 may be reassigned to different hosts and/or applications over time, which may result in a need for policy change. Still another example that may cause a policy change may involve multiple policies that are applied at different times, e.g., daytime vs nighttime or weekday vs weekend.
  • Flow then continues to 311 where the configuration of the Ethernet switch 101 is updated based on the update to the policy table at 310. Flow returns to 302. If at 306, no SLA policy is to be added, modified or deleted, flow returns (not shown) to 302.
  • FIG. 4 depicts a flow diagram of an example embodiment of a process 400 for monitoring the status of eSSDs and modifying a policy table based on the status of eSSDs according to the subject matter disclosed herein. The process begins at 401. Flow continues to 402 where the BMC determines whether the (health) status of all eSSDs in the system has been scanned. If not, flow continues to 403 where the (health) status of a first eSSD that has not been scanned is scanned using, for example, a NVMe-MI health status poll. At 404, the BMC reads the SMART/Health Information log of the eSSD. At 405, the BMC locally stores the current health status of the eSSD. Flow returns to 402.
  • If, at 402, the BMC has scanned the health status of all eSSDs in the system, flow continues to 406 where the BMC determines whether there has been a change of health status for any of the eSSDs. The health status parameters of an eSSD that may be monitored by the BMC may include, but are not limited to a data structure within the eSSD, an operating temperature, media integrity (i.e., BER), remaining life, available spare capacity, total capacity, used capacity, amount of data read/written since a predetermined time, a number of host commands responded to since a predetermined time, drive writes per day (DWPD), eSSD controller busy time, and any proprietary information.
  • If at 406, no changes of health status for any of the eSSDs have been determined or detected, flow returns to 402. If, at 406, there has been a change of health status for any of the eSSDs, flow continues to 407 where the BMC accesses a policy table relating to the health status of the affected eSSD. Other reasons for that policy change may be determined to be needed at 406 may include that one or more new eSSDs may be added (i.e., hot swapped) to the plurality of eSSDs 105, one or more eSSDs previously existing the plurality of eSSDs 105 may be removed (i.e., hot swapped), and or one or more eSSDs previously existing the plurality of eSSDs 105 may have become non-operational. Events such as, but not limited to these events may cause different policies to become active. Additionally, as mentioned eSSD characteristics, such as BER, operating temperature, storage capacity, etc., may change over time and thereby cause a need for a policy change. Further, one or more eSSDs 105 may be reassigned to different hosts and/or applications over time, which may result in a need for policy change. Still another example that may cause a policy change at 406 may involve multiple policies that are applied at different times, e.g., daytime vs nighttime or weekday vs weekend.
  • At 408, it is determined based on the change of health status for the eSSD and the contents of the policy table relating to the health status of the eSSD whether the Ethernet switch should be reconfigured. If not, flow returns to 402. If the Ethernet switch should be reconfigured, flow continues to 409 where the BMC reprograms the Ethernet switch to an updated configuration.
  • As will be recognized by those skilled in the art, the innovative concepts described herein can be modified and varied over a wide range of applications. Accordingly, the scope of claimed subject matter should not be limited to any of the specific exemplary teachings discussed above, but is instead defined by the following claims.

Claims (20)

What is claimed is:
1. A system comprising:
a network attached storage device;
a network switch coupled to a port of the network attached storage device; and
a controller coupled to the network switch, wherein the controller is configured to:
store, in a data structure of the controller, bandwidth information for the network attached storage device; and
program the network switch to provide a bandwidth to the network attached storage device by controlling, based on the bandwidth information, data transfers using the network switch;
wherein the network switch is configured to rate limit data transfers to the network attached storage device.
2. The system of claim 1, wherein the bandwidth information for the network attached storage device comprises at least one of an ingress bandwidth or an egress bandwidth for the network attached storage device.
3. The system of claim 2, wherein the bandwidth information is based on a service level associated with the network attached storage device.
4. The system of claim 3, wherein the controller is coupled to the network attached storage device, and the bandwidth information is based on one or more parameters of the network attached storage device.
5. The system of claim 1, further comprising an interconnect switch coupled between the controller and the network attached storage device.
6. The system of claim 1, wherein the controller is coupled to the network attached storage device using the network switch.
7. The system of claim 1, wherein the bandwidth information for the network attached storage device is based on at least one parameter associated with the network attached storage device.
8. The system of claim 7, wherein the controller is coupled to the network attached storage device, and the at least one parameter comprises one or more of a bit error rate or an operating temperature.
9. The system of claim 1, wherein the controller comprises a baseboard management controller.
10. A method comprising:
receiving, at a controller, bandwidth information for a network attached storage device;
storing, in a data structure of the controller, the bandwidth information;
configuring, by the controller, a bandwidth of a network switch for the network attached storage device by programming the network switch based on the bandwidth information, the network switch being coupled to a port of the network attached storage device; and
controlling, based on the bandwidth information, data transfers between the network switch and the network attached storage device;
wherein the controlling is performed, at least in part, by the network switch.
11. The method of claim 10, wherein the bandwidth information for the network attached storage device comprises at least one of an ingress bandwidth or a egress bandwidth.
12. The method of claim 11, wherein the bandwidth information is based on a service level associated with the network attached storage device.
13. The method of claim 12, wherein the bandwidth is based on one or more parameters of the network attached storage device.
14. The method of claim 10, wherein the configuring by the controller the bandwidth of the network switch comprises communicating, using an interconnect switch, a configuration to the network attached storage device.
15. The method of claim 10, wherein the configuring by the controller the bandwidth of the network switch comprises communicating, using a network, a configuration to the network attached storage device.
16. The method of claim 10, wherein the bandwidth for the network attached storage device is based on at least one parameter associated with the network attached storage device.
17. The method of claim 16, wherein the at least one parameter comprises one or more of a bit error rate or an operating temperature associated with the network attached storage device.
18. A method comprising:
receiving, at a controller, bandwidth information for a network attached storage device;
storing, in a data structure of the controller, the bandwidth information;
performing a data transfer between a host and the network attached storage device using a network switch coupled to a port of the network attached storage device; and
configuring the network switch to control, based on the bandwidth information, the data transfer between the host and the network attached storage device;
wherein the network switch is configured, at least in part, by the controller; and
wherein the data transfer is controlled, at least in part, by the network switch.
19. The method of claim 18, wherein the controller:
receives one or more commands; and
configures the network switch based on at least one of the one or more commands.
20. The method of claim 18, wherein the controller receives the bandwidth information from an administrator.
US18/092,432 2017-02-23 2023-01-02 METHOD FOR CONTROLLING BW SLA IN NVMe oF ETHERNET SSD STORAGE SYSTEM Pending US20230133604A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/092,432 US20230133604A1 (en) 2017-02-23 2023-01-02 METHOD FOR CONTROLLING BW SLA IN NVMe oF ETHERNET SSD STORAGE SYSTEM

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201762462901P 2017-02-23 2017-02-23
US15/487,431 US11543967B2 (en) 2017-02-23 2017-04-13 Method for controlling BW SLA in NVME-of ethernet SSD storage systems
US18/092,432 US20230133604A1 (en) 2017-02-23 2023-01-02 METHOD FOR CONTROLLING BW SLA IN NVMe oF ETHERNET SSD STORAGE SYSTEM

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US15/487,431 Continuation US11543967B2 (en) 2017-02-23 2017-04-13 Method for controlling BW SLA in NVME-of ethernet SSD storage systems

Publications (1)

Publication Number Publication Date
US20230133604A1 true US20230133604A1 (en) 2023-05-04

Family

ID=63167182

Family Applications (2)

Application Number Title Priority Date Filing Date
US15/487,431 Active 2037-04-23 US11543967B2 (en) 2017-02-23 2017-04-13 Method for controlling BW SLA in NVME-of ethernet SSD storage systems
US18/092,432 Pending US20230133604A1 (en) 2017-02-23 2023-01-02 METHOD FOR CONTROLLING BW SLA IN NVMe oF ETHERNET SSD STORAGE SYSTEM

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US15/487,431 Active 2037-04-23 US11543967B2 (en) 2017-02-23 2017-04-13 Method for controlling BW SLA in NVME-of ethernet SSD storage systems

Country Status (5)

Country Link
US (2) US11543967B2 (en)
JP (1) JP7108426B2 (en)
KR (1) KR102435004B1 (en)
CN (1) CN108512777B (en)
TW (1) TWI794198B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10459665B2 (en) 2017-04-03 2019-10-29 Samsung Electronics Co., Ltd. System and method of configuring NVMe-oF devices using a baseboard management controller (BMC)
US10733137B2 (en) * 2017-04-25 2020-08-04 Samsung Electronics Co., Ltd. Low latency direct access block storage in NVME-of ethernet SSD
US10275180B2 (en) * 2017-05-08 2019-04-30 Samsung Electronics Co., Ltd. Ethernet SSD system including storage offload engine (SoE) controller and ethernet switch
US10726930B2 (en) * 2017-10-06 2020-07-28 Western Digital Technologies, Inc. Method and system for a storage (SSD) drive-level failure and health prediction leveraging machine learning on internal parametric data
CN109840232B (en) * 2017-11-28 2022-05-13 华为技术有限公司 Storage device and element management method of storage device
US12481612B2 (en) 2018-03-02 2025-11-25 Samsung Electronics Co., Ltd. Mechanism to identify FPGA and SSD pairing in a multi-device environment
US12505053B2 (en) * 2018-03-02 2025-12-23 Samsung Electronics Co., Ltd. Method for supporting erasure code data protection with embedded PCIe switch inside FPGA+SSD
US12524180B2 (en) 2020-06-16 2026-01-13 Samsung Electronics Co., Ltd. Edge solid state drive (SSD) device and edge data system
US11770271B2 (en) 2020-08-21 2023-09-26 Samsung Electronics Co., Ltd. Data center
KR20220084844A (en) 2020-12-14 2022-06-21 삼성전자주식회사 Storage device and operating method thereof

Citations (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5862137A (en) * 1995-07-19 1999-01-19 Fujitsu Network Communications, Inc. Point-to-multipoint arbitration
US20010039582A1 (en) * 2000-05-19 2001-11-08 Mckinnon Martin W. Allocating access across a shared communications medium in a carrier network
US20020001287A1 (en) * 1998-12-31 2002-01-03 Martin Bergenwall Data transmission in a telecommunications system
US20020181397A1 (en) * 1998-01-30 2002-12-05 Robert C. Licht Frame relay network featuring frame relay nodes with controlled oversubscribed bandwidth trunks
US20030079016A1 (en) * 2001-10-23 2003-04-24 Sheng (Ted) Tai Tsao Using NAS appliance to build a non-conventional distributed video server
US6578186B1 (en) * 2000-12-22 2003-06-10 Juniper Networks Reset control for systems using programmable logic
US6594234B1 (en) * 2001-05-31 2003-07-15 Fujitsu Network Communications, Inc. System and method for scheduling traffic for different classes of service
US20050135435A1 (en) * 2003-12-18 2005-06-23 Fujitsu Limited Automatic change method of virtual concatenation bandwidth
US20070217339A1 (en) * 2006-03-16 2007-09-20 Hitachi, Ltd. Cross-layer QoS mechanism for video transmission over wireless LAN
US20080080377A1 (en) * 2006-09-28 2008-04-03 Fujitsu Limited Best-effort bandwidth allocating method and device
US20100118844A1 (en) * 2008-11-12 2010-05-13 At&T Intellectual Property I, Lp Dynamic lightweight remote management of hybrid femtocell gateways
US20110270987A1 (en) * 2010-04-30 2011-11-03 Michael Schlansker Method and system for allocating bandwidth
US8060643B2 (en) * 2002-08-30 2011-11-15 Hewlett-Packard Development Company, L.P. Method and apparatus for dynamically managing bandwidth for clients in a storage area network
US8060630B1 (en) * 2002-11-27 2011-11-15 Symantec Operating Corporation Creating and configuring virtual fabrics in storage area networks
US20120051281A1 (en) * 2009-04-29 2012-03-01 Yu Chen Method, bm-sc and base station for multiplexing mbms services in mbsfn
US20120224481A1 (en) * 2011-03-02 2012-09-06 3Inova Networks Inc. Traffic management in distributed wireless networks
US20130275568A1 (en) * 2012-04-16 2013-10-17 Dell Products, Lp System and Method to Discover Virtual Machine Instantiations and Configure Network Service Level Agreements
US20130336229A1 (en) * 2012-06-13 2013-12-19 Verizon Patent And Licensing Inc. Multistage hierarchical packet scheduling
US20160050470A1 (en) * 2012-02-13 2016-02-18 Ciena Corporation Systems and methods for managing excess optical capacity and margin in optical networks
US20160066070A1 (en) * 2014-08-26 2016-03-03 Terrence Trausch Network aggregation in a computing shelf/tray
US20160080502A1 (en) * 2014-09-16 2016-03-17 CloudGenix, Inc. Methods and systems for controller-based secure session key exchange over unsecured network paths
US20160094450A1 (en) * 2014-09-26 2016-03-31 Dell Products L.P. Reducing internal fabric congestion in leaf-spine switch fabric
US20160127492A1 (en) * 2014-11-04 2016-05-05 Pavilion Data Systems, Inc. Non-volatile memory express over ethernet
US9367298B1 (en) * 2012-03-28 2016-06-14 Juniper Networks, Inc. Batch configuration mode for configuring network devices
US20160227552A1 (en) * 2015-01-30 2016-08-04 Aruba Networks, Inc. Adaptive resource allocation in congested wireless local area network deployment
US20160381693A1 (en) * 2015-06-25 2016-12-29 Japan Communications Inc. Feedback-based packet control system
US20170017585A1 (en) * 2012-08-23 2017-01-19 Dell Products, Lp Fabric Independent PCIE Cluster Manager
US20170139592A1 (en) * 2015-11-13 2017-05-18 Via Alliance Semiconductor Co., Ltd. Chipset and server system using the same
US20180007455A1 (en) * 2016-07-04 2018-01-04 Huawei Technologies Co., Ltd. Method and system for distributed control of large photonic switched networks

Family Cites Families (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3956786B2 (en) 2002-07-09 2007-08-08 株式会社日立製作所 Storage device bandwidth control apparatus, method, and program
JP4214919B2 (en) 2004-01-16 2009-01-28 株式会社日立製作所 Storage switch with bandwidth control function
JP2007328604A (en) 2006-06-08 2007-12-20 Hitachi Ltd Computer system, storage management program, and computer management program
US7940785B2 (en) 2007-01-08 2011-05-10 International Business Machines Corporation Ethernet adapter packet management
US7852867B2 (en) * 2007-07-06 2010-12-14 Integrated Deoice Technology, Inc. Integrated memory for storing egressing packet data, replay data and to-be egressed data
US20100198965A1 (en) 2007-07-13 2010-08-05 Takaharu Tanaka Network control device, image display device and network control method
US7792046B2 (en) * 2008-06-05 2010-09-07 Vss Monitoring, Inc. Ethernet switch-based network monitoring system and methods
US8543685B1 (en) * 2008-08-12 2013-09-24 Eden Rock Communications, Llc Usage based multiplier modification in rate limiting schemes
JP4809413B2 (en) * 2008-10-08 2011-11-09 株式会社日立製作所 Storage system
US8774048B2 (en) 2009-01-29 2014-07-08 Qualcomm Incorporated Link management for multimedia content mobility
US8174969B1 (en) * 2009-11-24 2012-05-08 Integrated Device Technology, Inc Congestion management for a packet switch
US9210048B1 (en) 2011-03-31 2015-12-08 Amazon Technologies, Inc. Clustered dispersion of resource use in shared computing environments
US9300590B2 (en) * 2011-06-24 2016-03-29 Dell Products, Lp System and method for dynamic rate control in Ethernet fabrics
US8972611B2 (en) * 2011-08-11 2015-03-03 Cisco Technology, Inc. Multi-server consolidated input/output (IO) device
US20130311696A1 (en) 2012-05-18 2013-11-21 Lsi Corporation Storage processor for efficient scaling of solid state storage
US9363315B2 (en) * 2012-08-28 2016-06-07 Skyera, Llc Integrated storage and switching for memory systems
US20140195634A1 (en) * 2013-01-10 2014-07-10 Broadcom Corporation System and Method for Multiservice Input/Output
JP6072084B2 (en) * 2013-02-01 2017-02-01 株式会社日立製作所 Virtual computer system and data transfer control method for virtual computer system
US9396755B2 (en) 2013-02-20 2016-07-19 Kabushiki Kaisha Toshiba Temperature-defined data-storage policy for a hybrid disk drive
US20140337598A1 (en) * 2013-05-07 2014-11-13 Lsi Corporation Modulation of flash programming based on host activity
US20150378640A1 (en) * 2013-06-26 2015-12-31 Cnex Labs, Inc. Nvm express controller for remote access of memory and i/o over ethernet-type networks
US10063638B2 (en) 2013-06-26 2018-08-28 Cnex Labs, Inc. NVM express controller for remote access of memory and I/O over ethernet-type networks
KR20150047785A (en) 2013-10-25 2015-05-06 삼성전자주식회사 Server system and storage system
WO2016196766A2 (en) 2015-06-03 2016-12-08 Diamanti, Inc. Enabling use of non-volatile media - express (nvme) over a network
US9619164B2 (en) * 2014-05-06 2017-04-11 Nimble Storage, Inc. Cluster solid state drives
US9477295B2 (en) 2014-05-15 2016-10-25 Dell Products, L.P. Non-volatile memory express (NVMe) device power management
US9436628B2 (en) * 2014-05-30 2016-09-06 Apple Inc. Thermal mitigation using selective I/O throttling
AU2014401914B2 (en) 2014-08-13 2017-05-25 Huawei Technologies Co., Ltd. Storage system, method, and apparatus for processing operation request
US9529619B2 (en) * 2014-09-30 2016-12-27 Nicira, Inc. Method of distributing network policies of virtual machines in a datacenter
US9916087B2 (en) * 2014-10-27 2018-03-13 Sandisk Technologies Llc Method and system for throttling bandwidth based on temperature
KR101607136B1 (en) * 2014-12-31 2016-03-29 청주대학교 산학협력단 Organic light emitting diode device using SiOC thin film and manufacturing method thereof
US9857995B1 (en) * 2015-03-09 2018-01-02 Western Digital Technologies, Inc. Data storage device and method providing non-volatile memory buffer for real-time primary non-volatile memory protection
US10114778B2 (en) 2015-05-08 2018-10-30 Samsung Electronics Co., Ltd. Multi-protocol IO infrastructure for a flexible storage platform
US9933826B2 (en) * 2015-05-11 2018-04-03 Hewlett Packard Enterprise Development Lp Method and apparatus for managing nodal power in a high performance computer system
JP6381480B2 (en) 2015-05-12 2018-08-29 東芝メモリ株式会社 Semiconductor device
US10429909B2 (en) * 2015-06-01 2019-10-01 Hewlett Packard Enterprise Development Lp Managing power in a high performance computing system for resiliency and cooling
US10243791B2 (en) * 2015-08-13 2019-03-26 A10 Networks, Inc. Automated adjustment of subscriber policies
US9658791B2 (en) * 2015-08-14 2017-05-23 International Business Machines Corporation Managing temperature of solid state disk devices
GB2542174B (en) * 2015-09-10 2018-04-04 Openwave Mobility Inc An intermediate network entity for controlling bandwidth for an adaptive bit rate stream
KR20170046097A (en) * 2015-10-20 2017-04-28 (주)켐옵틱스 Low loss transmission type tunable optical filter
US20170195237A1 (en) * 2015-12-30 2017-07-06 Akamai Technologies, Inc. Distributed quality-of-service (QoS) mechanism in an overlay network having edge regions
US10263898B2 (en) * 2016-07-20 2019-04-16 Cisco Technology, Inc. System and method for implementing universal cloud classification (UCC) as a service (UCCaaS)
US10122651B2 (en) * 2016-08-31 2018-11-06 Inspeed Networks, Inc. Dynamic bandwidth control

Patent Citations (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5862137A (en) * 1995-07-19 1999-01-19 Fujitsu Network Communications, Inc. Point-to-multipoint arbitration
US20020181397A1 (en) * 1998-01-30 2002-12-05 Robert C. Licht Frame relay network featuring frame relay nodes with controlled oversubscribed bandwidth trunks
US20020001287A1 (en) * 1998-12-31 2002-01-03 Martin Bergenwall Data transmission in a telecommunications system
US20010039582A1 (en) * 2000-05-19 2001-11-08 Mckinnon Martin W. Allocating access across a shared communications medium in a carrier network
US6578186B1 (en) * 2000-12-22 2003-06-10 Juniper Networks Reset control for systems using programmable logic
US6594234B1 (en) * 2001-05-31 2003-07-15 Fujitsu Network Communications, Inc. System and method for scheduling traffic for different classes of service
US20030079016A1 (en) * 2001-10-23 2003-04-24 Sheng (Ted) Tai Tsao Using NAS appliance to build a non-conventional distributed video server
US8060643B2 (en) * 2002-08-30 2011-11-15 Hewlett-Packard Development Company, L.P. Method and apparatus for dynamically managing bandwidth for clients in a storage area network
US8060630B1 (en) * 2002-11-27 2011-11-15 Symantec Operating Corporation Creating and configuring virtual fabrics in storage area networks
US20050135435A1 (en) * 2003-12-18 2005-06-23 Fujitsu Limited Automatic change method of virtual concatenation bandwidth
US20070217339A1 (en) * 2006-03-16 2007-09-20 Hitachi, Ltd. Cross-layer QoS mechanism for video transmission over wireless LAN
US20080080377A1 (en) * 2006-09-28 2008-04-03 Fujitsu Limited Best-effort bandwidth allocating method and device
US20100118844A1 (en) * 2008-11-12 2010-05-13 At&T Intellectual Property I, Lp Dynamic lightweight remote management of hybrid femtocell gateways
US20120051281A1 (en) * 2009-04-29 2012-03-01 Yu Chen Method, bm-sc and base station for multiplexing mbms services in mbsfn
US20110270987A1 (en) * 2010-04-30 2011-11-03 Michael Schlansker Method and system for allocating bandwidth
US20120224481A1 (en) * 2011-03-02 2012-09-06 3Inova Networks Inc. Traffic management in distributed wireless networks
US20160050470A1 (en) * 2012-02-13 2016-02-18 Ciena Corporation Systems and methods for managing excess optical capacity and margin in optical networks
US9367298B1 (en) * 2012-03-28 2016-06-14 Juniper Networks, Inc. Batch configuration mode for configuring network devices
US20130275568A1 (en) * 2012-04-16 2013-10-17 Dell Products, Lp System and Method to Discover Virtual Machine Instantiations and Configure Network Service Level Agreements
US20130336229A1 (en) * 2012-06-13 2013-12-19 Verizon Patent And Licensing Inc. Multistage hierarchical packet scheduling
US20170017585A1 (en) * 2012-08-23 2017-01-19 Dell Products, Lp Fabric Independent PCIE Cluster Manager
US20160066070A1 (en) * 2014-08-26 2016-03-03 Terrence Trausch Network aggregation in a computing shelf/tray
US20160080502A1 (en) * 2014-09-16 2016-03-17 CloudGenix, Inc. Methods and systems for controller-based secure session key exchange over unsecured network paths
US20160094450A1 (en) * 2014-09-26 2016-03-31 Dell Products L.P. Reducing internal fabric congestion in leaf-spine switch fabric
US20160127492A1 (en) * 2014-11-04 2016-05-05 Pavilion Data Systems, Inc. Non-volatile memory express over ethernet
US20160227552A1 (en) * 2015-01-30 2016-08-04 Aruba Networks, Inc. Adaptive resource allocation in congested wireless local area network deployment
US20160381693A1 (en) * 2015-06-25 2016-12-29 Japan Communications Inc. Feedback-based packet control system
US20170139592A1 (en) * 2015-11-13 2017-05-18 Via Alliance Semiconductor Co., Ltd. Chipset and server system using the same
US20180007455A1 (en) * 2016-07-04 2018-01-04 Huawei Technologies Co., Ltd. Method and system for distributed control of large photonic switched networks

Also Published As

Publication number Publication date
JP7108426B2 (en) 2022-07-28
JP2018137749A (en) 2018-08-30
TW201832536A (en) 2018-09-01
TWI794198B (en) 2023-03-01
US20180239540A1 (en) 2018-08-23
KR102435004B1 (en) 2022-08-22
CN108512777A (en) 2018-09-07
US11543967B2 (en) 2023-01-03
KR20180097433A (en) 2018-08-31
CN108512777B (en) 2023-05-23

Similar Documents

Publication Publication Date Title
US20230133604A1 (en) METHOD FOR CONTROLLING BW SLA IN NVMe oF ETHERNET SSD STORAGE SYSTEM
US10275180B2 (en) Ethernet SSD system including storage offload engine (SoE) controller and ethernet switch
US20220291730A1 (en) Protocol Level Control for System on a Chip (SOC) Agent Reset and Power Management
KR101410292B1 (en) Traffic-load dependent power reduction in high-speed packet switching systems
US6981025B1 (en) Method and apparatus for ensuring scalable mastership during initialization of a system area network
US7843906B1 (en) Storage gateway initiator for fabric-backplane enterprise servers
CN104081718B (en) Network controller for remote system management
US7990994B1 (en) Storage gateway provisioning and configuring
US8868790B2 (en) Processor-memory module performance acceleration in fabric-backplane enterprise servers
US20130094499A1 (en) System and Method for High-Performance, Low-Power Data Center Interconnect Fabric
CN101729262B (en) Configuration method of non-webmaster type switchboard and non-webmaster type switchboard
US7953903B1 (en) Real time detection of changed resources for provisioning and management of fabric-backplane enterprise servers
CN102164044A (en) Networking method and system
US11460908B2 (en) Storage system and method for transmitting signal in storage system
CN1604550B (en) Method and system for maintaining a link between two network entities
US20210336895A1 (en) Data transmission method and network device
CN114745344B (en) Data exchange device and method comprising multiple communication interfaces based on Ethernet
CN103026317B (en) Control the method and system of the power consumption of polymerization I/O port
CN114356830A (en) Bus terminal control method, device, computer equipment and storage medium
US11671318B1 (en) Post-deployment updating of network device management switch configurations
CN103765398A (en) determining at least in part one or more respective quantities of the buffer memory fields
US20150178185A1 (en) Technique for Controlling Memory Accesses
HK1173005B (en) Systems and methods for reconfiguring a network adapter in sleep mode
HK1173005A1 (en) Systems and methods for reconfiguring a network adapter in sleep mode

Legal Events

Date Code Title Description
STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

AS Assignment

Owner name: SAMSUNG ELECTRONICS CO., LTD., KOREA, REPUBLIC OF

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KACHARE, RAMDAS P.;OLARIG, SOMPONG PAUL;PHAM, SON T.;REEL/FRAME:064909/0614

Effective date: 20170412

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION COUNTED, NOT YET MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED