User Tools

Site Tools


limulus_compute_blades

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
limulus_compute_blades [2021/05/04 14:53]
brandonm [History] Hyphenation and word fixes
limulus_compute_blades [2021/05/04 15:23] (current)
brandonm [Conclusion] Word and punctuation fixes
Line 25: Line 25:
 The following are some of the important capabilities of the Limulus 2.0 μATX blade system. The following are some of the important capabilities of the Limulus 2.0 μATX blade system.
  
-  * Support for all standard uATX commodity motherboards +  * Support for all standard μATX commodity motherboards 
-  * Single socket supporting up to 80-Watt (W) Thermal Power Design (TDP) processors +  * Single socket supporting up to 80-watt (W) Thermal Power Design (TDP) processors 
   * Up to 128 Gigabytes (GB) of DDR4 memory with Error Correction Code (ECC) option   * Up to 128 Gigabytes (GB) of DDR4 memory with Error Correction Code (ECC) option
   * One or two onboard M.2 NVMe drives (depends on motherboard)   * One or two onboard M.2 NVMe drives (depends on motherboard)
Line 42: Line 42:
  
 ====Design Details==== ====Design Details====
-All Limulus systems are based on a balanced design that incorporates low-cost commodity hardware and high performance.  Consult the [[Functional Diagram|Functional Diagram]] for design details. The design rational for the various blade components is as follows:+All Limulus systems are based on a balanced design that incorporates low-cost commodity hardware and high performance.  Consult the [[Functional Diagram|Functional Diagram]] for design details. The design rationale for the various blade components is as follows:
  
 ===Edge Computing=== ===Edge Computing===
-Since their inception, Limulus systems were designed to operate in a local or "non-datacenter" environment. As mentioned, this can be an office, lab, classroom, or factory. In general, this type of computing is now categorized as "Edge Computing" where processing occurs at the edge of the data center or cloud. The Limulus μATX blade is designed with the flexibility needed for this environment. Virtually all aspects of the computing node (processor, cores, memory, network, and power) can be configured to match Edge computing needs. +Since their inception, Limulus systems were designed to operate in a local or "non-datacenter" environment. As mentioned, this can be an office, lab, classroom, or factory. In general, this type of computing is now categorized as "Edge Computing," where processing occurs at the edge of the data center or cloud. The Limulus μATX blade is designed with the flexibility needed for this environment. Virtually all aspects of the computing node (processor, cores, memory, network, and power) can be configured to match Edge computing needs. 
  
 ===μATX Motherboards=== ===μATX Motherboards===
-The use of μATX represents a good compromise between size and motherboard capabilities. There are other smaller motherboard form factors (e.g. mini IDX) that require less space; however, these often do not provide as much flexibility in terms of processor choice, processor cooling options, supporting chip-sets, amounts of memory, and PCIe slots. Larger, full ATX motherboardsprimarily have more PCIe slots, but almost all the features found on these larger motherboards can be found on an appropriate μATX system. In addition, since the blades rely on motherboard "bus benders" to turn one PCIe socket by 90 degrees, additional PCIe ports available on full ATX systems would not be usable due to space considerations.+The use of μATX represents a good compromise between size and motherboard capabilities. There are othersmaller motherboard form factors (e.g. mini IDX) that require less space; however, these often do not provide as much flexibility in terms of processor choice, processor cooling options, supported chip-sets, amounts of memory, and PCIe slots. Larger, full ATX motherboards primarily have more PCIe slots, but almost all the features found on these larger motherboards can be found on an appropriate μATX system. In addition, since the blades rely on motherboard "bus benders" to turn one PCIe socket by 90 degrees, additional PCIe ports available on full ATX systems would not be usable due to space considerations.
  
 ===Use Low Core Count – High Clock Processors=== ===Use Low Core Count – High Clock Processors===
-The trend in processor design is to increase the number of cores on the processors die. While this may be advantageous for some workloads, the increased core count requires lower clock speeds allowing processors to fit into an air-cooled TDP envelope (above 275 W TDP air cooling gets difficult).  For instance, a current generation 32-core AMD EPIC has a base clock speed of 2.5 GHz and TDP of 180W.  +The trend in processor design is to increase the number of cores on the processorsdie. While this may be advantageous for some workloads, the increased core count requires lower clock speeds to fit processors into an air-cooled TDP envelope (TDP air cooling gets difficult above 275 W).  For instance, a current-generation 32-core AMD EPIC has a base clock speed of 2.5 GHz and TDP of 180 W.  
  
 Limulus μATX blades take a different approach. Using lower core count processors with higher clock speeds (in a clustered configuration) provides a less costly and higher performance solution. As mentioned above, the current generation of AMD Ryzen processors offers eight physical cores running at a base speed of 3.6 GHz with a total TDP of 65 W. Lowering the core count allows the clock speed to increase by 44% over the slower EPIC processor. (Note: Base clock speed is used for comparisons because full or nearly full utilization of the blade will prohibit turbo or boost clock increases.)  Limulus μATX blades take a different approach. Using lower core count processors with higher clock speeds (in a clustered configuration) provides a less costly and higher performance solution. As mentioned above, the current generation of AMD Ryzen processors offers eight physical cores running at a base speed of 3.6 GHz with a total TDP of 65 W. Lowering the core count allows the clock speed to increase by 44% over the slower EPIC processor. (Note: Base clock speed is used for comparisons because full or nearly full utilization of the blade will prohibit turbo or boost clock increases.) 
  
 ===Use Low Power Processors === ===Use Low Power Processors ===
-Another advantage of low core count high clock processors is the lower TDP. For example, both Intel and AMD manufacture processors that fit within a 65-80W TDP. Lower TDP means less heat and more relaxed cooling requirements (slower/quieter fans). In addition, less heat generation does not require data center cooling (e.g. hot isle/cold isle) and allows Limulus μATX blades to be used almost anywhere. +Another advantage of low core count high clock processors is the lower TDP. For example, both Intel and AMD manufacture processors that fit within a 65-80 W TDP. Lower TDP means less heat and more relaxed cooling requirements (slower/quieter fans). In addition, less heat generation does not require data center cooling (e.g. hot aisle/cold aisle) and allows Limulus μATX blades to be used almost anywhere. 
  
 In a 4-blade cluster environment, the cumulative TDP can reach 260 W; however, this heat is distributed, rather than emanating from a single location (i.e. using air cooling is more efficient to move four smaller amounts of heat vs. a large centrally-located heat source).  In a 4-blade cluster environment, the cumulative TDP can reach 260 W; however, this heat is distributed, rather than emanating from a single location (i.e. using air cooling is more efficient to move four smaller amounts of heat vs. a large centrally-located heat source). 
  
 ===CPU Only=== ===CPU Only===
-The size and compactness of the Limulus μATX blade excludes the use of GPU acceleration. In addition, the thermal constraints described above do not allow for large 200-300W GPUs to be placed on the blades.  The power requirements and heat generated by using multiple GPUs can limit where systems can be used.  +The size and compactness of the Limulus μATX blade excludes the use of GPU acceleration. In addition, the thermal constraints described above do not allow for large 200-300 W GPUs to be placed on the blades.  The power requirements and heat generated by using multiple GPUs can limit where systems can be used.  
  
 ===On-Board Video=== ===On-Board Video===
Line 78: Line 78:
  
 ===Networking and PCIe Slots=== ===Networking and PCIe Slots===
-Blades offer several networking options. First, virtually all μATX motherboards provide at least one 1 GbE port. There are other motherboards that provide two 1 GbE ports or one 10 GbE ports. These ports are almost always "Base-T" connectors (plastic "click in" type) using CAT 6 or 6A cabling. In addition to the motherboard ports, Limulus μATX blades can bring one of PCIe slots out to the font of the blade. This slot is normally used for a 10 GbE adaptor (either SFP+ or Base-T), although many types of smaller single slot PCIe cards can be used in this position. The 3D-Printed front of the blade can be modified to accept different PCIe cards with different physical port locations and types.+Blades offer several networking options. First, virtually all μATX motherboards provide at least one 1 GbE port. There are other motherboards that provide two 1 GbE ports or one 10 GbE port. These ports are almost always "Base-T" connectors (plastic "click in" type) using CAT 6 or 6A cabling. In addition to the motherboard ports, Limulus μATX blades can bring one of the PCIe slots out to the front of the blade. This slot is normally used for a 10 GbE adaptor (either SFP+ or Base-T), although many types of smaller single slot PCIe cards can be used in this position. The 3D-printed front of the blade can be modified to accept different PCIe cards with different physical port locations and types.
  
 ====Physical Configuration/RAS ==== ====Physical Configuration/RAS ====
-Blades are easily inserted (or removed) from the standard Limulus case. Power is applied through a blind 12V power connector in the rear of the blade. All other connections are brought to the front of the blade (see Figure Three). These connections include all motherboard IO connectors (e.g. USB ports, processor video (if available), keyboard, mouse, and 1 GbE network), power button and indicator light, two SATA (as eSATA connectors), and a 10 GbE network connector.+Blades are easily inserted (or removed) from the standard Limulus case. Power is applied through a blind 12V power connector in the rear of the blade. All other connections are brought to the front of the blade (see Figure Three). These connections include all motherboard IO connectors (e.g. USB ports, processor video (if available), keyboard, mouse, and 1 GbE network), power button and indicator light, two SATA connectors (as eSATA connectors), and a 10 GbE network connector.
  
 ===Blade Removal/Insertion=== ===Blade Removal/Insertion===
-On Limulus systems, blade removal begins by using the system control GUI (or command line tools) to remove power to the blade. Although blades are hot-pluggable, prior power removal ensures best results. Next the front of case is removed to expose the blades (no tools needed). Once blades are exposed, all networks and data storage cables are easily removed. Finally, the thumb screws are loosened and used to pull the blade out. To replace the blade, the steps are performed in reverse. +On Limulus systems, blade removal begins by using the system control GUI (or command line tools) to remove power to the blade. Although blades are hot-pluggable, prior power removal ensures best results. Next the front of the case is removed to expose the blades (no tools needed). Once the blades are exposed, all network and data storage cables are easily removed. Finally, the thumb screws are loosened and used to pull the blade out. To replace the blade, the steps are performed in reverse. 
  
 {{ :wiki:figure-4-node-power_ctl.png?400 |}} {{ :wiki:figure-4-node-power_ctl.png?400 |}}
Line 91: Line 91:
  
 ===Blade Power=== ===Blade Power===
-As mentioned, power is supplied though a single blind 12V (DC) power connector. Using an adaptor, this power rail is converted (efficient DC to DC) to the appropriate voltages required by the motherboard (see Figure Five). Also see [[Powering Up/Down Nodes|Powering Up/Down Nodes]] and [[Node Power Control|Node Power Control]]+As mentioned, power is supplied though a single blind 12V (DC) power connector. Using an adaptor, this power rail is converted (efficient DC to DC) to the appropriate voltages required by the motherboard (see Figure Five). Also see [[Powering Up/Down Nodes|Powering Up/Down Nodes]] and [[Node Power Control|Node Power Control]].
  
-===Video, Keyboard, Mice, USB Connection=== +===Video, Keyboard, Mouseand USB Connections=== 
-All motherboard ports are available on the front of the blade. In general, access to these ports is normally not needed when the system is operating a unified cluster. If these ports are needed and if the blade processor provides an onboard video option (mainly Intel processors), then these ports can be used for debugging (or updating BIOS).  All blade ports are easily accessible by removing the front panel (no tools needed).+All motherboard ports are available on the front of the blade. In general, access to these ports is normally not needed when the system is operating as a unified cluster. If these ports are needed and if the blade processor provides an onboard video option (mainly Intel processors), then these ports can be used for debugging (or updating BIOS).  All blade ports are easily accessible by removing the front panel (no tools needed).
  
 ===Cabling=== ===Cabling===
Line 100: Line 100:
  
 ===Blade Size=== ===Blade Size===
-Blades measure 11" x 11.3" x 1.9" (280mm x 287mm x 49mm) as shown in Figure Five.+Blades measure 11" x 11.3" x 1.9" (280mm x 287mm x 49mm)as shown in Figure Five.
  
 {{ :wiki:figure-5-blade-top-annotated-size.png?400 |}} {{ :wiki:figure-5-blade-top-annotated-size.png?400 |}}
-**Figure Five:** //μATX blade dimensions with front and top view. Note power connector is in lower right corner.// +**Figure Five:** //μATX blade dimensions with front and top view. Notepower connector is in lower right corner.// 
  
 ===Blade Enclosures=== ===Blade Enclosures===
Line 112: Line 112:
   -A mate for the power connector   -A mate for the power connector
  
-The following CAD drawing is for the enclosure that mounts in the standard Limulus case. Additional enclosures designs allow blades to be mounted in rack-mount cases. +The following CAD drawing is for the enclosure that mounts in the standard Limulus case. Additional enclosure designs allow blades to be mounted in rack-mount cases. 
  
 {{ :wiki:figure-6-limulus-blade-cage-front-side.png?400 |}} {{ :wiki:figure-6-limulus-blade-cage-front-side.png?400 |}}
Line 119: Line 119:
  
 ===Reliability, Availability and Serviceability (RAS)=== ===Reliability, Availability and Serviceability (RAS)===
-As described above, blades are easily removed by users for return to depot service. Because all blades are powered independently, any blade can be removed and reinserted without affecting other operating (powered) blades. In addition, if on-site component repair is an option, the open design of the blade allows for fast and easy repairs using no more than a Phillips screwdriver. +As described above, blades are easily removed by users for return-to-depot service. Because all blades are powered independently, any blade can be removed and reinserted without affecting other operating (powered) blades. In addition, if on-site component repair is an option, the open design of the blade allows for fast and easy repairs using no more than a Phillips screwdriver. 
  
 ==== 3-D Printed Flexibility==== ==== 3-D Printed Flexibility====
-The use of 3-D printing helps lower construction cost and provides a new level of system flexibility.  Costs are lower because there is no need to buy sheet metal or injection molded parts in high quantity. Since 3-D printed parts are printed "on demand," design changes and customized solutions are easily managed. In addition, 3-D printing can be used to create sophisticated shapes that might otherwise be expensive, difficult, or impossible to create with other manufacturing methodologies. +The use of 3-D printing helps lower construction cost and provides a new level of system flexibility.  Costs are lower because there is no need to buy sheet metal or injection-molded parts in high quantity. Since 3-D printed parts are printed "on demand," design changes and customized solutions are easily managed. In addition, 3-D printing can be used to create sophisticated shapes that might otherwise be expensive, difficult, or impossible to create with other manufacturing methodologies. 
  
 ====Environmental==== ====Environmental====
-Limulus systems are designed to work in non-data center environments (Edge Computing). The Limulus 2.0 μATX blades are designed to run quietly (equipped with efficient CPU coolers and fans) and produce low amounts of heat due to the use of lower TDP processors. +Limulus systems are designed to work in non-data center environments (Edge Computing). The Limulus 2.0 μATX blades are designed to run quietly (equipped with efficient CPU coolers and fans) and produce low amounts of heat due to the use of lower-TDP processors. 
  
 Blades are designed to be used in close quarters in ventilated, yet rugged, environments. Other than the processor cooler, there are no moving parts. All non-blade storage is external to the blade, allowing multiple removable storage devices to be used with each blade.  All blades have at least one internal SSD NVMe slot. Blades are designed to be used in close quarters in ventilated, yet rugged, environments. Other than the processor cooler, there are no moving parts. All non-blade storage is external to the blade, allowing multiple removable storage devices to be used with each blade.  All blades have at least one internal SSD NVMe slot.
  
-A central issue for all computing equipment is the accumulation of dust within the components (due to the need for moving air to cool hot components). Because the blades are designed to use cooler processors, the amount of cooling (moving air) is less than hotter processors. Thus, using a case with proper air filters (such as those used for Limulus systems) blades normally do not experience large amounts of dust accumulation. In addition, should dust find its way to a blade, it is easily removed and cleaned using compressed air. +A central issue for all computing equipment is the accumulation of dust within the components (due to the need for moving air to cool hot components). Because the blades are designed to use cooler processors, the amount of cooling (moving air) is less than for hotter processors. Thus, using a case with proper air filters (such as those used for Limulus systems)blades normally do not experience large amounts of dust accumulation. In addition, should dust find its way to a blade, it is easily removed and cleaned using compressed air. 
  
-Concerns about 3-D printed parts in a "hot" computing environment are not an issue. First, systems are designed to run cool and have plenty of moving air and case ventilation to remove heat. Second, during normal operation, the hottest parts of the system are the CPUs and active coolers. These devices are not near or in contact with any 3-D printed parts. Modern processors can reach temperatures of 70-85° C before they internally throttle the clock (thusreducing the amount of heat generated). Limulus nodes are also monitored and have a default throttling temperature (72°C) that will initiate kernel level throttling, if needed. Finally, all parts are printed (and annealed) using high performance PLA. The resulting parts start to become soft between 75-85°C. It would take some work to seal off the case so that the internal temperature reaches this range. In addition, at these temperatures, many  other things would not work. +Concerns about 3-D printed parts in a "hot" computing environment are not an issue. First, systems are designed to run cool and have plenty of moving air and case ventilation to remove heat. Second, during normal operation, the hottest parts of the system are the CPUs and active coolers. These devices are not near or in contact with any 3-D printed parts. Modern processors can reach temperatures of 70-85 °C before they internally throttle the clock (thus reducing the amount of heat generated). Limulus nodes are also monitored and have a default throttling temperature (72 °C) that will initiate kernel-level throttling, if needed. Finally, all parts are printed (and annealed) using high performance PLA. The resulting parts start to become soft between 75-85 °C. It would take some work to seal off the case so that the internal temperature reaches this range. In addition, at these temperatures, many  other things would not work. 
  
 ====Conclusion==== ====Conclusion====
-The Limulus 2.0 μATX blade design provides convenient "big compute" to non-data center environments. Optimal design and the use of 3-D printing provides a powerful yet flexible approach to local computing solutions (e.g. office, lab, classroom, or factory). In addition, the use of high-quality commodity priced components allows industry leading price to performance. The modular design allows virtually any standard μATX motherboard and off-the-shelf hardware to be combined into a dense high-performance computing resource.+The Limulus 2.0 μATX blade design provides convenient "big computing" to non-data center environments. Optimal design and the use of 3-D printing provides a powerful yet flexible approach to local computing solutions (e.g. for the office, lab, classroom, or factory). In addition, the use of high-quality commodity-priced components allows industry-leading price to performance. The modular design allows virtually any standard μATX motherboard and off-the-shelf hardware to be combined into a densehigh-performance computing resource.
  
limulus_compute_blades.1620139987.txt.gz · Last modified: 2021/05/04 14:53 by brandonm