Future Machines: ASCI Purple, ALC and M&IC MCR

Future Machines: ASCI Purple, ALC and M&IC MCR

Planned Machines: ASCI Purple, ALC and M&IC MCR Presented to SOS7 Mark Seager [email protected] 925-423-3141 ICCD ADH for Advanced Technology Lawrence Livermore National Laboratory This work was performed under the auspices of the U.S. Department of Energy by the University of California, Lawrence Livermore National Laboratory under Contract No. W-7405-Eng-48. Q1: What is unique in structure and function of your machine? Purples unique structure is fat SMPs with 16 rails of Federation interconnect MCR+ALCs unique structure is the shared global file system However, most important point is that applications are highly mobile between Purple, MCR+ALC, White, Q and other clusters of SMP systems Purples unique structure is fat SMPs with 16 rails of interconnect Fibre Channel 2 I/O Network I/O I/O I/O I/O I/O I/O I/O

I/O NFS Login NFS Login NFS Login NFS Login Login Net Login Net Login Net Login Net 16 Federation links SMP in four switch planes System Dataper and Control Networks System SystemData Dataand andControl ControlNetworks

Networks 191 Parallel Batch/Interactive/Visualization Nodes Purple System 100 TF/s + 30-45 TF/s delivered on sPPM+UMT2000 50 TB memory, 2.0 PB of disk @ 108 GB/s delivered 197 x 64-way Armada SMP w 16 Federation Links 4 Login/network nodes Login/network nodes for login/NFS 8x10 Gb/s for parallel FTP on each Login All external networking is 1-10 Gb/s Ethernet Clustered I/O services for cluster wide file system Fibre Channel2 I/O attach does not extend Programming/Usage Model Application launch over all compute nodes up to 8,192 tasks 1 MPI task/CPU and Shared Memory, full 64b support Scalable MPI (MPI_allreduce, buffer space) Likely usage multiple MPI tasks/node with 4-16 OpenMP/MPI task Single STDIO interface Parallel I/O to single file, multiple serial I/O (1 file/MPI task) Unique feature of ALC+MCR is Lustre Lite shared file system 1,116 P4 Compute Nodes 1,152 Port (10x96D32U+4x96D32U) QsNet Elan3 QsNet Elan3, 100BaseT Control 2 MetaData (fail-over) Servers 32 Gateway nodes @ 140 MB/s delivered Lustre I/O over 2x1GbE MDS MDS GW GW GW GW

GW GW GW GW 2 Service GbEnet Federated Switch 2 Login nodes with 4 Gb-Enet OST OST OST OST OST OST OST OST OST OST OST OST OST OST OST OST OST OST OST

OST OST OST OST OST Aggregated OST for Single Lustre file system OST OST OST OST OST OST OST OST GbEnet Federated Switch 2 Service MDS MDS GW QsNet Elan3, 100BaseT Control Cluster wide file system leverages DOE/NNSA ASCI PathForward Open Source Lustre development GW GW GW GW 960 Port (10x96D32U+4x80D48U) QsNet Elan3 924 P4 Compute Nodes

GW GW GW Q2: What characterizes your applications? Examples are: Intensities of message passing, memory utilization, computing, IO, and data. Applications characterized as multi-physics package simulations All applications compute/comms intensive Each package pushes performance envelope along a different dimension Some packages are MPI latency dominated Some packages are MPI BW dominated Memory BW is critical factor, but expensive memory subsystems dont perform much better than commodity ones Q3: What prior experience guided you to this choice? Mission and Applications Budgets Politics Delivered performance Balanced risk and cost performance Strategic Approach: straddle multiple curves to balance risk and opportunity of new disruptive technologies Three complementary curves Any given technology curve is ultimately

limited by Moores Law 1. Delivers to todays stockpiles demanding needs Cell-Based (IBM BG/L) IA32/ IA64/AMD + Linux Production environment For must have deliverables now Vendor integrated SMP Cluster (IBM SP, HP SC) Near production environment Provides cycles for science Provides cycles for stockpile Leading to next generation production systems These are the capacity systems in a strategic capacity/capability mix Performance 2. Delivers transition for next generation $7M/TF (Q) $ 500K /TF Mainframes (RIP) $2M/TF (Purple C)

$1.2M/TF (MCR) $10 M/TF (White) 3. Delivers affordable path to petaFLOP/s Research environment, leading transition to petaflop systems? Are there other paths to a breakthrough regime by 2006-7? $170K/TF Time Today FY05 Straddle strategy for stability and preeminence Q4. Other than your own machine, for your needs what are the best and worst machines? And, why? Clusters of SMPs with full node OS makes system administration and programming much easier, but scalability is an issue Vectors suck 10x potential speed-up from vectorization on Cray YMP class machines yielded only 1.5-2x in delivered performance boost to stockpile codes

Recently Viewed Presentations

  • Presentación de PowerPoint - Daniel Turp

    Presentación de PowerPoint - Daniel Turp

    COUR PÉNALE INTERNATIONALE 14 AFFAIRES EN COURS Dans le contexte de 7 situations ont été ouvertes devant la Cour. 1- Concernant les pays parties au Statut de Rome : - La situation en Ouganda : 1 affaire - La situation...
  • Discharge Note - WSU MED

    Discharge Note - WSU MED

    DISCHARGE NOTE. Brief note . indicating the disposition of the patient and the reason for conversion to full admission. TO AVOID NEEDING TO CLICK THROUGH WHOLE TEMPLATE: After opening up Discharge Note template, when asked if note is Deceased vs...
  • UNIT-IV Digital Subscriber Access ISDN  The telephone network

    UNIT-IV Digital Subscriber Access ISDN The telephone network

    ISDN. The Public Switched Telephone network is still analogue from the subscriber to the local exchange. The need has arisen to extend the digital network out to subscribers and to provide a single standardised interface to all different users of...
  • AP REVIEW UNITS I - VI UNIT 1

    AP REVIEW UNITS I - VI UNIT 1

    The Structure and Powers of Congress Delegated Powers to the National Government (Expressed, Implied, Inherent) Expressed/enumerated: actually stated in the Constitution (Article I, Section 8) Levy taxes (revenue bills must originate in the House) Coin money, Borrow money, Spend money...
  • OAM Configuration Framework and Technology Specific Extensions draft-ietf-ccamp-oam-configuration-fwk

    OAM Configuration Framework and Technology Specific Extensions draft-ietf-ccamp-oam-configuration-fwk

    CCAMP WG documents Individual drafts Framework (1) Ethernet OAM (2) MPLS-TP OAM (4) SDH and OTN OAM (3) End-to-end OAM Tandem Connection Monitoring Entities Non-intrusive Monitoring Entities LSP Attributes OAM Configuration TLV Technology specific OAM Configuration TLVs (Ethernet / SDH...
  • Facility Layout - Bradley University

    Facility Layout - Bradley University

    Facility Layout Ross L. Fink Facility Layout Types Classic Process (functional, departmental) Product (assembly line) Fixed Position Cellular Process Layout Product Layout Advantages of Process Layout More product variety Less investment in equipment Can share equipment across products Redundant equipment...
  • Title

    Title

    The student transcript should reflect the high school course, core or elective, the concurrently enrolled course is replacing. The instructional code within the student information system should be coded "college level." It is recommended that the local district distinguishes the...
  • Authentic Learning: Real-world Project Based Learning Paul Herring

    Authentic Learning: Real-world Project Based Learning Paul Herring

    This includes providing detailed design calculations to assess the proposed structure for its performance under normal service loads and under one selected extreme load (e.g. traffic, winds, surge, earthquakes, and tsunamis). The design is developed throughout the semester with students...