MARCC: Update FY18 Jaime E. Combariza 2018 IDIES Annual Symposium Associate Research Professor Department of Chemistry Director MARCC Johns Hopkins University
1 Configuration 2018 23,352 cores (growing) 900 nodes 1.5PFLOPs 60 condo-nodes 1680 condo-cores (60 nodes) 3
Metrics FY18 Research Groups (active): User Accounts (active): CPU-cycles used: Utilization Publications Classes/Workshops Scientific Applications 220+ 900
150M core-hours 90% > 400 15 >300 4 Utilization FY18 5
Utilization per Department 6 Impact on Research Tamer Zakis research group (Mechanical Engineering) In the study of turbulent flows, access to large-scale computing facilities is indispensable for research progress, and without MARCC this strand of research would atrophy and die at JHU. With MARCC, we are delivering some of the highest quality simulations of laminar-to-turbulence transition and of turbulent flows within the community. We span Newtonian, viscoelastic and particle-laden fluids, and flow regimes from low-speed viscous flows to hypersonic flight. One particular example is related to our high-speed simulations (flight Mach numbers equal to 4.5). At these speeds, transition from laminar to turbulent flow can be catastrophic for flight vehicles. Using MARCC for high-fidelity
simulations of the flow, we were able to establish strict nonlinear bounds on transition location. The work was recently reviewed by a panel of more than 30 members from DoD, DoE and industry and was highlighted as one of the most exciting new initiatives. The same work is also the foundation to a budding collaboration with APL. Tim Mueller (Material Science Engineering):MARCC has been a tremendous benefit to my research program, as our work is computationally intensive. Having access to MARCC has allowed us to explore fruitful new directions in our research which we would have been otherwise unable to pursue because of resource constraints. Johan Larssons research group (Mech. Eng. UMCP): Our research on hypersonic turbulent flows is performed entirely through large-scale numerical simulations, which require tens of thousands of processor-hours to complete. MARCC and BlueCrab gives us the ability to perform this research at a state-of-the-art level. As a direct consequence, the access to MARCC has helped secure two federal research grants totaling $900K in the last year. Dan Arking (SOM): The investigation of mitochondrial DNA copy number (mtDNA-CN) and its association with gene expression. The large amount of
available storage space on MARCC allowed for the direct download of gene expression and sequencing data from the Genotype-Tissue Expression (GTEx) database; data cleaning and analysis were then completed using unix and R interactively on MARCC. Additionally, MARCC is also used to for a development project to improve the current pipeline for calling mtDNA-CN and heteroplasmy from sequencing data. Alignment of sequence reads to the circular mitochondrial genome currently treats the mtDNA as a linear piece of DNA resulting in a marked coverage drop off at both 'ends' of the genome. We are optimizing a pipeline to correct this oversight so coverage remains uniform across the mitochondrial genome. We areal also working on generating a de-novo assembly pipeline to improve haplogroup detection on the individual level. This project is a collaboration with other Hopkins PIs; the ease of accessing and sharing data via MARCC has significantly improved cooperation between the different research groups. 7 Impact on Research Rebekka klausens research group: From the Stone Age to the Iron Age, eras of human achievement are named after the
materials that make technological breakthroughs possible. Research in my group is motivated by the conviction that discovering the transformative materials and technologies of the future will depend on discovering new materials with new properties. We design and synthesize unconventional molecular building blocks for the construction of diverse polymeric architectures. Because most of our research deals with the preparation of chemicals that have never been studied before, we invest significant research effort into understanding chemical structure. A major focus is chemical connectivity: how are the atoms that make up this structure connected? How do small changes in connectivity change the expected properties of the material? MARCC resources are invaluable in this pursuit. The calculations my students perform using MARCC have aided in the structure determination of large rings of silicon atoms (240 atoms). The computational power provided by MARCC allows for my calculations to be completed in a timely manner. The results I have obtained have been vital parts of published articles. Additionally, the use of MARCC resources led to the exciting prediction that
these large rings can tightly bind lithium ions, an insight very relevant to battery research. These predictions inform current research directions in the Klausen lab focused on experimental quantification of lithium ion binding. As not only a researcher but also an educator, the substantial documentation and education outreach is invaluable for training researchers at any level with no prior experience in computational work. MARCCs timely maintenance also makes it an exceptionally reliable resource. Because of the lowered learning curve and great computational capabilities, graduate students in my group have been able to use MARCC as a teaching tool for undergraduate researchers. Together, undergraduate and graduate students have created extensive data sets to inform our experimental hypotheses. 8
What is new Containers Power backup Coldfront for account management Globus+ features Integration with SciServer 9 Containers Singularity:
Mobility and reproducibility Enables the user to have full control of the environment. No root access needed, on MARCC User can build an image on his/her local machine, upload it to MARCC and then import it using singularity Build the image based on a particular pipeline, OS, data, local scientific software Compatible with docker (can import docker containers) 1 0
What is new Containers Power backup Coldfront for account management Globus+ features Integration with SciServer 11 Power backup Done
Colocation area plus critical servers (File systems, login nodes, main switches) are now on power backup More reliability Can house more critical systems 12 What is new Containers
Power backup Coldfront for account management Globus+ features Integration with SciServer 13 ColdFront (coldfront.CCR.Buffalo.edu) Allocation management tool MARCC Approve allocations based on PIs completing tasks
Collect external funding and Publications Non-funded research Generate reports Systems Administration Manage accounts, cleanup PIs Add/Delete users Approve accounts Easy report grants/project, publications 14
What is new Containers Power backup Coldfront for account management Globus+ features Integration with SciServer 15 Globus+ Features Sharing data with others at JHU or
external users Publishing data creating APIs 16 Globus+ Features 17 What is new
Containers Power backup Coldfront for account management Globus+ features Integration with SciServer 18 SciServer/MARCC Data analytics and traditional HPC Mount File Systems at SciServer (sshfs)
Add FS to container Access data through the network Submit jobs to MARCC 19 Return on Investment (ROI) ROI: MARCC has allowed many investigators to move their research forward Researchers can now do the research they really want as opposed to the research they
can over 120M in grants (low report) 20 Publications https://www.marcc.jhu.edu/newsand-events/publications/ Publications (FY18) that used MARCC for computing 21
Sustainability 3.5 years of freecomputing MARCC is getting old and needs refresh Better technology, faster computing, new computing We ALL have to contribute, grants (MRI), condos, schools, departments, research groups Part of the impact: Number of cores, low waiting times Newer technology Fast I/O (storage) and connectivity
22 Thanks [email protected] http://marcc.jhu.edu 23
