NSF CCF0938114

Visual Characterization of I/O System Behavior for High-End Computing


Principal Investigator: Kwan-Liu Ma, University of California at Davis
Co-Investigators: Pete Beckman and Kamil Iskra, University of Chicago

Period: September 15, 2009 - September 14, 2012

Project Summary

Modern supercomputers are complex, hierarchical systems consisting of huge numbers of cores, systems for disk storage, and nodes for I/O forwarding. These numbers continue to grow and the need for tools to understand the behavior of the system software becomes paramount: without these tools it is impossible to effectively tune that software, and high degrees of efficiency is unattainable by applications. This project addresses the challenge of understanding the behavior of complex system software on very large-scale compute platforms, like the current petascale computers. In particular, this project is developing software infrastructure to provide end-to-end analysis and visualization of I/O system software. Specifically, the objectives are to develop, improve, and deploy (1) end-to-end, scalable tracing integrated into the I/O system (MPI-IO, I/O forwarding, and file system); (2) information visualization tools for inspecting traces and extracting knowledge; (3) testing components that drive this system to generate example patterns, including a component to generate anomalies; and (4) tutorials and tools for helping other system software developers incorporate this analysis and visualization system into their production software. The software and techniques developed in this project will be directly applicable to and useful in other system software libraries which perform complex interactions on large systems.

NSF Award CCF-0938114

Other Participants

  • Jason Cope, Postdoc, University of Chicago and Argonne National Laboratory
  • Chris Muelder, PhD student
  • Carmen Sigovan, PhD student
  • Jeff Sukharev, PhD student
  • Sam Lang (Collaborator), Argonne National Laboratory
  • Robert Ross (Collaborator), Argonne National Laboratory
  • David Skinner (Collaborator), Lawrence Berkelely National Laboratory
  • Andrew Uselton (Collaborator), Lawrence Berkelely National Laboratory

Publications

  1. Chris W. Muelder et al. Visual Analysis of I/O System Behavior for High End Computing, in Proceedings of HDPC 2011 Workshop on Large-Scale System and Application Performance (LSAP). [pdf]
  2. C. Jones, R. Armstrong, and K.-L. Ma. Visualizing the Commonalities between Hierarchically Structured Data Queries, in Proceedings of VLC (Visual Languages and Computing) 2010. [pdf]
  3. Chris W. Muelder, Fracois Gygi, and Kwan-Liu Ma. Visual Analysis of Inter-Process Communication for Large-Scale Parallel Computing, IEEE Transaction on Visualization and Computer Graphics, Volume 15, Number 6, November/December 2009, pp. 1129-1136. [pdf]

Acknowledgments

The materials presented at this website are based upon work supported by the National Science Foundation under Grant No. 0938114. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the PI and do not necessarily reflect the views of the National Science Foundation.