MSST Conference

Sponsored by the IEEE Mass Storage Systems
Technical Committee

April 16 — 20, 2012
Asilomar Conference Grounds
Pacific Grove, CA

MSST 2012 Speaker

Andreas Dilger, Whamcloud

Tutorial Abstract: Lustre Metadata Scaling

Lustre is well known for providing the largest and fastest storage for the majority of large HPC systems in the world. As the number of cores and threads on these systems steadily increases, so does the demand for fast metadata performance. While metadata performance has not traditionally been Lustre's strongest area, this is being addressed in upcoming releases.

An overview of the structure of the Lustre storage cluster will be presented. This will lead into an overview of recent improvements to Lustre metadata performance and how this affects users and applications running on Lustre:

parallel directory operations allow fine-grained locking of a single directory to allow concurrent modification of a shared directory
statahead to allow a single client to efficiently fetch attributes from files in a directory for common workloads like "ls -l" and "find"
Distributed Namespace allows multiple servers for storing Lustre metadata, increasing scalability and providing dedicated resources when multiple jobs are running on the same cluster

MSST Abstract: Lustre Future Development

The Orion development project, nearing completion, allows Lustre to run atop ZFS backend filesystems. This code will be deployed for one of the largest (50PB) and fastest (500-1000 GB/s) filesystems in the world. An overview of the project will be given.

Also, the announcement two years ago about changes to the Lustre development and support model caused some confusion and turmoil in the user community. Experience with the new model, with the changes now seen as positive, will be discussed.

Working with HPC systems since 1995, first at IBM, Andreas was one of the original developers of the Lustre filesystem, working on the prototype in 1999. He is currently the Principal Engineer for Lustre at Whamcloud. Since 1997 he has been an active ext2/3/4 developer, and is currently one of the maintainers of the ext4 filesystem. He is very active in the Lustre and Linux communities. BS in Electrical and Computer Engineering at the University of Calgary, 1991.

Page Updated January 12, 2024