With MarFS in production at LANL since fall 2016, we have gained new insights, learned lessons, and expanded our future plans. We'll discuss the various hurdles required to deploy such an ambitious system with minimal manpower. Further, we'll delve into the challenges, triumphs, and defeats on the road to a new tier of inexpensive scalable storage.

Economic considerations and technology developments are necessitating widely usable tiered storage. Untroubled by the worries of transparency and performance, Campaign Storage—invented at Los Alamos National Laboratory—offers radical revisions of old workflows and adapts to new technologies. But it also leverages widely available technologies and interfaces to offer stability from the ground up and blend in with the past. We'll discuss how a simple combination of components can support scalability, data analytics and efficient integration with memory based storage.

For the last four years, Backblaze has collected and reported on the failure rates and SMART stats of the hard drives in use in our data centers. Currently we have over 80,000 drives ranging from 3 to 8TB. Let's take a look at what we learned over the years about hard drives, including failure rates, by model, and the ability to predict drive failure before it happens.

On August 12, 2014, Oracle announced the SPARC M7 processor, highlighting an architecture advancement the company refers to as "Software in Silicon". Putting software onto the computer chip (Software in Silicon) is a revolutionary new technology which fundamentally changes the way computer systems are built in the future. Oracle processor engineers worked closely with Oracle's software developers, in particular its database experts, in hard-wiring capabilities specific to application processes and performance onto the processor—hence the "Software in Silicon" moniker.

As distributed storage adds advanced features such as erasure coding, dedup, compression and encryption, the computational requirements can limit performance. ISA-L is an optimized software library for storage algorithms intended to maximize efficiency by targeting the highest cycle-per-byte functions in modern storage systems.

Library optimizations as well as development of new erasure coding algorithms have been keys to unlocking higher levels of erasure coding performance. Learn about performance improvements achieved on ARM and x86 processors.

SQL is the language for data in business and many science applications today. In this talk, we will outline Oracle’s Exadata design and architecture and how it achieves very high performance, capacity, and resilience in production.

NoSQL databases make some strong assertions about reasons for deviating from the conventional relational model. In this talk Aaron Cordova will present the design decisions and trade-offs made in building Apache Accumulo, a highly scalable open source implementation of Google's BigTable and will discuss how these decisions have enabled Accumulo to achieve extreme scalability in several dimensions.

Discover how to easily design and manage your large archives for the long term. This short talk will highlight the key components and features that are essential for a secure, future-proof long-lived archive. Understand how to leverage and scale the optimal mix of storage, ranging from flash to disk to tape to cloud. Get tips on how to architect an environment that provides you easy access to and infinite retention for your data, no matter where it is stored or how old.

Data growth and data analytics requirements are outpacing the compute and storage technologies that have provided the foundation of processor-driven architectures for the last five decades. This divergence requires a deep rethinking of how we build systems, and points towards a memory-driven architecture, where memory is the key resource and everything else, including processing, revolves around it.

Memory-driven computing (MDC) brings together byte-addressable persistent memory, a fast memory fabric, task-specific processing, and a new software stack to address these data growth and analysis challenges. At Hewlett Packard Labs, we are exploring MDC hardware and software design through The Machine. This talk will review the trends that motivate MDC, illustrate how MDC benefits applications, provide highlights from our Machine-related work in data management and programming models, and outline challenges that MDC presents for the storage community.

This talk will describe recent developments at the NASA Center for Climate Simulation, which is funded by NASA’s Science Mission Directorate, and supports the specialized data storage and computational needs of weather, ocean, and climate researchers, as well as astrophysicists, heliophysicists, and planetary scientists. To meet requirements for higher-resolution, higher-fidelity simulations, the NCCS augments its High Performance Computing (HPC) and storage/retrieval environment. As the petabytes of model and observational data grow, the NCCS is broadening data services offerings and deploying and expanding virtualization resources for high performance analytics.

The storage group in CERN’s IT department provides coherent storage services for the physics community at CERN, including the the experiments at the Large Hadron Collider. In this talk we give a status update on the storage technologies, workloads, and storage analytics at CERN. Moreover, we will discuss the strategic developments like a new archive backend for our EOS disk pools, a RAFT based implementation of EOS namespace and latest security enhancements to our data access protocol (xroot).

Scaling NAS filer technology to extreme-scale requires a mix of hardware and software as well as intelligent system design. In this talk, we outline one approach to scaling NAS name spaces.

Mitch will illustrate the power of separating content and metadata. When metadata is extracted into a database, the index powers web-scale file access, search, and audit in ways not before possible.

Storage systems continue to demand the seemingly impossible triad: faster, cheaper, and more reliable. As systems scale up, all three become increasingly hard to balance, with reliability generally taking the back seat. While flat protection schemas work well for small systems, they all compromise too much of the triad at the tens to hundreds of petabyte scale. This discussion will focus on the genesis of the tiered erasure system used in MarFS and how it leverages hardware accelerated erasure to implement a fast, safe, and efficient storage paradigm.

Ceph is an open-source, unified, distributed storage system designed with scale in mind. Ceph's performance doesn't suffer as your data storage grows, which makes it a good fit for Big Data projects. Ceph community has recently been focused on adding key enterprise features such as erasure coding, compression and encryption. In this presentation, we’ll talk about how Intel QuickAssist and ISA-L based offloads can help accelerate these functions in Ceph.

With the addition of the superblock namespace (essentially a user namespace for the kernel to filesystem boundary) much of the stage is now set for fixing one of the biggest underlying container problems: that of translating unprivileged container writes into real filesystem uid/gids. This talk will examine how this system works, why it is necessary and what solutions have been proposed so far, how the upstream discussions are going and what still need to be added for orchestration systems to make use of it.

What is so new about the container environment that a new class of storage software is emerging to address these use cases? And can container orchestration systems themselves be part of the solution? As is often the case in storage, metadata matters here. We are implementing in the open source OpenEBS.io some approaches that are in some regards inspired by ZFS to enable much more efficient scale out block storage for containers that itself is containerized. The goal is to enable storage to be treated in many regards as just another application while, of course, also providing storage services to stateful applications in the environment.

This presentation will describe the evolution and growth of the NCCS's flagship virtualization project known as "ADAPT". The ADAPT system is a KVM/QEMU-based virtualized environment hosting 11 PB of highly cost-effective disk-storage in the form of JBODS, heavily leveraging Linux's "mdadm" software-based RAID for data-integrity and reliability. The ADAPT environment enforces a true zoned architecture, such that nodes within a zone have no visibility whatsoever to network or storage resources in other zones. This is accomplished by providing each zone it's own distinct set of virtualized fileservers which access their media as logical block devices presented by their hypervisors.

Spin-Transfer-Torque Magnetic Random Access Memory (STT-MRAM) is emerging as a leading candidate for a variety of embedded memory applications ranging from embedded NVM to working memory and last level cache. In this talk, we will discuss the basic principles of STT-MRAM, as well as recent advances that bring perpendicular STT-MRAM closer to mass production. We will also address the specific challenges facing STT-MRAM for standalone and embedded applications, and its place in the emerging NVM landscape.

In this presentation, Andy will report on the latest developments around persistent memory programming. He’ll describing current discussions in the SNIA NVM Programming Technical Work Group, the current state of operating system support, recent tool and library development, and finally he’ll describe some of the upcoming challenges for high performance persistent memory use.

Fast non-volatile memories (NVMs) will soon usher in a new era of high-performance storage. Managing, protecting, accessing, and maintaining consistency for data stored in NVM file systems raises a host of challenges that neither conventional block-oriented file systems nor existing NVM file systems address. I will describe NOVA, an open-source file system that provides the higher performance, better reliability, and stronger consistency than any available NVMM file system. NOVA adapts conventional log-structured file system techniques to exploit the fast random access that NVM provides. It uses per-file logs to provide fast, atomic file operations, uses checksums and ECC to protect data and metadata from media errors and software bugs, and leverages DRAM to to maximize performance. Experimental results show that in write-intensive workloads, NOVA provides 22% to 216x throughput improvement compared to state-of-the-art file systems, and 3.1x to 13.5x improvement compared to file systems that provide equally strong data consistency guarantees.