7 Parallel file system tiering to object storage lets research computing centers, AI teams and media organizations combine the speed of a high-performance file system with the capacity and economics of object storage. Parallel file systems serve compute clusters with very high throughput and metadata performance, usually on flash. Object storage holds far more data at a much lower cost per terabyte. Tiering connects the two so that hot data stays fast while cold data moves to object storage, often without users needing to change how they find their files. This article explains the main tiering models, how they behave when data is read back, what to evaluate and how to size and operate a tiered environment. For the wider context, see our hub on HPC data tiering. Why tier parallel file systems to object storage Cost: flash-based parallel file systems are expensive per terabyte; object storage on high-capacity drives is far cheaper. Capacity: research and AI datasets grow faster than flash budgets. Performance: keeping the fast tier for active data preserves throughput and metadata responsiveness. Durability: object storage protects data with erasure coding across nodes and sites. Accessibility: data on object storage can be accessed over S3 by other tools and collaborators, not only through the file system. Tiering models Native tiering in the file system Some modern high-performance file systems tier natively to S3-compatible object storage. Metadata stays on the fast tier, while file data is moved to object storage based on policy, often in large chunks. Reads of tiered data are fetched transparently from object storage, sometimes with prefetching. This model is the most seamless for users, since the namespace never changes. Policy engine with external pools Some parallel file systems include policy engines that select files by attributes such as age, size, owner or path and move them to external storage pools, including cloud or object storage targets. A stub or placeholder remains in the file system, and access triggers a recall. Policies can also pre-migrate data so recall is faster. Hierarchical storage management HSM frameworks migrate file contents to an archive and leave the file’s metadata in the namespace. When a user opens a released file, the HSM restores it from the archive. Parallel file system HSM frameworks typically work this way, using data mover agents to move files to an archive tier, which can be S3-compatible object storage. Copy-based archiving Rather than tiering transparently, data is copied to object storage with tools or workflows and removed from the file system. The object storage copy becomes the record. This is simple and keeps data accessible over S3 in its native form, but users must know where to find it. Transparent vs native object format An important question is how data looks on object storage: File-system-managed format: some tiering approaches store data in chunks or proprietary layouts. It is efficient for the file system but not readable as normal files through S3. One file per object: other approaches store each file as an object with a recognizable name, so data can be accessed directly over S3 by other tools. Native S3 readability matters if you want object storage to serve as a shared data platform, for example for AI pipelines, data portals or collaboration, rather than only as a back end for the file system. Recall behavior Recall is where tiering succeeds or frustrates users: Latency: the first read of a tiered file must fetch data from object storage. With on-premises object storage, this is typically fast, but large files take time. Partial recall: some systems fetch only the needed portion of a large file, which helps with workloads that read parts of files. Bulk recall: jobs that read thousands of tiered files at once can create recall storms. Pre-staging data before jobs start avoids slowdowns. Prefetching: some systems prefetch data when a directory is accessed or a job is scheduled. Educate users about which data is tiered and provide tools to stage data before large jobs. What object storage must deliver High throughput for migrations and recalls, scaling with capacity. Large object performance for chunked or whole-file transfers. Strong consistency so recalled data is always current. Scale to billions of objects if one file per object is used. Multi-site protection for archived data. Network considerations Tiering moves large volumes between the file system and object storage. Ensure enough bandwidth between file system servers or data movers and object storage nodes, especially for initial migrations and bulk recalls. Placing object storage in the same data center as the parallel file system usually gives the best results. Designing policies Typical tiering policies consider: Age: files not accessed for a set period move to object storage. Size: large files benefit most from tiering, since small files save little capacity and recall overhead dominates. Project status: data from completed projects moves in bulk. Watermarks: when the fast tier reaches a utilization threshold, the oldest data is tiered. Exclusions: active datasets, software and small configuration files stay on the fast tier. Start conservatively, measure recall rates and adjust. Sizing a tiered environment Fast tier for active working sets, typically a fraction of total data. Object tier for total data minus the fast tier, plus growth and protection overhead. Migration throughput sized to keep up with data aging out of the fast tier. Recall throughput sized for peak job demand. Tiering for AI workloads AI training changes tiering assumptions. Training datasets are read repeatedly over many epochs, often in random order, so tiering them away during a training campaign causes constant recalls. A better pattern is to keep the canonical dataset on object storage and stage the working copy to the fast tier, or to local NVMe on GPU servers, before training starts, then release it afterward. Checkpoints written during training can land on the fast tier and be copied to object storage for durability. Operating and monitoring Monitor fast tier utilization, the volume of data migrated and recalled each day, recall latency and failures, object storage capacity and throughput, and the number of stub files in the namespace. Review policies regularly against recall statistics: frequent recalls of recently tiered data suggest policies are too aggressive, while a fast tier that keeps filling suggests they are too timid. Coordinate maintenance windows across both tiers so that object storage maintenance does not block recalls during critical jobs. Common pitfalls Tiering small files, generating huge object counts with little capacity benefit. Recall storms when large jobs read tiered data without staging. Opaque object formats that lock data into the file system. Insufficient network bandwidth between tiers. Backup confusion, where backup tools trigger recalls of every tiered file. Unclear user communication about tiered data behavior. Regional considerations Research centers in Europe, the UK, Japan and elsewhere often serve projects with data location requirements, such as health or national security data that must remain in-country. Tiering to on-premises object storage keeps archived data local, unlike tiering to public cloud. Checklist: parallel file system tiering to object storage Choose a tiering model: native, policy engine, HSM or copy-based. Decide whether objects must be readable over S3 as normal files. Set policies by age, size, project and watermarks. Exclude small files and active datasets. Size network and object storage for migration and recall throughput. Provide staging tools to avoid recall storms. Coordinate tiering with backup tools. Protect object storage data across sites. Communicate tiering behavior to users. Putting it together Parallel file system tiering to object storage gives research and AI environments fast storage where it counts and affordable capacity for everything else. Native tiering, policy engines and HSM all work, with different trade-offs in transparency, recall behavior and data format. Choose an approach that keeps recall predictable, size the network and object storage for real movement volumes and, where possible, keep data readable over S3 so object storage can serve as a shared research data platform. Frequently asked questions What is file system tiering to object storage? Moving data from a high-performance file system to lower-cost object storage based on policy, while keeping it accessible through the file system or over S3. Do users notice when data is tiered? With transparent tiering, files remain visible, but the first read of tiered data can be slower while it is recalled. What is a recall storm? A surge of recalls when many tiered files are accessed at once, often by large jobs, which can slow the system. Should small files be tiered? Usually not. They save little capacity and create many objects and recall overhead. Can tiered data be accessed directly over S3? Only if the tiering approach stores files as normal objects. Some store data in file-system-specific formats. Further reading HPC data tiering Instrument data storage University S3 storage services Tiered storage for AI
