6 Sequencing has gotten cheap, which is exactly why storage has gotten hard. A single whole-genome sequence produces hundreds of gigabytes of data, and a busy lab or core facility runs hundreds or thousands of samples a year. The result is relentless, compounding growth in data that has to be kept for years. The real challenge is rarely raw capacity. It is managing the whole life of that data, from fast pipelines today to a defensible long-term archive, without the cost spiraling and without losing the reproducibility, privacy, and shareability that science depends on. This guide explains what genomics data actually looks like, why it is uniquely hard to store, how the hot-to-cold lifecycle works, and how to think about architecting for it. It is vendor-neutral and written for the person who owns the storage. What genomics data actually looks like Genomics is not one file type but a pipeline, and each stage has a very different storage profile. Raw reads land as FASTQ files straight off the sequencer. These are large, approximately 90 to 120 gigabytes each for a whole genome, and a single project often generates 100 to 200 gigabytes of raw FASTQ. From there, the pipeline runs quality control, trims the reads, and aligns them to a reference genome, producing aligned data as BAM files, which commonly run around 200 to 300 gigabytes per run. CRAM is an increasingly popular alternative to BAM: it uses reference-based compression to cut file sizes by roughly half, which makes it well suited to long-term archiving of aligned data. Finally, variant calling produces VCF files, the small, standardized records of genetic variation such as SNPs, insertions, deletions, and structural variants. VCFs are what labs typically share with each other. Around all of this sit instrument and image data and shared reference genomes. The important point for storage: a completed project leaves behind large raw and aligned files that are rarely touched again, plus small variant files that are used constantly. One size does not fit all. Why genomics storage is uniquely hard Several pressures pile up at once. Growth is exponential. As sequencing costs fall, sample volume climbs, and for organizations running hundreds or thousands of samples a year, storage cost accumulates fast, especially when everything lands on expensive primary NAS. File sizes are enormous, so even routine operations like copying, backing up, or moving data become slow and costly at petabyte scale. Most data goes cold quickly. After a FASTQ file is processed, it is rarely read again, yet it often stays on primary storage because standard tools cannot easily tell a fresh file that may be needed imminently from a three-year-old archive from a finished study. Retention is long and non-negotiable. Raw data has to be kept for reproducibility, for regulatory reasons, and because re-analysis with improved reference genomes and methods can extract new findings from old samples years later. Deleting it destroys future value. On top of that come collaboration and sharing across institutions, strict privacy rules, and data-residency constraints. Genomic data is among the most demanding and most sensitive data in enterprise IT. The lifecycle: hot to cold The winning strategy treats genomics data by its stage of life rather than storing everything the same way. During active analysis, data is hot. Pipelines, alignment, and variant calling need fast, parallel access, and this is where performance matters most. Once a project is processed, the raw FASTQ and aligned BAM data turn cold. It still has to be retained, but it does not need to sit on the fastest, most expensive tier. This is where a deliberate hot-to-cold strategy pays off: compress aligned data to CRAM, and use automated lifecycle policies to move cold data off primary storage into cost-efficient capacity without anyone having to babysit it. The key discipline is to tier, not delete. Long-term archiving keeps the data available for the reanalysis that makes genomics cumulative, while getting it off the tier that is bleeding budget. Performance for pipelines, HPC, and AI Genomics is compute-heavy. Secondary and tertiary analysis run on HPC clusters, and increasingly on AI and machine-learning workflows for variant interpretation, multiomics, and even genomic foundation models. All of these read and write large files in parallel, so the storage layer has to deliver high aggregate throughput to many nodes at once. If storage cannot keep the pipeline and the AI data pipeline fed, expensive compute sits idle waiting on data. The goal is a platform that scales throughput with the cluster, so storage never becomes the bottleneck in a time-sensitive analysis. Privacy, reproducibility, and sovereignty Genomic data is deeply personal and, in many cases, identifying, which puts it squarely under regulations like HIPAA in the US and GDPR in the EU. Treating de-identification as a complete solution is risky, because genomic data is inherently hard to fully anonymize. Two storage properties matter here beyond access control. First, integrity and reproducibility: immutable, WORM-style storage lets a lab attest that raw data has not been altered since it was generated, which underpins both reproducibility and defense against ransomware. Second, data sovereignty: research consortia and national programs increasingly require that genomic data stay within specific borders, which shapes where it can be stored and who can access it. Architecting genomics data storage Put the requirements together and the profile is clear: huge files, parallel high-throughput access during analysis, cost-efficient long-term retention, integrity, privacy, and sovereignty. That is why scale-out object storage has become a common foundation for genomics at scale. An S3-compatible platform speaks the interface that modern bioinformatics tools, workflow managers, and AI frameworks already use. Erasure coding delivers durable capacity far more cost-effectively than replication when you are keeping years of large files. Lifecycle and tiering move cold FASTQ and CRAM off primary storage automatically. Object lock and WORM protect integrity and reproducibility. And on-premises or private deployment gives cores and institutions predictable cost, control of sensitive data, and freedom from cloud egress fees when large datasets are pulled back for reanalysis. This is not about a specific product. It is about matching the architecture to one of the most demanding data lifecycles in science. A checklist for evaluating genomics data storage High aggregate throughput for parallel pipeline and HPC accessPetabyte-plus scalability with predictable cost as sample volume growsAutomated hot-to-cold tiering and lifecycle so cold data leaves primary storageCost-efficient durability (for example erasure coding) for multi-year retentionSupport for CRAM and other compressed formats for archivingObject lock or WORM immutability for integrity and reproducibilityAccess control and audit aligned to HIPAA and GDPRData-residency and sovereignty controls for consortia and national programsS3-compatible access so bioinformatics and AI tools read the same store Putting it together Genomics data storage is a lifecycle problem, not a capacity problem. The organizations that handle it well keep hot data fast for analysis, retain cold raw data cheaply and immutably for the reanalysis that makes the science compound, and never let a completed study sit on the most expensive tier just because moving it is inconvenient. The practical path is to design for the full life of the data: fast parallel access while it is active, automated tiering as it cools, compression and cost-efficient durability for the long archive, and privacy, integrity, and sovereignty controls throughout. Do that, and storage stops being the thing that limits how much you can sequence. Frequently asked questions How big is a genome data file? It varies by sequencing depth and platform, but a whole-genome FASTQ file is roughly 90 to 120 gigabytes, an aligned BAM file is often around 200 to 300 gigabytes per run, and a VCF of variants is comparatively tiny. What is the difference between FASTQ, BAM, CRAM, and VCF? FASTQ holds raw sequencer reads, BAM holds reads aligned to a reference, CRAM is a more compressed alternative to BAM for archiving, and VCF stores the called genetic variants that labs typically share. What storage is best for genomics data? The workload favors scale-out, S3-compatible object storage: it delivers parallel throughput for pipelines, scales to petabytes, tiers hot to cold automatically, and provides cost-efficient durable retention with immutability. How long should genomic data be retained? Often for many years. Raw data is kept for reproducibility, regulatory reasons, and future reanalysis with improved references and methods, so most organizations archive rather than delete. Is genomic data covered by HIPAA? In the US, genomic data tied to an individual generally falls under HIPAA, and in the EU under GDPR. Because genomic data is hard to fully anonymize, it should be treated as sensitive and protected accordingly. Further reading (educational): object storage use cases, a look at hot vs cold storage, and erasure coding vs replication.
