Object storage

On this page 7

Object storage is a data storage architecture that manages data as discrete objects, each consisting of the data itself, descriptive metadata and a unique key, stored in a flat namespace and accessed through an HTTP-based API rather than a file system or disk interface. In cloud storage it is the default home for unstructured data such as images, video, backups, logs and data-lake files.

The model trades the fine-grained, in-place updates of disks and file systems for near-unlimited scale, very high durability and low cost per gigabyte. Objects are written and read as whole units, grouped into containers usually called buckets, and addressed by key from anywhere with network access and the right permissions.

How object storage organizes data

Object storage has no real directory tree. A key such as invoices/2026/11/0042.pdf looks like a path, but the slashes are ordinary characters; consoles and tools merely simulate folders by grouping keys that share a prefix and delimiter.

Every object carries three parts that the system stores together.

PartContentPurpose
KeyUnique name within the bucketLocates the object
DataByte sequence of any formatThe payload itself
System metadataSize, checksum, timestamp, content typeIntegrity and HTTP handling
User metadataCustom key-value pairsClassification and application logic
Version identifierGenerated per write when versioning is onRecovery of earlier states
Anatomy of a stored object

Applications interact through a small set of verbs: PUT writes a whole object, GET reads it or a byte range of it, DELETE removes it and LIST enumerates keys by prefix. The S3 API has become the de facto standard interface, and many other providers and on-premises systems implement it.

Object storage compared with block and file storage

The three storage models differ mainly in how data is addressed and modified, which determines the workloads each one serves well.

PropertyObject storageBlock storageFile storage
AddressingKey in a flat namespaceBlock number on a volumePath in a directory tree
Access methodHTTP APIDisk attached to one hostNFS or SMB over the network
Updating part of the dataRewrite the whole objectRewrite individual blocksModify bytes within a file
Typical latencyTens of millisecondsSub-millisecond to low millisecondsLow milliseconds
ScalePractically unlimitedLimited by volume sizeLimited by file system design
Typical workloadsMedia, backups, data lakesDatabases, boot volumesShared home and project folders
Object, block and file storage side by side

Because objects are immutable units, changing one byte of a 2 GB file means uploading a new 2 GB object. That makes object storage a poor fit for databases or files edited in place, and an excellent fit for write-once, read-many data.

How durability and consistency are achieved

Object storage reaches very high durability by spreading redundant data across devices and facilities. Major cloud services design for 99.999999999% (eleven nines) annual durability for standard classes, which statistically corresponds to losing about one object per 10,000 years when storing 10 million objects.

Two redundancy techniques dominate, and erasure coding is far more space-efficient than simple replication at comparable or better fault tolerance.

SchemeRaw capacity per 1 TB storedFragments that can be lostMain cost
Three-way replication3.0 TB2 of 3 copiesHigh capacity overhead
Erasure coding 10+41.4 TBAny 4 of 14 fragmentsCPU and network for reconstruction
Erasure coding 6+31.5 TBAny 3 of 9 fragmentsWider read fan-out
Replication versus erasure coding

Consistency semantics matter as much as durability. Several large services now offer strong read-after-write consistency for new and overwritten objects, but older systems and some S3-compatible products remain eventually consistent for listings or overwrites, so applications should confirm the guarantee of their specific platform.

Storage classes and lifecycle management

Providers offer storage classes that trade retrieval speed and access cost for lower monthly storage prices. Lifecycle rules move objects between classes or delete them automatically based on age, prefix or tags.

ClassAccess patternRetrieval timeCost profile
Hot or standardFrequent readsMillisecondsHighest storage price, no retrieval fee
Infrequent accessAbout monthlyMillisecondsLower storage price, per-gigabyte retrieval fee
ColdA few times per yearMilliseconds to minutesLow storage price, minimum storage duration
ArchiveRarely, for complianceMinutes to 12 hours or moreLowest storage price, highest retrieval cost
Typical object storage classes

Colder classes usually impose a minimum storage duration and a minimum billable object size. Moving millions of tiny objects into an archive class can therefore cost more than leaving them in standard storage.

Practical limits and common mistakes

Object storage is simple to start with, but production use exposes limits that are easy to overlook.

  • Large uploads without multipart: in S3 a single PUT is limited to 5 GB while an object may reach 5 TB; multipart upload splits data into up to 10,000 parts and allows parallel, resumable transfers.
  • Treating it as a file system: renaming a folder means copying and deleting every object under the prefix.
  • Hot key prefixes: request rates scale per prefix, so funneling heavy traffic through one prefix can cause throttling.
  • Public buckets by accident: misconfigured access policies remain a leading cause of cloud data exposure; block public access by default and share through presigned URLs.
  • Ignoring request and egress charges: workloads with many small reads or heavy downloads are often dominated by request and data transfer costs, not storage.

In everyday language

Object storage works like a valet coat check of enormous size. You hand over an item, receive a ticket with a unique number, and later use that ticket to get the item back. You do not care which shelf it sits on, and the coat check never runs out of space.

The item comes back exactly as you left it. If you want to change it, you hand in a new version rather than altering it on the shelf. That makes object storage cheap, safe and huge, but not the right place for data that changes constantly.

Seen in the real world

A video streaming platform stores about 400 TB of source footage and encoded renditions. New uploads land in a standard-class bucket, where an event triggers transcoding into several resolutions. Viewers fetch the files through a content delivery network that reads from the bucket, so the storage layer sees only cache misses.

Analytics show that titles older than 90 days receive less than 2% of views. A lifecycle rule moves source footage to an archive class after 90 days and keeps popular renditions in standard storage. Versioning protects against accidental overwrites, and a rule expires noncurrent versions after 30 days.

The platform scales storage without any capacity planning, and the bulk of its archive sits in the lowest-priced class while viewers still receive fast playback.