AWS makes S3 a file system

Contents

Amazon Web Services (AWS) has introduced “S3 Files”, a new feature that makes the S3 object storage service directly usable as a file system for the first time. Users can mount S3 buckets on EC2 instances, in containers under ECS and EKS, and in Lambda functions, and access them via ordinary file operations. AWS aims to blur the previously strict separation between object storage and file systems.

For twenty years, Amazon S3 has been storing data as objects in so-called buckets. Unlike in classic file systems, individual objects cannot be edited directly – the entire object must be replaced with every change. This scales well and is inexpensive, but excludes applications that rely on POSIX-compliant file systems.

Mount S3 buckets as a file system

S3 Files now maps S3 objects as files and directories and supports NFS operations from version 4.1 onwards – i.e., create, read, update, and delete. Applications access the data via standard file access instead of using S3's proprietary API. The service automatically synchronizes changes in the file system back to the S3 bucket; changes to the bucket usually appear in the file system within a few seconds, but sometimes only after a minute.

Multiple compute resources can access the same file system simultaneously without duplicating data. This is suitable for scenarios such as ML training or agentic AI systems where multiple instances access common data concurrently.

Performance: Fast cache and direct S3 access

For performance, S3 Files combines a fast storage layer with direct S3 access. The service keeps frequently used data in fast storage and, according to AWS, delivers it with latencies around one millisecond. For large files with sequential read access, the service streams directly from S3 to maximize throughput. In addition, there is prefetching and byte-range reads, which transfer only the data actually needed. Users can control whether the fast storage holds complete files or just their metadata.

Consistency follows the NFS-typical close-to-open model: changes become visible to other clients after a file is closed. This makes the model particularly suitable for collaborative workloads.

Security, Use Cases, and Pricing

Technically, S3 Files is based on Amazon EFS. Access rights can be controlled both via IAM policies and via POSIX file rights with UID and GID. Data transfer is via TLS 1.3, and storage is server-side encrypted – optionally with S3's own or customer-managed KMS keys.

AWS names ML pipelines, data-intensive analyses, and AI agents that rely on file-based tools and scripts as typical use cases. However, all applications designed for classic file systems can also access S3 data this way.

S3 Files is now available in all commercial AWS regions. AWS charges for, among other things, the storage used in the file system layer, write and small read operations, and S3 requests for synchronization between the file system and the bucket. Further information can be found in Amazon's blog post.

(fo)