Unveiling Amazon S3: What Is S3 and How Does It Work?

What is S3?- Demystify Amazon S3

Amazon S3 (Simple Storage Service) is easy to use, having a simple web service interface to store and retrieve an unlimited amount of data from anywhere on the web. It gives software developers and IT professionals with very secure, extremely durable, highly-scalable object storage in the Amazon cloud.

Click here to start the journey in AWS.

What is Amazon S3?

AWS S3, in short, called Simple Storage Service and it is one of the earliest and popular cloud services in AWS.AWS S3 is a very safe place to store files. S3 is Object-based storage.

Know the Basics:

As AWS S3 is object-based, it allows us to upload or store files. The stored file size can be from 0 bytes to 5TB. As far as we know from Amazon, it has unlimited storage. Files are stored in containers called “buckets”. Crazy right, well inside buckets we can create folders and files, just like the folders and files in our operating system's file system.

Let’s make it a bit interesting, S3 is a global or universal namespace. That means names should be unique globally. The example below shows an example of one bucket (container/folder/file) created in AWS S3;

https://myfirstbucket.amazonaws.com/myfirstfolder/myfirstfileobject

When we upload a file to S3, we receive an HTTP Status 200 if the upload is successful.

What are S3-Objects?

When I am mentioning as AWS S3 is Object-based, just think it as files. These Objects consists of Key(name of the object), Value(data made up of a sequence of bytes), Version-ID (used in versioning), Metadata(data about data we are storing), Sub-resources (like Access Control Lists and Torrent).

Data consistency in Amazon S3:

Data consistency in AWS S3 is achieved by Reading after Write consistency and Eventual Consistency models.

Read after write consistency is for PUTS of new Objects. If we successfully write a new file in S3 and read it immediately afterward, we can get that data from S3.

Eventual consistency for overwrite PUTS and DELETES. If we update an existing file or delete it from S3 and immediately try to read it from S3, we may get older versions or may not. Changes to existing objects take a little bit of time to propagate.

Amazon S3 confidence:

Any sort of confidence for the product comes from the guaranty. Amazon S3 has a couple of guarantees from Amazon. It is built for 99.99% availability for the S3 platform, Amazon’s Guarantee for availability is 99.9%. Amazon’s guarantee for durability of S3 information is 99.999999999%(11 * 9s)

Core features of Amazon S3:

Core features of AWS S3 are Tiered Storage, Lifecycle Management, Versioning, Encryption, MFA Delete, Secure data using Access Control Lists, and Bucket Policies.

In Lifecycle management we can move S3 objects to a different tier of storage class based on some timeframe logics. Versioning helps in keeping multiple versions of objects in our S3 bucket. Encryption allows us to encrypt our S3 data at rest. If we enable the MFA delete option in our S3 bucket then object deletion will ask for two-factor authentication like Google authenticator before deleting the object.

Let’s deep dive a bit into S3 storage classes;

S3 Standard: It is having 99.99% availability and 99.999999999% durability. This means, data is stored redundantly across multiple devices in multiple facilities. It is designed to sustain the loss of 2 facilities concurrently.

S3 Standard IA: This has been designed for data that is accessed less frequently or data access frequency is less, however, it requires rapid access when needed. It has a lower fee then S3 Standard, however, charged a retrieval fee.

S3 One Zone IA: This has been designed as a lower-cost option for accessing infrequently accessed data, however, multiple availability zone data resilience is not required.

S3 Intelligent Tiering: Machine Learning enabled designed storage class. It has been designed to optimize costs by learning data usage patterns and automatically moving data to the most cost-effective access tier. It does all these activities without hurting the performance or operational overhead.

S3 Glacier: It has been designed for data archiving. The glacier is a secure, durable, and low-cost storage class used for data archives. We can transfer any amount of data with competitive cost or cheaper than many on-premise solutions. Data retrieval time is configurable from minutes to hours.

S3 Glacier Deep Archive: Designed as the lowest cost option. Data retrieval time of 12 hours is acceptable for this storage class.

Amazon S3 charges:

Amazon S3 charges are based on Storage, Requests, Storage Management Pricing, Data Transfer Pricing, Transfer Acceleration, and Cross Region Replication Pricing. Click here to know more about amazon s3 pricing.

Cross-Region Replication:

This helps in replicating S3 data to multiple regions, very useful for making data redundancy for high availability and disaster recovery. Versioning must be enabled for both source and destination buckets. Regions must be unique. Files in existing buckets are not replicated automatically, once enabled then subsequent file upload and updates on existing files will be replicated automatically. File deletes are not replicated, individual version deletes are also not replicated.

Same-Region Replication:

Same Region Replication (SRR) in Amazon S3 enables data replication within the same AWS region, ensuring redundancy and improving data durability and availability. Like Cross Region Replication (CRR), SRR requires versioning to be enabled on both the source and destination buckets. It is particularly useful for maintaining multiple copies of data within a single region to support compliance requirements, enhance operational workflows, or separate environments for different applications. SRR does not automatically replicate existing files in the source bucket at the time of enabling; instead, subsequent uploads and updates are replicated automatically. However, file deletions and individual version deletes are not propagated, maintaining control and flexibility over replicated data.

S3 Transfer Acceleration:

It enables the fast, easy, and secure transfer of files over long distances between the end-user and an S3 bucket. It makes use of Amazon CloudFront global distributed edge locations, as data arrives at an edge location, data is routed to S3 over an optimized network path.

S3 Event Notification:

S3 Bucket Event Notifications allow you to configure Amazon S3 to send event notifications whenever certain actions occur on a bucket or its objects. This feature is particularly useful for automating workflows, integrating with other AWS services, and triggering custom logic in response to changes in your bucket.

Key Features:

1. Supported Events:

   • s3:ObjectCreated: Triggered when an object is uploaded or written (e.g., PUT, POST, COPY).

   • s3:ObjectRemoved: Triggered when an object is deleted.

   • s3:ObjectRestore: Triggered when an object is restored from Glacier or Deep Archive.

   • s3:Replication: Triggered for replication status changes.

   • s3:LifecycleExpiration: Triggered for lifecycle expiration events.

2. Destinations: You can send notifications to:

   • Amazon Simple Notification Service (SNS): Broadcast events to multiple subscribers.

   • Amazon Simple Queue Service (SQS): Queue events for asynchronous processing.

   • AWS Lambda Functions: Trigger custom processing or workflows.

3. Filtering: Event notifications can include filters based on object key name prefixes and suffixes. For example, you can configure notifications for only .jpg files or files in a specific folder.

What is Snowball?

It is a petabyte-scale data transfer solution. It transfers a large amount of data into and out from AWS using secure appliances. It takes care of challenges like high network cost, long transfer times, and security concerns. Comparatively, it is one-fifth of the cost of high-speed internet.

Some features of snowball;

Comes in two sizes,50TB and 80TB

Uses multi-layer security. It is designed to protect data including tamper-resistance enclosures,256-bit encryption, and an industry-standard trusted platform module to ensure security and full-data integrity of custody of data.

Once the data transfer job is complete AWS performs the software erasure procedure of the snowball appliance.

AWS Snowball Edge:

AWS Snowball Edge is a 100TB data transfer device with on-board storage and computes capability. It is used to move a large number of datasets in and out to AWS. It is quite efficient in executing local workloads in offline locations.

AWS Snowmobile:

It is an exabyte scale data transfer service used to move an extremely large amount of data to AWS. It helps easy to move massive volumes of data to AWS, video libraries, image repositories, and complete datacentres. It is a 45-foot long shipping container pulled by a semi-trailer truck.

S3 and Eventbridge Integration:

Amazon EventBridge Integration with S3 enables seamless event-driven architecture by allowing S3 to emit events directly to EventBridge. This integration helps build scalable, decoupled workflows that react to S3 bucket events without requiring additional infrastructure like SNS or SQS.

What is Amazon EventBridge?

Amazon EventBridge is a serverless event bus service that allows you to connect applications using events. It supports routing events from AWS services or custom applications to various targets like AWS Lambda, Step Functions, or even external APIs.

How S3 Integrates with EventBridge

1. Enable EventBridge Notifications:

   • Once EventBridge is enabled for an S3 bucket, all supported events are automatically sent to EventBridge, bypassing the need for manual notification configurations.

2. Supported Event Types: EventBridge receives the same events that S3 bucket notifications support, such as:

   • s3:ObjectCreated:* (e.g., PUT, POST, COPY)

   • s3:ObjectRemoved:*

   • s3:ObjectRestore:*

   • s3:Replication:*

   • s3:LifecycleExpiration:*

3. Rules in EventBridge:

   • Create rules in EventBridge to filter and route events to specific targets.

   • Rules can be defined based on event patterns, such as the bucket name, object key, or specific event types.

References and Credits:

A great way of learning Amazon cloud is learning material and resources from AWS