Table Of ContentAmazon Simple Storage Service
User Guide
API Version 2006-03-01
Amazon Simple Storage Service User Guide
Amazon Simple Storage Service: User Guide
Copyright © 2022 Amazon Web Services, Inc. and/or its affiliates. All rights reserved.
Amazon's trademarks and trade dress may not be used in connection with any product or service that is not
Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or
discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may
or may not be affiliated with, connected to, or sponsored by Amazon.
Amazon Simple Storage Service User Guide
Table of Contents
What is Amazon S3? ........................................................................................................................... 1
Features of Amazon S3 ............................................................................................................... 1
Storage classes.................................................................................................................. 1
Storage management......................................................................................................... 1
Access management........................................................................................................... 2
Data processing................................................................................................................. 2
Storage logging and monitoring.......................................................................................... 2
Analytics and insights......................................................................................................... 3
Strong consistency............................................................................................................. 3
How Amazon S3 works ............................................................................................................... 3
Buckets............................................................................................................................. 4
Objects............................................................................................................................. 4
Keys................................................................................................................................. 5
S3 Versioning.................................................................................................................... 5
Version ID ......................................................................................................................... 5
Bucket policy..................................................................................................................... 5
S3 Access Points................................................................................................................ 5
Access control lists (ACLs)................................................................................................... 6
Regions............................................................................................................................. 6
Amazon S3 data consistency model .............................................................................................. 6
Concurrent applications...................................................................................................... 7
Related services......................................................................................................................... 8
Accessing Amazon S3 ................................................................................................................. 9
AWS Management Console .................................................................................................. 9
AWS Command Line Interface ............................................................................................. 9
AWS SDKs......................................................................................................................... 9
Amazon S3 REST API.......................................................................................................... 9
Paying for Amazon S3 .............................................................................................................. 10
PCI DSS compliance.................................................................................................................. 10
Getting started ................................................................................................................................ 11
Setting up............................................................................................................................... 11
Sign up for an AWS account .............................................................................................. 11
Create an administrative user ............................................................................................ 12
Step 1: Create a bucket ............................................................................................................. 12
Step 2: Upload an object .......................................................................................................... 15
Step 3: Download an object...................................................................................................... 15
Using the S3 console ........................................................................................................ 15
Step 4: Copy an object ............................................................................................................. 16
Step 5: Delete the objects and bucket ........................................................................................ 16
Deleting an object ............................................................................................................ 17
Emptying your bucket ....................................................................................................... 17
Deleting your bucket ........................................................................................................ 17
Next steps............................................................................................................................... 18
Understand common use cases .......................................................................................... 18
Control access to your buckets and objects .......................................................................... 18
Manage and monitor your storage ...................................................................................... 19
Develop with Amazon S3 .................................................................................................. 19
Learn from tutorials ......................................................................................................... 20
Explore training and support ............................................................................................. 21
Access control.......................................................................................................................... 22
Creating a new bucket ...................................................................................................... 22
Storing and sharing data................................................................................................... 23
Sharing resources............................................................................................................. 24
Protecting data................................................................................................................ 24
API Version 2006-03-01
iii
Amazon Simple Storage Service User Guide
Tutorials.......................................................................................................................................... 27
Getting started........................................................................................................................ 20
Optimizing storage costs........................................................................................................... 20
Hosting videos and websites...................................................................................................... 20
Data processing ........................................................................................................................ 20
Protecting data........................................................................................................................ 21
Transforming data with S3 Object Lambda.................................................................................. 28
Prerequisites.................................................................................................................... 29
Step 1: Create an S3 bucket .............................................................................................. 30
Step 2: Upload a file to the S3 bucket ................................................................................ 31
Step 3: Create an S3 access point ....................................................................................... 31
Step 4: Create a Lambda function ...................................................................................... 32
Step 5: Configure an IAM policy for your Lambda function's execution role ............................... 36
Step 6: Create an S3 Object Lambda access point ................................................................. 36
Step 7: View the transformed data..................................................................................... 37
Step 8: Clean up.............................................................................................................. 39
Next steps....................................................................................................................... 41
Detecting and redacting PII data ................................................................................................ 42
Prerequisites: Create an IAM user with permissions ............................................................... 43
Step 1: Create an S3 bucket .............................................................................................. 44
Step 2: Upload a file to the S3 bucket ................................................................................ 45
Step 3: Create an S3 access point ....................................................................................... 45
Step 4: Configure and deploy a prebuilt Lambda function ...................................................... 46
Step 5: Create an S3 Object Lambda access point ................................................................. 47
Step 6: Use the S3 Object Lambda access point to retrieve the redacted file ............................. 48
Step 7: Clean up.............................................................................................................. 49
Next steps....................................................................................................................... 51
Hosting video streaming........................................................................................................... 52
Prerequisites: Register and configure a custom domain with Route 53 ..................................... 53
Step 1: Create an S3 bucket .............................................................................................. 54
Step 2: Upload a video to the S3 bucket ............................................................................. 54
Step 3: Create a CloudFront origin access identity................................................................ 55
Step 4: Create a CloudFront distribution .............................................................................. 55
Step 5: Access the video through the CloudFront distribution ................................................. 57
Step 6: Configure your CloudFront distribution to use your custom domain name ...................... 57
Step 7: Access the S3 video through the CloudFront distribution with the custom domain name .. 60
(Optional) Step 8: View data about requests received by your CloudFront distribution ................ 61
Step 9: Clean up.............................................................................................................. 61
Next steps....................................................................................................................... 64
Batch-transcoding videos.......................................................................................................... 65
Prerequisites.................................................................................................................... 66
Step 1: Create an S3 bucket for the output media files ......................................................... 66
Step 2: Create an IAM role for MediaConvert ....................................................................... 68
Step 3: Create an IAM role for your Lambda function ............................................................ 68
Step 4: Create a Lambda function for video transcoding ........................................................ 70
Step 5: Configure Amazon S3 Inventory for your S3 source bucket .......................................... 82
Step 6: Create an IAM role for S3 Batch Operations .............................................................. 84
Step 7: Create and run an S3 Batch Operations job .............................................................. 86
Step 8: Check the output media files from your S3 destination bucket ..................................... 90
Step 9: Clean up.............................................................................................................. 90
Next steps....................................................................................................................... 92
Configuring a static website...................................................................................................... 93
Step 1: Create a bucket ..................................................................................................... 93
Step 2: Enable static website hosting .................................................................................. 93
Step 3: Edit Block Public Access settings ............................................................................. 94
Step 4: Add a bucket policy that makes your bucket content publicly available .......................... 95
Step 5: Configure an index document ................................................................................. 96
API Version 2006-03-01
iv
Amazon Simple Storage Service User Guide
Step 6: Configure an error document .................................................................................. 97
Step 7: Test your website endpoint .................................................................................... 97
Step 8: Clean up.............................................................................................................. 98
Configuring a static website using a custom domain ..................................................................... 98
Before you begin .............................................................................................................. 99
Step 1: Register a custom domain with Route 53 .................................................................. 99
Step 2: Create two buckets ................................................................................................ 99
Step 3: Configure root Domain bucket .............................................................................. 100
Step 4: Configure subdomain bucket for redirect ................................................................ 101
Step 5: Configure logging ................................................................................................ 101
Step 6: Upload index and website content ......................................................................... 102
Step 7: Upload an error document .................................................................................... 103
Step 8: Edit Block Public Access ....................................................................................... 104
Step 9: Attach a bucket policy ......................................................................................... 105
Step 10: Test your domain endpoint ................................................................................. 106
Step 11: Add alias records ............................................................................................... 106
Step 12: Test the website ................................................................................................ 109
Speeding up your website with Amazon CloudFront ............................................................ 110
Cleaning up example resources ........................................................................................ 113
Working with buckets ..................................................................................................................... 115
Buckets overview.................................................................................................................... 115
About permissions.......................................................................................................... 116
Managing public access to buckets ................................................................................... 116
Bucket configuration....................................................................................................... 117
Naming rules......................................................................................................................... 119
Example bucket names .................................................................................................... 119
Creating a bucket ................................................................................................................... 120
Default settings for new S3 buckets FAQ ................................................................................... 126
Viewing bucket properties ....................................................................................................... 127
Methods for accessing a bucket ................................................................................................ 128
Virtual-hosted–style access.............................................................................................. 128
Path-style access............................................................................................................ 128
Accessing an S3 bucket over IPv6 ..................................................................................... 129
Accessing a bucket through S3 access points ...................................................................... 129
Accessing a bucket using S3:// ......................................................................................... 129
Emptying a bucket ................................................................................................................. 129
Deleting a bucket ................................................................................................................... 131
Setting default bucket encryption ............................................................................................ 134
Using SSE-KMS encryption for cross-account operations ...................................................... 135
Using default encryption with replication .......................................................................... 135
Using Amazon S3 Bucket Keys with default encryption ........................................................ 136
Enabling default encryption ............................................................................................. 136
Monitoring default encryption .......................................................................................... 138
Configuring Transfer Acceleration ............................................................................................. 139
Why use Transfer Acceleration? ........................................................................................ 139
Requirements for using Transfer Acceleration ..................................................................... 139
Getting Started .............................................................................................................. 140
Enabling Transfer Acceleration ......................................................................................... 141
Speed Comparison tool................................................................................................... 146
Using Requester Pays .............................................................................................................. 146
How Requester Pays charges work .................................................................................... 147
Configuring Requester Pays ............................................................................................. 147
Retrieving the requestPayment configuration ..................................................................... 149
Downloading objects in Requester Pays buckets ................................................................. 149
Restrictions and limitations ...................................................................................................... 150
Working with objects ...................................................................................................................... 152
Objects.................................................................................................................................. 152
API Version 2006-03-01
v
Amazon Simple Storage Service User Guide
Subresources.................................................................................................................. 153
Creating object keys ............................................................................................................... 153
Object key naming guidelines .......................................................................................... 154
Working with metadata ........................................................................................................... 156
System-defined object metadata ...................................................................................... 157
User-defined object metadata .......................................................................................... 158
Editing object metadata .................................................................................................. 160
Uploading objects................................................................................................................... 161
Using multipart upload ........................................................................................................... 170
Multipart upload process ................................................................................................. 170
Checksums with multipart upload operations ..................................................................... 171
Concurrent multipart upload operations ............................................................................ 172
Multipart upload and pricing ........................................................................................... 172
API support for multipart upload ..................................................................................... 172
AWS Command Line Interface support for multipart upload ................................................. 173
AWS SDK support for multipart upload ............................................................................. 173
Multipart upload API and permissions ............................................................................... 173
Configuring a lifecycle policy ........................................................................................... 175
Uploading an object using multipart upload ...................................................................... 177
Uploading a directory ..................................................................................................... 190
Listing multipart uploads ................................................................................................. 192
Tracking a multipart upload ............................................................................................. 194
Aborting a multipart upload ............................................................................................ 196
Copying an object .......................................................................................................... 200
Multipart upload limits .................................................................................................... 205
Copying objects...................................................................................................................... 205
To copy an object ........................................................................................................... 206
Downloading an object ........................................................................................................... 212
Checking object integrity ......................................................................................................... 219
Using supported checksum algorithms .............................................................................. 219
Using Content-MD5 when uploading objects ...................................................................... 225
Using Content-MD5 and the ETag to verify uploaded objects ............................................... 225
Using trailing checksums................................................................................................. 226
Using part-level checksums for multipart uploads ............................................................... 226
Deleting objects..................................................................................................................... 227
Programmatically deleting objects from a version-enabled bucket ........................................ 228
Deleting objects from an MFA-enabled bucket .................................................................... 228
Deleting a single object ................................................................................................... 228
Deleting multiple objects ................................................................................................. 235
Organizing and listing objects .................................................................................................. 247
Using prefixes................................................................................................................ 248
Listing objects................................................................................................................ 249
Using folders................................................................................................................. 259
Viewing an object overview ............................................................................................. 262
Viewing object properties ................................................................................................ 262
Using presigned URLs ............................................................................................................. 263
Limiting presigned URL capabilities ................................................................................... 263
Who can create a presigned URL ...................................................................................... 264
When does Amazon S3 check the expiration date and time of a presigned URL? ...................... 264
Sharing objects.............................................................................................................. 265
Uploading objects........................................................................................................... 268
Deleting an object .......................................................................................................... 283
Transforming objects.............................................................................................................. 284
Creating Object Lambda access points .............................................................................. 286
Using Amazon S3 Object Lambda Access Points ................................................................. 295
Security considerations.................................................................................................... 296
Writing Lambda functions ............................................................................................... 300
API Version 2006-03-01
vi
Amazon Simple Storage Service User Guide
Using AWS built functions ............................................................................................... 321
Best practices and guidelines for S3 Object Lambda ........................................................... 322
S3 Object Lambda tutorials ............................................................................................. 324
Debugging S3 Object Lambda .......................................................................................... 324
Working with access points .............................................................................................................. 325
Configuring IAM policies.......................................................................................................... 325
Access point policy examples ........................................................................................... 326
Condition keys............................................................................................................... 328
Delegating access control to access points ......................................................................... 329
Granting permissions for cross-account access points .......................................................... 330
Creating access points ............................................................................................................. 330
Rules for naming Amazon S3 access points ........................................................................ 330
Creating an access point .................................................................................................. 331
Creating access points restricted to a VPC ......................................................................... 332
Managing public access ................................................................................................... 334
Using access points ................................................................................................................. 335
Monitoring and logging................................................................................................... 335
Managing access points ................................................................................................... 337
Using a bucket-style alias for your access point .................................................................. 339
Using access points with Amazon S3 operations ................................................................. 340
Restrictions and limitations ...................................................................................................... 342
Working with Multi-Region Access Points ........................................................................................... 344
Creating Multi-Region Access Points .......................................................................................... 346
Rules for naming Amazon S3 Multi-Region Access Points ..................................................... 347
Rules for choosing buckets for Amazon S3 Multi-Region Access Points ................................... 348
Blocking public access with Amazon S3 Multi-Region Access Points ....................................... 348
Creating Amazon S3 Multi-Region Access Points ................................................................. 349
Configuring AWS PrivateLink ............................................................................................ 350
Using a Multi-Region Access Point ............................................................................................ 352
Multi-Region Access Point hostnames ................................................................................ 353
Multi-Region Access Points and Amazon S3 Transfer Acceleration ......................................... 354
Multi-Region Access Point permissions .............................................................................. 354
Request routing.............................................................................................................. 358
Failover configuration..................................................................................................... 359
Bucket replication........................................................................................................... 364
Supported operations..................................................................................................... 367
Managing Multi-Region Access Points ........................................................................................ 373
Monitoring and logging........................................................................................................... 373
Monitoring and logging requests made to Multi-Region Access Point management APIs ........... 374
Using CloudTrail............................................................................................................. 375
Restrictions and limitations ...................................................................................................... 375
Security......................................................................................................................................... 378
Data protection...................................................................................................................... 379
Data encryption..................................................................................................................... 379
Server-side encryption.................................................................................................... 380
Using client-side encryption ............................................................................................. 424
Internetwork privacy............................................................................................................... 428
Traffic between service and on-premises clients and applications .......................................... 428
Traffic between AWS resources in the same Region ............................................................. 429
AWS PrivateLink for Amazon S3 ............................................................................................... 429
Types of VPC endpoints .................................................................................................. 429
Restrictions and limitations of AWS PrivateLink for Amazon S3 ............................................. 430
Creating a VPC endpoint ................................................................................................. 430
Accessing Amazon S3 interface endpoints .......................................................................... 430
Accessing buckets and S3 access points from S3 interface endpoints ..................................... 431
Updating an on-premises DNS configuration ...................................................................... 434
Creating a VPC endpoint policy ........................................................................................ 435
API Version 2006-03-01
vii
Amazon Simple Storage Service User Guide
Identity and access management .............................................................................................. 437
Overview....................................................................................................................... 438
Access policy guidelines ................................................................................................... 444
Request authorization..................................................................................................... 448
Bucket policies and user policies ....................................................................................... 455
AWS managed policies .................................................................................................... 609
Managing access with ACLs .............................................................................................. 611
Using CORS ................................................................................................................... 630
Blocking public access ..................................................................................................... 641
Reviewing bucket access .................................................................................................. 651
Verifying bucket ownership .............................................................................................. 655
Controlling object ownership ................................................................................................... 659
Object Ownership settings ............................................................................................... 660
Changes introduced by disabling ACLs ............................................................................... 661
Prerequisites for disabling ACLs ........................................................................................ 663
Object Ownership permissions ......................................................................................... 664
Disabling ACLs for all new buckets ................................................................................... 664
Replication and Object Ownership .................................................................................... 665
Setting Object Ownership ................................................................................................ 665
Prerequisites for disabling ACLs ........................................................................................ 665
Creating a bucket ........................................................................................................... 674
Setting Object Ownership ................................................................................................ 677
Viewing Object Ownership settings ................................................................................... 679
Disabling ACLs for all new buckets ................................................................................... 680
Troubleshooting............................................................................................................. 682
Logging and monitoring.......................................................................................................... 684
Compliance Validation............................................................................................................. 685
Resilience.............................................................................................................................. 686
Backup encryption.......................................................................................................... 688
Infrastructure security............................................................................................................. 689
Configuration and vulnerability analysis .................................................................................... 690
Security Best Practices ............................................................................................................ 691
Amazon S3 preventative security best Practices .................................................................. 691
Amazon S3 Monitoring and auditing best practices ............................................................. 694
Managing storage........................................................................................................................... 696
Using S3 Versioning ................................................................................................................ 696
Unversioned, versioning-enabled, and versioning-suspended buckets ..................................... 697
Using S3 Versioning with S3 Lifecycle ............................................................................... 697
S3 Versioning ................................................................................................................. 698
Enabling versioning on buckets ........................................................................................ 701
Configuring MFA delete ................................................................................................... 706
Working with versioning-enabled objects ........................................................................... 707
Working with versioning-suspended objects ....................................................................... 725
Using AWS Backup for Amazon S3 ........................................................................................... 728
Working with archived objects ................................................................................................. 729
Archive retrieval options .................................................................................................. 730
Restoring an archived object ............................................................................................ 732
Using Object Lock .................................................................................................................. 736
S3 Object Lock ............................................................................................................... 737
Configuring Object Lock on the console ............................................................................ 741
Managing Object Lock .................................................................................................... 742
Managing storage classes........................................................................................................ 744
Frequently accessed objects ............................................................................................. 745
Automatically optimizing data with changing or unknown access patterns ............................. 745
Infrequently accessed objects ........................................................................................... 746
Archiving objects............................................................................................................ 747
Amazon S3 on Outposts .................................................................................................. 748
API Version 2006-03-01
viii
Amazon Simple Storage Service User Guide
Comparing storage classes............................................................................................... 748
Setting the storage class of an object ............................................................................... 749
Amazon S3 Intelligent-Tiering .................................................................................................. 750
How S3 Intelligent-Tiering works ..................................................................................... 750
Using S3 Intelligent-Tiering ............................................................................................ 752
Managing S3 Intelligent-Tiering ...................................................................................... 755
Managing lifecycle.................................................................................................................. 758
Managing object lifecycle ................................................................................................ 759
Creating a lifecycle configuration ...................................................................................... 759
Transitioning objects....................................................................................................... 760
Expiring objects.............................................................................................................. 765
Setting lifecycle configuration .......................................................................................... 765
Using other bucket configurations .................................................................................... 776
Configuring Lifecycle event notifications ........................................................................... 777
Lifecycle configuration elements ...................................................................................... 778
Examples of S3 Lifecycle configuration .............................................................................. 786
Managing inventory................................................................................................................ 796
Amazon S3 Inventory buckets .......................................................................................... 797
Inventory lists................................................................................................................ 797
Configuring Amazon S3 Inventory .................................................................................... 799
Setting up notifications for inventory completion ............................................................... 803
Locating your inventory .................................................................................................. 804
Querying inventory with Athena ....................................................................................... 806
Converting empty version ID strings to null strings ............................................................. 809
Replicating objects.................................................................................................................. 810
Why use replication ........................................................................................................ 811
When to use Cross-Region Replication .............................................................................. 812
When to use Same-Region Replication .............................................................................. 812
When to use two-way replication (bi-directional replication) ................................................. 813
When to use S3 Batch Replication .................................................................................... 813
Requirements for replication ............................................................................................ 813
What's replicated?........................................................................................................... 814
Setting up replication ..................................................................................................... 816
Replicate existing objects ................................................................................................ 857
Additional configurations................................................................................................. 864
Getting replication status ................................................................................................ 881
Troubleshooting............................................................................................................. 884
Additional considerations................................................................................................. 885
Using object tags ................................................................................................................... 887
API operations related to object tagging ........................................................................... 889
Additional configurations................................................................................................. 889
Access control................................................................................................................ 890
Managing object tags ...................................................................................................... 892
Using cost allocation tags ........................................................................................................ 896
More Info ...................................................................................................................... 897
Billing and usage reporting ...................................................................................................... 897
Billing reports................................................................................................................ 897
Usage report.................................................................................................................. 899
Understanding billing and usage reports ........................................................................... 901
Using Amazon S3 Select .......................................................................................................... 913
Requirements and limits .................................................................................................. 914
Constructing a request .................................................................................................... 914
Errors............................................................................................................................ 915
S3 Select examples ......................................................................................................... 915
SQL Reference............................................................................................................... 918
Using Batch Operations........................................................................................................... 942
Batch Operations basics.................................................................................................. 943
API Version 2006-03-01
ix
Amazon Simple Storage Service User Guide
S3 Batch Operations tutorial............................................................................................ 944
Granting permissions...................................................................................................... 944
Creating a job ................................................................................................................ 951
Supported operations..................................................................................................... 957
Managing jobs................................................................................................................ 982
Tracking job status and completion reports ....................................................................... 985
Using tags..................................................................................................................... 994
Managing S3 Object Lock .............................................................................................. 1005
S3 Batch Operations tutorial.......................................................................................... 1021
Monitoring Amazon S3.................................................................................................................. 1022
Monitoring tools................................................................................................................... 1022
Automated tools........................................................................................................... 1022
Manual tools................................................................................................................ 1023
Logging options................................................................................................................... 1023
Logging with CloudTrail ......................................................................................................... 1025
Using CloudTrail logs with Amazon S3 server access logs and CloudWatch Logs ..................... 1025
CloudTrail tracking with Amazon S3 SOAP API calls .......................................................... 1025
CloudTrail events.......................................................................................................... 1026
Example log files.......................................................................................................... 1030
Enabling CloudTrail ....................................................................................................... 1034
Identifying S3 requests .................................................................................................. 1035
Logging server access ............................................................................................................ 1042
How do I enable log delivery? ........................................................................................ 1042
Log object key format ................................................................................................... 1043
How are logs delivered?................................................................................................ 1043
Best effort server log delivery ........................................................................................ 1043
Bucket logging status changes take effect over time ......................................................... 1044
Enabling server access logging ....................................................................................... 1044
Log format.................................................................................................................. 1053
Deleting log files.......................................................................................................... 1063
Identifying S3 requests .................................................................................................. 1063
Monitoring metrics with CloudWatch ....................................................................................... 1067
Metrics and dimensions................................................................................................. 1068
Accessing CloudWatch metrics........................................................................................ 1078
CloudWatch metrics configurations ................................................................................. 1079
Amazon S3 Event Notifications ............................................................................................... 1085
Overview..................................................................................................................... 1085
Notification types and destinations................................................................................. 1086
Using SQS, SNS, and Lambda ......................................................................................... 1090
Using EventBridge........................................................................................................ 1109
Using analytics and insights ........................................................................................................... 1116
Storage Class Analysis........................................................................................................... 1116
How to set up storage class analysis ............................................................................... 1116
Storage class analysis.................................................................................................... 1117
How can I export storage class analysis data? ................................................................... 1118
Configuring storage class analysis................................................................................... 1119
S3 Storage Lens................................................................................................................... 1121
S3 Storage Lens metrics and features .............................................................................. 1122
Understanding S3 Storage Lens...................................................................................... 1124
Working with Organizations........................................................................................... 1130
S3 Storage Lens permissions.......................................................................................... 1132
Viewing storage metrics................................................................................................ 1134
Amazon S3 Storage Lens metrics use cases ...................................................................... 1153
Metrics glossary............................................................................................................ 1169
Working with S3 Storage Lens ....................................................................................... 1179
Tracing requests using X-Ray .................................................................................................. 1206
How X-Ray works with Amazon S3 .................................................................................. 1206
API Version 2006-03-01
x
Description:Mar 1, 2006 or service that is not Amazon's, in any manner that is likely to cause .. Request
Redirection and the REST API Using the SDK for Java .