Table Of Content

Amazon Elastic Container Service Developer Guide Amazon Elastic Container Service Developer Guide Amazon Elastic Container Service: Developer Guide Amazon Elastic Container Service Developer Guide Table of Contents What is Amazon ECS?......................................................................................................................... 1 Launch types............................................................................................................................. 1 Access Amazon ECS.................................................................................................................... 2 Pricing...................................................................................................................................... 2 Amazon ECS components............................................................................................................ 3 Clusters............................................................................................................................. 3 Containers and images....................................................................................................... 3 Task definitions.................................................................................................................. 3 Tasks................................................................................................................................ 4 Services............................................................................................................................ 4 Container agent................................................................................................................. 4 Fargate architecture overview .............................................................................................. 5 Common use cases..................................................................................................................... 6 Additional resources........................................................................................................... 6 Related services......................................................................................................................... 6 Getting started .................................................................................................................................. 8 Set up...................................................................................................................................... 8 Sign up for an Amazon Web Services account ........................................................................ 8 Secure IAM users ................................................................................................................ 8 Create the credentials to connect to your EC2 instance ........................................................... 9 Create a virtual private cloud ............................................................................................. 10 Create a security group ..................................................................................................... 10 Install the Amazon CLI ...................................................................................................... 12 Creating a container image........................................................................................................ 12 Prerequisites.................................................................................................................... 13 Create a Docker image ...................................................................................................... 14 Push your image to Amazon Elastic Container Registry ......................................................... 16 Clean up......................................................................................................................... 17 Next steps....................................................................................................................... 17 Using Amazon Copilot .............................................................................................................. 17 Prerequisites.................................................................................................................... 17 Deploy your application using one command ....................................................................... 17 Deploy your application step by step .................................................................................. 18 Using the Amazon CDK............................................................................................................. 21 Step 1: Set up your Amazon CDK project ............................................................................ 22 Step 2: Use the Amazon CDK to define a containerized web server on Fargate .......................... 24 Step 3: Test the web server ............................................................................................... 27 Step 4: Clean up.............................................................................................................. 28 Next steps....................................................................................................................... 28 Getting started using the classic console ..................................................................................... 28 Using the classic console with Linux containers on Amazon Fargate ........................................ 29 Using the classic console with Windows containers on Amazon Fargate ................................... 32 Using the classic console with Amazon EC2 ......................................................................... 36 Using the classic console with Windows containers ............................................................... 39 Developer tools overview.................................................................................................................. 44 Amazon Web Services Management Console ................................................................................ 44 Amazon Command Line Interface ............................................................................................... 44 Amazon CloudFormation........................................................................................................... 45 Amazon Copilot CLI .................................................................................................................. 45 Amazon CDK............................................................................................................................ 45 Amazon App2Container............................................................................................................ 46 Docker Desktop integration with Amazon ECS ............................................................................. 46 Amazon SDKs.......................................................................................................................... 46 Summary................................................................................................................................ 47 iii Amazon Elastic Container Service Developer Guide Using the Amazon Copilot CLI ................................................................................................... 47 Installing the Amazon Copilot CLI ...................................................................................... 48 Next steps....................................................................................................................... 53 Amazon Fargate............................................................................................................................... 54 Task definitions........................................................................................................................ 54 Network mode................................................................................................................. 55 Task Operating Systems .................................................................................................... 55 Task CPU architecture ....................................................................................................... 55 Task CPU and memory ...................................................................................................... 55 Task resource limits .......................................................................................................... 56 Logging........................................................................................................................... 57 Amazon ECS task execution IAM role .................................................................................. 57 Example Amazon Linux 2 task definition ............................................................................. 57 Example Windows task definition ....................................................................................... 58 Task storage.................................................................................................................... 59 Tasks and services .................................................................................................................... 59 Task networking ............................................................................................................... 59 Service load balancing ...................................................................................................... 60 Private registry authentication................................................................................................... 60 Clusters................................................................................................................................... 60 Fargate Spot............................................................................................................................ 61 Usage metrics.......................................................................................................................... 61 Task maintenance..................................................................................................................... 61 Savings plans........................................................................................................................... 62 Windows containers on Amazon Fargate considerations................................................................ 62 Platform Versions..................................................................................................................... 63 Linux platform versions ..................................................................................................... 63 Windows platform versions ............................................................................................... 67 Getting started walkthroughs.................................................................................................... 67 New Amazon Elastic Container Service console .................................................................................... 69 Getting started using the new Amazon ECS console ..................................................................... 69 Using the console with Linux containers on Amazon Fargate .................................................. 69 Using the console with Windows containers on Amazon Fargate ............................................. 72 Using the console with Amazon EC2 ................................................................................... 75 Cluster management in the new Amazon ECS console ................................................................... 79 Creating a cluster for the Fargate launch type using the new console ...................................... 79 Creating a cluster for the Amazon EC2 launch type using the new console ............................... 81 Creating a capacity provider using the new console.............................................................. 83 Updating a capacity provider using the new console............................................................. 83 Deleting a capacity provider using the new console .............................................................. 84 Deleting a cluster using the new console ............................................................................. 84 Task definition management in the new Amazon ECS console ........................................................ 85 Creating a task definition using the new console .................................................................. 85 Updating a task definition using the new console ................................................................. 90 Deregistering a task definition revision using the new console ................................................ 91 Task management in the new Amazon ECS console ...................................................................... 92 Service management in the new Amazon ECS console .................................................................. 92 Clusters........................................................................................................................................... 93 Cluster concepts....................................................................................................................... 93 Creating a cluster using the classic console .................................................................................. 94 Capacity providers.................................................................................................................... 97 Capacity provider concepts ................................................................................................ 97 Capacity provider types ..................................................................................................... 98 Capacity provider considerations ........................................................................................ 98 Amazon Fargate capacity providers ..................................................................................... 99 Auto Scaling group capacity providers ............................................................................... 103 Cluster auto scaling................................................................................................................ 109 iv Amazon Elastic Container Service Developer Guide How cluster Auto Scaling works ....................................................................................... 109 Managed termination protection ...................................................................................... 111 Managed scale-out behavior ............................................................................................ 111 Managed scale-in behavior .............................................................................................. 113 Target tracking considerations .......................................................................................... 113 Update on the way Amazon ECS creates resources for cluster auto scaling .............................. 114 Turn on cluster Auto Scaling ............................................................................................ 114 Turn off cluster auto scaling ............................................................................................ 115 Using Local Zones, Wavelength Zones, and Amazon Outposts ...................................................... 116 Local Zones................................................................................................................... 116 Wavelength Zones.......................................................................................................... 117 Amazon Outposts........................................................................................................... 117 Updating cluster settings ......................................................................................................... 117 Deleting a cluster using the classic console ................................................................................ 118 Task definitions.............................................................................................................................. 119 Amazon EC2 Windows task definition considerations ................................................................... 120 Additional configuration for Windows IAM roles for tasks .................................................... 120 Application architecture........................................................................................................... 121 Using the Fargate launch type ......................................................................................... 121 Using the EC2 launch type .............................................................................................. 121 Creating a task definition using the classic console ..................................................................... 122 Task definition template .................................................................................................. 130 Task definition parameters ...................................................................................................... 134 Family........................................................................................................................... 134 Launch types................................................................................................................. 134 Task role....................................................................................................................... 135 Task execution role ......................................................................................................... 135 Network mode............................................................................................................... 135 Runtime platform........................................................................................................... 136 Task size........................................................................................................................ 137 Container definitions....................................................................................................... 139 Elastic Inference accelerator name .................................................................................... 167 Task placement constraints .............................................................................................. 168 Proxy configuration......................................................................................................... 168 Volumes........................................................................................................................ 170 Tags.............................................................................................................................. 174 Other task definition parameters...................................................................................... 175 Launch types......................................................................................................................... 176 Fargate launch type ........................................................................................................ 176 EC2 launch type ............................................................................................................. 177 External launch type ....................................................................................................... 178 Working with GPUs on Amazon ECS ......................................................................................... 179 Considerations................................................................................................................ 180 Specifying GPUs in your task definition ............................................................................. 181 Using video transcoding on Amazon ECS ................................................................................... 182 Considerations................................................................................................................ 182 Using a VT1 AMI ............................................................................................................ 182 Task definition requirements ............................................................................................ 183 Using machine learning on Amazon ECS .................................................................................... 190 Using Amazon Neuron on Amazon Linux 2 on Amazon ECS .................................................. 190 Using deep learning DL1 instances on Amazon ECS ............................................................. 193 Working with 64-bit ARM workloads on Amazon ECS .................................................................. 195 Considerations................................................................................................................ 195 Specifying the ARM architecture in your task definition ....................................................... 196 Interfaces for configuring ARM ......................................................................................... 197 Using data volumes in tasks .................................................................................................... 197 Fargate task storage ....................................................................................................... 198 v Amazon Elastic Container Service Developer Guide Amazon EFS volumes ...................................................................................................... 199 FSx for Windows File Server volumes ................................................................................ 202 Docker volumes.............................................................................................................. 206 Bind mounts.................................................................................................................. 211 Managing container swap space ............................................................................................... 220 Container swap considerations ......................................................................................... 221 Task networking..................................................................................................................... 221 AWSVPC mode............................................................................................................... 223 Bridge mode.................................................................................................................. 227 Host mode..................................................................................................................... 227 Using the awslogs log driver .................................................................................................... 227 Turning on the awslogs log driver for your containers ......................................................... 227 Creating a log group ....................................................................................................... 228 Available awslogs log driver options ................................................................................. 229 Specifying a log configuration in your task definition .......................................................... 231 Viewing awslogs container logs in CloudWatch Logs ........................................................... 232 Custom log routing ................................................................................................................. 233 Considerations................................................................................................................ 234 Required IAM permissions ................................................................................................ 234 Fluentd buffer limit........................................................................................................ 236 Using Fluent logger libraries or Log4j over TCP .................................................................. 237 Using the Amazon for Fluent Bit image ............................................................................. 237 Creating a task definition that uses a FireLens configuration ................................................ 239 Filtering logs using regular expressions ............................................................................. 242 Concatenate multiline or stack-trace log messages ............................................................. 243 Example task definitions.................................................................................................. 258 Private registry authentication for tasks .................................................................................... 263 Required IAM permissions for private registry authentication ................................................ 263 Enabling private registry authentication ............................................................................ 264 Passing environment variables ................................................................................................. 265 Considerations for specifying environment variable files ...................................................... 267 Required IAM permissions ................................................................................................ 267 Passing sensitive data ............................................................................................................. 268 Using Secrets Manager .................................................................................................... 268 Using Systems Manager Parameter Store ........................................................................... 275 Example task definitions.......................................................................................................... 280 Example: Webserver........................................................................................................ 280 Example: splunk log driver ............................................................................................. 282 Example: fluentd log driver ........................................................................................... 282 Example: gelf log driver ................................................................................................ 282 Example: Amazon ECR image and task definition IAM role ................................................... 283 Example: Entrypoint with command ................................................................................. 283 Example: Container dependency ....................................................................................... 284 Windows sample task definitions ...................................................................................... 285 Updating a task definition using the classic console .................................................................... 286 Deregistering a task definition revision ...................................................................................... 286 Account settings............................................................................................................................. 288 Amazon Resource Names (ARNs) and IDs ................................................................................... 289 ARN and resource ID format timeline ........................................................................................ 290 Viewing account settings ......................................................................................................... 291 Modifying account settings ...................................................................................................... 292 Reverting to the default account settings .................................................................................. 293 Container instances......................................................................................................................... 295 Container instance concepts ..................................................................................................... 295 Container instance lifecycle ...................................................................................................... 296 Check the instance IAM role for your account ............................................................................. 297 Linux instances....................................................................................................................... 297 vi Amazon Elastic Container Service Developer Guide Amazon ECS-optimized AMI ............................................................................................. 298 Launching a container instance ........................................................................................ 325 Bootstrap Container Instances .......................................................................................... 333 Starting a task at container instance launch time ................................................................ 335 Elastic network interface trunking .................................................................................... 337 Memory Management..................................................................................................... 353 Connect to your container instance using the classic console ................................................ 355 Manage container instances remotely ................................................................................ 356 Windows instances................................................................................................................. 357 Amazon ECS-optimized AMI ............................................................................................. 358 Launching a container instance ........................................................................................ 376 Bootstrap Container Instances .......................................................................................... 382 Connect to your container Windows instance ..................................................................... 384 Deregister a container instance ........................................................................................ 385 External instances................................................................................................................... 386 Supported operating systems and system architectures ....................................................... 387 Considerations................................................................................................................ 388 IAM permissions............................................................................................................. 390 Registering an external instance to a cluster ...................................................................... 392 Deregistering an external instance .................................................................................... 396 Running workloads on external instances .......................................................................... 398 Updating the Amazon Systems Manager Agent and Amazon ECS container agent .................... 400 Monitoring............................................................................................................................. 403 CloudWatch Logs IAM Policy ............................................................................................ 403 Installing and configuring the CloudWatch agent ................................................................ 404 Viewing CloudWatch Logs ............................................................................................... 404 Container instance draining ..................................................................................................... 404 Draining behavior for services .......................................................................................... 404 Draining behavior for standalone tasks ............................................................................. 405 Draining container instances ............................................................................................ 405 Container agent............................................................................................................................. 407 Installing the Amazon ECS container agent ................................................................................ 407 Installing the Amazon ECS container agent on an Amazon Linux 2 EC2 instance ...................... 408 Installing the Amazon ECS container agent on an Amazon Linux AMI EC2 instance ................... 408 Installing the Amazon ECS container agent on a non-Amazon Linux EC2 instance .................... 409 Running the Amazon ECS agent with host network mode .................................................... 412 Container agent versions ......................................................................................................... 413 Amazon ECS-Optimized Amazon Linux 2 AMI container agent versions .................................. 413 Amazon ECS-Optimized Amazon Linux AMI container agent versions ..................................... 417 Amazon EC2 Windows containers ............................................................................................. 422 Windows container caveats .............................................................................................. 422 Getting started with Windows containers .......................................................................... 423 Updating the Amazon ECS container agent ................................................................................ 423 Checking the Amazon ECS container agent version ............................................................. 424 Updating the Amazon ECS container agent on an Amazon ECS-optimized AMI ........................ 425 Manually updating the Amazon ECS container agent (for non-Amazon ECS-Optimized AMIs) ..... 428 Container agent configuration .................................................................................................. 430 Available Parameters....................................................................................................... 430 Storing container instance configuration in Amazon S3 ....................................................... 444 Private registry authentication for container instances ................................................................. 445 Authentication formats................................................................................................... 445 Enabling private registries ............................................................................................... 447 Automated task and image cleanup.......................................................................................... 448 Tunable parameters........................................................................................................ 448 Cleanup workflow........................................................................................................... 449 Container metadata file........................................................................................................... 449 Turning on container metadata ........................................................................................ 449 vii Amazon Elastic Container Service Developer Guide Container metadata file locations..................................................................................... 450 Container metadata file format........................................................................................ 450 Task metadata endpoint .......................................................................................................... 453 Task metadata endpoint version 4 .................................................................................... 453 Task Metadata Endpoint version 3 .................................................................................... 470 Task Metadata Endpoint version 2 .................................................................................... 475 Container agent endpoint........................................................................................................ 480 Task scale-in protection endpoint ..................................................................................... 480 Container agent introspection .................................................................................................. 483 HTTP proxy configuration ........................................................................................................ 485 Amazon Linux container instance configuration .................................................................. 485 Windows container instance configuration ......................................................................... 488 Using gMSAs for Windows Containers ....................................................................................... 489 Considerations................................................................................................................ 489 Prerequisites.................................................................................................................. 489 Setting Up gMSA-capable Windows Containers on Amazon ECS ............................................ 490 Scheduling tasks............................................................................................................................. 493 Running a standalone task using the new Amazon ECS console .................................................... 494 Stopping tasks using the new console ....................................................................................... 498 Run a standalone task in the classic Amazon ECS console ............................................................ 498 Task placement...................................................................................................................... 501 Task groups................................................................................................................... 501 Task placement strategies ................................................................................................ 502 Task placement constraints .............................................................................................. 504 Cluster query language ................................................................................................... 508 Scheduled tasks..................................................................................................................... 512 Create a scheduled task .................................................................................................. 512 View your scheduled tasks in the classic console ................................................................. 516 Edit a scheduled task ...................................................................................................... 516 Task lifecycle......................................................................................................................... 516 Lifecycle states............................................................................................................... 517 Creating a scheduled task using the Amazon CLI ........................................................................ 518 Services......................................................................................................................................... 520 Service scheduler concepts ...................................................................................................... 520 Daemon........................................................................................................................ 521 Replica.......................................................................................................................... 522 Additional service concepts ...................................................................................................... 522 Service definition parameters ................................................................................................... 523 Launch type................................................................................................................... 523 Capacity provider strategy ............................................................................................... 523 Task definition............................................................................................................... 524 Platform operating system .............................................................................................. 525 Platform version............................................................................................................. 525 Cluster.......................................................................................................................... 525 Service name................................................................................................................. 526 Scheduling strategy........................................................................................................ 526 Desired count................................................................................................................. 526 Deployment configuration............................................................................................... 527 Deployment controller.................................................................................................... 528 Task placement.............................................................................................................. 529 Tags.............................................................................................................................. 530 Network configuration.................................................................................................... 531 Client token................................................................................................................... 537 Service definition template .............................................................................................. 537 Service management in the Amazon ECS console ....................................................................... 538 New Amazon ECS console ............................................................................................... 538 Classic Amazon ECS console ............................................................................................ 550 viii Amazon Elastic Container Service Developer Guide Deployment types.................................................................................................................. 564 Rolling update............................................................................................................... 564 Blue/Green deployment with CodeDeploy ......................................................................... 569 External deployment....................................................................................................... 573 Service load balancing ............................................................................................................ 578 Load balancer types ........................................................................................................ 579 Creating a load balancer ................................................................................................. 582 Registering multiple target groups with a service ............................................................... 587 Service auto scaling ................................................................................................................ 589 Service auto scaling and deployments ............................................................................... 590 IAM permissions required for service auto scaling ............................................................... 590 Considerations................................................................................................................ 591 Amazon CLI and SDK experience ...................................................................................... 592 Target tracking scaling policies ......................................................................................... 592 Step scaling policies........................................................................................................ 593 Interconnecting services.......................................................................................................... 594 Choosing an interconnection method ................................................................................ 594 Network mode compatibility table .................................................................................... 595 Service Connect.............................................................................................................. 595 Service discovery............................................................................................................ 612 Task scale-in protection ........................................................................................................... 615 Task scale-in protection mechanisms ................................................................................. 615 Task scale-in protection considerations .............................................................................. 616 IAM permissions required for task scale-in protection .......................................................... 617 Service throttle logic ............................................................................................................... 617 Resources and tags......................................................................................................................... 619 Tagging your resources ........................................................................................................... 619 Tag basics...................................................................................................................... 619 Tagging your resources ................................................................................................... 620 Tag restrictions.............................................................................................................. 621 Tagging your resources for billing ..................................................................................... 621 Working with tags using the console ................................................................................. 622 Working with tags using the CLI or API ............................................................................. 624 Service quotas........................................................................................................................ 626 Amazon ECS service quotas ............................................................................................. 626 Amazon Fargate service quotas ........................................................................................ 629 Managing your Amazon ECS and Amazon Fargate service quotas .......................................... 629 Amazon Fargate Regions ......................................................................................................... 629 Supported Regions for Linux containers on Amazon Fargate ................................................ 629 Supported Regions for Windows containers on Amazon Fargate ........................................... 631 Usage Reports........................................................................................................................ 631 Monitoring..................................................................................................................................... 633 Monitoring tools..................................................................................................................... 634 Automated Tools............................................................................................................ 634 Manual Tools................................................................................................................. 634 CloudWatch metrics................................................................................................................ 635 Using CloudWatch metrics ............................................................................................... 635 Available metrics and dimensions ..................................................................................... 636 Cluster reservation.......................................................................................................... 642 Cluster utilization........................................................................................................... 643 Service utilization........................................................................................................... 644 Service RUNNING task count ............................................................................................ 645 Viewing Amazon ECS metrics ........................................................................................... 646 Events and EventBridge ........................................................................................................... 647 Amazon ECS events ........................................................................................................ 648 Handling events............................................................................................................. 659 CloudWatch Container Insights ................................................................................................. 661 ix Amazon Elastic Container Service Developer Guide Container Insights considerations ..................................................................................... 661 Setting up CloudWatch Container Insights for cluster and service level metrics ........................ 662 To change the default for Container Insights for all users using the console ............................ 662 To change the default for Container Insights for all users using the command line ................... 662 To change the default for Container Insights for a specific user using the command line ........... 663 To turn on Container Insights for a specific cluster using the command line ............................ 663 Use CloudWatch Container Insights to view Amazon ECS lifecycle events ............................... 663 Container instance health ........................................................................................................ 665 Collecting application trace data .............................................................................................. 665 Required IAM permissions for Amazon Distro for OpenTelemetry integration with Amazon X- Ray............................................................................................................................... 666 Specifying the Amazon Distro for OpenTelemetry sidecar for Amazon X-Ray integration in your task definition................................................................................................................ 667 Collecting application metrics .................................................................................................. 668 Exporting application metrics to Amazon CloudWatch ......................................................... 668 Exporting application metrics to Amazon Managed Service for Prometheus ............................ 671 Logging Amazon ECS API calls with Amazon CloudTrail ............................................................... 673 Amazon ECS information in CloudTrail .............................................................................. 673 Understanding Amazon ECS log file entries ....................................................................... 674 Security......................................................................................................................................... 676 Identity and Access Management .............................................................................................. 676 Audience....................................................................................................................... 677 Authenticating with identities.......................................................................................... 677 Managing access using policies ......................................................................................... 679 How Amazon Elastic Container Service works with IAM ........................................................ 681 Identity-based policy examples ........................................................................................ 688 Amazon managed policies for Amazon ECS ........................................................................ 700 Using service-linked roles ................................................................................................ 712 Task execution IAM role .................................................................................................. 714 Container instance IAM role ............................................................................................. 718 ECS Anywhere IAM role ................................................................................................... 721 IAM roles for tasks ......................................................................................................... 723 CodeDeploy IAM Role ...................................................................................................... 729 CloudWatch Events IAM Role ........................................................................................... 733 Additional configuration for Windows IAM roles for tasks .................................................... 736 Troubleshooting............................................................................................................. 737 Logging and Monitoring.......................................................................................................... 739 Compliance validation............................................................................................................. 740 Infrastructure Security............................................................................................................. 740 Interface VPC endpoints (Amazon PrivateLink) ................................................................... 741 Working with other services ............................................................................................................. 744 Using Amazon ECR with Amazon ECS ........................................................................................ 744 Using Amazon ECR Images with Amazon ECS ..................................................................... 744 Creating Amazon ECS resources with Amazon CloudFormation ..................................................... 745 Amazon ECS and Amazon CloudFormation templates .......................................................... 745 Example templates......................................................................................................... 745 Using the Amazon CLI to create resources from templates ................................................... 750 Learn more about Amazon CloudFormation ....................................................................... 751 Amazon Elastic Container Service on Amazon Outposts ............................................................... 751 Prerequisites.................................................................................................................. 751 Limitations..................................................................................................................... 751 Network Connectivity Considerations ................................................................................ 751 Creating an Amazon ECS Cluster on an Amazon Outposts .................................................... 752 Use App Mesh with Amazon ECS .............................................................................................. 754 Tutorials........................................................................................................................................ 755 Tutorial: Creating a VPC .......................................................................................................... 755 Step 1: Create an Elastic IP Address for Your NAT Gateway .................................................. 755 x

Description:
Adding Amazon S3 Read-only Access to your Container Instance Role . The following diagram shows the architecture of an Amazon ECS environment using the Fargate launch type: API Version an additional 22-GiB volume that is attached at /dev/xvdcz that Docker uses for image and metadata.
ebook img

Amazon Elastic Container Service - Developer Guide PDF

497 Pages·2017·4.91 MB·English
by  Amazon Web Services
Save to my drive
Quick download
Download
Upgrade Premium
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Amazon Elastic Container Service - Developer Guide

Amazon Elastic Container Service Developer Guide Amazon Elastic Container Service Developer Guide Amazon Elastic Container Service: Developer Guide Amazon Elastic Container Service Developer Guide Table of Contents What is Amazon ECS?......................................................................................................................... 1 Launch types............................................................................................................................. 1 Access Amazon ECS.................................................................................................................... 2 Pricing...................................................................................................................................... 2 Amazon ECS components............................................................................................................ 3 Clusters............................................................................................................................. 3 Containers and images....................................................................................................... 3 Task definitions.................................................................................................................. 3 Tasks................................................................................................................................ 4 Services............................................................................................................................ 4 Container agent................................................................................................................. 4 Fargate architecture overview .............................................................................................. 5 Common use cases..................................................................................................................... 6 Additional resources........................................................................................................... 6 Related services......................................................................................................................... 6 Getting started .................................................................................................................................. 8 Set up...................................................................................................................................... 8 Sign up for an Amazon Web Services account ........................................................................ 8 Secure IAM users ................................................................................................................ 8 Create the credentials to connect to your EC2 instance ........................................................... 9 Create a virtual private cloud ............................................................................................. 10 Create a security group ..................................................................................................... 10 Install the Amazon CLI ...................................................................................................... 12 Creating a container image........................................................................................................ 12 Prerequisites.................................................................................................................... 13 Create a Docker image ...................................................................................................... 14 Push your image to Amazon Elastic Container Registry ......................................................... 16 Clean up......................................................................................................................... 17 Next steps....................................................................................................................... 17 Using Amazon Copilot .............................................................................................................. 17 Prerequisites.................................................................................................................... 17 Deploy your application using one command ....................................................................... 17 Deploy your application step by step .................................................................................. 18 Using the Amazon CDK............................................................................................................. 21 Step 1: Set up your Amazon CDK project ............................................................................ 22 Step 2: Use the Amazon CDK to define a containerized web server on Fargate .......................... 24 Step 3: Test the web server ............................................................................................... 27 Step 4: Clean up.............................................................................................................. 28 Next steps....................................................................................................................... 28 Getting started using the classic console ..................................................................................... 28 Using the classic console with Linux containers on Amazon Fargate ........................................ 29 Using the classic console with Windows containers on Amazon Fargate ................................... 32 Using the classic console with Amazon EC2 ......................................................................... 36 Using the classic console with Windows containers ............................................................... 39 Developer tools overview.................................................................................................................. 44 Amazon Web Services Management Console ................................................................................ 44 Amazon Command Line Interface ............................................................................................... 44 Amazon CloudFormation........................................................................................................... 45 Amazon Copilot CLI .................................................................................................................. 45 Amazon CDK............................................................................................................................ 45 Amazon App2Container............................................................................................................ 46 Docker Desktop integration with Amazon ECS ............................................................................. 46 Amazon SDKs.......................................................................................................................... 46 Summary................................................................................................................................ 47 iii Amazon Elastic Container Service Developer Guide Using the Amazon Copilot CLI ................................................................................................... 47 Installing the Amazon Copilot CLI ...................................................................................... 48 Next steps....................................................................................................................... 53 Amazon Fargate............................................................................................................................... 54 Task definitions........................................................................................................................ 54 Network mode................................................................................................................. 55 Task Operating Systems .................................................................................................... 55 Task CPU architecture ....................................................................................................... 55 Task CPU and memory ...................................................................................................... 55 Task resource limits .......................................................................................................... 56 Logging........................................................................................................................... 57 Amazon ECS task execution IAM role .................................................................................. 57 Example Amazon Linux 2 task definition ............................................................................. 57 Example Windows task definition ....................................................................................... 58 Task storage.................................................................................................................... 59 Tasks and services .................................................................................................................... 59 Task networking ............................................................................................................... 59 Service load balancing ...................................................................................................... 60 Private registry authentication................................................................................................... 60 Clusters................................................................................................................................... 60 Fargate Spot............................................................................................................................ 61 Usage metrics.......................................................................................................................... 61 Task maintenance..................................................................................................................... 61 Savings plans........................................................................................................................... 62 Windows containers on Amazon Fargate considerations................................................................ 62 Platform Versions..................................................................................................................... 63 Linux platform versions ..................................................................................................... 63 Windows platform versions ............................................................................................... 67 Getting started walkthroughs.................................................................................................... 67 New Amazon Elastic Container Service console .................................................................................... 69 Getting started using the new Amazon ECS console ..................................................................... 69 Using the console with Linux containers on Amazon Fargate .................................................. 69 Using the console with Windows containers on Amazon Fargate ............................................. 72 Using the console with Amazon EC2 ................................................................................... 75 Cluster management in the new Amazon ECS console ................................................................... 79 Creating a cluster for the Fargate launch type using the new console ...................................... 79 Creating a cluster for the Amazon EC2 launch type using the new console ............................... 81 Creating a capacity provider using the new console.............................................................. 83 Updating a capacity provider using the new console............................................................. 83 Deleting a capacity provider using the new console .............................................................. 84 Deleting a cluster using the new console ............................................................................. 84 Task definition management in the new Amazon ECS console ........................................................ 85 Creating a task definition using the new console .................................................................. 85 Updating a task definition using the new console ................................................................. 90 Deregistering a task definition revision using the new console ................................................ 91 Task management in the new Amazon ECS console ...................................................................... 92 Service management in the new Amazon ECS console .................................................................. 92 Clusters........................................................................................................................................... 93 Cluster concepts....................................................................................................................... 93 Creating a cluster using the classic console .................................................................................. 94 Capacity providers.................................................................................................................... 97 Capacity provider concepts ................................................................................................ 97 Capacity provider types ..................................................................................................... 98 Capacity provider considerations ........................................................................................ 98 Amazon Fargate capacity providers ..................................................................................... 99 Auto Scaling group capacity providers ............................................................................... 103 Cluster auto scaling................................................................................................................ 109 iv Amazon Elastic Container Service Developer Guide How cluster Auto Scaling works ....................................................................................... 109 Managed termination protection ...................................................................................... 111 Managed scale-out behavior ............................................................................................ 111 Managed scale-in behavior .............................................................................................. 113 Target tracking considerations .......................................................................................... 113 Update on the way Amazon ECS creates resources for cluster auto scaling .............................. 114 Turn on cluster Auto Scaling ............................................................................................ 114 Turn off cluster auto scaling ............................................................................................ 115 Using Local Zones, Wavelength Zones, and Amazon Outposts ...................................................... 116 Local Zones................................................................................................................... 116 Wavelength Zones.......................................................................................................... 117 Amazon Outposts........................................................................................................... 117 Updating cluster settings ......................................................................................................... 117 Deleting a cluster using the classic console ................................................................................ 118 Task definitions.............................................................................................................................. 119 Amazon EC2 Windows task definition considerations ................................................................... 120 Additional configuration for Windows IAM roles for tasks .................................................... 120 Application architecture........................................................................................................... 121 Using the Fargate launch type ......................................................................................... 121 Using the EC2 launch type .............................................................................................. 121 Creating a task definition using the classic console ..................................................................... 122 Task definition template .................................................................................................. 130 Task definition parameters ...................................................................................................... 134 Family........................................................................................................................... 134 Launch types................................................................................................................. 134 Task role....................................................................................................................... 135 Task execution role ......................................................................................................... 135 Network mode............................................................................................................... 135 Runtime platform........................................................................................................... 136 Task size........................................................................................................................ 137 Container definitions....................................................................................................... 139 Elastic Inference accelerator name .................................................................................... 167 Task placement constraints .............................................................................................. 168 Proxy configuration......................................................................................................... 168 Volumes........................................................................................................................ 170 Tags.............................................................................................................................. 174 Other task definition parameters...................................................................................... 175 Launch types......................................................................................................................... 176 Fargate launch type ........................................................................................................ 176 EC2 launch type ............................................................................................................. 177 External launch type ....................................................................................................... 178 Working with GPUs on Amazon ECS ......................................................................................... 179 Considerations................................................................................................................ 180 Specifying GPUs in your task definition ............................................................................. 181 Using video transcoding on Amazon ECS ................................................................................... 182 Considerations................................................................................................................ 182 Using a VT1 AMI ............................................................................................................ 182 Task definition requirements ............................................................................................ 183 Using machine learning on Amazon ECS .................................................................................... 190 Using Amazon Neuron on Amazon Linux 2 on Amazon ECS .................................................. 190 Using deep learning DL1 instances on Amazon ECS ............................................................. 193 Working with 64-bit ARM workloads on Amazon ECS .................................................................. 195 Considerations................................................................................................................ 195 Specifying the ARM architecture in your task definition ....................................................... 196 Interfaces for configuring ARM ......................................................................................... 197 Using data volumes in tasks .................................................................................................... 197 Fargate task storage ....................................................................................................... 198 v Amazon Elastic Container Service Developer Guide Amazon EFS volumes ...................................................................................................... 199 FSx for Windows File Server volumes ................................................................................ 202 Docker volumes.............................................................................................................. 206 Bind mounts.................................................................................................................. 211 Managing container swap space ............................................................................................... 220 Container swap considerations ......................................................................................... 221 Task networking..................................................................................................................... 221 AWSVPC mode............................................................................................................... 223 Bridge mode.................................................................................................................. 227 Host mode..................................................................................................................... 227 Using the awslogs log driver .................................................................................................... 227 Turning on the awslogs log driver for your containers ......................................................... 227 Creating a log group ....................................................................................................... 228 Available awslogs log driver options ................................................................................. 229 Specifying a log configuration in your task definition .......................................................... 231 Viewing awslogs container logs in CloudWatch Logs ........................................................... 232 Custom log routing ................................................................................................................. 233 Considerations................................................................................................................ 234 Required IAM permissions ................................................................................................ 234 Fluentd buffer limit........................................................................................................ 236 Using Fluent logger libraries or Log4j over TCP .................................................................. 237 Using the Amazon for Fluent Bit image ............................................................................. 237 Creating a task definition that uses a FireLens configuration ................................................ 239 Filtering logs using regular expressions ............................................................................. 242 Concatenate multiline or stack-trace log messages ............................................................. 243 Example task definitions.................................................................................................. 258 Private registry authentication for tasks .................................................................................... 263 Required IAM permissions for private registry authentication ................................................ 263 Enabling private registry authentication ............................................................................ 264 Passing environment variables ................................................................................................. 265 Considerations for specifying environment variable files ...................................................... 267 Required IAM permissions ................................................................................................ 267 Passing sensitive data ............................................................................................................. 268 Using Secrets Manager .................................................................................................... 268 Using Systems Manager Parameter Store ........................................................................... 275 Example task definitions.......................................................................................................... 280 Example: Webserver........................................................................................................ 280 Example: splunk log driver ............................................................................................. 282 Example: fluentd log driver ........................................................................................... 282 Example: gelf log driver ................................................................................................ 282 Example: Amazon ECR image and task definition IAM role ................................................... 283 Example: Entrypoint with command ................................................................................. 283 Example: Container dependency ....................................................................................... 284 Windows sample task definitions ...................................................................................... 285 Updating a task definition using the classic console .................................................................... 286 Deregistering a task definition revision ...................................................................................... 286 Account settings............................................................................................................................. 288 Amazon Resource Names (ARNs) and IDs ................................................................................... 289 ARN and resource ID format timeline ........................................................................................ 290 Viewing account settings ......................................................................................................... 291 Modifying account settings ...................................................................................................... 292 Reverting to the default account settings .................................................................................. 293 Container instances......................................................................................................................... 295 Container instance concepts ..................................................................................................... 295 Container instance lifecycle ...................................................................................................... 296 Check the instance IAM role for your account ............................................................................. 297 Linux instances....................................................................................................................... 297 vi Amazon Elastic Container Service Developer Guide Amazon ECS-optimized AMI ............................................................................................. 298 Launching a container instance ........................................................................................ 325 Bootstrap Container Instances .......................................................................................... 333 Starting a task at container instance launch time ................................................................ 335 Elastic network interface trunking .................................................................................... 337 Memory Management..................................................................................................... 353 Connect to your container instance using the classic console ................................................ 355 Manage container instances remotely ................................................................................ 356 Windows instances................................................................................................................. 357 Amazon ECS-optimized AMI ............................................................................................. 358 Launching a container instance ........................................................................................ 376 Bootstrap Container Instances .......................................................................................... 382 Connect to your container Windows instance ..................................................................... 384 Deregister a container instance ........................................................................................ 385 External instances................................................................................................................... 386 Supported operating systems and system architectures ....................................................... 387 Considerations................................................................................................................ 388 IAM permissions............................................................................................................. 390 Registering an external instance to a cluster ...................................................................... 392 Deregistering an external instance .................................................................................... 396 Running workloads on external instances .......................................................................... 398 Updating the Amazon Systems Manager Agent and Amazon ECS container agent .................... 400 Monitoring............................................................................................................................. 403 CloudWatch Logs IAM Policy ............................................................................................ 403 Installing and configuring the CloudWatch agent ................................................................ 404 Viewing CloudWatch Logs ............................................................................................... 404 Container instance draining ..................................................................................................... 404 Draining behavior for services .......................................................................................... 404 Draining behavior for standalone tasks ............................................................................. 405 Draining container instances ............................................................................................ 405 Container agent............................................................................................................................. 407 Installing the Amazon ECS container agent ................................................................................ 407 Installing the Amazon ECS container agent on an Amazon Linux 2 EC2 instance ...................... 408 Installing the Amazon ECS container agent on an Amazon Linux AMI EC2 instance ................... 408 Installing the Amazon ECS container agent on a non-Amazon Linux EC2 instance .................... 409 Running the Amazon ECS agent with host network mode .................................................... 412 Container agent versions ......................................................................................................... 413 Amazon ECS-Optimized Amazon Linux 2 AMI container agent versions .................................. 413 Amazon ECS-Optimized Amazon Linux AMI container agent versions ..................................... 417 Amazon EC2 Windows containers ............................................................................................. 422 Windows container caveats .............................................................................................. 422 Getting started with Windows containers .......................................................................... 423 Updating the Amazon ECS container agent ................................................................................ 423 Checking the Amazon ECS container agent version ............................................................. 424 Updating the Amazon ECS container agent on an Amazon ECS-optimized AMI ........................ 425 Manually updating the Amazon ECS container agent (for non-Amazon ECS-Optimized AMIs) ..... 428 Container agent configuration .................................................................................................. 430 Available Parameters....................................................................................................... 430 Storing container instance configuration in Amazon S3 ....................................................... 444 Private registry authentication for container instances ................................................................. 445 Authentication formats................................................................................................... 445 Enabling private registries ............................................................................................... 447 Automated task and image cleanup.......................................................................................... 448 Tunable parameters........................................................................................................ 448 Cleanup workflow........................................................................................................... 449 Container metadata file........................................................................................................... 449 Turning on container metadata ........................................................................................ 449 vii Amazon Elastic Container Service Developer Guide Container metadata file locations..................................................................................... 450 Container metadata file format........................................................................................ 450 Task metadata endpoint .......................................................................................................... 453 Task metadata endpoint version 4 .................................................................................... 453 Task Metadata Endpoint version 3 .................................................................................... 470 Task Metadata Endpoint version 2 .................................................................................... 475 Container agent endpoint........................................................................................................ 480 Task scale-in protection endpoint ..................................................................................... 480 Container agent introspection .................................................................................................. 483 HTTP proxy configuration ........................................................................................................ 485 Amazon Linux container instance configuration .................................................................. 485 Windows container instance configuration ......................................................................... 488 Using gMSAs for Windows Containers ....................................................................................... 489 Considerations................................................................................................................ 489 Prerequisites.................................................................................................................. 489 Setting Up gMSA-capable Windows Containers on Amazon ECS ............................................ 490 Scheduling tasks............................................................................................................................. 493 Running a standalone task using the new Amazon ECS console .................................................... 494 Stopping tasks using the new console ....................................................................................... 498 Run a standalone task in the classic Amazon ECS console ............................................................ 498 Task placement...................................................................................................................... 501 Task groups................................................................................................................... 501 Task placement strategies ................................................................................................ 502 Task placement constraints .............................................................................................. 504 Cluster query language ................................................................................................... 508 Scheduled tasks..................................................................................................................... 512 Create a scheduled task .................................................................................................. 512 View your scheduled tasks in the classic console ................................................................. 516 Edit a scheduled task ...................................................................................................... 516 Task lifecycle......................................................................................................................... 516 Lifecycle states............................................................................................................... 517 Creating a scheduled task using the Amazon CLI ........................................................................ 518 Services......................................................................................................................................... 520 Service scheduler concepts ...................................................................................................... 520 Daemon........................................................................................................................ 521 Replica.......................................................................................................................... 522 Additional service concepts ...................................................................................................... 522 Service definition parameters ................................................................................................... 523 Launch type................................................................................................................... 523 Capacity provider strategy ............................................................................................... 523 Task definition............................................................................................................... 524 Platform operating system .............................................................................................. 525 Platform version............................................................................................................. 525 Cluster.......................................................................................................................... 525 Service name................................................................................................................. 526 Scheduling strategy........................................................................................................ 526 Desired count................................................................................................................. 526 Deployment configuration............................................................................................... 527 Deployment controller.................................................................................................... 528 Task placement.............................................................................................................. 529 Tags.............................................................................................................................. 530 Network configuration.................................................................................................... 531 Client token................................................................................................................... 537 Service definition template .............................................................................................. 537 Service management in the Amazon ECS console ....................................................................... 538 New Amazon ECS console ............................................................................................... 538 Classic Amazon ECS console ............................................................................................ 550 viii Amazon Elastic Container Service Developer Guide Deployment types.................................................................................................................. 564 Rolling update............................................................................................................... 564 Blue/Green deployment with CodeDeploy ......................................................................... 569 External deployment....................................................................................................... 573 Service load balancing ............................................................................................................ 578 Load balancer types ........................................................................................................ 579 Creating a load balancer ................................................................................................. 582 Registering multiple target groups with a service ............................................................... 587 Service auto scaling ................................................................................................................ 589 Service auto scaling and deployments ............................................................................... 590 IAM permissions required for service auto scaling ............................................................... 590 Considerations................................................................................................................ 591 Amazon CLI and SDK experience ...................................................................................... 592 Target tracking scaling policies ......................................................................................... 592 Step scaling policies........................................................................................................ 593 Interconnecting services.......................................................................................................... 594 Choosing an interconnection method ................................................................................ 594 Network mode compatibility table .................................................................................... 595 Service Connect.............................................................................................................. 595 Service discovery............................................................................................................ 612 Task scale-in protection ........................................................................................................... 615 Task scale-in protection mechanisms ................................................................................. 615 Task scale-in protection considerations .............................................................................. 616 IAM permissions required for task scale-in protection .......................................................... 617 Service throttle logic ............................................................................................................... 617 Resources and tags......................................................................................................................... 619 Tagging your resources ........................................................................................................... 619 Tag basics...................................................................................................................... 619 Tagging your resources ................................................................................................... 620 Tag restrictions.............................................................................................................. 621 Tagging your resources for billing ..................................................................................... 621 Working with tags using the console ................................................................................. 622 Working with tags using the CLI or API ............................................................................. 624 Service quotas........................................................................................................................ 626 Amazon ECS service quotas ............................................................................................. 626 Amazon Fargate service quotas ........................................................................................ 629 Managing your Amazon ECS and Amazon Fargate service quotas .......................................... 629 Amazon Fargate Regions ......................................................................................................... 629 Supported Regions for Linux containers on Amazon Fargate ................................................ 629 Supported Regions for Windows containers on Amazon Fargate ........................................... 631 Usage Reports........................................................................................................................ 631 Monitoring..................................................................................................................................... 633 Monitoring tools..................................................................................................................... 634 Automated Tools............................................................................................................ 634 Manual Tools................................................................................................................. 634 CloudWatch metrics................................................................................................................ 635 Using CloudWatch metrics ............................................................................................... 635 Available metrics and dimensions ..................................................................................... 636 Cluster reservation.......................................................................................................... 642 Cluster utilization........................................................................................................... 643 Service utilization........................................................................................................... 644 Service RUNNING task count ............................................................................................ 645 Viewing Amazon ECS metrics ........................................................................................... 646 Events and EventBridge ........................................................................................................... 647 Amazon ECS events ........................................................................................................ 648 Handling events............................................................................................................. 659 CloudWatch Container Insights ................................................................................................. 661 ix Amazon Elastic Container Service Developer Guide Container Insights considerations ..................................................................................... 661 Setting up CloudWatch Container Insights for cluster and service level metrics ........................ 662 To change the default for Container Insights for all users using the console ............................ 662 To change the default for Container Insights for all users using the command line ................... 662 To change the default for Container Insights for a specific user using the command line ........... 663 To turn on Container Insights for a specific cluster using the command line ............................ 663 Use CloudWatch Container Insights to view Amazon ECS lifecycle events ............................... 663 Container instance health ........................................................................................................ 665 Collecting application trace data .............................................................................................. 665 Required IAM permissions for Amazon Distro for OpenTelemetry integration with Amazon X- Ray............................................................................................................................... 666 Specifying the Amazon Distro for OpenTelemetry sidecar for Amazon X-Ray integration in your task definition................................................................................................................ 667 Collecting application metrics .................................................................................................. 668 Exporting application metrics to Amazon CloudWatch ......................................................... 668 Exporting application metrics to Amazon Managed Service for Prometheus ............................ 671 Logging Amazon ECS API calls with Amazon CloudTrail ............................................................... 673 Amazon ECS information in CloudTrail .............................................................................. 673 Understanding Amazon ECS log file entries ....................................................................... 674 Security......................................................................................................................................... 676 Identity and Access Management .............................................................................................. 676 Audience....................................................................................................................... 677 Authenticating with identities.......................................................................................... 677 Managing access using policies ......................................................................................... 679 How Amazon Elastic Container Service works with IAM ........................................................ 681 Identity-based policy examples ........................................................................................ 688 Amazon managed policies for Amazon ECS ........................................................................ 700 Using service-linked roles ................................................................................................ 712 Task execution IAM role .................................................................................................. 714 Container instance IAM role ............................................................................................. 718 ECS Anywhere IAM role ................................................................................................... 721 IAM roles for tasks ......................................................................................................... 723 CodeDeploy IAM Role ...................................................................................................... 729 CloudWatch Events IAM Role ........................................................................................... 733 Additional configuration for Windows IAM roles for tasks .................................................... 736 Troubleshooting............................................................................................................. 737 Logging and Monitoring.......................................................................................................... 739 Compliance validation............................................................................................................. 740 Infrastructure Security............................................................................................................. 740 Interface VPC endpoints (Amazon PrivateLink) ................................................................... 741 Working with other services ............................................................................................................. 744 Using Amazon ECR with Amazon ECS ........................................................................................ 744 Using Amazon ECR Images with Amazon ECS ..................................................................... 744 Creating Amazon ECS resources with Amazon CloudFormation ..................................................... 745 Amazon ECS and Amazon CloudFormation templates .......................................................... 745 Example templates......................................................................................................... 745 Using the Amazon CLI to create resources from templates ................................................... 750 Learn more about Amazon CloudFormation ....................................................................... 751 Amazon Elastic Container Service on Amazon Outposts ............................................................... 751 Prerequisites.................................................................................................................. 751 Limitations..................................................................................................................... 751 Network Connectivity Considerations ................................................................................ 751 Creating an Amazon ECS Cluster on an Amazon Outposts .................................................... 752 Use App Mesh with Amazon ECS .............................................................................................. 754 Tutorials........................................................................................................................................ 755 Tutorial: Creating a VPC .......................................................................................................... 755 Step 1: Create an Elastic IP Address for Your NAT Gateway .................................................. 755 x

Description:
Adding Amazon S3 Read-only Access to your Container Instance Role . The following diagram shows the architecture of an Amazon ECS environment using the Fargate launch type: API Version an additional 22-GiB volume that is attached at /dev/xvdcz that Docker uses for image and metadata.
See more

The list of books you might like

Upgrade Premium
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.
DMCA & Copyright: Dear all, most of the website is community built, users are uploading hundred of books everyday, which makes really hard for us to identify copyrighted material, please contact us if you want any material removed.
Copyright @ PDFdrive.to, 2022 | We love our users