Table Of Content

Amazon Elastic Container Service Developer Guide Amazon Elastic Container Service Developer Guide Amazon Elastic Container Service: Developer Guide Copyright © 2022 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon. Amazon Elastic Container Service Developer Guide Table of Contents What is Amazon ECS?......................................................................................................................... 1 Launch types............................................................................................................................. 1 Access Amazon ECS.................................................................................................................... 2 Pricing...................................................................................................................................... 2 Amazon ECS components............................................................................................................ 3 Clusters............................................................................................................................. 3 Containers and images....................................................................................................... 3 Task definitions.................................................................................................................. 3 Tasks................................................................................................................................ 4 Services............................................................................................................................ 4 Container agent................................................................................................................. 4 Fargate architecture overview .............................................................................................. 5 Common use cases..................................................................................................................... 6 Additional resources........................................................................................................... 6 Related services......................................................................................................................... 6 Getting started .................................................................................................................................. 8 Set up...................................................................................................................................... 8 Sign up for an AWS account ................................................................................................ 8 Create an administrative user .............................................................................................. 8 Create the credentials to connect to your EC2 instance ........................................................... 9 Create a virtual private cloud ............................................................................................. 10 Create a security group ..................................................................................................... 11 Install the AWS CLI ........................................................................................................... 13 Creating a container image........................................................................................................ 13 Prerequisites.................................................................................................................... 13 Create a Docker image ...................................................................................................... 15 Push your image to Amazon Elastic Container Registry ......................................................... 16 Clean up......................................................................................................................... 17 Next steps....................................................................................................................... 17 Using AWS Copilot ................................................................................................................... 18 Prerequisites.................................................................................................................... 18 Deploy your application using one command ....................................................................... 18 Deploy your application step by step .................................................................................. 18 Using the AWS CDK.................................................................................................................. 22 Step 1: Set up your AWS CDK project ................................................................................. 23 Step 2: Use the AWS CDK to define a containerized web server on Fargate ............................... 24 Step 3: Test the web server ............................................................................................... 28 Step 4: Clean up.............................................................................................................. 28 Next steps....................................................................................................................... 28 Getting started using the classic console ..................................................................................... 29 Using the classic console with Linux containers on AWS Fargate ............................................. 29 Using the classic console with Windows containers on AWS Fargate ........................................ 32 Using the classic console with Amazon EC2 ......................................................................... 36 Using the classic console with Windows containers ............................................................... 40 Developer tools overview.................................................................................................................. 45 AWS Management Console ........................................................................................................ 45 AWS Command Line Interface .................................................................................................... 45 AWS CloudFormation................................................................................................................ 46 AWS Copilot CLI ....................................................................................................................... 46 AWS CDK................................................................................................................................ 46 AWS App2Container................................................................................................................. 47 Docker Desktop integration with Amazon ECS ............................................................................. 47 AWS SDKs............................................................................................................................... 47 Summary................................................................................................................................ 48 iii Amazon Elastic Container Service Developer Guide Using the AWS Copilot CLI ........................................................................................................ 48 Installing the AWS Copilot CLI ........................................................................................... 49 Next steps....................................................................................................................... 54 AWS Fargate.................................................................................................................................... 55 Task definitions........................................................................................................................ 55 Network mode................................................................................................................. 56 Task Operating Systems .................................................................................................... 56 Task CPU architecture ....................................................................................................... 56 Task CPU and memory ...................................................................................................... 56 Task resource limits .......................................................................................................... 57 Logging........................................................................................................................... 58 Amazon ECS task execution IAM role .................................................................................. 58 Example Amazon Linux 2 task definition ............................................................................. 58 Example Windows task definition ....................................................................................... 59 Task storage.................................................................................................................... 60 Tasks and services .................................................................................................................... 60 Task networking ............................................................................................................... 60 Service load balancing ...................................................................................................... 61 Private registry authentication................................................................................................... 61 Clusters................................................................................................................................... 61 Fargate Spot............................................................................................................................ 62 Usage metrics.......................................................................................................................... 62 Task maintenance..................................................................................................................... 62 Savings plans........................................................................................................................... 63 Windows containers on AWS Fargate considerations..................................................................... 63 Platform Versions..................................................................................................................... 64 Linux platform versions ..................................................................................................... 64 Windows platform versions ............................................................................................... 68 Getting started walkthroughs.................................................................................................... 68 New Amazon Elastic Container Service console .................................................................................... 70 Getting started using the new Amazon ECS console ..................................................................... 70 Using the console with Linux containers on AWS Fargate ....................................................... 70 Using the console with Windows containers on AWS Fargate .................................................. 73 Using the console with Amazon EC2 ................................................................................... 76 Cluster management in the new Amazon ECS console ................................................................... 80 Creating a cluster for the Fargate launch type using the new console ...................................... 80 Creating a cluster for the Amazon EC2 launch type using the new console ............................... 82 Creating a capacity provider using the new console.............................................................. 84 Updating a capacity provider using the new console............................................................. 84 Deleting a capacity provider using the new console .............................................................. 85 Deleting a cluster using the new console ............................................................................. 85 Task definition management in the new Amazon ECS console ........................................................ 86 Creating a task definition using the new console .................................................................. 86 Updating a task definition using the new console ................................................................. 91 Deregistering a task definition revision using the new console ................................................ 92 Task management in the new Amazon ECS console ...................................................................... 93 Service management in the new Amazon ECS console .................................................................. 93 Clusters........................................................................................................................................... 94 Cluster concepts....................................................................................................................... 94 Creating a cluster using the classic console .................................................................................. 95 Capacity providers.................................................................................................................... 98 Capacity provider concepts ................................................................................................ 98 Capacity provider types ..................................................................................................... 99 Capacity provider considerations ........................................................................................ 99 AWS Fargate capacity providers ........................................................................................ 100 Auto Scaling group capacity providers ............................................................................... 104 Cluster auto scaling................................................................................................................ 110 iv Amazon Elastic Container Service Developer Guide How cluster Auto Scaling works ....................................................................................... 110 Managed termination protection ...................................................................................... 112 Managed scale-out behavior ............................................................................................ 112 Managed scale-in behavior .............................................................................................. 114 Target tracking considerations .......................................................................................... 114 Update on the way Amazon ECS creates resources for cluster auto scaling .............................. 115 Turn on cluster Auto Scaling ............................................................................................ 115 Turn off cluster auto scaling ............................................................................................ 116 Using Local Zones, Wavelength Zones, and AWS Outposts ........................................................... 117 Local Zones................................................................................................................... 117 Wavelength Zones.......................................................................................................... 118 AWS Outposts................................................................................................................ 118 Updating cluster settings ......................................................................................................... 118 Deleting a cluster using the classic console ................................................................................ 119 Task definitions.............................................................................................................................. 120 Amazon EC2 Windows task definition considerations ................................................................... 121 Additional configuration for Windows IAM roles for tasks .................................................... 121 Application architecture........................................................................................................... 122 Using the Fargate launch type ......................................................................................... 122 Using the EC2 launch type .............................................................................................. 122 Creating a task definition using the classic console ..................................................................... 123 Task definition template .................................................................................................. 131 Task definition parameters ...................................................................................................... 135 Family........................................................................................................................... 135 Launch types................................................................................................................. 135 Task role....................................................................................................................... 136 Task execution role ......................................................................................................... 136 Network mode............................................................................................................... 136 Runtime platform........................................................................................................... 137 Task size........................................................................................................................ 138 Container definitions....................................................................................................... 140 Elastic Inference accelerator name .................................................................................... 168 Task placement constraints .............................................................................................. 168 Proxy configuration......................................................................................................... 169 Volumes........................................................................................................................ 170 Tags.............................................................................................................................. 174 Other task definition parameters...................................................................................... 175 Launch types......................................................................................................................... 176 Fargate launch type ........................................................................................................ 176 EC2 launch type ............................................................................................................. 177 External launch type ....................................................................................................... 178 Working with GPUs on Amazon ECS ......................................................................................... 179 Considerations................................................................................................................ 180 Specifying GPUs in your task definition ............................................................................. 181 Using video transcoding on Amazon ECS ................................................................................... 182 Considerations................................................................................................................ 182 Using a VT1 AMI ............................................................................................................ 182 Task definition requirements ............................................................................................ 183 Using machine learning on Amazon ECS .................................................................................... 190 Using AWS Neuron on Amazon Linux 2 on Amazon ECS ....................................................... 190 Using deep learning DL1 instances on Amazon ECS ............................................................. 194 Working with 64-bit ARM workloads on Amazon ECS .................................................................. 196 Considerations................................................................................................................ 196 Specifying the ARM architecture in your task definition ....................................................... 197 Interfaces for configuring ARM ......................................................................................... 198 Using data volumes in tasks .................................................................................................... 198 Fargate task storage ....................................................................................................... 199 v Amazon Elastic Container Service Developer Guide Amazon EFS volumes ...................................................................................................... 199 FSx for Windows File Server volumes ................................................................................ 203 Docker volumes.............................................................................................................. 206 Bind mounts.................................................................................................................. 212 Managing container swap space ............................................................................................... 221 Container swap considerations ......................................................................................... 222 Task networking..................................................................................................................... 222 AWSVPC mode............................................................................................................... 223 Bridge mode.................................................................................................................. 228 Host mode..................................................................................................................... 228 Using the awslogs log driver .................................................................................................... 228 Turning on the awslogs log driver for your containers ......................................................... 228 Creating a log group ....................................................................................................... 229 Available awslogs log driver options ................................................................................. 230 Specifying a log configuration in your task definition .......................................................... 231 Viewing awslogs container logs in CloudWatch Logs ........................................................... 232 Custom log routing ................................................................................................................. 234 Considerations................................................................................................................ 234 Required IAM permissions ................................................................................................ 235 Fluentd buffer limit........................................................................................................ 236 Using Fluent logger libraries or Log4j over TCP .................................................................. 237 Using the AWS for Fluent Bit image .................................................................................. 238 Creating a task definition that uses a FireLens configuration ................................................ 240 Filtering logs using regular expressions ............................................................................. 243 Concatenate multiline or stack-trace log messages ............................................................. 244 Example task definitions.................................................................................................. 259 Private registry authentication for tasks .................................................................................... 263 Required IAM permissions for private registry authentication ................................................ 264 Enabling private registry authentication ............................................................................ 265 Passing environment variables ................................................................................................. 266 Considerations for specifying environment variable files ...................................................... 267 Required IAM permissions ................................................................................................ 268 Passing sensitive data ............................................................................................................. 269 Using Secrets Manager .................................................................................................... 269 Using Systems Manager Parameter Store ........................................................................... 276 Example task definitions.......................................................................................................... 280 Example: Webserver........................................................................................................ 281 Example: splunk log driver ............................................................................................. 282 Example: fluentd log driver ........................................................................................... 283 Example: gelf log driver ................................................................................................ 283 Example: Amazon ECR image and task definition IAM role ................................................... 284 Example: Entrypoint with command ................................................................................. 284 Example: Container dependency ....................................................................................... 284 Windows sample task definitions ...................................................................................... 286 Updating a task definition using the classic console .................................................................... 286 Deregistering a task definition revision ...................................................................................... 287 Account settings............................................................................................................................. 288 Amazon Resource Names (ARNs) and IDs ................................................................................... 289 ARN and resource ID format timeline ........................................................................................ 290 Viewing account settings ......................................................................................................... 291 Modifying account settings ...................................................................................................... 292 Reverting to the default account settings .................................................................................. 293 Container instances......................................................................................................................... 295 Container instance concepts ..................................................................................................... 295 Container instance lifecycle ...................................................................................................... 296 Check the instance IAM role for your account ............................................................................. 297 Linux instances....................................................................................................................... 297 vi Amazon Elastic Container Service Developer Guide Amazon ECS-optimized AMI ............................................................................................. 298 Bottlerocket................................................................................................................... 334 Launching a container instance ........................................................................................ 338 Bootstrap Container Instances .......................................................................................... 346 Starting a task at container instance launch time ................................................................ 347 Elastic network interface trunking .................................................................................... 349 Memory Management..................................................................................................... 366 Connect to your container instance using the classic console ................................................ 367 Manage container instances remotely ................................................................................ 368 Windows instances................................................................................................................. 370 Amazon ECS-optimized AMI ............................................................................................. 370 Launching a container instance ........................................................................................ 393 Bootstrap Container Instances .......................................................................................... 399 Connect to your container Windows instance ..................................................................... 401 Deregister a container instance ........................................................................................ 402 External instances................................................................................................................... 403 Supported operating systems and system architectures ....................................................... 404 Considerations................................................................................................................ 405 IAM permissions............................................................................................................. 407 Registering an external instance to a cluster ...................................................................... 409 Deregistering an external instance .................................................................................... 413 Running workloads on external instances .......................................................................... 415 Updating the AWS Systems Manager Agent and Amazon ECS container agent ......................... 417 Monitoring............................................................................................................................. 420 CloudWatch Logs IAM Policy ............................................................................................ 420 Installing and configuring the CloudWatch agent ................................................................ 421 Viewing CloudWatch Logs ............................................................................................... 421 Container instance draining ..................................................................................................... 421 Draining behavior for services .......................................................................................... 422 Draining behavior for standalone tasks ............................................................................. 422 Draining container instances ............................................................................................ 422 Container agent............................................................................................................................. 424 Installing the Amazon ECS container agent ................................................................................ 424 Installing the Amazon ECS container agent on an Amazon Linux 2 EC2 instance ...................... 425 Installing the Amazon ECS container agent on an Amazon Linux AMI EC2 instance ................... 425 Installing the Amazon ECS container agent on a non-Amazon Linux EC2 instance .................... 426 Running the Amazon ECS agent with host network mode .................................................... 429 Container agent versions ......................................................................................................... 430 Amazon ECS-Optimized Amazon Linux 2 AMI container agent versions .................................. 430 Amazon ECS-Optimized Amazon Linux AMI container agent versions ..................................... 434 Amazon EC2 Windows containers ............................................................................................. 439 Windows container caveats .............................................................................................. 439 Getting started with Windows containers .......................................................................... 440 Updating the Amazon ECS container agent ................................................................................ 440 Checking the Amazon ECS container agent version ............................................................. 441 Updating the Amazon ECS container agent on an Amazon ECS-optimized AMI ........................ 442 Manually updating the Amazon ECS container agent (for non-Amazon ECS-Optimized AMIs) ..... 445 Container agent configuration .................................................................................................. 447 Available Parameters....................................................................................................... 447 Storing container instance configuration in Amazon S3 ....................................................... 461 Private registry authentication for container instances ................................................................. 462 Authentication formats................................................................................................... 462 Enabling private registries ............................................................................................... 464 Automated task and image cleanup.......................................................................................... 465 Tunable parameters........................................................................................................ 465 Cleanup workflow........................................................................................................... 466 Container metadata file........................................................................................................... 466 vii Amazon Elastic Container Service Developer Guide Turning on container metadata ........................................................................................ 466 Container metadata file locations..................................................................................... 467 Container metadata file format........................................................................................ 467 Task metadata endpoint .......................................................................................................... 470 Task metadata endpoint version 4 .................................................................................... 470 Task Metadata Endpoint version 3 .................................................................................... 487 Task Metadata Endpoint version 2 .................................................................................... 492 Container agent endpoint........................................................................................................ 497 Task scale-in protection endpoint ..................................................................................... 497 Container agent introspection .................................................................................................. 500 HTTP proxy configuration ........................................................................................................ 502 Amazon Linux container instance configuration .................................................................. 502 Windows container instance configuration ......................................................................... 505 Using gMSAs for Windows Containers ....................................................................................... 506 Considerations................................................................................................................ 506 Prerequisites.................................................................................................................. 506 Setting Up gMSA-capable Windows Containers on Amazon ECS ............................................ 507 Scheduling tasks............................................................................................................................. 510 Running a standalone task using the new Amazon ECS console .................................................... 511 Stopping tasks using the new console ....................................................................................... 515 Run a standalone task in the classic Amazon ECS console ............................................................ 515 Task placement...................................................................................................................... 518 Task groups................................................................................................................... 518 Task placement strategies ................................................................................................ 519 Task placement constraints .............................................................................................. 521 Cluster query language ................................................................................................... 525 Scheduled tasks..................................................................................................................... 529 Create a scheduled task .................................................................................................. 529 View your scheduled tasks in the classic console ................................................................. 533 Edit a scheduled task ...................................................................................................... 533 Task lifecycle......................................................................................................................... 533 Lifecycle states............................................................................................................... 534 Creating a scheduled task using the AWS CLI ............................................................................. 535 Services......................................................................................................................................... 537 Service scheduler concepts ...................................................................................................... 537 Daemon........................................................................................................................ 538 Replica.......................................................................................................................... 539 Additional service concepts ...................................................................................................... 539 Service definition parameters ................................................................................................... 540 Launch type................................................................................................................... 540 Capacity provider strategy ............................................................................................... 540 Task definition............................................................................................................... 541 Platform operating system .............................................................................................. 542 Platform version............................................................................................................. 542 Cluster.......................................................................................................................... 542 Service name................................................................................................................. 543 Scheduling strategy........................................................................................................ 543 Desired count................................................................................................................. 543 Deployment configuration............................................................................................... 544 Deployment controller.................................................................................................... 545 Task placement.............................................................................................................. 546 Tags.............................................................................................................................. 547 Network configuration.................................................................................................... 548 Client token................................................................................................................... 554 Service definition template .............................................................................................. 554 Service management in the Amazon ECS console ....................................................................... 555 New Amazon ECS console ............................................................................................... 555 viii Amazon Elastic Container Service Developer Guide Classic Amazon ECS console ............................................................................................ 567 Deployment types.................................................................................................................. 581 Rolling update............................................................................................................... 581 Blue/Green deployment with CodeDeploy ......................................................................... 586 External deployment....................................................................................................... 590 Service load balancing ............................................................................................................ 595 Load balancer types ........................................................................................................ 596 Creating a load balancer ................................................................................................. 599 Registering multiple target groups with a service ............................................................... 604 Service auto scaling ................................................................................................................ 606 Service auto scaling and deployments ............................................................................... 607 IAM permissions required for service auto scaling ............................................................... 607 Considerations................................................................................................................ 608 AWS CLI and SDK experience ........................................................................................... 609 Target tracking scaling policies ......................................................................................... 609 Step scaling policies........................................................................................................ 610 Interconnecting services.......................................................................................................... 611 Choosing an interconnection method ................................................................................ 611 Network mode compatibility table .................................................................................... 612 Service Connect.............................................................................................................. 612 Service discovery............................................................................................................ 628 Task scale-in protection ........................................................................................................... 631 Task scale-in protection mechanisms ................................................................................. 632 Task scale-in protection considerations .............................................................................. 633 IAM permissions required for task scale-in protection .......................................................... 634 Service throttle logic ............................................................................................................... 634 Resources and tags......................................................................................................................... 636 Tagging your resources ........................................................................................................... 636 Tag basics...................................................................................................................... 636 Tagging your resources ................................................................................................... 637 Tag restrictions.............................................................................................................. 638 Tagging your resources for billing ..................................................................................... 638 Working with tags using the console ................................................................................. 639 Working with tags using the CLI or API ............................................................................. 641 Service quotas........................................................................................................................ 643 Amazon ECS service quotas ............................................................................................. 643 AWS Fargate service quotas ............................................................................................. 646 Managing your Amazon ECS and AWS Fargate service quotas in the AWS Management Console. 646 AWS Fargate Regions .............................................................................................................. 647 Supported Regions for Linux containers on AWS Fargate ..................................................... 648 Supported Regions for Windows containers on AWS Fargate ................................................ 649 Usage Reports........................................................................................................................ 650 Monitoring..................................................................................................................................... 651 Monitoring tools..................................................................................................................... 652 Automated Tools............................................................................................................ 652 Manual Tools................................................................................................................. 652 CloudWatch metrics................................................................................................................ 653 Using CloudWatch metrics ............................................................................................... 653 Available metrics and dimensions ..................................................................................... 654 Cluster reservation.......................................................................................................... 660 Cluster utilization........................................................................................................... 661 Service utilization........................................................................................................... 662 Service RUNNING task count ............................................................................................ 663 Viewing Amazon ECS metrics ........................................................................................... 664 Events and EventBridge ........................................................................................................... 665 Amazon ECS events ........................................................................................................ 666 Handling events............................................................................................................. 677 ix Amazon Elastic Container Service Developer Guide CloudWatch Container Insights ................................................................................................. 679 Container Insights considerations ..................................................................................... 679 Setting up CloudWatch Container Insights for cluster and service level metrics ........................ 680 To change the default for Container Insights for all users using the console ............................ 680 To change the default for Container Insights for all users using the command line ................... 680 To change the default for Container Insights for a specific user using the command line ........... 681 To turn on Container Insights for a specific cluster using the command line ............................ 681 Use CloudWatch Container Insights to view Amazon ECS lifecycle events ............................... 681 Container instance health ........................................................................................................ 683 Collecting application trace data .............................................................................................. 683 Required IAM permissions for AWS Distro for OpenTelemetry integration with AWS X-Ray ........ 684 Specifying the AWS Distro for OpenTelemetry sidecar for AWS X-Ray integration in your task definition....................................................................................................................... 685 Collecting application metrics .................................................................................................. 686 Exporting application metrics to Amazon CloudWatch ......................................................... 686 Exporting application metrics to Amazon Managed Service for Prometheus ............................ 689 Logging Amazon ECS API calls with AWS CloudTrail .................................................................... 691 Amazon ECS information in CloudTrail .............................................................................. 691 Understanding Amazon ECS log file entries ....................................................................... 692 Security......................................................................................................................................... 694 Identity and Access Management .............................................................................................. 694 Audience....................................................................................................................... 695 Authenticating with identities.......................................................................................... 695 Managing access using policies ......................................................................................... 697 How Amazon Elastic Container Service works with IAM ........................................................ 699 Identity-based policy examples ........................................................................................ 706 AWS managed policies for Amazon ECS ............................................................................ 718 Using service-linked roles ................................................................................................ 729 Task execution IAM role .................................................................................................. 731 Container instance IAM role ............................................................................................. 736 ECS Anywhere IAM role ................................................................................................... 739 IAM roles for tasks ......................................................................................................... 741 CodeDeploy IAM Role ...................................................................................................... 747 CloudWatch Events IAM Role ........................................................................................... 750 Additional configuration for Windows IAM roles for tasks .................................................... 753 Troubleshooting............................................................................................................. 755 Logging and Monitoring.......................................................................................................... 757 Compliance validation............................................................................................................. 758 Infrastructure Security............................................................................................................. 758 Interface VPC endpoints (AWS PrivateLink) ........................................................................ 759 Working with other services ............................................................................................................. 762 Using Amazon ECR with Amazon ECS ........................................................................................ 762 Using Amazon ECR Images with Amazon ECS ..................................................................... 762 Creating Amazon ECS resources with AWS CloudFormation .......................................................... 763 Amazon ECS and AWS CloudFormation templates .............................................................. 763 Example templates......................................................................................................... 763 Using the AWS CLI to create resources from templates ........................................................ 768 Learn more about AWS CloudFormation ............................................................................ 769 Amazon Elastic Container Service on AWS Outposts .................................................................... 769 Prerequisites.................................................................................................................. 769 Limitations..................................................................................................................... 769 Network Connectivity Considerations ................................................................................ 769 Creating an Amazon ECS Cluster on an AWS Outposts ......................................................... 770 Use App Mesh with Amazon ECS .............................................................................................. 772 AWS Deep Learning Containers on Amazon ECS ......................................................................... 772 Deep Learning Containers with Elastic Inference on Amazon ECS .......................................... 772 Tutorials........................................................................................................................................ 774 x

Description:
Nov 13, 2014 Container Instance: an Amazon EC2 instance that is running the Amazon ECS agent and has been ing on this container instance. [22, 80
ebook img

Amazon EC2 Container Service Developer Guide - Amazon.com PDF

244 Pages·2016·2.59 MB·English
by  Amazon Web Services
Save to my drive
Quick download
Download
Upgrade Premium
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Amazon EC2 Container Service Developer Guide - Amazon.com

Amazon Elastic Container Service Developer Guide Amazon Elastic Container Service Developer Guide Amazon Elastic Container Service: Developer Guide Copyright © 2022 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon. Amazon Elastic Container Service Developer Guide Table of Contents What is Amazon ECS?......................................................................................................................... 1 Launch types............................................................................................................................. 1 Access Amazon ECS.................................................................................................................... 2 Pricing...................................................................................................................................... 2 Amazon ECS components............................................................................................................ 3 Clusters............................................................................................................................. 3 Containers and images....................................................................................................... 3 Task definitions.................................................................................................................. 3 Tasks................................................................................................................................ 4 Services............................................................................................................................ 4 Container agent................................................................................................................. 4 Fargate architecture overview .............................................................................................. 5 Common use cases..................................................................................................................... 6 Additional resources........................................................................................................... 6 Related services......................................................................................................................... 6 Getting started .................................................................................................................................. 8 Set up...................................................................................................................................... 8 Sign up for an AWS account ................................................................................................ 8 Create an administrative user .............................................................................................. 8 Create the credentials to connect to your EC2 instance ........................................................... 9 Create a virtual private cloud ............................................................................................. 10 Create a security group ..................................................................................................... 11 Install the AWS CLI ........................................................................................................... 13 Creating a container image........................................................................................................ 13 Prerequisites.................................................................................................................... 13 Create a Docker image ...................................................................................................... 15 Push your image to Amazon Elastic Container Registry ......................................................... 16 Clean up......................................................................................................................... 17 Next steps....................................................................................................................... 17 Using AWS Copilot ................................................................................................................... 18 Prerequisites.................................................................................................................... 18 Deploy your application using one command ....................................................................... 18 Deploy your application step by step .................................................................................. 18 Using the AWS CDK.................................................................................................................. 22 Step 1: Set up your AWS CDK project ................................................................................. 23 Step 2: Use the AWS CDK to define a containerized web server on Fargate ............................... 24 Step 3: Test the web server ............................................................................................... 28 Step 4: Clean up.............................................................................................................. 28 Next steps....................................................................................................................... 28 Getting started using the classic console ..................................................................................... 29 Using the classic console with Linux containers on AWS Fargate ............................................. 29 Using the classic console with Windows containers on AWS Fargate ........................................ 32 Using the classic console with Amazon EC2 ......................................................................... 36 Using the classic console with Windows containers ............................................................... 40 Developer tools overview.................................................................................................................. 45 AWS Management Console ........................................................................................................ 45 AWS Command Line Interface .................................................................................................... 45 AWS CloudFormation................................................................................................................ 46 AWS Copilot CLI ....................................................................................................................... 46 AWS CDK................................................................................................................................ 46 AWS App2Container................................................................................................................. 47 Docker Desktop integration with Amazon ECS ............................................................................. 47 AWS SDKs............................................................................................................................... 47 Summary................................................................................................................................ 48 iii Amazon Elastic Container Service Developer Guide Using the AWS Copilot CLI ........................................................................................................ 48 Installing the AWS Copilot CLI ........................................................................................... 49 Next steps....................................................................................................................... 54 AWS Fargate.................................................................................................................................... 55 Task definitions........................................................................................................................ 55 Network mode................................................................................................................. 56 Task Operating Systems .................................................................................................... 56 Task CPU architecture ....................................................................................................... 56 Task CPU and memory ...................................................................................................... 56 Task resource limits .......................................................................................................... 57 Logging........................................................................................................................... 58 Amazon ECS task execution IAM role .................................................................................. 58 Example Amazon Linux 2 task definition ............................................................................. 58 Example Windows task definition ....................................................................................... 59 Task storage.................................................................................................................... 60 Tasks and services .................................................................................................................... 60 Task networking ............................................................................................................... 60 Service load balancing ...................................................................................................... 61 Private registry authentication................................................................................................... 61 Clusters................................................................................................................................... 61 Fargate Spot............................................................................................................................ 62 Usage metrics.......................................................................................................................... 62 Task maintenance..................................................................................................................... 62 Savings plans........................................................................................................................... 63 Windows containers on AWS Fargate considerations..................................................................... 63 Platform Versions..................................................................................................................... 64 Linux platform versions ..................................................................................................... 64 Windows platform versions ............................................................................................... 68 Getting started walkthroughs.................................................................................................... 68 New Amazon Elastic Container Service console .................................................................................... 70 Getting started using the new Amazon ECS console ..................................................................... 70 Using the console with Linux containers on AWS Fargate ....................................................... 70 Using the console with Windows containers on AWS Fargate .................................................. 73 Using the console with Amazon EC2 ................................................................................... 76 Cluster management in the new Amazon ECS console ................................................................... 80 Creating a cluster for the Fargate launch type using the new console ...................................... 80 Creating a cluster for the Amazon EC2 launch type using the new console ............................... 82 Creating a capacity provider using the new console.............................................................. 84 Updating a capacity provider using the new console............................................................. 84 Deleting a capacity provider using the new console .............................................................. 85 Deleting a cluster using the new console ............................................................................. 85 Task definition management in the new Amazon ECS console ........................................................ 86 Creating a task definition using the new console .................................................................. 86 Updating a task definition using the new console ................................................................. 91 Deregistering a task definition revision using the new console ................................................ 92 Task management in the new Amazon ECS console ...................................................................... 93 Service management in the new Amazon ECS console .................................................................. 93 Clusters........................................................................................................................................... 94 Cluster concepts....................................................................................................................... 94 Creating a cluster using the classic console .................................................................................. 95 Capacity providers.................................................................................................................... 98 Capacity provider concepts ................................................................................................ 98 Capacity provider types ..................................................................................................... 99 Capacity provider considerations ........................................................................................ 99 AWS Fargate capacity providers ........................................................................................ 100 Auto Scaling group capacity providers ............................................................................... 104 Cluster auto scaling................................................................................................................ 110 iv Amazon Elastic Container Service Developer Guide How cluster Auto Scaling works ....................................................................................... 110 Managed termination protection ...................................................................................... 112 Managed scale-out behavior ............................................................................................ 112 Managed scale-in behavior .............................................................................................. 114 Target tracking considerations .......................................................................................... 114 Update on the way Amazon ECS creates resources for cluster auto scaling .............................. 115 Turn on cluster Auto Scaling ............................................................................................ 115 Turn off cluster auto scaling ............................................................................................ 116 Using Local Zones, Wavelength Zones, and AWS Outposts ........................................................... 117 Local Zones................................................................................................................... 117 Wavelength Zones.......................................................................................................... 118 AWS Outposts................................................................................................................ 118 Updating cluster settings ......................................................................................................... 118 Deleting a cluster using the classic console ................................................................................ 119 Task definitions.............................................................................................................................. 120 Amazon EC2 Windows task definition considerations ................................................................... 121 Additional configuration for Windows IAM roles for tasks .................................................... 121 Application architecture........................................................................................................... 122 Using the Fargate launch type ......................................................................................... 122 Using the EC2 launch type .............................................................................................. 122 Creating a task definition using the classic console ..................................................................... 123 Task definition template .................................................................................................. 131 Task definition parameters ...................................................................................................... 135 Family........................................................................................................................... 135 Launch types................................................................................................................. 135 Task role....................................................................................................................... 136 Task execution role ......................................................................................................... 136 Network mode............................................................................................................... 136 Runtime platform........................................................................................................... 137 Task size........................................................................................................................ 138 Container definitions....................................................................................................... 140 Elastic Inference accelerator name .................................................................................... 168 Task placement constraints .............................................................................................. 168 Proxy configuration......................................................................................................... 169 Volumes........................................................................................................................ 170 Tags.............................................................................................................................. 174 Other task definition parameters...................................................................................... 175 Launch types......................................................................................................................... 176 Fargate launch type ........................................................................................................ 176 EC2 launch type ............................................................................................................. 177 External launch type ....................................................................................................... 178 Working with GPUs on Amazon ECS ......................................................................................... 179 Considerations................................................................................................................ 180 Specifying GPUs in your task definition ............................................................................. 181 Using video transcoding on Amazon ECS ................................................................................... 182 Considerations................................................................................................................ 182 Using a VT1 AMI ............................................................................................................ 182 Task definition requirements ............................................................................................ 183 Using machine learning on Amazon ECS .................................................................................... 190 Using AWS Neuron on Amazon Linux 2 on Amazon ECS ....................................................... 190 Using deep learning DL1 instances on Amazon ECS ............................................................. 194 Working with 64-bit ARM workloads on Amazon ECS .................................................................. 196 Considerations................................................................................................................ 196 Specifying the ARM architecture in your task definition ....................................................... 197 Interfaces for configuring ARM ......................................................................................... 198 Using data volumes in tasks .................................................................................................... 198 Fargate task storage ....................................................................................................... 199 v Amazon Elastic Container Service Developer Guide Amazon EFS volumes ...................................................................................................... 199 FSx for Windows File Server volumes ................................................................................ 203 Docker volumes.............................................................................................................. 206 Bind mounts.................................................................................................................. 212 Managing container swap space ............................................................................................... 221 Container swap considerations ......................................................................................... 222 Task networking..................................................................................................................... 222 AWSVPC mode............................................................................................................... 223 Bridge mode.................................................................................................................. 228 Host mode..................................................................................................................... 228 Using the awslogs log driver .................................................................................................... 228 Turning on the awslogs log driver for your containers ......................................................... 228 Creating a log group ....................................................................................................... 229 Available awslogs log driver options ................................................................................. 230 Specifying a log configuration in your task definition .......................................................... 231 Viewing awslogs container logs in CloudWatch Logs ........................................................... 232 Custom log routing ................................................................................................................. 234 Considerations................................................................................................................ 234 Required IAM permissions ................................................................................................ 235 Fluentd buffer limit........................................................................................................ 236 Using Fluent logger libraries or Log4j over TCP .................................................................. 237 Using the AWS for Fluent Bit image .................................................................................. 238 Creating a task definition that uses a FireLens configuration ................................................ 240 Filtering logs using regular expressions ............................................................................. 243 Concatenate multiline or stack-trace log messages ............................................................. 244 Example task definitions.................................................................................................. 259 Private registry authentication for tasks .................................................................................... 263 Required IAM permissions for private registry authentication ................................................ 264 Enabling private registry authentication ............................................................................ 265 Passing environment variables ................................................................................................. 266 Considerations for specifying environment variable files ...................................................... 267 Required IAM permissions ................................................................................................ 268 Passing sensitive data ............................................................................................................. 269 Using Secrets Manager .................................................................................................... 269 Using Systems Manager Parameter Store ........................................................................... 276 Example task definitions.......................................................................................................... 280 Example: Webserver........................................................................................................ 281 Example: splunk log driver ............................................................................................. 282 Example: fluentd log driver ........................................................................................... 283 Example: gelf log driver ................................................................................................ 283 Example: Amazon ECR image and task definition IAM role ................................................... 284 Example: Entrypoint with command ................................................................................. 284 Example: Container dependency ....................................................................................... 284 Windows sample task definitions ...................................................................................... 286 Updating a task definition using the classic console .................................................................... 286 Deregistering a task definition revision ...................................................................................... 287 Account settings............................................................................................................................. 288 Amazon Resource Names (ARNs) and IDs ................................................................................... 289 ARN and resource ID format timeline ........................................................................................ 290 Viewing account settings ......................................................................................................... 291 Modifying account settings ...................................................................................................... 292 Reverting to the default account settings .................................................................................. 293 Container instances......................................................................................................................... 295 Container instance concepts ..................................................................................................... 295 Container instance lifecycle ...................................................................................................... 296 Check the instance IAM role for your account ............................................................................. 297 Linux instances....................................................................................................................... 297 vi Amazon Elastic Container Service Developer Guide Amazon ECS-optimized AMI ............................................................................................. 298 Bottlerocket................................................................................................................... 334 Launching a container instance ........................................................................................ 338 Bootstrap Container Instances .......................................................................................... 346 Starting a task at container instance launch time ................................................................ 347 Elastic network interface trunking .................................................................................... 349 Memory Management..................................................................................................... 366 Connect to your container instance using the classic console ................................................ 367 Manage container instances remotely ................................................................................ 368 Windows instances................................................................................................................. 370 Amazon ECS-optimized AMI ............................................................................................. 370 Launching a container instance ........................................................................................ 393 Bootstrap Container Instances .......................................................................................... 399 Connect to your container Windows instance ..................................................................... 401 Deregister a container instance ........................................................................................ 402 External instances................................................................................................................... 403 Supported operating systems and system architectures ....................................................... 404 Considerations................................................................................................................ 405 IAM permissions............................................................................................................. 407 Registering an external instance to a cluster ...................................................................... 409 Deregistering an external instance .................................................................................... 413 Running workloads on external instances .......................................................................... 415 Updating the AWS Systems Manager Agent and Amazon ECS container agent ......................... 417 Monitoring............................................................................................................................. 420 CloudWatch Logs IAM Policy ............................................................................................ 420 Installing and configuring the CloudWatch agent ................................................................ 421 Viewing CloudWatch Logs ............................................................................................... 421 Container instance draining ..................................................................................................... 421 Draining behavior for services .......................................................................................... 422 Draining behavior for standalone tasks ............................................................................. 422 Draining container instances ............................................................................................ 422 Container agent............................................................................................................................. 424 Installing the Amazon ECS container agent ................................................................................ 424 Installing the Amazon ECS container agent on an Amazon Linux 2 EC2 instance ...................... 425 Installing the Amazon ECS container agent on an Amazon Linux AMI EC2 instance ................... 425 Installing the Amazon ECS container agent on a non-Amazon Linux EC2 instance .................... 426 Running the Amazon ECS agent with host network mode .................................................... 429 Container agent versions ......................................................................................................... 430 Amazon ECS-Optimized Amazon Linux 2 AMI container agent versions .................................. 430 Amazon ECS-Optimized Amazon Linux AMI container agent versions ..................................... 434 Amazon EC2 Windows containers ............................................................................................. 439 Windows container caveats .............................................................................................. 439 Getting started with Windows containers .......................................................................... 440 Updating the Amazon ECS container agent ................................................................................ 440 Checking the Amazon ECS container agent version ............................................................. 441 Updating the Amazon ECS container agent on an Amazon ECS-optimized AMI ........................ 442 Manually updating the Amazon ECS container agent (for non-Amazon ECS-Optimized AMIs) ..... 445 Container agent configuration .................................................................................................. 447 Available Parameters....................................................................................................... 447 Storing container instance configuration in Amazon S3 ....................................................... 461 Private registry authentication for container instances ................................................................. 462 Authentication formats................................................................................................... 462 Enabling private registries ............................................................................................... 464 Automated task and image cleanup.......................................................................................... 465 Tunable parameters........................................................................................................ 465 Cleanup workflow........................................................................................................... 466 Container metadata file........................................................................................................... 466 vii Amazon Elastic Container Service Developer Guide Turning on container metadata ........................................................................................ 466 Container metadata file locations..................................................................................... 467 Container metadata file format........................................................................................ 467 Task metadata endpoint .......................................................................................................... 470 Task metadata endpoint version 4 .................................................................................... 470 Task Metadata Endpoint version 3 .................................................................................... 487 Task Metadata Endpoint version 2 .................................................................................... 492 Container agent endpoint........................................................................................................ 497 Task scale-in protection endpoint ..................................................................................... 497 Container agent introspection .................................................................................................. 500 HTTP proxy configuration ........................................................................................................ 502 Amazon Linux container instance configuration .................................................................. 502 Windows container instance configuration ......................................................................... 505 Using gMSAs for Windows Containers ....................................................................................... 506 Considerations................................................................................................................ 506 Prerequisites.................................................................................................................. 506 Setting Up gMSA-capable Windows Containers on Amazon ECS ............................................ 507 Scheduling tasks............................................................................................................................. 510 Running a standalone task using the new Amazon ECS console .................................................... 511 Stopping tasks using the new console ....................................................................................... 515 Run a standalone task in the classic Amazon ECS console ............................................................ 515 Task placement...................................................................................................................... 518 Task groups................................................................................................................... 518 Task placement strategies ................................................................................................ 519 Task placement constraints .............................................................................................. 521 Cluster query language ................................................................................................... 525 Scheduled tasks..................................................................................................................... 529 Create a scheduled task .................................................................................................. 529 View your scheduled tasks in the classic console ................................................................. 533 Edit a scheduled task ...................................................................................................... 533 Task lifecycle......................................................................................................................... 533 Lifecycle states............................................................................................................... 534 Creating a scheduled task using the AWS CLI ............................................................................. 535 Services......................................................................................................................................... 537 Service scheduler concepts ...................................................................................................... 537 Daemon........................................................................................................................ 538 Replica.......................................................................................................................... 539 Additional service concepts ...................................................................................................... 539 Service definition parameters ................................................................................................... 540 Launch type................................................................................................................... 540 Capacity provider strategy ............................................................................................... 540 Task definition............................................................................................................... 541 Platform operating system .............................................................................................. 542 Platform version............................................................................................................. 542 Cluster.......................................................................................................................... 542 Service name................................................................................................................. 543 Scheduling strategy........................................................................................................ 543 Desired count................................................................................................................. 543 Deployment configuration............................................................................................... 544 Deployment controller.................................................................................................... 545 Task placement.............................................................................................................. 546 Tags.............................................................................................................................. 547 Network configuration.................................................................................................... 548 Client token................................................................................................................... 554 Service definition template .............................................................................................. 554 Service management in the Amazon ECS console ....................................................................... 555 New Amazon ECS console ............................................................................................... 555 viii Amazon Elastic Container Service Developer Guide Classic Amazon ECS console ............................................................................................ 567 Deployment types.................................................................................................................. 581 Rolling update............................................................................................................... 581 Blue/Green deployment with CodeDeploy ......................................................................... 586 External deployment....................................................................................................... 590 Service load balancing ............................................................................................................ 595 Load balancer types ........................................................................................................ 596 Creating a load balancer ................................................................................................. 599 Registering multiple target groups with a service ............................................................... 604 Service auto scaling ................................................................................................................ 606 Service auto scaling and deployments ............................................................................... 607 IAM permissions required for service auto scaling ............................................................... 607 Considerations................................................................................................................ 608 AWS CLI and SDK experience ........................................................................................... 609 Target tracking scaling policies ......................................................................................... 609 Step scaling policies........................................................................................................ 610 Interconnecting services.......................................................................................................... 611 Choosing an interconnection method ................................................................................ 611 Network mode compatibility table .................................................................................... 612 Service Connect.............................................................................................................. 612 Service discovery............................................................................................................ 628 Task scale-in protection ........................................................................................................... 631 Task scale-in protection mechanisms ................................................................................. 632 Task scale-in protection considerations .............................................................................. 633 IAM permissions required for task scale-in protection .......................................................... 634 Service throttle logic ............................................................................................................... 634 Resources and tags......................................................................................................................... 636 Tagging your resources ........................................................................................................... 636 Tag basics...................................................................................................................... 636 Tagging your resources ................................................................................................... 637 Tag restrictions.............................................................................................................. 638 Tagging your resources for billing ..................................................................................... 638 Working with tags using the console ................................................................................. 639 Working with tags using the CLI or API ............................................................................. 641 Service quotas........................................................................................................................ 643 Amazon ECS service quotas ............................................................................................. 643 AWS Fargate service quotas ............................................................................................. 646 Managing your Amazon ECS and AWS Fargate service quotas in the AWS Management Console. 646 AWS Fargate Regions .............................................................................................................. 647 Supported Regions for Linux containers on AWS Fargate ..................................................... 648 Supported Regions for Windows containers on AWS Fargate ................................................ 649 Usage Reports........................................................................................................................ 650 Monitoring..................................................................................................................................... 651 Monitoring tools..................................................................................................................... 652 Automated Tools............................................................................................................ 652 Manual Tools................................................................................................................. 652 CloudWatch metrics................................................................................................................ 653 Using CloudWatch metrics ............................................................................................... 653 Available metrics and dimensions ..................................................................................... 654 Cluster reservation.......................................................................................................... 660 Cluster utilization........................................................................................................... 661 Service utilization........................................................................................................... 662 Service RUNNING task count ............................................................................................ 663 Viewing Amazon ECS metrics ........................................................................................... 664 Events and EventBridge ........................................................................................................... 665 Amazon ECS events ........................................................................................................ 666 Handling events............................................................................................................. 677 ix Amazon Elastic Container Service Developer Guide CloudWatch Container Insights ................................................................................................. 679 Container Insights considerations ..................................................................................... 679 Setting up CloudWatch Container Insights for cluster and service level metrics ........................ 680 To change the default for Container Insights for all users using the console ............................ 680 To change the default for Container Insights for all users using the command line ................... 680 To change the default for Container Insights for a specific user using the command line ........... 681 To turn on Container Insights for a specific cluster using the command line ............................ 681 Use CloudWatch Container Insights to view Amazon ECS lifecycle events ............................... 681 Container instance health ........................................................................................................ 683 Collecting application trace data .............................................................................................. 683 Required IAM permissions for AWS Distro for OpenTelemetry integration with AWS X-Ray ........ 684 Specifying the AWS Distro for OpenTelemetry sidecar for AWS X-Ray integration in your task definition....................................................................................................................... 685 Collecting application metrics .................................................................................................. 686 Exporting application metrics to Amazon CloudWatch ......................................................... 686 Exporting application metrics to Amazon Managed Service for Prometheus ............................ 689 Logging Amazon ECS API calls with AWS CloudTrail .................................................................... 691 Amazon ECS information in CloudTrail .............................................................................. 691 Understanding Amazon ECS log file entries ....................................................................... 692 Security......................................................................................................................................... 694 Identity and Access Management .............................................................................................. 694 Audience....................................................................................................................... 695 Authenticating with identities.......................................................................................... 695 Managing access using policies ......................................................................................... 697 How Amazon Elastic Container Service works with IAM ........................................................ 699 Identity-based policy examples ........................................................................................ 706 AWS managed policies for Amazon ECS ............................................................................ 718 Using service-linked roles ................................................................................................ 729 Task execution IAM role .................................................................................................. 731 Container instance IAM role ............................................................................................. 736 ECS Anywhere IAM role ................................................................................................... 739 IAM roles for tasks ......................................................................................................... 741 CodeDeploy IAM Role ...................................................................................................... 747 CloudWatch Events IAM Role ........................................................................................... 750 Additional configuration for Windows IAM roles for tasks .................................................... 753 Troubleshooting............................................................................................................. 755 Logging and Monitoring.......................................................................................................... 757 Compliance validation............................................................................................................. 758 Infrastructure Security............................................................................................................. 758 Interface VPC endpoints (AWS PrivateLink) ........................................................................ 759 Working with other services ............................................................................................................. 762 Using Amazon ECR with Amazon ECS ........................................................................................ 762 Using Amazon ECR Images with Amazon ECS ..................................................................... 762 Creating Amazon ECS resources with AWS CloudFormation .......................................................... 763 Amazon ECS and AWS CloudFormation templates .............................................................. 763 Example templates......................................................................................................... 763 Using the AWS CLI to create resources from templates ........................................................ 768 Learn more about AWS CloudFormation ............................................................................ 769 Amazon Elastic Container Service on AWS Outposts .................................................................... 769 Prerequisites.................................................................................................................. 769 Limitations..................................................................................................................... 769 Network Connectivity Considerations ................................................................................ 769 Creating an Amazon ECS Cluster on an AWS Outposts ......................................................... 770 Use App Mesh with Amazon ECS .............................................................................................. 772 AWS Deep Learning Containers on Amazon ECS ......................................................................... 772 Deep Learning Containers with Elastic Inference on Amazon ECS .......................................... 772 Tutorials........................................................................................................................................ 774 x

Description:
Nov 13, 2014 Container Instance: an Amazon EC2 instance that is running the Amazon ECS agent and has been ing on this container instance. [22, 80
See more

The list of books you might like

Upgrade Premium
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.
DMCA & Copyright: Dear all, most of the website is community built, users are uploading hundred of books everyday, which makes really hard for us to identify copyrighted material, please contact us if you want any material removed.
Copyright @ PDFdrive.to, 2022 | We love our users