Amazon ElastiCache for Redis

448 Pages·2017·4.44 MB·English

Amazon ElastiCache for Redis
User Guide
API Version 2015-02-02
Amazon ElastiCache for Redis User Guide
Amazon ElastiCache for Redis: User Guide
Amazon ElastiCache for Redis User Guide
Table of Contents
What is ElastiCache for Redis? ............................................................................................................. 1
Overview of ElastiCache for Redis ................................................................................................ 1
Related services......................................................................................................................... 2
Clusters..................................................................................................................................... 2
Amazon Regions and Availability Zones ........................................................................................ 3
Security.................................................................................................................................... 3
Monitoring................................................................................................................................ 3
ElastiCache resources.................................................................................................................. 4
Components and features ........................................................................................................... 5
Nodes............................................................................................................................... 5
ElastiCache for Redis shards ................................................................................................ 5
ElastiCache for Redis clusters ............................................................................................... 6
ElastiCache for Redis replication .......................................................................................... 7
Amazon Regions and availability zones................................................................................. 9
ElastiCache for Redis endpoints ......................................................................................... 10
Parameter groups............................................................................................................. 10
ElastiCache for Redis security ............................................................................................. 10
Security groups................................................................................................................ 11
Subnet groups................................................................................................................. 11
ElastiCache for Redis backups ............................................................................................ 11
Events............................................................................................................................. 12
ElastiCache for Redis terminology .............................................................................................. 13
Tools for managing your implementation .................................................................................... 15
Using the Amazon Web Services Management Console ......................................................... 15
Using the Amazon CLI ...................................................................................................... 15
Using the Amazon SDK ..................................................................................................... 15
Using the ElastiCache API.................................................................................................. 15
Additional resources......................................................................................................... 15
Common ElastiCache Use Cases and How ElastiCache Can Help ...................................................... 16
In-Memory Data Store ...................................................................................................... 16
Gaming Leaderboards (Redis Sorted Sets) ............................................................................ 17
Messaging (Redis Pub/Sub) ................................................................................................ 18
Recommendation Data (Redis Hashes) ................................................................................. 20
Other Redis Uses .............................................................................................................. 21
ElastiCache Customer Testimonials ..................................................................................... 21
Comparing Memcached and Redis ...................................................................................................... 22
Getting started with ElastiCache for Redis ........................................................................................... 26
Setting up............................................................................................................................... 26
Create your Amazon account ............................................................................................. 26
Getting an Amazon Access Key .......................................................................................... 27
Configuring Your Credentials ............................................................................................. 28
Downloading and Configuring the Amazon CLI .................................................................... 28
Set up your permissions (new ElastiCache users only) ............................................................ 28
Step 1: Create a subnet group................................................................................................... 29
Step 2: Create a cluster ............................................................................................................. 31
Creating a Redis (cluster mode disabled) cluster (Console)..................................................... 32
Step 3: Authorize access to the cluster ........................................................................................ 37
Step 4: Connect to the cluster's node ......................................................................................... 39
Find your node endpoints ................................................................................................. 39
Step 5: Deleting a cluster.......................................................................................................... 46
Tutorials and videos ................................................................................................................. 47
Where do I go from here? ......................................................................................................... 48
Getting Started with ElastiCache and Amazon SDKs ............................................................................. 49
Python and ElastiCache............................................................................................................. 49
API Version 2015-02-02
iii
Amazon ElastiCache for Redis User Guide
Tutorial Prerequisites........................................................................................................ 49
Creating Elasticache clusters and users ................................................................................ 49
Connecting to Elasticache .................................................................................................. 57
Usage examples............................................................................................................... 58
Online Migration to ElastiCache......................................................................................................... 62
Overview................................................................................................................................. 62
Migration steps........................................................................................................................ 62
Preparing your source and target Redis nodes for migration .......................................................... 62
Starting migration.................................................................................................................... 63
Verifying the data migration progress ......................................................................................... 64
Completing the data migration.................................................................................................. 65
Performing online data migration using the Console .................................................................... 65
Choosing regions and availability zones .............................................................................................. 68
Supported regions & endpoints .................................................................................................. 69
Locating your nodes ................................................................................................................. 72
Using Local zones ..................................................................................................................... 72
Enabling a local zone ........................................................................................................ 73
Using Outposts........................................................................................................................ 73
Using Outposts with the Redis console ............................................................................... 73
Using Outposts with the Amazon CLI .................................................................................. 75
Managing nodes............................................................................................................................... 76
Redis nodes and shards ............................................................................................................. 76
Connecting to nodes................................................................................................................. 78
Supported node types .............................................................................................................. 80
Supported node types by Amazon Region ........................................................................... 82
Rebooting nodes (cluster mode disabled only) ............................................................................. 83
Replacing nodes....................................................................................................................... 84
Reserved nodes........................................................................................................................ 88
Managing costs with reserved nodes ................................................................................... 88
Migrating previous generation nodes.......................................................................................... 96
Migrating nodes on a Redis cluster ..................................................................................... 96
If nodes are running on EC2-Classic .................................................................................... 97
Managing clusters............................................................................................................................ 99
Choosing a network type ......................................................................................................... 100
Configuring subnets for network type ............................................................................... 100
Using dual-stack............................................................................................................. 100
TLS enabled dual stack ElastiCache clusters....................................................................... 100
Using the Amazon Web Services Management Console ........................................................ 100
Using the CLI................................................................................................................. 101
Data tiering........................................................................................................................... 103
Best practices................................................................................................................. 103
Limitations..................................................................................................................... 103
Pricing........................................................................................................................... 104
Monitoring..................................................................................................................... 104
Using data tiering........................................................................................................... 104
Restoring data from backup into clusters with data tiering enabled ....................................... 105
Preparing a cluster ................................................................................................................. 107
Determining your requirements ........................................................................................ 107
Choosing your node size ................................................................................................. 109
Creating a cluster ................................................................................................................... 112
Viewing a cluster's details ........................................................................................................ 119
Modifying a cluster ................................................................................................................. 128
Adding nodes to a cluster ........................................................................................................ 131
Removing nodes from a cluster ................................................................................................ 136
Canceling pending add or delete node operations ...................................................................... 141
Deleting a cluster................................................................................................................... 142
Accessing your cluster or replication group................................................................................ 144
API Version 2015-02-02
iv
Amazon ElastiCache for Redis User Guide
Determine the cluster's platform ...................................................................................... 144
Grant access to your cluster or replication group ................................................................ 147
Finding connection endpoints .................................................................................................. 153
Shards................................................................................................................................... 160
Finding a shard's ID ........................................................................................................ 161
Managing your ElastiCache for Redis implementation .......................................................................... 165
Engine versions and upgrading ................................................................................................. 165
Supported Redis versions ................................................................................................ 166
Redis versions end of life schedule ................................................................................... 174
Upgrading engine versions ............................................................................................... 176
Major version behavior and compatibility differences .......................................................... 178
Getting started with JSON in ElastiCache for Redis ..................................................................... 182
Redis JSON data type overview ........................................................................................ 183
Supported Redis JSON commands .................................................................................... 191
Tagging your ElastiCache resources ........................................................................................... 219
Monitoring costs with tags .............................................................................................. 225
Managing tags using the Amazon CLI ................................................................................ 226
Managing tags using the ElastiCache API........................................................................... 229
Caching strategies and best practices ........................................................................................ 231
Caching strategies.......................................................................................................... 231
Restricted Redis Commands ............................................................................................. 236
Ensuring that you have enough memory to create a Redis snapshot ...................................... 237
Managing Reserved Memory ............................................................................................ 239
Mitigating failure issues when using Redis AOF .................................................................. 244
Best practices: Online cluster resizing ................................................................................ 245
Best practices: Minimizing downtime during maintenance .................................................... 246
Best practices: Redis clients and ElastiCache for Redis ......................................................... 246
Best practices: Pub/Sub ................................................................................................... 246
IPv6 Client Examples....................................................................................................... 247
Minimizing downtime with Multi-AZ ................................................................................. 250
Managing maintenance........................................................................................................... 250
Replication across Amazon Regions using global datastores ......................................................... 251
Overview....................................................................................................................... 252
Prerequisites and limitations ............................................................................................ 252
Using global datastores (console) ..................................................................................... 253
Using global datastores (CLI) ............................................................................................ 264
High availability using replication groups ................................................................................... 268
Understanding Redis replication ....................................................................................... 270
Replication: Redis (Cluster Mode Disabled) vs. Redis (Cluster Mode Enabled) ........................... 272
Minimizing downtime with Multi-AZ ................................................................................. 275
How synchronization and backup are implemented ............................................................. 287
Creating a replication group ............................................................................................ 288
Viewing a replication group's details ................................................................................. 310
Finding replication group endpoints .................................................................................. 315
Modifying a replication group .......................................................................................... 316
Deleting a replication group ............................................................................................ 318
Changing the number of replicas ...................................................................................... 319
Promoting a read replica ................................................................................................. 331
Backup and restore ................................................................................................................. 332
Constraints.................................................................................................................... 333
Costs............................................................................................................................. 334
Performance impact of backups ....................................................................................... 334
Backups when running Redis 2.8.22 and later .................................................................... 334
Backups when running Redis versions before 2.8.22 ............................................................ 334
Improving backup performance ........................................................................................ 334
Scheduling automatic backups......................................................................................... 336
Making manual backups.................................................................................................. 337
API Version 2015-02-02
v
Amazon ElastiCache for Redis User Guide
Creating a final backup ................................................................................................... 343
Describing backups......................................................................................................... 346
Copying a backup ........................................................................................................... 348
Exporting a backup ......................................................................................................... 350
Restoring from a backup ................................................................................................. 357
Seeding a cluster with a backup....................................................................................... 360
Scaling ElastiCache for Redis clusters ........................................................................................ 368
Scaling clusters for Redis (Cluster Mode Disabled) ............................................................... 370
Scaling clusters in Redis (Cluster Mode Enabled) ................................................................. 398
Auto Scaling ElastiCache for Redis clusters ................................................................................ 417
Prerequisites.................................................................................................................. 417
Managing Capacity Automatically with ElastiCache for Redis Auto Scaling .............................. 417
Auto Scaling policies....................................................................................................... 419
IAM Permissions Required for ElastiCache for Redis Auto Scaling .......................................... 421
Service-linked role.......................................................................................................... 421
Auto Scaling Best Practices .............................................................................................. 422
Using Auto Scaling with shards ........................................................................................ 423
Using Auto Scaling with replicas ....................................................................................... 435
Configuring engine parameters using parameter groups .............................................................. 446
Parameter management.................................................................................................. 448
Parameter group tiers ..................................................................................................... 449
Creating a parameter group ............................................................................................. 449
Listing parameter groups by name ................................................................................... 453
Listing a parameter group's values .................................................................................... 458
Modifying a parameter group .......................................................................................... 459
Deleting a parameter group ............................................................................................. 463
Redis-specific parameters................................................................................................ 465
Security......................................................................................................................................... 496
Data protection...................................................................................................................... 496
Data security in Amazon ElastiCache ................................................................................. 497
Internetwork traffic privacy ...................................................................................................... 537
Amazon VPCs and ElastiCache security .............................................................................. 538
Amazon ElastiCache API and interface VPC endpoints (Amazon PrivateLink) ........................... 558
Subnets and subnet groups ............................................................................................. 560
Security groups: EC2-Classic ............................................................................................. 569
Identity and access management .............................................................................................. 576
Authentication............................................................................................................... 577
How Amazon ElastiCache works with IAM .......................................................................... 578
Access control................................................................................................................ 582
Overview of managing access .......................................................................................... 584
Compliance validation............................................................................................................. 622
ElastiCache for Redis compliance ...................................................................................... 623
Resilience.............................................................................................................................. 627
Mitigating Failures.......................................................................................................... 627
Infrastructure security............................................................................................................. 630
Service updates...................................................................................................................... 630
Managing the service updates .......................................................................................... 630
Troubleshooting ..................................................................................................................... 632
Security groups.............................................................................................................. 632
Network ACLs................................................................................................................ 633
Route tables.................................................................................................................. 634
DNS resolution............................................................................................................... 634
Identifying issues with server-side diagnostics .................................................................... 635
Network connectivity validation ....................................................................................... 638
Network-related limits.................................................................................................... 639
CPU Usage..................................................................................................................... 640
Connections being terminated from the server side ............................................................ 642
API Version 2015-02-02
vi
Amazon ElastiCache for Redis User Guide
Client-side troubleshooting for Amazon EC2 instances ......................................................... 643
Dissecting the time taken to complete a single request ....................................................... 644
Logging and monitoring in Elasticache.............................................................................................. 647
Log delivery........................................................................................................................... 648
Contents of a slow log entry ........................................................................................... 648
Contents of an engine log entry ....................................................................................... 648
Permissions to configure logging ...................................................................................... 649
Log type and log format specifications .............................................................................. 649
ElastiCache logging destinations....................................................................................... 650
Specifying log delivery using the Console .......................................................................... 652
Specifying log delivery using the Amazon CLI .................................................................... 652
Monitoring use....................................................................................................................... 657
Host-Level Metrics.......................................................................................................... 657
Metrics for Redis ............................................................................................................ 659
Which Metrics Should I Monitor?...................................................................................... 669
Choosing Metric Statistics and Periods .............................................................................. 672
Monitoring CloudWatch Cluster and Node Metrics .............................................................. 672
Logging Amazon ElastiCache API calls with Amazon CloudTrail ..................................................... 675
Amazon ElastiCache information in CloudTrail .................................................................... 675
Understanding Amazon ElastiCache log file entries ............................................................. 675
Monitoring events.................................................................................................................. 678
Managing ElastiCache Amazon SNS notifications ................................................................ 678
Viewing ElastiCache events .............................................................................................. 682
Event Notifications and Amazon SNS ................................................................................ 685
Quotas.......................................................................................................................................... 690
Reference...................................................................................................................................... 691
Using the ElastiCache API........................................................................................................ 691
Using the query API ........................................................................................................ 691
Available libraries........................................................................................................... 693
Troubleshooting applications........................................................................................... 694
Set up the Amazon CLI for ElastiCache ...................................................................................... 694
Prerequisites.................................................................................................................. 695
Getting the command line tools ....................................................................................... 696
Setting up the tools ........................................................................................................ 696
Providing credentials for the tools .................................................................................... 697
Environmental variables.................................................................................................. 697
Error messages....................................................................................................................... 698
Notifications.......................................................................................................................... 699
General ElastiCache notifications...................................................................................... 699
ElastiCache for Redis specific notifications ......................................................................... 699
ElastiCache for Redis Documentation history..................................................................................... 700
Amazon glossary............................................................................................................................ 717
API Version 2015-02-02
vii
Amazon ElastiCache for Redis User Guide
Overview of ElastiCache for Redis
What is Amazon ElastiCache for
Redis?
Welcome to the Amazon ElastiCache for Redis User Guide. Amazon ElastiCache is a web service that
makes it easy to set up, manage, and scale a distributed in-memory data store or cache environment
in the cloud. It provides a high-performance, scalable, and cost-effective caching solution. At the same
time, it helps remove the complexity associated with deploying and managing a distributed cache
environment.
Note
Amazon ElastiCache works with both the Redis and Memcached engines. Use the guide for the
engine that you're interested in. If you're unsure which engine you want to use, see Comparing
Memcached and Redis (p. 22) in this guide.
Overview of ElastiCache for Redis
Existing applications that use Redis can use ElastiCache with almost no modification. Your applications
simply need information about the host names and port numbers of the ElastiCache nodes that you have
deployed.
ElastiCache for Redis has multiple features that help make the service more reliable for critical
production deployments:
• Automatic detection of and recovery from cache node failures.
• Multi-AZ for a failed primary cluster to a read replica, in Redis clusters that support replication.
• Redis (cluster mode enabled) supports partitioning your data across up to 500 shards.
• For Redis version 3.2 and later, all versions support encryption in transit and encryption at rest
encryption with authentication. This support helps you build HIPAA-compliant applications.
• Flexible Availability Zone placement of nodes and clusters for increased fault tolerance.
• Integration with other Amazon services such as Amazon EC2, Amazon CloudWatch, Amazon CloudTrail,
and Amazon SNS. This integration helps provide a managed in-memory caching solution that is high-
performance and highly secure.
• ElastiCache for Redis manages backups, software patching, automatic failure detection, and recovery.
• You can have automated backups performed when you need them, or manually create your own
backup snapshot. You can use these backups to restore a cluster. The ElastiCache for Redis restore
process works reliably and efficiently.
• You can get high availability with a primary instance and a synchronous secondary instance that you
can fail over to when problems occur. You can also use read replicas to increase read scaling.
• You can control access to your ElastiCache for Redis clusters by using Amazon Identity and Access
Management to define users and permissions. You can also help protect your clusters by putting them
in a virtual private cloud (VPC).
• By using the Global Datastore for Redis feature, you can work with fully managed, fast, reliable, and
secure replication across Amazon Regions. Using this feature, you can create cross-Region read replica
clusters for ElastiCache for Redis to enable low-latency reads and disaster recovery across Amazon
Regions.
• Data tiering provides a price-performance option for Redis workloads by utilizing lower-cost solid
state drives (SSDs) in each cluster node in addition to storing data in memory. It is ideal for workloads
API Version 2015-02-02
1
Amazon ElastiCache for Redis User Guide
Related services
that access up to 20 percent of their overall dataset regularly, and for applications that can tolerate
additional latency when accessing data on SSD. For more information, see Data tiering (p. 103).
Related services
Amazon MemoryDB for Redis
When deciding whether to use ElastiCache for Redis or Amazon MemoryDB for Redis consider the
following comparisons:
• ElastiCache for Redis is a service that is commonly used to cache data from other databases and data
stores using Redis. You should consider ElastiCache for Redis for caching workloads where you want to
accelerate data access with your existing primary database or data store (microsecond read and write
performance). You should also consider ElastiCache for Redis for use cases where you want to use the
Redis data structures and APIs to access data stored in a primary database or data store.
• Amazon MemoryDB for Redis is a durable, in-memory database for workloads that require an ultra-
fast, primary database. You should consider using MemoryDB if your workload requires a durable
database that provides ultra-fast performance (microsecond read and single-digit millisecond write
latency). MemoryDB may also be a good fit for your use case if you want to build an application using
Redis data structures and APIs with a primary, durable database. Finally, you should consider using
MemoryDB to simplify your application architecture and lower costs by replacing usage of a database
with a cache for durability and performance.
Clusters
The basic building block of ElastiCache for Redis is the cluster. A cluster is a collection of one or more
cache nodes, all of which run an instance of the Redis cache engine software. When you create a cluster,
you specify the engine and version for all of the nodes to use. Your ElastiCache for Redis instances are
designed to be accessed through an Amazon EC2 instance. You can create and modify a cluster by using
the Amazon CLI, the ElastiCache for Redis API, or the Amazon Web Services Management Console.
Each ElastiCache for Redis cluster runs a Redis engine version. Each Redis engine version has its own
supported features. Additionally, each Redis engine version has a set of parameters in a parameter group
that control the behavior of the clusters that it manages.
The computation and memory capacity of a cluster is determined by its instance, or node, class. You can
select the node type that best meets your needs. If your needs change over time, you can change node
types. For information, see Supported node types.
You can also leverage data-tiering when considering your node type needs. Data tiering is a feature
where some least frequently used data is stored on disk to mitigate against memory limitations on
applications that can tolerate additional latency when data on SSD (solid state drives) is accessed.
Note
For pricing information on ElastiCache instance classes, see Amazon ElastiCache pricing.
Cluster node storage comes in two types: Standard and memory-optimized. They differ in performance
characteristics and price, allowing you to tailor your storage performance and cost to your needs.
Each instance has minimum and maximum storage requirements depending on the storage type. It's
important to have sufficient storage so that your clusters have room to grow. Also, sufficient storage
makes sure that features have room to write content or log entries.
You can run a cluster on a virtual private cloud (VPC) using the Amazon Virtual Private Cloud (Amazon
VPC) service. When you use a VPC, you have control over your virtual networking environment. You
can choose your own IP address range, create subnets, and configure routing and access control lists.
API Version 2015-02-02
2
Amazon ElastiCache for Redis User Guide
Amazon Regions and Availability Zones
ElastiCache manages backups, software patching, automatic failure detection, and recovery. There's no
additional cost to run your cluster in a VPC. For more information on using Amazon VPC with ElastiCache
for Redis, see Amazon VPCs and ElastiCache security.
Amazon Regions and Availability Zones
Amazon cloud computing resources are housed in highly available data center facilities in different areas
of the world (for example, North America, Europe, or Asia). Each data center location is called an Amazon
Region.
Each Amazon Region contains multiple distinct locations called Availability Zones, or AZs. Each
Availability Zone is engineered to be isolated from failures in other Availability Zones. Each is engineered
to provide inexpensive, low-latency network connectivity to other Availability Zones in the same Amazon
Region. By launching instances in separate Availability Zones, you can protect your applications from
the failure of a single location. For more information, see Choosing regions and availability zones. You
can create your cluster in several Availability Zones, an option called a Multi-AZ deployment. When
you choose this option, Amazon automatically provisions and maintains a secondary standby node
instance in a different Availability Zone. Your primary node instance is asynchronously replicated
across Availability Zones to the secondary instance. This approach helps provide data redundancy and
failover support, eliminate I/O freezes, and minimize latency spikes during system backups. For more
information, see Minimizing downtime in ElastiCache for Redis with Multi-AZ.
Security
A security group controls the access to a cluster. It does so by allowing access to IP address ranges or
Amazon EC2 instances that you specify. For more information about security groups, see Security in
ElastiCache for Redis.
Monitoring an ElastiCache for Redis cluster
There are several ways that you can track the performance and health of a ElastiCache for Redis cluster.
You can use the CloudWatch service to monitor the performance and health of a cluster. CloudWatch
performance charts are shown in the ElastiCache for Redis console. You can also subscribe to ElastiCache
for Redis events to be notified about changes to a cluster, snapshot, parameter group, or security group.
For more information, see Monitoring Use with CloudWatch metrics.
API Version 2015-02-02
3