Amazon Personalize

259 Pages·2017·1.88 MB·English

Amazon Personalize
Developer Guide
Amazon Personalize Developer Guide
Amazon Personalize: Developer Guide
Copyright © 2023 Amazon Web Services, Inc. and/or its affiliates. All rights reserved.
Amazon's trademarks and trade dress may not be used in connection with any product or service that is not
Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or
discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may
or may not be affiliated with, connected to, or sponsored by Amazon.
Amazon Personalize Developer Guide
Table of Contents
What is Amazon Personalize? .............................................................................................................. 1
Pricing for Amazon Personalize .................................................................................................... 1
Guidance for first-time users ....................................................................................................... 2
Discovering Amazon Personalize with the Magic Movie Machine ............................................... 2
Navigating this guide.......................................................................................................... 2
Related AWS services and solutions .............................................................................................. 3
Third-party services.................................................................................................................... 3
Learn more............................................................................................................................... 4
How it works..................................................................................................................................... 5
Amazon Personalize workflows .................................................................................................... 5
Workflow for Domain dataset groups................................................................................... 5
Workflow for Custom dataset groups ................................................................................... 6
Amazon Personalize terms .......................................................................................................... 7
Data import and management ............................................................................................. 7
Training ............................................................................................................................ 9
Model deployment and recommendations ........................................................................... 10
Amazon Personalize data.......................................................................................................... 11
Interactions data.............................................................................................................. 11
User data........................................................................................................................ 14
Item data........................................................................................................................ 15
Setting up Amazon Personalize .......................................................................................................... 18
Sign up for AWS ...................................................................................................................... 18
Regions and endpoints.............................................................................................................. 18
Setting up permissions .............................................................................................................. 19
Creating a new IAM policy ................................................................................................. 19
Creating an IAM role for Amazon Personalize ....................................................................... 20
Giving Amazon Personalize access to Amazon S3 resources .................................................... 21
Giving Amazon Personalize permission to use your AWS KMS key ........................................... 25
Setting up the AWS CLI ............................................................................................................ 27
Setting up the AWS SDKs .......................................................................................................... 28
Getting started ................................................................................................................................ 29
Getting started prerequisites ..................................................................................................... 29
Creating the training data (Domain dataset group) ............................................................... 30
Creating the training data (Custom dataset group) ............................................................... 30
Getting started with a Domain dataset group .............................................................................. 31
Getting started with a Domain dataset group (console) ......................................................... 31
Getting started with a Domain dataset group (SDK for Java 2.x) ............................................. 36
Getting started with a Domain dataset group (SDK for Python (Boto3)) ................................... 42
Getting started with a Custom dataset group .............................................................................. 46
Getting started (console) ................................................................................................... 46
Getting started (AWS CLI) ................................................................................................. 54
Getting started (SDK for Python (Boto3)) ............................................................................ 61
Getting started (SDK for Java 2.x) ...................................................................................... 65
Cleaning up resources ............................................................................................................... 73
Cleaning up domain-based resources .................................................................................. 73
Cleaning up custom resources ............................................................................................ 74
Datasets and schemas....................................................................................................................... 75
Schema formatting requirements............................................................................................... 75
Schema data types ........................................................................................................... 76
Domain datasets and schemas................................................................................................... 76
VIDEO_ON_DEMAND datasets and schemas ......................................................................... 77
ECOMMERCE datasets and schemas .................................................................................... 82
Custom datasets and schemas................................................................................................... 87
Custom dataset and schema requirements ........................................................................... 87
iii
Amazon Personalize Developer Guide
Interactions dataset requirements ...................................................................................... 89
Users dataset requirements ............................................................................................... 90
Items dataset requirements ............................................................................................... 91
Creating a schema using Python ........................................................................................ 93
Domain dataset groups..................................................................................................................... 94
Creating a Domain dataset group ............................................................................................... 95
Creating a Domain dataset group and importing interaction data (console) .............................. 95
Creating a Domain dataset group and importing interaction data (AWS CLI) ............................. 97
Creating a Domain dataset group and importing interaction data (AWS SDKs) ......................... 100
Creating recommenders........................................................................................................... 108
Choosing recommender use cases ..................................................................................... 108
Minimum recommendation requests per second and auto-scaling ......................................... 112
Creating recommenders (console) ..................................................................................... 113
Creating recommenders (AWS CLI) .................................................................................... 114
Creating recommenders (AWS SDKs) ................................................................................. 114
Evaluating a recommender ...................................................................................................... 116
Retrieving metrics........................................................................................................... 116
Metric definitions........................................................................................................... 118
Example........................................................................................................................ 119
Additional resources........................................................................................................ 120
Getting recommendations from a recommender ......................................................................... 120
Getting recommendations with a recommender (console) .................................................... 120
Getting recommendations with a recommender (AWS CLI) ................................................... 121
Getting recommendations with a recommender (AWS SDKs) ................................................ 121
Promoting items............................................................................................................. 123
Managing Domain dataset group resources ................................................................................ 128
Creating datasets............................................................................................................ 129
Managing data............................................................................................................... 129
Managing recommenders................................................................................................. 131
Maintaining recommendation relevance ..................................................................................... 135
Custom dataset groups ................................................................................................................... 137
Determining your use case ....................................................................................................... 137
Amazon Personalize use cases .......................................................................................... 137
What data to import ....................................................................................................... 138
Preparing and importing data .................................................................................................. 138
Step 1: Creating a Custom dataset group .......................................................................... 139
Step 2: Creating a dataset and a schema ........................................................................... 142
Step 3: Importing your historical data .............................................................................. 147
Creating a solution ................................................................................................................. 168
Step 1: Choosing a recipe ................................................................................................ 168
Step 2: Configuring a solution.......................................................................................... 204
Step 3: Creating a solution version ................................................................................... 217
Step 4: Evaluating a solution version with metrics .............................................................. 223
Creating a campaign ............................................................................................................... 228
Minimum provisioned TPS ............................................................................................... 229
Creating a campaign (console) .......................................................................................... 229
Creating a campaign (AWS CLI) ........................................................................................ 230
Creating a campaign (AWS SDKs) ..................................................................................... 231
Updating a campaign...................................................................................................... 233
Getting recommendations....................................................................................................... 235
Recommendation scores.................................................................................................. 235
Getting real-time recommendations .................................................................................. 235
Getting batch recommendations and user segments ........................................................... 249
Maintaining recommendation relevance ..................................................................................... 263
Keeping datasets current ................................................................................................. 263
Keeping solution versions up to date ................................................................................ 264
Recording events............................................................................................................................ 266
iv
Amazon Personalize Developer Guide
Requirements for recording events and training a model ............................................................. 266
How real-time events influence recommendations ...................................................................... 267
Creating an event tracker ........................................................................................................ 267
Recording events with the PutEvents operation .......................................................................... 269
Recording events for anonymous users ...................................................................................... 273
Recording impressions data ..................................................................................................... 273
Event metrics and attribution reports ....................................................................................... 274
Sample Jupyter notebook ........................................................................................................ 275
Sample implementations......................................................................................................... 276
Exporting a dataset ........................................................................................................................ 277
Dataset export job permissions requirements ............................................................................. 277
Service role policy for exporting a dataset ......................................................................... 278
Amazon S3 bucket policy for exporting a dataset ............................................................... 278
Creating a dataset export job (console) ..................................................................................... 279
Creating a dataset export job (AWS CLI) .................................................................................... 279
Creating a dataset export job (AWS SDKs) ................................................................................. 280
Filtering results.............................................................................................................................. 283
Filter expressions.................................................................................................................... 283
Creating filter expressions ............................................................................................... 284
Filter expression examples ............................................................................................... 286
Filtering real-time recommendations ......................................................................................... 288
Filtering real-time recommendations (console) ................................................................... 289
Filtering real-time recommendations (AWS CLI) .................................................................. 293
Filtering real-time recommendations (AWS SDKs) ............................................................... 294
Filtering batch recommendations and user segments .................................................................. 298
Providing filter values in your input JSON ......................................................................... 298
Filtering batch workflows (console) ................................................................................... 299
Filtering batch workflows (AWS SDKs) ............................................................................... 299
Measuring impact of recommendations ............................................................................................. 301
Guidelines and requirements .................................................................................................... 301
Giving Amazon Personalize access to CloudWatch ............................................................... 302
Giving Amazon Personalize access to your Amazon S3 bucket ............................................... 303
Creating a metric attribution .................................................................................................... 304
Creating a metric attribution (console) .............................................................................. 304
Creating a metric attribution (AWS CLI) ............................................................................. 305
Creating a metric attribution (AWS SDKs) .......................................................................... 306
Managing a metric attribution .................................................................................................. 307
Updating a metric attribution .......................................................................................... 308
Deleting a metric attribution ............................................................................................ 311
Publishing and viewing results ................................................................................................. 312
Viewing metrics in CloudWatch ........................................................................................ 312
Publishing metrics to Amazon S3 ..................................................................................... 313
Tagging resources........................................................................................................................... 317
Managing tags....................................................................................................................... 317
Additional information.................................................................................................... 318
Adding tags to Amazon Personalize resources ............................................................................ 318
Adding tags (console) ...................................................................................................... 318
Adding tags (AWS CLI) .................................................................................................... 319
Adding tags (AWS SDKs) .................................................................................................. 319
Using tags in IAM policies................................................................................................ 322
Using tags in IAM policies........................................................................................................ 322
Troubleshooting ............................................................................................................................. 325
Frequently asked questions ...................................................................................................... 325
Data import and management ......................................................................................... 325
Creating a solution and solution version (Custom dataset group) .......................................... 326
Model deployment (Custom dataset group campaigns) ........................................................ 326
Recommendations.......................................................................................................... 327
v
Amazon Personalize Developer Guide
Filtering recommendations.............................................................................................. 327
Error messages....................................................................................................................... 328
Data import and management ......................................................................................... 328
Creating a solution and solution version (Custom dataset group) .......................................... 328
Model deployment (Custom dataset group campaigns) ........................................................ 329
Recommenders (Domain dataset group) ............................................................................ 329
Recommendations.......................................................................................................... 329
Filtering recommendations.............................................................................................. 329
Specifying resources with AWS CloudFormation ................................................................................. 331
Amazon Personalize and AWS CloudFormation templates ............................................................ 331
Example AWS CloudFormation templates for Amazon Personalize resources ................................... 331
CreateDatasetGroup........................................................................................................ 332
CreateDataset................................................................................................................ 332
CreateSchema................................................................................................................ 333
CreateSolution............................................................................................................... 333
Learn more about AWS CloudFormation .................................................................................... 334
Security......................................................................................................................................... 335
Data protection...................................................................................................................... 335
Data encryption............................................................................................................. 336
Identity and access management .............................................................................................. 337
Audience....................................................................................................................... 337
Authenticating with identities.......................................................................................... 337
Managing access using policies ......................................................................................... 339
How Amazon Personalize works with IAM .......................................................................... 341
Cross-service confused deputy prevention ......................................................................... 344
Identity-based policy examples ........................................................................................ 344
Troubleshooting............................................................................................................. 348
Logging and monitoring.......................................................................................................... 350
Monitoring..................................................................................................................... 350
CloudWatch metrics for Amazon Personalize ...................................................................... 353
Logging Amazon Personalize API calls with AWS CloudTrail .................................................. 356
Compliance validation............................................................................................................. 357
Resilience.............................................................................................................................. 358
Infrastructure security............................................................................................................. 358
Endpoints and quotas..................................................................................................................... 359
Amazon Personalize endpoints and regions ............................................................................... 359
Compliance............................................................................................................................ 359
Service quotas........................................................................................................................ 359
Requesting a quota increase .................................................................................................... 363
API reference................................................................................................................................. 364
Actions.................................................................................................................................. 364
Amazon Personalize........................................................................................................ 366
Amazon Personalize Events .............................................................................................. 539
Amazon Personalize Runtime ........................................................................................... 545
Data Types............................................................................................................................ 554
Amazon Personalize........................................................................................................ 555
Amazon Personalize Events .............................................................................................. 664
Amazon Personalize Runtime ........................................................................................... 670
Common Errors...................................................................................................................... 673
Common Parameters.............................................................................................................. 675
Document history........................................................................................................................... 677
AWS glossary................................................................................................................................. 683
vi
Amazon Personalize Developer Guide
Pricing for Amazon Personalize
What is Amazon Personalize?
Amazon Personalize is a fully managed machine learning service that uses your data to generate item
recommendations for your users. It can also generate user segments based on the users' affinity for
certain items or item metadata.
Common use case examples include the following:
• Personalizing a video streaming app – You can use preconfigured or customizable Amazon
Personalize resources to add multiple types of personalized video recommendations to your streaming
app. For example, Top picks for you, More like X and Most popular video recommendations.
• Adding product recommendations to an ecommerce app – You can use preconfigured or
customizable Amazon Personalize resources to add multiple types of personalized product
recommendations to your retail app. For example, Recommended for you, Frequently bought together
and Customers who viewed X also viewed product recommendations.
• Creating personalized emails – You can use customizable Amazon Personalize resources to generate
batch recommendations for all users on an email list. Then you can use an AWS service (p. 3) or
third party service (p. 3) to send users personalized emails recommending items in your catalog.
• Creating a targeted marketing campaign – You can use Amazon Personalize to generate segments
of users who will most likely interact with items in your catalog. Then you can use an AWS
service (p. 3) or third party service (p. 3) to create a targeted marketing campaign that
promotes different items to different user segments.
Amazon Personalize includes API operations for real-time personalization, and batch operations
for bulk recommendations and user segments. You can get started quickly with use-case optimized
recommenders for your business domain, or you can create your own configurable custom resources.
With Amazon Personalize, your data can come from both your historical bulk interaction records in a CSV
file, and real-time events from your users as they interact with your catalog. Before Amazon Personalize
can generate recommendations, your interactions data must have:
• At minimum 1000 interactions records from users interacting with items in your catalog. These
interactions can be from bulk imports, or streamed events, or both.
• At minimum 25 unique user IDs with at least 2 interactions for each.
Different use cases may have additional data requirements. If you don't have enough data, you can
use Amazon Personalize to first collect real-time event data. After you have recorded enough events,
Amazon Personalize can generate recommendations.
Topics
• Pricing for Amazon Personalize (p. 1)
• Guidance for first-time Amazon Personalize users (p. 2)
• Related AWS services and solutions (p. 3)
• Third-party services (p. 3)
• Learn more (p. 4)
Pricing for Amazon Personalize
With Amazon Personalize, there are no minimum fees and no upfront commitments. The AWS Free Tier
provides a monthly quota of up to 20 GB of data processing per available AWS region, up to 100 hours
1
Amazon Personalize Developer Guide
Guidance for first-time users
of training time per eligible AWS region, and up to 50 TPS-hours of real-time recommendations/month.
The free tier is valid for the first two months of usage.
For a complete list of charges and prices, see Amazon Personalize pricing.
Guidance for first-time Amazon Personalize users
If you're a first-time user of Amazon Personalize, the following resources can help you get started.
Topics
• Discovering Amazon Personalize with the Magic Movie Machine (p. 2)
• Navigating this guide (p. 2)
Discovering Amazon Personalize with the Magic
Movie Machine
The Magic Movie Machine is an interactive learning experience. It helps you discover Amazon Personalize
features and learn more about generating recommendations. For a short introduction, see the video
below. Then try the Magic Movie Machine.
Getting Started with Amazon Personalize
Navigating this guide
We recommend you read the following sections in order:
1. How it works (p. 5) – This section introduces the Amazon Personalize workflow and walks
you through the steps to create personalized experiences for your users. This section also includes
common Amazon Personalize terms and their definitions. Start with this section to make sure you
have good understanding of Amazon Personalize workflows and terms before you start getting
recommendations.
2. Setting up Amazon Personalize (p. 18) – In this section you set up your AWS account, set up the
required permissions to use Amazon Personalize, and set up the AWS CLI and the AWS SDKs to use
and manage Amazon Personalize.
3. Getting started (p. 29) – In this section you get started using Amazon Personalize with a simple
movie dataset. Complete these tutorials to get hands-on experience with Amazon Personalize. You can
choose to either get started with a Domain dataset group or a Custom dataset group:
• To get started creating a Domain dataset group, complete the Getting started
prerequisites (p. 29) and then start the tutorials in Getting started with a Domain dataset
group (p. 31).
• To get started with a Custom dataset group, complete the Getting started prerequisites (p. 29)
and then start the tutorials in Getting started with a Domain dataset group (p. 31).
4. Depending on your application, complete either of the following sections:
• Domain dataset groups (p. 94) – If you have a streaming video or ecommerce application,
follow the procedures in this section to create a Domain dataset group and recommenders for your
use cases. These procedures build upon what you learned when completing the Domain dataset
group getting started exercises. They provide more in-depth information about recommenders and
domain use cases.
• Custom dataset groups (p. 137) – If you don't have a streaming video or ecommerce application,
follow the procedures in this section to create a Custom dataset group. These procedures build
2
Amazon Personalize Developer Guide
Related AWS services and solutions
upon what you learned when completing the Custom dataset group getting started exercises. They
provide more in depth information about custom recipes and models.
5. Recording events (p. 266) – This section covers how to record user interaction events in real time.
After you have set up your Amazon Personalize resources, complete this section to learn how to keep
your Interactions dataset up to date with your users' behavior. You do this by recording interaction
events with an event tracker and the PutEvents (p. 540) operation.
6. Filtering recommendations and user segments (p. 283) – This section covers how to filter
recommendations. Complete this section to learn how to construct filter expressions to filter
recommendations based on custom criteria. For example, you might not want to recommend products
that a user has already purchased, or recommend movies that a user has already watched.
Related AWS services and solutions
Amazon Personalize integrates seamlessly with other AWS services and solutions. For example, you can:
• Use AWS Amplify to record user interaction events. Amplify includes a JavaScript library for recording
events from web client applications, and a library for recording events in server code. For more
information, see Amplify - analytics.
• Automate and schedule Amazon Personalize tasks with Maintaining Personalized Experiences with
Machine Learning. This AWS Solutions Implementation automates the Amazon Personalize workflow,
including data import, solution version training, and batch workflows.
• Use Amazon CloudWatch Evidently to perform A/B testing with Amazon Personalize
recommendations. For more information, see Perform launches and A/B experiments with CloudWatch
Evidently in the Amazon CloudWatch User Guide.
• Use Amazon Pinpoint to create targeted marketing campaigns. For an example that shows how to use
Amazon Pinpoint and Amplify to add Amazon Personalize recommendations to a marketing email
campaign and a web app, see Web Analytics with Amplify.
Third-party services
Amazon Personalize works well with various third-party services.
• Amplitude – You can use Amplitude to track user actions to help you understand your users' behavior.
For information on using Amplitude and Amazon Personalize, see the following AWS Partner
Network (APN) blog post: Measuring the Effectiveness of Personalization with Amplitude and Amazon
Personalize.
• Braze – You can use Braze to send users personalized emails recommending items in your catalog.
Braze is a market leading messaging platform (email, push, SMS). For a workshop that shows how to
integrate Amazon Personalize and Braze, see Amazon Personalize workshop.
• mParticle – You can use mParticle to collect event data from your app. For an example that shows how
to use mParticle and Amazon Personalize to implement personalized product recommendations, see
How to harness the power of a CDP for machine learning: Part 2.
• Optimizely – You can use Optimizely to perform A/B testing with Amazon Personalize
recommendations. For information on using Optimizely and Amazon Personalize, see Optimizely
integrates with Amazon Personalize to combine powerful machine learning with experimentation.
• Segment – You can use Segment to send your data to Amazon Personalize. For more information on
integrating Segment with Amazon Personalize, see Amazon Personalize Destination.
For a complete list of partners, see Amazon Personalize Partners.
3
Amazon Personalize Developer Guide
Learn more
Learn more
The following resources provide additional information about Amazon Personalize:
• For a quick reference to help you determine if Amazon Personalize fits your use case, see the Amazon
Personalize Cheat Sheet in the Amazon Personalize samples repository.
• For a series of videos on how to use Amazon Personalize, see the Amazon Personalize Deep Dive Video
Series found on YouTube.
• For in-depth tutorials and code samples, see the amazon-personalize-samples GitHub repository.
4