Azure AI Services at Scale for Cloud, Mobile, and Edge: Building Intelligent Apps with Azure Cognitive Services and Machine Learning

227 Pages·2022·46.641 MB·English

by  Simon Bisson, Mary Branscombe, Chris Hoder, Anand Raman

Azure AI Services
at Scale for Cloud,
Mobile, and Edge
Building Intelligent Apps with Azure Cognitive
Services and Machine Learning
B
is
s
o
Hn
od, B
era
r &ns
Aco
nm
a
nb
de
,
Simon Bisson, Mary Branscombe,
Chris Hoder & Anand Raman
Azure AI Services at Scale
for Cloud, Mobile, and Edge
Take advantage of the power of cloud and the latest AI
techniques. Whether you’re an experienced developer
“An invaluable
wanting to improve your app with AI-powered features or
you want to make a business process smarter by getting AI introduction to
to do some of the work, this book’s got you covered. Simon Microsoft’s family
Bisson, Mary Branscombe, Chris Hoder, and Anand Raman of AI services.”
show you how to build practical intelligent applications for
—Rik Hepworth
the cloud, mobile, browsers, and edge devices using a hands-
Chief Consulting Officer, Black Marble
on approach.
This book shows you how cloud AI services fit in alongside
familiar software development approaches, walks you
Simon Bisson is a freelance writer,
through key Microsoft AI services, and provides real-world specializing in enterprise technologies
examples of AI-oriented architectures that integrate different and development. He writes for several
Azure AI services. All you need to get started is a working publications, including InfoWorld,
ZDNet, and Computerworld.
knowledge of basic cloud concepts.
Mary Branscombe has worked as
• Become familiar with Azure AI offerings and capabilities
a freelance technology journalist
• Build intelligent applications using Azure Cognitive Services for many publications over three
decades. Recently she’s specialized
• Train, tune, and deploy models with Azure Machine Learning,
in AI, enterprise technology, and
PyTorch, and the Open Neural Network Exchange (ONNX) development.
• Learn to solve business problems using AI in the Power Chris Hoder is a principal program
Platform manager on the cognitive services
team at Microsoft. He’s responsible
• Use transfer learning to train vision, speech, and language
for product development of the Azure
models in minutes OpenAI Service.
B
is
Anand Raman leads program s
o
management for the AI services Hn
platform at Microsoft. Previously, he od, B
was the chief of staff for the Microsoft era
Azure AI and Data Group. r &ns
Aco
nm
a
nb
de
,
MACHINE LEARNING Twitter: @oreillymedia
linkedin.com/company/oreilly-media
US $59.99 CAN $74.99 youtube.com/oreillymedia
ISBN: 978-1-098-10804-5
Praise for Azure AI Services at Scale for Cloud, Mobile, and Edge
Creating modern applications powered by machine learning and AI is too hard, even
when you have the tools at hand to do it. In this book, Bisson, Branscombe, Hoder,
and Anand show how much simpler it can be, with up-to-date and clear
guidance on how to use the capabilities already built into Azure.
—John Montgomery, CVP, Azure AI Products, Microsoft
This book is an easy read for anyone looking to understand what Microsoft’s AI services
can do for them, how to use the AI services in their business and applications, and how
they should use AI responsibly for good. Highly recommended for anyone looking
to build intelligent applications at scale using Microsoft Azure AI services.
—Sundar Srinivasan, VP,
Search Technology Center India (STCI), Microsoft
An invaluable introduction to Microsoft’s family of AI services, with the added
bonus of a thought-provoking section on using AI responsibly. A genuine
something for everyone reference. Recommended without reservation.
—Rik Hepworth, Chief Consulting Officer, Black Marble
This book on Azure AI services is a must read for anyone looking for guidance on
building intelligent applications and implementing an AI-oriented architecture for their
organizations. Congratulations to the authors for delivering an insightful AI book that
has fused the nuts and bolts of AI with years of practical on-the-ground experience.
—Wee Hyong Tok, O’Reilly Book Author
and Principal GPM, Microsoft
The time has clearly come for building AI-oriented applications, to tap into
their immense potential, and to do so in a trustworthy, responsible way. This
book lays out a compelling path that will help innovators create the next
generation of intelligent systems using Azure’s popular AI services
in a way that will be both highly effective and sustainable.
—Dion Hinchcliffe, VP and Principal Analyst,
Constellation Research
This book is a one-stop-shop for what, how, why, and “whether
you should” questions for those looking to benefit from the
democratization of AI offered by Microsoft’s cloud services.
—Eric Boyd, CVP, Azure AI Platform, Microsoft
Azure AI Services at Scale for
Cloud, Mobile, and Edge
Building Intelligent Apps with Azure
,
Cognitive Services and Machine Learning
Simon Bisson, Mary Branscombe,
Chris Hoder, and Anand Raman
BBeeiijjiinngg BBoossttoonn FFaarrnnhhaamm SSeebbaassttooppooll TTookkyyoo
Azure AI Services at Scale for Cloud, Mobile, and Edge
by Simon Bisson, Mary Branscombe, Chris Hoder, and Anand Raman
Copyright © 2022 O’Reilly Media, Inc. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are
also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional
sales department: 800-998-9938 or [email protected].
Acquisitions Editor: Rebecca Novack Indexer: Ellen Troutman-Zaig
Development Editor: Gary O’Brien Interior Designer: David Futato
Production Editor: Beth Kelly Cover Designer: Karen Montgomery
Copyeditor: nSight, Inc. Illustrator: Kate Dullea
Proofreader: Piper Editorial Consulting, LLC
April 2022: First Edition
Revision History for the First Edition
2022-04-11: First Release
See http://oreilly.com/catalog/errata.csp?isbn=9781098108045 for release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Azure AI Services at Scale for Cloud,
Mobile, and Edge, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc.
The views expressed in this work are those of the authors, and do not represent the publisher’s views.
While the publisher and the authors have used good faith efforts to ensure that the information and
instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility
for errors or omissions, including without limitation responsibility for damages resulting from the use
of or reliance on this work. Use of the information and instructions contained in this work is at your
own risk. If any code samples or other technology this work contains or describes is subject to open
source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use
thereof complies with such licenses and/or rights.
This work is part of a collaboration between O’Reilly and Microsoft. See our statement of editorial
independence.
978-1-098-10804-5
[LSI]
Table of Contents
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Part I. Understanding AI-Oriented Architecture
1. An Introduction to AI-Oriented Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
What You Can Do with AI 3
From Milestones to Models to Architectures 4
Ready to Jump In? 7
Part II. Tools and Services to Help You Build AI-Oriented Architectures
2. Understanding AI Offerings and Capabilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
AI Services for All Types of Users 11
Microsoft’s AI Offerings 14
Managed AI Services and Infrastructure Options in Azure 15
Business Platforms with Extensible AI 17
AI for Big Data and Relational Data 17
Making Machine Learning More Portable 18
Cognitive Services 19
How to Determine What Tool Is Best for You 21
3. Train, Tune, and Deploy Models with Azure Machine Learning, ONNX, and PyTorch. . 27
Understanding Azure Machine Learning 27
Understanding Azure Machine Learning Studio 28
Getting Started with Azure Machine Learning 29
Setting Up a Machine Learning Environment 29
v
Integration with Azure Services 31
Using Visual Studio Code 32
The Azure Machine Learning Python SDK for Local Development 33
Azure Machine Learning and R 35
Build Your First Model Using Azure Machine Learning Studio 37
Use Automated Machine Learning 37
Using Designer 38
Using Azure Machine Learning with Notebooks and Python 39
Working with Azure Machine Learning Using Different Machine Learning
Frameworks 41
An Introduction to MLOps 43
Logging in Azure Machine Learning 45
Tuning Using Hyperparameters 45
Exporting with ONNX 46
Using ONNX with WinML 47
Using ONNX in Machine Learning Container Runtimes 47
Wrapping It Up 48
4. Using Azure Cognitive Services to Build Intelligent Applications. . . . . . . . . . . . . . . . . . 49
Using Prebuilt AI 49
The Core Azure Cognitive Services 54
Language 55
Azure OpenAI Service 61
Speech 64
Vision 69
Decision Making 74
Wrapping It Up 77
5. Using Azure Applied AI Services for Common Scenarios. . . . . . . . . . . . . . . . . . . . . . . . . . 79
Azure Applied AI Services 79
Azure Video Analyzer 81
Cognitive Search 83
Azure Form Recognizer 86
Azure Bot Service 91
Immersive Reader 93
Use Transfer Learning to Train Vision, Speech, and Language Models in
Minutes 94
Creating a Custom Vision Model 95
Creating a Custom Speech Model 99
Wrapping It Up 101
vi | Table of Contents
6. Machine Learning for Everyone: Low-Code and No-Code Experiences. . . . . . . . . . . . 103
The Microsoft Power Platform 104
Power BI and AI 105
AI Visualizations in Power BI 107
Using AI for Data Preparation in Power BI 109
Working with Custom Machine Learning Models in Power BI 110
Building Your Own Custom Models in Power BI 111
AI Builder 113
Training a Custom Form Processing Model 116
Using AI Builder Models 119
Using Cognitive Services and Other AI Models in Power Automate 126
Logic Apps and AI 134
Wrapping It Up 138
7. Responsible AI Development and Use. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Understanding Responsible AI 141
Responsible AI Improves Performance and Outcomes 142
Experiment and Iterate 143
Tools for Delivering Responsible AI 143
Tools for Transparency 145
Tools for AI Fairness 150
Tools for Reliability and Understanding Error 152
Human in the Loop Oversight 153
Wrapping It Up 155
Further Resources 156
8. Best Practices for Machine Learning Projects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
Working Well with Data 157
Sharing Data 158
Data Provenance and Governance 158
Making Machine Learning Projects Successful 162
Preparing Your Dataset 163
Establish Performance Metrics 165
Transparency and Trust 167
Experiment, Update, and Move On 167
Collaboration, Not Silos 168
Wrapping It Up 168
Table of Contents | vii
Part III. AI-Oriented Architectures in the Real World
9. How Microsoft Runs Cognitive Services for Millions of Users. . . . . . . . . . . . . . . . . . . . 171
AI for Anyone 172
Clusters and Containers 174
10. Seeing AI: Using Azure Machine Learning and Cognitive Services
in a Mobile App at Scale. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
Custom and Cloud Models 178
The Seeing AI Backend 180
Getting the Interface Right 181
11. Translating Multiple Languages at Scale for International Organizations. . . . . . . . . 183
Delivering Translations for an International Parliament 183
Connecting to Existing Audio-Visual (AV) Systems 184
Using Custom Speech Recognition for Specialized Vocabularies 184
From Specialized Prototype to General Application 185
Working within Constraints 186
12. Bringing Reinforcement Learning from the Lab to the Convenience Store. . . . . . . . 187
Two APIs, Eight Weeks, 100% Uplift 188
Afterword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
viii | Table of Contents