C o m pli m e nts of Reactive MMiiccrroossyysstteemmss The Evolution of Microservices at Scale Jonas Bonér Reactive Microsystems The Evolution of Microservices at Scale Jonas Bonér BBeeiijjiinngg BBoossttoonn FFaarrnnhhaamm SSeebbaassttooppooll TTookkyyoo Reactive Microsystems by Jonas Bonér Copyright © 2017 Lightbend, Inc. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com/safari). For more information, contact our corporate/institutional sales department: 800-998-9938 or [email protected]. Editor: Brian Foster Interior Designer: David Futato Production Editor: Melanie Yarbrough Cover Designer: Karen Montgomery Copyeditor: Octal Publishing Services Illustrator: Rebecca Demarest Proofreader: Matthew Burgoyne August 2017: First Edition Revision History for the First Edition 2017-08-07: First Release The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Reactive Microsys‐ tems, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limi‐ tation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsi‐ bility to ensure that your use thereof complies with such licenses and/or rights. 978-1-491-99433-7 [LSI] Table of Contents Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v 1. Essential Traits of an Individual Microservice. . . . . . . . . . . . . . . . . . . . 1 Isolate All the Things 1 Single Responsibility 2 Own Your State, Exclusively 3 Stay Mobile, but Addressable 6 2. Slaying the Monolith. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Don’t Build Microliths 9 3. Microservices Come in Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Embrace Uncertainty 11 We Are Always Looking into the Past 12 The Cost of Maintaining the Illusion of a Single Now 13 Learn to Enjoy the Silence 13 Avoid Needless Consistency 14 4. Events-First Domain-Driven Design. . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Focus on What Happens: The Events 17 Think in Terms of Consistency Boundaries 21 Manage Protocol Evolution 25 5. Toward Reactive Microsystems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Embrace Reactive Programming 28 Embrace Reactive Systems 35 Microservices Come as Systems 44 iii 6. Toward Scalable Persistence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Moving Beyond CRUD 49 Event Logging—The Scalable Seamstress 50 Transactions—The Anti-Availability Protocol 59 7. The World Is Going Streaming. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Three Waves Toward Fast Data 68 Leverage Fast Data in Microservices 68 8. Next Steps. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Further Reading 71 Start Hacking 72 iv | Table of Contents Introduction The Evolution of Scalable Microservices In this report, I will discuss strategies and techniques for building scalable and resilient microservices, working our way through the evolution of a microservices-based system. Beginning with a monolithic application, we will refactor it, briefly land at the antipattern of single instance—not scalable or resilient— microliths (micro monoliths), before quickly moving on, and step by step work our way toward scalable and resilient microservices (microsystems). Along the way, we will look at techniques from reactive systems, reactive programming, event-driven programming, events-first domain-driven design, event sourcing, command query responsibil‐ ity segregation, and more. v We Can’t Make the Horse Faster If I had asked people what they wanted, they would have said faster horses. —Henry Ford1 Today’s applications are deployed to everything from mobile devices to cloud-based clusters running thousands of multicore processors. Users have come to expect millisecond response times (latency) and close to 100 percent uptime. And, by “user,” I mean both humans and machines. Traditional architectures, tools, and products as such simply won’t cut it anymore. We need new solutions that are as dif‐ ferent from monolithic systems as cars are from horses. Figure P-1 sums up some of the changes that we have been through over the past 10 to 15 years. Figure P-1. Some fundamental changes over the past 10 to 15 years To paraphrase Henry Ford’s classic quote: we can’t make the horse faster anymore; we need cars for where we are going. So, it’s time to wake up, time to retire the monolith, and to decom‐ pose the system into manageable, discrete services that can be scaled individually, that can fail, be rolled out, and upgraded in isolation. 1 It’s been debated whether Henry Ford actually said this. He probably didn’t. Regardless, it’s a great quote. vi | Introduction They have had many names over the years (DCOM, CORBA, EJBs, WebServices, etc.). Today, we call them microservices. We, as an industry, have gone full circle again. Fortunately, it is more of an upward spiral as we are getting a little bit better at it every time around. We Need to Learn to Exploit Reality Imagination is the only weapon in the war against reality. —Lewis Carroll, Alice in Wonderland We have been spoiled by the once-believed-almighty monolith— with its single SQL database, in-process address space, and thread- per-request model—for far too long. It’s a fairytale world in which we could assume strong consistency, one single globally consistent “now” where we could comfortably forget our university classes on distributed systems. Knock. Knock. Who’s There? Reality! We have been living in this illusion, far from reality. We will look at microservices, not as tools to scale the organization and the development and release process (even though it’s one of the main reasons for adopting microservices), but from an architecture and design perspective, and put it in its true architectural context: distributed systems. One of the major benefits of microservices-based architecture is that it gives us a set of tools to exploit reality, to create systems that closely mimic how the world works. Don’t Just Drink the Kool-Aid Everyone is talking about microservices in hype-cycle speak; they are reaching the peak of inflated expectations. It is very important to not just drink the Kool-Aid blindly. In computer science, it’s all about trade-offs, and microservices come with a cost. Microservices can do wonders for the development speed, time-to-market, and Continuous Delivery for a large organization, and it can provide a great foundation for building elastic and resilient systems that can Introduction | vii take full advantage of the cloud.2 That said, it also can introduce unnecessary complexity and simply slow you down. In other words, do not apply microservices blindly. Think for yourself. 2 If approached from the perspective of distributed systems, which is the topic of this report. viii | Introduction