Kafka in Action PDF
Preview Kafka in Action
Dylan Scott Viktor Gamov Dave Klein Foreword by Jun Rao M A N N I N G Kafka in Action ii Kaf ka in Action DYLAN SCOTT VIKTOR GAMOV AND DAVE KLEIN FOREWORD BY JUN RAO MAN NING SHELTER ISLAND For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: [email protected] ©2022 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. The author and publisher have made every effort to ensure that the information in this book was correct at press time. The author and publisher do not assume and hereby disclaim any liability to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from negligence, accident, or any other cause, or from any usage of the information herein. Manning Publications Co. Development editor: Toni Arritola 20 Baldwin Road Technical development editors: Raphael Villela, Nickie Buckner PO Box 761 Review editor: Aleksandar Dragosavljevic´ Shelter Island, NY 11964 Production editor: Andy Marinkovich Copy editor: Frances Buran Proofreader: Katie Tennant Technical proofreaders: Felipe Esteban Vildoso Castillo, Mayur Patil, Sumant Tambe, Valentin Crettaz, and William Rudenmalm Typesetter and cover designer: Marija Tudor ISBN 9781617295232 Printed in the United States of America Dylan: I dedicate this work to Harper, who makes me so proud every day, and to Noelle, who brings even more joy to our family every day. I would also like to dedicate this book to my parents, sister, and wife, who are always my biggest supporters. Viktor: I dedicate this work to my wife, Maria, for her support during the process of writing this book. It’s a time-consuming task, time that I needed to carve out here and there. Without your encouragement, nothing would have ever happened. I love you. Also, I would like to dedicate this book to (and thank) my children, Andrew and Michael, for being so naïve and straightforward. When people asked where daddy is working, they would say, “Daddy is working in Kafka.” Dave: I dedicate this work to my wife, Debbie, and our children, Zachary, Abigail, Benjamin, Sarah, Solomon, Hannah, Joanna, Rebekah, Susanna, Noah, Samuel, Gideon, Joshua, and Daniel. Ultimately, everything I do, I do for the honor of my Creator and Savior, Jesus Christ. vi brief contents PART 1 GETTING STARTED .......................................................... 1 1 ■ Introduction to Kafka 3 2 ■ Getting to know Kafka 17 PART 2 APPLYING KAFKA .......................................................... 41 3 ■ Designing a Kafka project 43 4 ■ Producers: Sourcing data 66 5 ■ Consumers: Unlocking data 87 6 ■ Brokers 111 7 ■ Topics and partitions 129 8 ■ Kafka storage 144 9 ■ Management: Tools and logging 158 PART 3 GOING FURTHER ........................................................ 179 10 ■ Protecting Kafka 181 11 ■ Schema registry 197 12 ■ Stream processing with Kafka Streams and ksqlDB 209 vii viii BRIEF CONTENTS contents foreword xv preface xvi acknowledgments xviii about this book xx about the authors xxiii about the cover illustration xxiv PART 1 GETTING STARTED ............................................... 1 1 Introduction to Kafka 3 1.1 What is Kafka? 4 1.2 Kafka usage 8 Kafka for the developer 8 ■ Explaining Kafka to your manager 9 1.3 Kafka myths 10 Kafka only works with Hadoop® 10 ■ Kafka is the same as other message brokers 11 1.4 Kafka in the real world 11 Early examples 12 ■ Later examples 13 ■ When Kafka might not be the right fit 14 1.5 Online resources to get started 15 References 15 ix
The list of books you might like
