Learning Elastic Stack 6.0 A beginner’s guide to distributed search, analytics, and visualization using Elasticsearch, Logstash, and Kibana Pranav Shukla Sharath Kumar M N BIRMINGHAM - MUMBAI Learning Elastic Stack 6.0 Copyright © 2017 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. First published: December 2017 Production reference: 1201217 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-78728-186-8 www.packtpub.com Credits Authors Copy Editors Pranav Shukla Safis Editing Sharath Kumar M N Vikrant Phadkay Reviewer Project Coordinator Marcelo Ochoa Nidhi Joshi Commissioning Editor Proofreader Amey Varangaonkar Safis Editing Acquisition Editor Indexer Varsha Shetty Aishwarya Gangawane Content Development Editor Graphics Cheryl Dsa Tania Dutta Technical Editor Production Coordinator Sagar Sawant Shantanu Zagade Disclaimer Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries. Kibana is a trademark of Elasticsearch BV, registered in the U.S. and in other countries. Logstash is a trademark of Elasticsearch BV, registered in the U.S. and in other countries. Packetbeat is a trademark of Elasticsearch BV, registered in the U.S. and in other countries. Elastic is a trademark of Elasticsearch BV or Elastic Cloud is a trademark of Elasticsearch BV or Elastic Cloud Enterprise is a trademark of Elasticsearch BV or X-Pack is a trademark of Elasticsearch BV or Beats is a trademark of Elasticsearch BV or Winlogbeat is a trademark of Elasticsearch BV or Libbeat is a trademark of Elasticsearch BV or Metricbeat is a trademark of Elasticsearch BV or Filebeat is a trademark of Elasticsearch BV or Topbeat is a trademark of Elasticsearch BV or Heartbeat is a trademark of Elasticsearch BV. About the Authors Pranav Shukla is the founder and CEO of Valens DataLabs, a technologist, husband, and father of two. He is a big data architect and software craftsman who uses JVM-based languages. Pranav has diverse experience of over 14 years in architecting enterprise applications for Fortune 500 companies and start-ups. His core expertise lies in building JVM-based, scalable, reactive, and data-driven applications using Java/Scala, the Hadoop ecosystem, Apache Spark, and NoSQL databases. He is a big data engineering, analytics, and machine learning enthusiast. Pranav founded Valens DataLabs with a vision to help companies leverage data to their competitive advantage. Valens DataLabs specializes in developing next-generation, cloud- based, reactive, and data-intensive applications using big data and web technologies. The company believes in agile practices, lean principles, test-driven and behavior-driven development, continuous integration, and continuous delivery for sustainable software systems. In his free time, he enjoys reading books, playing musical instruments, singing, listening to music, and watching cricket. You can reach him via email at [email protected] and follow him on Twitter at @pranavshukla81. I would like to thank my wife Kruti Shukla for her unconditional love and support, our sons Sauhadra and Pratishth, my parents Dr Sharad Shukla and Varsha Shukla. I would like to thank my brother Vishal Shukla for playing an inspirational role in my career and also for inspiring me to write this book. I would like to thank Parth Mistry, Gopal Ghanghar, and Krishna Meet for their valuable feedback for the book. I am grateful to many who have contributed in shaping my career through fruitful interactions, particularly I would like to thank Umesh Kakkad, Eddie Moojen, Wart Fransen, Praveen Sameneni, Vinod Patel, Gopal Shah, and Sachin Bakshi. Sharath Kumar M N has done his masters in Computer Science at The University of Texas, Dallas, USA. He has been in the IT industry for more than ten years now and is the Elasticsearch Solutions Architect at Oracle. He is an Elastic Stack advocate, and being an avid speaker he has also given several tech talks in conferences such as the Oracle Code Event. Sharath is a certified trainer—Elastic Certified Instructor—one of the few technology experts in the world who has been certified by Elastic Inc to deliver their official from the creators of Elastic training. He is also a data science and machine learning enthusiast. In his free time, he enjoys trekking, listening to music, playing with his lovely pets Guddu and Milo and the geek in him loves exploring his Python skills for stock market analysis. You can reach him via email at [email protected]. I would like to thank my parents, Geetha and Nanjaiah, sister Dr Shilpa M N, brother-in- law Dr Sridhar and my friends - without their support I wouldn't have been able to finish my part of this book in time. I would also like to thank Packt Publishing team(specially Cheryl, Samuel, Varsha, Sagar) for providing a great opportunity for me to take part in this exciting journey. About the Reviewer Marcelo Ochoa works at the systems laboratory of Facultad de Ciencias Exactas, Universidad Nacional del Centro de la Provincia de Buenos Aires, Argentina. He is the CTO at www.scotas.com, a company that specializes in near-real-time search solutions using Apache Solr and Oracle. He divides his time between university jobs and external projects related to Oracle and big data technologies. He has worked on several Oracle-related projects, such as the translation of Oracle manuals and multimedia CBTs. His background is in database, network, web, and Java technologies. In the XML world, Marcelo is known as the developer of DB Generator for the Apache Cocoon project. He has worked on the open source projects DBPrism and DBPrism CMS, Lucene-Oracle integration using the Oracle JVM Directory implementation, and the Restlet.org project, where he worked on the Oracle XDB Restlet Adapter, an alternative to writing native REST web services inside a database- resident JVM. Since 2006, he has been part of an Oracle ACE program and has recently linked to a Docker Mentor program. Marcelo has coauthored Oracle Database Programming Using Java and Web Services by Digital Press and Professional XML Databases by Wrox Press. He has been a technical reviewer on several Packt books, such as Mastering Elastic Stack, Mastering Elasticsearch 5.x - Third Edition, Elasticsearch 5.x Cookbook - Third Edition, and so on. www.PacktPub.com For support files and downloads related to your book, please visit www.PacktPub.com. Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details. At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks. https://www.packtpub.com/mapt Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career. Why subscribe? Fully searchable across every book published by Packt Copy and paste, print, and bookmark content On demand and accessible via a web browser Customer Feedback Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at https://www.amazon.in/dp/1787281868. If you'd like to join our team of regular reviewers, you can email us at [email protected]. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!