Integrate Elasticsearch into Hadoop to effectively visualize and analyze your data
About This Book
- Build production-ready analytics applications by integrating the Hadoop ecosystem with Elasticsearch
- Learn complex Elasticsearch queries and develop real-time monitoring Kibana dashboards to visualize your data
- Use Elasticsearch and Kibana to search data in Hadoop easily with this comprehensive, step-by-step guide
Who This Book Is For
This book is targeted at Java developers with basic knowledge on Hadoop. No prior Elasticsearch experience is expected.
What You Will Learn
- Set up the Elasticsearch-Hadoop environment
- Import HDFS data into Elasticsearch with MapReduce jobs
- Perform full-text search and aggregations efficiently using Elasticsearch
- Visualize data and create interactive dashboards using Kibana
- Check and detect anomalies in streaming data using Storm and Elasticsearch
- Inject and classify real-time streaming data into Elasticsearch
- Get production-ready for Elasticsearch-Hadoop based projects
- Integrate with Hadoop eco-system such as Pig, Storm, Hive, and Spark
In Detail
The Hadoop ecosystem is a de-facto standard for processing terra-bytes and peta-bytes of data. Lucene-enabled Elasticsearch is becoming an industry standard for its full-text search and aggregation capabilities. Elasticsearch-Hadoop serves as a perfect tool to bridge the worlds of Elasticsearch and Hadoop ecosystem to get best out of both the worlds. Powered with Kibana, this stack makes it a cakewalk to get surprising insights out of your massive amount of Hadoop ecosystem in a flash.
In this book, you'll learn to use Elasticsearch, Kibana and Elasticsearch-Hadoop effectively to analyze and understand your HDFS and streaming data.
You begin with an in-depth understanding of the Hadoop, Elasticsearch, Marvel, and Kibana setup. Right after this, you will learn to successfully import Hadoop data into Elasticsearch by writing MapReduce job in a real-world example. This is then followed by a comprehensive look at Elasticsearch essentials, such as full-text search analysis, queries, filters and aggregations; after which you gain an understanding of creating various visualizations and interactive dashboard using Kibana. Classifying your real-world streaming data and identifying trends in it using Storm and Elasticsearch are some of the other topics that we'll cover. You will also gain an insight about key concepts of Elasticsearch and Elasticsearch-hadoop in distributed mode, advanced configurations along with some common configuration presets you may need for your production deployments. You will have “Go production checklist” and high-level view for cluster administration for post-production. Towards the end, you will learn to integrate Elasticsearch with other Hadoop eco-system tools, such as Pig, Hive and Spark.
Style and approach
A concise yet comprehensive approach has been adopted with real-time examples to help you grasp the concepts easily.
Vishal Shukla is the CEO of Brevitaz Systems (http://brevitaz.com) and a technology evangelist at heart. He is a passionate software scientist and a big data expert. Vishal has extensive experience in designing modular enterprise systems. Since his college days (more than 11 years), Vishal has enjoyed coding in JVM-based languages. He also embraces design thinking and sustainable software development. He has vast experience in architecting enterprise systems in various domains. Vishal is deeply interested in technologies related to big data engineering, analytics, and machine learning. He set up Brevitaz Systems. This company delivers massively scalable and sustainable big data and analytics-based enterprise applications to their global clientele. With varied expertise in big data technologies and architectural acumen, the Brevitaz team successfully developed and re-engineered a number of legacy systems to state-of-the-art scalable systems. Brevitaz has imbibed in its culture agile practices, such as scrum, test-driven development, continuous integration, and continuous delivery, to deliver high-quality products to its clients. Vishal is a music and art lover. He loves to sing, play musical instruments, draw portraits, and play sports, such as cricket, table tennis, and pool, in his free time. You can contact Vishal at vishal.shukla@brevitaz.com and on LinkedIn at https://in.linkedin.com/in/vishalshu. You can also follow Vishal on Twitter at @vishal1shukla2.
