Apache Flume: Distributed Log Collection for Hadoop - Second Edition - Couverture souple

Hoffman, Steve

Couverture souple

ISBN 10 : 1784392170 ISBN 13 : 9781784392178

Editeur : Packt Publishing, 2015

Synopsis

If you are a Hadoop programmer who wants to learn about Flume to be able to move datasets into Hadoop in a timely and replicable manner, then this book is ideal for you. No prior knowledge about Apache Flume is necessary, but a basic knowledge of Hadoop and the Hadoop File System (HDFS) is assumed.

A propos du livre

Présentation de l'éditeur

Design and implement a series of Flume agents to send streamed data into Hadoop

About This Book

• Construct a series of Flume agents using the Apache Flume service to efficiently collect, aggregate, and move large amounts of event data
• Configure failover paths and load balancing to remove single points of failure
• Use this step-by-step guide to stream logs from application servers to Hadoop's HDFS

Who This Book Is For

If you are a Hadoop programmer who wants to learn about Flume to be able to move datasets into Hadoop in a timely and replicable manner, then this book is ideal for you. No prior knowledge about Apache Flume is necessary, but a basic knowledge of Hadoop and the Hadoop File System (HDFS) is assumed.

What You Will Learn

• Understand the Flume architecture, and also how to download and install open source Flume from Apache
• Follow along a detailed example of transporting weblogs in Near Real Time (NRT) to Kibana/Elasticsearch and archival in HDFS
• Learn tips and tricks for transporting logs and data in your production environment
• Understand and configure the Hadoop File System (HDFS) Sink
• Use a morphline-backed Sink to feed data into Solr
• Create redundant data flows using sink groups
• Configure and use various sources to ingest data
• Inspect data records and move them between multiple destinations based on payload content
• Transform data en-route to Hadoop and monitor your data flows

In Detail

Apache Flume is a distributed, reliable, and available service used to efficiently collect, aggregate, and move large amounts of log data. It is used to stream logs from application servers to HDFS for ad hoc analysis.

This book starts with an architectural overview of Flume and its logical components. It explores channels, sinks, and sink processors, followed by sources and channels. By the end of this book, you will be fully equipped to construct a series of Flume agents to dynamically transport your stream data and logs from your systems into Hadoop.

A step-by-step book that guides you through the architecture and components of Flume covering different approaches, which are then pulled together as a real-world, end-to-end use case, gradually going from the simplest to the most advanced features.

Biographie de l'auteur

Steve Hoffman

Steve Hoffman has 32 years of experience in software development, ranging from embedded software development to the design and implementation of large-scale, service-oriented, object-oriented systems. For the last 5 years, he has focused on infrastructure as code, including automated Hadoop and HBase implementations and data ingestion using Apache Flume. Steve holds a BS in computer engineering from the University of Illinois at Urbana-Champaign and an MS in computer science from DePaul University. He is currently a senior principal engineer at Orbitz Worldwide (http://orbitz.com/). More information on Steve can be found at http://bit.ly/bacoboy and on Twitter at @bacoboy. This is the first update to Steve's first book, Apache Flume: Distributed Log Collection for Hadoop, Packt Publishing.

Détails de l'édition

• Éditeur: Packt Publishing
• Date d'édition: 2015
• Langue: anglais
• ISBN 10: 1784392170
• ISBN 13: 9781784392178
• Reliure: Broché
• Numéro d'édition: 2
• Nombre de pages: 178
• Coordonnées du fabricant: non disponible
• Personne responsable: gpsr@libri.de; gpsr@libri.de; ; Friedensallee 273; Hamburg; 22763; Allemagne