Dataproc Cookbook: Running Spark and Hadoop Workloads in Google Cloud - Couverture souple

Sadineni, Narasimha; Venkataraman, Anuyogam

Couverture souple

ISBN 10 : 1098157702 ISBN 13 : 9781098157708

Editeur : O'Reilly Media, 2025

Synopsis

Get up to speed with Dataproc, the fully managed and highly scalable service for running open source big data tools and frameworks, including Hadoop, Spark, Flink, and Presto. This cookbook shows data engineers, data scientists, data analysts, and cloud architects how to use Dataproc, integrated with Google Cloud, for data lake modernization, ETL, and secure data science at a fraction of the cost.

Narasimha Sadineni from Google and former Googler Anu Venkataraman show you how to set up and run Hadoop and Spark jobs on Dataproc. You'll learn how to create Dataproc clusters and run data engineering and data science workloads in long-running, ephemeral, and serverless ways. In the process, you'll gain an understanding of Dataproc, orchestration, logging and monitoring, Spark History Server, and migration patterns.

This cookbook includes hands-on examples for configuring, logging, securing clusters, and migrating from on-prem to Dataproc. You'll learn how to:

• Create Dataproc clusters on Compute Engine and Kubernetes Engine
• Run data science workloads on Dataproc
• Execute Spark jobs on Dataproc Serverless
• Optimize Dataproc clusters to be cost effective and performant
• Monitor Spark jobs in various ways
• Orchestrate various workloads and activities
• Use different methods for migrating data and workloads from existing Hadoop clusters to Dataproc

A propos du livre

À propos des auteurs

Narasimha Sadineni is a data engineer at Google who has 12 years of experience in Data & Analytics. While working as a professional services team member at Google and Cloudera, he helped 50+ organizations in solving BigData problems using tools like Hadoop and Google Cloud technologies. He has several years of teaching experience in Hadoop.

Anu Venkataraman is a Senior Program Manager. She previously served as a Data Lake Engineer at Google, accumulating extensive experience in data technologies. Anu assists customers in migrating large-scale distributed systems to the cloud. She finds joy in speaking at universities and contributing technical blogs and videos to the Data community, aiming to expedite customers' journeys to the cloud. Anu played a key role as one of the leads for the Professional Services Tech Talk playlist on the Google Cloud Tech YouTube channel. She holds a Master's degree in Electrical and Computer Engineering from Ryerson University, specializing in Medical Image Processing and Machine Learning.

Détails de l'édition

• Éditeur: O'Reilly Media
• Date d'édition: 2025
• Langue: anglais
• ISBN 10: 1098157702
• ISBN 13: 9781098157708
• Reliure: Broché
• Nombre de pages: 436
• Coordonnées du fabricant: O'Reilly; https://oreilly.com; ; 1005 Gravenstein Highway North; Sebastopol; CA; 95472; Etats-Unis
• Personne responsable: non disponible