Mastering Hadoop: Book Review

I came across a book Mastering Hadoop published by Packt and authored by Sandeep Karath. Here is my detail review about the book-

SUMMARY
This book is based on most popular massive parallel programming (MPP) framework "Hadoop" and its eco-system. This is an intermediate level book where author goes in depth on not only the principle subject but also on most of the supporting eco-systems like hive, pig, stream, etc. The book has 374 pages with 12 chapters, the ToC  itself is spanned across 7 pages! It has conceptual as well as hands on lab experiences with lot of code churned into.


OPINION
The book starts with genealogy of Hadoop where the author has nicely narrated the evolution of web search to current state and then various releases of Hadoop. Good reasoning as why Hadoop 2.0 was essential to move ahead from previous version. Touches the architecture starting from high level 3-layered, drilling down step by step to cluster and node level. Describes all the features of Hadoop2.x nicely and then talks about 4 major hadoop distros.

The concepts of MapReduce (MR) algorithm like merge and spills of intermediate outputs, stagglers, job counter, data joins packed in chapter 2. On the labs front, it has explained MapReduce example to great detail. explains custom RecordReader implementation. Some tips are really handy like heuristic formula for calculating optimum number of reducers. It is to be noted that the chapter assumes the reader has basic knowledge of this algo and it talks about the advance concepts.

Chapter 3, Pig talks about in-dept execution process of pig latin script and semantics along with many tips to optimize the query performance. It also shows practicle ways to use Pig for joining, as combiner, as abstract data analyzer for Data Acyclic Graph (DAG).

Hive in ch 4, another way to scoop the data from Hadoop in a conventional SQL-like style from RDBMS world. This is also covered in pretty details starting with its architecture to HiveQL semantics, execution steps and optimization tips like indexing, partioning, etc. It also captures exntensibles like UDF, UDAF and UDTF.

Hadoop Serialization and I/O talks about techniques of SerDe. After talking about Hadoop's own implementation and JDK's implementation, it slowly starts Apache's Avro tool with clearly stating its advantages and detail example. It explains the steps of Avro/Pig and Avro/Hive integration.

Chapter 6 & 7 talks about Yarn and Storm. YARN introduces the new architecture along with example of writing client and scheduling job plus ways to monitor it. Storm talks about low latency processing (aka real time processing), compares between Hadoop MR and Apache Storm with with the help of process diagrams and also explains the concepts of spout, bolt and topology with the help of java based example. It ends with installation on hadoop.

Then it flows with Hadoop off premise offerings (Cloud based!) like Anazon's AWS based EMR and Microsoft's Azure based HDInsight with enough comparison points as well as enough configuration steps.

Hadoop replacements gets into debating pros and cons of HDFS and possible extensions like AWS S3 which can make it more powerful. Actually adding more points on alternative systems likes of Cassandra, Ceph, GlusterFS would gave been value addition here.

Then it delves into features like HDFS Federation, Hadoop Security with its four pillars Authentication, Authorization, Auditing and Data Protection with each explained in great detail.

And here comes ch 12, Analytics using Hadoop: Its Machine Learning is a very interesting topic to have in this book but not sure if to that extent of detail. You need to have some statistical knowledge to understand some tpoics from the chapter as it talks about the terms/algos like tf-idf, k-means clustering. At the end, it talks about data analysis libraries: RHadoop, Mahout. Overall this chapter provides good handles on analytics.

The book ends with the appendix of "Hadoop for MS Windows". Thanks to Hortonworks! Now you can get Hadoop distro on win platform as well as their PaaS offering on MS Azure, more details follow in this chapter.

The author definitely seems having a rich experience in the field and is successful in conveying the depth of the subject through this book. Also the source code for the book is available at github.

In otherwise crowded Hadoop beginners' books, this one is different and catering an intermediate level. I wish all the very best to this efforts...

WHO SHOULD READ THIS BOOK
Anyone who has prior knowledge of Hadoop1.x can easily upgrade himself to Hadoop 2.x YARN. But then even the one with little knowledge of database and java can read this book to explore this new eco-system to enhance existing skills.