Setup Spark and Intellij on Windows to access a remote Hadoop cluster Posted on April 22, 2020, updated on November 29, 2021 by Yannick Jaquier Table of contents Preamble Spark installation and configuration Sbt installation and configuration Intellij IDEA from JetBrains Conclusion References Continue reading
On the importance to have good Hive statistics on your tables Posted on March 23, 2020, updated on March 20, 2020 by Yannick Jaquier Table of contents Preamble The problematic queries Problem has gone with good Hive statistics References Continue reading
How to use Livy server to submit Spark job through a REST interface Posted on February 24, 2020, updated on February 19, 2020 by Yannick Jaquier Table of contents Preamble Configuration Curl testing HTTP ERROR: 400 Python testing References Continue reading
Hive concatenate command issues and workaround Posted on January 24, 2020, updated on January 20, 2020 by Yannick Jaquier Table of contents Preamble Concatenate command failing for access right HDFS default access right with Hive and Spark To go further Worked well till... References Continue reading
How to handle HDFS blocks with corrupted replicas or under replicated Posted on December 25, 2019, updated on April 19, 2022 by Yannick Jaquier Table of contents Preamble Managing under replicated blocks Managing blocks with corrupted replicas References Continue reading
Hive fetch task really improving response time by bypassing MapReduce ? Posted on November 24, 2019, updated on February 28, 2020 by Yannick Jaquier Table of contents Preamble Identical queries not same response time Partitions statistics and concatenation Fetch task performing worst than MapReduce ? To go further References Continue reading