How to use Livy server to submit Spark job through a REST interface Posted on February 24, 2020, updated on February 19, 2020 by Yannick JaquierTable of contents Preamble Configuration Curl testing HTTP ERROR: 400 Python testing References Continue reading
Hive concatenate command issues and workaround Posted on January 24, 2020, updated on January 20, 2020 by Yannick JaquierTable of contents Preamble Concatenate command failing for access right HDFS default access right with Hive and Spark To go further Worked well till... References Continue reading
How to handle HDFS blocks with corrupted replicas or under replicated Posted on December 25, 2019, updated on January 30, 2023 by Yannick JaquierTable of contents Preamble Managing under replicated blocks Managing blocks with corrupted replicas References Continue reading
Hive fetch task really improving response time by bypassing MapReduce ? Posted on November 24, 2019, updated on February 28, 2020 by Yannick JaquierTable of contents Preamble Identical queries not same response time Partitions statistics and concatenation Fetch task performing worst than MapReduce ? To go further References Continue reading
Fetch Zookeeper information from Python with Kazoo to connect Hive Posted on October 25, 2019, updated on October 22, 2019 by Yannick JaquierTable of contents Preamble Kazoo development environment installation Python source code Kazoo testing References Continue reading
Hadoop backup: what parts to backup and how to do it ? Posted on September 27, 2019, updated on May 6, 2024 by Yannick JaquierTable of contents Preamble Mandatory parts to backup Configuration files Ambari server meta info NameNode metadata Ambari repository database Backup with Point In Time Recovery (PITR) capability Backup with no PITR capability Hive repository database Backup with Point In Time Recovery (PITR) capability Backup with no PITR capability Not mandatory parts to backup JournalNodes Parts nice to backup HDFS References Continue reading