Spark dynamic allocation how to configure and use it Posted on October 22, 2020, updated on October 13, 2020 by Yannick Jaquier Table of contents Preamble Spark dynamic allocation setup Spark dynamic allocation testing References Continue reading
INSERT OVERWRITE does not delete old directories Posted on September 22, 2020, updated on September 15, 2020 by Yannick Jaquier Table of contents Preamble INSERT OVERWRITE test case INSERT OVERWRITE does not delete old directories References Continue reading
Spark lineage issue and how to handle it with Hive Warehouse Connector Posted on August 23, 2020, updated on July 30, 2020 by Yannick Jaquier Table of contents Preamble Test case and problem Spark lineage solution Spark lineage second problem and partial solution References Continue reading
How to install and configure a standalone TEZ UI with HDP 3.x Posted on July 22, 2020, updated on September 15, 2020 by Yannick Jaquier Table of contents Preamble TEZ UI installation TEZ UI configuration TEZ UI bug correction trick References Continue reading
PySpark and Spark Scala Jupyter kernels cluster integration Posted on June 21, 2020, updated on June 9, 2020 by Yannick Jaquier Table of contents Preamble JupyterHub installation Jupyter kernels manual configuration Sparkmagic Jupyter kernels configuration References Continue reading
Hive Warehouse Connector integration in Zeppelin Notebook Posted on May 21, 2020, updated on May 18, 2020 by Yannick Jaquier Table of contents Preamble HWC on the command line Zeppelin Spark2 interpreter configuration Zeppelin Livy2 interpreter configuration References Continue reading