Automating an HDP Cluster with Puppet

What Is It?

A Puppet module from 2018–2019 that automates deploying an Ambari-managed Hortonworks Data Platform (HDP) cluster: installs and configures the Ambari server and agents, sets up PostgreSQL as the backing metastore for Hive and Oozie, and generates a self-signed certificate so the Ambari API runs over HTTPS instead of plain HTTP.

Chef, then Puppet. This wasn't the first time I automated an HDP deployment – I'd done it before with Chef, and this Puppet module was the second pass at the same problem: get Ambari installed, configured, and pointed at a database, without touching a server by hand. The Chef-era version didn't survive to today except for a note from that era on how to automate infrastructure application with Chef, but this Puppet one did.

The Stack

Built With

Puppet – module structure, Jenkins-built and installed onto a Puppet master
Ambari – cluster management for HDP, driven via its setup CLI and REST API
PostgreSQL – backing metastore for Hive and Oozie, provisioned with dedicated users per service
OpenSSL – self-signed certificate so the Ambari API runs over HTTPS
Jenkins – lint (puppet-lint), build the module package, then scp + ssh install it onto the Puppet master

What Actually Happens on a Node

Broken down by manifest, roughly in the order it runs:

Every Node (`hdp::default`)

Disable Python's SSL certificate verification – needed because the Ambari agent's Python-based tooling would otherwise reject the self-signed cert set up later.
Install java-1.8.0-openjdk.
Pull in Hortonworks' Ambari yum repo (2.5.1.0, CentOS 7).

Worker Nodes (`hdp::agent`)

Install ambari-agent.
Rewrite the agent's config so it points at the Ambari server's hostname instead of localhost.
Force PROTOCOL_TLSv1_2 in the agent's security settings.
Start and enable the ambari-agent service.

The Ambari Server (`hdp::server`)

Install ambari-server and run its silent setup.
Download the PostgreSQL JDBC driver and point Ambari at it instead of the default embedded database.
Override the default API port.
Generate a self-signed cert – write out the openssl req prompts as a canned answers file, then genrsa → req → self-sign, and enable HTTPS on the Ambari API with it.
Start the ambari-server service.
Create dedicated PostgreSQL users and databases for Hive and Oozie.

Scala, Spark, and This

This overlaps with when I was deep into Scala and Spark – the 2016 USA election analysis with Spark GraphX is from a couple years earlier, but HDP and Spark were the same world: bringing up and babysitting a cluster was the unglamorous prerequisite to actually being able to run Spark jobs on one.

The source is open, at jane-natucci/my-puppet-hdp.