README

This directory contains sample files to automate Hadoop Kerberos security setup with Centrify DirectControl Agent.

Hadoop distributions from Cloudera, Hortonworks, and MapR are supported.

Pre-requisites

Centrify DirectControl Agent 5.3.1 (Centrify Server Suite 2016.1) or later running on all cluster nodes.
Root privilege to run sample scripts.
SSH and SCP access to all cluster nodes without password prompt.
Valid TGT of Kerberos principal with administrative privilege to modify Active Directory.

Remarks:

This set of sample files has been available since Centrify DirectControl Agent 5.2.2. However, functionality may various from release to release. Please refer to the corresponding document in the same release for exact features supported.
Public key authentication or single sign-on (SSO) can be used to avoid password prompt on SSH and SCP. Centrify kerberized OpenSSH supports SSO.
kinit can be used to get a valid TGT of Kerberos principal with administrative privilege (e.g. Administrator@EXAMPLE.COM)

Sample files

kerberos_security_setup.pl:
Sample Perl script to automate Hadoop Kerberos security setup through Centrify DirectControl Agent. Please refer to the Usage section below for detail.

hadoop.conf:
Configuration file for kerberos_security_setup.pl.

host-principal-keytab-list.csv:
Sample CSV file used by kerberos_security_setup.pl as input. This CSV file can be downloaded from previous Apache Ambari v1.6.1 or created manually. Please refer to doc/FAQ for file format and other details.

host-principal-keytab-list-v212.csv:
Another sample CSV file also supported by kerberos_security_setup.pl as input. This new CSV file can be downloaded from Apache Ambari v2.1.2 or created manually. Please refer to doc/FAQ for file format and other details.

CreateADObject.tcl:
Sample adedit script to create new container (or organizational unit) on Active Directory through Centrify DirectControl Agent. This sample script is provided in case a new container (or organizational unit) is needed to store objects created by kerberos_security_setup.pl for Hadoop service principals, but other ways to access Active Directory are not available (e.g. through Windows UI). Note that Active Directory administrative privilege is required. For usage, please refer to comments in the sample script.

Usage

The sample script kerberos_security_setup.pl is used to automate Kerberos setup for Hadoop clusters.

Following the CSV file, the sample script can create AD objects and Kerberos keytab files for Hadoop service principals. To do so:

./kerberos_security_setup.pl --input host-principal-keytab-list.csv --create

The Kerberos keytab files are stored locally. Following the CSV file, the sample script can also distribute Kerberos keytab files to cluster nodes via SCP. To do so:

./kerberos_security_setup.pl --input host-principal-keytab-list.csv --deploy

The sample script is able to clean up deployed Kerberos keytab files from cluster nodes. To do so:

./kerberos_security_setup.pl --input host-principal-keytab-list.csv --undeploy

The sample script is also able to clean up AD objects and locally stored Kerberos keytab files using the --delete command. To do so:

./kerberos_security_setup.pl --input host-principal-keytab-list.csv --delete

You may also use this script to remove some default service principal(s), such as http, nfs, from computer objects created by Centrify DirectControl agents using the --remove-spn command. To do so:

./kerberos_security_setup.pl --input host-principal-keytab-list.csv --remove-spn

For a complete list of available options, please run the sample script with --help option.

For more detailed instructions to use this sample script on supported Hadoop distributions, please refer to doc/INSTALL.

Please also refer to doc/FAQ for usage detail.

Supported distributions

The sample files have been tested with the following Hadoop distributions:

Cloudera CDH-5.2.1 on CentOS 6 x64 with Cloudera Manager 5.2.1.
Hortonworks HDP-2.1 with Apache Ambari 1.6.1, or HDP-2.3.2 with Apache Ambari 2.1.2.1 on CentOS/RHEL 6 x64.
MapR 4.0 on CentOS 6 x64.

Licensing

Notice

The sample files might not address all production environments. Please always check before proceed. You are advised to test the solution in a lab environment as thoroughly as possible before deploying to a production environment.

High Availability (HA) mode is not supported.