...

In this Page

Setting Up Kerberos on Groundplex Nodes

To set up Kerberos on Groundplex nodes:

1. Before doing the following setups, make sure the Groundplex is working fine.

2. Install Kerberos packages on the Groundplex nodes.

   Code Block
   $ sudo yum install krb5-workstation krb5-libs krb5-auth-dialog

   
   

3. Copy the file /etc/krb5.conf from one of the target cluster nodes to /etc/krb5.conf on each Groundplex node. 

4. Install the JCE extension on each Groundplex node.
   1. Download the JCE extension zip file: http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html

   2. Copy the JCE extension zip file onto each Groundplex node and install the JCE extension with the following command. Restart the node after the installation.

      Code Block
      $ unzip -o -j -q jce_policy-8.zip -d /opt/snaplogic/pkgs/jre1.8.0_45/lib/security/

      
      

   3. To check if the JCE extension was correctly installed, run the command below and you should see the same output shown as below:

      Code Block
      $ zipgrep CryptoAllPermission /opt/snaplogic/pkgs/jre1.8.0_45/lib/security/local_policy.jar default_local.policy: permission javax.crypto.CryptoAllPermission;

      
      

5. Generate the keytab file for the Kerberos user, put it on each Groundplex node, and give snapuser access to the keytab file.

   Code Block
   $ sudo cp /path/to/keytab/file /home/snapuser/<keytab_file_name> $ sudo chown snapuser:snapuser /home/snapuser/<keytab_file_name> $ sudo chmod 400 /home/snapuser/<keytab_file_name>

   
   

6. To provide additional Hadoop Configuration details to the JCC, it should be passed as a JCC Configuration option. For more details on configuration, please check the Configuration page. The global . properties file should  should be updated to add the following configuration option. Value that jcc.jvm_options  points to the HDFS configuration directory.

   Code Block
   jcc.jvm_options=-DHADOOP_CLIENT_CONF_DIR=<PATH_TO_HDFS_CONF_DIRECTORY>

   for example:

   Code Block
   jcc.jvm_options=-DHADOOP_CLIENT_CONF_DIR=/home/snapuser/remote-hadoop/conf

   
   

7. To provide additional Hadoop Configuration details to the JCC from the UI, edit the Snapplex properties in Manager.

   Go to the Snapplex Snaplex Node Properties. Under Global Properties and add a key named "jvm_options" with the value:  

   Code Block
   -DHADOOP_CLIENT_CONF_DIR=<PATH_TO_HDFS_CONF_DIRECTORY>

   
   for example:

   Code Block
   -DHADOOP_CLIENT_CONF_DIR=/home/snapuser/remote-hadoop/conf

   

Setting Up Edge Node with Kerberos Configurations on Groundplex Nodes

Edge nodes are the interface between the Hadoop cluster and the outside network. The edge node can be a part of Hadoop Cluster or can be outside of the Hadoop Cluster. SnapLogic's suggested configuration for an edge node is to be a part of HDFS Cluster and running the Groundplex on the edge node. 

Different Hadoop distributions follows different steps to configure a node as an edge node. Here are the links to setting up the edge node.

• Cloudera
• HortonWorks

...

Testing Kerberos Configurations on Groundplex Nodes

The following commands can be used to test Kerberos configurations on the Groundplex nodes:

...

$ kinit -k -t /home/snapuser/snaplogic.keytab snaplogic/node1.snaplogic.dev.com@SNAPLOGIC.DEV.COM
$ klist
Ticket cache: FILE:/tmp/krb5cc_5112
Default principal: snaplogic/node1.snaplogic.dev.com@SNAPLOGIC.DEV.COM

Valid starting       Expires              Service principal
03/27/2017 18:39:59  03/28/2017 18:39:59  krbtgt/node1.snaplogic.dev.com@SNAPLOGIC.DEV.COM
	renew until 04/03/2017 18:39:59

Common Issues

How to Create a New User for Hive with Kerberos

1. Create a Linux user on each CDH cluster node.

2. Create a home directory on the CDH cluster's HDFS for the new user.

3. Create a Kerberos principal for the new user.

Snap fails with error: [Cloudera][HiveJDBCDriver](500051) ERROR processing query/statement.

Error Code: ERROR_STATE, SQL state: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask, Query: ......

In the Snap's account setting:

1. If the authentication type is Kerberos, make sure the all the steps in How to Create a New User for Hive with Kerberos have been done correctly.

2. If the authentication type is User ID or User ID and Password, make sure the first two steps of How to Create a New User for Hive with Kerberos have been done correctly.

3. If the Authentication type is None, a home directory for an anonymous user should be created on HDFS:

Snap failed with error: Unable to authenticate the client to the KDC using the provided credentials.

This error indicates that the Kerberos is not configured properly on the Groundplex nodes. See section Configure Kerberos on Groundplex Nodes.

Snap failed with error: Keytab file does not exist or is not readable.

As the error message says, the keytab file is either missing or snapuser does not have the access to the keytab file. See step 5 in Step-by-step Guide.

Common errors with HDFS Cluster authenticated with Kerberos

<property>
  <name>dfs.namenode.kerberos.principal</name>
  <value>hdfs/_HOST@YOUR-REALM.COM</value>
</property>

<property>
  <name>dfs.datanode.kerberos.principal</name>
  <value>hdfs/_HOST@YOUR-REALM.COM</value>
</property>

...