Running Beeline

While the user may use any client program to connect to HiveServer2, the MR3 release provides a script for creating Beeline connections based on the configuration files in the installation.

In order to run Beeline, set the following environment variables in env.sh as necessary:

USER_PRINCIPAL=gitlab-runner@RED
USER_KEYTAB=/home/gitlab-runner/gitlab-runner.keytab

HIVE_CLIENT_HEAPSIZE=2048
  • USER_PRINCIPAL and USER_KEYTAB specify the principal and keytab file for the user executing Beeline in a secure cluster with Kerberos.
  • HIVE_CLIENT_HEAPSIZE specifies the heap size (in megabytes) for Beeline.

In order to start a Beeline connection, execute hive/run-beeline.sh with the following options:

--local                   # Run jobs with configurations in conf/local/ (default).
--cluster                 # Run jobs with configurations in conf/cluster/.
--mysql                   # Run jobs with configurations in conf/mysql/.
--tpcds                   # Run jobs with configurations in conf/tpcds/.
--hivesrc1                # Choose hive1-mr3 (based on Hive 1.2.2).
--hivesrc2                # Choose hive2-mr3 (based on Hive 2.3.4).
--hivesrc5                # Choose hive5-mr3 (based on Hive 3.1.1) (default).
--hiveconf <key>=<value>  # Add a configuration key/value; may be repeated at the end.
<Beeline option>          # Add a Beeline option; may be repeated at the end.

The user can append as many Beeline options (for the command beeline from Hive) as necessary to the command.

In a secure cluster with Kerberos, Beeline uses the Kerberos ticket provided by the user in order to authenticate itself to HiveServer2. Hence the Kerberos ticket should be valid at the time of executing the script. In a non-secure cluster without Kerberos, the script reads the environment variable USER for both the user name and the password. In order to override them, the user can supply Beeline options, as in hive/run-beeline.sh -n username_foo -p password_bar.

In a secure cluster with Kerberos, Beeline may fail with org.ietf.jgss.GSSException even if a valid Kerberos ticket is available:

javax.security.sasl.SaslException: GSS initiate failed
...
Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
  at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) ~[?:1.8.0_112]
  at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122) ~[?:1.8.0_112]
  at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) ~[?:1.8.0_112]
  at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224) ~[?:1.8.0_112]
  at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) ~[?:1.8.0_112]
  at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) ~[?:1.8.0_112]
  at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ~[?:1.8.0_112]

In such a case, setting the Java property javax.security.auth.useSubjectCredsOnly to false may work. For example, the user can execute the following line before running Beeline:

export HADOOP_OPTS="$HADOOP_OPTS -Djavax.security.auth.useSubjectCredsOnly=false"

In order to show progress bars in Beeline output, update hive-site.xml as follows:

  • hive.server2.logging.operation.enabled should be set to true.
  • hive.server2.logging.operation.log.location should be set to a path for which the user has write permission.
  • In Hive 2.3.4 on MR3, hive.async.log.enabled should be set to false.
  • In Hive 3.1.1 on MR3, hive.async.log.enabled should be set to true.

beeline.progress

Hive 1.2.2 on MR3 does not support progress bars in Beeline.