HBase Kerberos SaslException: GSS initiate failed (Mechanism level: Failed to find any Kerberos tgt) Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern) Come Celebrate our 10 Year Anniversary!Apache2, Kerberos: gss_accept_sec_context() failed: An unsupported mechanism was requestedKerberos: connection from win app running from IIS to SQL failedTrack down source of event 4771: Kerberos pre-authentication failedLogin failed for user 'NT AUTHORITYANONYMOUS LOGON' and kerberos delegationKerberos Apache keeps asking for BASICAnsible - Kerberos. Can´t find client principaluser account successfully authenticated after multiple Kerberos pre-authentication failed/ account lock out event
AppleTVs create a chatty alternate WiFi network
A term for a woman complaining about things/begging in a cute/childish way
Karn the great creator - 'card from outside the game' in sealed
What order were files/directories output in dir?
Getting prompted for verification code but where do I put it in?
How could we fake a moon landing now?
How does Belgium enforce obligatory attendance in elections?
1-probability to calculate two events in a row
Random body shuffle every night—can we still function?
In musical terms, what properties are varied by the human voice to produce different words / syllables?
Strange behavior of Object.defineProperty() in JavaScript
What is best way to wire a ceiling receptacle in this situation?
Lagrange four-squares theorem --- deterministic complexity
Girl Hackers - Logic Puzzle
How do I find out the mythology and history of my Fortress?
The Nth Gryphon Number
What does Turing mean by this statement?
Is there any word for a place full of confusion?
Tannaka duality for semisimple groups
Did any compiler fully use 80-bit floating point?
What does it mean that physics no longer uses mechanical models to describe phenomena?
Did Mueller's report provide an evidentiary basis for the claim of Russian govt election interference via social media?
Putting class ranking in CV, but against dept guidelines
How do living politicians protect their readily obtainable signatures from misuse?
HBase Kerberos SaslException: GSS initiate failed (Mechanism level: Failed to find any Kerberos tgt)
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern)
Come Celebrate our 10 Year Anniversary!Apache2, Kerberos: gss_accept_sec_context() failed: An unsupported mechanism was requestedKerberos: connection from win app running from IIS to SQL failedTrack down source of event 4771: Kerberos pre-authentication failedLogin failed for user 'NT AUTHORITYANONYMOUS LOGON' and kerberos delegationKerberos Apache keeps asking for BASICAnsible - Kerberos. Can´t find client principaluser account successfully authenticated after multiple Kerberos pre-authentication failed/ account lock out event
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
1
I am trying to set up Kerberos authentication for HBase using this http://hbase.apache.org/0.94/book/security.html documentation and have very little progress so far.
HBase 1.1.1 from Apache without any Cloudera influences. Host machine is running under Centos 6.5.
I've already set up Kerberos KDC and client after following instruction https://gist.github.com/ashrithr/4767927948eca70845db
KDC is located on the same machine as HBase I'm trying to secure.
All-in-all, here's current environment state:
keytab file is here /opt/hbase.keytab
hbase-site.xml contents
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>hbase.rootdir</name>
<value>file:///opt/hbase-data/hbase</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/opt/hbase-data/zookeeper</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.security.authentication</name>
<value>kerberos</value>
</property>
<property>
<name>hbase.security.authorization</name>
<value>true</value>
</property>
<property>
<name>hbase.coprocessor.region.classes</name>
<value>org.apache.hadoop.hbase.security.token.TokenProvider</value>
</property>
<property>
<name>hbase.master.keytab.file</name>
<value>/opt/hbase.keytab</value>
</property>
<property>
<name>hbase.master.kerberos.principal</name>
<value>hbase/_HOST@XXX.MYCOMPANY.COM</value>
</property>
<property>
<name>hbase.regionserver.kerberos.principal</name>
<value>hbase/_HOST@XXX.MYCOMPANY.COM</value>
</property>
<property>
<name>hbase.regionserver.keytab.file</name>
<value>/opt/hbase.keytab</value>
</property>
</configuration>
It's a pseudo-distributed mode and I didn't bother with undelying HDFS to keep things as simple as possible.
However when I start hbase with ./start-hbase command I get following error in regionserver.log
2015-10-20 17:33:18,068 INFO [regionserver/xxx.mycompany.com/172.24.4.60:16201] regionserver.HRegionServer: reportForDuty to master=xxx.mycompany.com,16000,1445349909162 with port=16201, startcode=1445349910087 2015-10-20 17:33:18,071 WARN [regionserver/xxx.mycompany.com/172.24.4.60:16201] ipc.AbstractRpcClient: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 2015-10-20 17:33:18,071 FATAL [regionserver/xxx.mycompany.com/172.24.4.60:16201] ipc.AbstractRpcClient: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'. javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:609)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:154)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:735)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:732)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:732)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:885)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:854)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1180)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:8982)
at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2260)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:893)
at java.lang.Thread.run(Thread.java:745) Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
... 18 more 2015-10-20 17:33:18,072 WARN [regionserver/xxx.mycompany.com/172.24.4.60:16201] regionserver.HRegionServer: error telling master we are up com.google.protobuf.ServiceException: java.io.IOException: Could not set up IO Streams to xxx.mycompany.com/172.24.4.60:16000
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:223)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:8982)
at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2260)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:893)
at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Could not set up IO Streams to xxx.mycompany.com/172.24.4.60:16000
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:777)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:885)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:854)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1180)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
... 5 more Caused by: java.lang.RuntimeException: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'.
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$1.run(RpcClientImpl.java:677)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.handleSaslConnectionFailure(RpcClientImpl.java:635)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:743)
... 9 more Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:609)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:154)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:735)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:732)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:732)
... 9 more Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
... 18 more 2015-10-20 17:33:18,073 WARN [regionserver/xxx.mycompany.com/172.24.4.60:16201] regionserver.HRegionServer: reportForDuty failed; sleeping and then retrying.
I presume Kerberos works because I can obtain
$ klist -ekt hbase.keytab
Keytab name: FILE:hbase.keytab
KVNO Timestamp Principal
---- ----------------- --------------------------------------------------------
3 10/19/15 17:11:42 hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM (arcfour-hmac)
3 10/19/15 17:11:42 hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM (des3-cbc-sha1)
3 10/19/15 17:11:42 hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM (des-cbc-crc)
$ kinit -kt /opt/hbase.keytab hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM
[userx1@gms-01 logs]$ klist
Ticket cache: FILE:/tmp/krb5cc_2369
Default principal: hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM
Valid starting Expires Service principal
10/20/15 17:49:32 10/21/15 03:49:32 krbtgt/XXX.MYCOMPANY.COM@XXX.MYCOMPANY.COM
renew until 10/27/15 16:49:32
hbase shell produces the same exception as provided above when trying to run status (or whatever) command
If anyone has any suggestions or advices please let me know
Thanks in advance
kerberos centos6.5 hbase
asked Oct 20 '15 at 15:06
user1455836user1455836
1064
add a comment |
1
I am trying to set up Kerberos authentication for HBase using this http://hbase.apache.org/0.94/book/security.html documentation and have very little progress so far.
HBase 1.1.1 from Apache without any Cloudera influences. Host machine is running under Centos 6.5.
I've already set up Kerberos KDC and client after following instruction https://gist.github.com/ashrithr/4767927948eca70845db
KDC is located on the same machine as HBase I'm trying to secure.
All-in-all, here's current environment state:
keytab file is here /opt/hbase.keytab
hbase-site.xml contents
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>hbase.rootdir</name>
<value>file:///opt/hbase-data/hbase</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/opt/hbase-data/zookeeper</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.security.authentication</name>
<value>kerberos</value>
</property>
<property>
<name>hbase.security.authorization</name>
<value>true</value>
</property>
<property>
<name>hbase.coprocessor.region.classes</name>
<value>org.apache.hadoop.hbase.security.token.TokenProvider</value>
</property>
<property>
<name>hbase.master.keytab.file</name>
<value>/opt/hbase.keytab</value>
</property>
<property>
<name>hbase.master.kerberos.principal</name>
<value>hbase/_HOST@XXX.MYCOMPANY.COM</value>
</property>
<property>
<name>hbase.regionserver.kerberos.principal</name>
<value>hbase/_HOST@XXX.MYCOMPANY.COM</value>
</property>
<property>
<name>hbase.regionserver.keytab.file</name>
<value>/opt/hbase.keytab</value>
</property>
</configuration>
It's a pseudo-distributed mode and I didn't bother with undelying HDFS to keep things as simple as possible.
However when I start hbase with ./start-hbase command I get following error in regionserver.log
2015-10-20 17:33:18,068 INFO [regionserver/xxx.mycompany.com/172.24.4.60:16201] regionserver.HRegionServer: reportForDuty to master=xxx.mycompany.com,16000,1445349909162 with port=16201, startcode=1445349910087 2015-10-20 17:33:18,071 WARN [regionserver/xxx.mycompany.com/172.24.4.60:16201] ipc.AbstractRpcClient: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 2015-10-20 17:33:18,071 FATAL [regionserver/xxx.mycompany.com/172.24.4.60:16201] ipc.AbstractRpcClient: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'. javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:609)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:154)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:735)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:732)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:732)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:885)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:854)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1180)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:8982)
at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2260)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:893)
at java.lang.Thread.run(Thread.java:745) Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
... 18 more 2015-10-20 17:33:18,072 WARN [regionserver/xxx.mycompany.com/172.24.4.60:16201] regionserver.HRegionServer: error telling master we are up com.google.protobuf.ServiceException: java.io.IOException: Could not set up IO Streams to xxx.mycompany.com/172.24.4.60:16000
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:223)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:8982)
at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2260)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:893)
at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Could not set up IO Streams to xxx.mycompany.com/172.24.4.60:16000
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:777)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:885)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:854)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1180)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
... 5 more Caused by: java.lang.RuntimeException: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'.
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$1.run(RpcClientImpl.java:677)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.handleSaslConnectionFailure(RpcClientImpl.java:635)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:743)
... 9 more Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:609)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:154)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:735)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:732)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:732)
... 9 more Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
... 18 more 2015-10-20 17:33:18,073 WARN [regionserver/xxx.mycompany.com/172.24.4.60:16201] regionserver.HRegionServer: reportForDuty failed; sleeping and then retrying.
I presume Kerberos works because I can obtain
$ klist -ekt hbase.keytab
Keytab name: FILE:hbase.keytab
KVNO Timestamp Principal
---- ----------------- --------------------------------------------------------
3 10/19/15 17:11:42 hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM (arcfour-hmac)
3 10/19/15 17:11:42 hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM (des3-cbc-sha1)
3 10/19/15 17:11:42 hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM (des-cbc-crc)
$ kinit -kt /opt/hbase.keytab hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM
[userx1@gms-01 logs]$ klist
Ticket cache: FILE:/tmp/krb5cc_2369
Default principal: hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM
Valid starting Expires Service principal
10/20/15 17:49:32 10/21/15 03:49:32 krbtgt/XXX.MYCOMPANY.COM@XXX.MYCOMPANY.COM
renew until 10/27/15 16:49:32
hbase shell produces the same exception as provided above when trying to run status (or whatever) command
If anyone has any suggestions or advices please let me know
Thanks in advance
kerberos centos6.5 hbase
asked Oct 20 '15 at 15:06
user1455836user1455836
1064
add a comment |
1
1
1
I am trying to set up Kerberos authentication for HBase using this http://hbase.apache.org/0.94/book/security.html documentation and have very little progress so far.
HBase 1.1.1 from Apache without any Cloudera influences. Host machine is running under Centos 6.5.
I've already set up Kerberos KDC and client after following instruction https://gist.github.com/ashrithr/4767927948eca70845db
KDC is located on the same machine as HBase I'm trying to secure.
All-in-all, here's current environment state:
keytab file is here /opt/hbase.keytab
hbase-site.xml contents
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>hbase.rootdir</name>
<value>file:///opt/hbase-data/hbase</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/opt/hbase-data/zookeeper</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.security.authentication</name>
<value>kerberos</value>
</property>
<property>
<name>hbase.security.authorization</name>
<value>true</value>
</property>
<property>
<name>hbase.coprocessor.region.classes</name>
<value>org.apache.hadoop.hbase.security.token.TokenProvider</value>
</property>
<property>
<name>hbase.master.keytab.file</name>
<value>/opt/hbase.keytab</value>
</property>
<property>
<name>hbase.master.kerberos.principal</name>
<value>hbase/_HOST@XXX.MYCOMPANY.COM</value>
</property>
<property>
<name>hbase.regionserver.kerberos.principal</name>
<value>hbase/_HOST@XXX.MYCOMPANY.COM</value>
</property>
<property>
<name>hbase.regionserver.keytab.file</name>
<value>/opt/hbase.keytab</value>
</property>
</configuration>
It's a pseudo-distributed mode and I didn't bother with undelying HDFS to keep things as simple as possible.
However when I start hbase with ./start-hbase command I get following error in regionserver.log
2015-10-20 17:33:18,068 INFO [regionserver/xxx.mycompany.com/172.24.4.60:16201] regionserver.HRegionServer: reportForDuty to master=xxx.mycompany.com,16000,1445349909162 with port=16201, startcode=1445349910087 2015-10-20 17:33:18,071 WARN [regionserver/xxx.mycompany.com/172.24.4.60:16201] ipc.AbstractRpcClient: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 2015-10-20 17:33:18,071 FATAL [regionserver/xxx.mycompany.com/172.24.4.60:16201] ipc.AbstractRpcClient: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'. javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:609)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:154)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:735)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:732)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:732)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:885)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:854)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1180)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:8982)
at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2260)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:893)
at java.lang.Thread.run(Thread.java:745) Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
... 18 more 2015-10-20 17:33:18,072 WARN [regionserver/xxx.mycompany.com/172.24.4.60:16201] regionserver.HRegionServer: error telling master we are up com.google.protobuf.ServiceException: java.io.IOException: Could not set up IO Streams to xxx.mycompany.com/172.24.4.60:16000
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:223)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:8982)
at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2260)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:893)
at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Could not set up IO Streams to xxx.mycompany.com/172.24.4.60:16000
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:777)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:885)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:854)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1180)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
... 5 more Caused by: java.lang.RuntimeException: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'.
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$1.run(RpcClientImpl.java:677)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.handleSaslConnectionFailure(RpcClientImpl.java:635)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:743)
... 9 more Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:609)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:154)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:735)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:732)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:732)
... 9 more Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
... 18 more 2015-10-20 17:33:18,073 WARN [regionserver/xxx.mycompany.com/172.24.4.60:16201] regionserver.HRegionServer: reportForDuty failed; sleeping and then retrying.
I presume Kerberos works because I can obtain
$ klist -ekt hbase.keytab
Keytab name: FILE:hbase.keytab
KVNO Timestamp Principal
---- ----------------- --------------------------------------------------------
3 10/19/15 17:11:42 hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM (arcfour-hmac)
3 10/19/15 17:11:42 hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM (des3-cbc-sha1)
3 10/19/15 17:11:42 hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM (des-cbc-crc)
$ kinit -kt /opt/hbase.keytab hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM
[userx1@gms-01 logs]$ klist
Ticket cache: FILE:/tmp/krb5cc_2369
Default principal: hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM
Valid starting Expires Service principal
10/20/15 17:49:32 10/21/15 03:49:32 krbtgt/XXX.MYCOMPANY.COM@XXX.MYCOMPANY.COM
renew until 10/27/15 16:49:32
hbase shell produces the same exception as provided above when trying to run status (or whatever) command
If anyone has any suggestions or advices please let me know
Thanks in advance
kerberos centos6.5 hbase
asked Oct 20 '15 at 15:06
user1455836user1455836
1064
I am trying to set up Kerberos authentication for HBase using this http://hbase.apache.org/0.94/book/security.html documentation and have very little progress so far.
HBase 1.1.1 from Apache without any Cloudera influences. Host machine is running under Centos 6.5.
I've already set up Kerberos KDC and client after following instruction https://gist.github.com/ashrithr/4767927948eca70845db
KDC is located on the same machine as HBase I'm trying to secure.
All-in-all, here's current environment state:
keytab file is here /opt/hbase.keytab
hbase-site.xml contents
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>hbase.rootdir</name>
<value>file:///opt/hbase-data/hbase</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/opt/hbase-data/zookeeper</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.security.authentication</name>
<value>kerberos</value>
</property>
<property>
<name>hbase.security.authorization</name>
<value>true</value>
</property>
<property>
<name>hbase.coprocessor.region.classes</name>
<value>org.apache.hadoop.hbase.security.token.TokenProvider</value>
</property>
<property>
<name>hbase.master.keytab.file</name>
<value>/opt/hbase.keytab</value>
</property>
<property>
<name>hbase.master.kerberos.principal</name>
<value>hbase/_HOST@XXX.MYCOMPANY.COM</value>
</property>
<property>
<name>hbase.regionserver.kerberos.principal</name>
<value>hbase/_HOST@XXX.MYCOMPANY.COM</value>
</property>
<property>
<name>hbase.regionserver.keytab.file</name>
<value>/opt/hbase.keytab</value>
</property>
</configuration>
It's a pseudo-distributed mode and I didn't bother with undelying HDFS to keep things as simple as possible.
However when I start hbase with ./start-hbase command I get following error in regionserver.log
2015-10-20 17:33:18,068 INFO [regionserver/xxx.mycompany.com/172.24.4.60:16201] regionserver.HRegionServer: reportForDuty to master=xxx.mycompany.com,16000,1445349909162 with port=16201, startcode=1445349910087 2015-10-20 17:33:18,071 WARN [regionserver/xxx.mycompany.com/172.24.4.60:16201] ipc.AbstractRpcClient: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 2015-10-20 17:33:18,071 FATAL [regionserver/xxx.mycompany.com/172.24.4.60:16201] ipc.AbstractRpcClient: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'. javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:609)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:154)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:735)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:732)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:732)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:885)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:854)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1180)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:8982)
at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2260)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:893)
at java.lang.Thread.run(Thread.java:745) Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
... 18 more 2015-10-20 17:33:18,072 WARN [regionserver/xxx.mycompany.com/172.24.4.60:16201] regionserver.HRegionServer: error telling master we are up com.google.protobuf.ServiceException: java.io.IOException: Could not set up IO Streams to xxx.mycompany.com/172.24.4.60:16000
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:223)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:8982)
at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2260)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:893)
at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Could not set up IO Streams to xxx.mycompany.com/172.24.4.60:16000
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:777)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:885)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:854)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1180)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
... 5 more Caused by: java.lang.RuntimeException: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'.
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$1.run(RpcClientImpl.java:677)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.handleSaslConnectionFailure(RpcClientImpl.java:635)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:743)
... 9 more Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:609)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:154)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:735)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:732)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:732)
... 9 more Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
... 18 more 2015-10-20 17:33:18,073 WARN [regionserver/xxx.mycompany.com/172.24.4.60:16201] regionserver.HRegionServer: reportForDuty failed; sleeping and then retrying.
I presume Kerberos works because I can obtain
$ klist -ekt hbase.keytab
Keytab name: FILE:hbase.keytab
KVNO Timestamp Principal
---- ----------------- --------------------------------------------------------
3 10/19/15 17:11:42 hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM (arcfour-hmac)
3 10/19/15 17:11:42 hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM (des3-cbc-sha1)
3 10/19/15 17:11:42 hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM (des-cbc-crc)
$ kinit -kt /opt/hbase.keytab hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM
[userx1@gms-01 logs]$ klist
Ticket cache: FILE:/tmp/krb5cc_2369
Default principal: hbase/xxx.mycompany.com@XXX.MYCOMPANY.COM
Valid starting Expires Service principal
10/20/15 17:49:32 10/21/15 03:49:32 krbtgt/XXX.MYCOMPANY.COM@XXX.MYCOMPANY.COM
renew until 10/27/15 16:49:32
hbase shell produces the same exception as provided above when trying to run status (or whatever) command
If anyone has any suggestions or advices please let me know
Thanks in advance
kerberos centos6.5 hbase
kerberos centos6.5 hbase
asked Oct 20 '15 at 15:06
user1455836user1455836
1064
asked Oct 20 '15 at 15:06
user1455836user1455836
1064
asked Oct 20 '15 at 15:06
user1455836user1455836
1064
asked Oct 20 '15 at 15:06
user1455836user1455836
1064
asked Oct 20 '15 at 15:06
user1455836user1455836
1064
1064
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
0
I faced the same issue and got it resolved by starting region server separately. Make sure when you are starting daemons, you have valid Kerberos TGT.
answered Apr 1 '16 at 12:00
AnandAnand
1
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "2"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f730263%2fhbase-kerberos-saslexception-gss-initiate-failed-mechanism-level-failed-to-fi%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
0
I faced the same issue and got it resolved by starting region server separately. Make sure when you are starting daemons, you have valid Kerberos TGT.
answered Apr 1 '16 at 12:00
AnandAnand
1
add a comment |
0
I faced the same issue and got it resolved by starting region server separately. Make sure when you are starting daemons, you have valid Kerberos TGT.
answered Apr 1 '16 at 12:00
AnandAnand
1
add a comment |
0
0
0
I faced the same issue and got it resolved by starting region server separately. Make sure when you are starting daemons, you have valid Kerberos TGT.
answered Apr 1 '16 at 12:00
AnandAnand
1
I faced the same issue and got it resolved by starting region server separately. Make sure when you are starting daemons, you have valid Kerberos TGT.
answered Apr 1 '16 at 12:00
AnandAnand
1
answered Apr 1 '16 at 12:00
AnandAnand
1
answered Apr 1 '16 at 12:00
AnandAnand
1
answered Apr 1 '16 at 12:00
AnandAnand
1
1
add a comment |
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Server Fault!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f730263%2fhbase-kerberos-saslexception-gss-initiate-failed-mechanism-level-failed-to-fi%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
