CRITICAL : Check kerberos authenticated DNS updates

tbeluch · August 19, 2018, 1:59pm

Hello,

Following a recent update to a UCS AD Master from 4.2.X to 4.2.4 errata 490, i have seen a new Critical error appearing on the diagnostic center. This error is :

CRITICAL : Check kerberos authenticated DNS updates
Errors occured while running kinit or nsupdate
kinit for principal dns-ucsdc with keytab /var/lib/samba/private/dns.keytab failed.

I have tried the solution from the post “Critical: Check kerberos authenticated DNS update (on DC Master)”, with no success. The actions seem to work correctly and change the keytab file, but this doesn’t solve the issue.

In the same time, applications that used LDAP to check identities on this server claim that this server is no longer joinable.

Any help would be welcome on how to identify/fix the issue.

Best regards,

Thomas

Moritz_Bunkus · August 20, 2018, 7:07am

Hey,

There are two ways to repair such a problem, depending on what the actual problem is. First a couple of questions as you’re somewhat vague in your description:

Do you have more than one UCS server?
If so, what’s the name of your DC Master and which server is this problem occurring on?

You said you already tried what I wrote in this post, correct? Please really link to the topic (even better: to the post in the topic) instead of just referring to it by its subject as subjects can be ambiguous.

I assume that you only have one UCS server which is both the Kerberos KDC as well as the one your problem occurs on. If not, we’ll have to adjust the following commands.

First let’s make sure that the key the KDC has stored really doesn’t work. Please try the following (and be exact with the commands — ideally just copy & paste them; if not, be extra careful not to confuse @ with . etc.):

samba-tool domain exportkeytab test-keytab --principal=dns-$(ucr get hostname)@$(ucr get kerberos/realm)
samba-tool domain exportkeytab test-keytab --principal=DNS/$(ucr get hostname).$(ucr get kerberos/realm)
kinit -t test-keytab dns-$(ucr get hostname)@$(ucr get kerberos/realm)
kinit -t test-keytab DNS/$(ucr get hostname).$(ucr get kerberos/realm)

The first two commands should each state: Export one principal to test-keytab The latter two commands shouldn’t output anything.

If all of those succeed, we can continue to test an actual DNS update with those keys:

kinit -t test-keytab dns-$(ucr get hostname)@$(ucr get kerberos/realm)
eval "$(ucr shell)"
nsupdate -g <<%EOF
server $ldap_master
prereq yxdomain $hostname.$domainname
send
%EOF

Again, if everything works fine, the nsupdate command shouldn’t output anything. If this is the case, then please follow the steps shown in the aforementioned post precisely as they will solve your issue.

However, if those steps don’t work either, there’s one more thing you can do: re-create the whole Kerberos service account for DNS. The following steps are required:

udm users/user delete --dn=uid=dns-$(hostname),cn=users,$(ucr get ldap/base)
ldbdel -H /var/lib/samba/private/secrets.ldb "samAccountName=dns-$(hostname),CN=Principles"
# Now wait until your S4 connector has finished syncing the deleted objects. Watch /var/log/univention/connector-s4.log for corresponding messages.
/usr/share/univention-samba4/scripts/create_spn_account.sh --samaccountname "dns-$(hostname)" --serviceprincipalname "DNS/$(hostname).$(ucr get domainname)" --privatekeytab dns.keytab
# Again, wait until the S4 connector has finished syncing.
/usr/share/univention-s4-connector/resync_object_from_s4.py "cn=dns-$(hostname),cn=users,$(ucr get samba4/ldap/base)"
# Wait for the S4 connector a third time.
keytab=/var/lib/samba/private/dns.keytab
mv $keytab $keytab.$(date '+%Y%m%d%H%M%S')
samba-tool domain exportkeytab $keytab "--principal=DNS/$(hostname).$(ucr get domainname)"
samba-tool domain exportkeytab $keytab "--principal=dns-$(hostname)@$(ucr get kerberos/realm)"
chmod 0640 $keytab

Kind regards,
mosu

tbeluch · August 20, 2018, 2:02pm

Edit: Sorry I did not take time to read until the end of your message. The solution is in the end. This problem is solved now. Thx

Hello Moritz,

Thanks for your help, and sorry i did’t reopen the other post.
The two first command correctly export the keytab files, but the two next commands (kinit…) fail with a password expired message.

root@ucsdc:~# samba-tool domain exportkeytab test-keytab --principal dns-$(ucr get hostname)@$(ucr get kerberos/realm)
Export one principal to test-keytab
root@ucsdc:~# samba-tool domain exportkeytab test-keytab --principal DNS/$(ucr get hostname).$(ucr get kerberos/realm)
Export one principal to test-keytab
root@ucsdc:~# kinit -t test-keytab dns-$(ucr get hostname)@$(ucr get kerberos/realm)
kinit: krb5_get_init_creds: Password has expired
root@ucsdc:~# kinit -t test-keytab DNS/$(ucr get hostname).$(ucr get kerberos/realm)
kinit: krb5_get_init_creds: Password has expired
root@ucsdc:~#

I am quite new to kerberos, so any help on this subject would be nice.

Best regards,

Thomas

Moritz_Bunkus · August 21, 2018, 8:25am

Hey,

looks like you have password expiration configured for your users. That’s generally not a bad idea, but the passwords of certain system users must never expire. Otherwise services won’t work correctly anymore.

DNS updates is one of those services. It uses the user object called dns-<hostname>, in your case probably dns-ucsdc. As a workaround edit that user and disable its password expiration.

Then you should reconfigure your expiration policies so that they don’t apply to the aforementioned system users. Often the problem is that both regular users (human beings) as well as system users reside in the same container, cn=users,$ldap_base. The solution is often to create a new container for the regular users, move them there and apply the password expiration policies to that container only.

Kind regards
mosu

Moritz_Bunkus · August 21, 2018, 9:05am

You mean you re-created the dns-<hostname> user account? Sure, that works, but only until your password expiration policy strikes the next time. You should really look into applying that policy to your regular users only as I’ve described in today’s post.

tomshere · October 31, 2020, 1:09pm

SORRY to warm up this one again. I have the same error but can’t repair it with these commands due to another error when trying to apply the corrections:

Moritz_Bunkus:

samba-tool domain exportkeytab test-keytab --principal=dns-$(ucr get hostname)@$(ucr get kerberos/realm)
samba-tool domain exportkeytab test-keytab --principal=DNS/$(ucr get hostname).$(ucr get kerberos/realm)
kinit -t test-keytab dns-$(ucr get hostname)@$(ucr get kerberos/realm)
kinit -t test-keytab DNS/$(ucr get hostname).$(ucr get kerberos/realm)

The last command throws
kinit: krb5_get_init_creds: Client (DNS/ucs-001.FRITZ.BOX@FRITZ.BOX) unknown

So I tried to execute the lines starting with
udm users/user delete --dn=uid=dns-$(hostname),cn=users,$(ucr get ldap/base)

After this first command in the S4-log I can see the following:

31.10.2020 13:55:27.322 LDAP        (PROCESS): sync from ucs: [          user] [    delete] cn=dns-ucs-001,cn=users,DC=fritz,DC=box
31.10.2020 13:55:27.341 LDAP        (PROCESS): sync from ucs: [         group] [    modify] cn=domain users,cn=groups,DC=fritz,DC=box
31.10.2020 13:55:28.365 LDAP        (PROCESS): object_from_element: DN of removed object: u'cn=dns-ucs-001,CN=Users,DC=fritz,DC=box'
31.10.2020 13:55:28.370 LDAP        (PROCESS): sync to ucs:   [          user] [    delete] u'uid=dns-ucs-001,cn=users,dc=fritz,dc=box'
31.10.2020 13:55:28.371 LDAP        (WARNING): Object to delete doesn't exists, ignore (u'uid=dns-ucs-001,cn=users,dc=fritz,dc=box')

then:

root@ucs-001:~# ldbdel -H /var/lib/samba/private/secrets.ldb "samAccountName=dns-$(hostname),CN=Principles"
delete of 'samAccountName=dns-ucs-001,CN=Principles' failed - (No such object) ldb_wait from (null) with LDB_WAIT_ALL: No such object (32)

and nothing in the S4-log.

then:

root@ucs-001:~# /usr/share/univention-samba4/scripts/create_spn_account.sh --samaccountname "dns-$(hostname)" --serviceprincipalname "DNS/$(hostname).$(ucr get domainname)" --privatekeytab dns.keytab
Object created: uid=dns-ucs-001,cn=users,dc=fritz,dc=box
looking for spn account "dns-ucs-001" in local samba
looking for spn account "dns-ucs-001" in local samba
Modified 1 records successfully
Modified 1 records successfully
Expiry for user 'dns-ucs-001' disabled.

S4-log:

31.10.2020 13:58:19.947 LDAP        (PROCESS): sync from ucs: [          user] [       add] cn=dns-ucs-001,cn=users,DC=fritz,DC=box
31.10.2020 13:58:20.000 LDAP        (PROCESS): sync from ucs: [         group] [    modify] cn=domain users,cn=groups,DC=fritz,DC=box
31.10.2020 13:58:21.038 LDAP        (PROCESS): sync to ucs:   [          user] [    modify] u'uid=dns-ucs-001,cn=users,dc=fritz,dc=box'
31.10.2020 13:58:27.169 LDAP        (PROCESS): sync from ucs: [          user] [    modify] cn=dns-ucs-001,cn=users,DC=fritz,DC=box
31.10.2020 13:58:28.186 LDAP        (PROCESS): sync to ucs:   [          user] [    modify] u'uid=dns-ucs-001,cn=users,dc=fritz,dc=box'
31.10.2020 13:58:34.231 LDAP        (PROCESS): sync to ucs:   [          user] [    modify] u'uid=dns-ucs-001,cn=users,dc=fritz,dc=box'

then:

root@ucs-001:~# /usr/share/univention-s4-connector/resync_object_from_s4.py "cn=dns-$(hostname),cn=users,$(ucr get samba4/ldap/base)"
resync triggered for cn=dns-ucs-001,cn=users,DC=FRITZ,DC=BOX
Estimated sync in 50 seconds.

S4-log:

31.10.2020 14:02:10.996 LDAP        (PROCESS): sync to ucs: Resync rejected dn: cn=dns-ucs-001,cn=users,DC=FRITZ,DC=BOX
31.10.2020 14:02:11.012 LDAP        (PROCESS): sync to ucs:   [          user] [    modify] u'uid=dns-ucs-001,cn=users,dc=fritz,dc=box'
31.10.2020 14:02:16.110 LDAP        (PROCESS): sync from ucs: [          user] [    modify] cn=dns-ucs-001,cn=users,DC=fritz,DC=box
31.10.2020 14:02:17.156 LDAP        (PROCESS): sync to ucs:   [          user] [    modify] u'uid=dns-ucs-001,cn=users,dc=fritz,dc=box'

The resync-error is probably not what’s expected and at the end the kinit error still exists after applying all the rest.

Any advice to solve the ldap problem?