Interner Server-Fehler in "udm/containers (users/user)"

UCS - Univention Corporate Server
#1

Hello,
anyone got an idea how to fix this behaviour on my Backup-Server?
I can not configure or at least view anything on my Backup server:
image

Interner Server-Fehler in "udm/containers (users/user)".
Request: udm/containers (users/user)

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 359, in __error_handling
    six.reraise(etype, exc, etraceback)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 262, in execute
    function.__func__(self, request, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/udm/__init__.py", line 107, in _decorated
    request.options['module'] = self._get_module_by_request(request)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/udm/__init__.py", line 241, in _get_module_by_request
    return UDM_Module(module_name)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/udm/udm_ldap.py", line 410, in __init__
    self.load(force_reload=force_reload)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/udm/udm_ldap.py", line 432, in load
    self.module = _module_cache.get(module, None, force_reload, *self.get_ldap_connection())  # FIXME: template_object not used?!
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/udm/udm_ldap.py", line 419, in get_ldap_connection
    self.ldap_connection, po = get_user_connection(bind=get_bind_function(), write=True)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/ldap.py", line 94, in get_user_connection
    return connection()
  File "/usr/lib/python2.7/dist-packages/univention/management/console/ldap.py", line 149, in _decorated
    kwargs[loarg], kwargs[poarg] = lo, po = getter()
  File "/usr/lib/python2.7/dist-packages/univention/management/console/ldap.py", line 139, in getter
    conn = connection()
  File "/usr/lib/python2.7/dist-packages/univention/management/console/ldap.py", line 53, in connection
    bind(lo)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/udm/__init__.py", line 197, in bind_user_connection
    super(Instance, self).bind_user_connection(lo)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 416, in bind_user_connection
    lo.lo.bind_saml(self._password)
  File "/usr/lib/python2.7/dist-packages/univention/uldap.py", line 207, in _decorated
    return func(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/univention/uldap.py", line 319, in bind_saml
    self.lo.sasl_interactive_bind_s('', saml)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 962, in sasl_interactive_bind_s
    res = self._apply_method_s(SimpleLDAPObject.sasl_interactive_bind_s,*args,**kwargs)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 931, in _apply_method_s
    return func(self,*args,**kwargs)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 244, in sasl_interactive_bind_s
    return self._ldap_call(self._l.sasl_interactive_bind_s,who,auth,RequestControlTuples(serverctrls),RequestControlTuples(clientctrls),sasl_flags)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 106, in _ldap_call
    result = func(*args,**kwargs)
STRONG_AUTH_NOT_SUPPORTED: {'info': "SASL(-4): no mechanism available: Couldn't find mech SAML", 'desc': 'Authentication method not supported'}

Thank you
Greets Matthias

Whole domain slowly dying
#2

Hi,

did you try to re-join the backup server?

/KNEBB

#3

Hi,
i tryed to re-join.
No luck so far.

Matthias

#4

Then there should be some information why the re-join did fail. join.log and/ or other files from /var/log/univention

/KNEBB

#5

Well, i cleared join.log, started a rejoin and it seems it does not fail to join:
https://pastebin.ubuntu.com/p/zkFgPTX7HK/

I still wonder about the last line in the traceback:

STRONG_AUTH_NOT_SUPPORTED: {'info': "SASL(-4): no mechanism available: Couldn't find mech SAML

Maybe to mention an other thing: When i disconnet the Primary server from the Network the UMC on the Backup Server works.

Matthias

#6

Hello,
each time when i try to open a umc-Module like users or something else i get on the Primary Server such lines in the Syslog:

Aug 11 15:58:25 dc1 slapd[1529]: SASL [conn=1391] Failure: Couldn't find mech SAML

Matthias

#7

Here we have a first indicator:

rsync: opendir "/etc/univention/ssl/192.168.200.9" failed: Permission denied (13)
rsync: opendir "/etc/univention/ssl/rackstation.faschang.local" failed: Permission denied (13)
rsync: send_files failed to open "/etc/univention/ssl/horizon-a.faschang.local/cert.pfx": Permission denied (13)
rsync: send_files failed to open "/etc/univention/ssl/horizon-a.faschang.local/cert.pfx.tmp": Permission denied (13)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.2]

Further more the local ldap server does not start due to some (currently unknown) errors:

Job for slapd.service failed because the control process exited with error code.
See "systemctl status slapd.service" and "journalctl -xe" for details.

The reason might be this:

6113b1de /etc/ldap/slapd.conf: line 355: invalid path: No such file or directory
slaptest: bad configuration file!
[...]
slapd: Kein Prozess gefunden

A following error then later to be seen:

Could not find machine account in secrets database: Failed to fetch machine account password from secrets.ldb: Could not find entry to match filter: '(&(flatname=INTRANET)(objectclass=primaryDomain))' base: 'cn=Primary Domains': No such object: dsdb_search at ../../source4/dsdb/common/util.c:4733 and failed to open /var/lib/samba/private/secrets.tdb: NT_STATUS_CANT_ACCESS_DOMAIN_INFO
ERROR(<class 'samba.join.DCJoinException'>): uncaught exception - Can't join, error: Not removing account dns-DC2 which looks like a Samba DNS service account but does not have servicePrincipalName=dns/dc2.intranet.faschang.at

I suggest to do the following:

  1. Remove ALL DC2 entries from the master. Search in the forum here how to decommission a server in the right way. Follow the steps there.
  2. Uninstall Samba from DC2
  3. Uninstall (if installed) S4-connector
  4. Make sure ldap is porperly installed (apt --reinstall)
  5. Make sure ldap conf file is ok (ucr commit)
  6. Check existence of failed.ldif- delete if found
  7. run univention-join an join the server from scratch
  8. Install Samba on the properly joined server

/KNEBB

#8

Thank you very much knebb for your Help!

I fixed those Permissions, anyway i think those are only “cosmetical” They happen when a external Certificate is created or renewed with the root account.

When i run systemctl status slapd.service after the mentioned reboot after the rejoin it shows me slapd Active and running.

I really struggle with your suggestions because of serveral things:
For Point 1: I found How-To: Remove a Server in the Knowledge Base. There i have to shut down dc2 to correctly demote it.
Since yesterday i can not access the UMC on DC1 anymore to remove dc2. It simply stays Grey.
Syslog tells me:

Aug 12 08:18:53 dc1 simplesamlphp[31337]: 5 STAT [e598521abd] passive-saml20-idp-SSO https://dc1.intranet.faschang.at/univention/saml/metadata https://ucs-sso.intranet.faschang.at/simplesamlphp/sa
ml2/idp/metadata.php NA
Aug 12 08:18:54 dc1 python2.7: pam_ldap: error trying to bind as user "uid=Administrator,cn=users,dc=intranet,dc=faschang,dc=at" (Invalid credentials)

I reassured the correct Password with entering a definitly wrong one which throws an other Message, so i am sure that the Password from the above attempt was correct:

Aug 12 08:39:46 dc1 simplesamlphp[1948]: 4 [e25270e321] Returning error to SP with entity ID ''https://dc1.intranet.faschang.at/univention/saml/metadata''.
Aug 12 08:39:46 dc1 simplesamlphp[1948]: 4 [e25270e321] SimpleSAML\Module\saml\Error\NoPassive: Passive authentication not supported./NoPassive
Aug 12 08:39:48 dc1 check_nrpe: Remote 192.168.0.219 accepted a Version 3 Packet
Aug 12 08:39:50 dc1 simplesamlphp[1964]: 5 STAT [e25270e321] Unsuccessful login attempt from 192.168.0.43.

I think about Restore a Backup from Yesterday morning where both UMCs on Primary and Backup were at least accessable.

Point 2 and 3: Remove Samba and its Connector from a powered of Machine is hopefully unproblematical if Step 1 is done correct.

Point 4 and 5: Those sound accaptable. Anyway does it feel like fixing Symptoms not the source of Pain

May i mention that all those Troubles started with a Broken Master-Server:

  1. There is no failed.ldif on the backup-server or on the Primary

All in all at the moment i think the problem is still on the Primary-Server and all other Problems are only Symptoms caused by that Problems.
I really would appreciate your Opinion to this.

Thank you
Matthias

#9

Hi,

ok, I did not get it right you are havving issues on the primary server. My fault.

And yes, fix the primary first before further troubleshooting the secondary.
Run univention-run-diagnostic-checks -t all and post the results.

/KNEBB

#10

Thank you for responding.

Here is the output:

root@dc1:~# univention-run-diagnostic-checks -t all
Domain Admin Login: Administrator
Password:

You can find the logging messages of the diagnostic modules at /var/log/univention/management-console-module-diagnostic.log

ran 00_check_server_password successfully.
ran 01_ssh_connection successfully.
ran 02_certificate_check successfully.
ran 03_check_notifier_replication successfully.
############################### Start 04_saml_certificate_check ################################
## Check failed: 04_saml_certificate_check - Überprüfung der SAML-Zertifikate fehlgeschlagen! ##
Das Zertifikat des SAML Identity Providers stimmt nicht überein.
################################ End 04_saml_certificate_check #################################
ran 10_gateway successfully.
ran 11_nameserver successfully.
ran 12_proxy successfully.
ran 20_check_nameservers successfully.
ran 21_check_join_status successfully.
ran 22_kdc_service successfully.
ran 23_check_update_sites successfully.
ran 30_disk_usage successfully.
ran 31_file_permissions successfully.
ran 32_security_limits successfully.
ran 40_samba_tool_dbcheck successfully.
ran 41_samba_tool_showrepl successfully.
############################# Start 43_connectors4_rejects ############################
## Check failed: 43_connectors4_rejects - Nicht synchronisierte S4 Connector Objekte ##
1 nicht synchronisierte UCS Objekte und 1 nicht synchronisierte S4 Objekte. Weitere Hinweise finden Sie unter {sdb}.
Nicht synchronisierte UCS Objekte:
UCS DN: cn=Console Logon,cn=Builtin,dc=intranet,dc=faschang,dc=at, S4 DN: cn=console logon,cn=builtin,DC=intranet,DC=faschang,DC=at, Dateiname: /var/lib/univention-connector/s4/1626790040.507805
Nicht synchronisierte S4 Objekte:
S4 DN: CN=Console Logon,CN=Builtin,DC=intranet,DC=faschang,DC=at, UCS DN: cn=console logon,cn=builtin,dc=intranet,dc=faschang,dc=at
############################## End 43_connectors4_rejects ############################
ran 44_well_known_sid_check successfully.
ran 45_heimdal_on_samba4_dc successfully.
ran 46_kerberos_ddns_update successfully.
ran 50_check_ucr_templates successfully.
ran 51_hostname_check successfully.
ran 52_mail_acl_sync successfully.
ran 53_package_status successfully.
ran 54_sources_list_check successfully.
ran 55_user_migration successfully.
ran 56_univention_types successfully.
ran 57_univention_server_role_windows successfully.
ran 58_check_memberOf successfully.
ran 59_ldap_server_name successfully.
ran 60_old_schema_registration successfully.
root@dc1:~#

Matthias

#11

As this feels like riding a dead Horse i think about if backup2master would be a fast solution.

Matthias

#12

Only guessing: which will fail.

Anyways, use search function here for “SAML” in the Knowledgebase and you will find loads of matching topics where this one seems to be a good match. Try it.

/KNEBB

#13

Good Mornig,
i ran up and down the Results for Saml the last two Weeks with no working Result.

For Excample yours :

root@dc1:~# ldapsearch -LLL -x -ZZ -p7389 -h $'(hostname -f)' -D uid=sys-idp-user,cn=users,$(ucr get ldap/base) -w $(cat /etc/idp-ldap-user.secret) uid=sys-idp-user uid
dn: uid=sys-idp-user,cn=users,dc=intranet,dc=faschang,dc=at
uid: sys-idp-user

Looks like Credentials are still correct.

I have a bunch of apt logs where i can nearly reconstruct what happend before everythig went into this disaster.
It started with uninstall Nagios and Prometheus combined with two errata Updates. Prometheus uninstallation seems to set univention-server-master, univention-management-console an dozens of other Packages as Obsolete.
I did not recognize this first, i guess because the umc typically answers itself everything with yes.
After installing vmware tools and a reboot the packages were removed and the Troubles started.
Until now i reinstalled lots of Packages and the Master works as it works now.

Matthias

#14

Hi as far as I remember, there is some logfile or a list of installed packages somewhere. Your might try to install the missing ones.

But I guess a re-installation of the primary is needed here. Sorry for this.

Sometimes package dependecies are a nightmare.

Mastodon