Hi Valentin,
We’re only getting some different errors now, but no users/groups on the target system (see below). Any other logs i can check or some debug setting i can adjust to get some insight?
23.03.20 13:01:02.232 LISTENER ( WARN ) : initializing module univention_user_group_sync_source_generate
23.03.20 13:01:11.472 LISTENER ( WARN ) : finished initializing module univention_user_group_sync_source_generate with rv=0
Kind regards
Hi jester,
those messages mean that the module was initialized successfully, so it should be working.
Some things you can check now:
/var/lib/univention-user-group-sync when you change a user?/var/log/univention/user-group-sync.log on the destination for thatMake sure that the objects you’re trying to sync are matched by the default filter mentioned in the docs: Cool Solution - Sync Users and Groups into a second Domain
Search for the paragraph " Filter the LDAP objects to be synchronized".
If you have set a custom filter with ldap/sync/filter, make sure that the objects you’re trying to transfer are actually matched by that filter (use univention-ldapsearch to test both).
Best regards,
Valentin
Hi Valentin,
Yes, files are being generated (like: 01584964871.2601819). But these files do not appear on the ‘destination’ server.
Kind regards.
Hi Valentin,
I’m guessing the files are being transferred with scp to the ‘destination’, so i logged in as the ucs-sync user and tried it manually:
root:# su - ucs-sync
ucs-sync:$ cd ~
ucs-sync:$ pwd
/var/lib/univention-user-group-sync
ucs-sync:$ scp -i ~/.ssh/id_rsa.pub /var/lib/univention-user-group-sync/test.txt ucs-sync@192.168.0.16:/var/lib/univention-user-group-sync/
Enter passphrase for key '/var/lib/univention-user-group-sync/.ssh/id_rsa.pub':
Password:
This seems odd. I’ve re-created the rsa keys as root with your command from a few posts back and also as the ucs-sync user and made sure no password was entered… Result is the same.
This weem weird to me.
Did you copy the ssh key to the destination system? Your output looks like you’re being asked for the (empty) password for the key first and then for the password of ucs-sync@192.168.0.16. That hints at the ssh key not being present at the destination.
Please execute the ssh-copy-id from the cool solution again.
Done that, even checked if the key was added to the /var/lib/univention-user-group-sync/.ssh/authorized_keys on the destination system.
In that case I think somethings wrong with the key. Maybe it does have a password set for some reason?
Hi Valentin,
Will try a clean install of both vm’s and re-do the setup to see if it makes a difference. Will report back my findings.
Stay safe & kind regards.
Installing the univention-user-group-sync-source package on the ‘source’ server i straight away get 2 lines of text in red (full output at the end of post):
Warning: cron.service changed on disk. Run 'systemctl daemon-reload' to reload units.0 univention_user_group_sync_source_generate /usr/lib/univention-directory-listener/system/univention_user_group_sync_source_generate.pyThe first one does not seem to upsetting because there is a Reloading systemd daemon below it. But could the second one be the source of my trouble?
How can i help to troubleshoot this further?
root@ucs:~# univention-install univention-user-group-sync-source
Ign:1 https://updates.software-univention.de/4.0/maintained 4.0-0/all/ InRelease
...
Get:202 https://updates.software-univention.de/4.0/unmaintained/component cool-solutions/amd64/ Packages [2,499 B]
Fetched 49.0 kB in 7s (6,667 B/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
univention-user-group-sync-source
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 12.7 kB of archives.
After this operation, 73.7 kB of additional disk space will be used.
Get:1 https://updates.software-univention.de/4.4/unmaintained/component cool-solutions/all/ univention-user-group-sync-source 2.1.0-23A~4.4.0.202002251544 [12.7 kB]
Preconfiguring packages ...
Fetched 12.7 kB in 0s (77.8 kB/s)
Selecting previously unselected package univention-user-group-sync-source.
(Reading database ... 86911 files and directories currently installed.)
Preparing to unpack .../univention-user-group-sync-source_2.1.0-23A~4.4.0.202002251544_all.deb ...
Unpacking univention-user-group-sync-source (2.1.0-23A~4.4.0.202002251544) ...
Setting up univention-user-group-sync-source (2.1.0-23A~4.4.0.202002251544) ...
Script: /etc/univention/templates/scripts/restart-listener
Calling joinscript 99univention-user-group-sync-source_add-ucs-sync.inst ...
2020-03-24 18:00:13.449564124+01:00 (in joinscript_init)
Creating local service user
Object created: uid=ucs-sync,cn=users,dc=mydomain,dc=loc
Local service user ucs-sync exists now.
No passwd entry for user 'ucs-sync'
Done!
2020-03-24 18:00:14.480297173+01:00 (in joinscript_save_current_version)
Joinscript 99univention-user-group-sync-source_add-ucs-sync.inst finished with exitcode 0
Creating a synchronization cron job
Create cron/ldap-sync-src/command
Create cron/ldap-sync-src/time
Create cron/ldap-sync-src/user
Create cron/ldap-sync-src/description
File: /etc/cron.d/univention-ucr-cronjobs
Restarting univention-directory-listener Service
Restarting cron Service
Warning: cron.service changed on disk. Run 'systemctl daemon-reload' to reload units.
Reloading systemd daemon
Modules:
3 ad-connector /usr/lib/univention-directory-listener/system/ad-connector.py
3 app_attributes /usr/lib/univention-directory-listener/system/app_attributes.py
3 bind /usr/lib/univention-directory-listener/system/bind.py
3 faillog /usr/lib/univention-directory-listener/system/faillog.py
3 gencertificate /usr/lib/univention-directory-listener/system/gencertificate.py
3 hosteddomains /usr/lib/univention-directory-listener/system/hosteddomains.py
3 keytab-member /usr/lib/univention-directory-listener/system/keytab-member.py
3 keytab /usr/lib/univention-directory-listener/system/keytab.py
3 ldap_extension /usr/lib/univention-directory-listener/system/ldap_extension.py
3 ldap_server /usr/lib/univention-directory-listener/system/ldap_server.py
3 license_uuid /usr/lib/univention-directory-listener/system/license_uuid.py
3 nagios-client /usr/lib/univention-directory-listener/system/nagios-client.py
3 nfs-homes /usr/lib/univention-directory-listener/system/nfs-homes.py
3 nfs-shares /usr/lib/univention-directory-listener/system/nfs-shares.py
3 nscd_update /usr/lib/univention-directory-listener/system/nscd.py
3 nss /usr/lib/univention-directory-listener/system/nss.py
3 pkgdb-watch /usr/lib/univention-directory-listener/system/pkgdb-watch.py
3 portal_groups /usr/lib/univention-directory-listener/system/portal_groups.py
3 portal_server /usr/lib/univention-directory-listener/system/portal_server.py
3 quota /usr/lib/univention-directory-listener/system/quota.py
3 samba-privileges /usr/lib/univention-directory-listener/system/samba-privileges.py
3 samba-shares /usr/lib/univention-directory-listener/system/samba-shares.py
3 udm_extension /usr/lib/univention-directory-listener/system/udm_extension.py
3 umc-service-providers /usr/lib/univention-directory-listener/system/umc-service-providers.py
3 univention-admin-diary-backend /usr/lib/univention-directory-listener/system/univention-admin-diary-backend.py
3 univention-saml-idp-config /usr/lib/univention-directory-listener/system/univention-saml-idp-config.py
3 univention-saml-servers /usr/lib/univention-directory-listener/system/univention-saml-servers.py
3 univention-saml-simplesamlphp-configuration /usr/lib/univention-directory-listener/system/univention-saml-simplesamlphp-configuration.py
0 univention_user_group_sync_source_generate /usr/lib/univention-directory-listener/system/univention_user_group_sync_source_generate.py
3 well-known-sid-name-mapping /usr/lib/univention-directory-listener/system/well-known-sid-name-mapping.py
Processing triggers for univention-config (14.0.0-12A~4.4.0.201909041712) ...
dpkg-query: no packages found matching ldapacl_66univention-appcenter_app.acl
Kind regards.
Also the /var/log/univention/listener.log log is showing this error again:
24.03.20 18:00:21.373 LISTENER ( ERROR ) : import of filename=/usr/lib/univention-directory-listener/system/univention_user_group_sync_source_generate.py failed
Traceback (most recent call last):
File "/usr/lib/univention-directory-listener/system/univention_user_group_sync_source_generate.py", line 74, in <module>
owning_user_number = pwd.getpwnam('ucs-sync').pw_uid
KeyError: 'getpwnam(): name not found: ucs-sync'
24.03.20 18:00:21.373 LISTENER ( ERROR ) : import of filename=/usr/lib/univention-directory-listener/system/univention_user_group_sync_source_generate.py failed in module_import()
The ucs-sync user exists in the WebGUI. And a univention-check-join-status displays: Joined successfully
This output means that the initialization of the listener module failed. Very likely due to the error you posted in your last comment.
The user creation works and the user does appear in the local passwd db but a su ucs-sync does not work. I assume this is due to your AD Member Setup. I haven’t seen this cool solution work in such an environment - only in UCS domains, with and without a connected AD.
Depending on what you intend to use UCS for in your scenario, it might be feasible to install UCS not as a member but as an independent DC master and install the AD Connector from the App Center on it to get users and groups from AD. In that scenario the cool solution should work.
We can also modify the cool solution to work in such a scenario in the professional service. Feel free to drop me a message for that. I can get you in touch with sales then.
Hi @jester,
I might have another interesting finding here. By coincidence we were able to observe the problem on another standard UCS domain as well yesterday and I’ve implemented a fix. Package updates should be available soon. It seems to be a timing problem between restarting the listener, it noticing that the import of the listener module which generates the files succeeds and triggering a resync of that module.
Hi Valentin,
Sorry for the lack of response, awkward times at the moment.
Hope to find some time tomorrow to see if your fix will help my case… would be great.
Thanks for your perseverance.
Kind regards.
Hi Jester,
we have just released an update for the sync packages, that should fix the issue that occurs during the installation of the source package.
I suggest you clean up just to make sure, then update the package:
eval "$(ucr shell)" udm users/user remove --ignore_not_exists --dn "uid=ucs-sync,cn=users,$ldap_base" rm -rf /var/lib/univention-user-group-sync # This will put out an error from postrm, you can ignore that apt remove univention-user-group-sync-source univention-install univention-user-group-sync-source
Best regards,
Valentin
Hi Valentin,
Finally found some time to give the updated package a test.
The files in /var/lib/univention-user-group-sync are now being transferred to the ‘destination’ system, so that part is working now!
On the destination system some of the users from the source system have been imported, but only a small part. When running the command univention_user_group_sync_dest.py i get:
Reading file /var/lib/univention-user-group-sync/01586186643.9837890
W: Different objectClasses detected NEW ['krb5KDCEntry', 'person', 'automount', 'top', 'inetOrgPerson', 'krb5Principal', 'organizationalPerson', 'univentionPWHistory', 'univentionMail', 'univentionObject', 'shadowAccount', 'sambaSamAccount', 'posixAccount'] vs. OLD ['krb5KDCEntry', 'organizationalPerson', 'automount', 'top', 'inetOrgPerson', 'krb5Principal', 'person', 'univentionPWHistory', 'univentionMail', 'univentionSAMLEnabled', 'shadowAccount', 'sambaSamAccount', 'nextcloudUser', 'posixAccount', 'univentionObject']
W: Ignoring difference in objectClass as specified in ldap/sync/ignore_error/objectClass_difference : univentionSAMLEnabled
E: During User.modify_ldap: Traceback (most recent call last):
File "/usr/bin/univention_user_group_sync_dest.py", line 440, in _direct_update
lo.modify(user.position.getDn(), modlist)
File "/usr/lib/python2.7/dist-packages/univention/admin/uldap.py", line 902, in modify
raise univention.admin.uexceptions.ldapError(_err2str(msg), original_exception=msg)
ldapError: Object class violation: attribute 'nextcloudEnabled' not allowed
And in /var/log/univention/user-group-sync.log i see a similar error repeat:
Apr 06 18:50:02: Reading file /var/lib/univention-user-group-sync/01586186643.9837890
Apr 06 18:50:02: Modify User: 'uid=myuser,cn=users,dc=mydomain,dc=loc'
Apr 06 18:50:02: W: Different objectClasses detected NEW ['krb5KDCEntry', 'person', 'automount', 'top', 'inetOrgPerson', 'krb5Principal', 'organizationalPerson', 'univentionPWHistory', 'univentionMail', 'univentionObject', 'shadowAccount', 'sambaSamAccount', 'posixAccount'] vs. OLD ['krb5KDCEntry', 'organizationalPerson', 'automount', 'top', 'inetOrgPerson', 'krb5Principal', 'person', 'univentionPWHistory', 'univentionMail', 'univentionSAMLEnabled', 'shadowAccount', 'sambaSamAccount', 'nextcloudUser', 'posixAccount', 'univentionObject']
Apr 06 18:50:02: W: Ignoring difference in objectClass as specified in ldap/sync/ignore_error/objectClass_difference : univentionSAMLEnabled
Apr 06 18:50:02: E: During User.modify_ldap: Traceback (most recent call last):
File "/usr/bin/univention_user_group_sync_dest.py", line 440, in _direct_update
lo.modify(user.position.getDn(), modlist)
File "/usr/lib/python2.7/dist-packages/univention/admin/uldap.py", line 902, in modify
raise univention.admin.uexceptions.ldapError(_err2str(msg), original_exception=msg)
ldapError: Object class violation: attribute 'nextcloudEnabled' not allowed
I’m guessing the warnings can be ignored (possibly resolved by installing Active Directory-compatible Domain Controller component), but the error looks more serious.
Kind regards.
Hi jester,
please refer to the paragraph " Remove attributes/objectClasses before sync" in the docs on how to deal with attributes or objectClasses which are not present in both domains.
Best regards,
Valentin
Hi Valentin,
Thanks again for your reply!
Right, missed that. My test apps were installed on the destination system (in dmz) so was “Object class violation” paragraph in my case. But still it would not work, changes did not get replicated.
I’ve started over again, one freshly installed source UCS in LAN (just domaincontroller_master, no AD member this time) and a fresh UCS (also domaincontroller_master) destination install in DMZ. Both fully updated.
Ran through the ‘User roup sync’ installation again, everything fine. But even after >10min. no LDAP export files appearing in /var/lib/univention-user-group-sync.
On both source and target systems:
univention-check-join-status = .Joined successfully
univention-run-join-scripts = everything skipped (already executed) - all OKunivention-ldapsearch -LLL -b "uid=ucs-sync,cn=users,$(ucr get ldap/base)" - both return LDAP account info, so ucs-sync user is present.After a source server reboot: LDAP export files are created… And after some time they are transferred to the target system and my test user appears… But only this once!
Any changes to my test user on the source system do not get replicated, not even after a reboot.
18.04.20 16:18:47.004 LDAP ( PROCESS ) : connecting to ldap://ucs01.mydomain.loc:7389
18.04.20 16:18:47.008 LISTENER ( PROCESS ) : updating 'uid=user01,cn=users,dc=mydomain,dc=loc' command m
18.04.20 16:28:09.680 LDAP ( PROCESS ) : connecting to ldap://ucs01.mydomain.loc:7389
18.04.20 16:28:09.683 LISTENER ( PROCESS ) : updating 'uid=user01,cn=users,dc=mydomain,dc=loc' command m
18.04.20 17:21:22.009 LDAP ( PROCESS ) : connecting to ldap://ucs01.mydomain.loc:7389
18.04.20 17:21:22.013 LISTENER ( PROCESS ) : updating 'uid=user01,cn=users,dc=mydomain,dc=loc' command m
18.04.20 17:22:03.629 LISTENER ( WARN ) : received signal 15
18.04.20 17:22:30.465 DEBUG_INIT
18.04.20 17:22:30.480 LISTENER ( WARN ) : Notifier/LDAP server is ucs01.mydomain.loc:7389
18.04.20 17:22:30.480 LDAP ( PROCESS ) : connecting to ldap://ucs01.mydomain.loc:7389
18.04.20 17:22:31.675 LISTENER ( ERROR ) : import of filename=/usr/lib/univention-directory-listener/system/univention_user_group_sync_source_generate.py failed
Traceback (most recent call last):
File "/usr/lib/univention-directory-listener/system/univention_user_group_sync_source_generate.py", line 74, in <module>
owning_user_number = pwd.getpwnam('ucs-sync').pw_uid
KeyError: 'getpwnam(): name not found: ucs-sync'
18.04.20 17:22:31.675 LISTENER ( ERROR ) : import of filename=/usr/lib/univention-directory-listener/system/univention_user_group_sync_source_generate.py failed in module_import()
UNIVENTION_DEBUG_BEGIN : uldap.__open host=ucs01.mydomain.loc port=7389 base=dc=mydomain,dc=loc
UNIVENTION_DEBUG_END : uldap.__open host=ucs01.mydomain.loc port=7389 base=dc=mydomain,dc=loc
18.04.20 17:39:48.092 LDAP ( PROCESS ) : connecting to ldap://ucs01.mydomain.loc:7389
18.04.20 17:39:48.096 LISTENER ( PROCESS ) : updating 'uid=user01,cn=users,dc=mydomain,dc=loc' command m
Some more findings:
Looking at /var/lib/univention-user-group-sync there are no LDAP export files, even after reboot. Running univention-directory-listener-ctrl resync univention_user_group_sync_source_generate will generate LDAP export files again in /var/lib/univention-user-group-sync but they never get synced.
Changing to ucs-sync (su - ucs-sync) and manually running univention_user_group_sync_source_synchronize (as what etc/cron.d/univention-ucr-cronjobs does) will sync all LDAP exports. After the import process has finished my changes are visible on the destination server in DMZ.
So to sum things up: only the initial export gets synced, once after a reboot. And then it stops working, on the source system.
I think it would be wise to remove the ucs-4-4 tag from the article for now (because it’s not working on a clean UCS 4.4-4 install)… Just to indicate others to refrain from using it for the moment.
I’ve got the test VMs running with snapshots of a clean state right after a fresh install of UCS …so shoot if you want me to test something.
Kind regards.
Hi jester,
thanks for the details. Unfortunately, I don’t think I can assist you further without having a look at the system. Sorry about that. It seems that there is a problem with the ucs-sync user being replicated as a local user on your system or the Python library fails at least sometimes.
The sync is running without problems at several customers of ours on UCS 4.4 and we couldn’t reproduce this problem in any of the last quality assurances, so at this point I wouldn’t remove the tag. If you’d like to request further assistance by the Professional Service at Univention, feel free to contact us.
Best regards,
Valentin
Hi Valentin,
Just contacted the Univention Head Office to ask how to get further assistance. I explained my situation: user accounts from our Windows AD to a UCS in DMZ for Rocket.Chat. They wondered why i was using this Cool Solution and not the usual way this was set up: a read-only sync to the DMZ.
I was under the impression that a read only sync (I’m guessing we’re talking about a UCS Slave DC in the DMZ) would be less safe and a lot more ports would have to be opened on the firewall.
I was told to ask you this here on the forum, because the person i spoke to was no engineer… so:
Kind regards.