UCS 4.3/4.4 Domain and Management Console: Modules not loaded in most cases

UCS - Univention Corporate Server
#1

Hi there,

since the very first install of UCS, the web-based domain and management console does not load the chosen module when clicking on a module.

Sometimes it works and the module is loaded. Most time it fails. The problem occurs with almost all modules, like DNS, DHCP, Users. The problem occurs even when accessing the web-based domain and management console from different computers and different web browsers. Sometimes, the “Users” module does not load, whereas the “DNS” module loads. Sometimes, it is exactly vice verse. I access UCS usually from different Mac computers. I got the impression that the likelihood of getting a module loaded is higher when working with the Safari browser. But also there, I have a success rate of about 10% to get the module loaded. I need to try again and again, wait, and suddenly, the module loads and shows its content.

In most cases, clicking on a module opens a new page with the module name in the headline without showing any content in the window. This remains unchanged. Sometimes, I receive an error message:

Ein unbekannter Fehler mit Status-Code 502 trat während des Verbindungsaufbaus zum Server auf. Bitte versuchen Sie es später noch einmal.

However, in most cases, no response is shown. When I try 10-20 times, suddenly the module loads and I can work.

Does anybody has a hint to overcome this behavior? What debug information could I check to identify the source of this problem?

Your comments are welcome. Thank you very much.

Best regards,
Peter

#2

Hello @herrep,

HTTP Status Code 502 is “Bad Gateway”. This might be a problem in the network setup. I assume the error is shown via JavaScript in your browser (maybe you can share a screenshot of the error message?) and probably not caused by something in the UCS backend. Can you explain the network layout? Which path should network packets take from your browser to UCS and back again, which network devices need to be passed (firewalls, proxy, routers …)?

Best regards,
Michael

#3

Dear @Grandjean,

Thank you very much for your reply. The full error message (in German) is:

Ein unbekannter Fehler mit Status-Code 502 trat während des Verbindungsaufbaus zum Server auf. Bitte versuchen Sie es später noch einmal.

However, this error message does not appear often and only after a long waiting time, like a timeout. In most cases, simply nothing happens, i.e. the module page opens with the header containing the module name but without displaying any content.

The problem also occurs within my local network where UCS is installed as a VM on a Synology NAS so that there is one virtual network interface in between. Even if I use a client that is connected at the same switch like the Synology NAS, the timeout issues ocur, but less often as compared to a connection via Internet.

I wonder what debugging information should be retrieved to identify the problem.

Best regards,
Peter

#4

Hi,

I do not have a real solution for this issue but I noticed it happens more frequently when the UCS server does not get enough hardware power or is very busy with other tasks.

So your Synology might not be able to offer enough resources to the UCS-VM,

Could you monitor your underlying hardware to see if there is some sort of bottleneck?

/CV

#5

How much memory does your VM have? How much swap? And hoch much is used? = Please post the output of free

#6 
root@server:~# free
              total        used        free      shared  buff/cache   available
Mem:        6115112     3956844      298060       47004     1860208     1840780
Swap:       4194300     1856860     2337440
#7 
top - 09:34:35 up 32 days, 10:06,  2 users,  load average: 2,44, 2,28, 1,77
Tasks: 197 total,   2 running, 194 sleeping,   0 stopped,   1 zombie
%Cpu(s): 57,2 us,  2,8 sy,  0,0 ni,  0,0 id, 38,9 wa,  0,0 hi,  0,5 si,  0,7 st
KiB Mem :  6115112 total,   241992 free,  4031104 used,  1842016 buff/cache
KiB Swap:  4194300 total,  2393068 free,  1801232 used.  1766868 avail Mem
#8

Yeah, that server has way too little memory available. You should size a server’s memory so that it doesn’t have to swap during regular usage and still have at least 1.5 GB of memory available for buffers & cache. In you case your server needs to swap out 1.8 GB of memory, resulting not only in slower operation but likely in out-of-memory scenarios — and the symptoms you described sound a lot like out-of-memory situations.

I’m curious what you’re running on your server that takes up that much memory. Please post the output of ps auxw --sort=vsz

#9

Hi,

this is quite a high load (2,44) but it might depend on the count of CPUs you have assigned to the UCS. For two or one CPU it is a relative high load, for more CPUs it should be fine.
What bothers me is the high 38,9% of IO-Waits together with the very high and variable usage of the swap. I bet this is (at least) one reason for the behaviour.

Please chek this article regarding performance.

/CV

#10

I run a Nextcloud server which currently performs occ fulltextsearch:index over the entire files. The corresponding PHP process already occupies almost 2 GB RAM. Once this process is finished, I will add further 2 GB RAM to the VM.

#11

At present, 2 CPU cores are assigned. The host resources should also allow 4 CPU cores.

#12

Just to inform… using hp server dl360 gen5 (Intel® Xeon® CPU E5450 @ 3.00GHz )running xenserver i cannot have any ucs vm running… i have the same issues until i give up…

Server has 32 gb i give the vm 12 then 16 gb of ram all the cpus etc but always have the same issue…

In the similar server but with less memoy i can run it as physical server… i never can understand what was the issue

#13

Indeed, after added more resources to the UCS VM, I noticed that I could access UCS modules more often, but nevertheless, also with 4 cores and 10 GB RAM I still have some difficulties to get modules loaded in the UCS GUI.

#14

When you encounter the issue the next time, please post the output of

ps auxw --sort=rss
free

along with the content of the two log files /var/log/univention/management-console-web-server.log and …/management-console-server.log

#15

Thank you very much for your hints. The LDAP configuration module presently does not load. Here is the requested output:

root@server:~# ps auxw --sort=rss
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         2  0.0  0.0      0     0 ?        S    Apr01   0:00 [kthreadd]
root         3  0.6  0.0      0     0 ?        S    Apr01  22:44 [ksoftirqd/0]
root         7  1.0  0.0      0     0 ?        S    Apr01  37:49 [rcu_sched]
root         8  0.0  0.0      0     0 ?        S    Apr01   0:00 [rcu_bh]
root         9  0.0  0.0      0     0 ?        S    Apr01   0:05 [migration/0]
root        10  0.0  0.0      0     0 ?        S<   Apr01   0:00 [lru-add-drain]
root        11  0.0  0.0      0     0 ?        S    Apr01   0:02 [watchdog/0]
root        12  0.0  0.0      0     0 ?        S    Apr01   0:00 [cpuhp/0]
root        13  0.0  0.0      0     0 ?        S    Apr01   0:00 [cpuhp/1]
root        14  0.0  0.0      0     0 ?        S    Apr01   0:02 [watchdog/1]
root        15  0.0  0.0      0     0 ?        S    Apr01   0:06 [migration/1]
root        16  0.0  0.0      0     0 ?        S    Apr01   1:39 [ksoftirqd/1]
root        18  0.0  0.0      0     0 ?        S<   Apr01   0:00 [kworker/1:0H]
root        19  0.0  0.0      0     0 ?        S    Apr01   0:00 [cpuhp/2]
root        20  0.0  0.0      0     0 ?        S    Apr01   0:01 [watchdog/2]
root        21  0.0  0.0      0     0 ?        S    Apr01   0:06 [migration/2]
root        22  0.0  0.0      0     0 ?        S    Apr01   1:40 [ksoftirqd/2]
root        24  0.0  0.0      0     0 ?        S<   Apr01   0:00 [kworker/2:0H]
root        25  0.0  0.0      0     0 ?        S    Apr01   0:00 [cpuhp/3]
root        26  0.0  0.0      0     0 ?        S    Apr01   0:01 [watchdog/3]
root        27  0.0  0.0      0     0 ?        S    Apr01   0:07 [migration/3]
root        28  0.0  0.0      0     0 ?        S    Apr01   1:36 [ksoftirqd/3]
root        30  0.0  0.0      0     0 ?        S<   Apr01   0:00 [kworker/3:0H]
root        31  0.0  0.0      0     0 ?        S    Apr01   0:00 [kdevtmpfs]
root        32  0.0  0.0      0     0 ?        S<   Apr01   0:00 [netns]
root        33  0.0  0.0      0     0 ?        S    Apr01   0:00 [khungtaskd]
root        34  0.0  0.0      0     0 ?        S    Apr01   0:00 [oom_reaper]
root        35  0.0  0.0      0     0 ?        S<   Apr01   0:00 [writeback]
root        36  0.0  0.0      0     0 ?        S    Apr01   0:00 [kcompactd0]
root        38  0.0  0.0      0     0 ?        SN   Apr01   0:00 [ksmd]
root        39  0.0  0.0      0     0 ?        SN   Apr01   0:08 [khugepaged]
root        40  0.0  0.0      0     0 ?        S<   Apr01   0:00 [crypto]
root        41  0.0  0.0      0     0 ?        S<   Apr01   0:00 [kintegrityd]
root        42  0.0  0.0      0     0 ?        S<   Apr01   0:00 [bioset]
root        43  0.0  0.0      0     0 ?        S<   Apr01   0:00 [kblockd]
root        44  0.0  0.0      0     0 ?        S<   Apr01   0:00 [devfreq_wq]
root        45  0.0  0.0      0     0 ?        S<   Apr01   0:00 [watchdogd]
root        47  0.1  0.0      0     0 ?        S    Apr01   4:34 [kswapd0]
root        48  0.0  0.0      0     0 ?        S<   Apr01   0:00 [vmstat]
root        60  0.0  0.0      0     0 ?        S<   Apr01   0:00 [kthrotld]
root        61  0.0  0.0      0     0 ?        S<   Apr01   0:00 [ipv6_addrconf]
root       101  0.0  0.0      0     0 ?        S<   Apr01   0:00 [bioset]
root       102  0.0  0.0      0     0 ?        S<   Apr01   0:00 [bioset]
root       103  0.0  0.0      0     0 ?        S<   Apr01   0:00 [bioset]
root       104  0.0  0.0      0     0 ?        S<   Apr01   0:00 [bioset]
root       105  0.0  0.0      0     0 ?        S<   Apr01   0:00 [bioset]
root       106  0.0  0.0      0     0 ?        S<   Apr01   0:00 [bioset]
root       107  0.0  0.0      0     0 ?        S<   Apr01   0:00 [bioset]
root       108  0.0  0.0      0     0 ?        S<   Apr01   0:00 [bioset]
root       113  0.0  0.0      0     0 ?        S<   Apr01   0:00 [ata_sff]
root       138  0.0  0.0      0     0 ?        S    Apr01   0:00 [scsi_eh_0]
root       139  0.0  0.0      0     0 ?        S<   Apr01   0:00 [scsi_tmf_0]
root       140  0.0  0.0      0     0 ?        S    Apr01   0:00 [scsi_eh_1]
root       141  0.0  0.0      0     0 ?        S<   Apr01   0:00 [scsi_tmf_1]
root       142  0.0  0.0      0     0 ?        S    Apr01   0:00 [scsi_eh_2]
root       143  0.0  0.0      0     0 ?        S<   Apr01   0:00 [scsi_tmf_2]
root       148  0.0  0.0      0     0 ?        S<   Apr01   0:00 [bioset]
root       404  0.0  0.0      0     0 ?        S<   Apr01   0:00 [bioset]
root       405  0.0  0.0      0     0 ?        S<   Apr01   0:00 [bioset]
root       420  0.0  0.0      0     0 ?        S<   Apr01   0:00 [kdmflush]
root       421  0.0  0.0      0     0 ?        S<   Apr01   0:00 [bioset]
root       423  0.0  0.0      0     0 ?        S<   Apr01   0:00 [kdmflush]
root       424  0.0  0.0      0     0 ?        S<   Apr01   0:00 [bioset]
root       442  0.0  0.0      0     0 ?        S<   Apr01   0:00 [dm_bufio_cache]
root       478  0.0  0.0      0     0 ?        S<   Apr01   0:00 [kworker/u9:0]
root       490  0.5  0.0      0     0 ?        S    Apr01  18:56 [jbd2/dm-0-8]
root       491  0.0  0.0      0     0 ?        S<   Apr01   0:00 [ext4-rsv-conver]
root       539  0.0  0.0      0     0 ?        S    Apr01   0:00 [kauditd]
root       543  0.0  0.0      0     0 ?        S<   Apr01   0:00 [rpciod]
root       544  0.0  0.0      0     0 ?        S<   Apr01   0:00 [xprtiod]
root       697  0.0  0.0      0     0 ?        S<   Apr01   0:00 [ext4-rsv-conver]
root      1351  0.0  0.0      0     0 ?        S<   Apr01   0:00 [nfsd4_callbacks]
root      1352  0.0  0.0      0     0 ?        S    Apr01   0:00 [lockd]
root      1356  0.0  0.0      0     0 ?        S    Apr01   0:00 [nfsd]
root      1357  0.0  0.0      0     0 ?        S    Apr01   0:00 [nfsd]
root      1358  0.0  0.0      0     0 ?        S    Apr01   0:00 [nfsd]
root      1359  0.0  0.0      0     0 ?        S    Apr01   0:00 [nfsd]
root      1360  0.0  0.0      0     0 ?        S    Apr01   0:00 [nfsd]
root      1361  0.0  0.0      0     0 ?        S    Apr01   0:00 [nfsd]
root      1362  0.0  0.0      0     0 ?        S    Apr01   0:00 [nfsd]
root      1363  0.0  0.0      0     0 ?        S    Apr01   0:00 [nfsd]
root      1472  0.0  0.0      0     0 ?        S<   Apr01   0:00 [nfsiod]
root      7968  0.0  0.0      0     0 ?        S<   Apr01   0:15 [kworker/1:2H]
root      8513  0.0  0.0   4500     0 pts/0    Ss+  Apr01   0:00 /bin/sh -c bash start-libreoffice.sh
root      8555  0.0  0.0  18408     0 pts/0    S+   Apr01   0:00 bash start-libreoffice.sh
root      8558  0.0  0.0   6528     0 pts/0    S+   Apr01   0:00 /usr/bin/inotifywait -e modify /etc/loolwsd/loolwsd.xml
root      9044  0.0  0.0      0     0 ?        S<   08:08   0:01 [kworker/2:1H]
root      9755  0.0  0.0      0     0 ?        S    08:39   0:01 [kworker/u8:1]
root     14285  0.0  0.0      0     0 ?        S    09:39   0:00 [kworker/u8:0]
root     14765  0.1  0.0      0     0 ?        S    10:01   0:01 [kworker/2:1]
root     14770  0.8  0.0      0     0 ?        S<   10:02   0:12 [kworker/0:0H]
root     14778  0.0  0.0      0     0 ?        S    10:04   0:00 [kworker/1:2]
root     14814  0.0  0.0      0     0 ?        S    10:05   0:00 [kworker/3:0]
root     18481  0.2  0.0      0     0 ?        S    10:12   0:02 [kworker/0:2]
root     18489  0.0  0.0      0     0 ?        S    10:13   0:00 [kworker/2:0]
root     18613  0.4  0.0      0     0 ?        S    10:17   0:02 [kworker/0:0]
root     18629  0.0  0.0      0     0 ?        S    10:19   0:00 [kworker/1:0]
root     18652  0.5  0.0      0     0 ?        S<   10:20   0:02 [kworker/0:1H]
root     18660  0.2  0.0      0     0 ?        S    10:21   0:00 [kworker/3:3]
root     18667  0.4  0.0      0     0 ?        S    10:22   0:01 [kworker/0:1]
root     18719  0.0  0.0      0     0 ?        Z    10:25   0:00 [xmlsec1] <defunct>
root     18730  1.6  0.0      0     0 ?        S<   10:25   0:01 [kworker/0:2H]
root     18732  0.0  0.0      0     0 ?        S    10:25   0:00 [kworker/1:1]
root     18734  0.0  0.0      0     0 ?        S    10:26   0:00 [kworker/3:1]
root     32293  0.0  0.0      0     0 ?        S<   06:35   0:03 [kworker/3:2H]
root      8557  0.0  0.0  18408     4 pts/0    S+   Apr01   0:00 bash start-libreoffice.sh
root      8556  0.0  0.0  47328     8 pts/0    S+   Apr01   0:00 su -c /usr/bin/loolwsd --version --o:sys_template_path=/opt/lool/sy
root      8536  0.0  0.0  18412    12 pts/0    S+   Apr01   0:00 bash start-libreoffice.sh
root      1732  0.0  0.0  68248    20 ?        S    Apr01   0:06 /usr/lib/heimdal-servers/kdc --config-file=/etc/heimdal-kdc/kdc.con
root      1737  0.0  0.0  68248    24 ?        S    Apr01   0:06 /usr/lib/heimdal-servers/kdc --config-file=/etc/heimdal-kdc/kdc.con
root      1620  0.0  0.0   6272    92 ?        S    Apr01   0:33 /usr/sbin/ifplugd -i ens3 -q -f -u0 -d10 -w -I
root       570  0.0  0.0  20824   172 ?        Ss   Apr01   0:00 /usr/sbin/blkmapd
root       724  0.0  0.0  23344   208 ?        Ss   Apr01   0:00 /usr/sbin/rpc.idmapd
root      1749  0.0  0.0  49324   564 ?        Ss   Apr01   0:00 /usr/sbin/saslauthd -a pam -c -r -m /var/run/saslauthd -t 1800 -n 5
root      1750  0.0  0.0  49324   576 ?        S    Apr01   0:00 /usr/sbin/saslauthd -a pam -c -r -m /var/run/saslauthd -t 1800 -n 5
root      1751  0.0  0.0  49324   576 ?        S    Apr01   0:00 /usr/sbin/saslauthd -a pam -c -r -m /var/run/saslauthd -t 1800 -n 5
root      1752  0.0  0.0  49324   576 ?        S    Apr01   0:00 /usr/sbin/saslauthd -a pam -c -r -m /var/run/saslauthd -t 1800 -n 5
root      1753  0.0  0.0  49324   576 ?        S    Apr01   0:00 /usr/sbin/saslauthd -a pam -c -r -m /var/run/saslauthd -t 1800 -n 5
root       839  0.0  0.0   4048   640 ?        Ss   Apr01   0:00 runsv univention-directory-listener
root       840  0.0  0.0   4048   644 ?        Ss   Apr01   0:00 runsv univention-directory-notifier
root      1335  0.0  0.0  35568   696 ?        Ss   Apr01   0:00 /usr/sbin/rpc.mountd --manage-gids --port 32767
root      1064  0.0  0.0  20472  1048 ?        Ss   Apr01   0:00 /sbin/dhclient -4 -v -pf /run/dhclient.ens3.pid -lf /var/lib/dhcp/d
samlcgi   1773  0.0  0.0 113168  1068 ?        Ss   Apr01   0:00 /usr/bin/stunnel4 /etc/stunnel/univention_saml.conf
root       831  0.0  0.0   4200  1092 ?        S    Apr01   0:06 runsvdir -P /etc/service log: .....................................
root       572  0.0  0.0 107196  1284 ?        Ss   Apr01   0:00 /sbin/lvmetad -f
root      8494  0.0  0.0 357436  1360 ?        Sl   Apr01   0:01 docker-containerd-shim 1156d33d5fb7ba1afa38c32af7f1cc53fdc549294f3d
postgres   909  0.0  0.0 122944  1524 ?        Ss   Apr01   0:06 postgres: stats collector process   
root      8488  0.0  0.0 332180  1568 ?        Sl   Apr01   0:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 9980 -
statd     1511  0.0  0.0  35428  1768 ?        Ss   Apr01   0:00 /sbin/rpc.statd --no-notify --port 32765 --outgoing-port 32766
elastic+ 12101  0.0  0.0 135460  1796 ?        Sl   Apr01   0:00 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bi
daemon     786  0.0  0.0  27964  1984 ?        Ss   Apr01   0:00 /usr/sbin/atd -f
samlcgi   1775  0.1  0.0 327240  2192 ?        Ssl  Apr01   5:17 /usr/bin/memcached -m 64 -s /var/run/univention-saml/memcached.sock
root       722  0.0  0.0  35252  2372 ?        Ss   Apr01   0:00 /usr/sbin/rpc.gssd
postgres   905  0.0  0.0 267944  2512 ?        Ss   Apr01   0:00 postgres: checkpointer process   
postgres   907  0.0  0.0 267944  2512 ?        Ss   Apr01   0:07 postgres: wal writer process   
postgres   906  0.0  0.0 267944  2516 ?        Ss   Apr01   0:07 postgres: writer process   
root      1676  0.0  0.0  22516  2664 ?        Ss   Apr01   0:08 /usr/sbin/qemu-ga --daemonize -m virtio-serial -p /dev/virtio-ports
root       784  0.0  0.0  31200  2692 ?        Ss   Apr01   0:04 /usr/sbin/cron -f
nagios    1552  0.0  0.0  23908  2884 ?        Ss   Apr01   0:00 /usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f
root       768  0.0  0.0  36196  2956 ?        Ss   Apr01   0:34 /usr/sbin/inetd
root      2996  0.0  0.0  14492  2988 tty1     S    Apr01   0:00 -bash
root       764  0.0  0.0  49868  3068 ?        Ss   Apr01   0:00 /sbin/rpcbind -f -w
root     18755  6.0  0.0  31712  3308 pts/1    R+   10:26   0:00 ps auxw --sort=rss
root       810  0.0  0.0  37980  3328 ?        Ss   Apr01   0:01 /lib/systemd/systemd-logind
root       581  0.0  0.0  45964  3628 ?        Ss   Apr01   0:02 /lib/systemd/systemd-udevd
message+   791  0.0  0.0  47196  3660 ?        Ss   Apr01   0:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfi
redis     1344  2.1  0.0  42392  3692 ?        Rsl  Apr01  74:58 /usr/bin/redis-server 127.0.0.1:6379
root     17027  0.0  0.0  81180  3752 ?        Ss   Apr01   0:03 /usr/lib/postfix/sbin/master -w
ntp       1730  0.0  0.0  99960  3860 ?        Ssl  Apr01   0:46 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -c /run/ntp.conf.dhcp -u 119
root      1306  0.0  0.0  63656  4268 ?        Ss   Apr01   0:00 /usr/sbin/sshd -D
root      2304  0.3  0.0 384984  4572 ?        Ssl  Apr01  11:25 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-co
kopano     849  0.0  0.0 139964  4588 ?        S    Apr01   0:00 /usr/sbin/kopano-spooler -F
root      1606  0.0  0.0  68248  4588 ?        S    Apr01   0:00 /usr/lib/heimdal-servers/kdc --config-file=/etc/heimdal-kdc/kdc.con
root       775  0.0  0.0 250112  4616 ?        Ssl  Apr01   0:10 /usr/sbin/rsyslogd -n
kopano     851  0.0  0.0 140064  4668 ?        S    Apr01   0:00 /usr/sbin/kopano-dagent -l
listener  8559  0.1  0.0 740688  4724 ?        Ssl  Apr01   4:31 /usr/bin/loolwsd --version --o:sys_template_path=/opt/lool/systempl
root     18746  8.0  0.0  14292  4860 pts/1    Ss   10:26   0:00 -bash
root      1586  0.0  0.0 108464  4948 tty1     Ss   Apr01   0:00 /bin/login --
postgres   908  0.0  0.0 268360  5088 ?        Ss   Apr01   0:06 postgres: autovacuum launcher process   
root     12214  0.0  0.0 110260  6108 tty1     S+   Apr01   0:00 sudo -u www-data php /var/www/nextcloud/occ fulltextsearch:index
root         1  0.0  0.0 205300  6288 ?        Ss   Apr01   0:49 /sbin/init splash
postfix  17029  0.0  0.0  85492  6380 ?        S    Apr01   0:01 qmgr -l -t unix -u
postfix   9955  0.0  0.0  85316  6672 ?        S    08:52   0:00 pickup -l -t unix -u
postfix  17037  0.0  0.0  89548  6772 ?        S    Apr01   0:00 tlsmgr -l -t unix -u
root      1733  0.0  0.0 172420  8092 ?        S    Apr01   0:06 /usr/lib/heimdal-servers/kdc --config-file=/etc/heimdal-kdc/kdc.con
root      1735  0.0  0.0 172420  8096 ?        S    Apr01   0:06 /usr/lib/heimdal-servers/kdc --config-file=/etc/heimdal-kdc/kdc.con
root     30639  0.0  0.0 143964  9076 ?        S    Apr01   0:00 /usr/sbin/univention-directory-notifier -o -d 1 -v 2 -F
kopano    2417  0.0  0.0 301360  9172 ?        S    Apr01   0:00 /usr/bin/python2 /usr/sbin/kopano-search -F
root     18736  1.3  0.1 139808 10388 ?        Ss   10:26   0:00 sshd: root@pts/1
root       782  0.3  0.1 702792 12064 ?        Ssl  Apr01  10:40 /usr/sbin/nscd
kopano     770  0.0  0.1 139312 12356 ?        Ss   Apr01   0:02 /usr/sbin/kopano-ical -F
clamav     789  0.0  0.1 152528 12544 ?        Ss   Apr01   0:19 /usr/bin/freshclam -d --foreground=true
kopano    2414  0.0  0.1 299364 13100 ?        S    Apr01   0:00 /usr/bin/python2 /usr/sbin/kopano-search -F
kopano     777  0.0  0.1 339852 14136 ?        Ssl  Apr01   0:12 /usr/sbin/kopano-monitor -F
root       536  0.0  0.1  78936 14324 ?        Ss   Apr01   0:45 /lib/systemd/systemd-journald
root      2035  0.5  0.1 369732 16064 ?        Ssl  Apr01  20:12 /usr/bin/dockerd -H fd:// --bip=172.17.42.1/16 --storage-driver=ove
kopano     776  0.0  0.1 177616 16312 ?        Ssl  Apr01   0:02 /usr/sbin/kopano-dagent -l
kopano     767  0.0  0.1 151944 16844 ?        Ss   Apr01   0:02 /usr/sbin/kopano-gateway -F
bind      2052  0.0  0.1 494972 17624 ?        Ssl  Apr01   0:02 /usr/sbin/named -c /etc/bind/named.conf -p 7777 -u bind -f -d 0
kopano     785  0.0  0.1 246868 20008 ?        Ssl  Apr01   0:39 /usr/sbin/kopano-spooler -F
root      2028  0.4  0.2 4858868 20840 ?       Ssl  Apr01  15:57 /usr/sbin/slapd -h ldapi:/// ldap://:7389/ ldap://:389/ ldaps://:76
kopano    2331  0.1  0.2 330844 21024 ?        Ssl  Apr01   6:36 /usr/sbin/kopano-server -F
postgres   866  0.0  0.2 267944 21376 ?        S    Apr01   0:06 /usr/lib/postgresql/9.6/bin/postgres -D /var/lib/postgresql/9.6/mai
bind      1300  0.4  0.2 477208 25556 ?        Ssl  Apr01  15:50 /usr/sbin/named -c /etc/bind/named.conf.proxy -u bind -f -d 0
memcache  1289  1.2  0.2 336460 25864 ?        Ssl  Apr01  42:48 /usr/bin/memcached -m 64 -p 11211 -u memcache -l 127.0.0.1
root      1609  0.0  0.2 503908 27284 ?        S    Apr01   0:02 /usr/lib/heimdal-servers/kpasswdd
kopano     769  0.0  0.2 301360 28652 ?        Ssl  Apr01   2:54 /usr/bin/python2 /usr/sbin/kopano-search -F
root      1187  0.0  0.3  89060 31180 ?        Ss   Apr01   0:02 /usr/bin/python2.7 /usr/bin/univention-portal-server
www-data 18672  0.0  0.3 888976 31864 ?        S    10:23   0:00 /usr/sbin/apache2 -k start
www-data 18718  0.0  0.3 888984 32000 ?        S    10:25   0:00 /usr/sbin/apache2 -k start
listener 30638  0.0  0.3 2680076 34584 ?       S    Apr01   0:06 /usr/sbin/univention-directory-listener -F -d 2 -b dc=herre,dc=at -
root      1107  0.6  0.3 2950516 35744 ?       Sl   Apr01  22:55 /usr/bin/python2.7 /usr/sbin/univention-management-console-web-serv
root       948  0.0  0.4 698296 44536 ?        S    Apr01   1:18 /usr/bin/python2.7 /usr/sbin/univention-management-console-server s
listener  8568  0.0  0.5 1272272 52624 ?       Sl   Apr01   1:33 /usr/bin/loolforkit --losubpath=lo --systemplate=/opt/lool/systempl
www-data 18717  0.5  0.5 893508 59692 ?        S    10:25   0:00 /usr/sbin/apache2 -k start
www-data 18675  1.7  0.6 895908 65508 ?        R    10:24   0:02 /usr/sbin/apache2 -k start
www-data 18651  5.0  0.6 896096 65860 ?        S    10:20   0:20 /usr/sbin/apache2 -k start
root      4534  0.0  0.6 175196 66584 ?        S    07:13   0:00 spamd child
root      4535  0.0  0.6 175196 66760 ?        S    07:13   0:00 spamd child
www-data 18680 23.3  0.6 895844 67168 ?        S    10:24   0:40 /usr/sbin/apache2 -k start
root     16830  0.0  0.6 886120 71204 ?        Ss   Apr01   0:44 /usr/sbin/apache2 -k start
root      4508  0.1  0.7 175196 72252 ?        Ss   07:13   0:11 /usr/bin/perl -T -w /usr/sbin/spamd -d --pidfile=/var/run/spamd.pid
www-data 18681  1.1  0.7 1133820 73544 ?       S    10:24   0:01 /usr/sbin/apache2 -k start
www-data 18714  3.4  0.7 1135684 75760 ?       S    10:25   0:03 /usr/sbin/apache2 -k start
www-data 18713  3.4  0.7 1135992 76024 ?       S    10:25   0:03 /usr/sbin/apache2 -k start
listener 19274  0.0  0.7 1272588 77684 ?       Sl   Apr02   0:05 /usr/bin/loolforkit --losubpath=lo --systemplate=/opt/lool/systempl
listener 19271  0.0  0.7 1272588 77724 ?       Sl   Apr02   0:05 /usr/bin/loolforkit --losubpath=lo --systemplate=/opt/lool/systempl
listener 19272  0.0  0.7 1272588 77724 ?       Sl   Apr02   0:05 /usr/bin/loolforkit --losubpath=lo --systemplate=/opt/lool/systempl
listener 19287  0.0  0.7 1272588 77840 ?       Sl   Apr02   0:04 /usr/bin/loolforkit --losubpath=lo --systemplate=/opt/lool/systempl
www-data 18715  4.2  0.7 1174744 78108 ?       S    10:25   0:04 /usr/sbin/apache2 -k start
www-data 18658  3.6  0.7 1136296 78308 ?       S    10:20   0:13 /usr/sbin/apache2 -k start
www-data 18653 13.6  0.7 1168388 79172 ?       S    10:20   0:54 /usr/sbin/apache2 -k start
www-data  9781 40.8  0.7 1174764 81808 ?       S    08:40  43:34 /usr/sbin/apache2 -k start
www-data 18679  3.2  0.8 1174792 82136 ?       S    10:24   0:05 /usr/sbin/apache2 -k start
www-data  9563 36.1  0.8 1175052 82932 ?       R    08:31  41:36 /usr/sbin/apache2 -k start
www-data 18659  4.6  0.8 1175012 83428 ?       S    10:20   0:17 /usr/sbin/apache2 -k start
www-data 18624  4.3  0.8 1168744 83484 ?       S    10:19   0:19 /usr/sbin/apache2 -k start
www-data 14839 10.6  0.8 1174920 85872 ?       S    10:07   2:02 /usr/sbin/apache2 -k start
www-data 18654  4.2  0.8 1175020 86212 ?       S    10:20   0:16 /usr/sbin/apache2 -k start
www-data 14838  3.0  0.8 1175024 87136 ?       S    10:07   0:35 /usr/sbin/apache2 -k start
amavis    4567  0.0  1.2 273160 128516 ?       S    07:13   0:00 /usr/sbin/amavisd-new (virgin child)
amavis    4568  0.0  1.2 273160 128516 ?       S    07:13   0:00 /usr/sbin/amavisd-new (virgin child)
amavis    4519  0.0  1.2 271656 132592 ?       Ss   07:13   0:06 /usr/sbin/amavisd-new (master)
mysql     1470 22.9  2.6 790456 267716 ?       Ssl  Apr01 797:32 /usr/sbin/mysqld
elastic+ 12023  1.2  3.9 7212740 403148 ?      Ssl  Apr01  27:16 /usr/bin/java -Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:CMSInitiati
clamav     820  0.1  5.5 990884 569172 ?       Ssl  Apr01   4:59 /usr/sbin/clamd --foreground=true
www-data 12218 89.8 18.7 2974232 1916004 tty1  R+   Apr01 1884:09 php /var/www/nextcloud/occ fulltextsearch:index

root@server:~# free
              total        used        free      shared  buff/cache   available
Mem:       10243628     4172804      159168       56564     5911656     5718028
Swap:       4194300     2685800     1508500

Latest excerpts of the log files:

root@server:~# cat /var/log/univention/management-console-web-server.log
03.04.19 10:25:22.037  MAIN        ( PROCESS ) : SessionClient(0x7f69f4024f10): _authenticated: success=True  status=200  message=None
03.04.19 10:25:22.038  MAIN        ( PROCESS ) : auth_type=None
03.04.19 10:25:34.989  MAIN        ( PROCESS ) : CPCommand (24.134.101.253:41404) response status code: 511
03.04.19 10:25:34.989  MAIN        ( PROCESS ) : CPCommand (24.134.101.253:41404) response message: Connection to module process failed
03.04.19 10:25:34.989  MAIN        ( PROCESS ) : CPCommand (24.134.101.253:41404) response result: None
03.04.19 10:25:35.081  MAIN        ( PROCESS ) : CPCommand (24.134.101.253:41412) response status code: 511
03.04.19 10:25:35.081  MAIN        ( PROCESS ) : CPCommand (24.134.101.253:41412) response message: Connection to module process failed
03.04.19 10:25:35.081  MAIN        ( PROCESS ) : CPCommand (24.134.101.253:41412) response result: None
03.04.19 10:25:35.154  MAIN        ( PROCESS ) : CPCommand (24.134.101.253:41414) response status code: 511
03.04.19 10:25:35.155  MAIN        ( PROCESS ) : CPCommand (24.134.101.253:41414) response message: Connection to module process failed
03.04.19 10:25:35.155  MAIN        ( PROCESS ) : CPCommand (24.134.101.253:41414) response result: None
03.04.19 10:25:35.345  MAIN        ( PROCESS ) : CPCommand (24.134.101.253:41420) response status code: 511
03.04.19 10:25:35.346  MAIN        ( PROCESS ) : CPCommand (24.134.101.253:41420) response message: Connection to module process failed
03.04.19 10:25:35.346  MAIN        ( PROCESS ) : CPCommand (24.134.101.253:41420) response result: None

root@server:~# cat /var/log/univention/management-console-server.log
03.04.19 10:25:24.793  MAIN        ( PROCESS ) : running: ['/usr/sbin/univention-management-console-module', '-m', 'join', '-s', '/var/run/univention-management-console/948-1554279924793.socket', '-d', '2', '-l', 'de_DE.UTF-8']
03.04.19 10:25:24.889  MAIN        ( PROCESS ) : running: ['/usr/sbin/univention-management-console-module', '-m', 'appcenter', '-s', '/var/run/univention-management-console/948-1554279924888.socket', '-d', '2', '-l', 'de_DE.UTF-8']
03.04.19 10:25:24.950  MAIN        ( PROCESS ) : running: ['/usr/sbin/univention-management-console-module', '-m', 'updater', '-s', '/var/run/univention-management-console/948-1554279924949.socket', '-d', '2', '-l', 'de_DE.UTF-8']
03.04.19 10:25:25.194  MAIN        ( PROCESS ) : running: ['/usr/sbin/univention-management-console-module', '-m', 'udm', '-s', '/var/run/univention-management-console/948-1554279925193.socket', '-d', '2', '-l', 'de_DE.UTF-8']
03.04.19 10:25:34.997  MAIN        ( PROCESS ) : ModuleProcess: dying
03.04.19 10:25:34.997  MAIN        ( PROCESS ) : ModuleProcess: child stopped
03.04.19 10:25:35.087  MAIN        ( PROCESS ) : ModuleProcess: dying
03.04.19 10:25:35.087  MAIN        ( PROCESS ) : ModuleProcess: child stopped
03.04.19 10:25:35.161  MAIN        ( PROCESS ) : ModuleProcess: dying
03.04.19 10:25:35.162  MAIN        ( PROCESS ) : ModuleProcess: child stopped
03.04.19 10:25:35.343  MAIN        ( PROCESS ) : ModuleProcess: dying
03.04.19 10:25:35.344  MAIN        ( PROCESS ) : ModuleProcess: child stopped
03.04.19 10:25:40.307  MAIN        ( PROCESS ) : Connection timed out.
03.04.19 10:25:43.314  MAIN        ( PROCESS ) : Connection timed out.
03.04.19 10:25:47.182  MAIN        ( PROCESS ) : Connection timed out.
#16

Hi,

honestly, we have indicated it is for sure an issue of ressources. Somehow you doubt it telling us

And I already told you the following:

What bothers me is the high 38,9% of IO-Waits together with the very high and variable usage of the swap

Based on the output of the ps command I can see you are running Kopano, Nextcloud with Colabora and LibreOffice, PostgreSQL, MySQL and Amavis on the host.
Do you really expect this will run with just 10G of memory, especially when there are 2.6GB of Swap used? In my home server I run my UCS master with DHCP and no further Apps with 8GB of RAM!

So:
I guess you should assign at least 24GB of RAM to this machine otherwise you will have these performance issues.

As managemen-server are not running at all there could be some OutOfMemory (OOM)- check syslog for such entries.

And no, there is no fault. Match hardware to your needs. You do not want to use a Fiat 500 for a Formula 1 race…

/CV

#17

I’m running it on an old Poweredge with 4GB ram, SSD and quadcore Zeon.
Runs like a dream. I tried running it in a VM with 10GB ram and 2 cores but it’s basically unusable. I don’t see how memory is such an issue in a VM when with actual hardware its not.

#18

This is not related at all to virtualization.
I doubt you will be able to run a system with Kopano, Nextcloud with Colabora and LibreOffice, PostgreSQL, MySQL and Amavis on your mentioned 4GB physical host. At least not if you have a couple of users trying to use the services…

/CV

#19

@Christian_Voelker let me put some “question marks” to that afirmation… i’m having similiar issues in similar hardware… and running in virtual machine or in physical i have issues. But i can say that running others ucs in more recent hardware in VM i don’t have issues at all

Physical machine, ucs as slave, working ok

cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel® Xeon® CPU 5140 @ 2.33GHz
stepping : 6
microcode : 0xd2
cpu MHz : 2333.200
cache size : 4096 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant _tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vm x est tm2 ssse3 cx16 xtpr pdcm dca lahf_lm kaiser tpr_shadow dtherm
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds
bogomips : 4666.40
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel® Xeon® CPU 5140 @ 2.33GHz
stepping : 6
microcode : 0xd2
cpu MHz : 2333.200
cache size : 4096 KB
physical id : 0
siblings : 2
core id : 1
cpu cores : 2
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant _tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vm x est tm2 ssse3 cx16 xtpr pdcm dca lahf_lm kaiser tpr_shadow dtherm
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds
bogomips : 4666.40
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

free
total used free shared buff/cache available
Mem: 4049160 1573764 427884 72896 2047512 2124208
Swap: 8380412 39200 8341212

Virtual machine, ucs as slave, a lot of issues

cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel® Xeon® CPU E5450 @ 3.00GHz
stepping : 10
microcode : 0xa0b
cpu MHz : 3000.213
cache size : 6144 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush acpi mmx fxsr sse sse2 syscall nx lm constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 x2apic xsave hypervisor lahf_lm kaiser
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds
bogomips : 5999.99
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel® Xeon® CPU E5450 @ 3.00GHz
stepping : 10
microcode : 0xa0b
cpu MHz : 3000.213
cache size : 6144 KB
physical id : 2
siblings : 1
core id : 0
cpu cores : 1
apicid : 2
initial apicid : 2
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush acpi mmx fxsr sse sse2 syscall nx lm constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 x2apic xsave hypervisor lahf_lm kaiser
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds
bogomips : 6017.81
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel® Xeon® CPU E5450 @ 3.00GHz
stepping : 10
microcode : 0xa0b
cpu MHz : 3000.213
cache size : 6144 KB
physical id : 4
siblings : 1
core id : 0
cpu cores : 1
apicid : 4
initial apicid : 4
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush acpi mmx fxsr sse sse2 syscall nx lm constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 x2apic xsave hypervisor lahf_lm kaiser
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds
bogomips : 6018.74
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel® Xeon® CPU E5450 @ 3.00GHz
stepping : 10
microcode : 0xa0b
cpu MHz : 3000.213
cache size : 6144 KB
physical id : 6
siblings : 1
core id : 0
cpu cores : 1
apicid : 6
initial apicid : 6
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush acpi mmx fxsr sse sse2 syscall nx lm constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 x2apic xsave hypervisor lahf_lm kaiser
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds
bogomips : 6027.99
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
processor : 4
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel® Xeon® CPU E5450 @ 3.00GHz
stepping : 10
microcode : 0xa0b
cpu MHz : 3000.213
cache size : 6144 KB
physical id : 8
siblings : 1
core id : 0
cpu cores : 1
apicid : 8
initial apicid : 8
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush acpi mmx fxsr sse sse2 syscall nx lm constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 x2apic xsave hypervisor lahf_lm kaiser
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds
bogomips : 6024.72
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
processor : 5
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel® Xeon® CPU E5450 @ 3.00GHz
stepping : 10
microcode : 0xa0b
cpu MHz : 3000.213
cache size : 6144 KB
physical id : 10
siblings : 1
core id : 0
cpu cores : 1
apicid : 10
initial apicid : 10
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush acpi mmx fxsr sse sse2 syscall nx lm constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 x2apic xsave hypervisor lahf_lm kaiser
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds
bogomips : 6021.52
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:

free
total used free shared buff/cache available
Mem: 16414400 900592 15127196 27696 386612 15214152
Swap: 4321276 0 4321276

#20

Hi,

guys, if you are going to compare the performance of a virtualized system with the one of a physical system you should do it right!

Especially think about all the implications you assume when telling us the VMs dates like CPU or RAM.

Virtualization obviously has many advantages and therefore I would suggest ANYONE to virtualize it’s systems. We have loads of customers using VMs. My experience is in case users complain about issues with VMs it mostly a “user understanding error” based on not having enough information.

Regarding RAM/ Memory:
Assigning 10GB memory to a VM does not necessarily mean it will really have this memory really available. Purpose of virtualization is to share ressources. So you can easily do overcommitment by havin a physical virtualization host with 24GB of RAM and assigning 16GB of RAM two two VMs running on this host. You might see inside the both VMs still 2GB of free memory. But you might notice very slow performance because the host needs to use swap. Not the VMs. So you could easily shout: “VM has enough memory, it is not even using all”. But, it has not… it has enough assigned but not available.

Similar to CPU ressources. At least with VMware you might see a VM being faster with less vCPUs assigned. Example? Let’s say you have a host with 16 pCPUs. You have a single VM with 8 vCPUs and around 12 VMs with a single vCPU and the later using an average of 50% of CPU ressources. The 8 vCPU will only get pCPU time when 8 pCPUs are available at the same time. It will not get a single pCPU time slot and then later the next. However, in the end the 8 vCPU might starve on ressources and by reducing it to 4 vCPUs it is likely to get scheduled more often- so in the end it will run faster.

Having said all this in case you complain about virtual performance please consider every aspect of the environment. Let us know how many VMs are running on the virtualization host, how many vCPUs and vRAM they have assigned, how many pCPUs and pRAM is really available. Post the output of the monitoring tools of the host (esxtop in ESXi or top in UCS). Once you made sure your VM in question really has access to 100% of the assigned resources (in ESXi could be done with reservations) THEN a direct comparison to the physical host might be reliable. Otherwise it is tasseography!

Once the same prerequisites exists then monitor the system over a while. A simple snapshot usually does not give enough hints. See how swap usage changes, note an spikes in CPU usages, watch I/O waits and so on.

/CV

Mastodon