After update Apache randomly stops responding

UCS - Univention Corporate Server
#1

Hi all,

after upgrading the server of a client from UCS 4.4-5 errata 686 to 4.4-6 errata 786 (including updates of Nextcloud and Kopano webapp) I’m experiencing problems with the Apache Webserver. I’ve been searching and trying for two days now, but I can’t find a reasonable cause.

The problem is, that Apache once in a while suddenly stops responding to request. That can be after 5 minutes, but also after 3 hours. When this happens, I can’t connect to the UMC, the webapps (Kopano and Nextcloud) stop working, and ActivSync (Outlook and mobile devices) doesn’t get a connection to the server.

I’ve been monitoring the behaviour of Apache with apachetop and Apache’s error.log (loglevel debug) and other system tools like top. But I can’t find any cause, that would explain the “freeze” of the Apache service. The load of CPUs and harddisks is under 10%, and there are several GBs of free memory. In Apache’s error.log and in all the other logfiles I’ve been walking through, no serious error is reported. I also tried some changes in Apache configuration, like ressources and so on, but that did’t help at all.

The only “solution” for the problem I’ve found so far, is to watch the activity of Apache with apachetop the whole day, and if it suddenly drops to zero, manually restart the Apache service (otherwise I will be reminded to do so by the ringing of my phone after a couple of minutes…)

By now I have no more ideas, what I can do to fix the problem, except updating Apache itself from official UCS version 2.4.25 to a newer one. But to be honest, I’m a little afraid to do that. It might result in the primary problem remaining, and due to compatibility issues additional problems occurring.

Where can I look at or what can I try to do, to find and fix the cause of the problem, before I get completely mad?!?Thanks in advance for every hint!

Cheers, Thorger

#2

Does apachectl status or fullstatus show anything interesting?

#3

Thanks for the hint, I wasn’t aware of apachectl before, nice tool :slight_smile:

I’ve been watching the Apache server now for almost 24 hrs, and it seems the problem has solved itself… The Apache service is running without any problems. I have no idea, what was wrong the two days before:woozy_face:

If the error shows up again, I’ll have a closer look with apachectl and post the result here!

Cheers, Thorger

#4

Hi there,

i’ve been watching my problem with Apache server for some days now. What I can say:

  1. The problem occurs only during the working hours, when there is a lot of ActivSync (Z-Push) traffic to the server. At night or at the weekend, I did’nt observe the problem.

  2. When the Apache server is hanging, it has not completely crashed. First it doesn’t even respond to the apachectl status command (I think it is a https request as well?) After a while (some minutes), it starts responding again, the status request is executed, and page refresh in a browser works again.

  3. The result of the status request shows, that all Apache workers a busy with “Sending Reply”, and all workers are blocked by ActiveSync (Z-Push) tasks.

image

It seems to me , as if Z-Push is trying to send data to connected devices, but is - for whatever reason - not able to finish the transfer. So all channels are blocked, until Z-Push has finished its job(s)?

I think I will try to raise the number of allowed parallel tasks for the Apache server and / or to limit the Z-Push connections in the next step.

Cheers, Thorger

Mastodon