Zammad wont finish install

Hello togather,
i want to Install Zammad, but for some reason it doenst finishes its installation:

  Bitte diese Seite während der Installation nicht neu laden.
Press [ENTER] to continue
Creating data directories for zammad...
Registering UCR for zammad
Marking zammad=3.3.0-19 as installed
File: /etc/univention/service.info/services/univention-appcenter.cfg
Multifile: /etc/postgresql/9.6/main/pg_hba.conf
Module: kopano-cfg
File: /usr/share/univention-portal/apps.json
Setting ports for apache proxy
Multifile: /etc/apache2/sites-available/000-default.conf
Multifile: /etc/apache2/sites-available/default-ssl.conf
Module: kopano-cfg
File: /etc/apache2/sites-available/univention-letsencrypt.conf
Adding localhost to LDAP object
Setting overview variables
Module: kopano-cfg
Module: create_portal_entries
Reloading apache2 configuration (via systemctl): apache2.service.
Registering the container host zamma-02039479 for zammad
Module: kopano-cfg
Verifying Docker registry manifest for app image docker.software-univention.de/zammad-zammad-railsserver:3.3.0-19
Module: kopano-cfg
Downloading app images
Running command: docker-compose -p zammad pull
Pulling zammad-elasticsearch ... done
Pulling zammad-postgresql    ... done
Pulling zammad-init          ... done
Pulling zammad-memcached     ... done
Pulling zammad-railsserver   ... done
Pulling zammad-backup        ... done
Pulling zammad-scheduler     ... done
Pulling zammad-websocket     ... done
Pulling zammad-nginx         ... done

Initializing app image
Running command: docker-compose -p zammad up -d --no-build --no-recreate
Creating network "zammad_appcenter_net" with the default driver
Creating zammad_zammad-memcached_1     ... done
Creating zammad_zammad-elasticsearch_1 ... done
Creating zammad_zammad-postgresql_1    ... done
Creating zammad_zammad-railsserver_1   ... done
Creating zammad_zammad-init_1          ... done
Creating zammad_zammad-websocket_1     ... done
Creating zammad_zammad-scheduler_1     ... done
Creating zammad_zammad-backup_1        ... done
Creating zammad_zammad-nginx_1         ... done

Module: kopano-cfg
Preconfiguring container e0f09d8f9e13b60d33a26083e1dfcecfe2314b12887e8da6c430abcdb9ce6f81
Starting zammad-elasticsearch ... 
Starting zammad-memcached     ... 
Starting zammad-postgresql    ... 
Starting zammad-railsserver   ... 
Starting zammad-scheduler     ... 
Starting zammad-backup        ... 
Starting zammad-websocket     ... 
Starting zammad-nginx         ... 
Starting zammad-init          ... 
 tarting zammad-nginx         ...  done 
Configuring zammad=3.3.0-19
Unsetting zammad/build-version
Unsetting zammad/certificate
Unsetting zammad/certkey
Setting zammad/what-hostname to 'hdefault'
Unsetting zammad/setting-feedback
Unsetting zammad/hostname
Setting zammad/port-setting to 'pdefault'
Setting zammad/certificate-type to 'zucs'
Module: kopano-cfg
ucr cannot be found, falling back to changing the database file directly
Executing interface restore_data_before_setup for zammad
No interface defined
Executing interface restore_data_after_setup for zammad
No interface defined
Configuring zammad=3.3.0-19
Unsetting zammad/certificate
Unsetting zammad/hostname
Unsetting zammad/certkey
Setting zammad/certificate-type to 'zucs'
Setting zammad/what-hostname to 'hdefault'
Setting zammad/port-setting to 'pdefault'
 
 
#------------------------------------#
# Start of Zammad-updatescript-magic #
#------------------------------------#
 
 
Setting Status setting to empty...
Create zammad/setting-feedback
Module: kopano-cfg
UCS doesn't seem to know our build version setting yet, thus we'll set it very low.
With this we ensure we don't miss any migrations we need to do.
Create zammad/build-version
Module: kopano-cfg
Ensuring that the provided settings are valid ...
Emptying setting as it's not needed (we don't want to cause confusion)
Create zammad/certificate
Module: kopano-cfg
Emptying setting as it's not needed (we don't want to cause confusion)
Create zammad/certkey
Module: kopano-cfg
 
#--------------------------------------#
# Working on apache configurations ... #
#--------------------------------------#
 
Adding port 10412 to Firewall (allowing that port) ...
Setting security/packetfilter/package/zammad/tcp/10412/all
Setting security/packetfilter/package/zammad/tcp/10412/all/en
File: /etc/security/packetfilter.d/10_univention-firewall_start.sh
Module: kopano-cfg
File: /etc/security/packetfilter.d/80_univention-firewall_policy.sh
W: The config registry variable 'security/packetfilter/package/zammad/tcp/8443/all' does not exist
Restarting Firewall ....
Removing old redirect-configuration ...
Work-around 'Open'-Button (building redirect from /zammad to vHost) ...
Removing old zammad.conf ...
Creating my zammad.conf (sites-available) vHost-config ...
Reloading apache ...
Enabling site zammad.
To activate the new configuration, you need to run:
  systemctl reload apache2
 
#-------------------------------------#
# Working on secrets / migrations ... #
#-------------------------------------#
 
Looking for my secret files ...
  => conf-Directory found
  => Found my secrets and expect them to be fine
Looking for environment files ...
 
#-------------------------------------------------#
# Waiting for Docker, please stand by ...         #
# ... Checking Zammad-State: 000                  #
# ... Checking Zammad-State: 000                  #
# ... Checking Zammad-State: 000                  #
# ... Checking Zammad-State: 000                  #
# ... Checking Zammad-State: 000                  #
# ... Checking Zammad-State: 000                  #

But for some reason it doenst finish with chekcing zammad-state :(. I’ve already waited for some hours, but it wont finish its installation. I’ve got CPU Load at 100% too as long as zammad-docker-containers are runnig.
Does anyone have an idea to solve my issue?

It seems like it is hanging in the following lines in /usr/lib/univention-install/50zammad.inst:

# Check readiness of Zammad-Docker
ZAMMAD_HTTP_CODE="666"
while [ ! "$ZAMMAD_FRUN" ]; do
  sleep 2
  ZAMMAD_HTTP_CODE="$(curl -s -o /dev/null -w "%{http_code}" http://"$ZAMMAD_DOCKER_LOCAL")"
  echo "# ... Checking Zammad-State: $ZAMMAD_HTTP_CODE                  #"

  if [ "$ZAMMAD_HTTP_CODE" -eq "200" ]; then
    ZAMMAD_FRUN=false
  fi
done

So maybe you can check curl manually or via docker ps if the Zammad container is running and listening on port 80.

Hi Andreas_T
i’ve tried, i started install again, and sshed into it:

:~$ ssh administrator@192.168.178.100
Password: 
Univention DC Master 4.4-5:

The UCS management system is available at https://ucs(192.168.178.100)

You can log into the Univention Management Console - the principal tool to manage 
users, groups, etc. - using the "Administrator" account and the password selected 
for the root user on the master domain controller.


Last login: Fri Jul 24 12:44:29 2020 from 192.168.178.202
Administrator@ucs:~$ sudo docker ps
[sudo] Passwort für Administrator: 
CONTAINER ID        IMAGE                                                                COMMAND                  CREATED             STATUS              PORTS                           NAMES
b9c6f12416b0        docker.software-univention.de/zammad-zammad-nginx:3.3.0-19           "/docker-entrypoint.…"   49 seconds ago      Up 47 seconds       0.0.0.0:40002->80/tcp           zammad_zammad-nginx_1
df0a8a0f8674        docker.software-univention.de/zammad-zammad-websocket:3.3.0-19       "/docker-entrypoint.…"   51 seconds ago      Up 48 seconds                                       zammad_zammad-websocket_1
a730e2230be6        docker.software-univention.de/zammad-zammad-scheduler:3.3.0-19       "/docker-entrypoint.…"   51 seconds ago      Up 48 seconds                                       zammad_zammad-scheduler_1
ca58e7dcbc0d        docker.software-univention.de/zammad-zammad-backup:3.3.0-19          "/usr/local/bin/back…"   51 seconds ago      Up 48 seconds       5432/tcp                        zammad_zammad-backup_1
9e175294989a        docker.software-univention.de/zammad-zammad-railsserver:3.3.0-19     "/docker-entrypoint.…"   52 seconds ago      Up 50 seconds                                       zammad_zammad-railsserver_1
cf65b536983f        docker.software-univention.de/zammad-zammad-init:3.3.0-19            "/docker-entrypoint.…"   52 seconds ago      Up 15 seconds                                       zammad_zammad-init_1
8861aa69b1f6        docker.software-univention.de/zammad-zammad-postgresql:3.3.0-19      "docker-entrypoint.s…"   54 seconds ago      Up 51 seconds       5432/tcp                        zammad_zammad-postgresql_1
efe4c250532f        docker.software-univention.de/zammad-zammad-memcached:3.3.0-19       "docker-entrypoint.s…"   54 seconds ago      Up 52 seconds       11211/tcp                       zammad_zammad-memcached_1
1fb36847318b        docker.software-univention.de/zammad-zammad-elasticsearch:3.3.0-19   "/usr/local/bin/dock…"   54 seconds ago      Up 3 seconds        9200/tcp, 9300/tcp              zammad_zammad-elasticsearch_1
08de01253428        docker.software-univention.de/onlyoffice-ds:5.5.3.39                 "/bin/sh -c /app/ds/…"   3 weeks ago         Up 7 days           80/tcp, 0.0.0.0:5443->443/tcp   confident_kepler
4a2ebc4a6911        docker.software-univention.de/nextcloud:18.0.6-0                     "/bin/sh -c /usr/sbi…"   3 weeks ago         Up 7 days           0.0.0.0:40000->80/tcp           jolly_boyd
abb678cb3b78        docker.software-univention.de/dudle:1.2                              "/start.sh"              14 months ago       Up 7 days           0.0.0.0:40005->80/tcp           sleepy_pasteur
d63f53f78ef9        docker.software-univention.de/ucs-appbox-amd64:4.3-3                 "/sbin/init"             14 months ago       Up 7 days           0.0.0.0:40004->80/tcp           epic_morse
Administrator@ucs:~$ curl http://ucs.sys-fn.de/zammad
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://ucs.localdomain.local:10412/">here</a>.</p>
<hr>
<address>Apache/2.4.25 (Univention) Server at ucs.sys-fn.de Port 80</address>
</body></html>
Administrator@ucs:~$ curl https://ucs.sys-fn.de/zammad
curl: (51) SSL: no alternative certificate subject name matches target host name 'ucs.sys-fn.de'
Administrator@ucs:~$ curl https://ucs.localdomain.local:10412/
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>502 Proxy Error</title>
</head><body>
<h1>Proxy Error</h1>
<p>The proxy server received an invalid
response from an upstream server.<br />
The proxy server could not handle the request<p>Reason: <strong>Error reading from remote server</strong></p></p>
<hr>
<address>Apache/2.4.25 (Univention) Server at ucs.localdomain.local Port 10412</address>
</body></html>
Administrator@ucs:~$ curl https://ucs.localdomain.local:10412/zammad
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>502 Proxy Error</title>
</head><body>
<h1>Proxy Error</h1>
<p>The proxy server received an invalid
response from an upstream server.<br />
The proxy server could not handle the request<p>Reason: <strong>Error reading from remote server</strong></p></p>
<hr>
<address>Apache/2.4.25 (Univention) Server at ucs.localdomain.local Port 10412</address>
</body></html>
Administrator@ucs:~$ 

For some reason after abborting that zammad-installation again lets encrypt didnt work anymore

The curl check should use curl http://127.0.0.1:40002 (actually 127.0.0.1: followed by the port obtained via ucr get appcenter/apps/zammad/ports/80, please check if it is 40002). Maybe have a look at univention-app logs zammad.

Hi Andreas_T

root@ucs:/home/Administrator#  univention-app logs zammad
#### 'docker logs 0635ac95198d3fb0699eb72b30f3b23b30d4e89ffeebb10d442885cae8350243' output:
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...

and:

root@ucs:/home/Administrator# ucr get appcenter/apps/zammad/ports/80
40002
root@ucs:/home/Administrator# curl http://127.0.0.1:40002
curl: (52) Empty reply from server
root@ucs:/home/Administrator# curl http://127.0.0.1:40002
curl: (56) Recv failure: Die Verbindung wurde vom Kommunikationspartner zurückgesetzt


thanks in advance

Hello,

I observed a similar behavior here. Zammad uses several docker containers and the elastic search container is caught in a traceback loop. Here is what I did:

$ docker ps
docker ps                                                                                                                                                                                              |
CONTAINER ID        IMAGE                                                                COMMAND                  CREATED             STATUS              PORTS                   NAMES
60c186aaf929        docker.software-univention.de/zammad-zammad-nginx:3.3.0-19           "/docker-entrypoint.…"   22 minutes ago      Up 22 minutes       0.0.0.0:40001->80/tcp   zammad_zammad-nginx_1
98096e3ac9f9        docker.software-univention.de/zammad-zammad-scheduler:3.3.0-19       "/docker-entrypoint.…"   22 minutes ago      Up 22 minutes                               zammad_zammad-scheduler_1
9c19b1f3ae40        docker.software-univention.de/zammad-zammad-websocket:3.3.0-19       "/docker-entrypoint.…"   22 minutes ago      Up 22 minutes                               zammad_zammad-websocket_1
9a75f0526a3c        docker.software-univention.de/zammad-zammad-backup:3.3.0-19          "/usr/local/bin/back…"   22 minutes ago      Up 22 minutes       5432/tcp                zammad_zammad-backup_1
712e449e73ce        docker.software-univention.de/zammad-zammad-railsserver:3.3.0-19     "/docker-entrypoint.…"   22 minutes ago      Up 22 minutes                               zammad_zammad-railsserver_1
bac5122f2d44        docker.software-univention.de/zammad-zammad-init:3.3.0-19            "/docker-entrypoint.…"   22 minutes ago      Up 22 minutes                               zammad_zammad-init_1
bbb5d8a4310a        docker.software-univention.de/zammad-zammad-elasticsearch:3.3.0-19   "/usr/local/bin/dock…"   22 minutes ago      Up 5 seconds        9200/tcp, 9300/tcp      zammad_zammad-elasticsearch_1
583ec653af7b        docker.software-univention.de/zammad-zammad-memcached:3.3.0-19       "docker-entrypoint.s…"   22 minutes ago      Up 22 minutes       11211/tcp               zammad_zammad-memcached_1
fb20c0b2c39b        docker.software-univention.de/zammad-zammad-postgresql:3.3.0-19      "docker-entrypoint.s…"   22 minutes ago      Up 22 minutes       5432/tcp                zammad_zammad-postgresql_1

$ docker logs bbb5d8a4310a

{"type": "server", "timestamp": "2020-07-29T09:40:23,767Z", "level": "ERROR", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "docker-cluster", "node.name": "bbb5d8a4310a", "message":
 "uncaught exception in thread [main]",
"stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[Failure running machine learning native code. This could be due to running on an unsupported OS or distribution, missing OS li
braries, or a problem with the temp directory. To bypass this problem by running Elasticsearch without machine learning functionality set [xpack.ml.enabled: false].]",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:174) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.6.1.jar:7.6.1]",
"at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.6.1.jar:7.6.1]",
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.6.1.jar:7.6.1]",
"Caused by: org.elasticsearch.ElasticsearchException: Failure running machine learning native code. This could be due to running on an unsupported OS or distribution, missing OS libraries, or a problem with the
temp directory. To bypass this problem by running Elasticsearch without machine learning functionality set [xpack.ml.enabled: false].",
"at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:587) ~[?:?]",
"at org.elasticsearch.node.Node.lambda$new$9(Node.java:456) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]",
"at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1621) ~[?:?]",
"at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]",
"at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]",
"at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]",
"at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]",
"at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]",
"at org.elasticsearch.node.Node.<init>(Node.java:459) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.node.Node.<init>(Node.java:257) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.6.1.jar:7.6.1]",
"... 6 more"] }

How I came to it?

  1. I looked into /var/log/univention/appcenter.log. There I find lines like which point me to the app’s configure_host script which is run in the context of the UCS host. The configure script can be found at /var/cache/univention-appcenter/appcenter.software-univention.de/4.4/zammad_20190625122036.configure_host:
 30036 actions.configure                20-07-29 11:48:38 [    INFO]: #-------------------------------------#
 30036 actions.configure                20-07-29 11:48:38 [    INFO]: # Working on secrets / migrations ... #
 30036 actions.configure                20-07-29 11:48:38 [    INFO]: #-------------------------------------#
 30036 actions.configure                20-07-29 11:48:38 [    INFO]:  
 30036 actions.configure                20-07-29 11:48:38 [    INFO]: Looking for my secret files ...
 30036 actions.configure                20-07-29 11:48:38 [    INFO]:   => conf-Directory found
 30036 actions.configure                20-07-29 11:48:38 [    INFO]:   => Found my secrets and expect them to be fine
 30036 actions.configure                20-07-29 11:48:38 [    INFO]: Looking for environment files ...
 30036 actions.configure                20-07-29 11:48:38 [    INFO]:  
 30036 actions.configure                20-07-29 11:48:38 [    INFO]: #-------------------------------------------------#
 30036 actions.configure                20-07-29 11:48:38 [    INFO]: # Waiting for Docker, please stand by ...         #
 30036 actions.configure                20-07-29 11:48:40 [    INFO]: # ... Checking Zammad-State: 000                  #
 30036 actions.configure                20-07-29 11:48:42 [    INFO]: # ... Checking Zammad-State: 000                  #
  1. The configure_host script runs a curl command to check for the availability of the exposed zammad nginx container.
  2. Look into the logs of the zammad_zammad-nginx_1 container. There I can read that this container waits on the zammad init container.
  3. Look into the logs of the zammad_zammad-init_1 container. It waits on the elastic search container.
  4. Look at the elastic search container logs.

I don’t know the meaning of the elastic search error message. The effect though is that the Zammad app won’t start and is trapped in a loop that checks for the app.

@Tw33ki: Can you confirm that you have the same behavior?

Best regards,
Nico

Hello,

as recommended by the error message in ElasticSearch, I deactiaved the machine learning.

$ EL_ID=$(docker ps | grep elasticsearch | cut -f 1 -d ' ')
$ docker exec -it $EL_ID /bin/bash
$ [root@bbb5d8a4310a elasticsearch]$ echo "xpack.ml.enabled: false" >> config/elasticsearch.yml
$ [root@bbb5d8a4310a elasticsearch]$ exit

Wait until ElasticSearch takes the settings into account. Then the Zammad init container will finish and afterwards the Zammad nginx container continues which will let the configure_host script continue and finish the whole app installation process, including the app’s join script.

I don’t know though what the effect of a deactivated machine learning API for Zammad is. Here is the documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-settings.html

I hope this helps for the moment.

Best regards,
Nico

Hello,

just one follow up. The above settings won’t survive a container change. The setting can also be added to the environment variable for the ElasticSearch container.

  • Value to add: xpack.ml.enabled=false
  • Environment file: /var/lib/univention-appcenter/apps/zammad/conf/es.env
  • Command for single change: echo "xpack.ml.enabled=false" >> /var/lib/univention-appcenter/apps/zammad/conf/es.env

For environment variables to take effect on Docker containers, they need to be recreated. This can be done with univention-app reinitialize zammad.

I have not tested it!

Best regards,
Nico