Kopano broken after upgrade to 8.6.2.1

tpfann · June 25, 2018, 1:35pm

Hi all,

after upgrading to Kopano Core 8.6.2.1 today, I’m no longer able to connect to Kopano Core (it seems its not even running).

systemctl status kopano-server.service gives me:

● kopano-server.service - Kopano Core Storage Server
   Loaded: loaded (/lib/systemd/system/kopano-server.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Mon 2018-06-25 15:14:16 CEST; 13min ago
     Docs: man:kopano-server(8)
           man:kopano-server.cfg(5)
           man:kopano-admin(8)
  Process: 1822 ExecStart=/usr/sbin/kopano-server -F (code=exited, status=255)
 Main PID: 1822 (code=exited, status=255)
      CPU: 43ms

systemctl list-units | grep kopano gives me:

  kopano-dagent.service     	loaded active running   Kopano Core Delivery Agent         
  kopano-gateway.service        loaded active running   Kopano Core IMAP/POP3 Gateway      
  kopano-ical.service           loaded active running   Kopano Core iCal/CalDAV Gateway    
  kopano-monitor.service        loaded active running   Kopano Core Quota Manager          
  kopano-presence.service       loaded active running   Kopano Core Presence Daemon        
  kopano-search.service         loaded active running   Kopano Core Search Engine          
● kopano-server.service         loaded **failed failed**    Kopano Core Storage Server         
  kopano-spooler.service        loaded active running   Kopano Core Spooler                
  kopano-webmeetings.service    loaded active running   LSB: Kopano Webmeetings server

server.log:

Mon Jun 25 15:14:13 2018: [=======] Starting kopano-server version 8.6.2 (pid 1822)
Mon Jun 25 15:14:13 2018: [warning] Config warning: Option 'server_max_keep_alive_requests' is not used anymore.
Mon Jun 25 15:14:13 2018: [error  ] Coredumps will not be generated: kopano-server requires the fs.suid_dumpable sysctl to contain the value 2, not 0. See kopano-coredump(5) for details.
Mon Jun 25 15:14:16 2018: [error  ] SQL [00000003] Failed: Duplicate entry 'x-no-relay' for key 'gns', Query Size: 170, Query: "ALTER TABLE `names` ADD UNIQUE INDEX `gni` (`guid`(16), `nameid`), ADD UNIQUE INDEX `gns` (`guid`(16), `namestring`), DROP INDEX `guidnameid`, DROP INDEX `guidnamestring`"
Mon Jun 25 15:14:16 2018: [error  ] KDatabase::I_Update() query failed: "Duplicate entry 'x-no-relay' for key 'gns'", query: ALTER TABLE `names` ADD UNIQUE INDEX `gni` (`guid`(16), `nameid`), ADD UNIQUE INDEX `gns` (`guid`(16), `namestring`), DROP INDEX `guidnameid`, DROP INDEX `guidnamestring`
Mon Jun 25 15:14:16 2018: [error  ] K-1216: Cannot update to schema v69, because the "names" table contains unexpected rows. Certain prior versions of the server erroneously allowed these duplicates to be added (KC-1108).
Mon Jun 25 15:14:16 2018: [error  ] K-1220: To fix the excess rows, use `kopano-dbadm k-1216`. Consult the manpage and preferably make a backup first.
Mon Jun 25 15:14:16 2018: [error  ] K-1221: Alternatively, the server may be started with --ignore-da to forego the schema update.
Mon Jun 25 15:14:16 2018: [warning] WARNING: Unable to upgrade database from version 8.6.2.0.68 to 8.6.2.0.70
Mon Jun 25 15:14:16 2018: [warning]    You can force the server to start with --ignore-database-version-conflict
Mon Jun 25 15:14:16 2018: [warning]    Warning, you can lose data! If you don't know what you're doing, you shouldn't be using this option!
Mon Jun 25 15:14:16 2018: [=======] Server shutdown complete.

Any quick help would be much appreciated!

Thx and best regards
Thomas

bytemine · June 25, 2018, 1:56pm

Hi,

reading https://kopano.com/releases/kopanocore-8-5-7/ should answer your questions.

you need to fix the database, if you read your error log carefully you see:

Mon Jun 25 15:14:16 2018: [error  ] K-1220: To fix the excess rows, use `kopano-dbadm k-1216`. Consult the manpage and preferably make a backup first.

everything about that is explained in the link i have given above

tpfann · June 25, 2018, 2:27pm

Hi bytemine,

thx alot for your quick reply.
Running kopano-dbadm k-1216 indeed fixed it.

I’m just wondering if something went wrong on my system during the upgrade today, or if this is an expected behavior and users must find out the soltion themself?

Best regards
Thomas

bytemine · June 25, 2018, 3:06pm

Hi,

no this expected behaviour and nothing went wrong on your system.

“kopano-dbadm k-1216” can run many many hours on big systems and you are given the chance to start your server anyways with the option “–ignore-database-version-conflict”.

this way you can choose when to do the database fix and schedule it to a time where your users are not bothered by the downtime.

it also gives you the chance to do a database backup first before doing the update.

fbartels · June 25, 2018, 3:23pm

thanks for brining this up. we have now also added a small note to the readme_update to point admins towards this.

tpfann · June 25, 2018, 3:29pm

Hi fbartels,

… this makes perfect sense

@bytemine: thx for your detailed feedback - anyway … without any hint, I was not able to foresee this

Moritz_Bunkus · June 26, 2018, 7:06am

To be honest, Kopano has been very clear and vocal about this update. They’ve written extensive news posts about this, they’ve posted instructions in their wiki, they’ve spread the news via their partner network (who in turn spread it to paying customers). It’s perfectly fine if you’re only using the Open Source version, but in that case you should really try to follow the usual news channels for those Open Source products in order to stay up to date regarding important developments.

tpfann · June 26, 2018, 7:42am

Do you have the impression that my above posting was a complaint? It’s not.
I’m sorry that I’m a user of the Open Source version only and all these updates and postings you have mentioned didn’t reached me.

To make it clear, in generall we are absolutly on the same page and I’m very glad for the quick support I got here. But it would be really nice if you could give a short hint also within the UCS update center about such mandatory steps, that’s all. I hope you agree on that.

Best regards
Thomas

my_ucs_2017 · August 30, 2018, 10:06am

Sorry for the stupid question - but where did you run the “kopano-dbadm k-1216”. Just login as root an run or someone in the docker?

Thanks

F.

Moritz_Bunkus · August 30, 2018, 11:13am

Kopano does not run in a Docker container. You simply log in via ssh on the server Kopano runs on and run that command as root.