Letsencrypt verification failed

boospy · January 14, 2021, 11:52pm

Hello all,

after some recover from my Nextcloud VM, the certificate verification from Letsencrypt is failed. (Selftest)

Ungültiges Zertifikat '/etc/univention/letsencrypt/signed_chain.crt' gefunden:
error /etc/univention/letsencrypt/signed_chain.crt: verification failed

But the certificate is ok. Looks green in every browser Ok, now i reissued the cert with:
/usr/share/univention-letsencrypt/setup-letsencrypt
No error.

run-parts: executing /etc/univention/letsencrypt/setup.d//apache2
Setting apache2/ssl/certificate
Setting apache2/ssl/key
Multifile: /etc/apache2/sites-available/default-ssl.conf
run-parts: executing /etc/univention/letsencrypt/setup.d//dovecot
run-parts: executing /etc/univention/letsencrypt/setup.d//postfix
Fr 15. Jan 00:47:14 CET 2021
Refreshing certificate for following domains:
darkdevil.osit.cc
Parsing account key...
Parsing CSR...
Found domains: darkdevil.osit.cc
Getting directory...
Directory found!
Registering account...
Already registered!
Creating new order...
Order created!
Verifying darkdevil.osit.cc...
darkdevil.osit.cc verified!
Signing certificate...
Certificate signed!
Certificate refreshed at Fr 15. Jan 00:47:24 CET 2021
Setting letsencrypt/status

But if i check this cert from the selfcheck directly:

openssl verify /etc/univention/letsencrypt/signed_chain.crt

CN = darkdevil.osit.cc
error 20 at 0 depth lookup: unable to get local issuer certificate
error /etc/univention/letsencrypt/signed_chain.crt: verification failed

And i have tested this on 2 other servers, also from a customer. Same error on CMD and Webinterface. But there was an version 4.6.x and here i have 4.4-7 errata868. So is there generally a problem with Letsencrypt?

I found that too and ran some tests. But this was also ok.

Thanks and best Regards
boospy

Mornsgrans · January 15, 2021, 7:47am

This is the thrid thread about issues with Letsenrcyt.

See my opions here:

gulden · January 15, 2021, 9:36am

Hello @boospy,

it looks more like https://forge.univention.org/bugzilla/show_bug.cgi?id=52517 . Can you please check what openssl s_client -connect <yourserver> says?

Best regards,
Nico

Mornsgrans · January 15, 2021, 11:48am

Hello @gulden:
I tried the workarout and it did not work for me:

Ungültiges Zertifikat '/etc/univention/letsencrypt/signed_chain.crt' gefunden:
error /etc/univention/letsencrypt/signed_chain.crt: verification failed

Ungültiges Zertifikat '/etc/univention/letsencrypt/signed_chain.crt' gefunden:
error /etc/univention/letsencrypt/signed_chain.crt: verification failed

openssl says:

openssl s_client -connect ucs.<domain>.de:443 -prexit
CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = ucs.<domain>.de
verify return:1
---
Certificate chain
 0 s:/CN=ucs.<domain>.de
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIGbzCCBVegAwIBAgISA40AgVyo26SJWZTBnpspi+CDMA0GCSqGSIb3DQEBCwUA
...
-----END CERTIFICATE-----
subject=/CN=ucs.<domain>.de
issuer=/C=US/O=Let's Encrypt/CN=R3
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 3725 bytes and written 302 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 5E43DF7EBC6A3615D7E66EBAD5C547219C365BE4E83CD997ABA6A0509CA5903D
    Session-ID-ctx:
    Master-Key: 00336EDBB...05538C284501D263
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 35 01 1d db 81 d3 2f 61-80 28 c5 f4 65 3f 09 9d   5...../a.(..e?..
...
    Start Time: 1610710575
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---
closed
---
Certificate chain
 0 s:/CN=ucs.<domain>.de
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIGbzCCBVegAwIBAgISA40AgVyo26SJWZTBnpspi+CDMA0GCSqGSIb3DQEBCwUA
...
QxuLNPKx9oyvshRHIJh8rOiChQ==
-----END CERTIFICATE-----
subject=/CN=ucs.<domain>.de
issuer=/C=US/O=Let's Encrypt/CN=R3
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 3756 bytes and written 333 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 5E43DF7EBC6A3615D7E66EBAD5C547219C365BE4E83CD997ABA6A0509CA5903D
    Session-ID-ctx:
    Master-Key: 00336EDBBEF....4501D263
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 35 01 1d db 81 d3 2f 61-80 28 c5 f4 65 3f 09 9d   5...../a.(..e?..
   ...

    Start Time: 1610710575
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---

My sites-enabled/default-ssl.conf contains these lines:

...
        SSLCertificateFile /etc/univention/letsencrypt/signed_chain.crt
        SSLCertificateKeyFile /etc/univention/letsencrypt/domain.key
        SSLCACertificateFile /etc/univention/ssl/ucsCA/CAcert.pem
        SSLCertificateChainFile /etc/univention/letsencrypt/lets-encrypt-r3-cross-signed.pem
...

/etc/univention/letsencrypt contains the following files and folders:

-rw-r-----+ 1 letsencrypt www-data 3247 Jun 8 2019 account.key
-rw-r–r-- 1 letsencrypt root 2293 Jan 1 03:45 chain.pem
-rw-r–r-- 1 letsencrypt root 1724 Jun 30 2019 domain.csr
-rw-r-----+ 1 letsencrypt root 3243 Jun 8 2019 domain.key
-rw-r–r-- 1 letsencrypt root 112 Jan 8 19:23 domains
-rw-r–r-- 1 letsencrypt www-data 1586 Jan 15 12:30 lets-encrypt-r3-cross-signed.pem
drwxr-xr-x 2 root root 4096 Jun 8 2019 post-refresh.d
-rw-r–r-- 1 letsencrypt root 11096 Jan 1 03:45 private.pem
drwxr-xr-x 2 root root 4096 Jun 8 2019 setup.d
-rw-r–r-- 1 letsencrypt www-data 3880 Jan 1 03:30 signed_chain.crt

Verification of certificate still is failing:

openssl verify /etc/univention/letsencrypt/signed_chain.crt
CN = ucs.<domain>.de
error 20 at 0 depth lookup: unable to get local issuer certificate
error /etc/univention/letsencrypt/signed_chain.crt: verification failed

boospy · January 15, 2021, 3:32pm

@gulden here is the output:

CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = darkdevil.osit.cc
verify return:1
---
Certificate chain
 0 s:CN = darkdevil.osit.cc
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:O = Digital Signature Trust Co., CN = DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
blabla
-----END CERTIFICATE-----
subject=CN = darkdevil.osit.cc

issuer=C = US, O = Let's Encrypt, CN = R3

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 3673 bytes and written 435 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 53D5D149F1862F4E5ADBF3B870251DAB645B975902CD27AA58DF663C2AD55920
    Session-ID-ctx: 
    Master-Key: 5DACD1DEBB0239E1841E7B47EEC0DB7385FC45199C4C5C139C2981A8B76EEF0A7B3CA073B1B1F2A2AE8B0B03EB781FD8
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 5c a5 1e 12 6e 2f d1 9b-f0 3e 44 d4blabla

    Start Time: 1610724466
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no

@Mornsgrans works here also not. I checked the rights, rebuild the Letsencrypt Certificate and reboot the whole Server. Same error.

openssl verify /etc/univention/letsencrypt/signed_chain.crt
CN = darkdevil.osit.cc
error 20 at 0 depth lookup: unable to get local issuer certificate
error /etc/univention/letsencrypt/signed_chain.crt: verification failed

Thanks

Mornsgrans · January 15, 2021, 4:25pm

I don’t know, whether I am on the wrong way:
If I edit signed_chain.crt I find two certificates in the file.
After removing the second certificate the verification with

openssl verify /etc/univention/letsencrypt/signed_chain.crt

fails.
After removig the first certificate in signed_chain.crt I get a success after openssl verify:

openssl verify /etc/univention/letsencrypt/signed_chain.crt 
/etc/univention/letsencrypt/signed_chain.crt: OK

but Apache cannot start anymore, but his may be caused by other reason on my system.

icke · January 19, 2021, 1:32pm

I tried this solution. Yes, with only the second certificate in the signed_chain.crt-file the verify error succeeded. But Apache did not start anymore.

boospy · January 19, 2021, 4:24pm

Hope this is fixed in 4.4-8

Mornsgrans · January 19, 2021, 4:35pm

Thank you. Now I know, that this is not caused by my special configuration.

@boopsy:
But only in the case, that the Letsencrypt-App will have been upgraded, too.

damrose · January 21, 2021, 4:29pm

The UCS system diagnostic module has issues when trying to verify certificates from external CAs, so this may be a false positive.

Are any services currently restricted in the environment?

See also the following bugs, there is some information why openssl verify is not always the best tool to check cert chain validity, especially https://mail.python.org/pipermail/cryptography-dev/2016-August/000676.html

https://forge.univention.org/bugzilla/show_bug.cgi?id=52517
https://forge.univention.org/bugzilla/show_bug.cgi?id=52546

externa1 · January 21, 2021, 6:27pm

But the issue has arrived after errata update (don’t know which) as it is working till 4.4-7 errata850 also with the new Let’s encrypt CA ? (openssl version is the same there as on actual errata - so it must be something different not wroking anymore)

And yes the SSL Cert is still healthy - only the integrated univention test brings warnings

rg
Christian

Mornsgrans · January 21, 2021, 8:17pm

Letsencrypt had issued a new CA in September and deactivates the older ones stap by step.
There are a few more changes Letsencrypt pubilshed on their homepage.

I am not involved in Letsencrypt - I am a simple user - but I guess, that the Lentencrypt scrips of the Univention app will need to be adapted to the new certificate properties and chains.

lw3234 · January 23, 2021, 3:37pm

Same over here. After errata update to 873 I got that error message for let’s encrypt certificates.

Mornsgrans · January 23, 2021, 4:23pm

See:

damrose · January 25, 2021, 1:08pm

We have released a Let’s Encrypt App update. Version 1.2.2-16 should fix the errors reported in the system diagnostic module.

boospy · January 25, 2021, 3:58pm

Very thanks. It works perfectly. Good Work!

letmesetupthis · November 3, 2021, 2:04pm

I get this error after the update.
the cert itself is correct, I also renewed it on my host by running “/usr/share/univention-letsencrypt/setup-letsencrypt”

I run the latest version on UCS 5: LE:2.0.0-2

gabreitenbach · February 9, 2022, 11:44am

I got the same failure with UCS 5.01.
After fresh installation of UCS and enabling letsencrypt in the AppCenter all checks are running without error. Then I installed NexCloud HUB and I got the error messages when running the system analysis and the same by running the openssl verify command.
Also I can not start Nextcloud. Browser says unsafe connection. When say trust: The browser shows : Zugriff über eine nicht vertrauenswürdige Domain
Bitte kontaktiere Deinen Administrator. Wenn Du Administrator bist, bearbeite die „trusted_domains“-Einstellung in config/config.php. Siehe Beispiel in config/config.sample.php.
When I had this with UCS 4.7 with some checks there was a hint that I have wrong settings in a metafile regarding NextCloud.
How can this be solved? I think there is a failiure in the install-script or container.yml . Isn’t it?