Letsencrypt verification failed

[boospy]

Hello all,

after some recover from my Nextcloud VM, the certificate verification from Letsencrypt is failed. (Selftest)

Ungültiges Zertifikat '/etc/univention/letsencrypt/signed_chain.crt' gefunden:
error /etc/univention/letsencrypt/signed_chain.crt: verification failed

But the certificate is ok. Looks green in every browser Ok, now i reissued the cert with:
/usr/share/univention-letsencrypt/setup-letsencrypt
No error.

run-parts: executing /etc/univention/letsencrypt/setup.d//apache2
Setting apache2/ssl/certificate
Setting apache2/ssl/key
Multifile: /etc/apache2/sites-available/default-ssl.conf
run-parts: executing /etc/univention/letsencrypt/setup.d//dovecot
run-parts: executing /etc/univention/letsencrypt/setup.d//postfix
Fr 15. Jan 00:47:14 CET 2021
Refreshing certificate for following domains:
darkdevil.osit.cc
Parsing account key...
Parsing CSR...
Found domains: darkdevil.osit.cc
Getting directory...
Directory found!
Registering account...
Already registered!
Creating new order...
Order created!
Verifying darkdevil.osit.cc...
darkdevil.osit.cc verified!
Signing certificate...
Certificate signed!
Certificate refreshed at Fr 15. Jan 00:47:24 CET 2021
Setting letsencrypt/status

But if i check this cert from the selfcheck directly:

openssl verify /etc/univention/letsencrypt/signed_chain.crt

CN = darkdevil.osit.cc
error 20 at 0 depth lookup: unable to get local issuer certificate
error /etc/univention/letsencrypt/signed_chain.crt: verification failed

And i have tested this on 2 other servers, also from a customer. Same error on CMD and Webinterface. But there was an version 4.6.x and here i have 4.4-7 errata868. So is there generally a problem with Letsencrypt?

I found that too and ran some tests. But this was also ok.

Letsencrypt urlopen error [SSL: CERTIFICATE_VERIFY_FAILED]

Hey. The certificate has not been updated automatically. Attempting a manual update gave this error: /usr/share/univention-letsencrypt/setup-letsencrypt run-parts: executing /etc/univention/letsencrypt/setup.d//apache2 Setting apache2/ssl/certificatechain Setting apache2/ssl/certificate Setting apache2/ssl/key Multifile: /etc/simplesamlphp/metadata/saml20-idp-hosted.php Multifile: /etc/apache2/sites-available/default-ssl.conf run-parts: executing /etc/univention/letsencrypt/setup.d//dovecot ru…

Thanks and best Regards
boospy

[Mornsgrans]

This is the thrid thread about issues with Letsenrcyt.

See my opions here:

Let's Encrypt fails to renew / verify certificate in UCS 4.4-2 errata-291: traceback error in logfile

If I see it right, Letsencrypt has issued new root and intermediate certificates, but the Letsencrypt app of Univention still is on an old release.

[gulden]

Hello @boospy,

it looks more like https://forge.univention.org/bugzilla/show_bug.cgi?id=52517 . Can you please check what openssl s_client -connect <yourserver> says?

Best regards,
Nico

[Mornsgrans]

Hello @gulden:
I tried the workarout and it did not work for me:

Ungültiges Zertifikat '/etc/univention/letsencrypt/signed_chain.crt' gefunden:
error /etc/univention/letsencrypt/signed_chain.crt: verification failed

Ungültiges Zertifikat '/etc/univention/letsencrypt/signed_chain.crt' gefunden:
error /etc/univention/letsencrypt/signed_chain.crt: verification failed

openssl says:

openssl s_client -connect ucs.<domain>.de:443 -prexit
CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = ucs.<domain>.de
verify return:1
---
Certificate chain
 0 s:/CN=ucs.<domain>.de
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIGbzCCBVegAwIBAgISA40AgVyo26SJWZTBnpspi+CDMA0GCSqGSIb3DQEBCwUA
...
-----END CERTIFICATE-----
subject=/CN=ucs.<domain>.de
issuer=/C=US/O=Let's Encrypt/CN=R3
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 3725 bytes and written 302 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 5E43DF7EBC6A3615D7E66EBAD5C547219C365BE4E83CD997ABA6A0509CA5903D
    Session-ID-ctx:
    Master-Key: 00336EDBB...05538C284501D263
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 35 01 1d db 81 d3 2f 61-80 28 c5 f4 65 3f 09 9d   5...../a.(..e?..
...
    Start Time: 1610710575
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---
closed
---
Certificate chain
 0 s:/CN=ucs.<domain>.de
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIGbzCCBVegAwIBAgISA40AgVyo26SJWZTBnpspi+CDMA0GCSqGSIb3DQEBCwUA
...
QxuLNPKx9oyvshRHIJh8rOiChQ==
-----END CERTIFICATE-----
subject=/CN=ucs.<domain>.de
issuer=/C=US/O=Let's Encrypt/CN=R3
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 3756 bytes and written 333 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 5E43DF7EBC6A3615D7E66EBAD5C547219C365BE4E83CD997ABA6A0509CA5903D
    Session-ID-ctx:
    Master-Key: 00336EDBBEF....4501D263
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 35 01 1d db 81 d3 2f 61-80 28 c5 f4 65 3f 09 9d   5...../a.(..e?..
   ...

    Start Time: 1610710575
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---

My sites-enabled/default-ssl.conf contains these lines:

...
        SSLCertificateFile /etc/univention/letsencrypt/signed_chain.crt
        SSLCertificateKeyFile /etc/univention/letsencrypt/domain.key
        SSLCACertificateFile /etc/univention/ssl/ucsCA/CAcert.pem
        SSLCertificateChainFile /etc/univention/letsencrypt/lets-encrypt-r3-cross-signed.pem
...

/etc/univention/letsencrypt contains the following files and folders:

-rw-r-----+ 1 letsencrypt www-data 3247 Jun 8 2019 account.key
-rw-r–r-- 1 letsencrypt root 2293 Jan 1 03:45 chain.pem
-rw-r–r-- 1 letsencrypt root 1724 Jun 30 2019 domain.csr
-rw-r-----+ 1 letsencrypt root 3243 Jun 8 2019 domain.key
-rw-r–r-- 1 letsencrypt root 112 Jan 8 19:23 domains
-rw-r–r-- 1 letsencrypt www-data 1586 Jan 15 12:30 lets-encrypt-r3-cross-signed.pem
drwxr-xr-x 2 root root 4096 Jun 8 2019 post-refresh.d
-rw-r–r-- 1 letsencrypt root 11096 Jan 1 03:45 private.pem
drwxr-xr-x 2 root root 4096 Jun 8 2019 setup.d
-rw-r–r-- 1 letsencrypt www-data 3880 Jan 1 03:30 signed_chain.crt

Verification of certificate still is failing:

openssl verify /etc/univention/letsencrypt/signed_chain.crt
CN = ucs.<domain>.de
error 20 at 0 depth lookup: unable to get local issuer certificate
error /etc/univention/letsencrypt/signed_chain.crt: verification failed

[boospy]

@gulden here is the output:

CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = darkdevil.osit.cc
verify return:1
---
Certificate chain
 0 s:CN = darkdevil.osit.cc
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:O = Digital Signature Trust Co., CN = DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
blabla
-----END CERTIFICATE-----
subject=CN = darkdevil.osit.cc

issuer=C = US, O = Let's Encrypt, CN = R3

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 3673 bytes and written 435 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 53D5D149F1862F4E5ADBF3B870251DAB645B975902CD27AA58DF663C2AD55920
    Session-ID-ctx: 
    Master-Key: 5DACD1DEBB0239E1841E7B47EEC0DB7385FC45199C4C5C139C2981A8B76EEF0A7B3CA073B1B1F2A2AE8B0B03EB781FD8
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 5c a5 1e 12 6e 2f d1 9b-f0 3e 44 d4blabla

    Start Time: 1610724466
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no

@Mornsgrans works here also not. I checked the rights, rebuild the Letsencrypt Certificate and reboot the whole Server. Same error.

openssl verify /etc/univention/letsencrypt/signed_chain.crt
CN = darkdevil.osit.cc
error 20 at 0 depth lookup: unable to get local issuer certificate
error /etc/univention/letsencrypt/signed_chain.crt: verification failed

Thanks

[Mornsgrans]

I don’t know, whether I am on the wrong way:
If I edit signed_chain.crt I find two certificates in the file.
After removing the second certificate the verification with

openssl verify /etc/univention/letsencrypt/signed_chain.crt

fails.
After removig the first certificate in signed_chain.crt I get a success after openssl verify:

openssl verify /etc/univention/letsencrypt/signed_chain.crt 
/etc/univention/letsencrypt/signed_chain.crt: OK

but Apache cannot start anymore, but his may be caused by other reason on my system.

[icke]

I tried this solution. Yes, with only the second certificate in the signed_chain.crt-file the verify error succeeded. But Apache did not start anymore.

[boospy]

Hope this is fixed in 4.4-8

[Mornsgrans]

@icke

But Apache did not start anymore.

Thank you. Now I know, that this is not caused by my special configuration.

@boospy

Hope this is fixed in 4.4-8

@boopsy:
But only in the case, that the Letsencrypt-App will have been upgraded, too.

[damrose]

The UCS system diagnostic module has issues when trying to verify certificates from external CAs, so this may be a false positive.

Are any services currently restricted in the environment?

See also the following bugs, there is some information why openssl verify is not always the best tool to check cert chain validity, especially https://mail.python.org/pipermail/cryptography-dev/2016-August/000676.html

https://forge.univention.org/bugzilla/show_bug.cgi?id=52517
https://forge.univention.org/bugzilla/show_bug.cgi?id=52546

[externa1]

But the issue has arrived after errata update (don’t know which) as it is working till 4.4-7 errata850 also with the new Let’s encrypt CA ? (openssl version is the same there as on actual errata - so it must be something different not wroking anymore)

And yes the SSL Cert is still healthy - only the integrated univention test brings warnings

rg
Christian

[Mornsgrans]

Letsencrypt had issued a new CA in September and deactivates the older ones stap by step.
There are a few more changes Letsencrypt pubilshed on their homepage.

Chain of Trust - Let's Encrypt - Free SSL/TLS Certificates

I am not involved in Letsencrypt - I am a simple user - but I guess, that the Lentencrypt scrips of the Univention app will need to be adapted to the new certificate properties and chains.

[lw3234]

Same over here. After errata update to 873 I got that error message for let’s encrypt certificates.

[Mornsgrans]

See:

System diagnostic suddenly gives me: Found invalid certificate '/etc/univention/letsencrypt/signed_chain.crt'

Thanks, i just wanted to make sure. I understand and totally agree, it causes uncertainty. I will try to bump the priority in our backlog for Bug 52546 - LetsEncrypt signing chain broken - UCS System Diagnostic reports errors now Sorry if this came across wrong, i just wanted to provide more information for the technically interested readers.

[damrose]

We have released a Let’s Encrypt App update. Version 1.2.2-16 should fix the errors reported in the system diagnostic module.

[boospy]

Very thanks. It works perfectly. Good Work!

[letmesetupthis]

I get this error after the update.
the cert itself is correct, I also renewed it on my host by running “/usr/share/univention-letsencrypt/setup-letsencrypt”

I run the latest version on UCS 5: LE:2.0.0-2

[gabreitenbach]

I got the same failure with UCS 5.01.
After fresh installation of UCS and enabling letsencrypt in the AppCenter all checks are running without error. Then I installed NexCloud HUB and I got the error messages when running the system analysis and the same by running the openssl verify command.
Also I can not start Nextcloud. Browser says unsafe connection. When say trust: The browser shows : Zugriff über eine nicht vertrauenswürdige Domain
Bitte kontaktiere Deinen Administrator. Wenn Du Administrator bist, bearbeite die „trusted_domains“-Einstellung in config/config.php. Siehe Beispiel in config/config.sample.php.
When I had this with UCS 4.7 with some checks there was a hint that I have wrong settings in a metafile regarding NextCloud.
How can this be solved? I think there is a failiure in the install-script or container.yml . Isn’t it?

[boospy]

Yes the Bug is back again
5.0-7 errata1032

[pixel]

Yes, I can confirm that. 5.07 / 5.08