Let's encrypt issues

Hi there,

My https://domain.com is resolvable from the internet, I’m getting the Univention Web GUI but not http://domain.com/.well-known/acme-challenge/.
I’m using Cloudflare how would add this to my path?

Thank you for your quick replies by the way :slight_smile:

If https://domain.com is working, then just double check all your settings that relate to the port 443 on cloudflare and your firewall/router and make sure you also have opened/forwarded/have duplicated port 80 in all those places as well. It should work. If it doesn’t then there is something else involved in the equation that I am missing from the information included in your posts.

Hi Kevo,

Let me give you a better overview.

Cloudflare:

A record for my domain that points to my public IP address
Type: A
Name: domain.com
Content: xxx.xxx.xxx.xxx (Public IP address of my router OPNSense)

OPNSense:

NAT Port forward Rule on WAN that allows HTTP/HTTPS traffic from anywhere to my UCS server

Interface: WAN
TCP/IP Version: IPV4
Protocol: TCP/UDP
Source: ANY
Source port range: ANY
Destination: WAN Address
Destination port range: HTTP TO HTTPS
Redirect target IP: xxx.xxx.xxx.xxx (UCS internal Server IP)
Redirect target port: HTTP

Proxmox:

No Firewall rules on the VM

Like i mentioned i’m able to access the UCS WebGUI externally (tested via VPN).
However it looks like the http://domain.com/.well-known/acme-challenge/ does not work.
I’m not sure if I have to add another record to Cloudlfare or on my UCS server.
I’m using ACME on OPNSense and have no problems getting Certs.
One thing to note is that I use DNS-Challenges not HTTP-01. So it looks like something is not configured correctly for HTTP-01 challenges. If you have more insight into HTTP-01 challenges that would help.

Thank you for reaching out.

Sounds like an OPNSense config issue. Port range of http to https doesn’t make sense to me, although I don’t know that it is the issue. I would not want to blanket forward 80-443. Regardless, make sure to double check both NAT and Firewall rules in OPNSense. IIRC they can be out of sync depending on how things are setup.

Also, it’s been a while since I ran ACME on OPNSense, but I would test with it off just to make sure there is no conflict there.

The most likely issue is that something is using port 80 on the OPNSense install.

I used to run OPNSense with UCS behind it along with a second webserver. I ran haproxy on OPNSense as well. Not being able to schedule NAT rule changes so my Let’s Encrypt clients could all update their certs at different times is the only reason I moved off it. Overall I liked it quite a bit, but that one issue really got old having to manually manipulate the NAT rules to renew certs every few months.

I looked at all my NAT/WAN rules again and cannot find the issue.
No matter how I adjust the rules I can only get to the webgui via HTTP.
For some reason OPNSense won’t let me open Port 80.
I disabled all my plugins just to make sure nothing is running on Port 80 but no luck.

When you say you can only get to the webgui via http, do you mean the opnsense web gui?

If so then you might need to change that port to something else first. I think it might also be possible to insert a NAT rule higher in the rules list to redirect the port first, but it’s been a while since I’ve used it, and that might not be the case any more. I do remember being surprised that I could get my setup working the way I did, just needing to toggle one NAT rule on and off when renewing a cert. I know it took me a bit of trial and error to get it that way though.

Try this settings
OpnSense

put your WEB UI on another port like 9443 and no redirection to http

Hi Ben,

Thank you for your reply, my port is on a different port already :slight_smile:
I’m able to connect to all my services via HTTPS from outside my network but for some reason I’m not able to get port 80 to work.
I turned of Caddy off just to make sure I’m not hogging port 80 but no luck.
Would anyone know how to install the lets encrypt certificates manually?
I can just grab them from OPNSense and upload them to my UCS server right?

image

image

image

You can use the certificate fom your sense. Inside the acme client / automation you can copy the certificate with sftp to your server, but you have to use the right ssh key, he is inside /var/etc/acme-client/sftp-config/
and here is a helpful script so copy the key to your server

I didn’t realize there was a Caddy plugin for OPNSense. It wasn’t available last time I used it. I see in the docs there appears to be support for redirecting Let’s Encrypt to internal servers. Did you check the docs here?

https://docs.opnsense.org/manual/how-tos/caddy.html#id16

I only skimmed it, but it looks like you should be able to configure Caddy to do what you need.

Hi all,

So it turns out my ISP is blocking port 80 and there’s nothing I can do to open it, which is quite the bummer.
@Kevo, I already setup Caddy to forward HTTP-01 challenges I found the same guide :slight_smile:. But still without port 80 no go. Unfortunately I have to do this one manually for now until UCS adds DNS challenge support.

Would you guys know which certificates are needed?
I copied over the private.key and cert.pem from OPNSense location:
/var/etc/acme-client/keys/private.key
/var/etc/acme-client/certs/cert.pem

To UCS location:
/etc/univention/letsencrypt/cert.pem
/etc/univention/letsencrypt/private.key

And updated the config files for postfix and dovecot.

Screen Shot 2024-07-28 at 9.37.04 AM

Screen Shot 2024-07-28 at 9.37.23 AM

But I’m still getting a certificate error when setting up Thunderbird.

I just wanted to say I really appreciate the help here, I’m not well versed when it comes to certificates yet.

You should set dovecot and postfix certificates and keys in the registry, for example mail/dovecot/ssl/certificate and then reload the services.

Changing the files or the templates should be your last resort in case there is no registry option for it.

Hi @riess82,

I did that I used the ucs set command to update the dovecot and postfix config files.
But it looks like I’m importing the wrong certificates.
When trying to setup a mail account in Thunderbird I’m still getting an error that the cert is not trusted.
Looking at the cert it’s still the self signed one that UCS created.
Would you happen to know if I’m missing something?

Thank you

Hi
I don´t know Caddy, but if you read the post from Pixel and if you use the way via sftp copy from your sense, you will get the right certificate. I do this in this way and it works perfekt. The sftp copy from the automation inside the acme client will copy the right certificate to your server. Try this way :slight_smile:

Best Ben

just to make sure: have you restartet dovecot and postfix after setting the certificates?

I tried with a test account here. Thunderbird does something seemingly strange in the background. When setting up the account, right after successfully creating the account, Thunderbird connects to a domain controller, which is possibly different than the mailserver. There I get a request to manually permit a wrong certificate.

The certificates from postfix and dovecot seem okay in my test.

Hi @riess82,

Yes that seems to be the issue, it keeps pulling the self signed certificate from the domain controller itself, not the sense certs installed specifically for dovecote and postfix. Would it be possible to replace the initial certs created by univention and use them for the mail server?
Also there are a lot of different certificate types: .pem, .cert, .key.
Which ones are needed?

If someone would have more knowledge about how to manage certificates on Linux that would help.
I’m at the end of my knowledge at this point.

As a side note, I set up a brand new server with only certbot installed and installed the certificates I got from Lets Encrypt but it’s still the same problem. It keeps pulling the default self signed certificates from the UCS Domain Controller.

The bigger question for me is: what is Thunderbird requesting on the domain controller? I was not able to find out so far.

Plus, the request is made for domain, not a specific server, e.g. domain.example (without any servername). I am not even sure it is possible to request such a certificate from letsencrypt.

Hi @riess82,

Yes that seems very strange, I’m still looking for a solution to replace the Domain Controller Certificates.
Just didn’t have a lot of time the past 4 days.

Ok, I found a solution,

I set up a separate VM with certbot installed and requested new certificates for my domain.
This assumes that you have a DNS record publicly set up. I use Cloudflare for this.

On Debian/Ubuntu system use:

sudo apt install certbot

Then request the certificates for you domain (example.com << use your domain here)

sudo certbot --manual --preferred-challenges dns certonly -d example.com

Follow the instructions. Make sure to create a new TXT record in Cloudflare when prompted during the certbot request:

_acme-challenge.example.com

The new certificates can be found in the

/etc/letsencrypt/archive/example.com/cert1.pem
/etc/letsencrypt/archive/example.com/chain1.pem
/etc/letsencrypt/archive/example.com/fullchain1.pem
/etc/letsencrypt/archive/example.com/privkey1.pem

I copied the fullchain1.pem and privkey1.pem to a custom directory I created on my Univention server:

/etc/myssl/fullchain1.pem
/etc/myssl/privkey1.pem

Please note that the certificates will be renewed automatically but not copied to the Univention server. Perhaps a script can automate this process. Should I find a solution then I will update this post. If someone else has a better solution or a script please feel free to post it here :slight_smile:

With the certificates in place I followed this forum post:

Ensure you change the name of your certificate if needed.

Dovecot

Dovecot is the default IMAP server since UCS 4.0-2 and supersedes Cyrus.

The UCR variables mail/dovecot/ssl/certificate and mail/dovecot/ssl/key must be set for Dovecot:

ucr set mail/dovecot/ssl/certificate="/etc/myssl/fullchain1.pem"
ucr set mail/dovecot/ssl/key="/etc/myssl/privkey1.pem"

Then restart the daemon:
service dovecot restart

Postfix:

The UCR variables mail/postfix/ssl/certificate and mail/postfix/ssl/key need to be configured:

ucr set mail/postfix/ssl/certificate="/etc/myssl/fullchain1.pem"
ucr set mail/postfix/ssl/key="/etc/myssl/privkey1.pem"

Then the mail server has to be restarted:
service postfix restart

I want to thank everyone for your help and I hope this guide helps others which are in the same situation where http-01 challenges are not an option.
I hope Univention will add DNS challenges in the future.

Mastodon