@kopanogobigred #ucs Kopano - Require client-side certificates

I need to configure client certificate requirements on my email sync

I am seeing a lot of brute force password attempts against the UCS Kopano mail server’s ActiveSync.

I was seeing it on the mail portal before, but added a google authenticator inside my reverse proxy for the URLs.

OTP is not an option on ActiveSync.

My clients are Microsoft Outlook, Windows 10 Mail, iPhone, and Android.

I am looking for setup/configuration instructions for client certificate setup.

I am currently also looking at forcing users through a per-app OpenVPN but that solution is chewing up a ton of battery on my test device and it is not very use friendly.

Hi @kopanogobigred,

there are no z-push specific requirements to ssl client cert auth, as all the auth is carried on by the webserver itself. So you only have to follow instructions on setting up client certs either for the apache webserver that UCS uses, or for your reverse proxy.

Here’s what I’ve figured out on this

IMPORTANT! Watch deployed outlook versions.

  • Outlook 2013 and Outlook 2016 work with certificate based authentication (client certs HTTPS) just fine.
  • Outlook 2019’s TLS client stack does NOT include traditional CBA (certificate based authentication) support.

If you setup the server or reverse proxy for the mail with SSLVerifyClient it breaks Outlook’s ability to connect via Exchange Active Sync (EAS) even if it is only set to “optional”

In my environment I setup my reverse proxy apache host with the following defaults:
SSLVerifyClient none
SSLCACertificateFile /my/ca/path/CA.PEM
SSLCACertificatePath /my/ca/path

Client certificates are mostly broken in numerous TLS 1.3 stacks. TLS 1.2 is really what you’ll need to use.
SSLProtocol all -SSLv2 -SSLv3 -TLSv1.3 +TLSv1.2

Make the certificates optional to test stuff
<Location /Microsoft-Server-ActiveSync>
SSLVerifyClient optional
SSLVerifyDepth 5
SSLRequireSSL
RewriteEngine On
RewriteCond %{SSL:SSL_CLIENT_VERIFY} !^SUCCESS$
RewriteRule .* /help/ssl-client-auth-required.html [L]

Continue to make it options but with an error message rewrite after testing. This is what you may want for production so clients like Outlook will get an error message
<Location /Microsoft-Server-ActiveSync>
SSLVerifyClient optional
SSLVerifyDepth 5
SSLOptions +FakeBasicAuth
SSLRequireSSL
SSLOptions +StdEnvVars
RewriteEngine on
RewriteCond %{SSL:SSL_CLIENT_VERIFY} !=SUCCESS
RewriteRule .? - [F]
ErrorDocument 403 “You need a client side certificate issued by CAcert to access this site”

Provided the personal certificate is from a trusted CA on the machine and has a private key and matches the email address of the user this works in standalone (no AD join, etc) on a PC with Outlook 2013 & Outlook 2016.

It functions perfectly with IOS and Android devices, but IOS devices MUST be configured using Apple Configurator 2 profiles that specify a client CA certificate for Exchange Active Sync.

Certificate Based Authentication does NOT function on Outlook 2019. Not at all. Even disabling the rewrite and leaving the certs options NOT do any good.

The best work-around for Outlook 2019 is to setup a local loopback STUNNEL service.

Stunnel to EAS – (“proxy-out” below) setup this part to use the personal certificate. Give it the destination and port of the actual EAS server. Pick a port on the local machine that is firewalled and restricted to 127.0.0.1. It does not need to be accessible from the network at large

Client to Stunnel – (“https” below) Generate server cert for STUNNEL that is an unused name. For sanity perhaps one similar to the mail server. If the server is “server.mydomain.com” a name like “server2.mydomain.com” will do find. Put an entry on the machine’s host file or in DNS that points to 127.0.0.1 (loopback address)

STUNNEL config example. Lots of ways to actually do this…
;debug = info
;output = stunnel.log
;fips = yes
engine = capi
sslVersionMax = TLSv1.1

; used this for basic testing while troubleshooting stuff
[gmail-pop3]
client = yes
accept = 127.0.0.1:110
connect = pop.gmail.com:995
verifyChain = yes
CAfile = ca-certs.pem
checkHost = pop.gmail.com
OCSPaia = yes

[proxy-out]
client = yes
accept = 127.0.0.1:54225
connect = server.mydomain.com:443
engineId = capi

[https]
accept = 443
connect = 54225
cert = server2.pem
TIMEOUTclose = 0

!!IMPORTANT! This for BASIC testing. You’ll want to adjust your HTTPS client CA preferences to at least follow browser standards or you’ll get an error messages like this “Service [proxy-out] needs authentication to prevent MITM attacks” and indeed you are vulnerable to MITM

You could export the trusted root CA bundle on Windows using free software
mk-ca-bundle.vbs from https://raw.githubusercontent.com/curl/curl/master/lib/mk-ca-bundle.vbs

You’ll also need openssl.exe which is included in the latest version of git for windows

Create a .cmd with the following line:
if not exist ca-bundle.crt cscript mk-ca-bundle.vbs
save it and run it
if not exist ca-bundle.crt cscript mk-ca-bundle.vbs

Next use openSSL to convert the CRT to PEM
openssl x509 -inform DER -in ca-bundle.crt -out ca-bundle.pem -text

Now reference your ca-bundle.pem in your stunnel config

Or if you trust the CA certs included in STUNNEL use them.

This works for basic testing but it DOES NOT send a client certificate. Outlook 2019 will display the 403 error message you provide in the config once you switch to the production Apache config
[proxy-out]
client = yes
accept = 127.0.0.1:54225
connect = server.somedomain.com:443
verifyChain = yes
CAfile = ca-certs.pem
checkHost = server.somedomain.com
OCSPaia = yes
engineId = capi

In order to connect once you’ve added the rewrite and 403 error you’ll need a config like this
[proxy-out]
client = yes
accept = 127.0.0.1:54225
connect = server.somedomain.com:443
verifyChain = yes
CAfile = ca-certs.pem
checkHost = server.somedomain.com
OCSPaia = yes
engineId = capi
cert = mailuser1.crt.pem
key = mailuser1.key.pem

Certificate related stuff:
====== CA stuff =======
openssl genrsa -out CA.key 3072
openssl req -new -sha256 -key CA.key -out CA.csr
openssl x509 -req -days 3650 -sha256 -extensions v3_ca -signkey CA.key -in CA.csr -out CA.PEM
openssl x509 -outform der -in CA.PEM -out CA.crt

User related stuff: for “mailuser1” … needs to match user’s email address and be imported with private key. Be sure to include a private key password if you want to import this cert into an IOS device also

openssl genrsa -out mailuser1.key 2048
openssl req -new -sha256 -key mailuser1.key -out mailuser1.csr
openssl x509 -sha256 -req -in mailuser1.csr -out mailuser1.crt -CA CA.PEM -CAkey CA.key -CAcreateserial -days 3650
openssl pkcs12 -export -inkey mailuser1.key -in mailuser1.crt -out mailuser1.p12

user cert conversion for stunnel

#!/bin/bash
openssl rsa -in mailuser1.key -out mailuser1.key.pem
openssl x509 -in mailuser1.crt -out mailuser1.crt.pem

Fake server cert STunnel 443 listener related stuff:

openssl genrsa -out server2.key 2048
openssl req -new -sha256 -key server2.key -out server2.csr
openssl x509 -sha256 -req -in server2.csr -out server2.crt -CA CA.PEM -CAkey CA.key -CAcreateserial -days 3650
openssl pkcs12 -export -inkey server2.key -in server2.crt -out server2.p12

!! this is what you may think you need
openssl pkcs12 -export -in server2.p12 -out server2.pem

In most applications with SSL you would need to give a password at startup if you encrypted the pem. stunnel seems simply fail. It wants you to use this

openssl pkcs12 -in server2.p12 -out server2.pem -nodes

email mailuser1 the CA cert public key. Also the mailuser1 public/private key in .p12 with password protection

Import them on the devices.

Once done converting all of the users to CBA, adjust to “require” instead of “optional” for the apache site config.

Overall, you may want to create an Apache alias like /exchange that points to webmail and play with the certificates there first. There are a number of settings on modern browsers and applications that can cause clients to not even try to sent client-side certificates.

I put this in my UCS kopano server in /etc/apache2/sites-enabled/kopano-webapp.conf:
Alias /exchange /usr/share/kopano-webapp

Then put tested the reverse proxy with /exchange
<Location /exchange>
…start with the test config for ActiveSync above, then switch to the final production config suggested above

Go lock down your URLs with a certificate before you get brute forced.

After a few days of use I’ve ran into a few problems and have found the solutions.

ISSUE: I discovered after adding client certificate requirements to locations that I could not send attachments larger than 128KB via activesync

The root cause of this is that when client certificates are invoked and passed on the reverse proxy, the variables and cert pass through to Apache on UCS. Cient certificates have a known issue with default SSLRenegBufferSize values.

There are a few ways to skin this cat.

Option 1 – setup SSLRenegBufferSize on both reverse proxy and UCS. If you do this, you’ll also encounter PHP size limit problems that I up until this change for client certificates I had never ran into before.

The PHP defaults and/or Kopano .htaccess settings may have been to blame as they were 8M and 2M which is nowhere near large enough for a big file. I plan to add some private cloud file services that need to store up to 3GB per file so now is probably as good of time as any to correct this anyway.

#!/bin/bash
grep -Hirn post_max_size /etc/php/7.0/
grep -Hirn upload_max_filesize /etc/php/7.0/

post_max_size 3100M
php_value upload_max_filesize 3000M

There are also .htaccess files with kopano that have these defined
find / -name “.htaccess” -print

If using a reverse proxy, the reverse proxy host can error with too large errors unless this is set on the reverse proxy
SSLRenegBufferSize 300486000

The most critical thing …which turns out is what actually fixed this … was UCS Kopano server /etc/httpd/conf-available/z-push.conf
SSLRenegBufferSize 300486000

Option 2:

In /etc/httpd/sites-enabled/default-ssl.conf use
SSLVerifyClient none

Implement the SSLRenegBufferSize changes on the reverse proxy host only.

Notable discussion topics:

With option 1 an administrator could do some client cert things actually on UCS. Perhaps put user certs in an directory service an use lookups for greater functionality, etc. Perhaps in a way that effectively ties cert to user rather than any cert and any user. The current config is NOT true two-factor authentication. A bigger level of effort is needed to get to true 2FA.

With Option 2 the changes are primarily limited to the reverse proxy Apache host. The only change on UCS is within the SSL config and only to tell it to ignore client certs. Any tie of user to cert would have to happen on the reverse proxy with the type of config making lookups in a directory more complex and less scaleable.

Neither of my current config examples tie user cert and cert serial number together. That is the minimum necessary for true 2FA.

I will look at doing this tie in the future.

A step closer to 2FA is most easily implemented at the reverse proxy by using certificate parsing conditions that tie the user DN_CN to their assigned serial number.

Require expr %{SSL_CLIENT_S_DN_CN} == "user1@somedomain.com" && %{SSL_CLIENT_M_SERIAL} == "[USER1_SERIAL_NUMBER_IN_ALL_CAPITALS_GOES_HERE]" Require expr %{SSL_CLIENT_S_DN_CN} == "user2@somedomain.com" && %{SSL_CLIENT_M_SERIAL} == "[USER2_SERIAL_NUMBER_IN_ALL_CAPITALS_GOES_HERE]"

For instance
Require expr %{SSL_CLIENT_S_DN_CN} == “user5@univention.com” && %{SSL_CLIENT_M_SERIAL} == “8A244AAB7E03F8A5792AB78D54F8F5DF8D2166A0”

This makes it so a given valid certificate is only useful when the user email address and serial number are both expected preauthorized values.

This avoids the need for a CRL. It does still fall short of real 2FA.

What is missing? Making sure that the user logging in matches the user certificate being presented

Mastodon