UCS 4.3 transparent squid proxy setup not working

UCS - Univention Corporate Server
, ,
#1

I have been trying to setup a transparent proxy with squid on UCS 4.3 and I’m running into some problems.

I’b trying to access the squid interface at port 3128 and I’m getting an access denied page. I’ve setup all of my networks and yes my computer I’m accessing port 3128 is in one of those localnets listed in the conf file attached below.

It’s also unclear as how to setup the network interfaces. Is all traffic going to route through the sever so I have to have one NIC on the local network and the other outside? I have enabled echo “net.ipv4.ip_forward = 1” >> /etc/sysctl.conf and then sysctl -p which tells me I need to put this device inline with the traffic that leaves our switch and the filter (also setup as a transparent device) and firewall (which performs our NAT). From our current configuration of the filter and firewall both NICs if they had IPs would be on the same network, but on our filter I only have to set one NIC (the port facing the local networks) with an IP address and the other side connects to the firewall.

(Local networks) --> (UCS Squid) --> (Filter) --> (Firewall) --> Internet

#Warning: This file is auto-generated and might be overwritten by
#         univention-config-registry.
#         Please edit the following file(s) instead:
#Warnung: Diese Datei wurde automatisch generiert und kann durch
#         univention-config-registry ueberschrieben werden.
#         Bitte bearbeiten Sie an Stelle dessen die folgende(n) Datei(en):
#
#	/etc/univention/templates/files/etc/squid/squid.conf
#

include /etc/squid/local.conf

access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log

cache_dir ufs /media/squidcache/cache 512 16 256

acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY

# Adapted from squeeze default configuration
refresh_pattern ^ftp:		1440	20%	10080
refresh_pattern ^gopher:	1440	0%	1440
refresh_pattern -i (/cgi-bin/|\?) 0	0%	0
refresh_pattern .		0	20%	4320

# debug options
debug_options ALL,1

http_port 3128 transparent

########
# ACLs #
########

# default
# ACLs all, manager, localhost, and to_localhost are predefined.
acl purge method PURGE
acl CONNECT method CONNECT
acl web_ports port 80
acl web_ports port 21
acl web_ports port 8530
acl web_ports port 8531
acl web_ports port 8088

# network ACLs
acl localnet0 src 172.16.0.0/21
acl localnet1 src 192.168.16.0/20
acl localnet2 src 192.168.0.0/24
acl localnet3 src 10.0.100.0/24
acl localnet4 src 172.16.0.0/21
acl localnet5 src 172.16.8.0/22
acl localnet6 src 172.16.14.0/24
acl localnet7 src 172.16.24.0/23
acl localnet8 src 172.16.49.0/24
acl localnet9 src 172.16.50.0/24
acl localnet10 src 172.16.51.0/24
acl localnet11 src 172.16.52.0/24
acl localnet12 src 172.16.53.0/24
acl localnet13 src 172.16.54.0/24
acl localnet14 src 172.16.55.0/24
acl localnet15 src 172.16.56.0/24
acl localnet16 src 172.16.57.0/24
acl localnet17 src 172.16.58.0/24
acl localnet18 src 172.16.59.0/24
acl localnet19 src 172.16.60.0/24
acl localnet20 src 172.16.61.0/24
acl localnet21 src 172.16.62.0/24
acl localnet22 src 172.16.63.0/24
acl localnet23 src 172.16.64.0/24
acl localnet24 src 172.16.65.0/24
acl localnet25 src 172.16.66.0/24
acl localnet26 src 172.16.67.0/24
acl localnet27 src 172.16.68.0/24
acl localnet28 src 172.16.69.0/24

#############
# user ACLs #
#############


#########
# rules #
#########
include /etc/squid/local_rules.conf

# default rules

# Only allow cachemgr access from localhost
http_access deny !web_ports
http_access allow localhost manager
http_access deny manager
http_access allow purge localhost
http_access deny purge

# deny from user ACLs

# allow from user ACLs

# allow access based on networks
http_access allow localhost
http_access allow localnet0
http_access allow localnet1
http_access allow localnet2
http_access allow localnet3
http_access allow localnet4
http_access allow localnet5
http_access allow localnet6
http_access allow localnet7
http_access allow localnet8
http_access allow localnet9
http_access allow localnet10
http_access allow localnet11
http_access allow localnet12
http_access allow localnet13
http_access allow localnet14
http_access allow localnet15
http_access allow localnet16
http_access allow localnet17
http_access allow localnet18
http_access allow localnet19
http_access allow localnet20
http_access allow localnet21
http_access allow localnet22
http_access allow localnet23
http_access allow localnet24
http_access allow localnet25
http_access allow localnet26
http_access allow localnet27
http_access allow localnet28

# deny the rest
http_access deny all
http_reply_access allow all
icp_access allow all

coredump_dir /var/spool/squid

forwarded_for off


# this must be the last directive
include /etc/squid/local_bottom.conf
#2

Well, you have configured your squid to be transparent. Sorry, but I guess you should go and learn a little bit about networking before setting up such stuff.

Anyways:

First, you do not want to “access” you squid as it is not at all a web server! You will see on the logs if Squid is working properly or you might want to browse some sites (Search for “Am I behind a proxy?” or so) which will tell you if you use a cache.
Second, If your Squid should act as a transparent cache you should take care network packets trying by default to access the Internet gets properly redirected to the Squid host. This can (more or less) only be done on your default gateway for your local network.
Third, as your Squid hosts does have just a single IP and is in the same network as your clients you will additionally to step 2 NAT the source of the packets, too.

So before trying to falsely access Squid you should make sure your clients requests get properly forwarded to Squid. If this works, you can configure Squid.

Workaround
I would suggest to configure your clients to use Squid. This would skip the need to re-configure your firewall/ gateway and is (in your environment) possibly the easier way to do so. So not being a transparent proxy is in some ways much easier to configure.

#3

Knebb,

I was afraid of having to double NAT and wasn’t sure how squid operated because our filter doesn’t double NAT and it is transparent too. If you bothered to read my post instead of providing an off the cuff bitchy response, " you should go and learn a little bit about networking", then you might have been more helpful than you are currently being.

As I read it it said to access squid use port 3128. To me that meant a web interface for managing the cache and stuff. Apparently not. No, I have to have squid transparent because I cannot and will not singlehandedly configure a few thousand student owned (BYOD) laptops.

#4

Have you read further than “…stuff.”?

Part 1 (access) I have explained in my post. You got it.
Part 2 (NAT) I explained as well. My fault I missed the part where you were telling us you are aware of it.
Part 3 (not-transparent) was a suggestion without knowing your local environment. Indeed, now knowing it this is not a working solution.

So what you can do is proper doubleNATing on your filter, excluding packets from the Squid host.
Or configure your UCS-Squid to be the man-in-the-middle between filter and firewall (or acting as a local gateway in front of the filter) by indeed using two NICs. And yes, in both cases you need th ip_forward to be enabled to route all other traffic. For the http-part you will not need the forwarding…

So you might have enough information to decide how to setup. Depends on your environment and your knowledge (which both I am not aware of).

Oh, BTW:
As you mentioned thousands of devices be aware of the performance pitfalls! Squid needs fast disks. FAST! REALLY FAST! And Memory, RAM. Lots of.
IF one of these two items is not given you will have a bottleneck and your performance will be lower than you might have expected.
And, be aware, Squid in transparent mode does not handle https-traffic. So you might catch only few requests as currently most of Internet sites are protected by https.

BTW2: There is the possibility of automated proxy provisioning script with PAC. See if it will help you: See here. This way you might catch https-traffic as well?

#5

My question is, why did you use systemd’s resovle addresses.

$ systemd-resolve --status
Global
       LLMNR setting: no
MulticastDNS setting: no
  DNSOverTLS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      168.192.in-addr.arpa
                      17.172.in-addr.arpa
                      18.172.in-addr.arpa
                      19.172.in-addr.arpa
                      20.172.in-addr.arpa
                      21.172.in-addr.arpa
                      22.172.in-addr.arpa
                      23.172.in-addr.arpa
                      24.172.in-addr.arpa
                      25.172.in-addr.arpa
                      26.172.in-addr.arpa
                      27.172.in-addr.arpa
                      28.172.in-addr.arpa
                      29.172.in-addr.arpa
                      30.172.in-addr.arpa
Mastodon