WireGuard on Kubernetes with Adblocking
Lets be frank, the Internet is simply unusable with all the ads floating around.
I use the uBlock Origin extension in my browser, as do most of the people reading this genre of articles, but the same is not true for the majority of the population, including other members of my family. So in order to enhance their web browsing experience I decided to block ads at the DNS level.
But why stop there I thought, why not also improve their privacy while I'm at it. So I also decided to setup a VPN. Now let me clarify some things here, I'm not a big fan of VPNs, the way they're advertised by the big companies, here's a great video by Tom Scott explaining what I mean. But they also have their use cases, some of which are:
- Prohibiting ISPs from collecting data on my browsing patterns
- Circumvent internet censorship
- Connect to my home network from anywhere
I took a look at apps like Blokada and DNS66 which grant you device wide ad blocking on mobile devices. On Android the way this works is, by creating an internal VPN on the phone, so that all traffic from the device can be routed via it, and it has a file with all the blacklisted domains, to filter out traffic.
But there's a caveat with this approach. I cannot use another VPN to route my traffic, due to an Android limitation.
There can be only one VPN connection running at the same time. The existing interface is deactivated when a new one is created.
This means my browsing patterns are still accessible to my ISP. So, I started looking for ad blocking DNS servers, so that I could point Android's global DNS to it. I found AdGuard and PiHole to be the top projects. Hosting these at home, on Raspberry Pi seemed like a plausible solution, but then again one can't use it while traveling. The solution is to obviously host it on a publicly accessible server. Amongst the two I found AdGuard more appealing due to the following reasons
- It has out of the box support for DNS-over-TLS
- It maintains a single file for its entire configuration
- It's written in Golang, and is much lighter on resources compared to PiHole
You can find more about their differences here. Both are great projects, but AdGuard met my requirements perfectly.
When it comes to VPN, I did not even consider using OpenVPN, WireGuard was the obvious choice, because of it's speed, smaller, easily auditable codebase (not that I was going to audit it, but still), cross platform compatibility and integration into the Linux Kernel.
Being a big fan of Kubernetes and maintaining infrastructure as code, I wanted a way to be able to easily deploy and version control my deployment. After much searching I stumbled upon, kilo, a network overlay built on WireGuard for Kubernetes. It could do exactly what I wanted, while also enhancing the security of my cluster, by encrypting the inter pod communication, and allowing me to build secure clusters, over nodes spanning multiple cloud providers. Also it would give me the added benefit of easily debugging applications deployed on my Kubernetes Cluster, since when connected I would be a peer on the network, thereby getting access to the all the private IPs of the deployments, services etc. You can watch this talk by Lucas Servén Marín to know more.
Without further ado, let's jump right into the setup. I'll be explaining the steps for setting it up on a k3s cluster. You may need to modify them as per your cluster. After deploying k3s, the 1st thing which needs to be done is to setup kilo. First download the manifest for kilo on k3s.
curl -LO https://raw.githubusercontent.com/squat/kilo/master/manifests/kilo-k3s.yaml
Now you need to modify and add - --mesh-granularity=full to the DaemonSet section under the args for the kilo container.
...
containers:
- name: kilo
image: squat/kilo
args:
- --kubeconfig=/etc/kubernetes/kubeconfig
- --hostname=$(NODE_NAME)
- --mesh-granularity=full
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
...
This is done to ensure all our nodes are meshed together regardless of the datacenter. Then simply apply the manifest.
kubectl apply -f kilo-k3s.yaml
This will later be useful for setting up WireGuard VPN. More on this later. Now we can proceed to setup AdGuard. Here is the spec for AdGuard.
apiVersion: apps/v1
kind: Deployment
metadata:
name: adguardhome
spec:
selector:
matchLabels:
app: adguardhome
replicas: 1
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 0
template:
metadata:
labels:
app: adguardhome
spec:
volumes:
- name: tls-cert-secret
secret:
secretName: production-tls-cert
- name: adguard-config
hostPath:
path: "/path/to/store/conf"
type: DirectoryOrCreate
- name: adguard-logs
hostPath:
path: "/path/to/store/work"
type: DirectoryOrCreate
containers:
- name: adguardhome
image: adguard/adguardhome:v0.102.0
ports:
# Regular DNS Port
- containerPort: 53
hostPort: 53
protocol: UDP
- containerPort: 53
hostPort: 53
protocol: TCP
# DNS over TLS
- containerPort: 853
hostPort: 853
protocol: TCP
volumeMounts:
- name: tls-cert-secret
mountPath: /certs
- name: adguard-config
mountPath: /opt/adguardhome/conf
- name: adguard-logs
mountPath: /opt/adguardhome/work
terminationGracePeriodSeconds: 20
---
apiVersion: v1
kind: Service
metadata:
name: adguardhome
labels:
app: adguardhome
spec:
type: ClusterIP
selector:
app: adguardhome
ports:
- port: 80
# targetPort: 3000
targetPort: 80
protocol: TCP
The RollingUpdate is intentionally configured to not wait till a new pod is up, and directly terminate the existing pod during deploys. On the surface it seems like an anti-pattern, but since I'm using the hostPort directive, a new Pod wouldn't get scheduled unless port 53 was available on the host for it to bind to, so the existing Pod has to terminate before a new Pod can be deployed.
Also I initially intended to use a ConfigMap for holding the AdGuardHome.yml, but there was some issue with AdGuard trying to write to it while initally comming up, but since ConfigMap's are moundted as ReadOnly, Pod creation used to fail, so I decided to go with a Volume instead, until I could figure out the issue.
For the initial setup, the AdGuard admin UI will be accessible on port 3000, so you'll have to switch the targetPort to 3000 in the adguardhome service initially, access the admin UI, setup the password, and then revert the targetPort to 80.
You may also enable DNSSEC under DNS Settings for guaranteeing authenticity of DNS responses by signing them, and making tampering detectable.
The volume mounts for tls-cert-secret are only necessary if you want to enable DNS-over-TLS. And you need to configure your ingress resource before mounting it here.
For enabling DNS-over-TLS, on the AdGuard admin UI you can goto Encryption Settings -> Enable Encryption, and put in the Certificate path as /certs/tls.crt and the Key path as /certs/tls.key. Again let me re-iterate that your Ingress resource needs to be configured properly and you need to have a valid TLS certificate for the domain you're hosting AdGuard on.
On Android phones for Android Pie and later, you may goto Settings -> WiFi and Internet -> Private DNS. Select Private DNS Hostname Provider and set it to the domain name to the once you've configured on. You may also configure it on your home router to ensure all the devices get DNS level Ad Blocking.
This concludes the AdGuard part of the setup.
Now coming to setting up WireGuard.
I'll be referring to the k3s cluster as the server and the local laptop as the client from here on.
You'll need WireGaurd installed on both your server and client machine. Follow the steps as per your distribution to install the same. If you're using a bleeding edge distro like Archlinux or Gentoo, you don't need to do anything on the server side, since WireGuard is already baked into the kernel at this point. On the client side however you'll need to install it for getting the command line client to enable / disable the interface.
Another useful tool to have on the client side is kgctl. You can install it using
go get github.com/squat/kilo/cmd/kgctl
Now we need to create a private and a public key pair on the client.
wg genkey | tee privatekey | wg pubkey > publickey
This'll create 2 files with the respective key contents. This key pair needs to be authorized on the server. You can do this simply by creating a peer resource. Create a file named archie.yaml
apiVersion: kilo.squat.ai/v1alpha1
kind: Peer
metadata:
name: archie
spec:
allowedIPs:
- 10.120.120.1/32 # This is just and example, you can use any valid available CIDR here
publicKey: CLIENT_PUBLIC_KEY # Enter the public key here, the one you just generated
persistentKeepalive: 10
Finally apply the manifest
kubectl apply -f archie.yaml
Remember, the allowedIPs should be a valid CIDR, which is available on both the server and the client. Now we can use the kgctl tool to generate the peer section of the client WireGuard config.
kgctl showconf peer archie
This will return something like
[Peer]
AllowedIPs = 10.42.0.0/24, 10.42.0.0/32, 10.4.0.1/32
Endpoint = YOUR_SERVER_IP:51820
PersistentKeepalive = 10
PublicKey = SERVER_PUBLIC_KEY
Create a file on the client named adgaurd.yaml at the location /etc/wireguard and add the following contents to it
[Interface]
Address = 10.120.120.1/32 # Use the same CIDR whitelisted in the Peer manifest
PrivateKey = CLIENT_PRIVATE_KEY # The one you generated above on your client
DNS = YOUR_SERVER_IP # Enter the IP of the server to block ads, FQDN don't work on Android for some reason
[Peer]
AllowedIPs = 0.0.0.0/0, ::/0 # This modification is important to route all the traffic from your machine via wireguard interface
Endpoint = YOUR_SERVER_IP:51820
PersistentKeepalive = 10
PublicKey = SERVER_PUBLIC_KEY # as recieved from the above config
Now to enable the VPN on your client machine you may use
wg-quick up adguard
You can verify you're connected to the VPN server by visiting ifconfig.io. It should show your IP as the IP of your server.
To disconnect, you may use
wg-quick down adguard
Using the same process outlined above you can add multiple Peers and create multiple configs. You can also install and use the qrencode utility to convert the same config to a QR code for easy scanning on mobile phones.
qrencode -t ansiutf8 < /etc/wireguard/adguard.conf
This will print a QR code right in your terminal. You can install the WireGuard Android App on your phone and this scan this QR code to import the config.
And viola, whenever you're connected, your ISP can't snoop in on the websites you're visiting, add to that all your traffic will be filtered out for ads, while also routing your data through the server for protection against malicious actors in your network, and for the cherry on top, you can have all your devices connected and talking to each other regardless of the location / network they are on.
Guess this post made it to the front page on Y Combinator's Hacker News! You can find the HN post here. And here's the web archive link (8th one on this). This invited a good amount of traffic onto my blog :)
