Network Routing and Automation Fun with pfsense, Wireguard, and Tasker

Last year, I asked about some very specific home network routing ideas over on reddit, with the goal being to securely access home network resources from my phone while away from home while maintaining some coverage from my 3rd party VPN provider, Mullvad. The faster and more automated any network switcheroos are, the better.

This post is… very long. Here’s a table of contents in case you only care about parts of it:

1. Reqs and Specs
2. Prerequisites
3. Wireguard In, Wireguard Out
   1. Basic Interface Structure
   2. Mullvad Wireguard Tunnels
   3. Gateway Group
   4. Interface Group
   5. Firewall (Routing)
   6. Test!
4. At-home Mullvad Wireguard on Android
5. Tasker
6. Open Issues
   1. Detecting Wifi Disconnection in Tasker
   2. pfsense/Mullvad Instability
7. What’s Next

Reqs and Specs

I’ve refined the idea and changed a few technologies since that reddit post, so my requirements are now:

1. When I’m home, my phone stays connected to my 3rd party wireguard VPN (Mullvad), but uses my home DNS resolver (pfsense).
2. While I’m out, my phone connects to my home wireguard service (pfsense), which allows internal resource access.
3. My home wireguard service routes its traffic out through my 3rd party VPN provider.
4. My phone switches between these wireguard services automatically as I connect or disconnect from my home wifi (this is currently a little shaky).

At the time of the reddit post, I was running my own CalDav/CardDav server (Baikal) and pushing photo backups directly to my Synology NAS, so a fairly stable connection to my home network was a plus. That’s less the case now: my access away from home is primarily to get to Photoprism and FreshRSS, neither of which are background tasks benefiting a constant connection.

Here are some of things I didn’t have a great grasp on before starting this undertaking:

1. pfsense gateways and gateway groups
2. Firewall configuration within pfsense other than just basic allowing/blocking for games and services
3. Configuring wireguard connections manually (🎶 still only barely do 🎶)

Doing a lot better on them now, and even moreso after writing this post.

Final note here: Don’t be like me, folks. Backup your router settings early and often as you work, and take notes on what you’re doing.

Prerequisites

Have a functioning wireguard server that runs well for your phone/laptop/whatever. Mine uses the pfsense plugin, but if you’ve spun up your own, then just make sure you’ve already done the routing to make it work in pfsense. You can also do part or all of this whole thing with OpenVPN with some minor adjustments. Ask in the comments if you get stuck trying that path.

Wireguard In, Wireguard Out

We now enter the networking segment. It gets less complicated after this, I promise.

Basic Interface Structure

Speaking only to the relevant process here, you’ll end up with these interfaces defined within pfsense:

1. WAN
2. LAN
3. VLAN_xx_DHCP – standard home lan/wifi VLAN
4. WireguardServer – home wireguard server, configured when wg was set up
5. Mullvad1 – outbound connection to Mullvad
6. Mullvad2/etc – second+ outbound connection to Mullvad

If you don’t have #1-4 (or some stable, working variation on those) in place, definitely go do that. Adding in this new stuff on top of a barely-functioning network will only bring misery, trust.

Mullvad Wireguard Tunnels

I set up two tunnels with a couple peers each, but you could do this with any number for additional redundancy, or a single tunnel with multiple peers. Mullvad limits users to 5 wireguard keys, so my combination of devices + connections is limited.

SpookyGhost wrote a very good guide for getting Mullvad connected in pfsense. Follow that up through the setup of the Outbound NAT rules, since the rest of the guide funnels all traffic through Mullvad¹. If you want multiple tunnels, just rinse and repeat.

One deviation from the guide: in each peer, set a Keep Alive of 60 seconds. Since I’m not using the connections all the time, I found they did not wake gracefully when I needed them, and when they go down, it’s bullshit to get them reconnected.

You should then have one or more tunnels with peers persistently connected. Check Status → Wireguard → Status and expand out your Mullvad tunnels.

Gateway Group

We’re going to make two gateway groups, one for IPv4, one for IPv6.

Under System → Routing → Gateway Groups, add a gateway group. Name it something like “MullvadGateways_IPv4” and assign tiers to your Mullvad gateways. As soon as you pick either an IPv4 or IPv6 gateway, the other protocol gateways become unavailable. Then just rank your Mullvad gateways in whatever order you wish.

Make sure to set your normal home ISP gateway, probably called “WAN_DHCP” or something similar, to never be used (although see the “Open Issues” section below).

I am currently using a trigger level of “Packet Loss or High Latency” here for failing over. I haven’t tweaked the tolerances on each gateway, but that’s an option if you find the connections to be very up-and-down while no issues are reported on the Mullvad side.

Repeat the above process for the IPv6 gateway group.

Interface Group

Now we need to group the Mullvad intefaces created using SpookyGhost’s guide to help with firewall configuration.

Go to Interfaces → Interface Groups and make a new interface group called something like “Mullvad_IG”. Put all your Mullvad interfaces in it.

Firewall (Routing)

Now we need to make incoming VPN traffic use that outgoing gateway group.

If you don’t already have an Alias for your incoming wireguard connections, it’s good to have one. Go to Firewall → Aliases → IP and create one—I call mine “IncomingWireguard”—and set it to your wireguard network’s value range in CIDR format.

If you also don’t have an Alias for your LAN network, make that, too. In this case, “LAN network” means the network that your VPN’d users want access to. For me, that’s the network that includes my FreshRSS and Photoprism servers.

Then go to Firewall → NAT → Outbound and create a new rule. We want all traffic not aiming for local network resources to go through Mullvad, so use the following settings:

1. Interface: the Mullvad interface group we created
2. Address family: IPv4 + IPv6
3. Protocol: any
4. Source: Network, “IncomingWireguard” alias
5. Destination: Network, “LAN_Network” alias
6. Check the “Not” option. 
7. Set a useful description, like “Incoming Wireguard –> Mullvad”

Go to Firewall → Rules and then your wireguard interface (mine is very creatively called “WIREGUARDSERVER”). My guide is fuzzy here because I don’t remember what this looked like back when I had just setup wireguard. 

I have two rules:

1. Allow incoming wg traffic to my LAN network
   1. Interface: WIREGUARDSERVER
   2. Protocol: any
   3. Source: WIREGUARDSERVER net
   4. Destination: VLAN_xx_DHCP (from the original interface listing above)
   5. No advanced options set
2. Allow incoming wg to any and send all to the Mullvad gateway group:
   1. Interface: WIREGUARDSERVER
   2. Protocol: any
   3. Source: WIREGUARDSERVER net
   4. Destination: any
   5.  Advanced → Gateway: Mullvad gateway group

You’d think that only the second rule was necessary, but without it, you won’t be able to access local network resources. 

Test!

Verify that if you connect to your home VPN on your laptop or phone, you are:

1. Still able to access your desired network resources
2. Shown on (in my case) the Mullvad checker as being on VPN. Depending on your DNS configuration, you might be leaking DNS information.

If something goes awry there, double-check your config. Drop me a comment if you can’t find the issue and/or I missed something in my instructions.

And now, we’re done with pfsense: physically, emotionally, and spiritually.

At-home Mullvad Wireguard on Android

We made it this far. This next part is simpler: we need to setup wireguard on our Androids such that we have a connection to Mullvad except for DNS resolution, which uses our home DNS resolver. If you already have this in place using the stock Wireguard app, then you’re good to go.

There’s evidence that it might be possible to get Tasker to work with the Mullvad app, but there are a whole slew of complications with always-on VPN, which can only be set to a single app and blocks other apps from creating wireguard connections. It was a mess when I was toggling between OpenVPN and wireguard for the same reason.

So we do everything in the “official” wireguard app. 

Use Mullvad’s (or your own provider’s) configuration generator, get some config files onto your phone and import them into the Wireguard app. They should just work, although you’ll lose local network access.

I honestly only keep one standard Mullvad config in my phone, and I make sure to label it something like “mullvad-home” so that I know it’s the one with the DNS customization. I have multiple excluded applications, and I don’t enjoy keeping the list in sync across connection configs.

Anyway, edit that connection:

1. Set the DNS server(s) to your home DNS resolver
2. Add any excluded apps (phone, visual voicemail, whatever)
3. Check the box for “Exclude private IPs”
4. At the tail end of the “Allowed IPs”, remove your DNS resolver, which was added to the list

Bad news: every time you edit and save that wireguard config (which is probably only to update excluded applications), you’ll need to re-remove the DNS server(s) from that list so that they don’t route out through Mullvad.²

Last thing here: under the general settings in Wireguard, expand the advanced options and check the box for “Allow remote control apps”. That lets Tasker do its thing.

Tasker

Oh, Tasker. I love you in theory, but want to yeet you out of my office in practice. See the “Open Issues” section below as to why. But this is what we’ve got, so we’re rolling with it.

My Tasker project is too long to embed here, so it’s available here or as a gist. Basically, though:

1. When wifi is connected to a network, set the variable %CURRENTWIFI to the SSID of whatever network you’re on.
2. When wifi is disconnected, set %CURRENTWIFI to a fake value like the string “None”. I found that if I left it empty, I got less reliable results.
3. When connected to home wifi (%CURRENTWIFI ~R HomeSsidName*), disconnect the wireguard connection to home and connect your “mullvad-but-home-dns” connection.
4. When not connected to home wifi (%CURRENTWIFI !~R HomeSsidName*), disconnect the “mullvad-but-home-dns” connection and connect to your home wireguard server instead.

Take a look at the code, make sure there’s nothing sketch in there, and then import it into Tasker and customize it. I’d welcome ways to refine and improve it.

Open Issues

These are going to sound like Too Much when they’re written out, but honestly, given that I don’t even leave the house every day, these are things I can take my time on fixing or find workarounds for.

Detecting Wifi Disconnection in Tasker

Tasker is terribly inconsistent on picking up when I leave my home wifi network. Sometimes it picks it up within a few minutes, sometimes hours/effectively never, meaning that I just quietly don’t have internet on my phone, since my DNS is pointed to a server that isn’t available.

When I turn off wifi on my phone, Tasker immediately picks up on the change. 🤷🏾‍♀️ If I manually disconnect from my Mullvad connection, it’ll often immediately pick up that it should connect to my home wireguard. Other times, I have to manually switch.

I can’t find any missed power/sleep-related options that could be keeping Tasker from noticing the wifi disconnect consistently. I’d welcome suggestions.

pfsense/Mullvad Instability

There are rare times when all my pfsense Mullvad connections collapse in a fit of pique while I’m out and about, leading to my phone not having internet through my home Wireguard connection. I still have home network access, so I can prod at pfsense to restart those connections, but that’s annoying.

Setting the Keep Alive on the peers helped this go from a regular occurence to very rarely.

One solution might be to go ahead and include my regular WAN gateway in the Mullvad gateway group, but make it the lowest tier. If I did that, I’d definitely experiment with the Packet Loss and High Latency tolerances on the Mullvad gateways and keep an eye on my logs/notifications to see how often the failover to the ISP is happening.

What’s Next

My next super technical project is replacing Graylog/Elasticsearch/Grafana with Loki/Grafana. If I can get that sorted, that’ll be another post, because yeesh, Grafana + Loki is pretty annoying in some of its querying restrictions.

1. Why not just route everything instead of these shenanigans? Because I have too much going on on my home network to be on my router chewing on settings every time I want to stream anything on Netflix or Amazon, use my TV, etc., when I could just flip a switch on the device itself (or exclude the app on my phone). I legit do not know how people circumvent those services balking at VPN, but despite the content of this entire post being what it is, I do try to make less work for myself in the long run.
2. Emailing them about that bug is on my todo list.