AdGuard Home: Taming the DNS Loop

Installing AdGuard Home (AGH) via the iStoreOS app store was straightforward. The initial setup wizard (:3000) guided me through setting ports. I changed AGH’s DNS port to :5353 to avoid conflict with iStoreOS’s built-in dnsmasq (running on :53).

The goal: Clients -> iStoreOS (dnsmasq :53) -> AdGuard Home (:5353) -> Public DNS (e.g., 1.1.1.1).

Configuration involved:

1. dnsmasq: Set it to not forward upstream queries found in /etc/resolv.conf, and explicitly forward all queries to AGH listening on the loopback address: server=127.0.0.1#5353.
2. AdGuard Home: Set its upstream DNS servers to Cloudflare/Google.
3. DHCP: Configure iStoreOS’s DHCP server (Option 6) to advertise the router’s IP (10.0.0.1) as the DNS server for all clients.

Initially, this resulted in DNS resolution failures and context deadline exceeded errors in the AGH logs. Diagnosis: a DNS loop. Dnsmasq was forwarding to AGH, but AGH, possibly due to a misconfiguration or startup order issue, was somehow trying to query dnsmasq back.

Adding to the confusion was a “ghost configuration.” I had initially tried installing AGH via a curl script before using the app store version. This seemed to leave conflicting settings. The breakthrough came almost accidentally: running the AdGuard Home binary directly from the command line with the -s run flag (./AdGuardHome -s run) triggered some kind of self-reset or reinitialization. After this, restarting the service properly allowed the correct DNS chain (dnsmasq -> AGH -> Public) to establish, and DNS resolution started working beautifully across my network.

The Network Speed Crisis: A Virtualization Bottleneck

With DNS filtering active, I ran a speed test. Disaster! My gigabit fiber connection, which previously delivered ~950 Mbps, was now capped at a measly 400 Mbps.

The troubleshooting began:

• Disable AdGuard Home: No change. DNS filtering wasn’t the bottleneck.
• Check Physical Links: Used ethtool eth0 and ethtool eth1 within the iStoreOS VM to confirm the virtual NICs were negotiating at 2500 Mbps with the PVE bridges. Links were fine.
• Enable Software Flow Offloading: Toggled this common OpenWrt optimization. No significant change.

The problem had to lie deeper, likely within the virtualization layer itself. Research pointed towards a common issue with high-speed networking in VMs: the CPU struggling to handle the sheer volume of network packets being processed by a single core for the virtual NIC.

The Solution: Multiqueue

Proxmox VE (and the underlying KVM/QEMU) supports a feature called Multiqueue VirtIO-Net. This allows the network packet processing load for a virtual NIC to be spread across multiple vCPU cores assigned to the VM.

1. Shut down the iStoreOS VM.
2. In the PVE web UI, go to the VM’s “Hardware” tab.
3. Select the VirtIO network device corresponding to the LAN interface (net0 in my case, connected to vmbr1).
4. Click “Edit”.
5. Check the “Multiqueue” box.
6. Set the “Queues” number equal to the number of vCPU cores assigned to the VM (e.g., if you gave it 4 cores, set Queues to 4).
7. Repeat steps 3-6 for the WAN network device (net1, connected to vmbr0).
8. Start the iStoreOS VM.

Ran the speed test again. Success! Speeds jumped right back up to the ~950 Mbps range. The virtualization bottleneck was eliminated.

Tailscale Triumph: Conquering Remote Access

My final major goal was setting up remote access, primarily to manage services like my NAS (planned OMV VM) and potentially the router itself when away from home. I opted for Tailscale due to its ease of use and availability in the iStoreOS app store.

Installation was simple. Following the command-line instructions (tailscale up --advertise-routes=10.0.0.0/24) connected the router to my Tailnet and advertised my home LAN subnet. However, while the router appeared online in the Tailscale admin console, I couldn’t access any devices on my home LAN (like 10.0.0.1) from my phone connected via Tailscale.

Troubleshooting Round 1: Firewall

Pinging 10.0.0.1 from my phone resulted in “Destination Port Unreachable.” This screamed firewall. iStoreOS hadn’t automatically created a firewall zone or rules for the new tailscale0 interface.

Troubleshooting Round 2: Interface & Zone

Following guidance from GitHub issues, I manually edited /etc/config/network to define the tailscale0 interface:

config interface 'tailscale'
        option proto 'none'
        option device 'tailscale0'

Then, in the LuCI Firewall settings (Network -> Firewall), I:

1. Created a new firewall zone named tailscale.
2. Set its Input/Output/Forward policies to ACCEPT.
3. Assigned the tailscale0 network interface to this zone.
4. Crucially, created a Forwarding rule allowing traffic from the tailscale zone to the lan zone.

Saved and applied. Tried connecting again. The “Port Unreachable” error was gone, but now attempts to access 10.0.0.1just… hung. Loading indefinitely.

Troubleshooting Round 3: MTU/MSS

This new symptom – indefinite loading – strongly suggested an MTU (Maximum Transmission Unit) mismatch problem, common with VPNs and tunnels like Tailscale. Packets were likely getting fragmented or dropped because their size exceeded what the tunnel could handle. The standard fix is MSS Clamping (Maximum Segment Size), where the router rewrites TCP packet headers to request smaller segment sizes.

The Final Fix: iptables MSS Clamping

Added the following rules via Network -> Firewall -> Custom Rules:

# Tailscale MSS Clamping for IPv4
iptables -t mangle -A FORWARD -o tailscale0 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

# Tailscale MSS Clamping for IPv6 (if using IPv6 with Tailscale)
ip6tables -t mangle -A FORWARD -o tailscale0 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

Restarted the firewall. Tried accessing 10.0.0.1 from my phone via Tailscale one last time.

Victory! The iStoreOS login page loaded instantly. Remote access was finally conquered.

Conclusion: A System Forged in Debugging

This journey, from a simple IPv6 issue to a fully virtualized N100 soft router with custom services and robust remote access, was far more challenging than I initially anticipated. Every step seemed to uncover a new problem, requiring deep dives into networking concepts, Linux internals, and virtualization quirks.

However, each problem solved solidified my understanding and resulted in a final system that is incredibly powerful, stable, and perfectly tailored to my needs. The N100 platform with Proxmox and iStoreOS has proven to be an outstanding foundation. The debugging war is over, and the era of enjoying a truly capable home lab has begun.

Act I: The Struggles with Vanilla OpenWrt

Creating the VM in PVE (giving it VM ID 100) involved several steps:

1. Download Firmware: Get the official OpenWrt x86-64 combined-ext4 image (.img.gz).
2. Upload & Import: Upload the .img.gz file to PVE (e.g., via SCP or the web UI’s storage browser – note: uploading directly as an ISO image type fails). Import the disk image into the VM using qm importdisk 100 openwrt.img.gz local-lvm.
3. VM Hardware Config: Assign CPU cores, RAM. Remove the default SCSI disk. Attach the imported OpenWrt disk as a VirtIO block device. Add two VirtIO network cards – one connected to PVE’s vmbr0 (WAN) and one to vmbr1 (LAN). Set the boot order to use the OpenWrt disk.
4. Disk Resizing: The default image is tiny. Resize the virtual disk in PVE’s hardware tab. Boot the VM with a GParted Live ISO attached, use GParted to expand the OpenWrt partition to fill the larger virtual disk.
5. Initial Network Config: Boot OpenWrt. Access the console (via PVE’s web UI or qm terminal 100 - not qm console). Use the vi editor to modify /etc/config/network, changing the LAN interface’s IP address to 10.0.0.1.

So far, so good. I could access the OpenWrt LuCI web interface at 10.0.0.1. I configured PPPoE on eth1 (WAN, connected to vmbr0) and set up the LAN interface (eth0, connected to vmbr1) with DHCP and IPv6 server settings. Basic routing worked!

The Dependency Hell

The problems began when I tried to customize OpenWrt by installing packages. My goal was simple: install luci-app-adguardhome. But opkg update often failed to connect to the repositories. When it did connect, attempting to install AdGuard Home resulted in a cascade of errors about incompatible architectures or missing dependencies (luci-lua-runtime was a frequent culprit). Even manually downloading .ipk packages failed with similar architecture errors.

I spent hours trying different package versions, compiling dependencies, and scouring forums. I even managed to “force install” some packages, but this led to a broken LuCI interface – the “Services” menu simply wouldn’t appear, despite the package seemingly being installed.

[Placeholder: Add a screenshot of the terminal showing the opkg errors or the LuCI interface missing the Services menu.]

Clearing LuCI caches (rm /tmp/luci-indexcache) did nothing. Log files (logread) showed no obvious errors. File permissions seemed correct. The final nail in the coffin was a detailed error log clearly stating that key LuCI components and their dependencies were fundamentally incompatible with the base system. Trying to manually build a complex setup on vanilla OpenWrt felt like assembling a car from mismatched parts found in a junkyard.

Act II: The Turn Towards iStoreOS

Frustrated and realizing the “DIY package management” route was unsustainable, I revisited earlier advice: use a pre-integrated firmware like iStoreOS. Based on OpenWrt but comes with a user-friendly interface and, crucially, a curated app store where dependencies should just work.

The Migration Plan

The challenge: perform this migration without disrupting my family’s internet access for too long.

1. Preparation: In PVE, create a new VM (e.g., VM ID 101) for iStoreOS. Download the iStoreOS x86 .img.gz firmware.
2. Import & Basic Config: Upload and import the iStoreOS disk into VM 101. Configure its hardware similarly to the OpenWrt VM (CPU, RAM, 2x VirtIO NICs connected to vmbr0 and vmbr1).
3. Offline Config Attempt (Failed): Boot VM 101 with GParted. Try to mount the iStoreOS partition to change the default IP (192.168.100.1) to 10.0.0.1 offline. Failure! GParted couldn’t identify the filesystem. Manual mount commands failed, complaining about unknown filesystem types or missing superblocks. Offline config was impossible.
4. The Switch: Plan B - configure online. Physically disconnect the WAN cable from the N100. Shut down the old OpenWrt VM (100). Start the new iStoreOS VM (101). Set my Mac’s IP statically to 192.168.100.5. Connect my Mac to one of the N100’s LAN ports (connected to vmbr1). Try accessing 192.168.100.1.

The WAN/LAN Inversion Twist

Failure again! 192.168.100.1 was unreachable. In a moment of desperation (or maybe just random plugging), I moved my Mac’s cable to the N100 port I had designated for WAN (connected to vmbr0). And… success! I reached the iStoreOS login page.

Diagnosis: iStoreOS had, for some reason, reversed the network interface identification compared to OpenWrt. It saw eth0 (connected to PVE’s vmbr1, my LAN bridge) as its WAN interface, and eth1 (connected to PVE’s vmbr0, my WAN bridge) as its LAN interface, assigning the 192.168.100.1 address there.

Resolution: Logged into iStoreOS via the “wrong” port. Navigated to Network -> Interfaces. Edited the LAN and WAN interfaces, swapping their assigned physical devices (eth0 <-> eth1). Saved and applied. Moved my Mac’s cable back to a correct LAN port. Set my Mac back to DHCP. Accessed 192.168.100.1 (the default iStore IP). Success! Changed the iStoreOS LAN IP to 10.0.0.1. Plugged the WAN cable back into the N100’s WAN port.

The network migration was complete in under 15 minutes. The foundation was finally stable. iStoreOS provided the pre-packaged environment I needed. Now, it was time to deploy the services that started this whole journey.

Installing PVE: The First Hurdle

Installing PVE itself is relatively straightforward, similar to installing any Linux distribution:

1. Create Installation Media: Download the PVE ISO image. On my Mac, I used Balena Etcher to write the ISO to a USB drive. My first attempt failed – turns out the old USB drive I grabbed was faulty. A newer drive worked perfectly.
2. Boot from USB: Connect a monitor and keyboard to the N100 box, plug in the USB drive, and boot into the BIOS/UEFI settings (usually by pressing DEL or F2 during startup). Set the USB drive as the primary boot device.
3. PVE Installer: Follow the graphical installer prompts. Key settings during installation:
   • Target Hard Disk: Select the Fanxiang NVMe SSD.
   • Location/Timezone: Set appropriately.
   • Hostname: Set to something memorable, like pve.local.
   • Network Configuration: Crucially, assign a static IP address for PVE’s management interface (e.g., 10.0.0.2), along with the gateway (10.0.0.1 - my eventual router VM’s address) and DNS servers. I chose one of the four physical ports (e.g., enp1s0) for this initial management access.

The installation completed smoothly, and PVE rebooted. I could access the command line directly on the N100, and more importantly, access the web UI from my Mac at https://10.0.0.2:8006. Stage one complete!

The Networking Nightmare: Reconfiguring for VMs

PVE’s default network setup uses one physical port (enp1s0 in my case) solely for its own management. However, to allow my virtual machines (like the router VM) to connect to both the internet (WAN) and my home network (LAN), I needed to create Linux Bridges.

My plan:

• vmbr0: Use a different physical port (e.g., enp2s0) for the WAN connection (connecting to my ISP modem).
• vmbr1: Use the remaining two physical ports (e.g., enp3s0, enp4s0) bridged together for the LAN connection (connecting to my home switch).
• Crucially: Move PVE’s own management interface from the physical port enp1s0 onto the newly created LAN bridge (vmbr1). This way, I could manage PVE from any computer on my home network using the 10.0.0.2 address, freeing up enp1s0 perhaps for other uses.

This reconfiguration is done by editing the /etc/network/interfaces file directly via the PVE console (or SSH). I carefully modified the file, creating the bridge definitions and changing the management interface settings.

I applied the changes (systemctl restart networking or a reboot). And then… disaster.

I could no longer access the PVE web UI at 10.0.0.2. Pings failed. It seemed I had completely locked myself out.

Troubleshooting in the Dark

Panic set in. Had I made a typo in the config file? Did the bridge fail to come up? I had to go back to the “direct connection” method: plug a monitor and keyboard directly into the N100 box.

Logging into the PVE console, I ran ip a to check the network interface status. The output revealed the problem: vmbr1 (my intended LAN bridge and new management interface) had no IP address. My configuration changes hadn’t been applied correctly, or something had gone wrong during the network restart.

Carefully re-examining /etc/network/interfaces line by line, I eventually spotted the error – perhaps a typo, a missing auto vmbr1, or incorrect bridge port settings. I corrected the file using the nano editor (a bit friendlier than vi for quick edits).

After saving the corrected file and restarting the networking service again (systemctl restart networking), I ran ip a one more time. Success! vmbr1 now showed the correct 10.0.0.2 address.

Back on my Mac, I refreshed https://10.0.0.2:8006, and the Proxmox login screen reappeared. Access restored!

This harrowing experience was my first real taste of configuring Linux networking and the importance of meticulousness when editing critical system files. With the PVE host networking finally stable, I was ready for the main event: creating the virtual machine that would become my new home router.

What is a Soft Router?

Unlike traditional “hard” routers with proprietary hardware and software, a soft router is essentially a small, dedicated PC running a specialized router operating system (like OpenWrt, pfSense, OPNsense, or RouterOS). The “x86” refers to the standard PC processor architecture (Intel/AMD), offering significantly more power and flexibility than the ARM or MIPS chips found in most consumer routers.

The Allure of x86

The potential benefits were immense:

• Raw Performance: Capable of handling gigabit+ speeds with complex firewall rules, VPNs, and traffic shaping without breaking a sweat.
• Flexibility: Run virtually any router OS or even standard Linux distributions.
• Expandability: Add more RAM, faster storage, or even specialized network cards.
• Virtualization: This was the game-changer. An x86 platform could potentially run multiple operating systems simultaneously using a hypervisor like Proxmox VE (PVE). I could have my router, a NAS, a media server, and more, all running on one efficient box.

Choosing the Platform: N5105/J4125 vs. N100

My research focused on popular low-power x86 chips commonly used in mini PCs and soft routers:

• Older Generation (Jasper Lake N5105, Celeron J4125): Widely available, often cheaper, proven platforms. Decent performance for routing and light server tasks.
• New Generation (Alder Lake-N N100): Significant performance uplift (especially single-core, crucial for routing), much better integrated graphics (useful for PVE console/light desktop tasks), generally more power-efficient, supports faster DDR5 RAM. Often comes paired with newer Intel i226-V 2.5GbE network controllers, known for better driver support and stability compared to the sometimes-problematic Realtek chips found in cheaper units.

Given the relatively small price difference for new units and the significant performance and efficiency gains, the Intel N100 quickly became the clear winner for future-proofing.

Navigating the Hardware Maze

Choosing the N100 platform opened up a dizzying array of options, mostly from Chinese manufacturers like Beikong (Topton/Kingnovy), CWWK, etc., often sold as barebones systems (no RAM/SSD). Key considerations:

• Ports: How many? What speed? The standard was 4x 2.5GbE Intel i226-V ports, perfect for high-speed LAN and future ISP upgrades. Some offered 2x 2.5GbE + 2x 10GbE SFP+ (fiber) – tempting, but overkill and more expensive for my current needs.
• Cooling: Passive (fanless) is preferred for silent operation, requiring a well-designed chassis.
• RAM/SSD: DDR5 SO-DIMM and NVMe M.2 slots were standard.
• Price: Barebones N100 boxes with 4x 2.5GbE ports were readily available.
• Used Market: While tempting for cost savings, diagnosing potential issues on used hardware felt risky after my R8000 experience.

My Final Choice: Beikong H30W (N100)

After much comparison, I settled on a brand new Beikong H30W N100 barebones unit featuring:

• Intel N100 Processor
• 4x Intel i226-V 2.5GbE ports
• Passive cooling chassis
• DDR5 SO-DIMM slot
• NVMe M.2 slot

To complete the build with maximum cost-effectiveness:

• RAM: I salvaged a compatible 16GB DDR5 stick from an old laptop – a huge saving!
• SSD: I purchased a new, budget-friendly but well-regarded Fanxiang S500 Pro 256GB NVMe SSD. Plenty of space for PVE and multiple VMs.

The Grand Vision: Virtualization

With the hardware assembled, the real excitement began. This N100 wasn’t just going to be a router. It was going to be the heart of my home lab. The plan: install Proxmox VE (PVE) as the bare-metal hypervisor, then create virtual machines for:

1. OpenWrt/iStoreOS: My primary router, firewall, and network manager.
2. OpenMediaVault (OMV): A NAS for file storage and backups.
3. Future Possibilities: Home Assistant, AdGuard Home, a web server, maybe even AI experiments?

This N100 box represented a convergence of all my needs and aspirations. The hardware was ready. Now came the software challenge: installing and taming the Proxmox hypervisor.

Diving Deep into IPv6 and Network Structure

After some initial head-scratching, research pointed towards a concept called Prefix Delegation (PD). In simple terms, my ISP assigns a block (prefix) of IPv6 addresses to my router, and the router is then responsible for distributing individual addresses from that block to my devices. The symptom – WAN gets an address, LAN doesn’t – strongly suggested PD was failing.

Why? The prime suspect became my ISP’s modem (Optical Network Terminal or ONT). Was it running in router mode instead of bridge mode? If the modem itself was acting as a router and didn’t properly support delegating the prefix downstream, my AX6000 would never receive the necessary block.

I confirmed my network topology: ONT (in router mode) -> Switch -> Multiple Routers (including the AX6000). Unfortunately, due to other devices directly connected to the ONT via the switch, simply switching the ONT to bridge mode wasn’t feasible without restructuring everything.

As a temporary workaround, I enabled NAT6 on the AX6000. Success! My devices could now access IPv6 websites. However, I knew this wasn’t true end-to-end IPv6. NAT6 essentially translates internal IPv6 addresses, similar to how traditional NAT works for IPv4. It solved the immediate browsing issue but didn’t address the underlying PD failure and wouldn’t help with services requiring direct public IPv6 reachability.

Enter the Double NAT Monster

Around the same time, my Xbox started complaining: “Double NAT detected.” This made perfect sense. My network traffic was going through two layers of Network Address Translation: first at the AX6000, and then again at the ISP’s ONT (which was also acting as a router). This configuration is notorious for causing issues with online gaming, peer-to-peer connections, and services requiring open ports.

We discussed potential mitigations like UPnP, manual port forwarding, and placing the AX6000 in the ONT’s DMZ, but none felt like a clean solution.

The CGNAT Revelation

The real blow came when investigating the Double NAT further. By comparing the WAN IP address shown on my ONT’s status page with the public IP address reported by websites like “whatismyip.com”, I confirmed a dreaded reality: I was behind Carrier-Grade NAT (CGNAT). My ISP wasn’t assigning my ONT a unique, public IPv4 address. Instead, I was sharing one with potentially hundreds of other customers.

[Placeholder: Add a screenshot here showing the discrepancy between your ONT’s WAN IP (likely a private 10.x or 100.x address) and your public IP, confirming CGNAT.]

This revelation made two things crystal clear:

1. Solving the Xbox NAT issue reliably via IPv4 was going to be nearly impossible without resorting to complex (and often unreliable) tunneling solutions.
2. Getting native, public IPv6 working correctly was no longer just desirable; it was essential for any kind of future-proof home networking or self-hosting.

Back to the IPv6 Root: The Faulty ONT

With renewed focus, the investigation circled back to the ONT. I confirmed the ONT itself was receiving a /64 or similar public IPv6 prefix starting with 240e:. The problem was solely its inability to delegate this prefix downstream.

Analyzing the ONT’s configuration page revealed a peculiar “Port Binding” setting. Initially, only LAN1 was bound for internet services.

I modified the settings to bind all LAN ports. Hope surged… but was quickly dashed. Even after ensuring all ports were bound, testing with the AX6000 and other routers on different ports yielded the same result: no PD, no LAN-side public IPv6.

The conclusion was unavoidable: the ONT’s Prefix Delegation functionality was faulty, likely due to a hardware limitation or firmware bug. Relying on it was a dead end.

I needed a router powerful and flexible enough to potentially work around the ONT’s limitations, or perhaps even replace its routing functions altogether. This led me to revisit an old piece of hardware sitting in my closet… a supposedly bricked Netgear R8000 running OpenWrt. Could it be resurrected?