Adding ER-X between Cable Modem and Amplifi for SQM

  • This may be of interested to some. I recently added a TP-Link Easy managed switch for my wired devices so I could do 4 priority levels for the ports, so now I have basic QoS on my wired devices.

    The next step was to add an Edgerouter between my Bridged Cable Modem and Amplifi HD so I could have some sort of SQM to help with my bufferbloat rating (mine ranged from D to E).
    But I didn't want to bridge my Amplifi HD and loose all it's functionality and easy management from the app.

    After a couple nights and testing different setups and settings I think I found the one that works best for my setup. I tried many different variations to see how different settings reacted with each other.

    Per Edgerouter (ER-X):
    Pay for 150/15
    When speed testing 280/18
    D-E bufferbloat rating (it spikes as high at 650ms)
    Bridge Cable Modem -> Amplifi HD (with 2 ethernet backhaul Amplifi HDs) -> TP-Link Easy Managed switch (for wired QoS per port)

    Post Edgerouter (ER-X):
    When speed testing 150-170/15
    A+ bufferbloat rating (max spike is about 35ms)
    Bridge Cable Modem -> Bridge ER-X -> Amplifi HD (with 2 ethernet backhaul Amplifi HDs) -> TP-Link Easy Managed switch (for wired QoS per port)

    Here's how I configured the ER-X (from what I can remember, I tried many different setups).

    -Used the Wan+2Lan Wizard so I could leave eth0 as the main router port and plug my laptop into to monitor and setup the router (otherwise you have no way to access it from behind your Amplifi HD since it's in front of it).
    eth1 was automatically setup as the WAN and then eth2, eth3, eth4 were dropped onto switch0 automatically.

    -Changed switch0 to No IP and removed eth3 & eth4 from the switch0

    -Disabled eth3 and eth4 ports since I'm not using them

    -Removed DHCP from from eth1 (set it at No IP)

    -Made a Br0 bridge and set it as DHCP and connected eth1, eth2 & switch0 to it (very important you connect switch0 otherwise you get slower speeds with SQM for some reason).

    -Enabled hwnat and ipsec offloading through CLI (the only thing I used CLI for)

    -I believe there was a DNS server for switch0 that the wizard made also, so I disabled that since I don't need it.

    Now I had my public IP showing on Br0, and with my Amplifi HD plugged into eth1 my public IP was being passed directly onto it and all my devices had internet.

    With no SQM I would get my normal 280/17 and D bufferbloat rating. So I confirmed the ER-X was doing nothing more than passing the internet through it.

    I then turned SQM on eth1, set it at 1 Mp up and 1 Mb down because I wanted to make sure it was working. Ran a speed test and didn't get more than 1Mb. Which is perfect because I've now confirmed it can apply SQM to a bridge interface.

    I played with my SQM up and down numbers until I got an A+ on my bufferbloat rating.
    I set the up to 17. But the down it didn't matter how high I set it as I always got an A+ rating as long as my up was set at 17 or lower. My down speeds were always 150-180 as long as I set the down higher than 200. So I know that I'm maxing out the Edgerouters SQM CPU capabilities on the down (the CPU was hitting around 70%, which seems like it has more headroom but I couldn't get anymore than 150-180 down out of it).

    I'm thinking if I go to an ER-12 that has a better CPU I'll be able to get some down speed back. But for now I'm pretty happy with 150-180 down.

    I ran a couple rounds on my sons Fortnite to watch the pings (sorry, easiest way for me to see lag and ping spikes, there's definitely better ways to test).
    It had the lowest pings I've ever seen, normally it sits around 35-40ms, except when my NAS or desktop upload it would spike as high as 200ms. While the NAS was uploading last night consuming all the up bandwidth as usual, the Fortnite pings sat at an incredibly stable 27-30ms. I've never seen them so stable, sometimes even dropping as low as 19ms which I've never seen before.
    It was only 1 test though late last night when I finished setting it up so maybe the network was just "quiet", so I have more tests to run and want to see what it does when the desktop uploads which used to cause the worst ping spikes. I should know for sure over the next few days.

    We also have a nightly incoming backup from Filezilla on one of our NAS and our other NAS backups offsite to another QNAP and both backups had no issue getting through the ER-X -> Amplifi.

    Showing promising results so far.

    Oh and more more thing. I was able to enable DPI and see all the traffic on my system. If didn't seem to disable SQM either because I still got an A+ rating when I tested, but the CPU took a hit and speeds where slower as would be expected. I did leave it off though as I don't really need it, but just cool to see as I've never had anything that would break down traffic and show you exactly what your system is doing.

    Hope this helps for anyone who would like some SQM on the Amplifi but don't want to loose all it's features by bridging it.

  • Thanks @LainB - nice write up, great results...and some some déjà vu

  • Quick update.
    The ER-X has been running perfect for the last 4 days doing SQM on my non-bridged Amplifi HD. Super impressed with it.

    I let my son put in a lot of Fortnite time on the weekend so he could report back the pings to me, if there's any issues I want to know about it as soon as possible. It was pretty staple around 20-30ms the whole weekend, better than it's ever been. The lowest he saw it was 16ms, which he's never seen before.
    I still have to watch the ping when the desktop uploads, but the NAS was offsite backuping pretty much all weekend (there was a huge backup to do, about 200GB of work data completely saturating the upload) and didn't make a dent in his Fortnite pings.

    I've ordered an ER-12 to see if it's better CPU will get me higher than 150-180Mbps down.
    It's due in at the end of the month and I'll report back once I've tested it.
    (The ER-X will be going to my work since it's only a 20/20 connection).

    If Amplifi added SQM into the HD and/or the Alien it would be a game changer. Although I do think some home routers have SQM/fq_codel now. I'm kind of surprised a $70 ER-X router can pull it off where much more expensive routers can't.

  • Last update (unless I have issues).

    • I just saw that I actually removed everything from switch0 not just eth3 & eth4, so it is empty, but I needed to still add it as part of the bridge to get my 150-170Mbps of down speed.'
    • I was able watch the pings when the desktop was uploading. Sat around 30-40ms, normally would go to 100-150ms solid and sometimes spike as high as +200ms. So the ER-X is working perfect!

    My ER-12 came in way sooner than expected, it arrived yesterday and I just got it setup last night.

    Short version - it works awesome. I'm now around 250-270Mbps down (and still 17Mbps up which is my upload cap). So it got me about another 100Mbps down. I'm not sure if I'm maxing out the CPUs capabilities or if it's my down speed that's being max (which when I test without SQM I get about 280-290). But either way upgrading to the ER-12 got me a lot more down speed.

    Long version - I thought it would be an easy drop in, set it up the same as the ER-X.... was I wrong! I tried the WAN-2LAN wizard like I did with the ER-X so I could plug my WAN into eth1, eth0 would be the ER-X login port and eth2 would go to my Amplifi. I could not get any internet to pass through to any ports. I'm not a network wizard though, so maybe I was missing something, but with the ER-X I had internet on those ports right away. I even looked at the ER-X and ER-12 side by side and they were setup identical.

    I then noticed that the WAN-2LAN only setups up eth2, eth3 & eth4 as the 2nd LAN just like the ER-X. It makes no mention of all the other ports on the ER-12. So I'm wondering if WAN-2LAN is from the older ER-X days and when the ER-12 came out it was never updated, so maybe it's not compatible with the ER-12. Just a thought, but either way it burnt about 3 hrs of my night trying to get WAN-2LAN to work and even tried different firmwares.

    I then used the Basic setup which puts internet on eth9 and then used eth8 as my ER-12 login port. And pluged Amplifi into eth1.
    I was then able to proceed like I did with the ER-X, making the bridge, changing the DHCP, disabling the 2nd LAN, ect, and got it working great.

Log in to reply