RandomMemory wroteSo Instead of getting multiple network cards and adapters I can get a solid switch and do VLANs on 1 port?
Yes possible, MACVLAN may be simpler as OMR use it by default. If you want to assign VLAN, you have to write the interface name in "Add new interface @network page", followed by dot then vlan number. Ex: eth0 is the port, eth0.55 = VLAN tag 55. VLAN may affect MTU, though OMR constantly adjusts the MTU every 30 seconds per interface, I haven't tested the performance with VLAN, though I doubt it will affect it, I think it's only 4 bytes, performance is only a concern with VLAN due to it being software VLAN, unlike dedicated switches which have 0 impact. RPi3 ethernet switch hardware accelerated VLAN mode is quirky for example, even Intel based PCI ethernet suck. My setup: RPi3 + 10 dollar 5 port unmanaged gigabit network switch, you can also use any scrapped router as unmanaged switch. I recommend for each ISP to have it's own modem/router, e.g don't use TerraNet PPPoE connection in OMR. TerraNet WDSL gave me the credentials on paper btw, they asked if I needed a router. OMR doesn't like network adapters going down, PPPoE may go down due to provider issues, thus interrupting all connections (other ISPs), static IP only is advised, and let a cheapo router do the PPPoE, just like any ADSL modem.

TPL:
Terranet dish (ethernet)---> router --->
switch ---> OMR RPi3
Ogero phone line ----> modem ---> ^
|
Dumb bridged WiFi APs --------------------------
|
Ethernet devices ----------- switch ---------------
|
(optional depending on main switch no. of ports)

Total cost excluding WiFi APs was around 90 USD back then. (incl. extra Gb switch)
Okay, so I was being scammed I guess by the reseller, Ended up calling terranet and getting a direct subscription with them, the installation is scheduled for the 10th so I'll have to test all this stuff out when the connection is ready. How is Terranet's non parallels download speeds. I suffer with large github repos downloading at 40KB/s because github doesn't allow multiple connections while downloading a single zip.
RandomMemory wroteOkay, so I was being scammed I guess by the reseller, Ended up calling terranet and getting a direct subscription with them, the installation is scheduled for the 10th so I'll have to test all this stuff out when the connection is ready. How is Terranet's non parallels download speeds. I suffer with large github repos downloading at 40KB/s because github doesn't allow multiple connections while downloading a single zip.
2Mbit plan, you can see the bit burst effect, overshooting 2.2Mbit due to their PCQ limiter (to keep latency down during congestion)
It also tricks dumb speed testing services like the famous Ookla one during congestion to show 2Mbit, while in reality it's around 1.8Mbit average during peak hours, I had the same effect with 10Mbit plan. I recommend using DSLReports for testing.


(LuCI does smoothing to the graph displaying wrong average unless I run it for 1 hour +)

This is to all Europe servers, doing so with far west American server gets it down to around 200, you can use Cloudflare WARP+ (argo) Wireguard profile as these act like a middle box and tailor TCP settings per client, most VPN providers don't. This issue is universal to any ISP around the world, and it's exaggerated on 100mbit+ connections.
Beta0 wrote
RandomMemory wroteOkay, so I was being scammed I guess by the reseller, Ended up calling terranet and getting a direct subscription with them, the installation is scheduled for the 10th so I'll have to test all this stuff out when the connection is ready. How is Terranet's non parallels download speeds. I suffer with large github repos downloading at 40KB/s because github doesn't allow multiple connections while downloading a single zip.
2Mbit plan, you can see the bit burst effect, overshooting 2.2Mbit due to their PCQ limiter (to keep latency down during congestion)
It also tricks dumb speed testing services like the famous Ookla one during congestion to show 2Mbit, while in reality it's around 1.8Mbit average during peak hours, I had the same effect with 10Mbit plan. I recommend using DSLReports for testing.

https://i.ibb.co/bBmxnLy/image.png
(LuCI does smoothing to the graph displaying wrong average unless I run it for 1 hour +)

This is to all Europe servers, doing so with far west American server gets it down to around 200, you can use Cloudflare WARP+ (argo) Wireguard profile as these act like a middle box and tailor TCP settings per client, most VPN providers don't. This issue is universal to any ISP around the world, and it's exaggerated on 100mbit+ connections.
Thank you for your detailed responses. Much appreciated.
My final solution was to buy an extra 1 port network card for the terranet incoming connection. I installed proxmox on an i7 3770 computer. Then Installed pfsense and openmptcrouter in proxmox, pfsense handles the 2 incoming WANs, then made 2 rules in the firewall to make 2 certain IPs use WAN1 and WAN2 accordingly, Then I made 2 virtual interfaces in openmptcp and used those are incoming WAN connections and passed the bonded connection back to Pfsense as a third WAN to distribute it over the house with DHCP. Was very fun. (I still didn't get the terranet installation, but when I do it will be plug and play)
RandomMemory wroteMy final solution was to buy an extra 1 port network card for the terranet incoming connection. I installed proxmox on an i7 3770 computer. Then Installed pfsense and openmptcrouter in proxmox, pfsense handles the 2 incoming WANs, then made 2 rules in the firewall to make 2 certain IPs use WAN1 and WAN2 accordingly, Then I made 2 virtual interfaces in openmptcp and used those are incoming WAN connections and passed the bonded connection back to Pfsense as a third WAN to distribute it over the house with DHCP. Was very fun. (I still didn't get the terranet installation, but when I do it will be plug and play)
That's nice! Did you use static routing or double nat (port forwarding all ports)?
I ended up doing double nat and made a DMZ interface on pfsense
14 days later
Terranet installed today for me, this is the current speed with another bonded WAN that is 4mbps at this time of day
RandomMemory wroteTerranet installed today for me, this is the current speed with another bonded WAN that is 4mbps at this time of day https://www.speedtest.net/result/10627567849.png
Interesting, was this done with "Single" mode?
What is the monthly cost for this setup? I'm paying 120k LBP for Ogero openspeed(200gb) + Terranet 2mbit. Getting aggregated 15 download and 3 up.,
Well for some reason the VPN tunnel always shows as down but it's still bonding if that's what you mean by single mode. This setup is expensive for someone who is not working remotely or getting paid in USD. It's 150k LBP for local wireless guy where the speed goes from 20mbit at 5 am to 7 at 12 pm to 3.5 after 4 pm. Terranet is 12mbps at 270K LBP. so I guess around 420 a month. For me, this lets me do my work extremely efficiently and removes every headache I've had for the past 5 years, so 100% worth it. Also local cable resellers seem to always have high upload speeds for some reason and ogero says my max line speed is 1mbps (imagine 1mbps in 2020)
RandomMemory wroteWell for some reason the VPN tunnel always shows as down but it's still bonding if that's what you mean by single mode. This setup is expensive for someone who is not working remotely or getting paid in USD. It's 150k LBP for local wireless guy where the speed goes from 20mbit at 5 am to 7 at 12 pm to 3.5 after 4 pm. Terranet is 12mbps at 270K LBP. so I guess around 420 a month. For me, this lets me do my work extremely efficiently and removes every headache I've had for the past 5 years, so 100% worth it. Also local cable resellers seem to always have high upload speeds for some reason and ogero says my max line speed is 1mbps (imagine 1mbps in 2020)
Wow that's pretty bad, 420K is very high, VPN down but UDP is still aggregating? Just an info for others on how this works:


| WAN 1____________________
| |
VPS __| TCP to MPTCP by the kernel |__Shadowsocks TCP proxy__Glorytun TCP VPN
| WAN 2___________________| | |
|______________________|
\ Firewall splitting /
|__________________|
TCP Everything else minus TCP
\ /
LAN


TCP through VPN:
--------------------|\ VPN /------- MPTCP in this case is congested and
--------------------| }----------{-------- doing paths on one TCP connection
--------------------|/ ------- via VPN, lowering performance.
TCP through Proxy:
--------------------|\ ---Proxy--/-------Side note:Glorytun TCP VPN which is
--------------------| }-----------{-------- responsible for (UDP and others) is the only path
--------------------|/ ------------------- that is congested.(one TCP connection for possibly hundred UDP)

Shadowsocks is used to implement "TCP through Proxy"

UDP based clients are smart enough to be not affected by this, they implement their own packet flow algo, unlike TCP which is a very old standard.
OMR is configured to prioritize Glorytun by marking it with DSCP, then the prioritization is done and detected at Shorewall on the VPS.
what do you recommend to fix it. How is my IP showing up from OVH, is that the proxy's work?
RandomMemory wrotewhat do you recommend to fix it. How is my IP showing up from OVH, is that the proxy's work?
Sorry, late edit. See above, I recommend hitting Save and apply in the wizard again, see if that fix it, else look in the System log to see if there is any error.
And yeah, most websites use HTTP1/2 which is TCP, what shadowsocks serve. Youtube, google and few others with the latest Chrome (Windows) are using QUIC/HTTP3 which is UDP, but it fallbacks to HTTP2 if UDP/QUIC is blocked or unsupported. OMR actually blocks QUIC and HTTP3 since TCP over MPTCP performs better in this setup instead of being tunneled via the UDP VPN (glorytun). QUIC/HTTP3 is better on UDP only VPN or direct connection.
RandomMemory wroteWell for some reason the VPN tunnel always shows as down but it's still bonding if that's what you mean by single mode. This setup is expensive for someone who is not working remotely or getting paid in USD. It's 150k LBP for local wireless guy where the speed goes from 20mbit at 5 am to 7 at 12 pm to 3.5 after 4 pm. Terranet is 12mbps at 270K LBP. so I guess around 420 a month. For me, this lets me do my work extremely efficiently and removes every headache I've had for the past 5 years, so 100% worth it. Also local cable resellers seem to always have high upload speeds for some reason and ogero says my max line speed is 1mbps (imagine 1mbps in 2020)
Terranet is 12Mbps all day? or does it fluctuate? what is its upload speed alone?
wollyka wrote
RandomMemory wroteWell for some reason the VPN tunnel always shows as down but it's still bonding if that's what you mean by single mode. This setup is expensive for someone who is not working remotely or getting paid in USD. It's 150k LBP for local wireless guy where the speed goes from 20mbit at 5 am to 7 at 12 pm to 3.5 after 4 pm. Terranet is 12mbps at 270K LBP. so I guess around 420 a month. For me, this lets me do my work extremely efficiently and removes every headache I've had for the past 5 years, so 100% worth it. Also local cable resellers seem to always have high upload speeds for some reason and ogero says my max line speed is 1mbps (imagine 1mbps in 2020)
Terranet is 12Mbps all day? or does it fluctuate? what is its upload speed alone?
I'll update the image at night if vnstat does not clip few hours, this is from yesterday 12AM till now, wan2 is terranet 2mbit plan. I would be interested to see if the 12mbit plan is the same, I had 10mbit for 2 months with no issues but that was almost 2 years ago.
Downloading MSFS whopping 150GB :)

I expect direct usage will maintain 2mbit average, this is proxied over MPTCP, single flow.
Beta0 wrote
RandomMemory wrotewhat do you recommend to fix it. How is my IP showing up from OVH, is that the proxy's work?
Sorry, late edit. See above, I recommend hitting Save and apply in the wizard again, see if that fix it, else look in the System log to see if there is any error.
And yeah, most websites use HTTP1/2 which is TCP, what shadowsocks serve. Youtube, google and few others with the latest Chrome (Windows) are using QUIC/HTTP3 which is UDP, but it fallbacks to HTTP2 if UDP/QUIC is blocked or unsupported. OMR actually blocks QUIC and HTTP3 since TCP over MPTCP performs better in this setup instead of being tunneled via the UDP VPN (glorytun). QUIC/HTTP3 is better on UDP only VPN or direct connection.
Thanks for the great read... I tried changing all settings and for some reason none of them were able to put the VPN tunnel up again. This morning there was an update. I did an update on the server and it's working now (for some reason it's saying terranet has multipath blocked.



this is my speedtest right now with terra being at 12mbps and the other ISP almost dying...



I am only keeping the other ISP because they have 20mbps post 2 am and 15mbps cache on all popular services all day (netflix, youtube)
wollyka wrote
RandomMemory wroteWell for some reason the VPN tunnel always shows as down but it's still bonding if that's what you mean by single mode. This setup is expensive for someone who is not working remotely or getting paid in USD. It's 150k LBP for local wireless guy where the speed goes from 20mbit at 5 am to 7 at 12 pm to 3.5 after 4 pm. Terranet is 12mbps at 270K LBP. so I guess around 420 a month. For me, this lets me do my work extremely efficiently and removes every headache I've had for the past 5 years, so 100% worth it. Also local cable resellers seem to always have high upload speeds for some reason and ogero says my max line speed is 1mbps (imagine 1mbps in 2020)
Terranet is 12Mbps all day? or does it fluctuate? what is its upload speed alone?
It's a symmetrical up and down. I have the connection for 2 days and freaking amazing. it's a consistent 11-11.5mbps all day and it's unlimited. I also have their IPTV subscription running on 2 TVs which takes 6mbps on each TV but the route is different from your internet so it will use up intranet bandwidth instead of internet bandwidth.
RandomMemory wrote
wollyka wrote
RandomMemory wroteWell for some reason the VPN tunnel always shows as down but it's still bonding if that's what you mean by single mode. This setup is expensive for someone who is not working remotely or getting paid in USD. It's 150k LBP for local wireless guy where the speed goes from 20mbit at 5 am to 7 at 12 pm to 3.5 after 4 pm. Terranet is 12mbps at 270K LBP. so I guess around 420 a month. For me, this lets me do my work extremely efficiently and removes every headache I've had for the past 5 years, so 100% worth it. Also local cable resellers seem to always have high upload speeds for some reason and ogero says my max line speed is 1mbps (imagine 1mbps in 2020)
Terranet is 12Mbps all day? or does it fluctuate? what is its upload speed alone?
It's a symmetrical up and down. I have the connection for 2 days and freaking amazing. it's a consistent 11-11.5mbps all day and it's unlimited. I also have their IPTV subscription running on 2 TVs which takes 6mbps on each TV but the route is different from your internet so it will use up intranet bandwidth instead of internet bandwidth.
I wasn't aware that their IPTV also works on WDSL with separate bandwidth (unlike DSL), thanks for the heads up.
Ignore the Multipath warning, the script tends to do false negatives on heavy load, I'm surprised the author re-enabled it. You can always check if MPTCP is filtered in the MPTCP page. You should see "TCPoptionmptcp capable = sender's key". I hope it's not pfSense that is doing any filtering.
Beta0 wrote
RandomMemory wrote
wollyka wrote Terranet is 12Mbps all day? or does it fluctuate? what is its upload speed alone?
It's a symmetrical up and down. I have the connection for 2 days and freaking amazing. it's a consistent 11-11.5mbps all day and it's unlimited. I also have their IPTV subscription running on 2 TVs which takes 6mbps on each TV but the route is different from your internet so it will use up intranet bandwidth instead of internet bandwidth.
I wasn't aware that their IPTV also works on WDSL with separate bandwidth (unlike DSL), thanks for the heads up.
Ignore the Multipath warning, the script tends to do false negatives on heavy load, I'm surprised the author re-enabled it. You can always check if MPTCP is filtered in the MPTCP page. You should see "TCPoptionmptcp capable = sender's key". I hope it's not pfSense that is doing any filtering.
take a test from here https://terravision.com.lb/. make sure the device you are testing on has a terranet IP
RandomMemory wrote
wollyka wrote
RandomMemory wroteWell for some reason the VPN tunnel always shows as down but it's still bonding if that's what you mean by single mode. This setup is expensive for someone who is not working remotely or getting paid in USD. It's 150k LBP for local wireless guy where the speed goes from 20mbit at 5 am to 7 at 12 pm to 3.5 after 4 pm. Terranet is 12mbps at 270K LBP. so I guess around 420 a month. For me, this lets me do my work extremely efficiently and removes every headache I've had for the past 5 years, so 100% worth it. Also local cable resellers seem to always have high upload speeds for some reason and ogero says my max line speed is 1mbps (imagine 1mbps in 2020)
Terranet is 12Mbps all day? or does it fluctuate? what is its upload speed alone?
It's a symmetrical up and down. I have the connection for 2 days and freaking amazing. it's a consistent 11-11.5mbps all day and it's unlimited. I also have their IPTV subscription running on 2 TVs which takes 6mbps on each TV but the route is different from your internet so it will use up intranet bandwidth instead of internet bandwidth.
Thanks! I think it will be a great solution in Mansourieh.

The local cable guy is slow and alfa 4G bundles are way to expensive and very limited in quota! How much the installation costed you? I am willing to pay 270K sine it will help with remote work and it will be cheaper than 4G!