Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Significant performance difference between upload and download aggregation #88

Open
pipe2null opened this issue Aug 17, 2023 · 1 comment

Comments

@pipe2null
Copy link

Hello, and thank you for all the awesomeness of OMR!

One issue (as in title) and a few related Q's.
Upload appears to aggregate bandwidth very well nearly to the full sum of both ISP's available upload bandwidth, but download is usually slower than any single ISP connection's bandwidth. Realtime Traffic monitor shows that download aggregation IS working and aggregating, but the VPS->OMR traffic is not utilizing the available bandwidth determined by regular single-ISP speedtests. Based on how effective upload aggregation is working, I would expect download aggregation to yield ~80% better performance than it currently is.

As I was playing with settings, there was one time that the planets aligned and download aggregation worked perfectly and was a truly beautiful sight seeing an overall double of my normal single-ISP bandwidth, but upload performance was really bad, so kept playing with settings and I have not been able to reproduce the same download performance since with any of the many combinations of settings I have tried. NOTE the VPS-side inconsistent state mentioned below, which took me a while to figure out.

VPS is Debian 11 installed with current official VPS install script
Hosting on IONOS: 2 Core, 2GB, 80GB SSD, unlimited 1Gbit, $5/mo
OMR x64 non-EFI ext image: v0.59.1-5.4 r0+16594-ce92de8
Currently multi-nat, but working to get IP passthrough setup for both ISPs.
Speedtests via speedtest.net
ISPs: Starlink and TMobile5G

AFAIK more-or-less best perf config I've found for my widely varying minute-to-minute latency and bandwidth, excellent upload but still has above download aggregation issue: Fullmesh+RoundRobin+Nanqinlang

In general:
I've found that the current official VPS installation can get into a dirty inconsistent state when playing with OMR router-side settings attempting to find the optimal config, especially when trying different options under "Show advanced settings == checked". Even if you change router settings back to default, or even do a fresh install of the router and push default settings to VPS via the Wizard, the overall setup has really bad performance after VPS gets into a dirty/inconsistent state. The only way I've found that fixes this is to nuke the VPS and do a fresh install, and even without changing a thing on the router-side the extremely poor performance problem goes away, the above up/download speed problem still exists but overall perf is 10x better compared to the inconsistent state. So it seems to me that the VPS gets into a dirty inconsistent state leading to poor performance. Which also prompts me to believe that the above download aggregation issue might be VPS-side.

Q's:

  • I noticed you've been working hard on getting Debian 12 VPS directly supported (with it's already enabled MPTCP kernel). Any idea when your VPS install script might be stable enough to use?
  • Is there a way to restore VPS state to a known good clean condition without a full VM nuke and reimage/resinstall?
  • What should I try doing to fix the download aggregation issue?

Thanks!

@Ysurac
Copy link
Owner

Ysurac commented Aug 18, 2023

Issues should be on https://github.com/Ysurac/openmptcprouter/issues
This part is only on VPS script issues.

There is often more upload issues than download issues. You should not use RoundRobin and Nanqinlang, like BBR, give bad result when connections are not stable. Default or Blest scheduler should give a good result en BBR2 should be good.
What is the proxy used ? By default it's shadowsocks that should give good download speed, v2ray use more CPU but it also provide UDP instead of using a VPN for that.
If you have MPTCP over VPN enabled, disabled this.
If you have SQM enabled, check download/upload speed.

Debian 12 is supported on snapshot VPS script with snapshot router 6.1, it's still in testing.
You can run VPS script with UPDATE="no", but not sure this will reset all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants