Test problem with slow uplink ISPs such as DSL

[Sean]

From my testing so far, the Beta appears to work well with my 4G based Internet connection at home.  However, when I managed to give it a quick test run at my workplace, the beta kept delivering speeds under 1Mbps down in Chrome even though they have a 10Mbps DSL connection.

 

From further testing at home and setting upload / download limits on my MikroTik router, I found that when I set the upload speed to 512Kbps to match the DSL uplink at my workplace, I am able to replicate the issue here and also uncovered a few other small issues.

 

With my workplace DSL connection the following is the Beta test followed by the linear test in Chrome with the UK server:

 

Retest with the German server.  As I write this post, I see the up/down rows are swapped on the left. 🙃

 

I did one more test in Edge and although it performed better than Chrome, the upload and download was still around half the linear download test:

 

Other observations:

 

The Beta test does not mention it being a multithread test in the test result.  For comparison, the non-beta multithread test mentions "Multithread":

 

Download tests with a block size under 1MB incorrectly show the KB as MB in the test results page.  For example, the following test result on the left shows a test block of 205kB, however in the test results page, it shows "205 MB":

If the download or upload test is unable to fetch all the blocks, it gets stuck.  This happened a few times, probably due to the small packet loss on my 4G connection, such as the following screenshot where it endlessly waited here for the final 2 kB block.

 

 

 

[CA3LE]

Can you please run a few manual tests for me, selecting 10 MB using the beta?

[xs1]

interesting.. i cannot reproduce this.   Linear or automatic, my results seem to be the same.. 

 

 

[Sean]

11 hours ago, CA3LE said:

Can you please run a few manual tests for me, selecting 10 MB using the beta?

I tried retesting with my router upload limited to 0.5Mbps up (to mimic my workplace DSL uplink) and the 10MB manual download block performs better giving about 7-8Mbps in Chrome after a few tests.  With a 102MB manual block it gets around 50Mbps:

With linear, it gives around the full download speed like when I do not have the upload throttled to 512Kbps. 

 

1 hour ago, xs1 said:

interesting.. i cannot reproduce this.   Linear or automatic, my results seem to be the same.. 

Going by your test results above, it appears you either did not limit the upload speed to 512Kbps on your gateway / router (to mimic a slow 512Kbps DSL upload) or it was measuring the 512Kbps upload limit as 39.2Mbps. 

 

The following shows the 512Kbps upload configured on my router to mimic the 512Kbps upload limit on my workplace's DSL connection.  With MikroTik routers, FastTrack must be disabled (IP -> Firewall) for speed limits to take effect. 

 

[CA3LE]

With a slower upload speed or higher latency it interferes with the normal flow.  If the requests are lagged it will affect the result.

 

With 100+ smaller requests the connection has to negotiate each one.   The latency and upload can effect this.

 

This will be less pronounced with linear because we don't have to keep initiating requests over and over.  

 

I went around and around with this one, trying to get those connections to ramp up quicker.  Originally I was trying to make the test ramp up quicker by adjusting test parameters for that situation.  Then realized that it's only doing what it's supposed to do.  This happens when the connection is weak, it's only showing you what happened.  If something slows down the requests or the process... it affects the end result.

 

So keep in mind when you're using the beta, it's splitting the multithread process much more than my previous version.  100 elements for < 1MB tests and 200 elements beyond that, where the production multithread at 10MB you only open 12 threads and 200MB opens 30 threads.  Big difference.  The beta is more demanding.

 

The difference is before I adjusted the process to meet the connection.  Smaller tests were done with less elements.  I've decided going forward that TMN shouldn't scale based on the connection, rather measure every connection the same.  As the linear test does.  Remember I'm only talking about the multithread process.

 

The beta upload test works the same way, 100 and 200 elements.

 

A couple of things to can do.  Click [customize] and Enable Linear Boost or test linear on connections like that one.

 

21 hours ago, Sean said:

If the download or upload test is unable to fetch all the blocks, it gets stuck.  This happened a few times, probably due to the small packet loss on my 4G connection, such as the following screenshot where it endlessly waited here for the final 2 kB block.

 

I've seen that too, always on crappy connections.  I think you're right about it being due to packet loss.  I'm going to see about detecting when a thread gets stuck like that and then reinitiate that thread and report the event in the results.

 

It's all about how the data is being rendered.  The beta is an entirely new test with different variables.  These new variables seem to favor more modern connection types because they're better designed for this type of load.  A bunch of small requests may be harder to render in some cases than a few large ones.  But that's what we're here to test.

[xs1]

8 hours ago, CA3LE said:

I've seen that too, always on crappy connections.  I think you're right about it being due to packet loss.  I'm going to see about detecting when a thread gets stuck like that and then reinitiate that thread and report the event in the results.

 

It's all about how the data is being rendered.  The beta is an entirely new test with different variables.  These new variables seem to favor more modern connection types because they're better designed for this type of load.  A bunch of small requests may be harder to render in some cases than a few large ones.  But that's what we're here to test.

 

@CA3LE  I have noticed since slowing down to Comcast, that quite a few times now, primarily on the download test, that it will ccrruuuzzeee to the end, than pause before the result, thus skewing the test (slower) results.   Same pc, browser, & settings as before with Frontier Commy-unications. Possible correlation?