why the big difference for multithread test results?

[ginahoy]

I'm new to the forum. My ISP is wireless (microwave?) and I pay for 5mbs download. Over the 1-1/2 years or so since I discovered TMN, I consistently get lower results with TMN than with other speed test sites. I read in another discussion here that setting multithread might produce different results. It did. Whereas my default test results were typically between 1 & 3 mbs, the multithread tests I ran this evening all came back at ~4.7 mbs. Why is that? 

 

I have an unrelated question, if I may... Now that I registered with TMN (to post this question), I noticed my previous test results are only accessible when I log out. I'm not sure how my system is identified by the site, but it seems like it would be able to consolidate my previous tests into my new account, no?

[Pgoodwin1]

When you look at your results, select the ConnectID link and it will show your prior results. It looks like there are results going back to about a year ago under that ConnectID.

 

interesting that your multithread test results are better than your regular test results. On mine, I always get lower results when multithreading. I assume that your non-multithread tests were when testing to the TMN Test Server located the closest to you among all the TMN test servers.

[ginahoy]

4 hours ago, Pgoodwin1 said:

When you look at your results, select the ConnectID link and it will show your prior results. It looks like there are results going back to about a year ago under that ConnectID.

 

Yup, I didn't look very hard did I ?

 

Quote

interesting that your multithread test results are better than your regular test results. On mine, I always get lower results when multithreading. I assume that your non-multithread tests were when testing to the TMN Test Server located the closest to you among all the TMN test servers.

 

My automatic (non-multi) tests always select Dallas, although geographically I'm a bit closer to LA.  I guess I don't understand what multithread means in this context. What is the rationale behind selecting multithread?

[Sean]

The default test is a linear test.  Basically it downloads a known block size from the chosen server and calculates the speed based on the time it took.  This is just like using a stopwatch to time how long it takes to download a file and using a calculator to calculate the throughput.  E.g. if a 10MB file takes 20 seconds, then 10 / 20 * 8 = 4Mbps.  TestMy will also return 4Mbps if the 10MB block took 20 seconds to complete.

 

A real world analogy would be like drawing water from a tap with a hose.  Based on how long it takes to fill a bucket, you can work out the flow rate.   E.g. if it takes a minute to fill a 10 litre bucket, the flow rate is 10 litres per minute.

 

The multithread test is quite different.  The server splits the block up into multiple pieces and your browser downloads multiple pieces simultaneously.  This process repeats until all the pieces have downloaded.  For example, when I start a 50MB Multithread test, it says "Testing your connection with 46 x 1109 kB elements simultaneously...", so if my browser allows 10 simultaneous connections, it will download the first 10 elements, followed successive elements after each completes until all 46 have downloaded.  Once all the elements have finished downloading, it calculates the speed based on how long it took to download the lot.  Multithread tests tend to perform faster on shared networks, such as wireless, public Wi-Fi, cellular, etc.

 

For multithread, a real world analogy would be like carrying bottles of water from a water source, e.g. 10 at a time depending how many you can carry.  Calculating the flow rate is much the same, e.g. if takes a minute to carry 20 x 1 litre bottles from A to B, that's 20 litres per minute. 

[ginahoy]

Thanks guys. My connection to ISP (Wi-Power by Transworld Network) is wireless (small dish, line-of-sight). Might that explain why I'm getting close to my contract speed with multithread tests but not with linear tests? 

[Sean]

That is quite likely the case.  With wireless networks, the tower you connect to is shared with many customers.  The tower itself is very likely microwave fed off another highsite tower that also feeds towers in other regions.  The means that main tower is potentially serving several hundred or thousand customers. 

 

Let's say it has a Gigabit link and there are currently 1000 simultaneous downloads/streams across all its subscribers, this gives an average of 1Mbps per individual download.  If you start a multithread download and it makes 10 parallel connections, then technically each connection will get an average of up to 1Mbps.  As your plan has a 5Mbps cap, the tower you connect to will limit the overall speed to within your plan. 

[ginahoy]

8 hours ago, Sean said:

That is quite likely the case.  With wireless networks, the tower you connect to is shared with many customers.  The tower itself is very likely microwave fed off another highsite tower that also feeds towers in other region

 

Thanks, that's helpful. As this is a very rural area, I believe the local tower serves on the order of dozens customers, for sure not hundreds. The ISP taps into fiber at the tower, so not wireless past that point (the same tower supports several cell antennas). 

 

Quote

Let's say it has a Gigabit link and there are currently 1000 simultaneous downloads/streams across all its subscribers, this gives an average of 1Mbps per individual download.  If you start a multithread download and it makes 10 parallel connections, then technically each connection will get an average of up to 1Mbps.  As your plan has a 5Mbps cap, the tower you connect to will limit the overall speed to within your plan. 

 

I'd like to better understand how this relates to my typical internet performance. The only real demand I put on my network is streaming TV shows...  we have a couple of TV's with Roku boxes. So if I'm streaming on one TV, the tower would see that as a single thread, right?  But if I'm streaming on 2 TV's, would the tower see that as two threads? If so, that should put us close to our 5 Mbs contract and would explain how we can simultaneously stream 2 HD videos without buffering.

 

Nonetheless, I'm paying (a lot) for 5 Mbs service so I need to collect some non-threaded test results at random hours so I can complain to my ISP.  Interestingly, I get the same poor non-threaded test results even in the middle of the night (typically 2 - 3 Mps) when most if not all of my neighbors are asleep. So this doesn't look like a network loading issue at the local node. 

[Sean]

That's correct - A single streaming TV is the equivalent of one thread (one TCP connection).  Two streaming devices would be two threads (two TCP connections).

 

Your off-peak performance should be very close to your subscribed speed with a single thread, ideally with a flat throughput from start to finish, like the following example:

Before complaining to your ISP, you will need to connect only the computer you are testing with to the router, e.g. turn off its Wi-Fi or disconnect all devices from it an disconnect any other Ethernet connection.  Connect your computer with an Ethernet cable to the router for the tests.  If you use Wi-Fi or have anything else connected, the ISP will usually blame the Wi-Fi or something else skewing the tests. 

 

Bring up the Windows task manager (Shift+Ctrl+ESC, Performance tab, Ethernet on the left) and make sure the 'Receive' is mainly showing 0Kbps.  You can take a screenshot to show that your PC's network adapter was idle before the test.  Then run a large manual block size such as 25MB.  As you mention you are closer to Los Angeles, I suggest trying a manual test with that server also. When the test completes, you can take a screenshot of the Task Manager's Ethernet graph to show that you are only getting 2-3Mbps with a single connection with each server.

[Pgoodwin1]

17 hours ago, ginahoy said:

 

My automatic (non-multi) tests always select Dallas, although geographically I'm a bit closer to LA. 

You can do some Latency testing (select Latency pulldown on home page). Compare results between Dallas and LA. Choose the one with  more consistent results. The tests are pretty quick. The server with the better latency will usually give better test results, but not always, as it matters when tests are performed and what’s going on between you and the test server.

[ginahoy]

6 hours ago, Sean said:

That's correct - A single streaming TV is the equivalent of one thread (one TCP connection).  Two streaming devices would be two threads (two TCP connections).

 

Thanks, Sean, that's very helpful to know. I can see further evidence of this by running a multithread and a single-thread test while one TV is streaming. In that case, both tests show similar results, which is exactly what one would expect... i.e., the streaming connection plus the single-thread test together max out my contract bandwidth.

 

Quote

When the test completes, you can take a screenshot of the Task Manager's Ethernet graph to show that you are only getting 2-3Mbps with a single connection with each server.

 

Great suggestion! Better yet, a screenshot of my router's WAN Traffic screen, since Task Manager can't show that other network devices are idle or off.

 

The problem is, they're going to ask me to run a speed test on their preferred site. I don't yet know which site they'll point me to, but as I mentioned in my original post, other test sites typically report close to my contract speed. For example, I just ran turned off everything and ran a TMN single-thread test and it shows 3.6 Mbs with middle variance of 47%. Next I ran tests at CenturyLink (see attached screenshot), Ookla, and Spectrum. All graphed consistent D/L speeds at 4.7 Mbs for the duration of the test.  Is this because other sites are running multithread tests? I'm going to have a hard time trying to explain that one to my ISP!

[ginahoy]

On 7/20/2020 at 5:51 AM, Pgoodwin1 said:

You can do some Latency testing (select Latency pulldown on home page). Compare results between Dallas and LA. Choose the one with  more consistent results. 

 

Thanks for pointing that out. I did multiple tests to Dallas (930 mi), Colorado Springs (800 mi), and Los Angeles (550 mi). Interestingly, I had slightly better average latency to Dallas, and deviation was much better with Dallas.

 

Quote

interesting that your multithread test results are better than your regular test results. On mine, I always get lower results when multithreading. 

 

Based on what Sean described, I'm not sure I understand why anyone would get worse results with multithreading. Does this work differently with cable & DSL networks versus wireless networks like mine? 

[Sean]

From checking the CenturyLink speed test page, they are using the speedtestcustom domain in an iframe, which is hosted by Ookla.  The Ookla website and its custom embedded offerings are all multithread tests:

 

As a second example you can show, you can download a test file from a hosting provider.  Most hosting providers offer test files, such as the following example:

https://mirror.leaseweb.com/speedtest/

[ginahoy]

 

1 hour ago, Sean said:

The Ookla website and its custom embedded offerings are all multithread tests:

 

Again, very helpful. This is getting very interesting. So I'm getting the impression that the reason I benefit from multi-threaded demand (e.g., two Roku streams) is unique to the type of network my ISP operates (wireless). I just learned Chrome supports 6 parallel TCP connections, so for general browsing, it seems unlikely I would see much difference between 3 Mbps and 5 Mbps single-thread performance, especially given that browsing is very bursty. More importantly, with up to six concurrent connections working, my poor single thread performance shouldn't impose much of a limitation for browsing. 

 

OTOH, I'm often clicking on links that open large image or PDF files (sometimes tens of MB's). My assumption is that's handled as a single connection. However, I just found this from a 2018 article on parallel downloading: "Google Chrome can establish multiple connections to download a single file in parts." (emphasis added; if you Google this quote, you'll get lots of hits). If I'm understanding this correctly, it raises the question why the TMN single-thread test isn't split across multiple connections. Is this something TMN can control, or maybe I need to do some testing to see if Chrome really does use multiple connections when downloading a single file. But I'll have to find an app like Resource Monitor for my Linux OS that shows the number of TCP connections open.

[Sean]

That's a good find as I was unaware of Chrome having this feature.  From checking here, Chrome does not have this enabled by default, so will download over a single thread until the hidden setting is enabled.  I checked a few sites and it appears that only some allow multiple connections.

 

For example, with the setting enabled, the Leaseweb test files still only download over a single connection:

 

With the GIMP software download, Chrome ran it over three connections:

[ginahoy]

@Sean, I discovered the following Linux command to monitor TCP connections:

 

sudo tcptrack -i <ethernet interface>

 

But first I determined the name of my ethernet interface with this command:

 

ifconfig

 

So with tcptrack, I was able to verify that downloads from my hosting company's server only open a single TCP connection. But like you, when I downloaded the GIMP file, 3 TCP connections were opened. I did more speed tests at CenturyLink and it opens 4 TCP connections, but that's independent of the "parallel downloads" setting in Chrome. Likewise, single-thread tests at TMN only open a single connection when the parallel download setting is enabled.

 

Now I need to collect a series of single-thread test results with wifi turned off and screenshots of my router's traffic monitor so I can take this to my ISP. Thanks for all the help!

 

EDIT: Actually, a single file download from my hosting company opens 4 TCP connections in Chrome. I previously checked this using a FTP client (Filezilla), not Chrome. Duh. Filezilla of course can be set to download multiple files in parallel, each opens a new TCP connection. But it will only open one connection per file.