How much speed variation is acceptable?

[Numi]

My speed tests' graphs show a LOT of fluctuation from 0 to 200 Mbps, and that my  speed varied by 200%, which I assume is really terrible. Firstly, what does it mean for speed to vary by 200%? 200% of what exactly, my average speed? Since a downward fluctuation greater than 100% is nonsensical, I assume it means something else. Secondly, how much speed variation would be 'acceptable?' Lastly, there are two graphs in the speed test results, one on top of the other, and both have the same labels on the x and y axes. However, they show different results (and only the bottom one contains my upload speed data as well as download). What is the difference between the two graphs?

 

Thanks in advance

[CA3LE]

First, I will be changing variance to standard deviation in the future.

 

Let's use this result of your as an example.

TestMy.net Test ID : 5zlOE7tGc

Ideally this flat lines and doesn't deviate from start to finish.  Sometimes though, if the result is much lower than your line speed this flat line can be an indication of a bottleneck.  Most of the time a flat line with 0% variance is a good thing.

 

One of my recent results as an example.

TestMy.net Test ID : fUJ4g~VHj

Here's the actual calculation from the program.

 

round(($maximumThruput - $minimumThruput)/(($maximumThruput + $minimumThruput)/2)*100)

 

So your example above would be 

round((0.32 - 8.32)/((8.32 + 0.32)/2)*100) = -185%

 

The difference from the min and max divided by the average of the min and max... then calculated into percent.

 

The higher the number the more it indicates that the connection was heavily fluctuating during the test.

 

Using standard deviation from your example above you'd get 2.4 Mbps [https://testmy.net/working/deviation/standard-deviation.php?arr=3.28,2.91,1.79,1.84,3.04,4.93,1.43,0.32,0.34,1.31,1.91,3.34,1.89,1.73,3.9,7.28,8.32,8.18] -- Again, ideally this number would be 0 Mbps.

 

My result above's standard deviation is 42 Mbps... higher number but not relative to the result. [https://testmy.net/working/deviation/standard-deviation.php?arr=155.34,345.77,338.49,338.49,340.88,347.01,334.96,331.51,338.49,338.49,345.77,349.53,348.26,342.09,348.26,347.01,339.68,343.31,344.53]

 

So the standard deviation then needs to then be turned into a percentage of your average.

 

$standardDeviation / $middleAverage

Your example: 2.4/2.93 = .82 ... or 82%

My example: 42/342.24 = .12 ... or 12%

 

Using standard deviation I think will be much easier for everyone to understand.  To understand the current formula myself I had to go into the program... my users don't have that luxury.

 

Long story short: Variance shows what I wanted to show but makes it overly complicated.  I'll work on that for you.... actually -- I kinda just did, just need to work that all into the program.  

[CA3LE]

If you go to https://testmy.net/mysettings you can now change middle variance to standard deviation.

 

Note that it's calculating using TiP points from 10% to 90% ... start and finish points are not in this calculation (my calc in the post above was including all points).

[Numi]

Oh I see, it means variance. I probably should've played around with the numbers more n figured it out for myself. Yes, standard deviation is more readable (especially when its expressed in mbps and not a percentage) so thanks for that! Also, what is 'My Host Avg?' Is that like my ISP's average speed or something?

 

I absolutely love this site, so thanks for that as well as the detailed explanation above!

 

[Numi]

oh, and a quick follow up question: most of the tests (on this site I mean) gave my connection between 3 and 4 stars (inclusive), despite my speed variance never dipping below ~80%. Does this mean that that much speed variance is 'acceptable' or that the star rating doesn't take it into account? I find that I can stream 1080p videos just fine, which I assume is because it buffers ahead when the speed is high and I therefore don't notice the troughs in the speed. On the other hand, when playing online games (Dota 2), I find that even within, say, 5 second windows I go from lag to no lag and back to lag. Which I guess would be because the speed is fluctuating wildly, unable to maintain stability even for 5 seconds. Does that analysis make sense? If yes, how do I describe this to my ISP? 'Speed' doesn't seem to be the correct term, so what would be?

[CA3LE]

On 9/23/2019 at 9:21 PM, Numi said:

Also, what is 'My Host Avg?' Is that like my ISP's average speed or something?

 

I absolutely love this site, so thanks for that as well as the detailed explanation above!

 

Thank you!  Yes, 'My Host Avg' is your ISP's average.

 

On 9/23/2019 at 10:35 PM, Numi said:

most of the tests (on this site I mean) gave my connection between 3 and 4 stars (inclusive), despite my speed variance never dipping below ~80%. Does this mean that that much speed variance is 'acceptable' or that the star rating doesn't take it into account?

 

The star rating is something that's followed TMN through the years, since the beginning.  There is logic behind it but it doesn't take your TiP readings into account (the graph that shows how the data flowed during the speed test).  Although it wouldn't be very hard to implement a few changes to have it take the variation of the individual test into account.

 

Star Rating calculation history

First started by only calculating based on your speed.  Years later TMN started logging to a database.  The stars were then calculated based on the Index (a recent average for all TMN users).  Then when TMN started logging host results it started taking those into consideration.  So now it's based on averages of yourself, the index and your host... ...but definitely, if the connection is deviating by X%, it would be helpful if the star rating was effected as a result.  --- on my list now.

 

-D

[Numi]

is this website a non-profit? Is there any way I can contribute (besides donating cash (of which I have little) and sending users to the site (which I've already been doing xD)? I'm a programmer and I'd be happy to help