Confused about average results - how about a histogram?

[wmertens]

I'm trying to compare iWay and Init7 in Switzerland, and in the rankings iWay seems to score twice as fast as Init7, but when you go to the individual stats, Init7 way outscores iWay.

 

Ranking: https://testmy.net/country/ch#isps

• #1 iWay: 1811.4 Mbps 2152 Mbps  316 Mbps   47 clients took 356 tests recently
• #2 Init7: 624 Mbps 871 Mbps  154 Mbps   67 clients took 183 tests recently

 

Individual stats:

• iWay: https://testmy.net/hoststats/iway, Averages 326.7 Mbps / 85.3 Mbps | 81 ms, last 1000 download: 195.7, upload: 117.4
• Init7: https://testmy.net/hoststats/init7, Averages 621.6 Mbps / 227.8 Mbps | 72 ms, last 1000 download: 1265.3, upload: 146.9
  
  I don't understand how these numbers are related?

In any case, I think a histogram would be way more useful for the individual stats, it would show the different kind of connections an ISP has.

[CA3LE]

The stats that show under hoststats are taken over longer periods and have more results considered.

 

The stats that show under the ISP ranking tab are done in the background every 15 minutes and only consider the most recent results.

 

My next major hoststats update will use the same numbers for everything.  Using the overall average shown on the current hoststats pages.

 

I think a histogram is an excellent idea, I'll be working on that.

 

With less than 300 results each, it's hard to compare.  The dataset on those providers is very small.  If you click the "Log" button on the hoststats pages you can see the number of results.  iWay looks better... but Init7 has a slightly larger sample.

 

[wmertens]

Ok I see, by using a continuous stream you're signaling something that the ISP may optimize for, and it's not representative of actual use, right? I think it bears explicitly stating this in the explanation.

 

Really loving the beta BTW, it's so great to see the packets come in, and also to see how speed varies during the test. When will it go live?

[CA3LE]

Working on the histogram... here's the return on some raw queries I'm forming.

 

Histogram of your full history (download bin size 10 Mbps, upload bin size 2 Mbps)

    -> WHERE  User_Name = 'wmertens' AND Test_Type = 'download'
+-----------+-------+------+
| bin_floor | COUNT | bar  |
+-----------+-------+------+
|         0 |    14 | ***  |
|     10000 |    21 | ***  |
|     20000 |    16 | ***  |
|     30000 |     9 | **   |
|     40000 |    11 | **   |
|     50000 |    18 | ***  |
|     60000 |     2 | *    |
|     70000 |     2 | *    |
|     80000 |     3 | *    |
|     90000 |     9 | **   |
|    100000 |     7 | **   |
|    110000 |     3 | *    |
|    120000 |     1 |      |
|    130000 |     1 |      |
|    140000 |     1 |      |
|    150000 |     2 | *    |
|    210000 |     1 |      |
|    220000 |     1 |      |
|    250000 |     1 |      |
+-----------+-------+------+
19 rows in set (0.002 sec)

    -> WHERE  User_Name = 'wmertens' AND Test_Type = 'upload'
+-----------+-------+------+
| bin_floor | COUNT | bar  |
+-----------+-------+------+
|         0 |     4 | *    |
|      2000 |     4 | *    |
|      4000 |     7 | **   |
|      6000 |     8 | **   |
|      8000 |    13 | ***  |
|     10000 |     9 | **   |
|     12000 |     7 | **   |
|     14000 |     8 | **   |
|     16000 |     2 | *    |
|     18000 |    19 | ***  |
|     20000 |     6 | **   |
|     24000 |     4 | *    |
|     26000 |     4 | *    |
|     28000 |     1 |      |
|     30000 |     2 | *    |
|     32000 |     2 | *    |
|     34000 |     4 | *    |
|     36000 |     4 | *    |
|     38000 |     1 |      |
|     40000 |     4 | *    |
|     60000 |     1 |      |
|     62000 |     1 |      |
+-----------+-------+------+
22 rows in set (0.002 sec)

 

My results, on Comcast, this year only... (download bin size 25 Mbps, upload bin size 2.5 Mbps)

    -> WHERE  User_Name = 'CA3LE' AND Test_Type = 'upload' AND `Date` > '20240101000000' AND `User_Host` = 'comcast_cable'
+-----------+-------+-------+
| bin_floor | COUNT | bar   |
+-----------+-------+-------+
|         0 |    12 | **    |
|      2500 |    11 | **    |
|      5000 |     1 |       |
|      7500 |    14 | ***   |
|     10000 |     3 | *     |
|     12500 |     4 | *     |
|     15000 |     6 | **    |
|     17500 |     8 | **    |
|     20000 |    14 | ***   |
|     22500 |    13 | ***   |
|     25000 |    13 | ***   |
|     27500 |     9 | **    |
|     30000 |    14 | ***   |
|     32500 |    35 | ****  |
|     35000 |    61 | ****  |
|     37500 |   203 | ***** |
|     40000 |    20 | ***   |
+-----------+-------+-------+
17 rows in set (0.212 sec)

    -> WHERE  User_Name = 'CA3LE' AND Test_Type = 'download' AND `Date` > '20240101000000' AND `User_Host` = 'comcast_cable'
+-----------+-------+------+
| bin_floor | COUNT | bar  |
+-----------+-------+------+
|         0 |    48 | **** |
|     25000 |    17 | ***  |
|     50000 |     7 | **   |
|     75000 |    10 | **   |
|    100000 |    26 | ***  |
|    125000 |    31 | ***  |
|    150000 |    35 | **** |
|    175000 |    21 | ***  |
|    200000 |    25 | ***  |
|    225000 |    29 | ***  |
|    250000 |    22 | ***  |
|    275000 |    21 | ***  |
|    300000 |    24 | ***  |
|    325000 |    36 | **** |
|    350000 |    35 | **** |
|    375000 |    23 | ***  |
|    400000 |    18 | ***  |
|    425000 |    25 | ***  |
|    450000 |    33 | ***  |
|    475000 |    44 | **** |
|    500000 |    22 | ***  |
|    525000 |     2 | *    |
|    550000 |     4 | *    |
|    575000 |     6 | **   |
|    600000 |     9 | **   |
|    625000 |     8 | **   |
|    650000 |    10 | **   |
|    675000 |    13 | ***  |
|    700000 |    15 | ***  |
|    725000 |    22 | ***  |
|    750000 |     9 | **   |
|    775000 |    12 | **   |
|    800000 |    20 | ***  |
|    825000 |    13 | ***  |
|    850000 |     3 | *    |
|    875000 |     1 |      |
+-----------+-------+------+
36 rows in set (0.177 sec)

 

Optimal bin size will vary case by case, I'll give you some control over all of that. 

 

"bar" is there just for visual reference in the query, the charts will be drawn from the count and be better represented in the final output to the client.

 

These are just raw returned queries.  Now I have to build it into something.  But this shows how I can easily return a histogram, at the mysql level.

[wmertens]

See those are super interesting 😊

 

At some point it would probably also be nice to use stacked bars as the histogram bars so you can e.g. also group by age.