jimerman

@mudmanc4 maybe not a bad idea - I think I have Dallas selected by default, but maybe some other ones. I'll post after some decent period of testing what happens. Thanks for the idea. @xs1 I don't get what you're saying. It looks like TestMy is showing results consistent with the other 2.

The speed tests I use regularly are: Comcast speed test (Ookla)* http://speedtest.xfinity.com Google Fiber http://speedtest.googlefiber.net/ Speed Of Me http://speedof.me and this site. * - I know Ookla is skewed - they do the test, then discard the bottom 30% of results and top 10% - this means the average speed shown is higher than a true average However, the 3 sites above all show consistent speeds from 90 Mbps to 120 Mbps. As soon as a test completes, I then run both Multistream and single stream on TestMy.Net, and I get about half (e.g. 63 Mbps). I retest several times, but consistently TestMy.Net shows inconsistent results that vary all over the map, while the other 3 sites show consistent speeds (either towards the top end of my service speed, or toward the bottom end when I am having a problem). Has anyone experienced this? Does this call into question the validity of using TestMy.Net? Or, is there something else going on?

Single thread shows 77 down, multi shows 79.9 down from the same browser at similar moments in time. But apps and browsers are typically multithreaded, and that threading utilizes the bandwidth more fully, testing its peak performance.

Thanks, that was it! I didn't realize that all those locations were checked, and that meant it was testing them all. That explains it. I have to still disagree with you about single/multi thread, I think multi thread is the best way to test and tell what my pipe is allowing. I do understand there are technical differences in the different testing web sites, but at a gross level, at least you return stats on all the packets (rather than Ookla). And, you use PoP servers, instead of geographically-close servers.

Ever since the first week in October, TestMy.Net has shown a very large (slower) result than SpeedOf.Me and SpeedTest.net. For example, the other 2 are showing 85-90 Mbps Download, while TestMy.Net is showing 25 Mbps. I turned on Multithreading, and they are running from the same browsers. I tried Internet Explorer 10 and Firefox from Windows 7, plus Safari 9 and 8 from OS X 11.11 and 11.10. Are you aware of any system issues with your infrastructure that would cause that problem? I know, for example, that SpeedTest has local servers and discards a disproportionate portion of the test results, so their report is faster than reality. However, it should not be 300%+ discrepancy. And SpeedOf.Me is similar to TestMyNet in methodology and infrastructure.

OK, thanks.

@editorsean, all great points. My suggestion was that with a slower connection, the disparity is smaller, and conversely bigger with a faster one. Also one small correction, you stated to multiply by 8 to figure Mbps - actually you should have said multiply by 8 to figure MBps, but Mbps is typically how line speed is measured, while MBps is how throughput is measured by application software (like web browsers), but truly it is confusing as all hell because take for example Filezilla, it measures in a completely different base of measurement - KiBps (Kibibytes per second), which is the computer science base-2 multiple number. I don't know why we don't just settle on a single standard of measurement - like the rest of the world has settled on the Metric system, I think the Base-10 is a good way to do it. (And for heaven's sake, why don't operating systems report disk space in the base-10 instead of base-2? Drive capacity is sold on Base-10, so you buy a 1 TB drive, but only get 0.7 TiB - which is the same, but confusing?) So, in the interest of cutting through all this confusion, and determining A ) what is the best method to use for testing my Internet speed, and B ) what is the best site to use for testing my Internet speed, here's what I am thinking. Testing a known data provider (like Canonical) is great. However, we need to provide complete consistency. Putting a response server as a PoP on a somewhat local backbone does not provide a "real-world" example of, say, downloading a distro ISO. However, it does give us a "best case scenario" of completely consistent data. As well, multi-streaming also is realistically how software transfers data, and so it seems tests should also be multi-streamed. But the ad-hoc discarding of data, that to me is unethical and misleading (seemingly deliberate, and thus the "unethical" label). So I am glad that TestMyNet does not do this. Testing to a backbone PoP server does provide useful data, eliminating such issues as a "public/commercial" server, and all the connections between it and its backbone. So the "test" as I see it, is testing "what's my best-case (aka multi-streamed) throughput between me and the backbone" - in order to answer a question like "why is it slow right now downloading the latest OS X from Apple, when usually it goes faster? Is it some line problem on my end?" For example, last October, I began having intermittent downloads of 2-5 Mbps on an 18 Mbps DSL line. Obviously something wrong - but where? Since it was widespread (across sites), it was assumed to be between my computers and the backbone, and not out to the remote endpoints to which I was connecting. Worse was, when AT&T would come out, the line would show nominal, and speed was also nominal (around 18-24 Mbps), with no reported errors on the system. With a series of visits and equipment replacement, we finally identified the cause as a faulty connection at the main street junction - and accurate speed tests were an invaluable tool in determining when issues occurred. @editorsean, I'm a bit curious on the math at the end of your post? You show the first download, it says size is 681.5 MB, Completed 50.5 MB. What exactly does this mean, how much was transferred? I assume 50.5 MB? So we multiply by 8 to get Mb, and divide by 120 seconds (2m 0s), I get a throughput of 3.37 Mbps. Likewise, 3.449 Mbps on the second one. And like I said, maxing out a connection to test its throughput is exactly what I want when I go to a speed test web site - otherwise, if I wanted a real world test, I would just go to an actual site I wanted to download from, and just download. I want to know, what's the fastest my line can go right now, and is it what my ISP promises?

I had been choosing 100 Mbps on TestMyNet. The other 2 sites don't let me choose sample size. In contrast to what you say above, when I test 6 MB sample size, I get a slower speed (30 Mbps) than when I test 50 MB and 100 MB, where I get 57 Mbps and 60 Mbps respectively. And that makes sense intuitively, as a transfer of large data is typically faster than lots of smaller data, simply because of the overhead in establishing and de-establishing communications for the data transfer.

Another thought here, up until recently my ISP was AT&T with 18 Mbps, and the 3 speed test web sites I used showed pretty much similar speeds when testing. Now that I have Comcast with 50 Mbsp, maybe that is why there is more disparity with the speed tests - because it is a higher bandwidth (and slightly different line technology). I am testing around 40 Mbps with TestMy.net single-threaded test, while multi-threaded is showing almost 60 Mbps. However, don't web browsers and download managers download multi-threaded anyhow?

That is a lot to think about, thanks for that fantastic reply. As a "lay" person with regards to net speed, it seemed to me like a simple "what's my speed" question, and I did not realize the complexities (and misleading methodologies in some cases) involved. I'll also try running Performance Monitor and Activity Monitor on the systems I run speed tests from, but since I have 6 people relying on the connection, I don't think I will attempt the disruptive complete disconnect/reconnect. As far as OS X, I do have all machines set to update to the latest (the one I primarily test from is 10.11), so I don't think it's the OS. That brings up the question of what I should hold my ISP accountable to? If they say they provide me 50 Mbps average speed, and it tests at 30, they will go to their Ookla test and voila, with multithreading and convenient ignoring of outlier packet statistics, they will say it is 50. So then do I get into an argument with the tech, whose place it isn't to set policy, and who has no power over it whatsoever? I guess it's a moot point, but it also reminds me of a rant years ago that the storage industry snuck in a new standard of memory measurement, and that is what they use to advertise disk size, instead of what we were taught in computer class. Were you aware that the terms "Megabyte" and "Gigabyte" mean, precisely, 1 million bytes and 1 trillion bytes, and not (which I thought) 1,073,741,824 bytes and 1,099,511,627,776 bytes (1024 multiplied out)? Those numbers I thought, are called Mebibytes and Gibibytes - a highly technical and phonetically confusing obfuscation. So it looks like this similar obfuscation is going on here. Oh, and by the way - when they say Mbps, do they mean Megabits or Mebibits????

I regularly use 3 sites to test my network speed: Ookla (flash-based), SpeedOf.me (HTML5-based), and TestMy.net. However, the first 2 seem to report similar results, while TestMy.Net reports very different (and slower) results. Ookla I can rationalize, as they have local servers that might give lower latency. SpeedOf.Me correlates to Ookla. I am regularly getting 52 - 60 Mbps download, and 10-12 Mbps upload. However, when I run TestMy - I see 10 Mbps down, and 8 Mbps up. WAAAY off - this is the auto test. When I run the Download only test, I got 39 Mbps down, which is still way off. Does anyone know why? Before I switched ISP, TestMy seemed to be similar results to the other 2, and I really do like the automatic test every hour so I can collect stats across time.