Paga

TestMyNet CLI for Linux

11 posts in this topic

Hello there,

 

Given a recent networking problem on my networking setup, I've decided to write a simple CLI client for testmynet. I know, we can test download speeds using wget, but a proper client good have some nice goodies such as:

- Keeping the internal list of servers.

- Choosing the closest server by latency.

- Choosing a random server.

- Being a collectd plugin (my main motivation).

 

Before I embark on this (admittedly short) journey, I have a few basic questions:

- Is there anything similar out there? I couldn't find it.

- Anyone would be interested in such a tool? I find the collectd integration super useful.

- Is there any limit on how many times I can query a server before being blacklisted? I do not want to cause problems for TMN. The idea is to use the /dl-XXXXX URL to download a fixed amount of data.

 

Thanks in advance.

Tom Hyatt and Simplify like this

Share this post


Link to post
Share on other sites

Interesting. Myself, considering testmy.net does most everything you are suggesting, I'm not sure adding more clutter to my local system, or Nagios box is something I would entertain, when I can log into testmy.net and analyze, search the database for either automated, multi threaded, or manual testing results otherwise.

 

There are means incorporated within testmy.net, which prevent abuse, however this is not made public as far as I understand it.

 

I'm not posting to discourage you in any way, in fact personally I encourage good useful scripting.

Share this post


Link to post
Share on other sites
On 11/10/2016 at 6:19 AM, Paga said:

Hello there,

 

Given a recent networking problem on my networking setup, I've decided to write a simple CLI client for testmynet. I know, we can test download speeds using wget, but a proper client good have some nice goodies such as:

- Keeping the internal list of servers.

- Choosing the closest server by latency.

- Choosing a random server.

- Being a collectd plugin (my main motivation).

 

Before I embark on this (admittedly short) journey, I have a few basic questions:

- Is there anything similar out there? I couldn't find it.

- Anyone would be interested in such a tool? I find the collectd integration super useful.

- Is there any limit on how many times I can query a server before being blacklisted? I do not want to cause problems for TMN. The idea is to use the /dl-XXXXX URL to download a fixed amount of data.

 

Thanks in advance.

I have been looking for such a tool myself as it would be very handy to be able to automate this process as I collate the data and use it with my ISP's here in Australia.  The different technologies used on the Australian NBN deliver different results for the same connection speeds....  I don't trust speedtest.net and the Australian ISP's prioritise traffic to this site - so the results are never going to be correct for all the reasons we know and hate.  Paga, if you are continuing with this project and I VERY interested.

 

Cheers!

Share this post


Link to post
Share on other sites

Sorry I'm seeing this way after it was posted.

 

On 10/10/2016 at 4:19 PM, Paga said:

- Is there any limit on how many times I can query a server before being blacklisted? I do not want to cause problems for TMN. The idea is to use the /dl-XXXXX URL to download a fixed amount of data.

 

You can do that but be careful that you aren't repeatedly hitting it... testing only seconds apart, over and over for instance is sure to trigger a blacklist.  But if you're testing using normal intervals you're fine.  

Share this post


Link to post
Share on other sites
1 minute ago, CA3LE said:

Sorry I'm seeing this way after it was posted.

 

 

You can do that but be careful that you aren't repeatedly hitting it... testing only seconds apart, over and over for instance is sure to trigger a blacklist.  But if you're testing using normal intervals you're fine.  

Hi Ca3le,

 

I was looking at running the test every 15 minutes - is that going to trigger a blacklist?

Share this post


Link to post
Share on other sites
13 minutes ago, Simplify said:

Hi Ca3le,

 

I was looking at running the test every 15 minutes - is that going to trigger a blacklist?

 

Not at all.  In case you didn't know yet, TestMy.net has an auto test built in.  Members can test up to every 5 minutes automatically.  It's only when people make their own scripts to run the test more often that it will trigger intervention.  Like if you made a program that ran the test every 5 seconds, indefinitely... you're going to get banned because it violates TMN terms of use.  Keep in mind, you're degrading your connection while the test is running... it's literally designed to suck up your bandwidth.  So, personally I wouldn't want to run it too often... it defeats the purpose if it's run too often.  That's coming from the guy wrote wrote the program and benefits from you using it.  Once every 5 minutes for example should only be used for short periods to aid in troubleshooting.  You wouldn't want to run that test constantly.  There are other tools better suited for consistent monitoring... TestMy.net has a few in development right now that will monitor your connection health while only using bytes of data.  Keep visiting.

 

Don't worry about getting banned if you're using TMN normally in your browser as it's designed.  

Share this post


Link to post
Share on other sites
25 minutes ago, CA3LE said:

 

Not at all.  In case you didn't know yet, TestMy.net has an auto test built in.  Members can test up to every 5 minutes automatically.  It's only when people make their own scripts to run the test more often that it will trigger intervention.  Like if you made a program that ran the test every 5 seconds, indefinitely... you're going to get banned because it violates TMN terms of use.  Keep in mind, you're degrading your connection while the test is running... it's literally designed to suck up your bandwidth.  So, personally I wouldn't want to run it too often... it defeats the purpose if it's run too often.  Once every 5 minutes for example should only be used for short periods to aid in troubleshooting.  You wouldn't want to run that test constantly.  There are other tools better suited for consistent monitoring... TestMy.net has a few in development right now that will monitor your connection health while only using bytes of data.  Keep visiting.

 

Don't worry about getting banned if you're using TMN normally in your browser as it's designed.  

Hi Ca3le,

 

I am aware of your automated tool and have been using it.  I would like to run the tests and capture the results to CSV locally (programmatically) for collating reports that use with the ISP's here in Australia.  previously I had been using speedtest-cli to do these tests.  Due to my own ignorance, I didn't know about the differences in the tests - now with the new knowledge, I would like to use testmy.net to gather the results, hence the reason for my keen interest in the tool that Paga is talking about developing.  This is the type of reports I have been collating previously with the speedtest-cli tool. https://docs.google.com/spreadsheets/d/1KKrkZUjA7Q1QL-HoFprcaOSNptyj4RNV-WbGI_plw-Q/edit?usp=sharing 

 

I generate these reports for myself, family and friends and also for clients as ISP's in Australia (particularly with new NBN installations) like to play the blame game and these reports are extremely important to hold the ISP's accountable.

 

Cheers,

Darren

Share this post


Link to post
Share on other sites
On 6/9/2017 at 9:18 PM, CA3LE said:

Sorry I'm seeing this way after it was posted.

 

 

You can do that but be careful that you aren't repeatedly hitting it... testing only seconds apart, over and over for instance is sure to trigger a blacklist.  But if you're testing using normal intervals you're fine.  

 

What would be an acceptable limit? 1 test every X minutes? X MB downloaded every Y minutes?

Share this post


Link to post
Share on other sites

I'm a test addict. That's the first step right?

 

I managed to get a couple of my IPs black listed. :) (sorry, but honestly, it's a mark of a good engineer to be able to break things - at least that's what I tell myself).

 

The inadvertent issue I had was we have multiple sites (2500+) that get their webfiltering from two internal sites which means the public IP for my 2500+ sites is the same. They are in 4 different time zones, and I try to randomize the test in a one hour period, but 2500 sites doing http gets of 1,4,12,24 MB across three hours is clearly blacklist worthy.

 

Questions:

 

1) Can I get unblacklisted please?

2) How can I best configure these tests to not trigger the auto blacklist? Instead of 2500 once a week, different 300 once a day across a couple of hours?

 

You guys run a great service here, much more reliable than speedtest.net in my opinion and have been very helpful with my requests. Even if I am an addict.

Share this post


Link to post
Share on other sites

@SalmanAhmed , getting removed from the blacklist might be possible, however if this happens I would advise doing the math before running the tests next time, as you have already realized ;)

Share this post


Link to post
Share on other sites

Mr. @mudmanc4 it's about to hit again tonight. I work for a decent sized retailer and it's not always the easiest to get changes through to production quickly. Paper trail and what not.

 

So... don't remove me quite yet. :) 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now