jimharle

Glad you got it sorted; my suggestion was going to be EasyBCD.

For the sake of getting this off my mind I decided to go ahead and say it; as simple as I can to avoid a technical conundrum... 
 
As a network admin, one would surely be well aware results shown locally will not always match gateway results. We know packets are differentiated between one another on various levels throughout the route, once said packets reach any 'destination' (which there are many in it's travels) header bits may or may not be included / stripped, reordered ect. 
 
Therefore the possibility of catching an exact real time congruent match on throughput is very thin. As well as variations between calculations performed (after the fact) locally. (machine dependent latency / local filtering due to firewalls, routers, OS specific settings and load across a plethora of levels not to even mention QOS on more levels imaginable)
 
Stopping eons short of any technicals here, when we alter the very data we seek to explore in any way, or it's resulting computed findings, on any level, this defeats the very reason we initiated the test to begin with, since altered data within the flow is exactly what we were looking for to start with. And should be considered null and void.
 
Therefore we look for "real world" results. What is actually happening. The entire process as a whole while surfing, uploading or downloading, streaming, basically every day usage, never distorts , never alters or flaunts, it just is. 
 
If i look at myself in the mirror, and choose to not see the grey hairs on my chin in the morning, and try to convince myself they are still their original color, I would call that mental masturbation , not reality. 

The problem with our office connection is excessive packet loss in the "last mile," which makes single-threaded transfers max out at around 8Mbps on our 30Mbps symmetrical circuit, unless we are transferring files over low-latency routes (directly from our ISP or from providers whom they peer with), in which case we can hit 27Mbps.
 
It has been a long struggle, but we are in the process of ditching the last mile provider (XO Communications).  Our [local] ISP is either going to resell us a Comcast Business 150Mbs connection, or we're going to get that directly.  It's a long story in another thread; this is the short version. 

FWIW there is one major problem with an on-net test, whether it's Ookla-based or not.  An on-net endpoint is going to be low-latency to the customer (~ 7ms or so), and *if* there are packet loss problems within the last mile, those will be masked by a low-latency test because the TCP retransmissions will "recover" so quickly.  When performing a low-latency test, it needs to be done in parallel with a packet capture to ensure there is no significant packet loss.
 
See this thread for more:  https://testmy.net/ipb/topic/32071-upload-speed-issue-comcast/

FWIW there is one major problem with an on-net test, whether it's Ookla-based or not.  An on-net endpoint is going to be low-latency to the customer (~ 7ms or so), and *if* there are packet loss problems within the last mile, those will be masked by a low-latency test because the TCP retransmissions will "recover" so quickly.  When performing a low-latency test, it needs to be done in parallel with a packet capture to ensure there is no significant packet loss.
 
See this thread for more:  https://testmy.net/ipb/topic/32071-upload-speed-issue-comcast/

Here’s the response I filed with the BBB yesterday.
 
CASE ID: 22168058
On August 11, 2014, you provided the following information:
(The consumer indicated he/she DID NOT accept the response from the business.)
After Mr. Erb's response was posted to the BBB site, I e-mailed him and other Veracity personnel, indicating that their request to have Technicolor pay the XO ETL was not fair nor equitable. Mr. Erb then invited us to their Draper office for a meeting, which we did accept and attend on August 5th. In that meeting, we shared our mutual frustrations, and agreed on efforts to work together to address the problem to resolution.

In the days that followed, we gathered additional technical information, and then on Friday the 8th, participated in a technical "deep dive" with their high-level engineering staff, which took place over several hours. At the conclusion of that technical session, both parties had gathered sufficient technical evidence indicating that the root of the problem is persistent packet loss/congestion within XO's network, which sits between our building and Veracity's infrastructure. Consequently, Veracity's engineering staff is pushing back on XO to get the problem corrected. Neither Technicolor nor Veracity have direct control over XO's network, although Veracity contracts with XO to provide "last mile" service for our connection, since no Veracity fiber is present in our building.

We are hopeful that Veracity is successful in getting XO to correct the problem, and feel it would be reasonable to give them until August 31st to get the problem corrected. Technicolor has provided Veracity remote access to virtual machines within our building, so that they can perform any necessary testing 24/7 without relying on our involvement, although we are happy to assist with any troubleshooting should it be necessary.

We maintain that the relationship with XO rests with Veracity, and that it is their responsibility to drive them to resolution. If resolution can be obtained by August 31st, we feel it would be reasonable to finish out the contract. If not, we feel that it would be reasonable for Veracity to release us from the contract, with no ETL or other penalties.

Sincerely,

Jim Harle

Awesome!  Please post your test results for the 10/2 service. I will bet your download performance is better than our office, and we allegedly have a 30/30 symmetrical commercial connection.  I just tested it while signed in, and got 4.5 down and 9.7 up, which is typical...but that's another saga

You're absolutely right but... happy wife happy life.
 
 
Yes, it's 5GHz... as soon as the bed was moved my speed returned to 100 Mbps+.

Why would you want to hide that thing of beauty behind a nightstand?

So we met with Mr. Erb and other folks from his company on Tuesday afternoon.
 
To make a long story short, they seem to be determined to solve our problem, and are “just as frustrated as we are.”  We talked at length about tests that they and we have performed.  I believe we generated some mutual respect, and a determination to fix the problem.  It does seem that we are an isolated case for them, and that their service doesn’t suck for all of their customers.

We are scheduled for a conference call on Friday, during which time we’ll be disconnecting our router and doing some detailed troubleshooting.  They also requested some packet captures, which I provided today.

I also installed an Ethernet switch between our respective routers, and plugged a vswitch from our ESXi hypervisor into it, so that I could give them a couple of VMs on the outside they could test remotely with.  So there’s a Linux box and a Windows box configured with veracity user accounts.  I told them they could do “whatever they want” with them.

The BBB response to their response is due Monday, so I was going to hold off on that until then.

Cheers!

Jim

Why would you want to hide that thing of beauty behind a nightstand?

So we met with Mr. Erb and other folks from his company on Tuesday afternoon.
 
To make a long story short, they seem to be determined to solve our problem, and are “just as frustrated as we are.”  We talked at length about tests that they and we have performed.  I believe we generated some mutual respect, and a determination to fix the problem.  It does seem that we are an isolated case for them, and that their service doesn’t suck for all of their customers.

We are scheduled for a conference call on Friday, during which time we’ll be disconnecting our router and doing some detailed troubleshooting.  They also requested some packet captures, which I provided today.

I also installed an Ethernet switch between our respective routers, and plugged a vswitch from our ESXi hypervisor into it, so that I could give them a couple of VMs on the outside they could test remotely with.  So there’s a Linux box and a Windows box configured with veracity user accounts.  I told them they could do “whatever they want” with them.

The BBB response to their response is due Monday, so I was going to hold off on that until then.

Cheers!

Jim

Here's the e-mail response I sent this morning:
 
Dear Mr. Erb,
 
I received your response to the BBB complaint I filed against your company, and have attached it here for reference.  I trust you realize that is a public record.
 
So to confirm that I’m understanding this correctly:  Veracity Networks is unable, or unwilling, to provide us Internet service with speeds reasonably close to what we’re paying for.  Veracity Networks will allow us to cancel the contract for Internet service, provided we pay the ETL for the “last mile” fiber provider (XO Communications).  Even though no new fiber had to be laid to the building from which we are leasing office space, Veracity is paying XO a monthly fee for riding that fiber transport, which is then passed on to us as part of the monthly bill.  Presumably, Veracity made a commitment with XO on how long that term would be, and cancelling early will trigger the ETL.  You wish to collect that ETL from us, and excuse yourselves from any responsibility or liability.
 
There is one word which comes to my mind in reaction to this:  Stunning.
 
Both Technicolor and Veracity agree that the problems we are experiencing are unrelated to the underlying XO circuit.  Therefore, the problem must be originating from within Veracity’s network infrastructure, in terms of routing or peering with your upstream providers.  However, Veracity continually fails to acknowledge this, conveniently hiding behind the notion that you can’t guarantee the performance of the “greater Internet.”  While it is true you can’t control the performance of the “greater Internet,” or specific endpoints on it, you can, and do, control the size/type/quality of the connections to your upstream providers.  From our perspective, these are woefully insufficient, with the two known exceptions being your connections to XMission and the University of Utah.
 
It is clear that your technical staff is either incapable of determining root cause and correcting the problem, or unable to because of business decisions over which they have no control.  The way I see it, there are two possibilities:
Veracity maintains [mostly] poor/oversubscribed peering connections with their upstream providers, meaning that all of Veracity’s customers are experiencing the same problems we are, though may not be aware they are getting ripped off. There is some idiosyncrasy with the way our circuit terminates within Veracity’s network infrastructure, be that routing, a physical port or device, whatever.  This would mean that the problem is isolated to our connection, and not impacting other customers. I really want to believe that it’s the second one, but I feel that if it were that, then all that would need to happen would be, you’d move our termination point to a different device or location (the same as one of your “working” customers), and we’d be off to the races.  But that hasn’t happened, so it begs the question, is it really the first one?  If it were that, it may not be “illegal” to maintain inadequate peering connections, but it would be [at the very least] highly unethical.  Which is it?
 
Asking us to pay the ETL is insulting, and I’m struggling with what it would take for you to face reality here.  Do we need to approach the local news media?  Do I need to publish my “story” on social networks so that all of my LinkedIn contacts can read it and potentially distribute it?  What needs to happen for you guys to do the right thing?
 
Some of my frustrated colleagues in the office want us to just stop paying the bill, let you sue us, and go with Comcast…which would be faster and cheaper.  But personally, I think we should be willing to finish out the contract we originally committed to.  While it is expensive, if it were performing as advertised, it would be meeting our needs, and we would be supporting the local economy.  But the problem is, you are not holding up your end, it is impacting our business, and that is just plain wrong.  What if the water pressure at your house was only averaging 25% of normal all of the time?  That would make you pretty ticked, wouldn’t it?
 
Disclaimer: I realize that this isn’t a “perfect” analogy, as one can’t choose between different suppliers of water; but please stay with me, the general points are the same.
 
Me:        I need to report a problem with the water supply to our house.  The water pressure is very low, and has been that way since we moved in.  We and our kids have to get up several hours earlier than normal, so that we have time to take the resultant long showers, so we’re not late for church.  We had a plumber come in to take a look, and after measuring the flow, he said we’re only receiving an average of 25% of the water we should be.  He said we should be able to receive twenty gallons per minute, but we’re only averaging five.
 
City:       Okay, that sounds odd.  Are you sure you don’t have a leak somewhere?  Perhaps a toilet is malfunctioning and running all the time?
 
Me:        No, we’ve checked for things like that, and everything appears normal.  We’ve also analyzed the water bills from the city, and they verify that our water consumption has been very low.  Incidentally, our bill is ten times higher than what our friends pay in different neighborhoods.  Why is that?
 
City:       Well, your neighborhood requires us to install special main pipes, which connect your house to our water system.  Those pipes are very high quality, made out of a titanium alloy, and capable of handling over a thousand gallons per minute, although we have that limited to twenty, as that’s what your currently paying for.  However, unlike your friends, you are allowed to consume as much water as you want every month, for no additional charge.  Much of what you are paying for is that special pipe.
 
Me:        Okay, that sounds odd.  But whatever, please send someone out to troubleshoot, as we’re only getting five instead of twenty.
 
City:       Okay, we’ll send someone out.
 
Days/weeks pass, as the city sporadically sends technicians out to troubleshoot.  Each time they do, the technician has to disconnect the house from the water supply, which is disruptive and inconvenient.
 
City:       We haven’t been able to find a problem, so we changed the setting on your titanium pipe to allow thirty gallons per minute.  Try it now.
 
Me:        That hasn’t made a difference; we’re still only averaging five gallons per minute.  A different plumber came to the house and verified that.  Will you send someone out again, please?
 
City:       We’ll check some things and get back to you.
 
More days/weeks pass.
 
City:       Good news!  We tested your titanium pipe, and it is functioning normally – we can push thirty gallons of water per minute through it just fine, when we hook up our water truck to it.
 
Me:        That’s great, but we’re still only getting five.  Since the titanium pipe is not the problem, perhaps the problem is with your water supply not being good enough to handle the neighborhood’s capacity needs?
 
City:       Well, we can’t guarantee that your house, or anyone else’s, will always receive the maximum allotment.  What if twenty people in the neighborhood are taking showers at the same time?  We don’t have control over that.
 
Me:        Yes, I understand that, but shouldn’t your water system be built with enough capacity, so that your customers can receive reasonable amounts of water and pressure most of the time?
 
City:       What?
 
Me:        What?
 
City:       We turned your dial up to one hundred gallons per minute.  Try it now.
 
Me:        Nope, we’re still only getting five.
 
City:       We don’t know what to tell you.  Your titanium pipe is capable of thirty, and we’ve proven that.
 
Me:        Yes, but you can only deliver five, which is not acceptable.  We’d like to switch to a different water provider.
 
City:       No problem, we’re here to help.  Just pay us seven thousand dollars, and you can be on your way.
 
Me:        Seven thousand dollars?!  For what?
 
City:       For the titanium pipe.  Someone has to pay for that.  That stuff is expensive!  Tony (our Titanium supplier) is going to make us pay for it, whether you use it or not.  So you need to pay for that.
 
Me:        But you’re only able to deliver five gallons of water anyway, not near the twenty we’ve been paying for (for months), and even further from the thirty and one hundred you set it to…what would happen if we needed and paid for one thousand?  Why should we have to pay for your ineptitude?
 
City:       You don’t seem to understand how water delivery works.  Pay us our money and move along.
 
Thanks for reading.
 
-Jim

We haven't heard anything yet from the ISP after sending the comparison matrix. We are asking our legal team if it is okay to stop approving their invoices, so that if the ISP sues us for breach of contract, they were aware it was coming.  Additionally, I filed a complaint with the BBB this morning.  One of the questions on that was "would you be willing to speak to the media?" to which I answered yes.

I have no idea what "normal" is for cable modems, but here are the details from mine (I'm not having any problems...at home that is)
 

 
Model Name: SB6121
Vendor Name: Motorola
Firmware Name: SB_KOMODO-1.0.6.10-SCM00-NOSH
Boot Version: PSPU-Boot(25CLK) 1.0.12.18m3
Hardware Version: 5.0
Firmware Build Time: Oct 29 2012 18:07:13

I have no idea what "normal" is for cable modems, but here are the details from mine (I'm not having any problems...at home that is)
 

 
Model Name: SB6121
Vendor Name: Motorola
Firmware Name: SB_KOMODO-1.0.6.10-SCM00-NOSH
Boot Version: PSPU-Boot(25CLK) 1.0.12.18m3
Hardware Version: 5.0
Firmware Build Time: Oct 29 2012 18:07:13