You should be able to stream without issues... except when your speed looks like this.
https://testmy.net/db/3jtqGbupv
At that time you may experience quality fluctuation but I would still bet it wouldn't cause major buffering issues.
If you're able to hook directly into the modem and get better results we should look to improve your wifi. Could be as simple as better router placement or changing the channel in the router settings.
For all of your testing in this instance, pick a size and stick with it throughout your testing. In your case, 50 MB should do the trick. You're already doing the right thing by focusing on the download test. Just navigate to the download page and manually select the size.
* I know you already did but test directly to the modem again, reset the power to have the modem get the computer Mac address. Make sure we get a nice baseline from right now. Run the download test like 3 times.
* Next plug the router in, reset the power on the modem, plug directly into the router this time and run 3 more tests. Keep the test variables the same for a more scientific comparison. The only thing that should change is the router being added.
* Do 3 more tests on wifi about 10 feet from the router. Do another 3 tests further away in an area you often use the wifi. Bring your laptop near your TV and test the wifi to that location too.
Compare those results. If the wifi results are terrible you may just have interference in the area. Sometimes going into the router settings (often found at http://192.168.1.1) and changing the wifi channel helps. Change it and test to see if there is an improvement. Also look at the placement of the router. You want it away from other electronics, even devices that don't emit wifi can and will interfere. The best placement is at the center of your home (away from any brick), the worst is in the corner. Most routers have omnidirectional antenna, so the radio waves go in 360 degrees. If you place it in the corner of your home you're only effectively getting 90 degrees of the total output where you need it, 270 degrees is going away from your home. Having said that, the router in my own home is placed in a way that wastes 180 degrees and I get great performance everywhere. You can actually take a reflective surface (in a pinch the inside of a Pringles can works) and place it behind the antenna in a way to make it reflect the signal back to the house. Just make sure it doesn't have lines, crinkles or wrinkles, nice smooth reflective surface.