It's quite possible the connection is intermittently dropping during the show. This will not show up in a speed test unless you happen to test the moment this happens. However, you can test for this with a continuous ping test while watching a show.
If you have a laptop, bring up a command prompt (right-click the Start menu and select either "Command Prompt" or "Windows PowerShell"). Type the following command and press enter: ping -t 8.8.8.8
Ideally this should run on the same Wi-Fi network as what your Netflix is streaming on, unless you are streaming on a wired network connection (e.g. network cable plugged into the TV).
When the show starts buffering, have look at the ping responses on the laptop. If it shows multiple "Request timed out" messages, this indicates a dropped connection. If it's still showing replies, but the "time=" value spikes, this indicates network congestion at the time, such as either something hogging the network, Wi-Fi interference or a link issue between your modem and your Internet provider.
The next test you can try is check whether the issue is with the Wi-Fi or your ISP by pinging the gateway IP. To find this out, type in the following command: ipconfig
Look for the line "Default Gateway" and then type in ping -t followed by the gateway IP address. E.g. if it's 192.168.1.1, type in ping -t 192.168.1.1
Continue watching until you see it buffering again. If the ping replies gave the same symptom as above, e.g. timeout or "time=" value spikes, the issue is with the Wi-Fi connection, such as interference. Otherwise the issue is likely with your Internet provider's link.