You may have to unplug your modem for like 10 seconds and plug it back in to get it to assign to the MAC address of the newly connected device. Once it pairs to the new device you'll have to do it again if you want to switch it back or try a different directly connected device.