Log in to your router by typing in "192.168.1.1" or "192.168.0.1", it depends in what brand of router you have. You should get a password prompt...For Netgear routers, the default Username is "admin" and default password is "password". For Linksys routers, leave the username field blank, and the password "1234" or "password" or "admin". If none of these work, here is a list of all of the router's default passwords: http://www.phenoelit.de/dpl/dpl.html
Once you are logged in, it should be under "basic settings". Choose "Use these DNS servers" and type in 208.67.222.222 (primary) and 208.67.220.220 (secondary). These are the OpenDNS servers. On some routers, you can assign more than two, so you can put your ISP's DNS servers also if you want...