I doubt that your router is the limiting factor, though you may want to check/replace the cables if they are suspect. Also, occasionally the DSL filter goes bad and needs to be replaced. Do try power-cycling both the modem and the router, as suggested above.
Sometimes Wi-Fi can be slowing down things, but you can easily check for that by connecting your computer to the router (or even the modem if you know how to do that) directly with the Ethernet cable and running the speed tests.
You can also run some diagnostics, like Ping or Tracert commands, if you are familiar with them. For example, from DOS command line run:
PING yahoo.com -t
(instead of yahoo you can put some other site that responds to ping) - this command will run until you stop it with Ctrl-C. It will tell you how many packets are lost. If you get a lot of lost packets, then you have a problem somewhere: bad connection, noisy line, or something else. Though the problem may not be on your end. Tracert command can help you figure out where exactly the packets are lost. Just google Tracert for detailed instructions.