This is the problem... I haven't touched anything.
This is why it frustrates me when you guys report issues and I don't see them here. About the only one I have had see is the blue line on top when submitting posts, and that happens because one of the external services is having issues.
I monitor the site and most times it loads instantly. I really wish I knew where it was.
But with that said if I had to money I would hire someone to fully optimize the server just to make sure I don't have something screwed up.