Doing some testing over the past few days, it appears after restarting the services (Nginx, Apache) the system run's its fastest and then after about 48 hours starts to slow down a bit.
I have created a job that will automatically restart those services ever morning at 4:45 AM Eastern time. The first time this job was done was this morning and I was awake for it and the total down time was about 7 seconds while these services were stopped and restarted.