Site is slooooow

[Scott Greczkowski]

The strange part about this for you guys posting about issues, there are no issues shown in the logs when searching your IP addresses.

I was thinking this could be an IPV6 issue, but its about a 50/50 split between this being reported by IPV4 users and IPV6 users.

Again no changes were made to the server. So not sure why this started happening this afternoon.

 

[Scott Greczkowski]

been speedy the last few minutes

This is how I want it to be always. I strive to provide speed!

 

[TRG]

The strange part about this for you guys posting about issues, there are no issues shown in the logs when searching your IP addresses.

I was thinking this could be an IPV6 issue, but its about a 50/50 split between this being reported by IPV4 users and IPV6 users.

Again no changes were made to the server. So not sure why this started happening this afternoon.

I thought maybe it was my ISP at first. (crummy DSL) But I've seen it several times on 5G as well.

 

[Scott Greczkowski]

Looking at the logs over the past 15 minutes I only see 2 slow transactions.

And that was viewing these posts...

/xen/threads/shuttle-launch-tonite-scrubbed-until-sunday.167649/post-1736528
/xen/threads/anything-change-on-82w.412346/

We are monitoring the Apache, NGINX, Php, and SQL servers.

 

[FTA4PA]

Just had a 30+ second page reload.

 

[TRG]

Looking at the logs over the past 15 minutes I only see 2 slow transactions.

And that was viewing these posts...

/xen/threads/shuttle-launch-tonite-scrubbed-until-sunday.167649/post-1736528
/xen/threads/anything-change-on-82w.412346/

We are monitoring the Apache, NGINX, Php, and SQL servers.

 

[HIFI]

Just had a 30+ second page reload.

as did I

 

[charlesrshell]

as did I

I'm running good now and fast

 

[HIFI]

honestly its been hit and miss

 

[Scott Greczkowski]

I am really starting to think this is something outside SatelliteGuys, as the logs are not showing anything. I am going to log into our ISP's forum and see if others are reporting issue there too.

 

[HipKat]

Everything coming up instant here. Server load is 0.51.

Anyone else?

Slow for me loading home page just now, then clicking on next page was fast, then the next click hung for about the noted 20 seconds

 

[Scott Greczkowski]

Nothing on the ISP's forum. But the logs are coming up empty and we have logging on everything running.

We did have a resolv issue before but that has been fixed and tested a number of times and everything DNS wise out from the server is great. We turned off the CDN just in case one of the CDN server was having issues...

Just did a 10 minute test, and we did not have one web (apache / nginx) one PHP, one cache or mysql process last over 2 seconds. Yet during this time a few of you reported issues. With nothing being logged I am more and more thinking this is something happening outside our servers.

I am running.a few more tests but this is what its looking like and I need to get to bed as 4:30 am comes early.

Wish I had a fix.

 

[Jimbo]

10 minutes ago, I clicked on Alert and it took a good 10 seconds to load, several times.
Then I went out and came back in and it took a good 30 seconds to load anything.

Using Dark theme, Chrome on PC desktop

 

[Scott Greczkowski]

Here is what ChatGPT is saying... (Running a 30 minute test now)

If the next 10–20 second backend stall occurs and the strace still shows nothing blocked for several seconds, that becomes a very meaningful clue: PHP is likely spending the time inside user-space code rather than sitting on a kernel-visible wait.

That would shift us toward XenForo/add-on execution, possibly a loop or expensive processing path, rather than MariaDB, Redis, DNS, or socket waiting.

For now, I'd keep the MariaDB slow log enabled and leave all service settings alone. We have enough instrumentation in place that the next genuine stall should tell us much more.

 

[navychop]

9:20-9:24 It's slow loading

iPhone 15ProMax Safari.

 

[FTA4PA]

Not sure if this means anything but when it hangs I see it hanging at moderate.cleantalk.org

 

[Scott Greczkowski]

Yes you guys are on to something which ChatGPT just caught as well... our Bot Guard is trying to do reverse DNS lookups for bots and in addition is trying to access the bot guard database which is hosted by the Bot Guard folks... It is timing out to them, and that could cause the issue we are seeing...

I just disabled on feature on it to see if it corrects the problem. If it does not I will disable the add on completely.

 

[FTA4PA]

Well after you disabled that I just got an immediate page load!

 

[FTA4PA]

Working great now!

 

[Scott Greczkowski]

The suggestion from ChatGPT is this...

I'd also leave that BotGuard DNS option off tonight. Given that we directly observed its reverse-DNS routine spending 5 + 1.75 + 3 seconds waiting on DNS during PHP execution, it's entirely plausible that other IPs could produce even longer retry chains.

So we will eave that off and go to bed.