Site issues

I am training KE4EST how to reset things if I am away from a computer.

Today I only knew I was down as the server emailed me and told me all he wasn’t running. And I saw it on my watch.

At my job work has increased greatly for me. The person who was over me has left and all his work has been thrown in my lap.

I may have to look into getting us onto Sucuri or Cloudflare if these attacks keep happening.

I will really look into it when I get a chance hopefully by this weekend.


Sent from my iPhone using SatelliteGuys
 
Ok... not fixed yet... an we are still getting attacked. But I think I found a temporary work around which will hopefully stop the web server from running.

Lets see how this goes. I have to get to bed soon as I have a important early meeting tomorrow at work. So fingers crossed this works. :D
 
Good morning,

Well my fix worked for overnight... sort of. It kept things running however the Apache server still kept locking up giving a Connection Refused message a few times overnight. Luckily with the change I made it restarted the Apache server a few seconds after the Connection Refused issues happened. The log show the ModSecurity seems to be the one causing it.

I am in the office early today, had a vendor in doing some work here and I had to let them in and meet with them, so now I am here at my desk and I plan to work on the server again and hopefully fix this mess once and for all.

Because of this the site may be unavailable for a few minutes at a time. I will be recompiling and upgrading our Apache web server, as well as ModSecurity.

I would rather not put us behind a Sucuri setup again as that caused issues itself, not to mention the expense, but if its something we need to do then I will have to do it.
 
Ok one issue I kept seeing in the logs was issues with ElasticSearch, which is our search engine we use here on SatelliteGuys. We were running version 6.8.19 which I have removed and have installed the latest version which is 7.15.0.

The search index is still building as I type this almost 3 million out of the 4.5 million posts have been indexed. As you can imagine this is both CPU and Memory intensive as its doing its indexing. But it should be done shortly within a half hour.

As I am going through the logs and looking at the processes I can see other things I can adjust as well to help with performance.

Thank god for Google. :D Its helping me find, identify and fix some of these things. :D
 
Sounds like a job for Fail2Ban.
Not quite.

The issue is the attack is trying to use the Xenforo proxy.php to pull bad images from other sites. Mod security sees the proxy.php of our server being the bad guy as its the one that is trying (and failing) to grab the bad images... so it keeps trying to ban our server from the server.

I have put in new rules last night found at owasp-modsecurity-crs/REQUEST-903.9006-XENFORO-EXCLUSION-RULES.conf at v3.3/dev · SpiderLabs/owasp-modsecurity-crs and have been tweaking them over time.

I have been tweaking things all morning, updated APache, Php and ModSecurity, upgraded ElasticSearch and have just removed my cron job which was restarting the services every 45 minutes.... let see how this goes.
 
  • Like
Reactions: charlesrshell