Oh boy...

[Scott Greczkowski]

fail2ban can parse all manner of text from a log. It isn't limited to failed logins. If there's a pattern (such as a "[security2:error]") and an IP address, that's all you need to get the address blocked at the operating system level (extremely efficient).

Getting the database involved extends the DOS attack to the database.

From what I know that database is not stored in the MySQL database. Its stored within the ModSecurity itself they just call it a database is its a list of things.

Here is a look at warnings and attacks.

[Thu May 02 12:42:34.783512 2024] [security2:error] [pid 304485:tid 23447536985856] [client 87.121.69.52:50276] [client 87.121.69.52] ModSecurity: Warning. Match of "rx ^(?i:(?:[a-z]{3,10}\\\\s+(?:\\\\w{3,7}?://[\\\\w\\\\-\\\\./]*(?::\\\\d+)?)?/[^?#]*(?:\\\\?[^#\\\\s]*)?(?:#[\\\\S]*)?|connect (?:\\\\d{1,3}\\\\.){3}\\\\d{1,3}\\\\.?(?::\\\\d+)?|options \\\\*)\\\\s+[\\\\w\\\\./]+|get /[^?#]*(?:\\\\?[^#\\\\s]*)?(?:#[\\\\S]*)?)$" against "REQUEST_LINE" required. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "47"] [id "920100"] [rev "2"] [msg "Invalid HTTP Request Line"] [data "CONNECT google.com:443 HTTP/1.1"] [severity "WARNING"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "9"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/INVALID_REQ"] [tag "CAPEC-272"] [hostname "google.com"] [uri "/"] [unique_id "ZjPCekGBSwD4nUIAxBLr-QAAAQs"]
[Thu May 02 12:44:14.841972 2024] [security2:error] [pid 305046:tid 23447587415808] [remote 45.8.227.175:47724] [client 45.8.227.175] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "157"] [id "920180"] [rev "1"] [msg "POST request missing Content-Length Header."] [data "0"] [severity "WARNING"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "9"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/INVALID_HREQ"] [tag "CAPEC-272"] [hostname "www.satelliteguys.us"] [uri "/xen/"] [unique_id "ZjPC3r7ndAQvjnG_WIqr3QADhA0"]
[Thu May 02 12:46:00.448709 2024] [proxy_fcgi:error] [pid 305046:tid 23447509669632] [client 17.241.227.221:59316] AH01071: Got error 'Primary script unknown'
[Thu May 02 12:56:13.952462 2024] [security2:error] [pid 292846:tid 23447547492096] [client 14.215.163.132:48184] [client 14.215.163.132] ModSecurity: Warning. Pattern match "\\\\b(keep-alive|close),\\\\s?(keep-alive|close)\\\\b" at REQUEST_HEADERS:Connection. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "198"] [id "920210"] [rev "2"] [msg "Multiple/Conflicting Connection Header Data Found."] [data "keep-alive, close"] [severity "WARNING"] [ver "OWASP_CRS/3.0.0"] [maturity "6"] [accuracy "8"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/INVALID_HREQ"] [hostname "154.38.162.185"] [uri "/"] [unique_id "ZjPFrWWzP8b3C7KDm9AeOQAAAEY"]
[Thu May 02 13:02:32.253476 2024] [security2:error] [pid 292846:tid 23447536985856] [client 17.246.19.207:38694] [client 17.246.19.207] ModSecurity: Warning. Pattern match "(?i:(?:[\\\\s()]case\\\\s*?\\\\()|(?:\\\\)\\\\s*?like\\\\s*?\\\\()|(?:having\\\\s*?[^\\\\s]+\\\\s*?[^\\\\w\\\\s])|(?:if\\\\s?\\\\([\\\\d\\\\w]\\\\s*?[=<>~]))" at ARGS:_xfRequestUri. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf"] [line "65"] [id "942230"] [rev "2"] [msg "Detects conditional SQL injection attempts"] [data "Matched Data: having-same-problem.338790/ found within ARGS:_xfRequestUri: /xen/threads/replaced-hopper-with-sling-and-having-same-problem.338790/"] [severity "CRITICAL"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "8"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-sqli"] [tag "OWASP_CRS/WEB_ATTACK/SQL_INJECTION"] [tag "WASCTC/WASC-19"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE1"] [tag "PCI/6.5.2"] [hostname "www.satelliteguys.us"] [uri "/xen/index.php"] [unique_id "ZjPHKGWzP8b3C7KDm9Af4gAAAEs"], referer: https://www.satelliteguys.us/xen/threads/replaced-hopper-with-sling-and-having-same-problem.338790/
[Thu May 02 13:02:32.256735 2024] [security2:error] [pid 292846:tid 23447536985856] [client 17.246.19.207:38694] [client 17.246.19.207] ModSecurity: Access denied with code 403 (phase 2). Operator GE matched 5 at TX:anomaly_score. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "30"] [id "949110"] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [severity "CRITICAL"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "www.satelliteguys.us"] [uri "/xen/index.php"] [unique_id "ZjPHKGWzP8b3C7KDm9Af4gAAAEs"], referer: https://www.satelliteguys.us/xen/threads/replaced-hopper-with-sling-and-having-same-problem.338790/
[Thu May 02 13:02:32.257174 2024] [security2:error] [pid 292846:tid 23447536985856] [client 17.246.19.207:38694] [client 17.246.19.207] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/RESPONSE-980-CORRELATION.conf"] [line "37"] [id "980130"] [msg "Inbound Anomaly Score Exceeded (Total Inbound Score: 5 - SQLI=5,XSS=0,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): Detects conditional SQL injection attempts"] [tag "event-correlation"] [hostname "www.satelliteguys.us"] [uri "/xen/index.php"] [unique_id "ZjPHKGWzP8b3C7KDm9Af4gAAAEs"], referer: https://www.satelliteguys.us/xen/threads/replaced-hopper-with-sling-and-having-same-problem.338790/
[Thu May 02 13:02:32.759008 2024] [security2:error] [pid 304937:tid 23447541188352] [client 17.246.19.28:43864] [client 17.246.19.28] ModSecurity: Warning. Pattern match "(?i:(?:[\\\\s()]case\\\\s*?\\\\()|(?:\\\\)\\\\s*?like\\\\s*?\\\\()|(?:having\\\\s*?[^\\\\s]+\\\\s*?[^\\\\w\\\\s])|(?:if\\\\s?\\\\([\\\\d\\\\w]\\\\s*?[=<>~]))" at ARGS:_xfRequestUri. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf"] [line "65"] [id "942230"] [rev "2"] [msg "Detects conditional SQL injection attempts"] [data "Matched Data: having-same-problem.338790/ found within ARGS:_xfRequestUri: /xen/threads/replaced-hopper-with-sling-and-having-same-problem.338790/"] [severity "CRITICAL"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "8"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-sqli"] [tag "OWASP_CRS/WEB_ATTACK/SQL_INJECTION"] [tag "WASCTC/WASC-19"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE1"] [tag "PCI/6.5.2"] [hostname "www.satelliteguys.us"] [uri "/xen/index.php"] [unique_id "ZjPHKHqfSzvk8YgV5pejGQAAAwk"], referer: https://www.satelliteguys.us/xen/threads/replaced-hopper-with-sling-and-having-same-problem.338790/
[Thu May 02 13:02:32.759372 2024] [security2:error] [pid 304937:tid 23447541188352] [client 17.246.19.28:43864] [client 17.246.19.28] ModSecurity: Access denied with code 403 (phase 2). Operator GE matched 5 at TX:anomaly_score. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "30"] [id "949110"] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [severity "CRITICAL"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "www.satelliteguys.us"] [uri "/xen/index.php"] [unique_id "ZjPHKHqfSzvk8YgV5pejGQAAAwk"], referer: https://www.satelliteguys.us/xen/threads/replaced-hopper-with-sling-and-having-same-problem.338790/
[Thu May 02 13:02:32.759617 2024] [security2:error] [pid 304937:tid 23447541188352] [client 17.246.19.28:43864] [client 17.246.19.28] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/RESPONSE-980-CORRELATION.conf"] [line "37"] [id "980130"] [msg "Inbound Anomaly Score Exceeded (Total Inbound Score: 5 - SQLI=5,XSS=0,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): Detects conditional SQL injection attempts"] [tag "event-correlation"] [hostname "www.satelliteguys.us"] [uri "/xen/index.php"] [unique_id "ZjPHKHqfSzvk8YgV5pejGQAAAwk"], referer: https://www.satelliteguys.us/xen/threads/replaced-hopper-with-sling-and-having-same-problem.338790/
[Thu May 02 13:02:33.286451 2024] [security2:error] [pid 304861:tid 23447534884608] [client 17.246.23.107:37974] [client 17.246.23.107] ModSecurity: Warning. Pattern match "(?i:(?:[\\\\s()]case\\\\s*?\\\\()|(?:\\\\)\\\\s*?like\\\\s*?\\\\()|(?:having\\\\s*?[^\\\\s]+\\\\s*?[^\\\\w\\\\s])|(?:if\\\\s?\\\\([\\\\d\\\\w]\\\\s*?[=<>~]))" at ARGS:_xfRequestUri. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf"] [line "65"] [id "942230"] [rev "2"] [msg "Detects conditional SQL injection attempts"] [data "Matched Data: having-same-problem.338790/ found within ARGS:_xfRequestUri: /xen/threads/replaced-hopper-with-sling-and-having-same-problem.338790/"] [severity "CRITICAL"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "8"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-sqli"] [tag "OWASP_CRS/WEB_ATTACK/SQL_INJECTION"] [tag "WASCTC/WASC-19"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE1"] [tag "PCI/6.5.2"] [hostname "www.satelliteguys.us"] [uri "/xen/index.php"] [unique_id "ZjPHKScItfrHDHUl3Lr1rQAAAsw"], referer: https://www.satelliteguys.us/xen/threads/replaced-hopper-with-sling-and-having-same-problem.338790/
[Thu May 02 13:02:33.286922 2024] [security2:error] [pid 304861:tid 23447534884608] [client 17.246.23.107:37974] [client 17.246.23.107] ModSecurity: Access denied with code 403 (phase 2). Operator GE matched 5 at TX:anomaly_score. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "30"] [id "949110"] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [severity "CRITICAL"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "www.satelliteguys.us"] [uri "/xen/index.php"] [unique_id "ZjPHKScItfrHDHUl3Lr1rQAAAsw"], referer: https://www.satelliteguys.us/xen/threads/replaced-hopper-with-sling-and-having-same-problem.338790/
[Thu May 02 13:02:33.287273 2024] [security2:error] [pid 304861:tid 23447534884608] [client 17.246.23.107:37974] [client 17.246.23.107] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/RESPONSE-980-CORRELATION.conf"] [line "37"] [id "980130"] [msg "Inbound Anomaly Score Exceeded (Total Inbound Score: 5 - SQLI=5,XSS=0,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): Detects conditional SQL injection attempts"] [tag "event-correlation"] [hostname "www.satelliteguys.us"] [uri "/xen/index.php"] [unique_id "ZjPHKScItfrHDHUl3Lr1rQAAAsw"], referer: https://www.satelliteguys.us/xen/threads/replaced-hopper-with-sling-and-having-same-problem.338790/

 

[harshness]

From what I know that database is not stored in the MySQL database. Its stored within the ModSecurity itself they just call it a database is its a list of things.

Even if it is in redis or memcached, it is still consuming resources in deciding what needs to be done with each successive volley.

Here is a look at warnings and attacks.

Given that each address only shows up three times, it appears that something like fail2ban is already in play (or that fail2ban is what the attack is designed to thwart by changing IP addresses every three tries).

In a DDOS attack (distributed between many remote machines), you may be looking at something like geo-blocking that can be done with fancier firewalls (i.e. pfSense) that implement region blocking.

There is a tool called GeoIP that makes iptables-level address blocking possible on a Linux server based on the geographical area (not great for world-wide botnets but if they're all from Hong Kong, this could be a big help). This can make the server non-responsive and the kernel level to specific geographical regions.

https://docs.rackspace.com/docs/block-ip-range-from-countries-with-geoip-and-iptables

I've not been subject to a DDOS attack but fail2ban has squashed many a DOS attack for me.

 

[Scott Greczkowski]

We have that installed, but still some get around those blocks.

 

[waylew]

I had a hiccup at 9:07PM PDT last night. I was reading posts and at that time when I went to a different post it hung up like there wasn't a website to connect to. I tried again at 9:09 and it was the same. I tried again at 9:29 and all was well.

I'll confirm Bobby's hiccup,only at 12am EDT,same bat time,same bat channel.

 

[Scott Greczkowski]

I'll confirm Bobby's hiccup,only at 12am EDT,same bat time,same bat channel.

The 12 am time jives with the backups.

Made adjustments to those start times and hope all is well after.

 

[savarese04]

Or maybe go for a new M3 MacBook Air. This one is a 2019 and has served me well.

If you have a donation drive, I'll happily contribute. Every worker needs good equip to do the job, regardless of whether it's a hobby or not.

I will say that your 2019 MBP has great resale value, broken screen or not, and that you can definitely get away with the base model M3 MBA. It's absolutely INSANE how fast the Apple silicon is. I have an M2 and I am convinced it will last me 10 years. You probably will end up spending less than $700 doing it that way and have a brand new machine.

 

[Bruce]

If you have a donation drive, I'll happily contribute. Every worker needs good equip to do the job, regardless of whether it's a hobby or not.

I will say that your 2019 MBP has great resale value, broken screen or not, and that you can definitely get away with the base model M3 MBA. It's absolutely INSANE how fast the Apple silicon is. I have an M2 and I am convinced it will last me 10 years. You probably will end up spending less than $700 doing it that way and have a brand new machine.

I have the ipad pro with the M2, you are correct, so fast.

Rumors are the new model pros will skip over the M3 and go right to the M4, supposed to be announced next week, we shall see, most people are usually wrong on guessing what Apple will announce.

 

[Scott Greczkowski]

The Apple Macbook Air would do me fine... I just need 16 gig... and I guess if you get 16 gig of ram you have to get the 512 meg of storage as well, every time I spec it out and choose 16 gig of memory it automatically upgrades the hard drive size as well, and that setup is around $1,300. For my use I can't see getting another Macbook Pro.

Its my Birthday on the 30th and my father asked what I wanted so I told him that and he laughed at me.

Money is tight but good news is we are done with our Chapter 13 bankruptcy in September. Then I will have so free cash again.

But again my hobby is running this place and my weather website. Those keep me busy and I am happy with that. My wife has the dogs and she is happy with them. (Don't get me wrong I like the dogs too, but they are hers.)

 

[Bruce]

Another blip, 12:15pm, page took about a minute to load.

 

[Scott Greczkowski]

Anyone else or just Bruce. Logs are not showing anything. (Unlike before) and no other reports

I wasn’t online then so I can’t say something happened.


Sent from my iPhone using Tapatalk

 

[Bruce]

Anyone else or just Bruce. Logs are not showing anything. (Unlike before) and no other reports

I wasn’t online then so I can’t say something happened.


Sent from my iPhone using Tapatalk

Took a minute to load, as I was waiting, opened a different tab, went to Raw Story, loaded as fast as usual, while this site was still trying to load.

Also was watching CNN via MAX at the same time using the Roku, no issues there either.

But since then, it has been fine, plenty fast.

 

[harshness]

Anyone else or just Bruce. Logs are not showing anything.

I didn't notice anything during that interval (around 09:15 PDT).

 

[Jimbo]

Anyone else or just Bruce. Logs are not showing anything. (Unlike before) and no other reports

I wasn’t online then so I can’t say something happened.


Sent from my iPhone using Tapatalk

I had an issue today when it wouldn't load, came back a few minutes later and it loaded normally ...
Figured you were working on something.

Don't remember what time for sure ....noonish might be right ...

 

[Bruce]

Another, 10:31pm, lasted a minute, all other sites loaded fine.

 

[telstar_1]

Bruce you are on Eastern Time, and Bobby is on Pacific time. I was online and active here at that time (Eastern Time) and had no issues no see no log file of any issues.

With this said there DID seem to be some kind of outside internet issues that started at approximately 2:28 am Eastern Time where people from outside the USA could not get to the server and those out of the country could not get in for 4 hours and 22 minutes (6:47am ET). This issue was outside our server as most of our remote monitoring were able to get to the server, but in places like Australia and New Zealand couldn't get here during those times.

I just paid for extra monitoring so our server is tested every 60 seconds from 30 different locations across the globe.

Bobby, I think the issue you may have seen was because of the backups, looks like 3 different backups of the SatelliteGuys database were happening at the same time. Cpanel does one... Xenforo does one and Jetbackup does one.

I just changed the Jetbackup one to 3am. Working on changing the Xenforo one to start at a different time (not midnight) depending on the backup being done it is a resource heavy thing as not only is it dumping a huge database, but then it also gzip's it up for sending it offsite. gzip is a hog. I need to change gzip for pigz which is much better and handles more CPU cores, so you don't even notice when its running. I am adding that to my todo list now.

How to Compress Files Faster with Pigz Tool in Linux

Written by Mark Adler, Pigz is an acronym for Parallel Implementation of GZip. It’s a nifty compression tool that helps you compress files faster in Linux.

www.tecmint.com

pigz is not a hog...?

 

[Bruce]

Massive blip, about 10:15am till finally now, everything else was loading fine on a different tab

 

[charlesrshell]

Massive blip, about 10:15am till finally now, everything else was loading fine on a different tab

Same here. Noticed it around 09:20 CST.

 

[Scott Greczkowski]

Had the log files still up and running at the time and noticed a major burp in the http2 protocol.

I recompiled again removing the http2.


Sent from my iPhone using Tapatalk

 

[harshness]

Those IP addresses are all property of Amazon Technologies so I'm guessing this is yet another DDOS attack. This time they're using domestic addresses but I doubt that they would be humans.

Removing HTTP/2 isn't the solution to a DDOS attack. HTTP/2 brings significant performance gains when it isn't being hammered.

 

[navychop]

I am SO GLAD I stopped keeping up with this stuff so long ago. 20+ years ago. It’s now a lifestyle.

I started programming in HEX, and sometimes in OCTAL, after a brief stint in binary. Taught me to appreciate high order languages.