SatelliteGuys Server Update

[Scott Greczkowski]

Hi folks, here is an update on our server. I feel it is important to let you guys know whats going on, what will be happening and why.

For the most part our server is running great. However we are running up against some deadlines which I will have to address.

We are currently running CentOS 7 under PHP 7.4 using Apache and Nginx compiled through Engintron. Our control Panel is WHM / Cpanel.

Nginx was setup through Engintron as Cpanel did not support Nginx when we moved to this server. However Cpanel now supports Nginx natively.

PHP 7.4 is coming to its end of life, and we will need to upgrade to PHP 8.1. This move could break some of our plugins which have not been updated (by the authors) in some time, although in researching most should work.

With CentOS we are being encouraged to upgrade to AlmaLinux 8. AlmaLinux OS - Forever-Free Enterprise-Grade Operating System With AlmaLinux we are told we can do an in place migration. However to do this we must be using default WHM / CPANEL items, which means we would have to get away from our
Engintron installation of NGINX and run the Cpanel installation of NGINX. The NGINX implemntation in Cpanel was a mess when it was first added, but now it appears to be rather stable.

In addition some of the caching we use such as redis cache may need to be disabled for the upgrade. But once done I should be able to add it back in again.

I worked hard to optimize the server we have now and did a lot of work using CDN's to optimize delivery of items to you from servers close to your location. By doing this upgrade we are going into the great unknown. I don't know what issues we will run into or what effect the changes will have on the speed of the site. And while I would like to stay on the server we are on now, if I have issues we may have to move to a new server. I am trying to avoid that due to the costs, but will if necessary.

Now when do I expect to do this work? Good question. I do expect it will take a few hours to do and we will be down during the time I am working. I will try to keep the downtime to a minimum. With everything going on and the holidays I assume it will be on a Saturday or Sunday in the afternoon / evening. Because my schedule is always changing I will do my best to give a 24 hour notice. And when i do work I will be working to get the site back online as quick as possible. With that said there may be optimization that will need to be done to get the site humming nicely. And I am not sure if I am going to upgrade everything at once or save the operating system upgrade for another time. I am thinking my first priority is to get the site running default Cpanel addons first before trying to migrate to the new OS.

And that is the server status for now. I will update things when I know more.

Thanks for your support!

 

[TheKrell]

Just keep a disaster recovery backup in case there is a big snafu. I've had a few of those.

 

[Scott Greczkowski]

Doing a quick check of the system looks like I have a few things to take care of before upgrading anything...

* 06-11:11:38 (2671) [WARN] *** Elevation Blocker detected: ***
You have postgresql-server version 9.2 installed.
This is upgraded irreversably to version 10.0 when you switch to AlmaLinux 8
We recommend data backup and removal of all postgresql packages before upgrade to AlmaLinux 8.
To re-install postgresql 9 on AlmaLinux 8, you can run: `dnf -y module enable postgresql:9.6; dnf -y install postgresql-server`


* 06-11:11:38 (2671) [WARN] *** Elevation Blocker detected: ***
You are using MySQL 5.7 community server.
This version is not available for AlmaLinux 8.
You first need to update your MySQL server to 8.0 or later.

You can update to version 8.0 using the following command:

/usr/local/cpanel/bin/whmapi1 start_background_mysql_upgrade version=8.0

Once the MySQL upgrade is finished, you can then retry to elevate to AlmaLinux 8.


* 06-11:11:39 (3425) [ERROR] 2 package(s) installed from unsupported YUM repo 'ul' from /etc/yum.repos.d/ul.repo
* 06-11:11:39 (3436) [WARN] Unsupported YUM repo enabled 'EA4-experimental' without packages installed from /etc/yum.repos.d/EA4-experimental.repo
* 06-11:11:39 (3436) [WARN] Unsupported YUM repo enabled 's3tools' without packages installed from /etc/yum.repos.d/s3tools.repo
* 06-11:11:39 (3425) [ERROR] 1 package(s) installed from unsupported YUM repo 'ul_webhostbox' from /etc/yum.repos.d/UL_webhostbox.repo
* 06-11:11:39 (2638) [WARN] Unknown error while checking blockers: Wide character in syswrite at /usr/local/cpanel/3rdparty/perl/536/cpanel-lib/Log/Log4perl/Appender/File.pm line 306.

* 06-11:11:39 (692) [WARN] Please fix the detected issues before performing the elevation process.

 

[navychop]

. I do expect it will take a few hours to do and we will be down during the time I am working.

Must

Find

WHISKEY!

 

[Scott Greczkowski]

I have all the issues fixed from the upgrade checker except for updating the MySQL server to version 8. I may do that tonight. It will save me work later, and may help to speed up the site more too.

 

[Scott Greczkowski]

Overnight tonight the server may be slow as I will be doing a full server backup starting at 1am Eastern Time.

 

[navychop]

Gave up sleeping?

 

[c-spand]

We are back! Yeaaaah.

 

[Scott Greczkowski]

OK... I updated our MySQL from version 5 to 8. The long part was converting our charset from utf8mb3 to utf8mb4.

I wasn't planning on doing it but support logged into the server earlier trying to fix our other issue and told me to update the MySQL immediately. (Of course when they were posting around THEY crashed the server for a few minutes with their fix which didn't work)

I was happy to see they fixed things and got the site back online by the time I drove home.

Sorry for the downtime. . Just trying to move us ahead. Most other forums just setup a server, load the software and let it run and never touch it again. I l ike keeping us updated and secure.

 

[Scott Greczkowski]

Now that the MySQL is up to date support is playing with the server again. Hopefully no more downtime, but can't guarantee it. I am going to step away from the PC and have some dinner.

 

[charlesrshell]

Now that the MySQL is up to date support is playing with the server again. Hopefully no more downtime, but can't guarantee it. I am going to step away from the PC and have some dinner.

Thanks Scott

 

[dweber]

Thank you for all the time you spend working on the site. Your work is greatly appreciated.


Sent from my iPhone using Tapatalk

 

[Scott Greczkowski]

If you get some service unavailable messages, that just me again. Shouldn't happen for more than a few seconds.

 

[TheKrell]

Of course when they were posting around THEY crashed the server for a few minutes with their fix which didn't work

Yeah, I was getting an error 508 earlier. Gone now.

If you get some service unavailable messages, that just me again. Shouldn't happen for more than a few seconds.

I'm liking what I see so far. Now, on IE11 I get to see and use all the different "like" emoticons on any post, and not just the OP as I noticed this morning.

 

[Scott Greczkowski]

I believe the likes issue you are seeing is due to caching and optimization. I have been working to find the sweet spot.

With that said I have been working on this since early this morning. Time to turn off the computer for tonight.

 

[Scott Greczkowski]

Been working on things again since I got up. Made some more minor changes to the optimization which hopefully does not harm TheKrell's IE 11 stuff. Lol.

 

[Scott Greczkowski]

Ok I went through all the Xenforo Settings (forum software) and believe I have setup / optimized everything the way I want it for best performance.

Yesterday I upgraded the MySQL server from 5 to 8, and have also been optimizing things there as well.

I still need to move SatelliteGuys over to PHP 8.1 from 4.7, I have tried moving us over and keep on getting errors, so I will need to clone the site on the server and get the clone working first.

Once I am done with that then we have about a year to move from CentOS 7 to a new OS. Again I want to get rid of the plugins (such as Nginx) and move to the one that is built into CPANEL which should help when moving to the new OS since this move will be done by supported cpanel scripts.

Its been a busy two days but I am happy with the results so far.

 

[raoul5788]

Thanks for all the great work Scott.