Aware of the RabbitEars Outage

[Trip]

Good morning,

I am aware of the outage, and have no information to share at this time.

- Trip

 

[Trip]

Looks like we're back up and running. Not sure what happened, so doing a full backup out of pattern right now just in case.

- Trip

 

[Trip]

I'm aware that it's down again.

This is the third outage in the last month. It appears there may be some kind of sporadic hardware issue but I've been unable to get any kind of error message to be able to diagnose it. Today's issue is particularly dumb. The server was being flaky but there was nothing in the logs that looked particularly helpful, so I decided to try rebooting to see if it would clear up, but it didn't come back up. I shouldn't have done that, and quite frankly, I should know better than to do that without someone handy to poke it for me if necessary.

Hoping to have it back up and running in the next few hours. Good news is that in the worst case, I do have a full backup from April 8, including the Live Bandscan data and coverage maps.

- Trip

 

[Trip]

We're up again.

- Trip

 

[Trip]

Made a post about the recent issues on the RabbitEars Blog.

Third Unscheduled Outage in a Month - RabbitEars Blog

I wanted to take a moment to talk a little about the sudden, recent unreliability of the RabbitEars server. Yesterday's 10 hour outage was the third unscheduled outage in the past

www.rabbitears.info

- Trip

 

[LMckin]

Made a post about the recent issues on the RabbitEars Blog.

Third Unscheduled Outage in a Month - RabbitEars Blog

I wanted to take a moment to talk a little about the sudden, recent unreliability of the RabbitEars server. Yesterday's 10 hour outage was the third unscheduled outage in the past

www.rabbitears.info

- Trip

Looks like it down again.

 

[Trip]

Looks like it down again.

We're back.

- Trip

 

[Trip]

All,

The current outage is actually a power outage at the facility where the server is hosted. So whenever the power comes back, so too should RabbitEars.

EDIT: We're back.

- Trip

 

[Teehar]

All,

The current outage is actually a power outage at the facility where the server is hosted. So whenever the power comes back, so too should RabbitEars.

EDIT: We're back.

- Trip

Really love the rabbitears site,appreciate all the information provided there.

 

[Trip]

I'm aware of the current outage.

- Trip

 

[Trip]

We're back. I'll be making a blog post on the RabbitEars Blog about the outage later today.

- Trip

 

[Trip]

Blog post: Today's Lengthy Outage - RabbitEars Blog

- Trip

 

[hancox]

Down again this morning...

 

[Trip]

Yep. Note sent off to Silica Broadband. Server's acting up again, but no idea yet what's going on specifically today.

- Trip

 

[Scott Greczkowski]

Just got notice its back online.

 

[Trip]

The logs ended before the outage, but it's complaining earlier in the logs about disk issues. Might have to try to move up my visit out there to replace things...

- Trip

 

[Mr Tony]

Not working right now. Get half the main page then
Error connecting to RabbitEars database: 2002 No such file or directory

 

[Mr Tony]

Working now
Yay!!!!

 

[Trip]

One or more of the disks is definitely on the way out. I've worked with Dave at Silica Broadband and we have a plan to hopefully replace both disks and load a fresh operating system install by Monday evening. So don't be shocked if it's up and down until then; it's very clearly unhappy and unstable right now.

- Trip

 

[Trip]

Knock on wood, I've been having good luck with my backups so far. So good, in fact, that I went ahead and reenabled the Signal Search Map this morning and have made some updates, but have left the Blog which is far less used. I'll do another full database backup to restore from tomorrow morning, I think.

The Live Bandscan is also still running, though it had a brief outage I can't explain this morning. I'm trying to leave that for the last moment but since a full backup of that takes about three hours, I'm probably going to have to kill it and back it up early afternoon tomorrow, depending on when the parts are scheduled to arrive at the server.

- Trip