06/02 - 03 Downtime
#1
06/02 - 03 Downtime
Hey this is just a short statement regarding the down time we had today and yesterday evening.

I made the horrible mistake of having a bad web host for the forum and wiki. 

I will be moving this off shortly after which I will release full details of all of the issues that I underwent with them.  For safety reasons until I have every infrastructure move to another provider I am going to refrain from giving out a lot of details.

Please keep watching the thread at the moment.

As far as I can tell there doesn't appear to be any data loss and I'm hoping to take tomorrow and get us moved off because I'm not going to risk this again. 

Thank you guys for your patience and understanding


Raion
Operator
*******

Posts: 581
Threads: 113
Joined: Nov 2017
Website Find Reply
06-03-2019, 09:26 PM
#2
RE: 06/02 - 03 Downtime
Lol. I thought ole Phantom 309 did it.

Octane2 - Octane II: 2x600MHZ R14000 CPU - 2.5GB RAM - V12 Graphics - DCD

Indigo2 IMPACT  - Indigo2: R10000 195MHZ CPU - 128MB RAM - MaxIMPACT graphics - being sold

Octane - Octane: 2x360MHZ CPU - 1GB RAM - EMXI graphics - sold.

O2 - O2: R5000 180MHZ CPU - 128MB RAM - sold
Krokodil
Octane II

Posts: 41
Threads: 2
Joined: Dec 2017
Find Reply
06-04-2019, 04:03 AM
#3
RE: 06/02 - 03 Downtime
Now that backups are taken and downloaded against host retaliation, allow me to talk at length about the horror that is: TechVPS (https://techvps.com)

I signed up with them after I did a one-month trial a few months ago. The performance was more than good, pricing phenomenal, and they answered basic sales and support questions within 4 hours.

But then this happens. 

It was about 9PM EST on June 2nd when I noticed the forum was down (I was in the middle of a lengthy post I was trying to recover). This resulted in a panic as I couldn't SSH or ping the VPS node, which led me to believe a node crash. I opened a ticket within 15 mins as I was also locked out of SolusVM (a VPS control panel solution.) It is copied below.

I may as well have pissed into a black hole. No response, not to this fucking minute, has anyone gotten back to me. 

I decided to go to sleep, as I normally do on Sunday, around 1AM EST.

I got up at 9. Still no forums, still no correspondence. 

I put in a management request, and even though i work in the hosting industry and hate this, I fired off another response.

This provider uses WHMCS, like most smaller-scale providers do. The ticket doesn't get responded to, it will stay in "Open" status. The status for it to have been answered, is usually "Answered" or "Awaiting Response" (This is usually configurable within WHMCS or your ticket software.)

So this means nobody has answered the ticket, which tells you nothing about this.

Well, I headed off to Annapolis to deliver MassiveRobot his PI 4D/35. and on the way up, it's back up. I finished the job and got back home at 11PM, where I have begun frantically backing things up (if you noticed a blip slightly before this post came up, that was me backing stuff up.)

Then the moment of truth as I logged into the box:

root@kagura ~ % w
11:14PM  up 78 days, 22:12, 1 user, load averages: 0.59, 0.44, 0.36

This means that the server was up the entire time and it was a network issue. 

This raises several questions for me, as a client:

1. Would it have been too much to ask to fucking email anyone about this? Update your status page? Etc. At my hosting company I work at (which I don't use for this because I don't mix work and personal projects) if an issue lasts beyond 20 mins, we open a status post, which posts to twitter, facebook and our own RSS powered status page. Pretty nifty. 

2. Why did my ticket not get a reply???

3. What are these morons doing?

Looking back, several flags:

I only ever spoke to two people at the company:

Dementi Borodin - an obviously foreign frontline worker.
John - has no surname in the ticket system, who handled some issues for me with setup.

I was also frantic because my backups I had done, mostly relying on ZFS snaps, came back as nonfunctional. I was of course frantic, worrying that we'd lose months of data. 

That's not gonna happen again. Praetor, and several others behind the scenes here, are being recruited to keep this backed up at multiple sources. 

Downtime is inevitable for a host, but TechVPS has handled this about as well as a dog handles chocolate. The worst possible way. 

Fuck TechVPS. Now that I have backups, I could give two shits if they try to turn us off. I'm going to spend the next day or two moving our asses off this joker of a VPS company. Meanwhile, they will be dragged through the mud repeatedly.

I'd like to thank everyone who has contacted me with messages of encouragement and more through this difficult time. You live and you learn, and this place means too much for me to lose it all.


And for TechVPS, if you read this - and you wanna make it right, refund me at least half of my hosting plan. I'll give you 48 hours with your management ticket if three fucking days go by from the time that was filed and I get nothing? You're getting charged back on EVERY cent I've paid. and I'll make sure your reputation is ruined. Fucking cocksuckers.


Attached Files Thumbnail(s)
       


Raion
Operator
*******

Posts: 581
Threads: 113
Joined: Nov 2017
Website Find Reply
06-04-2019, 05:42 AM
#4
RE: 06/02 - 03 Downtime
Moved us over successfully to a new provider. I've charged back everything with TechVPS. Knowing paypal, I'll get every cent back.


Raion
Operator
*******

Posts: 581
Threads: 113
Joined: Nov 2017
Website Find Reply
06-04-2019, 08:46 AM
#5
RE: 06/02 - 03 Downtime
TechVPS resells servers from Colocrossing and according to LowEndTalk this generally is a source of issues.


Raion
Operator
*******

Posts: 581
Threads: 113
Joined: Nov 2017
Website Find Reply
06-04-2019, 09:10 AM
#6
RE: 06/02 - 03 Downtime
The VPS service I have, who shall remain nameless, they have awfully slow support times as well. It used to be 20GB of disk space, then it was changed to 40, but I had to chase them down to get the upgrade and it took several days of waiting before they finally got around to doing it.

A year ago I signed up for a Canadian hosting provider, who claimed they were still in business when I asked them on social media, despite their site looking pretty ragged. I got an invoice (with an incorrect date), but never, ever heard anything back from them, although no charges were applied. Just stupid.

Octane2 - Octane II: 2x600MHZ R14000 CPU - 2.5GB RAM - V12 Graphics - DCD

Indigo2 IMPACT  - Indigo2: R10000 195MHZ CPU - 128MB RAM - MaxIMPACT graphics - being sold

Octane - Octane: 2x360MHZ CPU - 1GB RAM - EMXI graphics - sold.

O2 - O2: R5000 180MHZ CPU - 128MB RAM - sold
(This post was last modified: 06-04-2019, 02:11 PM by Krokodil.)
Krokodil
Octane II

Posts: 41
Threads: 2
Joined: Dec 2017
Find Reply
06-04-2019, 02:04 PM
#7
RE: 06/02 - 03 Downtime
Good work! Guess we're all a bit jittery after having the rug pulled under us with Nekochan (which had its fair share of downtime previously it has to be said). Many thanks for all your efforts and for sorting things out. Big thumbs up from me!

<p>[Image: Indigo2_IMP.png] R10000 195MHz, 384MB RAM, MaxIMPACT (1MB), 36GB 15k &amp; 300GB 10k drives, new/quiet fans, 100Mb G160 NIC, IRIX 6.5.22 <br>[Image: Fuelb.png] R14000 600MHz, 4GB RAM, V10 Graphics, 72GB 15k &amp; 300GB 10k drives, new/quiet fans, 1Gb NIC, IRIX 6.5.30 <br>[Image: O2.png] in storage...</p>
Trippynet
Indigo2 IMPACT

Posts: 158
Threads: 5
Joined: Dec 2017
Find Reply
06-04-2019, 07:28 PM
#8
RE: 06/02 - 03 Downtime
Gods above you had no idea how stressful it was. Especially since I was concerned about the backup situation.

Over the next couple of weeks I'm going to be aggressively restructuring how I do backups and we're going to have at least three other people with copies of all the site information stored in safe spots.


Raion
Operator
*******

Posts: 581
Threads: 113
Joined: Nov 2017
Website Find Reply
06-04-2019, 10:55 PM
#9
RE: 06/02 - 03 Downtime
(06-04-2019, 10:55 PM)Raion Wrote:  Gods above you had no idea how stressful it was. Especially since I was concerned about the backup situation.

Over the next couple of weeks I'm going to be aggressively restructuring how I do backups and we're going to have at least three other people with copies of all the site information stored in safe spots.

Thank god you noticed!
Nice job Smile

[Image: mnTvJeq.png]
BackPlaner
Jurassic Technologist

Posts: 180
Threads: 27
Joined: Sep 2018
Find Reply
06-05-2019, 11:11 AM
#10
RE: 06/02 - 03 Downtime
Great job Raion! Thanks for taking care of this.

Any word from TechVPS about this already? Or did they decide to stick their heads in the sand?
dexter1
Global Moderator
******

Posts: 242
Threads: 10
Joined: May 2018
Find Reply
06-08-2019, 06:10 PM


Forum Jump:


Users browsing this thread: 1 Guest(s)