# 04Jan Outage



## Aldryic C'boas (Jan 4, 2014)

Well, that was rather annoying.  My apologies for the downtime folks - the power cable on KVM06 (which is hosting VPSB's SQL) was apparently of.. import quality:







The damn thing actually burned itself up, and brought the node offline.  Fran's checking on all of the other cables now to ensure we don't have a repeat of this.


----------



## Ishaq (Jan 4, 2014)

Who's chewing on the wires? >_>


----------



## drmike (Jan 4, 2014)

Gnarly cable destruction.

You lads are lucky it wasn't worse than that.  Pretty good scorching on the plug side 

Wondering the gauge and what the core of that cable is made from...

(as I check all my cables this afternoon)


----------



## johnlth93 (Jan 4, 2014)

lucky it din't like on fire or something  :huh:


----------



## tragic (Jan 4, 2014)

That's pretty dangerous, how did it happen?


----------



## Aldryic C'boas (Jan 4, 2014)

Just a bad cable from what we can tell.  It's happened before, but it's a rare occurrence.  Fran's checking over the rest of the cables to make sure none of the others are doing the same.


----------



## Francisco (Jan 4, 2014)

The rest are all fine.

Given where it was, there would have been the serial/sticker over it so it wouldn't have been easily spotted by anyone.

I was pretty surprised but I knew it was either a loose cable or a bad PSU when the IPMI wasn't pinging either. Little did I know the cable melted >_>

Francisco


----------



## tonyg (Jan 4, 2014)

A cable just doesn't fail in the middle run without outside influence.

That cable looks pinched or chewed (rodents) which broke through the insulation and shorted out.

Edit: Actually that cable looks cut...look at the sharp incision on the left side.


----------



## perennate (Jan 4, 2014)

tonyg said:


> A cable just doesn't fail in the middle run without outside influence.
> 
> That cable looks pinched or chewed (rodents) which broke through the insulation and shorted out.
> 
> Edit: Actually that cable looks cut...look at the sharp incision on the left side.


Maybe the metal was hot.


----------



## notFound (Jan 4, 2014)

Yeah, cables don't burn out like that. It would be uniform otherwise, likely not stared by an electrical fault but outside influence as mentioned. The charring would be as a result of the cut and probably some arcs and sparks.

Btw. Might be a bit more organised if you used them APC strip ones that have female kettle connectors instead of a standard plug, saves space etc., I've got that on my rack here and it's harder to get bad quality cables with that unless it's intentionally, I think some of mine are even from APC themselves. 

Sent from my GT-I8190N using Tapatalk


----------



## Francisco (Jan 4, 2014)

notFound said:


> Yeah, cables don't burn out like that. It would be uniform otherwise, likely not stared by an electrical fault but outside influence as mentioned. The charring would be as a result of the cut and probably some arcs and sparks. Btw. Might be a bit more organised if you used them APC strip ones that have female kettle connectors instead of a standard plug, saves space etc., I've got that on my rack here and it's harder to get bad quality cables with that unless it's intentionally, I think some of mine are even from APC themselves. Sent from my GT-I8190N using Tapatalk


I'll go poke around a little later to see if i can find the sticker that usually goes over that area. There's usually a barcode or things like that over that area. I don't even know if this was a cable we bought or one we took from FH's "power cable" bin.

Given what i'm seeing, there's soooo little copper being carried through this end of the cable. I chopped the cable in half further down and see that it's stranded (a run for each prong), but by the end of it there's almost no strand.



You can see that 1 of the feeds isn't even there now.

Francisco


----------



## Aldryic C'boas (Jan 4, 2014)

notFound said:


> Yeah, cables don't burn out like that. It would be uniform otherwise


In 5+ years as a fireman, I've _NEVER_ seen any type of electrical burn/fire (or any type of burn/fire) be _uniform_.


----------



## notFound (Jan 4, 2014)

Yeah not good, that looks like it can barely take an amp safely. I would just buy a bunch of my own rather than using the DCs, they probably got those straight from China probably with fake EC compliance labels etc. ;p

Sent from my GT-I8190N using Tapatalk


----------



## Francisco (Jan 4, 2014)

notFound said:


> Yeah not good, that looks like it can barely take an amp safely. I would just buy a bunch of my own rather than using the DCs, they probably got those straight from China probably with fake EC compliance labels etc. ;p Sent from my GT-I8190N using Tapatalk


I'd say 95%+ of our cables are the thick, heavy duty, ones that supermicro ships.

A few are the lighter ones but we ordered them off newegg. I'm not sure if this is a newegg or 'bin one. I'll give each cable a feel down and see if i can spot anything off.

Francisco


----------



## cubixcloud (Jan 4, 2014)

That's probably 16 gauge wire.


----------



## SkylarM (Jan 4, 2014)

Ishaq said:


> Who's chewing on the wires? >_>


Fran got hungry obviously!


----------



## pcan (Jan 4, 2014)

I reject power cables that are not bearing safety marks on the rubber plugs. I sectioned some of the rejects, and the copper wire is usually undersized. The cable may seem OK externally, but weight is mostly due to plastic/insulation. I believe this is related to increasing cost of copper, because I don't recall experiencing this in the past. Undersized cables may even feel warm to the touch; that's a very bad sign.

edit: I see from the picture that the plug has been damaged by the short-circuit current. The outlet contacts have been surely damaged in the same way; outlet should not be used again until replacement.


----------



## trewq (Jan 4, 2014)

Aldryic C said:


> In 5+ years as a fireman, I've _NEVER_ seen any type of electrical burn/fire (or any type of burn/fire) be _uniform_.


You're just the master of all trades aren't you?


----------



## Aldryic C'boas (Jan 4, 2014)

Not a master, no.  But I find that being idle is anathema to my sanity;  I tend to find more things to do rather than relax or take a break.


----------



## MCH-Phil (Jan 4, 2014)

It may be an expense, but safety should always come first.  Why not replace all questionable cables?  Understandably, you say this is rare.  Is it worth the chance of someone not catching it and something turning to rubble?  I've had a house burn down and can tell you it's not worth the chance.  Be safe '

@Aldryic C'boas, I have some family that are firefighters and have lived with a few.  Much respect.


----------



## Francisco (Jan 4, 2014)

MCH-Phil said:


> It may be an expense, but safety should always come first.  Why not replace all questionable cables?  Understandably, you say this is rare.  Is it worth the chance of someone not catching it and something turning to rubble?  I've had a house burn down and can tell you it's not worth the chance.  Be safe '
> 
> @Aldryic C'boas, I have some family that are firefighters and have lived with a few.  Much respect.


Cost is no concern, i've already been making a list of what cables to replace 

I got some of the fatty cables here that I'll drop in during one of our maint windows tomorrow.

Francisco


----------



## notFound (Jan 4, 2014)

Aldryic C said:


> In 5+ years as a fireman, I've _NEVER_ seen any type of electrical burn/fire (or any type of burn/fire) be _uniform_.


Of course, my bad, what I meant was basically, you of all people should be able to tell that it's probably originally due to 'other reasons', rather than originating by an electrical fault. That is unless there were fewer strands in that part of the cable for example then it's possible. ;-)


----------



## GIANT_CRAB (Jan 4, 2014)

INTRUDER ALERT: Red spy is in the base!


----------



## Aldryic C'boas (Jan 4, 2014)

WE NEED TO PROTECT THE BRIEFCASE

So listen up boy, or pornography starring your mother will be the _second_ worst thing to happen to you today.


----------



## DomainBop (Jan 4, 2014)

Aldryic C said:


> Well, that was rather annoying.  My apologies for the downtime folks - the power cable on KVM06 (which is hosting VPSB's SQL) was apparently of.. import quality:
> 
> 
> 
> ...


Reminds me of the time in 2001 I received a call from ServInt at 2AM in the morning telling me the power supply on one of my servers had shorted out and the server was a total loss (motherboard, hard drives both got fried when the power supply blew).


----------



## KuJoe (Jan 5, 2014)

I <3 Monoprice's 1ft and 3ft power cables. We buy them in bulk and have had 0 issues with them and cable management is a dream when you don't have 10ft power cables that are near impossible to manage if you don't use cable management arms.


----------



## drmike (Jan 5, 2014)

SkylarM said:


> Fran got hungry obviously!


There is a name for that behavior when horses start chewing up the stalls and such.  It's called cribbing and is caused by nutrient deficiencies.

Pregnant women are notorious for the same crazy behavior.


----------



## drmike (Jan 5, 2014)

Aldryic C said:


> WE NEED TO PROTECT THE BRIEFCASE
> 
> So listen up boy, or pornography starring your mother will be the _second_ worst thing to happen to you today.


You outing Obama again Aldryic?


----------



## dano (Jan 5, 2014)

I always use Tripplite heavy-duty cable when I power up a server that has "PC" size cords that came with it(paranoid about losing power supplies). Most time when I pull something from Ebay and it comes with a power cord, the cord looks like it's for a regular pc; recently bought a new HP DL360 and the cords that it came with were heavy guage wire, so I didn't use Tripplite power cable on this particular colo machine. I have had two experiences with machines that had cords that were so thin and had become so hot, that I decided that if the machine was going to be working hard and pulling AMPs, it's going to need some good quality power cables that are well insulated and of heavy gauge wire.


----------



## KS_Phillip (Jan 5, 2014)

This is one of the reasons we buy our own power cables, rated for [email protected]


----------



## shovenose (Jan 6, 2014)

Seen this at work several times - customers bought PCs, they ended up with a faulty or cheap power cable, similar end result to above.

And that's on a standard desktop computer, that's probably not even 2A


----------



## mtwiscool (Jan 6, 2014)

And this why vpsborad. Should use a master to master mysql configuration.


----------



## SrsX (Jan 6, 2014)

mtwiscool said:


> And this why vpsborad. Should use a master to master mysql configuration.


Oh god... I'm trying so hard not to say what I want to right now....


----------



## drmike (Jan 6, 2014)

mtwiscool said:


> And this why vpsborad. Should use a master to master mysql configuration.


vpsBoard will introduce more complexity this year no doubt


----------



## Coastercraze (Jan 7, 2014)

Extreme Cheapskates.

Yeah, I'd buy those 3Ft ones from Monoprice.


----------



## Francisco (Jan 7, 2014)

mtwiscool said:


> And this why vpsborad. Should use a master to master mysql configuration.


We shall call it *the cluster*.

What? At least some skilled people would be behind building this one.

Francisco


----------



## MannDude (Jan 8, 2014)

Francisco said:


> We shall call it *the cluster*.
> 
> 
> What? At least some skilled people would be behind building this one.
> ...


You must not have been around for this morning's timezone fiasco. 

Woops.


----------



## Coastercraze (Jan 8, 2014)

Francisco said:


> We shall call it *the clusterf**k*.
> 
> 
> What? At least some skilled people would be behind building this one.
> ...


FTFY


----------



## Francisco (Jan 8, 2014)

MannDude said:


> You must not have been around for this morning's timezone fiasco.
> 
> Woops.


Goddammit Curtis.

Hey that felt pretty good, no wonder the guys like saying it to me all the time >_>

Francisco


----------



## mtwiscool (Jan 8, 2014)

Francisco said:


> We shall call it *the cluster*.
> 
> 
> What? At least some skilled people would be behind building this one.
> ...


i buit one before 

worked perfect.


----------

