Caffeinated-Geek.com

Received a complaint from a co-worker today – the long and short of it is his battery backup (UPS) wasn’t working properly. Upon suggesting that I try testing/replacing the batteries, I was told the batteries were new. I mentioned that since I was working on a customer-affecting issue, I’ll take a look shortly, and asked him to please email me with the battery date on the sticker affixed to the case of the UPS – and that will help determine our plan of action. His response was that “that’s not my job” and to “go look at it yourself”.

I figure he was just having a bad morning, but it brings up the topic of what you can do to help IT help you with your problems (also known as “how to NOT piss off IT”). Without getting into too many intricate details, my list is:

• Be patient. I realize your issue is important, but troubleshooting problems (be they yours or someone who is in queue ahead of you) takes time. Please don’t assume because something isn’t done immediately that you need to follow up by phone or in person – I work on tickets according to priority, then submission date. If you choose to call, you’re probably going to end up in voicemail and getting a call back when I’ve cleaned out my ticket queue. If you come to my desk, you’re likely going to end standing there waiting until I wrap up my current ticket or call.
• Set realistic priorities. At any point throughout the day, I probably have a minimum of 5 – 10 tickets open in my own queue, often many more; that means there’s only about a 1-in-10 chance that yours really is more important than everyone else’s. Continually marking your low-priority issues as an “emergency” will not get them fixed faster, but may get your true high-priority issue you open later pushed to the bottom of the queue.
• Don’t lie. If you changed something that may be related to your problem, man up. If you’re consistently going to let me waste my time troubleshooting instead of coming clean and providing me the whole story, then when the time comes that you have a real issue it’s going to take longer to get resolved since I’ll be spending my time looking to see what you screwed up but won’t admit to.
• Give details and note error messages. Provide usernames, email addresses, and callback numbers. It’s much faster for you to provide the information I need to troubleshoot than it is for me to go back and forth trying to squeeze information from you or wading through a metric ton of server logs. I’m not going to troubleshoot in the dark — if you send a ticket saying “email is down”, I’ll respond with an equally vague message saying “it’s working fine for me”. Doing this one step alone could mean the difference between having the problem persist for a few minutes or a few hours.
• Don’t argue, clarify. If you think I’m wrong about something, ask for clarification or explain that you thought it worked differently, but don’t simply start an argument. Not to be rude, but you called me for help and I’ve been dealing with issues like this for a long time. Almost every time someone wants to argue, it comes down to them not completely understanding the intricacies of protocols and services such as BGP, ATM, PPP, DNS, and SMTP. If you know I’m wrong, explain why and I’ll listen and admit it if so — and we can continue with getting your problem fixed. If you feel the need to have an argumentative conversation, please don’t waste my time – there are plenty of Internet forums out there for you to troll.
• Don’t play the blame game. Bickering about who is at fault for your document getting deleted, your workstation crashing, or your email bouncing is not going to resolve the issue. Technology breaks, mistakes happen, life goes on. Deal with SLA’s per your contract, but AFTER the service-affecting issue is resolved; don’t attack the person on the other end of the phone when it comes to settling the dispute.

Anything else I’m missing?

Complaints, Systems Admin headdesk, lusers

Dear user “jerryleecooper” at ZDnet’s talkback forums,

Enclosed is your invoice for replacement of one pair of soiled blue jeans. Please remit payment immediately upon receipt. Said soiling was caused due to excessive laughter in regards to your post titled “Why Linux will not displace Windows” at http://talkback.zdnet.com/5208-12355-0.html?forumID=1&threadID=31199&messageID=579806&start=43

I may also seek punitive damages caused by a thirty minute fit of laughter from your final remark:

“Its just not possible that a freeware like the Linux could be extended to the point where it runs the entire computer fron start to finish, without using some of the more critical parts of windows. Not possible.”

Thank you for calling out a community of technical users on something which you have made apparent that you have absolutely no knowledge of.

Sincerely,

Me

(Why have I never seen this post until today? Freakin’ classic.)

Humor headdesk, linux, lusers

I love three day weekends. Three consecutive 24 hour periods of hanging out with friends and family, finishing projects, and all around laziness.

A wicked summers-almost-over barbecue with the whole family, working on my stereo install in my car, and lounging around at home were on my agenda – but not dealing with people who forgot that this was the end of the month, and that they needed to pay their bill to us (or they’d get shut off).

Background: I’m a systems and network admin for a wholesale ISP. We provide dial-up, DSL, hosting, etc. Some of our wholesale customers use their own RADIUS system for authentication, some use a managed system on our side. It’s in violation of our contract with the wholesale ISP to activate accounts/tinker with the accounting functions directly for a subscriber in our managed system, and it’s impossible for me to activate an account on a system that they manage.

There’s something about a three day weekend when the calendar month rolls over that makes our wholesale customers forget to do little things like paying their bills. I can’t take it out on the poor technical people who have to call me; they’re usually just reacting to customers yelling at them. It’s their management, bookkeepers, accounts payable, whoever is responsible in their organization that has dropped the ball. What irks me the most is that we notify people if they haven’t paid NUMEROUS times before shutoff — and it doesn’t help. And that’s what causes my cellphone to ring non-stop this weekend.

Since the ISP’s tech folks don’t usually know that their management has neglected our invoice, it simply looks like a massive technical issue as their retail customers can’t log on, and they call our emergency outage paging system, which patches them through to me – which is when I get to inform them that their boss never paid us. Most of them, I can turn back on right away and have them take care of it on the next business day. There are others that are persistently late, and that I need confirmed payment from to turn back on. Of course, the person who handles that is out of town for the holiday, too. Great.

Better than the wholesale calls, though, are the retail customers — who aren’t supposed to be calling us at all. They usually come across the NOC phone number by stumbling across it in WHOIS, or by talking to the phone company (who gives our contact info as the service provider for their DSL, since they’re unaware of our wholesale program), or when given it by the wholesale partner. Note that a wholesale partner doing the latter is grounds to have them stuffed into a cannon and shot at the Earth’s sun. Oh, and I can’t forget to mention that part of the telephone IVR greeting says that if you’re an end-user, to not use the emergency paging system. They never listen and proceed to the paging system anyway.

The fun really begins when they get connected with me; the end users want to argue with me about how they are consistently on time with payments, and this is unfair, and how they’re going to go to another service provider — even after I’ve explained that I’m at a wholesale/upstream provider level and have no access to the accounting and user login functions for their service provider. Yes, they might be the perfect customer or they may have been turned down mistakenly, but it doesn’t change the fact that I cannot do anything for them. Yet, somehow, I’m expected to turn them back on, offer a credit for an account that doesn’t belong to us, and publish a three page letter to the local newspaper apologizing for the actions of one of our customer.

I’ll get right on that first thing tomorrow.

This all brings to mind an old statement I first heard several years ago said by a co-worker to a member of the sales department:

“Failure to plan on your part does not constitute an emergency on mine.”

Complaints, Network Admin, Systems Admin headdesk, lusers

Time for a bit of a rant.

I came in to work early today, hoping to get in some project time. No less than 10 minutes after my arrival, I’m noticing our bandwidth is seriously pegged. Pegged, as in one Fast Ethernet connection and one DS3 completely saturated. Some investigation showed that a site on one of our hosting servers had apparently received enormous press coverage, and we had thousands of users browsing through an image gallery with hundreds of photos on each page, and of course the “thumbnails” created via HTML. Ugh, IMG WIDTH and HEIGHT attributes do NOT make thumbnails people. Add to that video downloads of a 400MB AVI from poorly configured streaming clients, and I was not happy. Anyway…

Now that I’ve located the geyser, it’s time to find out why it’s dragging everything down. While attempting to stop the problem, I notice the culprit — a web server upgrade killed the bandwidth throttling module. Shit. Well, there’s more than one way to skin a domesticated feline — time to filter on the network level, or so I thought. You see, the version of IOS in use has a lovely bug on the Cisco 7200 platform which doesn’t allow me to filter outbound traffic on any interfaces. 30 minutes into attempting to beat my ACLs into submission, I jump 5 feet into the air as my cellphone rings. Maybe I should lay off the coffee.

Time to talk to my favorite customer. For 45 minutes. About the issue I’m attempting to fix. I let him know that I was attempting to fix the problem at that very moment, and that I’d be more than happy to call him back as soon as I was able to finish working on it, but that my hands were tied while he was complaining and throwing random questions of a questionably technical nature at me. Then, like magic, one of the hosting servers crash. Unfortunately, it’s not hosting the site that’s hammering our bandwidth. I can’t win today.

I ended up forcibly terminating the call with the customer, as they refused to let me defer their conversation to fix issues affecting thousands of people. I’m sure they’ll call back to complain. Of course, for him to understand, despite being told several times, that I’m unable to be on my phone at my desk and in the data center at the same time would require just a tiny fraction of logic. My theory is that he was in the john while the logic handouts happened.

How I love feeling helpless during service affecting conditions due to one subscriber’s lack of a logical thought process. I can’t wait to see where the rest of the day goes, as it’s only 9:05 am.

Ouch.

Complaints, Systems Admin headdesk, lusers

Something to ponder from a spam prevention mailing list I’m subscribed to:

“I have often wondered why I am in this business myself. I should have opened a Deli. You work reasonable hours and when you put your pastrami in the fridge at the end of the day, it doesn’t call you up in the middle of the night. When you come back in the morning it is pretty much exactly the same way you left it.”

Oye. Fitting for me today, as never in my life did I imagine I’d be taking a 4:15 am call because one user with a mis-configured modem can’t get online. :/

Complaints, Systems Admin headdesk, lusers