Help IT help you (aka: “How to NOT piss off IT”)

June 16th, 2009

Received a complaint from a co-worker today – the long and short of it is his battery backup (UPS) wasn’t working properly. Upon suggesting that I try testing/replacing the batteries, I was told the batteries were new. I mentioned that since I was working on a customer-affecting issue, I’ll take a look shortly, and asked him to please email me with the battery date on the sticker affixed to the case of the UPS – and that will help determine our plan of action. His response was that “that’s not my job” and to “go look at it yourself”.

I figure he was just having a bad morning, but it brings up the topic of what you can do to help IT help you with your problems (also known as “how to NOT piss off IT”). Without getting into too many intricate details, my list is:

  • Be patient. I realize your issue is important, but troubleshooting problems (be they yours or someone who is in queue ahead of you) takes time. Please don’t assume because something isn’t done immediately that you need to follow up by phone or in person – I work on tickets according to priority, then submission date. If you choose to call, you’re probably going to end up in voicemail and getting a call back when I’ve cleaned out my ticket queue. If you come to my desk, you’re likely going to end standing there waiting until I wrap up my current ticket or call.
  • Set realistic priorities. At any point throughout the day, I probably have a minimum of 5 – 10 tickets open in my own queue, often many more; that means there’s only about a 1-in-10 chance that yours really is more important than everyone else’s. Continually marking your low-priority issues as an “emergency” will not get them fixed faster, but may get your true high-priority issue you open later pushed to the bottom of the queue.
  • Don’t lie. If you changed something that may be related to your problem, man up. If you’re consistently going to let me waste my time troubleshooting instead of coming clean and providing me the whole story, then when the time comes that you have a real issue it’s going to take longer to get resolved since I’ll be spending my time looking to see what you screwed up but won’t admit to.
  • Give details and note error messages. Provide usernames, email addresses, and callback numbers. It’s much faster for you to provide the information I need to troubleshoot than it is for me to go back and forth trying to squeeze information from you or wading through a metric ton of server logs. I’m not going to troubleshoot in the dark — if you send a ticket saying “email is down”, I’ll respond with an equally vague message saying “it’s working fine for me”. Doing this one step alone could mean the difference between having the problem persist for a few minutes or a few hours.
  • Don’t argue, clarify. If you think I’m wrong about something, ask for clarification or explain that you thought it worked differently, but don’t simply start an argument. Not to be rude, but you called me for help and I’ve been dealing with issues like this for a long time. Almost every time someone wants to argue, it comes down to them not completely understanding the intricacies of protocols and services such as BGP, ATM, PPP, DNS, and SMTP. If you know I’m wrong, explain why and I’ll listen and admit it if so — and we can continue with getting your problem fixed. If you feel the need to have an argumentative conversation, please don’t waste my time – there are plenty of Internet forums out there for you to troll.
  • Don’t play the blame game. Bickering about who is at fault for your document getting deleted, your workstation crashing, or your email bouncing is not going to resolve the issue. Technology breaks, mistakes happen, life goes on. Deal with SLA’s per your contract, but AFTER the service-affecting issue is resolved; don’t attack the person on the other end of the phone when it comes to settling the dispute.

Anything else I’m missing?

Complaints, Systems Admin ,

December 2008 site update

December 30th, 2008

Another quick update on the status of this site. I’ve moved everything back to WordPress, away from Drupal. I gave Drupal almost two years, but it was much more than I needed for a simple blog. The simplicity of writing my own plugins for WP is what got me to move back.

Please let me know if you notice any issues with the move. Thanks!

Meta

AT&T cell voicemail vulnerability STILL exists?

September 23rd, 2008
Comments Off

Decided to screw around with my AT&T voicemail service today; our local GSM carrier, Edge Wireless, was recently acquired by AT&T, and the time came to swap SIM cards and set up voicemail on the new system.

Noticed a voicemail option (enabled by default) allowing you to skip password entry if calling from your mobile — of course, my interest was in breaking into voicemail boxes with this option enabled. You see, I manage telephone systems with PRI (ISDN Primary Rate Interface) connections, and can very easily spoof my Calling Party Number (Caller ID).

I’ve heard of this issue before, and I’ve toyed around with doing this with other telephone services that “authenticate” based on CID before, too. I just cannot believe that this security vulnerability STILL exists – it’s been widely known for over 2 years!

Fudging caller ID is extremely easy. Most carriers will let PRI customers do it (we use it for forwarding calls with the original caller ID info intact, and for calling from our call center numbers). Many VOIP companies let their customers do it (for the same reasons). There are even services that take advantage of this and will allow you to visit a web site and enter a CID to call from, number to call to, and your telephone number – then bridge the call for you.

With regards to AT&T’s voicemail system, I can simply set my Caller ID to that of my victim and call the number, and hope it rolls to voicemail, but getting a call from yourself might arouse suspicion. Even better, I can just call the voicemail service center number. This is the number that the provider call-forwards busy/no answer/unavailable calls to, and can be viewed from most phones, and you can reach it from the PSTN (our local area’s service center number is 5037030985).

The iPhone uses a completely different voicemail system (to support the visual voicemail), and the service center number for regular VM users doesn’t seem to support login for iPhone users. However, I’ve tested with a couple of iPhone users here, and calling the user’s phone directly and going to voicemail still allows this vulnerability to be exploited.

I had hoped that this feature had at least used ANI for identification purposes, but sometimes ANI’s not much better; one of my carriers have a misconfigured switch which sends whatever I stuff as the Caller ID field as my ANI — I’ve even told them about it, but they don’t seem to care. After all, it could be handy for “calling from a home phone” to toll-free credit card activation services, making harassing telephone calls to toll-free operators, or screwing with your E-911 operator. Yes, we arranged a test call with 911, and the calltaker saw the address of the telephone number I forged.

The short story? Enable your voicemail password if you haven’t already. Call your mailbox, and if it doesn’t ask for your password, select Personal Options, Administrative Options, Password, and Turn Password On. Do it now, it only takes a minute. Go on, do it.

Seriously, I’ll wait.

You done?

Good.

Now, if you desire, call AT&T and complain. I’ve done it already, as have many other people, and it hasn’t helped — yet. But the more people who make them aware of this glaring security vulnerability that is a DEFAULT SETTING, the better.

Security

Some user password statistics

August 21st, 2008
Comments Off

So, a thread about stupid user passwords recently came up on a group that I frequent, and I thought I’d post this here.

We store customer information in MySQL, and have to keep a cleartext password for PPP CHAP authentication. A while back, I did some querying to see just how terrible our users’ passwords were.

Here were some of the more interesting/amusing results (remember, in SQL quotes surround literal strings and “%” is a wildcard):


SELECT COUNT(*) FROM customers: 32112
SELECT COUNT(*) FROM customers WHERE password = “password”: 151
SELECT COUNT(*) FROM customers WHERE password = username: 660
SELECT COUNT(*) FROM customers WHERE password LIKE “123%”: 364
SELECT COUNT(*) FROM customers WHERE password LIKE “%321″: 44
SELECT COUNT(*) FROM customers WHERE password LIKE “qwerty%”: 8
SELECT COUNT(*) FROM customers WHERE password LIKE “asdf%”: 11
SELECT COUNT(*) FROM customers WHERE password = “********”: 16
SELECT COUNT(*) FROM customers WHERE LENGTH(password) <= 4: 5151

…and I thought our users were doing surprisingly well — until I executed the last query.

Humor, Systems Admin ,

MySpace will collapse the Internets

July 12th, 2008
Comments Off

This, my friends, is why MySpace will be the cause of the technological collapse of the Internets.

Total page size (HTML and images): 9.18 megabytes

myspace-bandwidth-hogs

Mother of God.

All for glittery text, a crummy slideshow, three autostarting videos, pictures of Vin Diesel, and some other worthless drivel…

Complaints, Humor ,