Caffeinated-Geek.com

There are some days that I get completely fed up with the IT career path I’ve picked for myself. Then, there are some where I love what I do. I thoroughly enjoy when, even though there is a problem, everything gets worked through to completion properly and I can make someone happy – especially when it’s my parents.

Mom and Dad finally decided it’s time to upgrade from dial-up to DSL. Both of them had high-speed access at work (in fact, I set up the DSL at Dad’s office), but there’s been nothing faster than dial-up in that house since they moved in — and that was finally about to change.

When I lived with them, I had dial-up on a dedicated second pair coming into the MPOE/NID that was cross-connected via CAT3 that we pulled around the house and drilled into a wall/interconnected in a modular jack in my old room. At the time for where we lived, dial-up at 33.6 was definitely better than any neighbors could ever ask for. When I moved out, there was no real need for the second phone line for net access anymore, so it was dropped.

Unfortunately, dial-up access on the primary line was horribly slow and unreliable – I always chalked it up to the inside wiring, and threw some inline microfilters on all extensions, which helped a bit. It was never too much of a concern for them, since they didn’t use the connection a lot, and they didn’t want to be a bother about it – Mom always says, “don’t worry about it, it’s not important” and Dad jokes about not being able to type quickly enough for faster internet access anyway.

Today was their DSL due date. I come out to help hook it up, disable automated dialing of the old connection, and to make sure it worked well. Unfortunately, it was working much slower than expected, especially for a 1.5mbit product. Assuming it was the inside wiring once again, I went around replacing microfilters through the house and checked the inside jack’s wiring – but to no effect. What a killjoy; 1.5mbit down/1.0mbit up DSL that works at 256kbit down and 128kbit up, which is slower than the lowest speed offering from the phone company.

Not in the mood to be beaten by the house wiring again, I head out to the car to grab my test set and other tools. I take out a box knife and run it along the seam of the NID, removing a nice layer of paint that’s been curing in the sun over it for the past 6 years. Screwdriver in one hand, cordless phone in another, and lineman’s handset hanging to my side, I make entry into the NID, bound and determined to make the horrid copper bundle inside cower in fear at my geekiness.

I note the wiring job from over 10 years ago when we ran the CAT3 cable from the second line into the house, and admire how it has held up. I again start to wonder why the original wiring was done with an interesting 7-pair bundle running to two modular jacks, and who in the heck decided to ignore the standards for color coding telecom wiring by hooking up each jack to random color pairs. Time to get down to business – off comes the house wiring, when I notice one of the joys of copper – a familiar green foe. Oxidation, my old enemy – at last we meet again. I clamp the test set to the terminal posts and all is well until I brush one of the insulated wires and unleash a crackle of fury out of the handset speaker.

Off with the terminal posts and wires; out comes the metal brush. Goodbye, gunky green evilness. Hook it all back up and test with the lineman’s handset again, and the quality sounds great. Plug the home wiring interconnect back in and head inside to watch the DSL sync up at full speed. Success!

Head back outside, make pelvic thrusting motion towards the NID to show my superiority, and close it up.

I’m happy. Mom’s ecstatic. Dad still types slowly, but is happy too. I am rewarded with unending gratitude and food to take home for dinner. It has been a good day.

Network Admin, Systems Admin

Why do people completely filter ICMP echo?  I have no problem with folks re-prioritizing or rate-limiting it, but outright filtering a VERY useful diagnostic tool for network guys like myself is really annoying.

Anyway, if you do decide to filter ICMP echo packets, please be sure you don’t send an ICMP unreachable from the device that’s filtered:

kgasso@wibbly:~$ ping -c 1 66.131.100.81
PING 66.131.100.81 (66.131.100.81) 56(84) bytes of data.

From 66.131.100.81 icmp_seq=1 Packet filtered

--- 66.131.100.81 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

So, the host I pinged – to test reachability – sent me a response saying that I’m not allowed to do that, confirming what I wanted to know in the first place, that the host was reachable. Uh, yeah… You kinda blew your cover there, buddy.

On second thought, I like this strategy.  I’m going to start answering my doorbell by audibly saying “nobody’s home”.

Complaints, Network Admin, Security firewall

I love three day weekends. Three consecutive 24 hour periods of hanging out with friends and family, finishing projects, and all around laziness.

A wicked summers-almost-over barbecue with the whole family, working on my stereo install in my car, and lounging around at home were on my agenda – but not dealing with people who forgot that this was the end of the month, and that they needed to pay their bill to us (or they’d get shut off).

Background: I’m a systems and network admin for a wholesale ISP. We provide dial-up, DSL, hosting, etc. Some of our wholesale customers use their own RADIUS system for authentication, some use a managed system on our side. It’s in violation of our contract with the wholesale ISP to activate accounts/tinker with the accounting functions directly for a subscriber in our managed system, and it’s impossible for me to activate an account on a system that they manage.

There’s something about a three day weekend when the calendar month rolls over that makes our wholesale customers forget to do little things like paying their bills. I can’t take it out on the poor technical people who have to call me; they’re usually just reacting to customers yelling at them. It’s their management, bookkeepers, accounts payable, whoever is responsible in their organization that has dropped the ball. What irks me the most is that we notify people if they haven’t paid NUMEROUS times before shutoff — and it doesn’t help. And that’s what causes my cellphone to ring non-stop this weekend.

Since the ISP’s tech folks don’t usually know that their management has neglected our invoice, it simply looks like a massive technical issue as their retail customers can’t log on, and they call our emergency outage paging system, which patches them through to me – which is when I get to inform them that their boss never paid us. Most of them, I can turn back on right away and have them take care of it on the next business day. There are others that are persistently late, and that I need confirmed payment from to turn back on. Of course, the person who handles that is out of town for the holiday, too. Great.

Better than the wholesale calls, though, are the retail customers — who aren’t supposed to be calling us at all. They usually come across the NOC phone number by stumbling across it in WHOIS, or by talking to the phone company (who gives our contact info as the service provider for their DSL, since they’re unaware of our wholesale program), or when given it by the wholesale partner. Note that a wholesale partner doing the latter is grounds to have them stuffed into a cannon and shot at the Earth’s sun. Oh, and I can’t forget to mention that part of the telephone IVR greeting says that if you’re an end-user, to not use the emergency paging system. They never listen and proceed to the paging system anyway.

The fun really begins when they get connected with me; the end users want to argue with me about how they are consistently on time with payments, and this is unfair, and how they’re going to go to another service provider — even after I’ve explained that I’m at a wholesale/upstream provider level and have no access to the accounting and user login functions for their service provider. Yes, they might be the perfect customer or they may have been turned down mistakenly, but it doesn’t change the fact that I cannot do anything for them. Yet, somehow, I’m expected to turn them back on, offer a credit for an account that doesn’t belong to us, and publish a three page letter to the local newspaper apologizing for the actions of one of our customer.

I’ll get right on that first thing tomorrow.

This all brings to mind an old statement I first heard several years ago said by a co-worker to a member of the sales department:

“Failure to plan on your part does not constitute an emergency on mine.”

Complaints, Network Admin, Systems Admin headdesk, lusers

Ran into a fun little issue recently configuring a new router while taking into account the changes we’re going to see in Daylight Saving Time here in the U.S.

As of the year 2007, Daylight Saving Time in most of the U.S. and Canada starts earlier than past years due to the U.S. Energy Policy Act of 2005. DST now begins on the 2nd Sunday of March and ends on the 1st Sunday of November, both at 02:00 AM local time.

If your network equipment keeps time, and you set it to your local time (not GMT), then you may need to change this by hand — often the software powering these systems do not yet follow the new DST rules.

In order to do this automatically on most versions of Cisco IOS, you can try the following…

First, check your current time configuration – there’s no need in doing this if the version of IOS on your router obeys the new DST rules:

grps-edge-rtr-1>show clock detail
01:18:39.439 PST Fri Feb 9 2007
Time source is NTP
Summer time starts 02:00:00 PST Sun Apr 1 2007
Summer time ends 02:00:00 PDT Sun Oct 28 2007

In this case, the router is following the old DST rules, indicated by the incorrect dates. We will need to update this for this version of IOS.

Let’s update the DST settings. This should be pretty painless:

grps-edge-rtr-1>en
Password:
grps-edge-rtr-1#conf t
Enter configuration commands, one per line. End with CNTL/Z.
grps-edge-rtr-1(config)#clock timezone PST -8
grps-edge-rtr-1(config)#clock summer-time PDT recurring 2 Sun Mar 2:00 1 Sun Nov 2:00
grps-edge-rtr-1(config)#end

This set of commands first applies the standard timezone with the label “PST” as -8 hours offset from UTC. Then we state that the summer-time zone we label as “PDT” starts on the 2nd Sunday in March at 2:00am, and ends on the 1st Sunday in November. Obviously, change the timezone labels and UTC offsets to suit your needs.

You then should confirm the new settings with the following command:

grps-edge-rtr-1#show clock detail
01:19:53.644 PST Fri Feb 9 2007
Time source is NTP
Summer time starts 02:00:00 PST Sun Mar 11 2007
Summer time ends 02:00:00 PDT Sun Nov 4 2007


I’ve tested this on a handful of IOS versions, including from 12.1 through 12.4. As a note, all versions of 12.4 that I have access to already have the new DST rules set by default, however setting the summer-time rules this way won’t hurt anything.

Don’t forget to commit your changes to NVRAM!

Network Admin cisco, daylight savings

I have a problem. A serious one.

You see, I offered a neighbor (who also happens to be related) free wireless internet access. However, I don’t want them on the LAN segment with my machines, and I need to offer web content filtering and some decent packet filtering to keep the kids away from pr0n, MySpace, and annoying malware.

Well, I already have two Linksys WRT54G’s running OpenWRT, so I could just set up one for NAT’d public access and the other just as an AP (I still want access to my LAN segment from my laptop). No, that would just be way too easy. So what’s a geek to do?

I find an older PC in the closet, slap 5 Fast Ethernet cards in it, and install FreeBSD. Oh how I love pf + NAT + transparent proxying to Squid. I know there are easier ways, but seriously, what’s more fun than pissing off a 17 year old by cutting off access to social network sites, IM, and just about anything that could possibly be considered “fun” right at the router?

So here I am, DSL modem linked into one Ethernet port, a wireless AP into another, my local switch and wireless AP off of another, and yet another segment for setting up a fully open wireless AP/packet sniffer. The wife is looking at me with a raised eyebrow right about now as I laugh maniacally.

I wonder how many passwords I’ll happen to pick up flying through the air in cleartext off of the open AP.

Like I said, I have a problem.

Network Admin, Systems Admin bofh, squid, wifi