Caffeinated-Geek.com

So, I guess the big news today is that Comcast is threatening to de-peer with Level3 due to the possibility of future out-of-contract traffic ratios across their peering links, as Level3 has acquired Netflix as a customer. This has got to be one of the most stupid things I’ve heard of an ISP doing. By having settlement-free interconnects (peering links) with one another, Level3 is doing Comcast a huge favor — and I’ll explain why.

In the red corner, we have Comcast… a Tier 2 network provider and purveyor of Hybrid Fibre-Coaxial (cable) internet service to millions of end-users. Comcast is not a Tier 1 provider; they pay for transit alongside using their peering interconnects.

In the blue corner, we have Level3… a Tier 1 network provider and VOIP, dial-up, and DSL wholesaler as well as transit, colocation, and content delivery network (CDN) provider. Level3 does NOT pay anyone for transit; they have settlement-free connections with other Tier 1 and Tier 2 providers, and are a huge part of the Internet’s backbone.

Let’s think about this for a moment. The majority of Comcast’s traffic by volume is inbound to retail end-users, whose primary use of the service is to download content. The majority of Level3′s traffic which traverses the edge of their network is outbound from colocation or CDN customers. So, we’ve established that Comcast and Level3 both have nearly opposing traffic metrics when dealing with the services that this dispute revolves around.

Peering is a beautiful thing; it allows for fast, direct connections between two providers at minimal cost for the benefit of both. In this case, both Comcast and Level3 customers see a marked improvement in speed and latency when this traffic doesn’t have to traverse several different networks in order reach from Level3 to Comcast.

So, Comcast now wants to charge Level3 for these peering interconnects, claiming that it’s not fair for Level3 to dump that much data onto their network for free. That’s right, Comcast wants to double-dip and charge THEIR end-user for service along with charging the originating content provider for sending data to their user.

Here’s the rub: whether or not these peering links exist, that data’s going to traverse Comcast’s network to reach their end users anyway. Since Comcast’s not a Tier 1 provider, it’s all going to fall back on their transit links. Level3 is a Tier 1, so it’s going to traverse their other settlement-free links to reach Comcast.

What happens if Comcast pulls the plug? Comcast’s customers are going to request the data anyway, so it now traverses their transit links and increases their costs, and Comcast customers’ experience while using these services will likely be degraded, resulting in customer dissatisfaction and support/service calls.

Comcast seems to be staring down the barrel of a shotgun with their toe in the trigger guard, one involuntary twitch away from blowing their face clean off. Not that this surprises me in the least.

Complaints, Network Admin

One of our local newspapers, The Mail Tribune (along with their sister paper The Ashland Daily Tidings) has recently instituted a “pay wall” on their website. This pay wall permits viewers to read 3 stories anonymously, 10 stories upon registering for free, and has no limits with a paid registration. Your number of allotted page views are tracked in cookies set by the site, and in order to keep you from viewing more than your limit, JavaScript is used to cloak the page content and pop up a “Subscribe now to view” layer/div.

There’s a huge problem with this: the stories are still indexed by Google, and searching for terms present in the stories — the ones that I might not be able to read now — still show up in search results. They use JavaScript in interactive user agents to show different content to actual visitors than what the search engine sees and indexes. This is a blatant violation of Google’s Webmaster Guidelines:

Quality guidelines – basic principles

Sites that are not willing to display content publicly should not reap the benefits of displaying it to search engines and having it indexed.

Other websites which have done this sort of trickery have been forced to either modify their site to not display full content to the search engines or to allow search engine visitors to view all content (as was done with Experts Exchange, probably one of the most notable violators of this rule).

I have created a Greasemonkey script for Firefox (my first such one) which disables the JavaScript used to obscure the content. You can download it from my script page on userscripts.org.

NOTE: This script does not modify any queries sent to the remote server or utilize any weakness in the websites’ authentication mechanisms; it only tells the browser to not load the particular functions used by this site to hide content which has already been sent to the user agent. The data always comes across the pipe to the browser.

Complaints, Security

My current job as a wholesale ISP sysadmin has me responsible for our email servers as well as abuse handling. Part of this includes signing up for AOL’s SCOMP Feedback Loop (FBL), which is a requirement to have our SMTP mail exchangers whitelisted by AOL.

The idea is that we get copies of email originating from our server that AOL users have reported as spam. A nice idea, in theory.

Let’s take a quick look at one day’s worth of AOL’s SCOMP notifications that come to me:

• 18 are messages forwarded to the user from friends or family — jokes, pictures, videos, etc.
• 9 are regular, personal email to the user — family updates, exchanging phone numbers, sharing social networking site IDs, etc.
• 6 are messages to the user from confirmed-opt-it mailing lists — this ranges from customer newsletters to class and family reunion notifications.
• 2 are spam quarantine reports to the user — which the the user explicitly requested be sent weekly to their AOL account
• 1 is a legitimate spam email to the user — sent from a compromised webmail account.

It’s now clear to me that AOL users must have some sort of instinctual tendency to mash the “This is Spam” button instead of the “Delete” button. If you want Uncle Ted to stop emailing you political jokes, maybe it’s best that you contact him — because until it becomes a real “abuse” case, I’m not touching it. Not that I could even tell him who he needs to stop emailing, because AOL redacts anything that could even remotely be used to identify the email recipient! Oh, but it gets better…

I can’t even do anything about that one actual spam email that originated from our systems, because it has all possible identifying information removed. So, what am I to do when I get a complaint about an email user — one user out of over 40,000 — who sent email through our webmail system? Normally, the Received lines and a custom X-Abuse-Tracking header we add contain the originating account’s address. Alas, AOL removes these.

A 3% accuracy rate on the reports along with no useful information in the report has led me to set up a new mailbox rule to trash these messages. There’s no point in wasting my time when I can get real reports from real users, spamcop, or other feedback loops that are actually functional.

Complaints, Systems Admin

Yes, this is a water cooler. The same one we hacked to display our own messages:

And, yes, there are occasions when we have too much free time on our hands.

Humor, Photos

Oh dear… have I just created peachse.cx?

Humor, Photos