Michael J. Radwin

Tales of a software engineer who keeps kosher and hates the web.

Israelis love “West Wing”

Nov-2002-West-Wing-small.jpg We have many friends living in Israel this year. Last year we used to have a regular gathering on Wednesday evenings to watch The West Wing on our projection TV.

Now they’ve gotta wait months for someone to send ’em a tape from the States. They recently got together to watch a couple of hours and sent us this digital photo.

We’ll send ’em another tape soon. In the meantime, I can hardly wait for this week’s episode:

“Swiss Diplomacy”

The Iranian leader makes a secret request of Bartlet to allow his son to be flown to the United States for life-saving surgery.

I sure miss my friends.

Opt-out of Telemarketers and Junk Mail

operation-opt-out.gif There are already several good tools for combatting e-mail spam, but what about telephone calls and U.S. Postal Service junk mail?

Few people are aware of an organization called the Direct Marketing Association which is used by many (but not all) telemarketers and junk mailers. From what I understand, the DMA rents a centralized database of names and phone numbers to reputable marketers.

What’s important about the DMA is that they provide something called the Mail Preference Service and the Telephone Preference Service which are opt-out lists that consumers can join. If you tell the DMA that you don’t want junk mail or telephone calls by registering with these lists, they will refuse to give your name to their customers. Pretty cool.

There are other groups to contact as well, such as the Big 3 credit reporting agencies (Equifax, Experian and Trans Union). After you ask these companies to stop sharing your name & address, and suddenly all of those credit card solicitations disappear!

The easiest way to get rid of all of this offline spam is to go Operation Opt-Out and print out the 7 or 8 forms and mail them off to the various agencies. The $3.00 investment in stamps will go a long way towards reducing aggravation.

We like cars, the cars that go boom

Grab It! - L'Trimm My parents bought me an external Firewire CD burner for a birthday present (back in June) but the thing didn’t work. When I met them for dinner last night, they gave me a new one which although is a wimpy USB 1.1 model, at least it’s a name brand (Iomega Zip CD650). Turns out that it actually works. I guess there’s something to be said for buying name brands.

So this morning I finally made the “Car Songs” CD that I’ve been thinking about since this summer! Here’s what’s on the CD:

  • Barenaked Ladies – In the Car
  • Beach Boys – Fun, Fun, Fun
  • Beatles – Drive My Car
  • Billy Ocean – Get Out of My Dreams Get into My Car
  • Don McLean – American Pie
  • Geggy Tah – Whoever You Are
  • L’Trimm – Cars that go boom
  • Prince – Little Red Corvette
  • Roger Miller – King of the Road
  • Rose Royce – Car Wash
  • Sammy Hagar – I Can’t Drive 55
  • Tracy Chapman – Fast Car
  • Trio – Da Da Da
  • Willie Nelson – On The Road Again

Yes, I know that “Da Da Da” isn’t actually a song about cars or driving, but those Volkswagen commercials have infused the song with a whole new set of car imagery. Kinda like I always think of United Airlines when I hear Rhapsody in Blue.

Secret lyrics to the Ashim Theme

I recently got a cell phone that allows you to program custom ring tones.

So I programmed in the first 8 bars of the Ashim Theme that Mike Cafarella and I composed (with apologies to the Norwegian Folk Song) back in 1996.

I can’t reveal the whole set of lyrics to the Ashim Theme due to the blood oath that I swore to Caf back on that overcast day in Providence. But here’s an excerpt:

As I walk through the dark forest,

Swimming through waves of terror.

Oh my goodness, there he is now.

It’s the spirit of Roberto.

I must run, I must flee,

He will kill me (holy cow)!

Have I escaped? Am I free now?

Is this heaven, not hell?

I must try to find Nirvana: Taco Bell.

Now, every time my phone rings I think fondly of Ashim.

How do we fix the blogrot problem?

I’m concerned about linkrot in blogs. Blog entries tend to mention interesting stuff by hyperlinking to news articles, websites, and other blogs. Since it’s so easy to create a link to something (rather than excepting a relevant paragraph), you’re pretty much guaranteed to get a 404 when you try to visit that link at a later date. This might not be such an obvious problem on the surface because blogs are an ephemeral, fresh and up-to-the-minute medium, and linkrot usually takes a few weeks or months to set in.

But blogs are also supposed to serve as a diary or journal, so you should be able to go back 6 months from now and revisit all of the cool stuff you used to think about. That’s when linkrot is going to burn you the worst, because you’ll want to re-read an article or another person’s blog, and most likely it won’t be there anymore.

I think it would be cool if MovableType or some other popular blogging software could provide a PermaLink feature for external content. I’m thinking of something like the Google cache, which would mirror the content locally and add a header that would say something like:

This is Michael J. Radwin’s blog’s cache of http://www.newsfactor.com/perl/story/19912.html. This cache is the snapshot that I took of the page as I wrote my blog.

The page may have changed since that time. Click here for the current page.

It would work even better if there was some clever integration with your browser that says to visit this HREF first, but if you get a 404, try this alternative HREF (which happens to point to a snapshot of the page in your blog archives). I’m sure XHTML has something like this when you go beyond xlink:type=”simple” but I doubt browsers do anything intelligent with it.

Heck, even blogs themselves are prone to linkrot. I recently decided to switch my MoveableType settings to use Date-based archives instead of Individual Item archives because I rarely write more than one blog per day. Clicking on that convenient “Rebuild Site” button caused everything to get rebuilt. But what if someone had already linked one of my old Individual archives that’s no longer there? Apparently that PermaLink feature is not so “perma”.

I’m encouraged that people are working on solving the linkrot problem in a generalized way but not everyone is going to care to do it right.


I’ve gotten a handful of the Nigerian money-laundering scam emails over the past year, although it seems like I’m getting one or two a week now. I’m reminder of a story in Wired Magazine about some Netizen who decided to catch the scammers on film. Apparently this person had a lot of time to waste because they exchanged over 50 emails. Read the Wired synopsis now for some quick entertainment, and read the full story when you’re really bored (and don’t mind a dirty sense of humor).

But spam is a not-so-funny problem. I publish my email address pretty openly on my website, so I end up getting a lot of spam. I have been filtering email it for a couple of years with some home-made procmail recipes. I came up with a heuristic in 1999 that worked quite well:

  1. if the subject line happened to mention toner cartridges, it probably was spam.
  2. if my email address (or one of the mailing lists I subscribed to) was on the To: or Cc: line, it probably wasn’t spam.
  3. everything else was probably spam.

This heuristic worked pretty well, but had a few drawbacks. Most notably, when my friends would throw a big party and Bcc: me, the invitation would end up in my possible-spam folder.

In addition, spammers started getting more intelligent. Beginning around 2001, they started sending individual spam messages that were actually addressed directly to me! So my underlying heuristic was starting to fail me.

To solve the “Bcc” and “unknown sender” problem, I came up with a new plan. I was going to load my addressbook into a DBM hash and add a procmail rule that classified anything that came from this list of approved senders as guaranteed non-spam, and anything from someone unknown as likely spam. Then, I would add people to the DBM hash one-by-one when I confirmed that they were an actual friend of mine.

I was about to get started on this project but I hadn’t found the time to do it. It didn’t even occur to me to look for someone else’s software to solve my problem. (This is the problem with being a software engineer. You know how to solve problems like this, and it’s so easy to do, that you often start working on a solution without checking to see if anyone else has done it yet. We call it re-inventing the wheel).

Luckily, before I could waste a whole bunch of my time, someone at work mentioned a nifty server-side spam filter called SpamAssassin. I took a look and installed it on my ISP. It’s not perfect, but it does a remarkable job of detecting spam, and it’s about 50 times better than anything I could’ve written.

SpamAssassin works well because it’s got a group of volunteers who are constantly updating a rules database that says what patterns in an email make it more (or less) likely to be a spam message. For example, if the email message mentions “herbal Viagra” or toner cartridges, it’s likely to be spam. It even has something called a whitelist, which matches my idea of allowing people in your addressbook to send you mail. But I’m not even using that feature, because the 2.4x series of SpamAssassin works well enough out-of-the-box.

Yahoo! has a completely different system that it uses for Yahoo! Mail. Instead of running pattern detection on the email (an effective but labor-intensive solution), our former Chief Scientist (who recently left Yahoo! for a job at Amazon.com) came up with an automated algorithm. Unfortunately, I can’t say much about Udi’s approach without giving away trade secrets, but you’ll be able to read the patent when it’s finally approved.

ISPs should take a long, serious look at providing SpamAssassin as a service to their users. It won’t catch every piece of spam, but it’s probably got the best ratio of low sysadmin effort yielding a high quantity of spam detection.

Messin around with my Webcam

Picture 1.jpg We bought a webcam a couple of months ago so we could use Yahoo! Messenger to video-conference with our friends Rob & Kelley in Israel. I just discovered the Still Photo feature today, and I’m screwin’ around.

Ok, I’m gonna go back to work now.

Here comes the rain again

It has been raining in LA the past 24 hours. This is fantastic. I can hardly even remember the last time it rained. Certainly not this summer. This has lots of great side-effects:

  1. I don’t need to water the lawn this week.
  2. The streets are going to get nice ‘n’ clean.
  3. All of that gross LA smog goes away (and when the rain stops, the visibility will be superb).
  4. I can go outside and splash in some puddles if I need a break!

Of course, there are a bunch of drawbacks, too:

  1. LA drivers don’t really know what to do in the rain, so there might be some more traffic accidents today. God willing, nobody will get hurt.
  2. Our new house has a flat roof, and I’m a wee bit worried about whether it’s going to leak or not. So far, we’re drip-free! (my grandma would add “Kin-a hura poo-poo-poo”)
  3. My brother-in-law, who is in town this weekend from New York, happened to visit LA in some of the worst weather we’ve had in a year.

Ariella and I were musing about writing blogs this morning, and about how it’s stimulating a healthy flow of creative energy. Even if nobody is actually reading this, it’s fantastic exercise to organize my thoughts and put them into words. Although I don’t have anything profound to say this morning (the weather isn’t exactly what I would call the makings of an intellectual conversation), it’s still such a thrill to be able to do some writing. I sure missed that.

I’m going to muse about something more profound than the weather next time. Look for an upcoming blog in the Intellectual Property category on micropayments, PayPal, and online music.

Wal-Mart’s sub-$200 PCs

0066044810401_75X75.gif I’ve read with great interest about the dirt-cheap PCs that Wal-Mart has been selling.

It turns out that these not-so-wimpy internet appliances (128Mb of RAM, 10Gb of disk, and built-in 10/100 Ethernet) are running an new operating system called LindowsOS which is really just Linux + KDE 3.0 and some really easy to use management tools.

I’ve never been a fan of Linux on the desktop. Going against the Microsoft grain is such an uphill battle, and if you’re going to go that route it seems like Mac OS X is the right approach anyways. But I’m now rooting for Lindows because it appears that there’s a little bit of a trademark battle going on over the use of the Lindows name.

If the true test of trademark infringement is confusion in the marketplace, I think it’s going to be difficult to make the argument that people would think that Lindows was a Microsoft product.

Prime numbers

Last night on the plane ride home I was reading a copy of Dr. Dobb’s Journal, a magazine for programmers (I think I got a free subscription to this when I registered for PHPCon). I came across Michael Swaine’s column and read about a polynomial-time algorithm for testing primes that was discovered this summer.

I know this is old news (blogs are supposed to be up-to-the-minute, right?) but it’s still totally fascinating for anyone who understands how public-key cryptography works (I learned about it in Math 42 at Brown).

Although this algorithm is considered super-fast for what it does, it is actually slower than the ones used by RSA and PGP when generating new public/private key pairs. The difference is that this new algorithm has the distinct advantage of telling you definitively whether or not a number is prime. The more common apporach is to run a probabilistic algorithm so that there still exists a possibility that the number you’re evaluating is not prime, but it’s more likely that you’ll be struck by lightning. Although the traditional probablistic approach is still faster than this new technique, it doesn’t give you the 100% confidence that some people (bankers?) would rather have.

From a practical standpoint, 99.99999999999999% confidence ought to be enough for anyone, because insurance policies aren’t that expensive when you’re talking about covering something that has a 0.00000000000001% likelihood of happening.

But from a theorhetical computer science standpoint, this discovery is just plain cool. If they could only prove that P != NP, I’d be really psyched.