htdig has been retired from radwin.org.
Many years ago I set up a site search on radwin.org using a tool called ht://Dig. It was a C/C++ app that would spider/crawl a website and also serve out search results in an HTML-friendly fashion through a CGI.
The rest of the world moved onto Lucene/Solr, but I never bothered to make the move – in large part because htdig was good enough, and in part because DreamHost doesn’t support Tomcat/J2EE in their shared hosting environment (I don’t blame them).
The last time I successfully compiled htdig was on October 31, 2005. DreamHost has upgraded hardware several times, and the old 32-bit binaries have continued to run. Recently they did another server migration/upgrade, and finally everything broke. I tried recompiling both the stable 3.1.6 (released February 2002) and the most recent release (3.2.0b6, released June 2004) but neither compiles cleanly with gcc 4.4.
And it’s not worth fixing.