htdig 1:3.2.0b6-3.1 (amd64 binary) in ubuntu gutsy

 The ht://Dig system is a complete World Wide Web indexing and searching
 system for a small domain or intranet. This system is not meant to
 replace the need for powerful internet-wide search systems like Lycos,
 Google, or Yahoo!. Instead it is meant to cover the search needs of a
 single company, campus, or even a particular subsection of a website.
 .
 As opposed to some WAIS-based or web-server based search engines,
 ht://Dig can span several web servers at a site. The type of these
 different web servers doesn't matter as long as they understand the
 HTTP 1.0 protocol.
 .
 Features:
    * Intranet searching
    * It is free
    * Robot exclusion is supported
    * Boolean expression searching
    * Configurable search results
    * Fuzzy searching (different algorithms supported)
    * Searching of HTML and text files
    * Keywords can be added to HTML documents
    * Email notification of expired documents
    * A Protected server can be indexed
    * Searches on subsections of the database
    * Full source code included
    * The depth of the search can be limited
    * Full support for the ISO-Latin-1 character set
 .
 Please note that ht://Dig is a resource-hog, with respect to processor usage,
 when indexing.
 .
 Disk space requirements:
 .
 13.000 documents indexed: 150MB disk space with a 'wordlist database'
                                93MB disk space without a 'wordlist'
 .
 Multiplying the number of documents to index by 12.000 comes pretty close
 to the real disk space used.