A new update is scheduled for release on 24 February 2014. This update will fix a bug that prevented the Intranet crawler to fetch some robots.txt files if the robots.txt file was behind a password protected area.

Anyone wanting to test this in advanced may do so by:

* Download the file http://www.searchdaimon.com/div/bbh-001/boitho-crawlManager-1.4.124-1.i386.rpm
* Then go to Software updates->Advanced in the admin panel and install the rpm.