I know that the ‘insufficient disk space’ issue has been raised before but I still have problems.
I’m trying to crawl a file share with >500,000 documents, initially I hit an issue with /boithoData filling up so I resized the disk to 100Gb.

The crawl now runs to a point (around 8500 documents) before it stops with the ‘insufficient disk space’ message, although df-h still shows 99% free space in /boithoData.  After a short time the crawl will start again, it takes a long time to get back to the point it was at before crawling another 2000 or so documents before giving the ‘insufficient disk space’ message again.

The crawl has now been running 24hrs and is only up to 430,000 documents and /boithoData is showing 94% free - is this how long a crawl such as this should take?  Would each subsequent crawl take just as long or is it just the initial crawl that takes time?  I’d really like to get to the bottom of this as Searchdaimon looks like a great product!

Any suggestions?

I am sorry to hear that you have encountered this bug. Unfortunately we have not been able to reproduce the error on our end, and that make this error extremely hard to track down. It is also suspected to be caused by the underlying operating system, making it quit complex. In addition only 4 cases is known, so it must be caused by something that do not occur so often.

We are treating this with top priority and have come up with several possibly cases, but noting conclusive yet. Will get back to everyone when we find a solution.

I have also encountered this if you need logs or anything to help you, let me know.

Thank you for offering your help Slogstorm. Much appreciated. Thankfully this issue has already been resolved now, and an automatic update are being sent out to everyone this week.

You will get this update in a couple of days, or if you don’t want to wait you can follow the easy steps described at: http://www.searchdaimon.com/forum/viewthread/1009/

Thanks for the quick assistance! :D

However, after installing the patches, this happens when i try to crawl:

uriindex_open: Trying to open lotfile “/boithoData/lot/1/1/Fellesomrade/urls.db”
uriindex_open: finished
DBcursor->get: DB_NOTFOUND: No matching key/data pair found
search for “<<filename>>”, len 104
uriindex_close: closeing

I’ve tried creating a new collection, same result. This also started happening to other collections.. Any ideas?

Thank you for bringing this to our attention. I have had a quick look, and it turns out that the underlying database library is set to output debug information. These messages are seeing that the url was not found in the index from before, and thus should be crawled. You can safely ignore the messages.

Expected for thus error messages in the console, dos the crawl work ok?

This should not change anything; it is only the messages in the console that are nosey. Probably therefore it did not show up when we tested this, but I will have a fix made as soon as possible.

It didn’t work initially, but after forcing a full crawl, it looks good! Thank you for the quick assistance

Sins this only was a configuration issue we have been able to make a fix already. If you want you can try it out yourself by doing the following:

* Download the file
http://bbh-001.boitho.com/div/boitho-crawlManager-1.4.120-1.i386.rpm
* Then go to Software updates->Advanced in the admin panel and install the rpm

Works perfectly!

I am getting this error with latest version just installed 2 days ago. I am testing Search Daimon. Started the crawl, then realise I won’t have enough space. So I increase the I 2nd drive to 100GB. With 6GB disk usage and 93GB of 98GB free.
I got the insufficient disk space error. How can I fix this? Can install the patch as stated above on the latest version of Search Daimon?
THanks.

I would suggest to delete the collection restart again.

jmcentee - 15 January 2014 03:39 PM

I would suggest to delete the collection restart again.

Testing it out again now. So far been working ok. Thanks.

Thank you @jmcentee. That is a good suggestion. Restart crawling shouldn’t really be necessary, but it is known to fix a lot of issues.

The ‘Insufficient Disk Space’ error when it was enough space available was Searchdaimons longest running bug, but was finally tracked down some time ago. A fix has been sent out to everyone through the automatic update system, but if you have a firewall that block outgoing http you may not have gotten this fix. You can check if you have it from the admin interface by going to “Software updates” and click on the “List installed software”. Find the “boitho-crawlManager.i386” package. It should be version 1.4.120-1.

When you increased the hard drive, did you reboot the ES afterwards (Ctrl+Alt+Del from the console)? You have to reboot before the ES becomes aware of the extra space. You can check if the extra disk space has become available from the admin interface by going to “System diagnostics” and see under “Information for disk /boithoData”.

Let me know how it goes.

Runar Buvik - 16 January 2014 06:50 PM

Thank you @jmcentee. That is a good suggestion. Restart crawling shouldn’t really be necessary, but it is known to fix a lot of issues.

The ‘Insufficient Disk Space’ error when it was enough space available was Searchdaimons longest running bug, but was finally tracked down some time ago. A fix has been sent out to everyone through the automatic update system, but if you have a firewall that block outgoing http you may not have gotten this fix. You can check if you have it from the admin interface by going to “Software updates” and click on the “List installed software”. Find the “boitho-crawlManager.i386” package. It should be version 1.4.120-1.

When you increased the hard drive, did you reboot the ES afterwards (Ctrl+Alt+Del from the console)? You have to reboot before the ES becomes aware of the extra space. You can check if the extra disk space has become available from the admin interface by going to “System diagnostics” and see under “Information for disk /boithoData”.

Let me know how it goes.

Thanks. Have installed this now and I think it works. I am unable to run the List installed software as my SearchDaimon appliance isn’t able to talk to the internet as it’s behind a TMG. Is it possible to get SearchDaimon working behind a TMG? If not, I’ll have to get the appliance to bypass our TMG proxy.
Also, if I have only just installed the appliance less than 2 weeks ago, would it not have all the latest update and patches installed?
Thanks.

Great to hear that it works now That should mean that this isn’t a new bug, but just a case of an missing patch.

I am unable to run the List installed software as my SearchDaimon appliance isn’t able to talk to the internet as it’s behind a TMG. Is it possible to get SearchDaimon working behind a TMG? If not, I’ll have to get the appliance to bypass our TMG proxy.

How to configure an external firewall to allow automatic updates are documented at http://www.searchdaimon.com/documentation/C29/#configure_an_external_firewall

Also, if I have only just installed the appliance less than 2 weeks ago, would it not have all the latest update and patches installed?

Unfortunately so will a new installation not have all the newest patches installed. Making new install images are an expensive process that required us to install and test the ES on many different systems (VMware, Xen, VirtualBox etc.). We dos that from time to time, then have automatic update that covers the differences between the different releases.

This is a common technique used by many companies. For example Microsoft dos not make a new installable iso each time they make a security patch for Windows. Instead they normally release an cd-iso per service pack version. You thus have to run automatic updates after a new Windows install.

Therefore listed below are the reasons I believe most people make an application for the bloggers’ health program: The majority of bloggers tend to be misinformed; The majority of bloggers tend to be lazy; The majority of bloggers absence skills; The majority of bloggers absence Commitment; The majority of bloggers absence business way of thinking. legendsoflaw

From the actual BlackBerry towards the light bulb towards the telephone, Canadian creations are part of the normal life of the planet. But most finish up in US dependent companies. We depend down the very best ten Canadian technology inventions which stayed you’ll come to home. financeformanagers

Visit of Ruth Porat, because Chief Monetary Officer from Google, might be symbolized being an empowerment associated with women within tech advertising jobs. However, it’s sad this glory is actually dimmed down using the actual information showing the actual decline within the ratio associated with tech work held through women. medicomtech

Having your business known within the local area could be hard. Perhaps you have put the fundamentals in location? Follow these types of five easy local SEARCH ENGINE OPTIMIZATION principles to start to find the exposure your company needs. charterschoolpolicy

Perhaps you have faced the issue of finding the right canine house? Although thousands of styles flood the marketplace now, picking out a dog house could be tough, especially if you have a specific design in your mind or when you are interested in some genuinely unique canine house. thebeautyresource

Hollywood from the North. There’s valid reason why Vancouver is actually honoured with this type of glamourous pseudonym. In the end, it is a filmmaking centre for up to a hundred years. savenow4college

Traveling overseas is pricey. Some journey companies take your hard earned money and neglect to manage your own trip properly. Some strategies for contemplating choosing a organization that cares for that traveler below their view. bkkshopping

Faith, wish, and love would be the components associated with life. These 3 alone consider us from denote point within our lives, and provide us a chance to live the actual virtuous existence. With these types of three we could move within alignment towards the will associated with God. top—designer-brands

Finding journey deals within Europe is really a general topic however, many very obvious paths to follow along with. Lose them and will also be lost into a massive world associated with not fascinating opportunities. A large number of offers are available, with really tight costs one together. lifestyle-navi

Earlier throughout the 19th hundred years in Luxembourg, people took horse attracted carriages with regard to transportation. There had been obviously absolutely no wristwatches in those days. So metal carriage clocks were accustomed to keep an eye on time. womenfashionblog