Hi,

I’m trying out SearchDaimon, with a view to using it for our intranet. I’ve got a VirtualBox installation working and can successfully crawl sites that don’t require authentication.

However, when I try to crawl our intranet - Which uses https:// and authentication, I just keep getting the error:

Can’t get url https://.../: 401 Unauthorized

(Note: ... contains the URL of our intranet, which I have removed before posting here)

I did see the following error in the crawling management log yesterday, but have not seen it since:

Noting {500 Can’t locate object method “new” via package “LWP::Protocol::https::Socket”} error at https://.../

I wondered if the problem was related to a missing Perl-Crypt-SSLeay (as mentioned in http://www.searchdaimon.com/forum/viewthread/46/ ), but if I try to install the RPM I just see the message that it is already installed.

Here’s what I’ve done so far then - Within the collection settings page:

I’m using Fake ad as we only want to create a public search
I’ve put the URL including https:// within the url text field

Finally, for authentication, I’ve tried username and password without a user prefix (our Active Directory name) and with a user prefix. Whichever way I do it, it still doesn’t work.

Our intranet is accessible via the internet, so isn’t configured to only use Integrated Windows Authentication - I can connect to it using any browser without any problems by just entering my username into the popup login box.

I don’t really know what else to try!

Any help would be much appreciated.

Thanks,

Adam.

Have I misunderstood how authentication has to work?

Does the username for authentication when crawling have to exist within the “primary user system”?

Still confused…

Thank you for a exceptional good error report. You obviously know what you are doing

As I see it there are two possibilities for errors her. One is that https don’t work for some reason ( a bug?). Have you tried to index some other https site? For example https://www.iana.org/domains/example

The other possibility is that something is bugging with the authentication. Both basic authentication and NTLM should work find, but you may want to test to switch from one to the other just to eliminate it as an possibility for this error.

You mention that your intranet is accessible through the internet. If you want you can email me directly on “rb [att] searchdaimon [dot] com” and I can try to setup a crawl from her (but then you obviously will have to give out a username and password to your intranet ).

> Have I misunderstood how authentication has to work?
>
> Does the username for authentication when crawling have to exist within the “primary user system”?
>

No, you haven’t misunderstood how authentication work. You are supposed to set it up in the collection settings page just as you mentioned.

Hi,

Sorry for the delay in posting a reply to your email.

I’ve tried a few other things - But without success.

I’ve configured our development intranet server (IIS 7.5) to accept both Basic and Windows Authentication (NTLM) and tried with out with Searchdaimon. This site is just HTTP, not HTTPS. Even so, I still get a 401 unauthorised if it’s set to only Basic Authentication, only Windows Authentication, or both together.

Everything works fine if I don’t have any authentication switched on.

I’ve even tried changing the version within the application to Test and Development, just to see if it makes any difference (it doesn’t).

I’m using the standard VirtualBox Virtual application download from your website, and have not changed any under-the-hood settings at all.

If I’m honest, I just don’t think it works with authentication against IIS. Sorry.

Oh, and I forgot to say, I don’t think I’ll be able to give you access to our server (not unless we actually bought the product from you), due to security considerations.

I know it would help a lot, but it’s not something that we could do.

Sorry!

This is very strange! We have of course done a lot of testing with Windows and IIS. You mentions that the server is ISS 7.5, but what operating system are you using? Windows server 2008?

We use Windows Server 2008 R2.

I’ve looked at the traffic going to the intranet server from our SearchDaemon install using TCPTrace and whilst I can see the requests for authentication, it seems that SearchDaimon isn’t actually passing any credentials over.

Just not sure if I can go any further with this.

Adam.

How good an insurance plan is is determined by how effectively it suits the consumer. But, if you desire a starting point for, it could possibly be good to look into these about three top health care insurance providers 1st. tophealth

According on the Law involving Attraction, you’re as being a giant magnetic field pulling into your health whatever you target. You could apply this specific basic principle in the Law involving Attraction to create out the top in people. lawcafe

Now every day in the periods of financial system spending money on unnecessary things is often a very unwise course of action. With your recent belt securing measures involving governments across the world shopping pertaining to non-essential things is often a very bad best option. shoppingguru

The proper way to improve your own home security should be to install an outdoors camera stability camera technique. It is not hard to accomplish and more affordable than it may seem. Read on to acquire more information about the most effective there can be. improveyourhomes

This is a directory of creative take a trip blogs i read along with follow. These are written by simply independent take a trip writers, the listing include people who I look at as heavy-weights throughout travel writing a blog. drtravel

A brand-new series regarding the world involving fashion plus the top developer labels comprise it. Each fitting will target a significant top manner brand, their mission, as well as history. In your first fitting we check out Virgins, Saints, and Angels. fashionsaint

Marketing just about any business not online with organization cards and share fliers must be in your game plan. These are generally two great strategies to real entire world non-internet promoting possibilities that one could take benefit from today. anybusiness

Law jobs are very satisfying along with fulfilling jobs around. Winning in a situation is similar to striking gold if you, absolutely need way be aware that justice have been served, and all as a consequence of your efforts, you experience absolutely wonderful. lawcareer

It can be never to soon to start off your The holiday season shopping. The sensible shoppers know that one could already find what the scorching products are going to be this slide. Use each of our list that may help you avoid your rush. shoppingnow

No matter the reasons why you’ll probably decide to a somewhat more luxury at your residence, the luxury diy tips gives you the concepts on where do you start. luxuryhomeimprovement

By figuring out multiple situations which could confront leaders of their career course, this content brings to focus forcefully that will leadership style is not static for the leader all of which will require these people wearing distinct leadership to select from, suiting various situations. stylekeeper

For those people considering subscribing to the jet-to-let real estate investment opportunities set allow me to share the about three main possibilities open in relation to raising real estate property finance, loans as well as mortgages to acquire property in foreign countries. localfinance

Well, take time to find out your rights after which it flip your antics back for the debt collectors’. This guidebook from Credit card debt settlement Legal Guides will disclose how. legalguide

A similar applies throughout home upgrading; such minor changes may give your home a whole new and increased look. This may also enhance your living environment from it. A simple diy guide are able to do great issues for greater living. homeimprovementguide

Travelers have the advantages of learning along with gathering know-how about different spots. Not only that they can see along with explore distinct places nonetheless they also discover those spots intentionally and also unintentionally. toptravelers

Free magazine-style web site themes may help your site look slick and specialized. You don’t have to have a high-priced web marketer to update your web site. Here are generally five reasons to consider using a new web site theme as well as template. styleblog

B2B websites are most successful for developing new company communities and look after improved interaction with latest business correspondents. It in addition helps throughout reducing some time cycle linked to making services easily accessible to the buyers. gobusiness

The very same laws occur today which are in influence yesterday and will be tomorrow. When many of us enact legislation today it does not change recently but will offer an superior experience pertaining to tomorrow. lawtoday

It can be of wonderful importance while searching to get the best home advancement loans that you just do a number of legwork to make certain you find what is best in your case. If you do have a large diy project coming on your path finding the top home loan might be just what exactly you need. besthomeimprovement

Travelling is surely an enjoyable expertise. It is usually an even greater experience in case you prepare for ones journey, and as well as purchasing travel cover, there can be a host involving other actions to better plan for your getaway. travel101

Part in the beauty involving deciding to go into the fashion sector is thinking about getting yourself into a prime fashion institution. Every future dressmaker or aspiring university student of manner has thought of attending one of several top manner schools inside nation eventually or anothe topfashion

Creating Organization Plans is often a meaningful help the development associated with an online or Real world business, no matter how large you wish to become. It forms an excellent foundation along with reference for anyone involved. mrbusiness

The majority us wish to buy brand-new and exciting products, but the task of journeying and gonna many shops so as to search ideal items inflame us. Currently, one should purchase all the desired items directly from your own home by making use of Internet Searching. internetshopping

Home renovation might be fun along with rewarding pertaining to someone that’s prepared to the problems along with aggravation which a home advancement project might cause. Just just be sure you understand what we are getting yourself into. homeimprovement101

In this specific exciting brand-new century you will find there’s new way to turn into a travel adviser. The days in the expensive take a trip agent courses have grown to be ancient record, and doing work in a local mall booth is just not required ever again. protraveler

 
Get yourself a great dash of redecorating enthusiasm to make a home furnishings that exercises your creativeness. Make style a huge impact and also production any time decorating over a powerful mental wave. Take the decorative style over a journey regarding elegance to be able to refine, shift forwards and create beautiful internal spaces. make style

B2B web sites are very useful for developing home based business communities and observe after improved associations with existing business correspondents. It furthermore helps inside reducing enough time cycle associated with making products easily offered to the consumers. do business

A legislation career gives prestige and also unlimited making potential. But becoming an attorney encompasses far more than in which. This write-up guides you detail by detail through the method of legislation education and also practice. law forum

Whether you are carrying out your each week food purchasing or creating electrical purchases to your home venture, a few guidelines could help you save some funds while with it. Smart purchasing means using control with the situation to obtain the best value it is possible to afford. smart shopping

Are you hearing a whole lot about vacation clubs these days and wondering just what they have been? Becoming knowledgeable about them will save you big money. about travel