Slow search results (10-15 seconds)

1 of 2

stevepeters23

Posted: 24 July 2013 05:03 PM

[ Ignore ]

Newbie

Total Posts: 6

Joined 2013-07-24

I have a brand new installation that I’m testing on VMWare. It has two collections: a small SharePoint site and an SMB share of about 13,000 documents. When I search it takes between 10 and 15 seconds to return results. If I search for something that has no results it will return immediately.

So far, I’ve:
1. Increased the second hard drive on VMWare to 90GB
2. Increased the RAM on the VM to 4GB
3. Increased the number of CPUs to two

Any other suggestions?

Runar Buvik

Posted: 24 July 2013 05:08 PM

[ Ignore ] [ # 1 ]

Administrator

Total Posts: 253

Joined 2010-09-13

Dos this happen each time you do a search, or only the first time?

Is only the first search is slow please see this faq answer: http://www.searchdaimon.com/documentation/faq/#cID26

Runar Buvik

Posted: 24 July 2013 05:19 PM

[ Ignore ] [ # 2 ]

Administrator

Total Posts: 253

Joined 2010-09-13

There is also a way to see exactly where the time is spent. If you add &debug=showxml to the url in the address bar you will view the raw xml (if there is a “#” sign at the end, remove it first). Find the text between the “<searchnode>” and “</time_profile>” tags and post it her.

Example:

Url: http://demo.searchdaimon.com/webclient2/indexanon.pl?query=test&lang=en&debug=showxml

Time profile:

<searchnode>
<nodename>localhost</nodename>
<totaltime>0.057974</totaltime>
<hits>24</hits>
<filtered>0</filtered>
<shown>10</shown>
<time_profile>
<AuthorSearch>0.000000</AuthorSearch>
<UrlSearch>0.000000</UrlSearch>
<MainSearch>0.001999</MainSearch>
<MainAuthorMerge>0.000000</MainAuthorMerge>
<popRank>0.000000</popRank>
<responseShortening>0.000000</responseShortening>
<allrankCalc>0.000000</allrankCalc>
<indexSort>0.000000</indexSort>
<searchSimple>0.000000</searchSimple>
<popResult>0.041981</popResult>
<adultcalc>0.000000</adultcalc>
<filetypes>0.001000</filetypes>
<iintegerGetValueDate>0.000000</iintegerGetValueDate>
<dateview>0.000000</dateview>
<FilterCount>0.000000</FilterCount>
<pathaccess>0.000000</pathaccess>
<urlrewrite>0.094959</urlrewrite>
<getUserObjekt>0.000000</getUserObjekt>
<cmc_connect>0.003999</cmc_connect>
<html_parser_run>0.000000</html_parser_run>
<generate_snippet>0.000000</generate_snippet>
</time_profile> 

stevepeters23

Posted: 24 July 2013 05:21 PM

[ Ignore ] [ # 3 ]

Newbie

Total Posts: 6

Joined 2013-07-24

Every time. I can put in a search term click “search” and when it returns results hit search again and it will take roughly the same amount of time.

stevepeters23

Posted: 24 July 2013 05:24 PM

[ Ignore ] [ # 4 ]

Newbie

Total Posts: 6

Joined 2013-07-24

stevepeters23

Posted: 24 July 2013 05:26 PM

[ Ignore ] [ # 5 ]

Newbie

Total Posts: 6

Joined 2013-07-24

Formatted better:

<searchnode>
<nodename>localhost</nodename>
<totaltime>13.231073</totaltime>
<hits>81</hits>
<filtered>0</filtered>
<shown>10</shown>
<time_profile>
<AuthorSearch>0.000000</AuthorSearch>
<UrlSearch>0.000000</UrlSearch>
<MainSearch>0.005999</MainSearch>
<MainAuthorMerge>0.000000</MainAuthorMerge>
<popRank>0.000000</popRank>
<responseShortening>0.000000</responseShortening>
<allrankCalc>0.000000</allrankCalc>
<indexSort>0.000000</indexSort>
<searchSimple>0.000000</searchSimple>
<popResult>13.218076</popResult>
<adultcalc>0.000000</adultcalc>
<filetypes>0.000000</filetypes>
<iintegerGetValueDate>0.000000</iintegerGetValueDate>
<dateview>0.000000</dateview>
<FilterCount>0.000999</FilterCount>
<pathaccess>47.103141</pathaccess>
<urlrewrite>0.013997</urlrewrite>
<getUserObjekt>0.000000</getUserObjekt>
<cmc_connect>0.000999</cmc_connect>
<html_parser_run>0.000000</html_parser_run>
<generate_snippet>0.000000</generate_snippet>
</time_profile> 

Runar Buvik

Posted: 24 July 2013 06:21 PM

[ Ignore ] [ # 6 ]

Administrator

Total Posts: 253

Joined 2010-09-13

This is not related to the lack of resources for the ES, but how fast it can look up information in other systems. In the xml you posted we see that most of the time is spent in the pathaccess and popResult functions. The pathaccess routine is a function that looks up user’s access rights for every file during search.

If the crawler supports it, the ES will ask the source system for access rights for every file during search. This is done since the access rights of a file can have changes sins the ES crawled it. If the end-user doesn’t have access anymore we don’t want to show him the file etcher.

While an important part of security and allows for removing results from the search system in real time, this slows down the search if the source system is slow to answer. Typically we see this when showing results from windows file servers.

Unfortunately there is currently no way of opting out of this. Our only option is:

* Change the security model used for each collection to something other than system permissions. To do so, go to the “Manage” function for a collection, and open the “Access level“ tab. Then use something other than “System permissions”.
* If the source of your files is something other than a file server you may be able to changes the connector manually.
* Make the source system answer faster….

stevepeters23

Posted: 24 July 2013 09:52 PM

[ Ignore ] [ # 7 ]

Newbie

Total Posts: 6

Joined 2013-07-24

Hmmmm…your first two suggestions aren’t really doable right now. I’m not sure how to make the source system respond faster. It runs everything else ok. Any suggestions on that?

Are there any plans to add the ability to opt-out of the ES asking the source feature? For me, I would be ok with just using the data from the latest crawl.

Runar Buvik

Posted: 26 July 2013 02:10 PM

[ Ignore ] [ # 8 ]

Administrator

Total Posts: 253

Joined 2010-09-13

Are there any plans to add the ability to opt-out of the ES asking the source feature? For me, I would be ok with just using the data from the latest crawl.

Yes, we will make it possible to opt-out of the online security scan in the future ( it is on the road map http://www.searchdaimon.com/wiki/Road_map_to_the_future ), but we are probably 3-6 mounts away from the version that will have that possibility.

In the meantime we may be able to do a quick fix for you, by removing the online security scan from the connector you are using, and sending you a new one. What connector are you using?

stevepeters23

Posted: 26 July 2013 10:06 PM

[ Ignore ] [ # 9 ]

Newbie

Total Posts: 6

Joined 2013-07-24

I’m using the SMB connector but for now I’m going a little different route to work around the issue but I’m running into incredibly slow crawling. I’m crawling a share with close to 600,000 files and at the rate it’s going now it will be around 2 weeks before it is finished. Any way to speed that up?

I may want the fix at some point but for now the crawl time is a more important issue.

Runar Buvik

Posted: 30 July 2013 12:11 AM

[ Ignore ] [ # 10 ]

Administrator

Total Posts: 253

Joined 2010-09-13

It is not normal that it takes 2 weeks to crawl 600,000 file. It sounds to me that there is a lack of recourses somewhere.

The file server must appears very slow from the ES, especially if you still have set the ES VM to have 4 GB of ram. The time profile you posted above also showed that the communication was going slow. The pathaccess profile sees 47 seconds, where 0.5 would have been more normal.

What are the specifications on your fileserver and the network between them?

The ES has to read (crawl) every file and extract its text before it can make it searchable. If the file server is slow to serve files the performance will suffer. Have you tried to do other copying task from this file server? For example just copy a folder with some thousand files in it and see what happens?

Is it enough resources on the host system to run the VM? There are no issues with CPU steal time etc.

Generalservices

Posted: 16 March 2016 02:02 PM

[ Ignore ] [ # 11 ]

Sr. Member

Total Posts: 805

Joined 2016-03-16

Life after the Chapter 7 bankruptcy will not be as daunting countless would brand-new think. The fact is, most of your negative information to choose from is spewed by credit card providers that lose out when you seek bankruptcy relief protection. www.skincarepearls.net

Honesty plus integrity essential to a building with society. All these cornerstones make prosperity plus peaceful public order. Dishonesty plus greed will be destructive problems that disparaging offer society all the way down. www.fashionismydrug.org

Mongolia contains a significant proved area to get coal exploration sector. Around Mongolia a annual fossil fuel production is definitely approximate pertaining to 5 , 000, 000 metric a lot. Almost 85% of your production is required for water vapor and electric power generation. www.healthbeautycare.net

Throughout the last decade Taupo around New Zealand includes continued so that you can attract innovative house building firms and establish that you cannot assume all small places are weak. Read the next few paragraphs and see three explanation why Taupo keeps increasing both around population measurements and fiscally. www.mensmeds.net

There will be some stages you might follow to produce a flourishing lawn caution business. Knowing these tips, it is critical to frequently remind yourself analysts or you can definitely find yourself screwing up and bit by bit lose prospects. www.interior2han.net

Home overall health service now offers a great deal of to patients who sadly are recuperating out of surgery and also illness. These expert services offered allow for patients to recover comfortably for their own dwelling, without troubled families who seem to can’t provide many of those services. www.fatburningfurnacereport.org

Here I demonstrate research I actually conducted so that they can understand the key reason why people believe a good diet is much more expensive as compared with unhealthy food items. I examine the feedback I made and form a feeling based on the results out of my example. www.insurancehealthquote.org

Narrowly characterized, “moderate drinking consumption” minimizes health probability from coronary disease. But them raises probability for diverse cancers, hepatitis, brittle bones, immune reductions, accidents, plus suicide. They cannot tell you actually that section. www.intuitionandhealing.org

The second one half of getting suffering is fussing together with the health insurance cover claims approach. I have some tips so that you can pass-on from by myself health says experiences which could help you actually navigate the following maze. www.dailyhealthtrends.net

Changes while in the health caution environment will be here. When physician leaders, are you all set to build formidable, flexible heath care treatment teams that should thrive? Three practices are presented here that is going to position you your team achievement. www.salesmanagementsystems.net

Many people are extracting high demands grass from them yards and opting for a pleasurable garden preferably. If you aren’t ready to give up on a person’s green floor, here will be some approaches for a a nutritious green, grass free turf. www.cheapclothesshoppingonline.net

Generalservices

Posted: 16 March 2016 02:02 PM

[ Ignore ] [ # 12 ]

Sr. Member

Total Posts: 805

Joined 2016-03-16

Education to get Sustainable Progression (ESD) is actually a rather innovative field with education. We will see it for an innovative types of future instruction for institutions linking a child’s development together with the future complications of world. www.ehealthplanners.net

There absolutely are a small list of young Americans who sadly are already profiting from education foreign. They get credits for their degree at your cheaper amount, get to achieve the everyday living overseas plus travel all over Asia. www.heavenhomes.org

Master here the key reason why large enterprises, small models and professionals all are profiting from cloud business enterprise computing for several different reasons. The impair system is definitely described plus predictions are built for it has the future business expansion. www.gloucesterplumbing.net

Law will not be the starting point place, nor do you find it the conclusion place in regards to ethics. Our ethical ideas normally shape the type of all of our laws. Criticism with laws plus proposals to get change in the laws are sometimes based for a shared ethical ideal that isn’t being obtained. www.weightlossshakeshq.org

This article is actually directed during men nonetheless ladies to be able to comment in that case feel absolutely free! Understand just what exactly it has been to be a dude! I’m not preaching about what you actually learned through the dad or your family members, what Get real is how you would as anyone project you to ultimately others. www.prolawteam.net

In 2009 the BRITAIN winter ignited a significant freeze, he did this unusual, as a lot of winters in great britain are fertilized by light westerly winds with the Atlantic river, but a bitterly cold winter of 2009/2010 was that they are much more advanced than normal. www.clubsimenahotel.net

Fashion contains a rippling effects on people out of different strolling of everyday living. It slowly but surely gains global recognition among several strata with society. To enable fashion get in touch with the individuals, a present selection professionals are expected. www.certifiedhealthcoach.net

Shopping to get camping chairs may look simple more than enough, but knowing the perfect places to search just might help you find the only one you really want while helping you save both time frame and capital. Hint: don’t get started your search along at the supermarket! www.marketyourpractice.net

Food shopping for a budget? While you enter every supermarket, you are going to fall sorry victim to promotion. Marketing is only the perfectly researched scientific research of why is people get. Don’t get caught in the promotion trap. Here will be 14 good tips that can assist you save 10 : 15% within your money if food browsing… www.beautyandstyling.net

The Pappa Rocking Shore Chair is where beach fits the hall! So when your buying beach desk chair that stones likes your porch chair and it’s light, mobile and retracts flat, then the following beach chair is a one for yourself! www.safeguardfinancialusa.com

Generalservices

Posted: 16 March 2016 02:02 PM

[ Ignore ] [ # 13 ]

Sr. Member

Total Posts: 805

Joined 2016-03-16

If you will be like people and like designer attire at cheap prices then you will see this posting useful. It is stuffed with tips on restaurants to shop in New york city and learn how to get Designer music labels at good buy prices, and on learn how to get around the hub. www.firstcallservices.org

Shopping to get gifts never been a lot easier. This is particularly true if you ever go online to surf for a person’s gifts. Millions of merchants can advertise really attractive, quality gift ideas at acceptable prices and you could shop with the comfort within your living home and pc! www.veghealth.net

Everyone is definitely buzzing about environment friendly these days as well as home has become the best places to begin with. Since most people spend a great deal of time within our homes, it just is a good idea to get started being extra eco-friendly now there. Here, we’ll evaluate five ways you and your folks can start environment friendly with a strong eco dwelling. www.seniorsinsuranceadvice.net

So what on earth is it wish to own element of a home inside of a corner of your largest wine-producing district in Spain? Imagine a number your highest quality wines plus similar descriptions enter in to play: track record, depth, sophistication, uniqueness with character. How sweet its. www.shopfisher.net

If that you were to invest in a property to the beachfront, wouldn’t you end up picking the site that combines a thriving role to the international field with attainable prices? If you ever said certainly, consider real estate or apartment in Cancun. www.lyfood.net

That really we’ve taken care of IT appliance, computer basic safety and problems recovery (i highly recommend you see portions 1 & 2 of your series), let’s evaluate the vast choice of other bits and pieces that you have to make elements happen immediately and efficiently in your own home business. www.betaeducation.net

Luxury housing in Victoria, B . C ., is truly several of the very best we know of. Macleans paper selected Victoria as the number one city around Canada whereby to relocate kids business. Conde Nast Criminal magazine voted Victoria the perfect over-all urban center in Canada due to its environment plus ambiance. www.childrensoralhealthohio.net

Dramatic changes took place while in the market in Marbella. In point over the entire Costa delete Sol. Property here has been in demand from customers by North Europeans as being the climate is rather warm plus sunny many of the year and the buying price of living is historically very low. www.myattorneyatlaw.net

Buyers plus sellers, please you should definitely entrust the following transaction to the full time housing professional by using experience including a track track record. Someone who will be ready give you actually expert suggestions about homes around Summit NJ-NEW JERSEY or a person’s market vicinity. www.buddha-yoga.net

During the last week, the community has witnessed a series of events this influenced the several sectors of your growing world economy. Similarly, some incidents promised extra developments plus expansion but not only in a international trade and in precise countries. www.ansaricapital.com