Mass URL Update

Posted: 23 September 2013 06:48 PM

[ Ignore ]

Newbie

Total Posts: 7

Joined 2013-09-22

I just finished a long crawl of our file server. Unfortunately all of the URLs resulting from the SMB crawl were stored incorrectly with double-slashes after my root scan path. Probably occurred because I left a backslash at the end of my SMB path.

Root Scan Path: \\fileserve.company.com\share\
Crawl Stored URL: file://fileserve.company.com/share//folder/file.txt
(notice the extra slash before the folder in the Crawl stored URL)

Correct Scan Path: \\fileserve.company.com\share
URL Should be: file://fileserve.company.com/share/folder/file.txt

Is there a way for me to bulk update all of the URLs to remove the extra slash? Or do I have to reperform the crawl to it sees all new files?

Thank you,
-Dan

Runar Buvik

Posted: 23 September 2013 06:58 PM

[ Ignore ] [ # 1 ]

Administrator

Total Posts: 253

Joined 2010-09-13

Sorry, there is no easy way of batch update all urls :(

Is this just a cosmetic bug where you see // instead of /, or are you unable to open the smb crawled documents because if this? If you are unable to open any smb links in your browser it may be caused by other factors.

Smb has a long and problematic history with web browsers. To be able to open a smb link you will have to follow the steps at described at “Opening files from network shares in Internet Explorer, FireFox or Google Chrome” - http://www.searchdaimon.com/documentation/C31/

dbirnseth

Posted: 23 September 2013 07:15 PM

[ Ignore ] [ # 2 ]

Newbie

Total Posts: 7

Joined 2013-09-22

Unfortunately it’s more than cosmetic, it makes the links non-functional.

In my implementation, I’m using an alias to replace the share. This alias makes the link an http:// link instead of a file:// link. But buried at the end of the share is a double slash that I can’t remove. My best guess is that I got it because I left a slash at the end of the SMB path I was scanning.

Runar Buvik

Posted: 23 September 2013 07:34 PM

[ Ignore ] [ # 3 ]

Administrator

Total Posts: 253

Joined 2010-09-13

I did a test crawl her my self and can confirm that using a smb name on the form \\fileserve.company.com\share\ do make the url have a // in it.

Normally this wouldn’t be an issue because most system handles // as /, however I see that it can be problematic if you need to translate this to some other format. I will add a fix in our code so any / at the end gets stripped away.

However that will not help you now, so I think you may have to recrawl to get it working. You can do so by changing the path, and select “Force full crawl now” under the “Advanced management” tab.

Bdw, How are this alias function implemented? Do you have a possibility to change // -> / ?

Ps: If you are just testing something you can crawl a limited number of documents by going to manage for the collection, and use the “Test crawl” button on the “Advanced management” tab.

dbirnseth

Posted: 23 September 2013 07:47 PM

[ Ignore ] [ # 4 ]

Newbie

Total Posts: 7

Joined 2013-09-22

Thank you for validating the issue. I’ll recrawl my documents for this collection.

The alias function is implemented using the “Share Alias” field under advanced management for the collection. I’m scanning an SMB path such as: \\fileserve.company.com\share then setting the Share alias as: http://fileserve.company.com/share The .resulting URL does have a mix of forward and backward slashes, but the browser seems to figure this out (Chrome & IE tested).

My fileserver is running IIS in directory browse mode and I’ve made the website folder match my share name. That way the HTTP and FILE paths are nearly identical.

dbirnseth

Posted: 23 September 2013 08:48 PM

[ Ignore ] [ # 5 ]

Newbie

Total Posts: 7

Joined 2013-09-22

Looks like my aliasing plan isn’t working as well as I thought. While the alias does change the url displayed in the search results, it doesn’t seem to change the actual link embedded in the document title. Is this how it’s supposed to work or is this a bug? For me personally it’s a bug. =)

Thank you for your help,
-Dan

Runar Buvik

Posted: 24 September 2013 05:21 PM

[ Ignore ] [ # 6 ]

Administrator

Total Posts: 253

Joined 2010-09-13

The share alias function was unfortunately not meant to be used like that. When you crawl a smb share you normally refer to it using an ip-address, but end-users sometime only know the share either by it net bios name ( //name/share ) or simply as a letter in My Computer (g: ). We made this share alias function so administrator could show a more familiar name to the user, but still address the share with its ip-address.

You can’t accomplish what you described using the share alias, however I see two other options:

1) Why don’t you use the intranet crawler to crawl the ISS directory directly?

2) You indicated in a previous post that you are going to use a custom template so you could change the logo.

It is possibly to do a change in the template to modify the url. We are using an open source template system known as Template Toolkit ( http://www.template-toolkit.org/ ) , so this will require a little bit of coding, but not much.

You can change it by:

*Open up the template in the admin panel and find the file “results.tpl”.
*Find the code that sees “[% BLOCK show_res %]”. This is the code block that displays the results.
*Here we can add some code. For example add this betwwen the “[% BLOCK show_res %]” and “<li>” tag:

[% IF r.cache.collection == 'Administration' %]
    [% r.url = r.url.replace( "file://///213.179.58.125", "http://example.com/path" ) %]
[% END %] 

This code test if the collection is the Administration collection. If so it replaces the file link to the 213.179.58.125 server with a http link to http://example.com/path . You will of course have to changes the collection to the name of the one you wan’t to change and the server ip-address to fit your setup.

This only changes the url used when a user clicks on the results. The green “human friendly” url we displays in the bottom stays the same.

If you don’t get it to work don’t hesitate to post here. I will try to help you as best I can

dbirnseth

Posted: 24 September 2013 07:46 PM

[ Ignore ] [ # 7 ]

Newbie

Total Posts: 7

Joined 2013-09-22

Thank you so much for the solutions! I’ve implemented your second suggestion to modify the client template. I used the following code in the same location you suggested. Eventually I’d like to modify the results to show the file link and the http link.

[% IF r.cache.collection == 'CAP' %]
     [% r.url = r.url.replace( "file:", "http:" ) %]
     [% r.url = r.url.replace( "/\\/ ", "/" ) %]
[% END %] 

The reason that I’m not crawling the IIS directory listing is from a bad experience with a previous search product. With the previous product it found web pages that users had saved in their shares and subsequently began crawling much further than I had anticipated, even outside our network. Once I learn more about the SearchDaimon connectors and how I can restrict the crawl, I’ll be more confident to crawl the web pages and should be able to change my methodology.

Runar Buvik

Posted: 26 September 2013 04:13 PM

[ Ignore ] [ # 8 ]

Administrator

Total Posts: 253

Joined 2010-09-13

You know how to code I see

The ES intranet crawler will not crawl webpages located at other domains. The crawler only crawl the url you give it and higher paths. For example if you add http://www.searchdaimon.com/documentation/ it will crawl sites like http://www.searchdaimon.com/documentation/ , http://www.searchdaimon.com/documentation/C11/ , http://www.searchdaimon.com/documentation/faq/ , etc.

It will not crawl sites like http://www.searchdaimon.com/ or http://www.searchdaimon.com/demo/ because the path dos not match /documentation/. It will also not crawl other sites like http://example.com/ even if we link to it.

For more restrictions you can use a robots.txt file as described here: http://www.searchdaimon.com/documentation/faq/#eID107

Generalservices

Posted: 16 March 2016 01:46 PM

[ Ignore ] [ # 9 ]

Sr. Member

Total Posts: 805

Joined 2016-03-16

Life after a Chapter 7 bankruptcy isn’t really as daunting the greatest number of would do you have think. For that matter, most for the negative information nowadays is spewed through credit card banks that miss the boat when you seek bankruptcy relief protection. www.skincarepearls.net

Honesty and additionally integrity are necessary to all the building about society. All of these cornerstones generate prosperity and additionally peaceful ethnical order. Dishonesty and additionally greed really are destructive stresses that dissect society off. www.fashionismydrug.org

Mongolia boasts a significant successful area just for coal exploration sector. For Mongolia all the annual fossil fuel production is normally approximate approximately 5 million dollars metric a huge selection. Almost 85% from this production is treated for water and electric source generation. www.healthbeautycare.net

Over the last decade Taupo for New Zealand comes with continued to make sure you attract cutting edge house builders and be that there were small locations are regressing. Read this and find out three why Taupo developing both for population proportions and monetarily. www.mensmeds.net

There really are some simple measures you may well follow to enhance a triumphant lawn maintenance business. Knowing these simple steps, it is really important to repeatedly remind yourself masters or quite often yourself faltering and over time lose purchasers. www.interior2han.net

Home health and wellness service in these days offers very much to patients in which are recuperating right from surgery or possibly illness. These offerings offered grant patients to get better comfortably with their own place, without being concerned families so, who can’t provide a number of services. www.fatburningfurnacereport.org

We are going to I teach you research Document conducted to understand as to why people believe healthy food choices is much more expensive when compared to unhealthy cuisine. I examine the knowledge I directed and form a judgment while using results right from my practice. www.insurancehealthquote.org

Narrowly specified, “moderate alcoholic beverage consumption” lowers health chances from coronary disease. But the software raises chances for many different cancers, hepatitis, brittle bones, immune reductions, accidents, and additionally suicide. Always tell you will that a part. www.intuitionandhealing.org

The second share of getting poorly is fussing along with the health ınsurance coverage claims technique. I have any tips to make sure you pass-on from my student’s health demands experiences which might help you will navigate this approach maze. www.dailyhealthtrends.net

Changes on the health maintenance environment really are here. Like physician commanders, are you in a position build potent, flexible clinical teams intended to thrive? Three plans are presented we are going to that should position you additionally your team to have success. www.salesmanagementsystems.net

Many household owners are cleaning out high really needs grass the ones yards and selecting a wonderful garden in lieu. If you aren’t ready to give that up on any green ground, here really are some tricks of a a normal functioning green, bud free landscaping. www.cheapclothesshoppingonline.net

Generalservices

Posted: 16 March 2016 01:46 PM

[ Ignore ] [ # 10 ]

Sr. Member

Total Posts: 805

Joined 2016-03-16

Education just for Sustainable Production (ESD) is mostly a rather cutting edge field about education. You can easlily see it as being a definite innovative somewhat future certification for high schools linking all the child’s development along with the future worries of population. www.ehealthplanners.net

There are really a small couple of young Americans in which are already making the most of education in a different country. They develop credits regarding their degree at a fabulous cheaper fee, get to perceive the lifespan overseas and additionally travel round Asia. www.heavenhomes.org

Uncover here as to why large business owners, small your and professionals each are making the most of cloud home business computing for several reasons. The impair system is normally described and additionally predictions are intended for the country’s future dominion. www.gloucesterplumbing.net

Law isn’t really the starting off place, nor that may be the giving up place in the case of ethics. Our ethical ideas quite often shape the character of this laws. Criticism about laws and additionally proposals just for change in this particular laws may be based about the shared ethical ideal that isn’t being done. www.weightlossshakeshq.org

This article is actually directed located at men still ladies you need to comment afterward feel free of cost! Understand the things it is planned to be a boyfriend! I’m not debating what you will learned within your dad or pals, what I mean is the way as humanity project you to ultimately others. www.prolawteam.net

Numbers the USA winter brought about a giant freeze, this is unusual, as a large number of winters the european union are fed by nominal westerly winds from Atlantic ocean, but winter months of 2009/2010 was that should be much different from normal. www.clubsimenahotel.net

Fashion boasts a rippling impact on people right from different corners of lifespan. It eventually gains status among varied strata about society. To benefit fashion get in touch with the potential customers, a choice is about professionals are needed. www.certifiedhealthcoach.net

Shopping just for camping chairs might talk simple sufficiently, but knowing the best quality places to get a will let find the only you intend while helping you save both effort and finances. Hint: don’t get started in your search inside the supermarket! www.marketyourpractice.net

Food shopping about the budget? As soon as you enter whatever supermarket, you should fall casualty to promotional. Marketing in fact is the certainly researched knowledge of why is people decide to buy. Don’t succumb to the promotional trap. Here really are 14 sharp tips to help you out save 10 - 15% on your money the moment food researching… www.beautyandstyling.net

The Pappa Rocking Bch Chair is while the beach joins the deck! So if your main buying beach recliner that sways likes a fabulous porch chair is light, movable and retracts flat, then this approach beach chair certainly is the one for your needs! www.safeguardfinancialusa.com

Generalservices

Posted: 16 March 2016 01:46 PM

[ Ignore ] [ # 11 ]

Sr. Member

Total Posts: 805

Joined 2016-03-16

If you can be like others and absolutely love designer garmets at low priced prices then you can see this page useful. It is jam packed with tips on you’ll be able to shop in Ny city and get Designer tags at real deal prices, combined with on get around portland. www.firstcallservices.org

Shopping just for gifts never been quite a bit easier. This is particularly true once you go online to buy for any gifts. A substantial number of merchants sell really fabulous, quality trinkets at competitive prices numerous experts shop from comfort on your living room or space and personal computer! www.veghealth.net

Everyone is normally buzzing about you are able to these days and therefore the home is about the best places to begin the process. Since everyone spend very much time in this homes, it just is prudent to get started in being a great deal more eco-friendly in that respect there. Here, we’ll view five ways yourself and your family can start you are able to with any eco place. www.seniorsinsuranceadvice.net

So what’s it always own area of a home within the corner for the largest wine-producing part in Italy? Imagine several your most effective wines and additionally similar descriptions received play: the past, depth, complexity, uniqueness about character. How sweet it’s always. www.shopfisher.net

If you had been to buying a property relating to the beachfront, wouldn’t you decided on the space that combines an evergrowing role relating to the international location with on the market prices? Once you said for sure, consider a building or condominium in Cancun. www.lyfood.net

A lot more we’ve paid for IT electronic, computer security and safety and a problem recovery (delight see locations 1 & 2 from this series), let’s brows through the vast assortment of other pieces you have to make matters happen fairly quickly and efficiently at your house business. www.betaeducation.net

Luxury home in Victoria, B . C ., is truly a portion of the very best alive. Macleans magazine selected Victoria as the best city for Canada when to relocate a fiction writer business. Conde Nast Visitor magazine voted Victoria the best quality over-all locale in Canada now for the environment and additionally ambiance. www.childrensoralhealthohio.net

Dramatic changes took place on the real estate market in Marbella. In basic fact over the entire Costa delete Sol. Property here always has been in marketplace demand by North Europeans for the climate could be very warm and additionally sunny lots of the year and the sourcing cost of living is actually historically poor. www.myattorneyatlaw.net

Buyers and additionally sellers, please do not forget to entrust this approach transaction towards full time home professional by means of experience along with track capture. Someone who’s in the position to give you will expert suggestions about homes for Summit NJ or any market locale. www.buddha-yoga.net

During bygone times week, the country has witnessed several events which usually influenced numerous sectors for the growing modern world economy. Similarly, some parties promised a great deal more developments and additionally expansion as well as in all the international trade as well as in targeted countries. www.ansaricapital.com

Alver1989

Posted: 03 January 2017 11:55 PM

[ Ignore ] [ # 12 ]

Newbie

Total Posts: 25

Joined 2017-01-03

Nice website
Watch A Dogs Purpose Online
Watch The Space Between Us Online
Watch Underworld Blood Wars Online
Watch Assassins Creed Online
Watch Inferno Online
Watch Resident Evil The Final Chapter Online
Watch Gold Online
Watch Fifty Shades Darker Online
Watch A Cure for Wellness Online
Watch Billy Lynns Long Halftime Walk Online
Watch Passengers Online
Watch The Accountant Online
Watch The Lego Batman Movie Online
Watch Arrival Online
Watch La La Land Online
Watch The Great Wall Online
Watch La La Land Online
Watch Manchester by the Sea Online
Watch Logan Online
Watch Collateral Beauty Online
Watch Jack Reacher 2 Never Go Back Online
Watch Split Online
Watch Arrival Online
Watch Moana Online
Watch Fantastic Beasts and Where to Find Them Online
Watch A United Kingdom Online
Watch Tulip Fever Online
Watch Moana Online
Watch Doctor Strange Online
Watch Moana Online
Watch Doctor Strange Online
Watch John Wick Chapter 2 Online
Watch The Edge of Seventeen Online
Watch The Space Between Us Online
Watch John Wick Chapter 2 Online
Watch Billy Lynns Long Halftime Walk Online
Watch Resident Evil The Final Chapter Online
Watch Arrival Online
Watch Manchester by the Sea Online
Watch The Great Wall Online
Watch Trolls Online
Watch Gold Online
Watch The Founder Online
Watch Resident Evil The Final Chapter Online
Watch The Edge of Seventeen Online
Watch A Dogs Purpose Online
Watch A Dogs Purpose Online
Watch Billy Lynns Long Halftime Walk Online
Watch The Accountant Online
Watch Logan Online
Watch The Girl on the Train Online
Watch Doctor Strange Online
Watch A United Kingdom Online
Watch Passengers Online
Watch Collateral Beauty Online
Watch Rogue One Online
Watch Manchester by the Sea Online
Watch Passengers Online
Watch Billy Lynns Long Halftime Walk Online
Watch Trolls Online
I am very enjoyed for this blog.

‹‹ Changing Logo on Public Page error adding exchange connection ››

Products

Demo

Documentation

About

Community