Search

SharePoint not crawling links

shehan — Tue, 25 Aug 2009 05:30:46 GMT

We have a document library which makes use of the 'Link to a Document' content type which lets users add links to a document library. We noticed that users were using this feature to add links to documents hosted on sites outside of the SharePoint farm. This is all good as far as they are concerned because the content type does its job and end-user can access the external documents by clicking the link.

However, we noticed that none of these links were showing up in the search results. When we looked into the crawl log (/ssp/admin/_layouts/LogSummary.aspx) we found that all these links had an Error status type. The error text was:

This item could not be crawled because the crawler could not connect to the repository. (The item was deleted because it was either not found or the crawler was denied access to it.)

It took some digging, but the issue was because the index server was behind a firewall that was preventing traffic to external sites. Once we made exclusions to the firewall rules (by adding exceptions to the IP address and ports of the external site) and performed a full crawl the links were indexed and started showing up in the search results.

Technorati Tags: Search,Link to a Document,Crawl,Firewall

A little 'gotcha' when restarting the search service

shehan — Fri, 21 Aug 2009 00:15:34 GMT

If you were to restart (i.e stop and then start) the search service, in a SharePoint farm, from the 'Services on server' page in Central Administration chances are that you might encounter both or either of the following errors when yo do a search on your web application:

Your search cannot be completed because of a service error. Try your search again or contact your administrator for more information.

The search request was unable to connect to the Search Service.

This happens because after restarting the service on the index server you need to assign an index server to the SSP. Here are the steps:

1. Start the search services on all the servers that you stopped.

2. Navigate to the Manage SSP page (_admin/managessp.aspx).

3. Place you mouse over the SSP name and select 'Edit Properties' from the drop down menu.

4. In page that opens up, scroll to the end and select the index server from the drop down list box.

5. Hit the OK button to save.

6. Give around 5 to 10 minutes for index propagation and initialization to take place and Search would start working again.

Another 'gotcha' to keep note of is that whenever you restart services or assign an index server to the SSP it should be done at a time when the crawler is 'Idle'. Else there is a high probability of the crawler performing an endless crawl.

Technorati Tags: Search,SSP,Index Server,Error,Crawl

Exclude task items from appearing in search results

shehan — Thu, 21 May 2009 13:30:01 GMT

I had a request to exclude items contained in the approval tasks list from appearing in SharePoint's search results. At first glance it could be as simple as making changes to the lists properties (List Settings -> Advanced Settings) to explicitly exclude it from appearing in search results. This would work if you only had a few lists to update, but lets say, that like me, you had well over 100 lists to update....You could develop a utility that makes use of SharePoint's API to update the list, but that would work only for existing approval lists and not ones created later on.

The approach I used is a simple and one-off task - I update the search scope by adding an exclusion rule.

To do this:

Open up the Search Settings / Search Administration page
Open up the Scopes page and place the mouse pointer over the scope that needs to be updated. A dropdown menu will appear; select Edit Properties and Rules
In the scope properties screen, click New Rule
In the add scope rule page:

Scope Rule Type: Property Query
Property Query: contentclass = STS_ListItem_Tasks
Behavior: Exclude
Click OK

Return to the Search Settings page and click 'Start update now' under the scopes section

Technorati Tags: SharePoint,Search

Invalid host file on index server when using a dedicated crawler

shehan — Thu, 19 Mar 2009 19:07:25 GMT

Incase you didn't know- when you set your index server to use a dedicated server for crawling, SharePoint updates the host file on the index server with the IP address of the WFE. However, there are times where SharePoint adds an invalid host entry when there are 2 or more WFE's.

Example - Say there have 2 WFE's - MOSSWFE01 & MOSSWFE02. Setting MOSSWFE02 as the dedicated crawler will cause SharePoint to update the host file in the index server. It adds an entry for MOSSWFE01 with the IP address of MOSSWFE02. Hence, the index server cannot access the index file location on MOSSWFE01. No use trying to update the host file manually since SharePoint reverts your change every few minutes.

With the invalid host file entry you'll start seeing errors on both the index server and the WFE (see below). The errors are due to the index servers inability to propagate the index file to the WFE. If you modify the index server to use all WFE's for crawling then no entries are added to the index file.

I couldn't find a supported resolution for this, but there is a 'hack' to overcome this! Instead of setting a dedicated crawler, select the option to use all WFE's for crawling. Next open the host file on the index server and add the host names of the web apps along with the IP address of the WFE you want to act as the crawler.

Here is the TechNet article on this:http://technet.microsoft.com/en-us/library/cc261810.aspx

Error on WFE:

Description:
Application Server Administration job failed for service instance Microsoft.Office.Server.Search.Administration.SearchServiceInstance (4d1c24af-411b-42b0-9f05-f657644e7d4b).

Reason: Object 0ad5f391-9f4b-4f49-8f85-3550343690e1 not found.

Techinal Support Details:
System.Collections.Generic.KeyNotFoundException: Object 0ad5f391-9f4b-4f49-8f85-3550343690e1 not found.
   at Microsoft.Office.Server.Search.Administration.SearchApi.get_App()
   at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.Synchronize()
   at Microsoft.Office.Server.Administration.ApplicationServerJob.ProvisionLocalSharedServiceInstances(Boolean isAdministrationServiceJob)

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Error in Index Server:
Description:
The Execute method of job definition Microsoft.Office.Server.Search.Administration.AddQueryServerJob (ID bcd3ce3e-b058-46ef-81a9-1bda53fa6c45) threw an exception. More information is included below.

Index location '\\MOSSWFE01\searchindexpropagation' not found on indexer 'M0SSWFE1'. Aborting query server deployment for application SharedServices1.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Technorati Tags: Search,Host File,Index

Search only items of a custom list

shehan — Thu, 10 Jul 2008 00:27:33 GMT

Had a requirement where search results should be limited to only items of a list. Naturally, I created a new Search Scope and set a rule of type Web Address to include the particular site holding the lists.

Well...it did work, but not to my expectations. What was happening was that in addition to displaying the list items, the list views were also being displayed and so were other pages in the site....DOH!

Doing a search on the net led me to this site. It was very informative, but it didn't provided the contentclass value I should use for a list created using the Custom List template....

Took me a while, but eventually tracked down the contentclass value - "STS_ListItem_GenericList"

So, long story short; here are the two rules I used for the scope:

Rule1:
Scope Rule Type: Web Address
Folder: <the site I need to search>
Behavior: Include

Rule 2:
Scope Rule Type: Property Query
Property Query: contentclass = STS_ListItem_GenericList
Behavior: Require

Technorati Tags: moss,search,scope

Display Search Scopes in the Search DoropDown and Advanced Search page

shehan — Thu, 28 Feb 2008 22:15:27 GMT

Most of you all would probably be knowing this, but for the newbie out there who is just getting into MOSS's Enterprise Search features this would probably prove useful...

Lets say that through the SSP Search Configurations page you've gone and created a couple of Scopes. But when you get back to your site collection you notice that none of the scopes appear in the search scope dropdown (normally located on the top right hand corner of the page), you only see the default scopes.

What you need to do is navigate to Site Settings page (_layouts/settings.aspx) of the top most level site and once there click on the ‘Search keywords’ link under the ‘Site Collection Administration’ section.

In the Scopes page you will notice that all the custom scopes created are contained in the ‘Unused Scopes’ section while the ‘Display Group: Search Dropdown’ contains ‘All Sites’ and ‘People’ while the ‘Display Group: Advanced Search’ contains only ‘All Sites’.

Click the ‘Search Dropdown’ link and add the Scopes by checking the appropriate ones. Do the same for the ‘Advanced Search’ link.

Once done, the custom scopes will appear on all search dropdowns in the site, however they will not appear on the default Advance Search page.

To get the scopes to appear on the Advanced Search page, navigate to that page (SearchCenter/Pages/advanced.aspx) and from the 'Site Actions' menu select 'Edit Page'. The page will now be checked out to you and be in the edit mode.

In the 'Advanced Search Box' web part click the 'edit' menu and then select 'Modify Shared Web Part'. The properties box for the web part appears. Expand the 'Scopes' section and check the 'Show the scope picker' option. Hit OK and the web part should update showing the scopes.

Remember to check in and publish the page so your changes will be seen by everyone who access the page.

Change the "Welcome" link font color

shehan — Mon, 25 Feb 2008 23:18:33 GMT

As part of configuring enterprise search for a company's portal, one of the requirements was changing the look and feel of the SearchCenter. In short, they wanted a very Google like UI - I.e. nothing too flashy or heavy.

So, I started off creating a minimal master page and adjusting the layouts - nothing major just positioning the controls etc.. Now, to change the appearance of the webparts required modifications to the default styles located in the core.css file and thanks to Heather Solomons CSS Reference Chart this was pretty much easy. But when it came to changing the font color of the "Welcome" link, changes to the styles mentioned by her weren't working....

Took me a while, but managed to find the solution by examining the source of the generated web page. Seems that the style in use for the font color of the "Welcome" link is: .ms-SPLink A:link,.ms-SPLink A:visited

To change the font color, modify the existing class like:

.ms-SPLink A:link,.ms-SPLink A:visited
{
color: #FFFFFF; /*#2A4666; this is the font color for the "Welcome" Link*/
text-decoration:none;
}

and if you want to change the font color on mouse over then add a new class:
.ms-SPLink A:hover
{

color: #66FFFF; /*this is the font color for the "Welcome" Link - mouse over*/
text-decoration:none;
}

Do remember that changes to the core.css file will result in changes that will be reflected all over the portal. So, if you only want the change to appear in on site don't go change the cosre.css file....

Highlight Search Results

shehan — Fri, 22 Feb 2008 01:38:47 GMT

OOTB the Search Core Results webpart simply emphasizes the search string you entered in the Search Box by wrapping it within the tags. But let’s say you need to highlight the search terms in different colors; what do you do? Write a new results webpart? No! you only need to adjust the Search Core Results XLS.

This is how it’s done:

Open up the XLS editor and copy the contents to notepad or some other editor (if your fine with editing it in the MOSS’s default editor, go right ahead)
Search for the term <xsl:template match="ddd">
Just below that you will see :

<xsl:template match="c0">

<xsl:value-of select="."/>

</xsl:template>

<xsl:template match="c1">

<xsl:value-of select="."/>

</xsl:template>

And this goes on till “c9”. The values “c0” to “c9” correspond to the search terms you entered, where “c0” is the first, “c1” is the second and so on…

Replace this with something like:

<xsl:template match="c0">

 

 <xsl:value-of select="."/>

 

</xsl:template>

<xsl:template match="c1">

 

 <xsl:value-of select="."/>

As you can see I’ve used the style “background-color” to give each search term its own color.

Once done copy the updated XLS file back to the webpart and save

Searching for "microsoft sharepoint server"

with the default XLS:

with the modified XLS:

Technorati Profile

Crawl Log Exporter

shehan — Thu, 21 Feb 2008 23:44:36 GMT

As part of setting up Enterprise Search for a major IT company, I had to create content sources for a number of web sites hosted on the company’s intranet. Pretty easy you say, well yes. But the problems arose when MOSS began crawling the websites.

There were more Crawl Errors in the crawl log than Crawl Successes. Seems that the web applications MOSS was crawling were not being maintained properly and there were loads of broken links. Naturally, I pointed this fact out to the administrators and naturally they asked me for the list of broken links, etc. Well, if there were only a couple of errors I could easily copy and paste the offending URLs, but these sites had loads and I couldn’t find an “export log” feature, drat!!!

Right, so the only option was to cook up a little C# winform application to export the crawl log. Easier said than done :) Other than for MSDN I couldn’t find anything else online to get me started, so, had to use Reflector quite a bit to figure out the internal workings of the assemblies, but managed to get something going….

I’ve uploaded the complied application on ProjectDistributor and looking to soon put it up on CodePlex.

As of now the log can be exported to only CSV format and there is no threading, so don’t freak out when the app starts hanging. I’ll be refactoring the code and adding more export options once I get the time…

Get the application from here:
http://www.projectdistributor.net/Projects/Project.aspx?projectId=259