Topic Last Modified: 2010-11-11

The Microsoft Exchange Server 2010 Management Pack for System Center Operations Manager includes a performance data collection engine that is used to query performance counter objects on computers running Exchange 2010. For this Operations Manager rule, data is collected by using the performance counter specified in the Details table.

To review the value of the performance counter that generated this alert, in Operations Manager, double-click this alert, and then click the General tab. Review the description of the alert that includes the variables specific to your environment.

Details

Product Name

Exchange

Product Version

14.0 (Exchange 2010)

Object Name

MSExchange Search Indices

Counter Name

Time Since Last Notification Was Indexed

Sample Interval

300

Server Role

Ex14. Mailbox

Critical Error Threshold

3600

Rule Path

Microsoft Exchange Server/Exchange 2010/Mailbox/Database Search Copy

Rule Name

Indexing Stalled: Time Since Last Notification Was Indexed (>1hr behind for 30min).

Explanation

This alert indicates that a performance or corruption issue may affect the content indexing catalogs on a passive mailbox database copy in a particular Microsoft Exchange Server 2010 database availability group (DAG).

A database availability group (DAG) is a set of Mailbox servers that provide automatic recovery from failures that affect individual databases. Any server in a DAG can host a copy of a mailbox database from any other server in the DAG. When a server is added to a DAG, it works together with the other servers in the DAG to provide automatic recovery from failures that affect mailbox databases.

The MS Search (Exchange) and Microsoft Exchange Search Indexer services provide content indexing functionality for the Exchange databases.

System Center Operations Manager monitors the indexing process to determine whether an indexing backlog occurs. An alert is generated when more than one hour has passed since the last notification event was indexed. In this scenario, if the passive mailbox copy becomes the active copy, users may be unable to search on or locate items in Exchange.

User Action

To resolve this error, do one or more of the following:

  • Review the Application log and System log on your Exchange 2010 servers for related events. For example, events that occur immediately before and after this event may provide more information about the root cause of this error.

  • Review the Operations Console in Operations Manager for detailed information about the cause of this problem. For more information, see the "Introduction" section in this article.

  • Run the Troubleshoot-CI.ps1 PowerShell script to determine whether an issue affects content indexing on the server. This script is located in the Scripts folder under the Exchange installation folder. In this scenario, check for the following items:

    • Corruption - This is indicated by a CatalogHealth error code of CatalogNeedReset.

    • Stalled indexing - This is indicated by a The Time Since the Last Notification was indexed value of more than two hours.

    • Deadlock issues - This is indicated by a CatalogHealth time stamp that is more than two hours old for each catalog entry.

    • Backlogged indexing - This is indicated when the Age of the Last Notification Indexed counter is more than 48 hours old.

  • For deadlock issues and for stalled indexing issues, restart the MS Search (Exchange) and Microsoft Exchange Search Indexer services.

  • For corruption issues, reset the search index, and then reseed the indexing catalog. To do this, follow these steps:

    1. Start the Exchange Management Shell, and then run the cd $exscripts command to change to the Scripts directory.

    2. Run the ResetSearchIndex.ps1 script together with the name of the Exchange mailbox database. For example, .\ResetSearchIndex.ps1 DB1. The following appear:

      WARNING: Waiting for service 'Microsoft Exchange Search Indexer (MSExchangeSearch)' to finish stopping...

      MSExchangeSearchService stopped

      Deleting catalog for DB1

      Removing: C:\ProgramFiles\Microsoft\ExchangeServer\V14\Mailbox\DBX1\CatalogData-<GUID>

      Control-C to quit or Enter to continue:

      MSExchangeSearchService Started.

    3. Allow sufficient time for the new search catalog to become up-to-date. To determine whether the catalog is up-to-date, examine the value of the Full Crawl Mode Status performance counter. This counter is located in the MSExchange Search Indices performance object. When you configure this counter, select the database on which the catalog has been created. While the database is being crawled, Full Crawl Mode Status has a value of 1. When the full crawl has completed successfully, Full Crawl Mode Status has a value of 0 (zero).

    4. To reseed the search catalog on other nodes in the DAG, use the following command:

      Copy Code
       Update-MailboxDatabaseCopy -Identity DB1\MBX1 -SourceServer MBX2 -CatalogOnly
      
      In this command, the SourceServer value represents the server on which you ran the ResetSearchIndex.ps1 script. For more information, see Update a Mailbox Database Copy.

  • Examine the current replication status for each replica database. To do this, use the Get-MailboxDatabaseCopyStatus cmdlet. This cmdlet returns information about all copies of a particular database, information about a specific copy of a database on a specific server, or information about all database copies on a server.

For more information, see the following topics in Exchange 2010 Help:

  • Understanding Mailbox Database Copies

  • Understanding Database Availability Groups

For More Information

If you are not already doing so, consider running the Exchange tools created to help you analyze and troubleshoot your Exchange environment. These tools can help make sure that your configuration aligns with Microsoft best practices. They can also help you identify and resolve performance issues, improve mail flow, and better manage disaster recovery scenarios. To run these tools, go to the Toolbox node of the Exchange Management Console. To learn more about these tools, see Managing Tools in the Toolbox.