top of page
  • Mayur Khatale

Popularity trends not pulling the data

Issue : Usage report is showing up 0’s even though there are actual hits on the sites.



Root Cause:


Verified and found that below two timer services ‘Last run time’ is showing up as N/A, due to which we are not getting the data to the Usage reports.

  • Analytics Timer Job for Search Service Application

  • Usage Analytics Timer Job for Search Application



All the remaining settings are fine.


Troubleshooting and Resolution:


1. Verify Search Health using the below powershell script :



<# =====================================================================
## Title       : Get-SPSearchInformation
## Description : This script will collect information regarding Search and the SSA's in the Farm.
## Author      : Anthony Casillas | Brian Pendergrass | PG
## Date        : 08-16-2016
## Input       : 
## Output      : 
## Usage       : .\Get-SP2013SearchInfo.ps1
## Notes       : 
## Tag         : Search, Sharepoint, Powershell
## Change log  : 1.1 [acasilla]
##             :   - Added Content Source SharePointCrawlBehavior type / If the Behavior is not CrawlVirtualServers, then the report will state the Start addresses are not local to the farm ( this is normal )
## 
## =====================================================================
#>

Add-PSSnapin Microsoft.SharePoint.PowerShell
Import-Module WebAdministration

$timestamp = $(Get-Date -format "MM-dd-yyyy_hhmm")
$output = Read-Host "Enter a location for the output file (For Example: C:\Temp)" 
$outputfile = $output + "\SP2013SearchInfo_" + $timestamp +".txt"
Write-Output ""
Write-Output "Collecting Some SharePoint and Search Info and writing that output to $outputfile"
Write-Output ""

Function WriteErrorAndExit($errorText)
{
    Write-Host -BackgroundColor Red -ForegroundColor Black $errorText
    Write-Host -BackgroundColor Red -ForegroundColor Black "Aborting script"
    exit
}


# ------------------------------------------------------------------------------------------------------------------
# GetSSA: Get SSA reference 
# ------------------------------------------------------------------------------------------------------------------
function GetSSA
{
    $ssas = @(Get-SPEnterpriseSearchServiceApplication)
    if ($ssas.Count -ne 1)
    {
        WriteErrorAndExit("This script only supports a single SSA configuration. If you have more than one SSA, please edit this script ( around line 24, to target a specific SSA")
        
    }
    

    $global:ssa = $ssas[0]
    if ($global:ssa.Status -ne "Online")
    {
        $ssaStat = $global:ssa.Status
        WriteErrorAndExit("Expected SSA to have status 'Online', found status: $ssaStat")
    }

}

Function GetFarmBuild()
{
    "[SharePoint Farm Build:" + (Get-SPFarm).BuildVersion + "]"
    ""
}


function GetServersInFarm()
{
    "#########################################################################################"
    "   Servers in the Farm "
    "#########################################################################################"

    foreach($svr in Get-SPServer | Select DisplayName, Id, Status, Role)
    {
    $svr
    }
    ""
}


function GetServiceInstances()
{
    "#########################################################################################"
    "   What Service Instanes are running and on what Server? "
    "#########################################################################################"
    ""
     $serviceInstances = Get-SPServiceInstance |?{$_.Status -ne "Disabled"}
     foreach ($si in $serviceInstances)
     {
       $si.Server.Address + " -- "  + $si.TypeName + " -- " + $si.Status
     }
     ""
}


function GetSSAFullObject()
{
    $fullSsaObject = New-Object PSObject
     "#########################################################################################"
    "   " + $global:ssa.Name + " Object  " 
    "#########################################################################################" 

    $fullSsaObject | Add-Member ServiceName $global:ssa.ServiceName
    $fullSsaObject | Add-Member Name $global:ssa.Name
    $fullSsaObject | Add-Member DisplayName $global:ssa.DisplayName
    $fullSsaObject | Add-Member TypeName $global:ssa.TypeName
    $fullSsaObject | Add-Member ApplicationName $global:ssa.ApplicationName
    $fullSsaObject | Add-Member Id $global:ssa.Id
    $fullSsaObject | Add-Member DefaultSearchProvider $global:ssa.DefaultSearchProvider
    $fullSsaObject | Add-Member LocationConfigurations $global:ssa.LocationConfigurations
    $fullSsaObject | Add-Member AlertNotificationFormat $global:ssa.AlertNotificationFormat
    $fullSsaObject | Add-Member QueryLoggingEnabled $global:ssa.QueryLoggingEnabled
    $fullSsaObject | Add-Member CloudIndex $global:ssa.CloudIndex
    $fullSsaObject | Add-Member QuerySuggestionsEnabled $global:ssa.QuerySuggestionsEnabled
    $fullSsaObject | Add-Member PersonalQuerySuggestionsEnabled $global:ssa.PersonalQuerySuggestionsEnabled
    $fullSsaObject | Add-Member SearchCenterUrl $global:ssa.SearchCenterUrl
    $fullSsaObject | Add-Member SharedSearchBoxSettings $global:ssa.SharedSearchBoxSettings
    $fullSsaObject | Add-Member UrlZoneOverride $global:ssa.UrlZoneOverride
    $fullSsaObject | Add-Member HeadQueryFrequencyThreshold $global:ssa.HeadQueryFrequencyThreshold
    $fullSsaObject | Add-Member NameNormalizationEnabled $global:ssa.NameNormalizationEnabled
    $fullSsaObject | Add-Member NameNormalizationPreferredNamePID $global:ssa.NameNormalizationPreferredNamePID
    $fullSsaObject | Add-Member QueryLogSettings $global:ssa.QueryLogSettings
    $fullSsaObject | Add-Member DiacriticSensitive $global:ssa.DiacriticSensitive
    $fullSsaObject | Add-Member VerboseQueryMonitoring $global:ssa.VerboseQueryMonitoring
    $fullSsaObject | Add-Member VerboseSubFlowTiming $global:ssa.VerboseSubFlowTiming
    $fullSsaObject | Add-Member SearchAdminDatabase $global:ssa.SearchAdminDatabase.Name
    $fullSsaObject | Add-Member CrawlStores $global:ssa.CrawlStores
    $fullSsaObject | Add-Member LinksStores $global:ssa.LinksStores
    $fullSsaObject | Add-Member AnalyticsReportingDatabases $global:ssa.AnalyticsReportingDatabases
    $fullSsaObject | Add-Member AnalyticsReportingStore $global:ssa.AnalyticsReportingStore
    $fullSsaObject | Add-Member CrawlLogCleanupIntervalInDays $global:ssa.CrawlLogCleanupIntervalInDays
    $fullSsaObject | Add-Member DefaultQueryTimeout $global:ssa.DefaultQueryTimeout
    $fullSsaObject | Add-Member MaxQueryTimeout $global:ssa.MaxQueryTimeout
    $fullSsaObject | Add-Member MaxKeywordQueryTextLength $global:ssa.MaxKeywordQueryTextLength
    $fullSsaObject | Add-Member DiscoveryMaxKeywordQueryTextLength $global:ssa.DiscoveryMaxKeywordQueryTextLength
    $fullSsaObject | Add-Member DiscoveryMaxRowLimit $global:ssa.DiscoveryMaxRowLimit
    $fullSsaObject | Add-Member MaxRowLimit $global:ssa.MaxRowLimit
    $fullSsaObject | Add-Member MaxRankingModels $global:ssa.MaxRankingModels
    $fullSsaObject | Add-Member AllowQueryDebugMode $global:ssa.AllowQueryDebugMode
    $fullSsaObject | Add-Member AllowPartialResults $global:ssa.AllowPartialResults
    $fullSsaObject | Add-Member AllowedMaxRowLimitSp14 $global:ssa.AllowedMaxRowLimitSp14
    $fullSsaObject | Add-Member AlertsEnabled $global:ssa.AlertsEnabled
    $fullSsaObject | Add-Member FarmIdsForAlerts $global:ssa.FarmIdsForAlerts
    $fullSsaObject | Add-Member AlertNotificationQuota $global:ssa.AlertNotificationQuota
    $fullSsaObject | Add-Member ResetAndEnableAlerts $global:ssa.ResetAndEnableAlerts
    $fullSsaObject | Add-Member SystemManagerLocations $global:ssa.SystemManagerLocations
    $fullSsaObject | Add-Member ApplicationClassId $global:ssa.ApplicationClassId
    $fullSsaObject | Add-Member ManageLink $global:ssa.ManageLink
    $fullSsaObject | Add-Member PropertiesLink $global:ssa.PropertiesLink
    $fullSsaObject | Add-Member MinimumReadyQueryComponentsPerPartition  $global:ssa.MinimumReadyQueryComponentsPerPartition 
    $fullSsaObject | Add-Member TimeBeforeAbandoningQueryComponent $global:ssa.TimeBeforeAbandoningQueryComponent
    $fullSsaObject | Add-Member EnableIMS $global:ssa.EnableIMS
    $fullSsaObject | Add-Member FASTAdminProxy $global:ssa.FASTAdminProxy
    $fullSsaObject | Add-Member UseSimpleSchemaUI $global:ssa.UseSimpleSchemaUI
    $fullSsaObject | Add-Member LatencyBasedQueryThrottling $global:ssa.LatencyBasedQueryThrottling
    $fullSsaObject | Add-Member CpuBasedQueryThrottling $global:ssa.CpuBasedQueryThrottling
    $fullSsaObject | Add-Member LoadBasedQueryThrottling $global:ssa.LoadBasedQueryThrottling
    $fullSsaObject | Add-Member IisVirtualDirectoryPath $global:ssa.IisVirtualDirectoryPath
    $fullSsaObject | Add-Member ApplicationPool $global:ssa.ApplicationPool.DisplayName
    $fullSsaObject | Add-Member PermissionsLink $global:ssa.PermissionsLink
    $fullSsaObject | Add-Member DefaultEndpoint $global:ssa.DefaultEndpoint
    $fullSsaObject | Add-Member Uri $global:ssa.Uri
    $fullSsaObject | Add-Member Shared $global:ssa.Shared
    $fullSsaObject | Add-Member Comments $global:ssa.Comments
    $fullSsaObject | Add-Member TermsOfServiceUri $global:ssa.TermsOfServiceUri
    $fullSsaObject | Add-Member Service $global:ssa.Service
    $fullSsaObject | Add-Member ServiceInstances $global:ssa.ServiceInstances
    $fullSsaObject | Add-Member ServiceApplicationProxyGroup $global:ssa.ServiceApplicationProxyGroup
    $fullSsaObject | Add-Member ApplicationVersion $global:ssa.ApplicationVersion
    $fullSsaObject | Add-Member CanUpgrade $global:ssa.CanUpgrade
    $fullSsaObject | Add-Member IsBackwardsCompatible $global:ssa.IsBackwardsCompatible
    $fullSsaObject | Add-Member NeedsUpgradeIncludeChildren $global:ssa.NeedsUpgradeIncludeChildren
    $fullSsaObject | Add-Member NeedsUpgrade $global:ssa.NeedsUpgrade
    $fullSsaObject | Add-Member UpgradeContext $global:ssa.UpgradeContext
    $fullSsaObject | Add-Member Status $global:ssa.Status
    $fullSsaObject | Add-Member Parent $global:ssa.Parent
    $fullSsaObject | Add-Member Version $global:ssa.Version
    $fullSsaObject | Add-Member Properties $global:ssa.Properties.Values
    $fullSsaObject | Add-Member Farm $global:ssa.Farm
    $fullSsaObject | Add-Member UpgradedPersistedProperties $global:ssa.UpgradedPersistedProperties
    $fullSsaObject | Add-Member CanSelectForBackup $global:ssa.CanSelectForBackup
    $fullSsaObject | Add-Member DiskSizeRequired $global:ssa.DiskSizeRequired
    $fullSsaObject | Add-Member CanSelectForRestore $global:ssa.CanSelectForRestore
    $fullSsaObject | Add-Member CanRenameOnRestore $global:ssa.CanRenameOnRestore

    $fullSsaObject | fl
}

function GetSSALegacyAdminComponent()
{
    " -------------------------------------------------------------------"
    "   Legacy Admin Component"
    " -------------------------------------------------------------------"
    ""
    $global:ssa.AdminComponent
    ""
    " -------------------------------------------------------------------"
}

function GetSearchTopo()
{
    
    $activeTopo = Get-SPEnterpriseSearchTopology -SearchApplication $global:ssa -Active
    "#################################################################################################################"
    " Search Topology ( ID: " + $global:ssa.ActiveTopology.TopologyId.Guid.ToString() + " ) for: " + "(  " + $global:ssa.Name + " )"
    "#################################################################################################################"
    
    Get-SPEnterpriseSearchComponent -SearchTopology $activeTopo | Select * | Sort  -Property Name
}

Function GetContentSources()
{
    
      $crawlAccount = (New-Object Microsoft.Office.Server.Search.Administration.Content $global:ssa).DefaultGatheringAccount
      "#########################################################################################"
      "  *** Content Sources (" + $global:ssa.Name + ") *** " + " Crawl Account: " + $crawlAccount
      "#########################################################################################"
      $contentSources = Get-SPEnterpriseSearchCrawlContentSource -SearchApplication $global:ssa;
      foreach ($contentSrc in $contentSources) {
        ""
		"------------------------------------------------------------------------------------------------------- "
		$contentSrc.Name + " | ( ID:" + $contentSrc.ID + " TYPE:" + $contentSrc.Type + "  Behavior:" + $contentSrc.SharePointCrawlBehavior + ")"
		"------------------------------------------------------------------------------------------------------- "
        foreach ($startUri in $contentSrc.StartAddresses) 
        { 
          if ($contentSrc.Type.toString() -ieq "SharePoint")
          {
            $spSrc = @{}
            if ($startUri.Scheme.toString().toLower().startsWith("http")) 
            {
              $isRemoteFarm = $true ## Assume Remote Farm Until Proven Otherwise ##
              foreach ($altUrl in Get-SPAlternateUrl) {
                if ($startUri.AbsoluteUri.toString() -ieq $altUrl.Uri.toString()) 
                {
                  $isRemoteFarm = $false                
                  if ($altUrl.UrlZone -ieq "Default") 
                  {
                    "  Start Address: " + $startUri
                    "  AAM Zone: [" + $altUrl.UrlZone + "]" 
                    $inUserPolicy = $false;    #assume crawlAccount not inUserPolicy until verified
                    $webApp = Get-SPWebApplication $startUri.AbsoluteUri;
                    $IIS = $webApp.IisSettings[[Microsoft.SharePoint.Administration.SPUrlZone]::($altUrl.UrlZone)]
                    $isClaimsBased = $true
                    if ($webApp.UseClaimsAuthentication) 
                    { 
                        "  Authentication Type: [Claims]"
                        if (($IIS.ClaimsAuthenticationProviders).count -eq 1) 
                        {
                           "  Authentication Provider: " + ($IIS.ClaimsAuthenticationProviders[0]).DisplayName
                        } 
                        else 
                        {
                          "  Authentication Providers: "
                          foreach ($provider in ($IIS.ClaimsAuthenticationProviders)) 
                          {
                            "     - " + $provider.DisplayName
                          }
                        }
                    }
                    else {
                      $isClaimsBased = $false
                      "  Authentication Type: [Classic]"                  
                      if ($IIS.DisableKerberos) { "  Authentication Provider: [Windows:NTLM]" }
                      else { "  Authentication Provider:[Windows:Negotiate]" }
                    }
                    foreach ($userPolicy in $webApp.Policies) 
                    {
                      if($isClaimsBased)
                      {
                       $claimsPrefix = "i:0#.w|" 
                      }
                      if ($userPolicy.UserName.toLower().Equals(($claimsPrefix + $crawlAccount).toLower())) 
                      {
                        $inUserPolicy = $true;
                        "  Web App User Policy: {" + $userPolicy.PolicyRoleBindings.toString() + "}";
                      }
                    }
                    if (!$inUserPolicy) 
                    {
                      "  ---"  + $crawlAccount + " is NOT defined in the Web App's User Policy !!!";
                    }
                  }
                  else { 
                    "    [" + $altUrl.UrlZone + "] " + $startUri;
                    "  --- Non-Default zone may impact Contextual Scopes (e.g. This Site) and other search functionality"
                    "  ----- Check out http://blogs.msdn.com/b/sharepoint_strategery/archive/2014/07/08/problems-when-crawling-the-non-default-zone-explained.aspx " 
                  }
                }
              }
          
              if($isRemoteFarm)
              {
                "  This Start Address is NOT local to the Farm"
                "  Start Address: " + $startUri
              } 
            
            } 
            else {
              if ($startUri.Scheme.toString().toLower().startsWith("sps")) { "  " + $startUri + " [Profile Crawl]" }
              else 
              {
                if ($startUri.Scheme.toString().toLower().startsWith("bdc")) { "  URL: " + $startUri + " [BDC Content]" }
                else { "    -" + $startUri; }
              }
            }
          }
          else { "  Web Address: " + $startUri; }
          "  ---------------------------------------------"
        }
        ""
      }
}


function GetServerNameMappings()
{
    "#########################################################################################"
    "  *** Server Name Mappings (" + $global:ssa.Name + ") ***"
    "#########################################################################################"  
    $global:ssa  | Get-SPEnterpriseSearchCrawlMapping
    ""

}

function GetCrawlRules()
{
    "#########################################################################################"
    "  *** Crawl Rules (" + $global:ssa.Name + ") ***"
    "#########################################################################################"  
    $Rules = $global:ssa | Get-SPEnterpriseSearchCrawlRule 
    if ($Rules.count -lt 20) { $Rules }
    else 
    {
    ""
    "Top 20 (of " + $Rules.count + ") Crawl Rules"
    "---------------------------------------------"
    for ($i = 0; $i -le 21; $i++) { $Rules[$i]; }
    }
    ""
}



function displayGlobalSearchService
{
    "#########################################################################################"
    "  Search Service "
    "#########################################################################################"
    Get-SPEnterpriseSearchService
    $searchAdminProxy = (Get-SPEnterpriseSearchService).WebProxy
        if($searchAdminProxy.Address -ne $null)
        {
            " The Search Service has a Web Proxy defined. This will impact ALL SSA's and route crawl traffic to the Proxy regardless if the IE settings are set to NO PROXY"
            $searchAdminProxy
        }
}

Function GetSQSS()
{
    "#########################################################################################"
    "  Search Query and Site Settings (SQSS) - These should Only be running on your QPCs"
    "#########################################################################################"
    $instances = Get-SPServiceInstance | where {$_.TypeName -like "Search Query*"} | where {$_.Status -eq "Online"} | select Server,Id,Status | fl
    if ($instances -ne $null) {
      $instances
    }
}

function VerifyServiceEndpoints
{
	ForEach ($pt in $global:ssa.EndPoints)
    {  
		Get-WebUrl -url $pt.ListenUris.AbsoluteUri
    }
}



Function GetSSIs
{
    "#########################################################################################"
    "  Search Service Instance(s) :  Represents whats seen in CA > Services on Server "
    "#########################################################################################"
    ""
    $instances = Get-SPEnterpriseSearchServiceInstance | where {$_.Status -eq "Online"} | select Server,Id,Status,DefaultIndexLocation | fl
    if ($instances -ne $null) 
    {
      "Online Instance(s)"
      "-------------------"
      $instances
      ""
    }
    $instances = Get-SPEnterpriseSearchServiceInstance | where {$_.Status -ne "Online"} | select Server,Id,Status | fl
    if ($instances -ne $null) 
    {
      "Disabled Instance(s)" 
      "---------------------"
      $instances
      ""
    }
}

Function GetAAMs
{
	"#########################################################################################"
	" Alternate Access Mappings" 
	"#########################################################################################"
	    foreach($wa in Get-SPWebApplication -IncludeCentralAdministration){
        "-----------------------------------------------------------------"
        $wa.Name
        "-----------------------------------------------------------------"
        Get-SPAlternateURL -WebApplication $wa | Select IncomingUrl, Zone, PublicUrl | fl
        ""
        }
}


#################################################################################################################
#################################################################################################################

function HealthCheck ()
{

# ------------------------------------------------------------------------------------------------------------------
# GetCrawlStatus: Get crawl status
# ------------------------------------------------------------------------------------------------------------------
Function GetCrawlStatus
{
    if ($global:ssa.Ispaused())
    {
        switch ($global:ssa.Ispaused()) 
        { 
            1       { $pauseReason = "ongoing search topology operation" } 
            2       { $pauseReason = "backup/restore" } 
            4       { $pauseReason = "backup/restore" } 
            32      { $pauseReason = "crawl DB re-factoring" } 
            64      { $pauseReason = "link DB re-factoring" } 
            128     { $pauseReason = "external reason (user initiated)" } 
            256     { $pauseReason = "index reset" } 
            512     { $pauseReason = "index re-partitioning (query is also paused)" } 
            default { $pauseReason = "multiple reasons ($($global:ssa.Ispaused()))" } 
        }
        Write-Output "$($global:ssa.Name): Paused for $pauseReason"
    }
    else
    {
        $crawling = $false
        $contentSources = Get-SPEnterpriseSearchCrawlContentSource -SearchApplication $global:ssa
        if ($contentSources) 
        {
            foreach ($source in $contentSources)
            {
                if ($source.CrawlState -ne "Idle")
                {
                    Write-Output "Crawling $($source.Name) : $($source.CrawlState)"
                    $crawling = $true
                }
            }
            if (! $crawling)
            {
                Write-Output "Crawler is idle"
            }
        }
        else
        {
            Write-Output "Crawler: No content sources found"
        }
    }
}

# ------------------------------------------------------------------------------------------------------------------
# GetTopologyInfo: Get basic topology info and component health status
# ------------------------------------------------------------------------------------------------------------------
Function GetTopologyInfo
{
    $at = Get-SPEnterpriseSearchTopology -SearchApplication $global:ssa -Active
    $global:topologyCompList = Get-SPEnterpriseSearchComponent -SearchTopology $at

    # Check if topology is prepared for HA
    $adminFound = $false
    foreach ($searchComp in ($global:topologyCompList))
    {
        if ($searchComp.Name -match "Admin")
        { 
            if ($adminFound) 
            { 
                $global:haTopology = $true 
            } 
            else
            {
                $adminFound = $true
            }
        }
    }    

    #
    # Get topology component state:
    #
    $global:componentStateList=Get-SPEnterpriseSearchStatus -SearchApplication $global:ssa

    # Find the primary admin component:
    foreach ($component in ($global:componentStateList))
    {
        if ( ($component.Name -match "Admin") -and ($component.State -ne "Unknown") )
        {
            if (Get-SPEnterpriseSearchStatus -SearchApplication $global:ssa -Primary -Component $($component.Name))
            {
                $global:primaryAdmin = $component.Name
            }
        }
    }    
    if (! $global:primaryAdmin)
    {
        Write-Output ""
        Write-Output "-----------------------------------------------------------------------------"
        Write-Output "Error: Not able to obtain health state information."
        Write-Output "Recommended action: Ensure that at least one admin component is operational."
        Write-Output "This state may also indicate that an admin component failover is in progress."
        Write-Output "-----------------------------------------------------------------------------"
        Write-Output ""
        throw "Search component health state check failed"
    }
}

# ------------------------------------------------------------------------------------------------------------------
# PopulateHostHaList: For each component, determine properties and update $global:hostArray / $global:haArray
# ------------------------------------------------------------------------------------------------------------------
Function PopulateHostHaList($searchComp)
{
    if ($searchComp.ServerName)
    {
        $hostName = $searchComp.ServerName
    }
    else
    {
        $hostName = "Unknown server"
    }
    $partition = $searchComp.IndexPartitionOrdinal
    $newHostFound = $true
    $newHaFound = $true
    $entity = $null

    foreach ($searchHost in ($global:hostArray))
    {
        if ($searchHost.hostName -eq $hostName)
        {
            $newHostFound = $false
        }
    }
    if ($newHostFound)
    {
        # Add the host to $global:hostArray
        $hostTemp = $global:hostTemplate | Select-Object *
        $hostTemp.hostName = $hostName
        $global:hostArray += $hostTemp
        $global:searchHosts += 1
    }

    # Fill in component specific data in $global:hostArray
    foreach ($searchHost in ($global:hostArray))
    {
        if ($searchHost.hostName -eq $hostName)
        {
            $partition = -1
            if ($searchComp.Name -match "Query") 
            { 
                $entity = "QueryProcessingComponent" 
                $searchHost.qpc = "QueryProcessing "
                $searchHost.components += 1
            }
            elseif ($searchComp.Name -match "Content") 
            { 
                $entity = "ContentProcessingComponent" 
                $searchHost.cpc = "ContentProcessing "
                $searchHost.components += 1
            }
            elseif ($searchComp.Name -match "Analytics") 
            { 
                $entity = "AnalyticsProcessingComponent" 
                $searchHost.apc = "AnalyticsProcessing "
                $searchHost.components += 1
            }
            elseif ($searchComp.Name -match "Admin") 
            { 
                $entity = "AdminComponent" 
                if ($searchComp.Name -eq $global:primaryAdmin)
                {
                    $searchHost.pAdmin = "Admin(Primary) "
                }
                else
                {
                    $searchHost.sAdmin = "Admin "
                }
                $searchHost.components += 1
            }
            elseif ($searchComp.Name -match "Crawl") 
            { 
                $entity = "CrawlComponent" 
                $searchHost.crawler = "Crawler "
                $searchHost.components += 1
            }
            elseif ($searchComp.Name -match "Index") 
            { 
                $entity = "IndexComponent"
                $partition = $searchComp.IndexPartitionOrdinal
                $searchHost.index = "IndexPartition($partition) "
                $searchHost.components += 1
            }
        }
    }

    # Fill in component specific data in $global:haArray
    foreach ($haEntity in ($global:haArray))
    {
        if ($haEntity.entity -eq $entity)
        {
            if ($entity -eq "IndexComponent")
            {
                if ($haEntity.partition -eq $partition)
                {
                    $newHaFound = $false
                }
            }
            else 
            { 
                $newHaFound = $false
            }
        }
    }
    if ($newHaFound)
    {
        # Add the HA entities to $global:haArray
        $haTemp = $global:haTemplate | Select-Object *
        $haTemp.entity = $entity
        $haTemp.components = 1
        if ($partition -ne -1) 
        { 
            $haTemp.partition = $partition 
        }
        $global:haArray += $haTemp
    }
    else
    {
        foreach ($haEntity in ($global:haArray))
        {
            if ($haEntity.entity -eq $entity) 
            {
                if (($entity -eq "IndexComponent") )
                {
                    if ($haEntity.partition -eq $partition)
                    {
                        $haEntity.components += 1
                    }
                }
                else
                {
                    $haEntity.components += 1
                    if (($haEntity.entity -eq "AdminComponent") -and ($searchComp.Name -eq $global:primaryAdmin))
                    {
                        $haEntity.primary = $global:primaryAdmin
                    }
                }
            }
        }
    }
}

# ------------------------------------------------------------------------------------------------------------------
# AnalyticsStatus: Output status of analytics jobs
# ------------------------------------------------------------------------------------------------------------------
Function AnalyticsStatus
{
    Write-Output "Analytics Processing Job Status:"
    $analyticsStatus = Get-SPEnterpriseSearchStatus -SearchApplication $global:ssa -JobStatus

    foreach ($analyticsEntry in $analyticsStatus)
    {
        if ($analyticsEntry.Name -ne "Not available")     
        {
            foreach ($de in ($analyticsEntry.Details))
            {
                if ($de.Key -eq "Status")
                {
                    $status = $de.Value
                }
            }
            Write-Output "    $($analyticsEntry.Name) : $status"
        }
        # Output additional diagnostics from the dictionary
        foreach ($de in ($analyticsEntry.Details))
        {
            # Skip entries that is listed as Not Available
            if ( ($de.Value -ne "Not available") -and ($de.Key -ne "Activity") -and ($de.Key -ne "Status") )
            {
                Write-Output "        $($de.Key): $($de.Value)"
                if ($de.Key -match "Last successful start time")
                {
                    $dLast = Get-Date $de.Value
                    $dNow = Get-Date
                    $daysSinceLastSuccess = $dNow.DayOfYear - $dLast.DayOfYear
                    if ($daysSinceLastSuccess -gt 3)
                    {
                        Write-Output "        Warning: More than three days since last successful run"
                        $global:serviceDegraded = $true                        
                    }
                }
            }
        }
    }
    Write-Output ""
}

# ------------------------------------------------------------------------------------------------------------------
# SearchComponentStatus: Analyze the component status for one component
# ------------------------------------------------------------------------------------------------------------------
Function SearchComponentStatus($component)
{
    # Find host name
    foreach($searchComp in ($global:topologyCompList))
    {
        if ($searchComp.Name -eq $component.Name)
        {
            if ($searchComp.ServerName)
            {
                $hostName = $searchComp.ServerName
            }
            else
            {
                $hostName = "No server associated with this component. The server may have been removed from the farm."
            }
        }
    }
    if ($component.State -ne "Active")
    {
        # String with all components that is not active:
        if ($component.State -eq "Unknown")
        {
            $global:unknownComponents += "$($component.Name):$($component.State)"
        }
        elseif ($component.State -eq "Degraded")
        {
            $global:degradedComponents += "$($component.Name):$($component.State)"
        }
        else
        {
            $global:failedComponents += "$($component.Name):$($component.State)"
        }
        $global:serviceDegraded = $true
    }
    
    # Skip unnecessary info about cells and partitions if everything is fine
    $outputEntry = $true
    
    # Indent the cell info, logically belongs to the component. 
    if ($component.Name -match "Cell")
    {
        if ($component.State -eq "Active")
        {
            $outputEntry = $false
        }
        else
        {
            Write-Output "    $($component.Name)"
        }
    }
    elseif ($component.Name -match "Partition")
    {
        if ($component.State -eq "Active")
        {
            $outputEntry = $false
        }
        else
        {
            Write-Output "Index $($component.Name)"
        }
    }
    else
    {
        # State for search components
        $primaryString = ""
        if ($component.Name -match "Query") { $entity = "QueryProcessingComponent" }
        elseif ($component.Name -match "Content") { $entity = "ContentProcessingComponent" }
        elseif ($component.Name -match "Analytics") { $entity = "AnalyticsProcessingComponent" }
        elseif ($component.Name -match "Crawl") { $entity = "CrawlComponent" }
        elseif ($component.Name -match "Admin") 
        { 
            $entity = "AdminComponent" 
            if ($global:haTopology)
            {
                if ($component.Name -eq $global:primaryAdmin)
                {
                    $primaryString = " (Primary)"
                }
            }
        }
        elseif ($component.Name -match "Index") 
        { 
            $entity = "IndexComponent"
            foreach ($searchComp in ($global:topologyCompList))
            {
                if ($searchComp.Name -eq $component.Name) 
                {
                    $partition = $searchComp.IndexPartitionOrdinal
                }
            }
            # find info about primary role
            foreach ($de in ($component.Details))
            {
                if ($de.Key -eq "Primary")
                {
                    if ($de.Value -eq "True")
                    {
                        $primaryString = " (Primary)"
                        foreach ($haEntity in ($global:haArray))
                        {
                            if (($haEntity.entity -eq $entity) -and ($haEntity.partition -eq $partition))
                            {
                                $haEntity.primary = $component.Name

                            }
                        }                        
                    }
                }
            }
        }
        foreach ($haEntity in ($global:haArray))
        {
            if ( ($haEntity.entity -eq $entity) -and ($component.State -eq "Active") )
            {
                if ($entity -eq "IndexComponent")
                {
                    if ($haEntity.partition -eq $partition)
                    {
                        $haEntity.componentsOk += 1
                    }
                }
                else 
                { 
                    $haEntity.componentsOk += 1
                }
            }
        }
        # Add the component entities to $global:compArray for output formatting
        $compTemp = $global:compTemplate | Select-Object *
        $compTemp.Component = "$($component.Name)$primaryString"
        $compTemp.Server = $hostName
        $compTemp.State = $component.State
        if ($partition -ne -1) 
        { 
            $compTemp.Partition = $partition 
        }
        $global:compArray += $compTemp

        if ($component.State -eq "Active")
        {
            $outputEntry = $false
        }
        else
        {
            Write-Output "$($component.Name)"
        }
    }
    if ($outputEntry)
    {
        if ($component.State)
        {
            Write-Output "    State: $($component.State)"
        }
        if ($hostName)
        {
            Write-Output "    Server: $hostName"
        }
        if ($component.Message)
        {
            Write-Output "    Details: $($component.Message)"
        }
    
        # Output additional diagnostics from the dictionary
        foreach ($de in ($component.Details))
        {
            if ($de.Key -ne "Host")
            {
                Write-Output "    $($de.Key): $($de.Value)"
            }
        }
        if ($global:haTopology)
        {
            if ($component.Name -eq $global:primaryAdmin)
            {
                Write-Output "    Primary: True"            
            }
            elseif ($component.Name -match "Admin")
            {
                Write-Output "    Primary: False"            
            }
        }
    }
}

# ------------------------------------------------------------------------------------------------------------------
# DetailedIndexerDiag: Output selected info from detailed component diag
# ------------------------------------------------------------------------------------------------------------------
Function DetailedIndexerDiag
{
    $indexerInfo = @()
    $generationInfo = @()
    $generation = 0

    foreach ($searchComp in ($global:componentStateList))
    {
        $component = $searchComp.Name
        if ( (($component -match "Index") -or ($component -match "Content") -or ($component -match "Admin")) -and ($component -notmatch "Cell") -and ($searchComp.State -notmatch "Unknown") -and ($searchComp.State -notmatch "Registering"))
        {
            $pl=Get-SPEnterpriseSearchStatus -SearchApplication $global:ssa -HealthReport -Component $component
            foreach ($entry in ($pl))
            {
                if ($entry.Name -match "plugin: number of documents") 
                { 
                    foreach ($haEntity in ($global:haArray))
                    {
                        if (($haEntity.entity -eq "IndexComponent") -and ($haEntity.primary -eq $component))
                        {
                            # Count indexed documents from all index partitions:
                            $global:indexedDocs += $entry.Message
                            $haEntity.docs = $entry.Message
                        }
                    }
                }
                if ($entry.Name -match "repartition")
                    { $indexerInfo += "Index re-partitioning state: $($entry.Message)" }
                elseif (($entry.Name -match "splitting") -and ($entry.Name -match "fusion")) 
                    { $indexerInfo += "$component : Splitting index partition (appr. $($entry.Message) % finished)" }
                elseif (($entry.Name -match "master merge running") -and ($entry.Message -match "true")) 
                { 
                    $indexerInfo += "$component : Index Master Merge (de-fragment index files) in progress" 
                    $global:masterMerge = $true
                }
                elseif ($global:degradedComponents -and ($entry.Name -match "plugin: newest generation id"))
                {
                    # If at least one index component is left behind, we want to output the generation number.  
                    $generationInfo += "$component : Index generation: $($entry.Message)" 
                    $gen = [int] $entry.Message
                    if ($generation -and ($generation -ne $gen))
                    {
                        # Verify if there are different generation IDs for the indexers
                        $global:generationDifference = $true
                    }
                    $generation = $gen
                }
                elseif (($entry.Level -eq "Error") -or ($entry.Level -eq "Warning"))
                {
                    $global:serviceDegraded = $true
                    if ($entry.Name -match "fastserver")
                        { $indexerInfo += "$component ($($entry.Level)) : Indexer plugin error ($($entry.Name):$($entry.Message))" }
                    elseif ($entry.Message -match "fragments")
                        { $indexerInfo += "$component ($($entry.Level)) : Missing index partition" }
                    elseif (($entry.Name -match "active") -and ($entry.Message -match "not active"))
                        { $indexerInfo += "$component ($($entry.Level)) : Indexer generation controller is not running. Potential reason: All index partitions are not available" }
                    elseif ( ($entry.Name -match "in_sync") -or ($entry.Name -match "left_behind") )
                    { 
                        # Indicates replicas are out of sync, catching up. Redundant info in this script
                        $global:indexLeftBehind = $true
                    }                
                    elseif ($entry.Name -match "full_queue")
                        { $indexerInfo += "$component : Items queuing up in feeding ($($entry.Message))" }                                
                    elseif ($entry.Message -notmatch "No primary")
                    {
                        $indexerInfo += "$component ($($entry.Level)) : $($entry.Name):$($entry.Message)"
                    }
                }
            }
        }
    } 

    if ($indexerInfo)
    {
        Write-Output ""
        Write-Output "Indexer related additional status information:"
        foreach ($indexerInfoEntry in ($indexerInfo))
        {        
            Write-Output "    $indexerInfoEntry"
        }
        if ($global:indexLeftBehind -and $global:generationDifference)
        {
            # Output generation number for indexers in case any of them have been reported as left behind, and reported generation IDs are different.
            foreach ($generationInfoEntry in ($generationInfo))
            {        
                Write-Output "    $generationInfoEntry"
            }
        }
        Write-Output ""
    }
}

# ------------------------------------------------------------------------------------------------------------------
# VerifyHaLimits: Verify HA status for topology and index size limits
# ------------------------------------------------------------------------------------------------------------------
Function VerifyHaLimits
{
    $hacl = @()
    $haNotOk = $false
    $ixcwl = @()
    $ixcel = @()
    $docsExceeded = $false
    $docsHigh = $false
    foreach ($hac in $global:haArray)
    {
        if ([int] $hac.componentsOk -lt 2)
        {
            if ([int] $hac.componentsOk -eq 0)
            {
                # Service is down
                $global:serviceFailed = $true
                $haNotOk = $true   
            }
            elseif ($global:haTopology)
            {
                # Only relevant to output if we have a HA topology in the first place
                $haNotOk = $true   
            }

            if ($hac.partition -ne -1)
            {
                $hacl += "$($hac.componentsOk)($($hac.components)) : Index partition $($hac.partition)"
            }
            else
            {
                $hacl += "$($hac.componentsOk)($($hac.components)) : $($hac.entity)"
            }
        }
        if ([int] $hac.docs -gt 10000000)
        {
            $docsExceeded = $true 
            $ixcel += "$($hac.entity) (partition $($hac.partition)): $($hac.docs)"
        }
        elseif ([int] $hac.docs -gt 9000000)
        {
            $docsHigh = $true   
            $ixcwl += "$($hac.entity) (partition $($hac.partition)): $($hac.docs)"
        }
    }
    if ($haNotOk)
    {
        $hacl = $hacl | sort
        if ($global:serviceFailed)
        {
            Write-Output "Critical: Service down due to components not active:"
        }
        else
        {
            Write-Output "Warning: No High Availability for one or more components:"
        }
        foreach ($hc in $hacl)
        {
            Write-Output "    $hc"
        }
        Write-Output ""
    }
    if ($docsExceeded)
    {
        $global:serviceDegraded = $true
        Write-Output "Warning: One or more index component exceeds document limit:"
        foreach ($hc in $ixcel)
        {
            Write-Output "    $hc"
        }
        Write-Output ""
    }
    if ($docsHigh)
    {
        Write-Output "Warning: One or more index component is close to document limit:"
        foreach ($hc in $ixcwl)
        {
            Write-Output "    $hc"
        }
        Write-Output ""
    }
}

# ------------------------------------------------------------------------------------------------------------------
# VerifyHostControllerRepository: Verify that Host Controller HA (for dictionary repository) is OK
# ------------------------------------------------------------------------------------------------------------------
Function VerifyHostControllerRepository
{
    $highestRepVer = 0
    $hostControllers = 0
    $primaryRepVer = -1
    $hcStat = @()
    $hcs = Get-SPEnterpriseSearchHostController
    foreach ($hc in $hcs)
    {
        $hostControllers += 1
        $repVer = $hc.RepositoryVersion
        $serverName = $hc.Server.Name
        if ($repVer -gt $highestRepVer)
        {
            $highestRepVer = $repVer
        }
        if ($hc.PrimaryHostController)
        {
            $primaryHC = $serverName
            $primaryRepVer = $repVer
        }
        if ($repVer -ne -1)
        {
            $hcStat += "        $serverName : $repVer"
        }
    }
    if ($hostControllers -gt 1)
    {
        Write-Output "Primary search host controller (for dictionary repository): $primaryHC"
        if ($primaryRepVer -eq -1)
        {
            $global:serviceDegraded = $true
            Write-Output "Warning: Primary host controller is not available."
            Write-Output "    Recommended action: Restart server or set new primary host controller using Set-SPEnterpriseSearchPrimaryHostController."
            Write-Output "    Repository version for existing host controllers:"
            foreach ($hcs in $hcStat)
            {
                Write-Output $hcs
            }
        }
        elseif ($primaryRepVer -lt $highestRepVer)
        {
            $global:serviceDegraded = $true
            Write-Output "Warning: Primary host controller does not have the latest repository version."
            Write-Output "    Primary host controller repository version: $primaryRepVer"
            Write-Output "    Latest repository version: $highestRepVer"
            Write-Output "    Recommended action: Set new primary host controller using Set-SPEnterpriseSearchPrimaryHostController."
            Write-Output "    Repository version for existing host controllers:"
            foreach ($hcs in $hcStat)
            {
                Write-Output $hcs
            }
        }
        Write-Output ""
    }
}

#---added by bspender--------------------------------------------------------------------------------------------------
# VerifyApplicationServerSyncJobsEnabled: Verify that Application Server Admin Service Timer Jobs are running
# ---------------------------------------------------------------------------------------------------------------------
function VerifyRunningProcesses
{
	$components = $SSA.ActiveTopology.GetComponents() | Sort ServerName | SELECT ServerName, Name

	foreach ($hostname in $global:hostArray.Hostname) {
	    Write-OutPut ("---[$hostname]---") -ForegroundColor Cyan

	    Write-OutPut ("Components deployed to this server...") 
	    $crawler = $components | Where {($_.Servername -ieq $hostname) -and ($_.Name -match "Crawl") } 
	    if ($crawler -ne $null) {
	        Write-OutPut ("    " + $crawler.Name + ":") -ForegroundColor White
	        $mssearch = (Get-Process mssearch -ComputerName $hostname -ErrorAction SilentlyContinue)
	        Write-OutPut ("        " + $mssearch.ProcessName + "[PID: " + $mssearch.Id + "]")
	        $mssdmn = (Get-Process mssdmn -ComputerName $hostname -ErrorAction SilentlyContinue)
	        $mssdmn | ForEach {
	            Write-OutPut ("        " + $_.ProcessName + "[PID: " + $_.Id + "]")
	        }
	    }

	    $junoComponents = $components | Where {($_.Servername -ieq $hostname) -and ($_.Name -notMatch "Crawl") }     
	    $noderunnerProcesses = (Get-Process noderunner -ComputerName $hostname -ErrorAction SilentlyContinue)

	    foreach ($node in $noderunnerProcesses) {
	        $node | Add-Member -Force -MemberType NoteProperty -Name _ProcessCommandLine -Value $(
			    (Get-WmiObject Win32_Process -ComputerName $hostname -Filter $("processId=" + $node.id)).CommandLine
		    )

	        $junoComponents | Where {$_.Servername -ieq $hostname} | ForEach {
	            $component = $($_).Name
	            if ($node._ProcessCommandLine -like $("*" + $component + "*")) {
	                Write-OutPut ("    " + $component + ":") -ForegroundColor White
	                Write-OutPut ("        " + $node.ProcessName + "[PID: " + $node.Id + "]")
	            }
	        }
	    }

	    #if this is a custom object, wrap it in an array object so we can get a count in the step below
	    if ($junoComponents -is [PSCustomObject]) { $junoComponents = @($junoComponents) } 

	    if ($junoComponents.Count  -gt $noderunnerProcesses.Count) {
	        Write-OutPut ("One or more noderunner processes is not running for components") -ForegroundColor Yellow 
	    }

	    Write-OutPut
	    $services = Get-Service -ComputerName $hostname -Name SPTimerV4, SPAdminV4, OSearch15, SPSearchHostController 
	    $running = $services | Where {$_.Status -eq "Running"}
	    if ($running) {
	        Write-OutPut ("Service Instances...") -ForegroundColor Green
	        $running | ft -AutoSize
	    }
	    $stopped = $services | Where {$_.Status -eq "Stopped"}
	    if ($stopped) {
	        Write-OutPut ("`"Stopped`" Services...") -ForegroundColor Red
	        $stopped | ft -AutoSize
	    }
	    $other   = $services | Where {($_.Status -ne "Running") -and ($_.Status -ne "Stopped")}
	    if ($other) {
	        Write-OutPut ("Service in an abnormal or transient state...") -ForegroundColor Yellow
	        $other | ft -AutoSize
	    }

	}
}

#---added by bspender--------------------------------------------------------------------------------------------------
# VerifyApplicationServerSyncJobsEnabled: Verify that Application Server Admin Service Timer Jobs are running
# ---------------------------------------------------------------------------------------------------------------------
function VerifyApplicationServerSyncJobsEnabled
{
	$timeThresholdInMin = 5
	
	$sspJob = $((Get-SPFarm).Services | where {$_.TypeName -like "SSP Job Control*"})
	if ($sspJob.Status -ne "Online") { 
		Write-Warning ("SSP Job Control Service is " + $sspJob.Status)
		$global:serviceDegraded = $true
	}

	$serverNames = $((Get-SPFarm).Servers | Where {$_.Role -eq "Application"}).Name
	foreach ($server in $serverNames) {
		$sspJobServiceInstance = $((Get-SPFarm).Servers[$server].ServiceInstances | where {$_.TypeName -like "SSP Job Control*"})
		if ($sspJobServiceInstance.Status -ne "Online") { 
			Write-Warning ("SSP Job Control Service Instance is " + $sspJobServiceInstance.Status + " on " + $server)
			$global:SSPJobInstancesOffline.Add($sspJobServiceInstance) | Out-Null
			$global:serviceDegraded = $true
		} 
	}

	if ($serverNames.count -eq 1) {
		$jobs = Get-SPTimerJob | where {$_.Name -like "job-application-*"}
	} else {
		$jobs = Get-SPTimerJob | where {$_.Name -eq "job-application-server-admin-service"}
	}

	foreach ($j in $jobs) { 
		Write-OutPut ($j.Name)
		Write-OutPut ("-------------------------------------------------------")
		if (($j.Status -ne "Online") -or ($j.isDisabled)) { 
			if ($j.Status -ne "Online") { Write-Warning ($j.Name + " timer job is " + $j.Status) }
			if ($j.isDisabled) { Write-Warning ($j.Name + " timer job is DISABLED") }
			$global:ApplicationServerSyncTimerJobsOffline.Add($j) | Out-Null 
			$global:serviceDegraded = $true
		} else {
			$mostRecent = $j.HistoryEntries | select -first ($serverNames.count * $timeThresholdInMin) 
			foreach ($server in $serverNames) { 
				$displayShorthand = $server+": "+$($j.Name)
				$mostRecentOnServer = $mostRecent | Where {$_.ServerName -ieq $server} | SELECT -First 1
				if ($mostRecentOnServer -eq $null) {
					Write-Warning ($displayShorthand + " timer job does not appear to be running")
					#and add this server to the list
					$global:ApplicationServerSyncNotRunning.Add($displayShorthand) | Out-Null
					$global:serviceDegraded = $true
				} else {
					$spanSinceLastRun = [int]$(New-TimeSpan $mostRecentOnServer.EndTime $(Get-Date).ToUniversalTime()).TotalSeconds
					if ($spanSinceLastRun -lt ($timeThresholdInMin * 60)) {
						Write-OutPut ($displayShorthand + " recently ran " + $spanSinceLastRun + " seconds ago")
					} else {
						Write-Warning ($displayShorthand + " last ran " + $spanSinceLastRun + " seconds ago")
						$global:ApplicationServerSyncNotRunning.Add($displayShorthand) | Out-Null
						$global:serviceDegraded = $true
					}
					#(For added verbosity, uncomment the following line to report the last successful run for this server) 
					#$mostRecentOnServer		
				}		
			}	
		}
	}
}



# ------------------------------------------------------------------------------------------------------------------
# Main
# ------------------------------------------------------------------------------------------------------------------

Write-Output ""
Write-Output "###############################################"
Write-Output "Search Topology health check"
Write-Output "###############################################"
Write-Output ""
# ------------------------------------------------------------------------------------------------------------------
# Global variables:
# ------------------------------------------------------------------------------------------------------------------

$global:serviceDegraded = $false
$global:serviceFailed = $false
$global:unknownComponents = @()
$global:degradedComponents = @()
$global:failedComponents = @()
$global:generationDifference = $false
$global:indexLeftBehind = $false
$global:searchHosts = 0
$global:ssa = $null
$global:componentStateList = $null
$global:topologyCompList = $null
$global:haTopology = $false
$global:primaryAdmin = $null
$global:indexedDocs = 0
$global:masterMerge = $false

#---added by bspender------------------------
$global:SSPJobInstancesOffline = $(New-Object System.Collections.ArrayList)
$global:ApplicationServerSyncTimerJobsOffline = $(New-Object System.Collections.ArrayList)
$global:ApplicationServerSyncNotRunning = $(New-Object System.Collections.ArrayList)
#--------------------------------------------
$global:UnreachableSearchServiceSvc = $(New-Object System.Collections.ArrayList)
$global:UnreachableSearchAdminSvc = $(New-Object System.Collections.ArrayList)
#--------------------------------------------

# Template object for the host array:
$global:hostTemplate = New-Object psobject
$global:hostTemplate | Add-Member -MemberType NoteProperty -Name hostName -Value $null
$global:hostTemplate | Add-Member -MemberType NoteProperty -Name components -Value 0
$global:hostTemplate | Add-Member -MemberType NoteProperty -Name cpc -Value $null
$global:hostTemplate | Add-Member -MemberType NoteProperty -Name qpc -Value $null
$global:hostTemplate | Add-Member -MemberType NoteProperty -Name pAdmin -Value $null
$global:hostTemplate | Add-Member -MemberType NoteProperty -Name sAdmin -Value $null
$global:hostTemplate | Add-Member -MemberType NoteProperty -Name apc -Value $null
$global:hostTemplate | Add-Member -MemberType NoteProperty -Name crawler -Value $null
$global:hostTemplate | Add-Member -MemberType NoteProperty -Name index -Value $null

# Create the empty host array:
$global:hostArray = @()

# Template object for the HA group array:
$global:haTemplate = New-Object psobject
$global:haTemplate | Add-Member -MemberType NoteProperty -Name entity -Value $null
$global:haTemplate | Add-Member -MemberType NoteProperty -Name partition -Value -1
$global:haTemplate | Add-Member -MemberType NoteProperty -Name primary -Value $null
$global:haTemplate | Add-Member -MemberType NoteProperty -Name docs -Value 0
$global:haTemplate | Add-Member -MemberType NoteProperty -Name components -Value 0
$global:haTemplate | Add-Member -MemberType NoteProperty -Name componentsOk -Value 0

# Create the empty HA group array:
$global:haArray = @()

# Template object for the component/server table:
$global:compTemplate = New-Object psobject
$global:compTemplate | Add-Member -MemberType NoteProperty -Name Component -Value $null
$global:compTemplate | Add-Member -MemberType NoteProperty -Name Server -Value $null
$global:compTemplate | Add-Member -MemberType NoteProperty -Name Partition -Value $null
$global:compTemplate | Add-Member -MemberType NoteProperty -Name State -Value $null

# Create the empty component/server table:
$global:compArray = @()

# Get the SSA object and print SSA name:
GetSSA

# Get basic topology info and component health status
GetTopologyInfo

#---added by bspender------------------------
VerifyRunningProcesses
VerifyApplicationServerSyncJobsEnabled


# Traverse list of components, determine properties and update $global:hostArray / $global:haArray
foreach ($searchComp in ($global:topologyCompList))
{
    PopulateHostHaList($searchComp)
}

# Analyze the component status:
foreach ($component in ($global:componentStateList))
{
    SearchComponentStatus($component)
}

# Look for selected info from detailed indexer diagnostics:
DetailedIndexerDiag

# Output list of components with state OK:
if ($global:compArray)
{
    $global:compArray | Sort-Object -Property Component | Format-Table -AutoSize
}
Write-Output ""

# Verify HA status for topology and index size limits:
VerifyHaLimits

# Verify that Host Controller HA (for dictionary repository) is OK:
VerifyHostControllerRepository

# Output components by server (for servers with multiple search components):
if ($global:haTopology -and ($global:searchHosts -gt 2))
{
    $componentsByServer = $false
    foreach ($hostInfo in $global:hostArray)
    {
        if ([int] $hostInfo.components -gt 1)
        {
            $componentsByServer = $true
        }
    }
    if ($componentsByServer)
    {
        Write-Output "Servers with multiple search components:"
        foreach ($hostInfo in $global:hostArray)
        {
            if ([int] $hostInfo.components -gt 1)
            {
                Write-Output "    $($hostInfo.hostName): $($hostInfo.pAdmin)$($hostInfo.sAdmin)$($hostInfo.index)$($hostInfo.qpc)$($hostInfo.cpc)$($hostInfo.apc)$($hostInfo.crawler)"
            }
        }
        Write-Output ""
    }
}

# Analytics Processing Job Status:
AnalyticsStatus

if ($global:masterMerge)
{
    Write-Output "Index Master Merge (de-fragment index files) in progress on one or more index components."
}

if ($global:serviceFailed -eq $false)
{
    Write-Output "Searchable items: $global:indexedDocs"
}

GetCrawlStatus
Write-Output ""
    
if ($global:unknownComponents)
{
    Write-Output "The following components are not reachable:"
    foreach ($uc in ($global:unknownComponents))
    {
        Write-Output "    $uc"
    }
    Write-Output "Recommended action: Restart or replace the associated server(s)"
    Write-Output ""
}

if ($global:degradedComponents)
{
    Write-Output "The following components are degraded:"
    foreach ($dc in ($global:degradedComponents))
    {
        Write-Output "    $dc"
    }
    Write-Output "Recommended action for degraded components:"
    Write-Output "    Component registering or resolving:"
    Write-Output "        This is normally a transient state during component restart or re-configuration. Re-run the script."

    if ($global:indexLeftBehind)
    {
        Write-Output "    Index component left behind:"
        if ($global:generationDifference)
        {
            Write-Output "        This is normal after adding an index component or index component/server recovery."
            Write-Output "        Indicates that the replica is being updated from the primary replica."
        }
        else
        {
            Write-Output "        Index replicas listed as degraded but index generation is OK."
            Write-Output "        Will get out of degraded state as soon as new/changed items are being idexed."
        }
    }
    Write-Output ""
}

if ($global:failedComponents)
{
    Write-Output "The following components are reported in error:"
    foreach ($fc in ($global:failedComponents))
    {
        Write-Output "    $fc"
    }
    Write-Output "Recommended action: Restart the associated server(s)"
    Write-Output ""
}

if ($global:serviceFailed)
{
    Write-OutPut -BackgroundColor Red -ForegroundColor Black "Search service overall state: Failed (no queries served)"
}
elseif ($global:serviceDegraded)
{
    Write-OutPut "Search service overall state: Degraded"
}
else
{
    Write-OutPut "Search service overall state: OK"
}
Write-Output ""
}
#################################################################################################################



GetSSA
GetFarmBuild | Out-File $outputfile -Append
"" | Out-File $outputfile  -Append
Get-Date | Out-File $outputfile -Append
GetServersInFarm |  ft -auto | Out-File $outputfile -Append
GetServiceInstances | Out-File $outputfile -Append
GetSSAFullObject | Out-File $outputfile -Append
GetSSALegacyAdminComponent | Out-File $outputfile -Append
GetSearchTopo | Out-File $outputfile -Append
GetContentSources | Out-File $outputfile -Append
GetServerNameMappings | Out-File $outputfile -Append
GetCrawlRules | fl | Out-File $outputfile -Append
displayGlobalSearchService | Out-File $outputfile -Append
GetSQSS | Out-File $outputfile -Append
VerifyServiceEndpoints | select ResponseUri, Description | ft -auto -HideTableHeaders | Out-File $outputfile -Append
GetSSIs | Out-File $outputfile -Append
GetAAMs | Out-File $outputfile -Append
HealthCheck | Out-File $outputfile -Append



2. Try below



$aud = Get-SPUsageDefinition | where {$_.Name -like "Analytics*"} 
$aud |fl




3. Check below command :



$a = Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.AnalyticsJobDefinition
$sa = $a.GetAnalysis("Microsoft.Office.Server.Search.Analytics.SearchAnalyticsJob")
$sa.StartAnalysis()


We Got below error :


We need to correct the link store Input :


4. Delete old corrupted below timer jobs :


Analytics Timer Job for Search Service Application Usage Analytics Timer Job for Search Application



$tj1= Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.AnalyticsJobDefinition
$tj1.delete()

$tj2= Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.UsageAnalyticsJobDefinition
$tj2.delete()



5. Recreate both deleted timer jobs again



$ssa = Get-SPEnterpriseSearchServiceApplication
$assembly = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.Office.Server.Search")
$bindingflags = [System.Reflection.BindingFlags]::NonPublic, [System.Reflection.BindingFlags]::Static
======================
$jobtypes = @("Microsoft.Office.Server.Search.Analytics.UsageAnalyticsJobDefinition","Microsoft.Office.Server.Search.Analytics.AnalyticsJobDefinition")
# Register analytics timer jobs
foreach ($jobtype in $jobtypes)
{
$type = $assembly.GetType($jobtype)
$registerJob = $type.GetMethod("RegisterJob", $bindingflags)
$registerJob.Invoke($type, $ssa)
}


6. Check running schedule details of both new timer jobs:



$tj1= Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.AnalyticsJobDefinition
$tj2= Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.UsageAnalyticsJobDefinition

Its showing garbage values, as below



7. Run it again with below commands



$tj1= Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.AnalyticsJobDefinition
$tj1.RunNow()
--------------------------------------
$tj2= Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.UsageAnalyticsJobDefinition
$tj2.RunNow()



8. Now check new corrected running schedule:



$tj1= Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.AnalyticsJobDefinition
$tj2= Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.UsageAnalyticsJobDefinition



After 24hours reports got generated as expected :


512 views0 comments
Post: Blog2 Post
bottom of page