Geocortex Core: Remove corrupt transient indices

Background

Geocortex Essentials and Geocortex Analytics both depend on Geocortex Core for clustering and document storage.

Geocortex Core is a Windows service that runs sub-processes to perform various tasks necessary for the continued operation of Geocortex software.

Geocortex Core contains an internal database known as the Document Store that contains user and application data.

Problem

Geocortex Core can stop working if its document store data becomes corrupted due to several potential reasons, such as an unexpected outage or running out of disk space.

Symptoms

There are a few symptoms that may imply Geocortex Core is not working correctly. What we've identified:

• Geocortex Essentials:
  • Viewers fail to load
  • Essentials Manager shows an error on load: "Server Error in '/Geocortex/Essentials/Apps/RestManager' Application. - Runtime Error"
• Geocortex Analytics:
  
  • Analytics Reports shows an error on load: "There was an error loading Analytics Reports"
• Task Manager on the host server shows only a few Geocortex Core processes running rather than the normal ~12-14 Geocortex-named processes
• We see errors like these in the Geocortex Core logs contained in C:\Program Files\Latitude Geographics\Geocortex Core\Data\Logs in DocumentStore.log or BootStrapper.console:
  • Elasticsearch failed to recover shard 1 from index geocortex.search.messagequeue
  • Elasticsearch failed to recover shard 0 from index geocortex.htmlreporting.messagequeue
  • Failed to recover all Elasticsearch shards. Shutting down Elasticsearch. Contact Latitude support
  • Failed to delete Elasticsearch index: geocortex.internal.messagequeue. Code: 404. Message: no such index

Solution

Some data for the Document Store is transient (temporary) and it can be safely removed.  This is a quick operation that often will restore functionality without requiring additional investigation. Our recommended practice is to stop the Core service (on ALL nodes in the cluster), remove the transient index/indices, then restart Geocortex Core.

Details

The default path for Geocortex Core data is C:\Program Files\Latitude Geographics\Geocortex Core\Data\ - this could differ for your install. 

Within the Data directory there is a path to the Document Store indices, based on your cluster name. This path looks like C:\Program Files\Latitude Geographics\Geocortex Core\Data\DocumentStore\DocumentStore\(clustername)\nodes\0\indices

1. On the Geocortex server(s), stop the Geocortex Core service.
2. Open a File Explorer, browse to the Document Store indices folder
   e.g. C:\Program Files\Latitude Geographics\Geocortex Core\Data\DocumentStore\DocumentStore\(clustername)\nodes\0\indices
3. Delete the transient indices folders as listed below.
   • geocortex.core.configuration
   • geocortex.core.roles.documentstore.monitor
   • geocortex.core.roles.elasticsearch.watcher
   • geocortex.core.rolestatus
   • geocortex.internal.messagequeue
   • geocortex.search
   • geocortex.search.messagequeue
     Caution: Any folder not in the above list may contain user or application data and should not be removed
     Additional note: renaming the transient indices listed above and leaving them in the indices folder is not sufficient - they should be deleted.
4. On the Geocortex server(s), start the Geocortex Core service.
5. Wait 2-5 minutes for services to start back up, retest the initial issue.

If you need further help, please open a Support ticket and we'll be happy to assist.