Instant search set up - scanning takes too long
I recently upgraded our Geocortex Essential to version 4.0.2 on our test web server(windows 2008 R2-64bit). I downloaded the instant search module from http://blog.geocortex.com/2014/03/31/instant-search-is-here/ and installed the module on the same web server.
I tried to enable instant search for one of our geocortex site by following the geocortex essentials admin guilde. In Geocortex Essential Manager, edit site->choose the map service,edit service->click start button to start search index. The map service I chose to build search index only contains one feature layer with only 10 feature records. I started the scanning an hour ago, the scanning status is still showing 0%. See below attached screen shot. Based on the video http://support.geocortex.com/introducing-instant-search, it supposed to scan ~500,000 features per hour.
/customer/servlet/servlet.FileDownload?file=00P6000000eluFxEAI
I am wondering if I have missed any step on getting instant search to work?
Any help is appreciated.
Thank you and regards,
Jin
-
Hello Jin,
That small of a service should have been scanned in a minute or two, so it's clear that something is blocking the scanner.
First, could you verify that your service is not secured in any way? We do not yet support secured services.
Second, could you take a look at http://{ArcGIS-server-url}/robots.txt? If it exists and is blocking access to all crawlers (denoted by a *), you will need to update it to allow the "latbot" crawler. I can give details on that if needed.
Please let me know if either of those two suggestions solves the problem.
Chris
0 -
Hi Chris,
Thank you for your prompt response. The service I tried to build search index is under a secured folder on ArcGIS server site. So i published this map service to a non secured folder on ArcGIS Server site. The Scan is still showing 0 status after 5 mins.
what exactally would be the url to http://{ArcGIS-server-url}/robots.txt? If the REST URL to our ArcGIS server in test is http://XX.X.XX.22/arcgis/rest/services, will the url be http://XX.X.XX.22/robots.txt? I tried that and I got a 404 error saying File or directory does not found.
BTW, our ArcGIS server is running behind a proxy server.
Regards,
Jin
0 -
Assuming you added that new un-secured service to your Site and started indexing it (note that your secured service will never scan, so feel free to press the pause button on the Instant Search tab for that service), you might have to wait a little longer than 5 minutes for the features to be crawled. Is it complete yet?
You are correct in how you figured the url to the robots.txt file. The 404 is perfectly normal and indicates that your ArcGIS server imposes no restrictions to any crawlers.
If you still haven't gotten your un-secured service scanned fully, then there might be a problem more specific to your environment. If so, please don't hesitate to contact our support team. We would be happy to get you sorted out.
Chris
0 -
I'm having the same problem and my service is not secured. Its been 3 hours now. I searched for the robot.txt and couldnt find that file on my system.
Stephanie
0 -
Jin,
I'm glad your initial service scanned, and I hope that this new one has completed by now. While we don't advertise this fact in our documentation, please be aware that there can be a good 10-20 minutes of "lag time" before we're able to scan at full speed.
In answer to your question about the repeated GetStatus calls, this is perfectly expected. You might have noticed that they occur every 30 seconds. This is your Essentials instance requesting the status of the scan to update the Instant Search tab. If you exit that tab, you will find that the GetStatus calls will stop.
Stephanie, has your issue been resolved? If not, our support team and I would be happy to help you figure it out.
Chris
0 -
Thanks Chris for your reponse. Yes, the scan was completed in 1 hour.
0 -
I'm still having my issue. I ran it 4 hours today with zero progress.
Stephanie
0 -
I set up Instant Search a couple of weeks ago no problem. Last week I noticed that it wasn't finding features it should be finding (features that haven't changed, that it used to find). When I open the Instant Search tab for a map service in Manager, I get the following strange behaviors:
- I can't uncheck the Instant Search checkbox. However, if I hit Apply Settings, it get cleared. The Search Index progress bar says 0, and there is no garbage can icon to delete the service from the index
- If I click the scan button to create a search index, it briefly flashes "Processing". It stays at 0% (for several hours, on a small, unsecured service). The scan button remains - it doesn't change to a pause buton. A garbage can button appears, but clicking it doesn't seem to do anything.
Any ideas what might have happened? Is there a way of checking (maybe in Task Manager?) to see if it's actually scanning?
Thanks
Chris Dunlop
0 -
Stephanie, I can't tell what might be going on with the little bit of information you've provided. Could you give more details as to what you're tried?
Chris, that list of symptoms is odd, but I have a potential thought. Was the index, to your knowledge, ever deleted? Note that the index is shared across all your Essentials Sites; someone else in another Site could have deleted your index. Even if you don't think it happened, please look in the following file for the string "ElasticSearch successfully deleted all documents in type(s)" matching the service that's causing you problems:
C:\Program Files\Latitude Geographics\Geocortex Application Services 2\Data\Logs\FrontEnd.log
Was it deleted?
0 -
Hi Chris. No, there's no reference in the FrontEnd.log that shows the index was deleted.
Is there an easy way I can start fresh and try it again?
Chris D
0 -
Definitely, but not on a service-by-service basis. You can delete the Scanner folder (with the Geocortex Application Services 2 Service shut down) found at:
C:\Program Files\Latitude Geographics\Geocortex Application Services 2\Data\Scanner
This will delete all your scanned data and require that you start all your scans from scratch. By leaving the remainder of the folders in the Data directory, you will preserve your existing search index while you rebuild your indices.
0 -
Chris, I stopped the service, and deleted all the folders in the Data directory. Restarting the service re-created the folders. I'm now able to index again. I'll keep tabs on it and will let you know if I have any more problems.
Thanks!
Chris0 -
@Chris Hairfield,
When I attempt to generate the index it generates for hours with zero progress. Can you tell me what additional information I can provide to get this resolved? I am on the most current version of Geocortex Essentials and Silverlight Viewer to date. As there is very little documentation on this, I am not sure how to even start troubleshooting this. Suggestions would be appreciated.
Stephanie Patterson
0 -
Stephanie,
I understand your lack of insight into the product, and I certainly appreciate your patience. One thing you haven't mentioned is whether you have tried indexing any other services (and whether that was a success). Assuming that you haven't, would you mind retrying by following a similar approach as I mentioned to Chris? Turn off the Geocortex Application Services 2 service and delete all folders within the Data directory, found here:
C:\Program Files\Latitude Geographics\Geocortex Application Services 2\Data
Turn on the service and start scanning your map service anew from Essentials. Perhaps the second try is a charm?
Just to reiterate, you have already verified that your ArcGIS service is not secured in any way and that your browser returns a 404: Not Found when you navigate to http:\\{ArcGIS-Server-Url}\robots.txt.
If it still isn't working, a useful data point would be to test whether you're able to scan another service. Could you give that a try to see if the problem is isolated to this particular service or whether if affects your entire scanner? Next, could you let me know if you see a very large number of folders being created (perhaps every 5 minutes) in this folder:
C:\Program Files\Latitude Geographics\Geocortex Application Services 2\Data\Scanner\Nutch-1.0\crawl-onprem\segments
If you're not noticing hundreds of folders there and nothing above worked, could you please contact our support team for further help? There might be something specific to your environment preventing scanning from working.
Best of luck,
Chris
0
Bitte melden Sie sich an, um einen Kommentar zu hinterlassen.
Kommentare
14 Kommentare