Commitlogs overload
Hi All
Our production environment is running in a load balanced scenario on two Virtual servers:
- Microsoft Windows Server 2012 R2 Standard
- 4 core 2.27 GHz processors
- 16 GB Ram
- GXE 4.3.1
For some reason, usually after an unexpected restart, the second server in our cluster keeps filling up with commitlog files from 'Program Files\Latitude Geographics\Geocortex Core\Data\SecurityStore\PRODUCTION\data\commitlog'. I tried to open the file to review the contents, but is just garbled text and make no sense.
It creates a new 32mb file every 10 secs or so and keeps going til it fills the hard drive then the server fails. This has happened on a couple of occasions and usually if I stop both servers and restart them in again in the order on which the software was installed the commitlogs stop. However after a department system restart over the weekend the server is filling up again with these files, and my usual workaround its working this time. as a result, I have had to create a scheduled task to delte the contents of the folder above every 5 minutes.
Our single server Dev and QA servers don't/haven't experienced this problem.
I dont know why or what creates these log files and cant find anyway to stop them.
I have logged a request with out local reseller, but I am hoping someone in these forums may have some ideas???
Cheers
Chris
-
Hello Chris,
Thank you for informing us about the issue you are having.There have been a few other reports of this issue and we have provided a fix for this issue which will be available in the next Essentials release. We have placed a limit on the number of logs that can appear in this location. The files themselves are actually a type of cached information by Cassandra's database, which explains why they can't be opened directly.
Thanks,
Stefan0 -
Thanks for that Stefan
In the interim, how do I stop the log files being created? As I mentioned they are being created continuosly at the moment, and I have had to create a scheduled task to delete the content every 5 minutes. I tried a restart of the servers but it didnt do anything.
Cheers
Chris
0 -
Hello Chris,
The log files being created continuously indicates that there's likely issues occuring with your SecurityStore role configuration, which is difficult to troubleshoot without more information about your system's security configuration, and probably a GCXInfo trace.
There may be some useful information in the Geocortex Core/Data/Logs directory, specifically the SecurityStore.log file.
Thanks,
Stefan0 -
I've got the same problem...here's what my log files say....
[2015-09-30 09:16:29,022][ERROR][Role: SecurityStore InstanceID: 304c2593-c479-48a4-9160-ba0b2243606f][SecurityStore][Platform.Roles.SecurityStore.SecurityStoreRole] Error unregistering role: _connection should not be null.
Parameter name: _connection
System.ArgumentNullException: _connection should not be null.
Parameter name: _connection
at CuttingEdge.Conditions.RequiresValidator`1.ThrowExceptionCore(String condition, String additionalMessage, ConstraintViolationType type)
at CuttingEdge.Conditions.ValidatorExtensions.IsNotNull[T](ConditionValidator`1 validator)
at Geocortex.Platform.Configuration.ConfigurationService.DeleteConfiguration(IEnumerable`1 configurationItems)
at Geocortex.Platform.Role.RoleBase.Stop()
[2015-09-30 09:16:55,488][INFO ][1 ][SecurityStore
Any idea on what is wrong?0 -
Our problem is still persisting. Additonally to this I had discovered that there are also 132kb .so files being created in Windows\Temp every 10 secs or so. I am hoping that the latest version of GXE will solve this and am just waiting on a support call from our local reseller.
Incidently my log file just shows hundreds of line as below
10-08 04:52:08,389][INFO ][25 ][SecurityStore][Platform.Roles.SecurityStore.SecurityStoreRole] Looking for peers finished.
[2015-10-08 04:52:13,515][INFO ][25 ][SecurityStore][Platform.Roles.SecurityStore.SecurityStoreRole] This Cassandra node seems to be completely ready for operation (advertising it).
[2015-10-08 04:52:14,030][INFO ][27 ][SecurityStore][Platform.Roles.SecurityStore.SecurityStoreRole] Looking in RoleRegistry for peers...
[2015-10-08 04:52:14,030][INFO ][27 ][SecurityStore][Platform.Roles.SecurityStore.SecurityStoreRole] Looking for peers finished.
[2015-10-08 04:52:19,093][INFO ][27 ][SecurityStore][Platform.Roles.SecurityStore.SecurityStoreRole] This Cassandra node seems to be completely ready for operation (advertising it).
[2015-10-08 04:52:19,608][INFO ][38 ][SecurityStore][Platform.Roles.SecurityStore.SecurityStoreRole] Looking in RoleRegistry for peers...
[2015-10-08 04:52:19,608][INFO ][38 ][SecurityStore][Platform.Roles.SecurityStore.SecurityStoreRole] Looking for peers finished.0 -
Just ran into same problem. I can't update to the latest version, just times out when trying to connect to the Geocortex Service during the post installation configuration and the Essentials Manager continues to not function.
Anyone get a solution to this? We are not functioning at all right now, so I will absolutely be calling in to support.0 -
Tech support has a solution, worked for me anyway! The root problem is resolved with the latest update to 4.2.2 but even after updating the Security Store portion of the Geocortex service was still crashing because it had 212,000 tiny bad files spread throughout the directory in addition to the 60gb of data in the commitlogs sub-directory as 32mb files. We renamed (or deleted) the bad SecurityStore directory and relaunched the post installer after the update. Everything worked, a new SecurityStore directory was created, post installer ran and the site was back up.
The only caveat, you will lose any security information you had, logins and such. We did not have any, so it wasn't an issue, but there may be another solution if you have a lot of info to lose.0
Bitte melden Sie sich an, um einen Kommentar zu hinterlassen.
Kommentare
7 Kommentare