This chapter explains the functionality of the Python-Script which georeferences IP-Addresses out of the statistic data provided by the WebOffice application.
The Script is using the statistics which are generated by the WebOffice application. Those include the IP-Addresses of every user that called the application, from which the coordinates can be estimated. The in the process generated CSV-File can be used for further analysis. It is possible for example to generate a heat map of the access locations using ArcGIS Online or ArcGIS Pro.
Note: The estimation of the coordinates to the IP-Addresses is no concern of data protection because the data is still anonymous and no individual is identified.
Note: To configured user tracking (saving of personal data) use the application configuration in the WebOffice author standalone.
All requirements are met if ArcMap is already installed. If not Python 2.7 as well as the Python-Libraries numpy and pandas need to be installed.
Pick a date for the analysis
First of all the time span for the analysis needs to be identified.
The following entries are possible:
- D => Analysis for a day
- M => Analysis for a month
- Q => Analysis for a quarter
- Y => Analysis for a year
The required entry which is required next will differ depending of the chosen time span. The next step was performed for the analysis of a day (Entry => "D")
Date entry and folder selection
The entered date needs to be confirmed with "Y". If the wrong date was entered the script can be started over by typing "N" instead.
Next, the path for the statistics folder needs to be entered. If the data is located at the recommended default path which is C:\Tomcat\webapps\WebOffice\WEB-INF\work\statistics, only "D" for default needs to be entered. Otherwise the absolute path to the statistics folder needs to be entered. Furthermore, the output file will be named after the WebOffice application.
Reading of the statistic data
If the entered path is correct, the statistics data will be imported. This can take up to a few minutes depending on the time span chosen for the analysis. The progress can be viewed in the command prompt. After the importing process is finished, the total number of the found entries are displayed. Now the filtering process can be carried out. If there is no project name entered before pressing enter, all projects will be included in the analysis.
The following entries are possible:
- ProjectnameA,ProjektnameB,ProjectnameC, ...
- No entry (all projects will be analysed)
Locating of the IP-Addresses
After generating a definite IP-List, the total number of all entries are displayed. Now there is the possibility to locate the remaining IP-Addresses. For that there simply needs to be entered a "Y" for the next query.
Locating of the IP-Addresses done
The locating of the IP-Addresses can take from a few minutes up to an hour. Due to technical limitations only 150 IP-Addresses per minute can be located. After a package of 150 IP-Addresses was calculated, there is a one minute break before the next package can be estimated.
The CSV-File which was created during the process can now be further processed with ESRI products.
As mentioned before, the path of the statistics folder is requested by the script. The default path therefore is C:\Tomcat\webapps\WebOffice\WEB-INF\work\statistics.
The now described sub folders are stored at the same location where the script is located.
All output files will be stored in the the output folder.
The naming is as follows:
- Day: <Name of wO-Application>_YYMMDD
- Month: <Name of wO-Application>_YYMM
- Quarter: <Name of wO-Application>_YY_Q
- Year: <Name of wO-Application>_YY
The sub folder log contains the log file and the processed IP-Lists without Geocoding.
The naming is as follows:
- Day: <Name of wO-Application>_LOG_YYMMDD
- Month: <Name of wO-Application>_LOG_YYMM
- Quarter: <Name of wO-Application>_LOG_YY_Q
- Year: <Name of wO-Application>_LOG_YY
Further processing of the CSV-Files
After creating the IP-Address list with the corresponding coordinates, it is now possible to visualize that data in form of a map. Therefore various options are feasible which will be shown in the following sub chapters.
Using ArcGIS Online it is possible with just a few steps to create a heat map. For this purpose ArcGIS Online needs to be opened in a Webbrowser and a new map must be created by clicking on Map.
Create a new map
The CSV-File can be inserted via drag&drop in the opened map view.
Drag&Drop of the CSV-File
After the data is integrated in ArcGIS Online, different drawing styles can be chosen. When choosing the drawing style Heat Map and clicking Done, a heat map from the provided data will be created. That map can now be shared online and edited further.
Using ArcGIS Pro also gives the possibility to edit the CSV-File further. First of all a Folder Connection to the CSV-File needs to be created. After that the file can be dragged from the Catalog right into Contents. Clicking right on the inserted file opens a context menu.
Display XY Data in ArcGIS Pro
Choosing Display XY Data... opens a dialog field in the functional area for creating a feature class.
Create feature class
If the desired coordinate system is chosen, the feature class can be created by clicking Run.
To create a heat map out of the generated feature class, the feature class needs to be selected first in the Contents area.
Select feature class
Now it is possible to edit the display of the signatures in the Symbology tab which can be found in the function area.
Display of signatures
Clicking on Single Symbol opens a drop down menu where the entry Heat Map can be chosen.
Symbology Heat Map
The now created heat map is now ready to be published and shared on ArcGIS Online.
Heat Map in ArcGIS Pro
ArcMap also holds the possibility to edit the CSV-Data further. First of all a Folder Connection needs to be established to access the CSV-File. Once displayed in the Catalog the file can be dragged in the Table of Contents.
Clicking right on the inserted file opens a context menu.
Display XY Data in ArcMap
Clicking on Display XY Data... opens a dialog window to create a layer.
A coordinate system needs to be selected if there is none already preselected. This can be done by clicking on the button Edit...
Select coordinate system
The choice of the desired coordinate system is confirmed by clicking OK.
Create Layer with desired coordinate system
The field description should now show information regarding the chosen coordinate system. If that is the case, the dialog can be affirmed by clicking OK which creates the layer.
The resulting layer is now ready to be further edited.