Georeferencing of IP-Addresses

This chapter explains the functionality of the Python-Script which georeferences IP-Addresses out of the statistic data provided by the WebOffice application.

The Script is using the statistics which are generated by the WebOffice application. Those include the IP-Addresses of every user that called the application, from which the coordinates can be estimated. The in the process generated CSV-File can be used for further analysis. It is possible for example to generate a heat map of the access locations using ArcGIS Online or ArcGIS Pro.

Note: The estimation of the coordinates to the IP-Addresses is no concern of data protection because the data is still anonymous and no individual is identified.

Note: To configured user tracking (saving of personal data) use the application configuration in the WebOffice author standalone.

 

System requirements

All requirements are met if ArcMap is already installed. If not Python 2.7 as well as the Python-Libraries numpy and pandas need to be installed.

 

Program use

 

First of all the time span for the analysis needs to be identified.

The following entries are possible:

• D => Analysis for a day
• M => Analysis for a month
• Q => Analysis for a quarter
• Y => Analysis for a year

The required entry which is required next will differ depending of the chosen time span. The next step was performed for the analysis of a day (Entry => "D")

 

 

The entered date needs to be confirmed with "Y". If the wrong date was entered the script can be started over by typing "N" instead.

 

 

Next, the path for the statistics folder needs to be entered. If the data is located at the recommended default path which is C:\Tomcat\webapps\WebOffice\WEB-INF\work\statistics, only "D" for default needs to be entered. Otherwise the absolute path to the statistics folder needs to be entered. Furthermore, the output file will be named after the WebOffice application.

 

 

If the entered path is correct, the statistics data will be imported. This can take up to a few minutes depending on the time span chosen for the analysis. The progress can be viewed in the command prompt. After the importing process is finished, the total number of the found entries are displayed. Now the filtering process can be carried out. If there is no project name entered before pressing enter, all projects will be included in the analysis.

The following entries are possible:

• Projectname
• ProjectnameA,ProjektnameB,ProjectnameC, ...
• No entry (all projects will be analysed)

 

 

After generating a definite IP-List, the total number of all entries are displayed. Now there is the possibility to locate the remaining IP-Addresses. For that there simply needs to be entered a "Y" for the next query.

 

 

The locating of the IP-Addresses can take from a few minutes up to an hour. Due to technical limitations only 150 IP-Addresses per minute can be located. After a package of 150 IP-Addresses was calculated, there is a one minute break before the next package can be estimated.

The CSV-File which was created during the process can now be further processed with ESRI products.

 

Inputfiles

As mentioned before, the path of the statistics folder is requested by the script. The default path therefore is C:\Tomcat\webapps\WebOffice\WEB-INF\work\statistics.

 

Outputfiles

The now described sub folders are stored at the same location where the script is located.

 

Output

All output files will be stored in the the output folder.

The naming is as follows:

• Day: <Name of wO-Application>_YYMMDD
• Month: <Name of wO-Application>_YYMM
• Quarter: <Name of wO-Application>_YY_Q
• Year: <Name of wO-Application>_YY

 

Log

The sub folder log contains the log file and the processed IP-Lists without Geocoding.

The naming is as follows:

• Day: <Name of wO-Application>_LOG_YYMMDD
• Month: <Name of wO-Application>_LOG_YYMM
• Quarter: <Name of wO-Application>_LOG_YY_Q
• Year: <Name of wO-Application>_LOG_YY

 

 

Further processing of the CSV-Files

After creating the IP-Address list with the corresponding coordinates, it is now possible to visualize that data in form of a map. Therefore various options are feasible which will be shown in the following sub chapters.

 

ArcGIS Online

Using ArcGIS Online it is possible with just a few steps to create a heat map. For this purpose ArcGIS Online needs to be opened in a Webbrowser and a new map must be created by clicking on Map.

 

 

 The CSV-File can be inserted via drag&drop in the opened map view.

 

 

After the data is integrated in ArcGIS Online, different drawing styles can be chosen. When choosing the drawing style Heat Map and clicking Done, a heat map from the provided data will be created. That map can now be shared online and edited further.

 

 

 

ArcGIS Pro

Using ArcGIS Pro also gives the possibility to edit the CSV-File further. First of all a Folder Connection to the CSV-File needs to be created. After that the file can be dragged from the Catalog right into Contents. Clicking right on the inserted file opens a context menu.

 

 

Choosing Display XY Data... opens a dialog field in the functional area for creating a feature class.

 

 

If the desired coordinate system is chosen, the feature class can be created by clicking Run.

To create a heat map out of the generated feature class, the feature class needs to be selected first in the Contents area.

 

 

Now it is possible to edit the display of the signatures in the Symbology tab which can be found in the function area.

 

 

Clicking on Single Symbol opens a drop down menu where the entry Heat Map can be chosen.

 

 

The now created heat map is now ready to be published and shared on ArcGIS Online.

 

 

 

ArcMap 10.x

ArcMap also holds the possibility to edit the CSV-Data further. First of all a Folder Connection needs to be established to access the CSV-File. Once displayed in the Catalog the file can be dragged in the Table of Contents.

Clicking right on the inserted file opens a context menu.

 

 

Clicking on Display XY Data... opens a dialog window to create a layer.

 

 

A coordinate system needs to be selected if there is none already preselected. This can be done by clicking on the button Edit...

 

 

The choice of the desired coordinate system is confirmed by clicking OK.

 

 

The field description should now show information regarding the chosen coordinate system. If that is the case, the dialog can be affirmed by clicking OK which creates the layer.

The resulting layer is now ready to be further edited.