Arctic Logging
I just re-read that title line and realized people will probably come here thinking this has something to do with high latitudes and trees.
Arctic is currently a random collection of ideas focused on management of GIS services.
I am evaluating server log processing tools as a subproject for inclusion in Arctic. This page will have notes on what I discover.
As usual on this web site, what follows are notes I wrote while exploring. Don't expect a tutorial or a complete copy of information available already in books or other web sites. Just expect notes.
What I know
Logging is complex on Esri because the logs are split up. The Esri logs tell about service requests, but don't have information about the requestor. So you can't tell if a batch of failed login attempts are from a friend who can't type or a foe attempting a break-in.
I know there are probably 100 different approaches to log analysis. I know many involve pulling logs into a database and then querying that and that's what I am looking for.
Logs I need to think about
- IIS
- Portal
- Server
- Data Store
- License manager
- Mapproxy
- Reverse proxy ??
- Other microservices ??
- SQL Server ??
As with the entire Arctic project, I am aiming for small sites like the ones I work with. Rural county level.
Possible starting points
I am thinking of something based on a specialized database, not a general purpose SQL database. I am also thinking I have few constraints at this point because of our small size. I think I could set up a separate VM for it.
Last time I looked at this project (2/2021) I picked logstash and started building a docker for it.
Elastic stack - this option would scale beyond anything comprehensible by a human brain. See https://en.wikipedia.org/wiki/Elasticsearch
- Elastic (or likely OpenSearch) is the information retrieval engine
- logstash does the data collection and log parsing
- Beats does the shipping
- Kibana = visualization see OpenSearch Dashboards
nagios icinga Apache project?? Lucene / Solr
It might be too soon to lean heavily on OpenSearch. But naaaa it will be fine. I mean, I can put it in Docker Compose! They say that right up front!!
Setting up OpenSearch (and friends)
I suspect this will move to its own page sometime soon.
Remember to set up /etc/sysctl.d/40-opensearch.conf with vm.max_map_count=262144
Resources
Book available via ACM, "Logging and Log Management: The Authoritative Guide to Understanding the Concepts Surrounding Logging and Log Management" 2013 so it's dated but all the concepts are there. It's UN*X oriented.
