GeoServer

From Wildsong
Jump to navigationJump to search

Overview

GeoServer is the Java based GIS web server that I am using with PostgreSQL and PostGIS for the Mapping Vietnam project

Home page: http://geoserver.org/

"GeoServer is a Java-based software server that allows users to view and edit geospatial data. Using open standards set forth by the Open Geospatial Consortium (OGC), GeoServer allows for great flexibility in map creation and data sharing."

The crucial feature for me is that it allows edits via WFS-T.

WFS-T comments: http://weblogs.java.net/blog/2006/01/09/geoserver-wfs-t

History

  • 2017 Sep -- Setting up Docker Compose version
  • 2016 Sep -- updated Dart with 2.9.1 (latest stable release)
  • 2012 Sep -- 2.2 installed on Kilchis and Dart.
  • 2011 Oct -- I am working with 2.1.2 now on HuPI
  • 2009 Feb 06 -- I decided to take a crack at installing GeoServer 1.7.2 today.

Example GeoServer site

The Portland TriMet system map is on GeoServer via the OpenGeo stack.

This is from the Portland Trimet site... GeoServer, OpenLayers, TileCache, PostgreSQL, PostGIS, GDAL, Extjs, Freemarker, Solr, Antlr, Balance Ant, Http Client, Tomcat, Apache, and CentOS are the underlying technologies used on this application. We're indebted to the contributors of these (and many other: eclipse, firefox and firebug to name a few) open source projects for sharing their efforts and expertise.

Some GeoServer Capabilities

  • WCS = Web Coverage Service = queryable feature data (OGC)
  • WFS = Web Feature Service = vector data (OGC)
  • WMS = Web Map Service = delivers rasters (not tiles) (OGC)
  • TMS = Tile map service = delivers raster tiles (OSGEO)
  • WMS-C = Web Map Service - Cached = WMS with tile caching (OSGEO draft)
  • WMTS = Web Map Tile service (OGC)

I think WCS can return features that are related from several feature classes whereas WFS returns features from one class only??

Map vs feature services

What if you want features to overlay a map but want symbology defined by the server? You define styles and then pick the correct encoding.

With WFS there are several GML flavours, GeoJSON, shapefiles and CSV. None of these work for me because they only encode the geography and not the visual properties. I probably want to use a WMS encoded with KML.

Lew's approach is to convert the KML into a custom JSON format on the server for transfer. There must be a better way than that. But it has to be compatible with the Google Maps Android API too.

The official way to put markers and shapes onto a Google map is via their API. I don't want to go the route of putting shapes onto the screen ignoring the API because I hate having to do all the projection and transformation stuff on my own.

Being able to directly read GPX files would be a good thing.

Tiles

If you are implementing a tile-based client you should probably use TMS or maybe WMTS. WMS-C was a OSGEO draft. WMTS is more complex so use TMS unless you really need WMTS features.

On the server side for performance and the best cartography you'd probably be better off pre-rendering and caching all the tiles. What's the best way to do that? Tilemill or gdal2tiles ? What's the best way to serve the prerendered tiles? TileCache? Or just use a web server (Apache)?

TMS reference implementation: see http://TileCache.org/

Status

On Dart I have data loaded into PostGIS and published it in GeoServer.

2017-09-19 Now I am setting it up in Docker.

Building the Geo-CEG Geoserver Docker images

When runnning both PostgreSQL/PostGIS and Geoserver inside Dockers, I use Compose. Normally I run a PostgreSQL server on the host though, not in a Docker.

Upgrade

I just updated the image from 2.13.0 to 2.13.1 and want to upgrade my installed image. Working from these instructions: https://www.jetbrains.com/help/hub/upgrade-with-docker.html 

docker stop geoserver
docker pull geoceg/geoserver:latest
docker run -t -p 8888:8080 --name geoserver geoceg/geoserver:latest

Geoserver

I created a docker image and put it on github, https://github.com/Geo-CEG/docker-geoserver and it autobuilds an image at https://hub.docker.com/r/geoceg/geoserver/ and I continue to refine that image as needed in this project.

I use it in Docker Compose but for testing it can be run like this 

docker run -t -p 8888:8080 --name geoserver geoceg/geoserver:latest

I am mapping it to port 8888 because a Docker running UniFi is using port 8080 already. I proxy that through nginx on Bellman so that it shows up as https://maps.wildsong.biz/geoserver/web/

Because it's tucked behind the proxy, it should be accessible everywhere.

Geonode

See http://geonode.org -- "GeoNode is a web-based application and platform for developing geospatial information systems (GIS) and for deploying spatial data infrastructures (SDI)."

Alternatively, using Docker Compose

To install Docker Compose, see https://docs.docker.com/compose/install/#install-compose for example, 

sudo -s
curl -L https://github.com/docker/compose/releases/download/1.16.1/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
docker-compose --version
docker-compose version 1.16.1, build 6d1ac21

Here is my Docker Compose file, so far. 

# Match this to your docker engine, I am at 17.06 in Debian 9
version: '3.3'

services:
  db:
    image: camptocamp/postgres:latest
    environment:
      POSTGRES_PASSWORD: example
      #POSTGRES_USER:
      #PGDATA:
      #POSTGRES_DB:
      #POSTGRES_INITDB_ARGS:
      #POSTGRES_INITDB_XLOGDIR:
    ports:
    - "5433:5432"
    volumes:
    - "/var/lib/postgresql/docker:/var/lib/postgresql"

    # I will want this behind an nginx reverse proxy instead of exposing the port

  geoserver:
    image: geoceg/geoserver:latest
    ports:
    - "8080:8080"

Fire it up

docker-compose up -d

For this to be accessible I have to punch through the firewall Aurora on ports 8080 and 5433.

Connect to it to test it psql -U postgres -h localhost

In this case I have mapped the standard postgres port to the same port on the host, I can get away with this because there is no instance of Postgres already running here. Then I can connect to it as if it were running locally with the psql command.

From the psql prompt, I can check out how the postgis set up went with 

\c geonode_imports
select postgis_full_version();
POSTGIS="2.3.1 r15264" GEOS="3.4.2-CAPI-1.8.2 r3921" PROJ="Rel. 4.8.0, 6 March 2012" GDAL="GDAL 1.10.1, released 2013/08/26" LIBXML="2.9.1" LIBJSON="0.11.99" RASTER

This looks reasonably complete and reasonably up to date. (Hmph, GDAL is a bit out of date, version 2 is out.)

You can run an interactive shell and look around. 

docker exec -it postgis /bin/bash

Plumbing the network

Create a network to hook together the geoserver and postgis containers 

docker network create pg
docker run --name postgis --net=pg -p 5432 -d wildsong/postgis
docker run --name geoserver --net=pg -p 8888:8080 -d wildsong/geoserver
docker network inspect pg

The two containers can see each other now with the names 'postgis' and 'geoserver' resolving correctly. The 'inspect' command shows the two containers are connected to the pg network.

Persisting the data

Add to the Dockerfile for PostGIS, at the end: 

ENV PGDATA /pgdata
VOLUME ${PGDATA}

The ENV statement will tell Postgres you want to keep all its files in a place called /pgdata. The VOLUME command allows you to tie that place "/pgdata" to a spot on your host computer when you run the container. Change the run command by added a mapping between the host and /pgdata/, for example, 

docker run --name postgis -v /home/myfavoriteplace/pgdata:/pgdata/ etc

will cause the files to be written to /home/myfavoriteplace/pgdata. This gives you the flexibility to use the same Docker image on different computers, keeping the data stored in the local filesystem, but in a different place on each computer.

Dealing with passwords

I don't want passwords to be coded into my Docker images, so how do I get them out of there? I know I am supposed to put them into environment settings but then how do I load the environment?

Getting a custom docker onto a Synology

I can do this by pushing my custom built Docker image into the Docker Hub and then pull it onto the Synology. I have set up both an organizational account and a personal account. I log in with my personal credentials and then push to the organizational account. 

docker login -u=brian32768 -p=paswword
docker push wildsong/postgis

Once the image is pushed then I should be able to pull it onto the Synology. Passing everything through a repository some 1000's of miles away might not work for you; you could set up your own registry. From the command line was easiest for me, 

sudo docker pull geoceg/postgis

I used the web interface to create a container and launch it once the image was copied to the server.

Setting up GeoServer on Linux

The downloadable binary for Linux includes jetty; this precludes the need for Tomcat. Indeed this is great for testing on a desktop but for any server application, I find that geoserver/java will eat ram and crash after a few days.

So I install Tomcat and expand the settings to give it lots of space.

Tomcat 8

I used to install Oracle manually (direct tarball download) and then was forced to install Tomcat from source, too. Otherwise doing "apt-get install tomcat7" got me an apt packaged outdated java.

Now I first set up the Oracle repo for Java and install it first so that everything can load from apt.

Fix for Debian 8.5, to get the add-apt-repository command (which is an Ubuntu thing) 

cd /tmp
wget http://blog.anantshri.info/content/uploads/2010/09/add-apt-repository.sh.txt
mv add-apt-repository.sh.txt /usr/local/sbin
chmod 755 /usr/local/sbin/add-apt-repository

Now I can install Oracle version 8 JDK as an apt package. 

apt-get install python-software-properties
add-apt-repository ppa:webupd8team/java
apt-get update
apt-get install oracle-java8-installer

Now since I have Oracle java installed as a package, I can install the tomcat8 package and it won't try to install whatever open sourced java it ordinarily installs. It will be happy to use Oracle's version.

Startup of tomcat fails at this point because JAVA_HOME is not set. They still use the old /etc/init.d /etc/default startup files, so I put my custom settings at the end of /etc/default/tomcat8: 

JAVA_HOME="/usr/lib/jvm/java-8-oracle"
GEOCEG_HOME="/home/geoceg"
GEOSERVER_HOME="$GEOCEG_HOME/geoserver"

JAVA_OPTS="-Djava.awt.headless=true -Xmx768m -Xrs -XX:PerfDataSamplingInterval=500 \          
 -XX:MaxPermSize=256m -Dorg.geotools.referencing.forceXY=true \                                       
 -DGEOSERVER_DATA_DIR=$GEOCEG_HOME/geoserver/data \                                                   
 -DGEOWEBCACHE_DATA_DIR=$GEOCEG_HOME/geowebcache \                                                    
 -DGEOEXPLORER_DATA=$GEOCEG_HOME/geoexplorer \                                                        
 "

Then start it up, fix any problems, try try again 

systemctl start tomcat8

At this point there should be a servlet container running at http://dart.wildsong.biz:8080/

How big is my heap now?

java -XX:+PrintFlagsFinal -version 2>&1 | grep -i MaxHeapSize

Geoserver 2.9.1

Download stable release of the WAR file from http://geoserver.org/ -- Today that would be here: 

cd ~/Downloads
wget http://downloads.sourceforge.net/project/geoserver/GeoServer/2.9.1/geoserver-2.9.1-war.zip?r=http%3A%2F%2Fgeoserver.org%2Frelease%2F2.9.1%2F&ts=1473614756&use_mirror=superb-sea2
unzip geoserver*

Based on the settings I used in /etc/default/tomcat8 above I need to install the WAR file at /var/lib/tomcat8/webapps/ and the rest of the files go in /home/geoceg Create the folders first, when you move the WAR file it will be deployed automatically and start searching for the folders. 

mkdir /home/geoceg
chdir /home/geoceg
mkdir -p geoexplorer geowebcache geoserver/data
chown -R tomcat8.wildsong /home/geoceg/*

mv geoserver.war /var/lib/tomcat8/webapps
chown root.root /var/lib/tomcat8/webapps/geoserver.war

systemctl restart tomcat8

If deployment succeeds then some files will be created in /home/geoceg/geoserver for you, and you should be able to see an empty geoserver at http://dart.wildsong.biz:8080/geoserver/

As you work on getting everything correct, the log file /var/lib/tomcat8/logging/catalina.out will be most helpful.


Change admin password now.

You have to look up the one that was created for you when geoserver.war ran the first time. It is in geoserver/data/security/masterpw.info Login as root in the browser and change it now.

Next I have to set up my own workspace to serve my PostGIS data.

Connecting GeoServer to PostGIS

Some notes on using WFS-T http://blogs.law.harvard.edu/jreyes/2007/08/03/geotools-wfs-t-update-request/

GeoServer has

  • workspaces aka namespaces
  • stores aka datastores
  • layers

These are about organizing the data accessible to GeoServer.

You store data in stores and a store has to be in a workspace. A PostGIS store connects to a database, and the tables therein become accessible as layers (once you publish them)

So for example, you can create a workspace for Oregon and then put all your state-level data into stores that are in the Oregon workspace.

OpenStreetMap

I have already set up the OpenStreetMap data for western US in PostGIS. There is a nice page to get started here: http://blog.geoserver.org/2009/01/30/geoserver-and-openstreetmap/

You will need SLD files, this one is uselessly outdated github.com/mitchelljj/openstreetmap-sld.git Trying https://github.com/ROGUE-JCTD/geoserver-osm-styles.git

Here is what I have so far, http://dart.wildsong.biz:8080/geoserver/osm/wms?service=WMS&version=1.1.0&request=GetMap&layers=osm:planet_osm_roads&styles=&bbox=-1.4011076E7,3631139.0,-1.1346024E7,6468468.0&width=721&height=768&srs=EPSG:3857&format=application/openlayers

Other PostGIS data

To create a connection to data in your PostGIS server you have to know what's in there. You can poke around with psql if you are text kind of guy or you can use pgAdmin III, like so:

Retrieved from "https://wiki.wildsong.biz//index.php?title=GeoServer&oldid=10978"