PostGIS

Given: I have to be able to manage data in a SQL database server. I can't hack geoprocessing on the desktop any more. The datasets are too large.

Enterprise geodatabase server

The hardware

2 dual core Xeon processors,

12 GB of RAM

RAID stripe (for performance) with 3 drives

The software

All installed from packages unless noted otherwise

Debian (32 bit but should be upgraded to 64 bit soon)

postgresql-8.2  8.2.6-2
postgresql-8.3  8.3.5-1
postgresql-client-8.2   8.2.6-2
postgresql-client-8.3   8.3.7-1
postgresql-client-common        98
postgresql-common       94lenny1
postgresql-doc-8.2      8.2.6-2
postgresql-plpython-8.3 8.3.7-1

postgis 1.3.5-1

proj    4.6.1-1

libgdal1-1.4.0  1.4.4-1
libgdal1-1.5.0  1.5.4-2
libgdal1-dev    1.5.4-2

mapserver
python-mapscript
php5-mapscript

Server management

Mostly I use command line. I used to have webmin installed. For PostgreSQL I have phppgadmin installed

Internal link: http://dev.alseageo.com/phppgadmin/

Loading data into PostGIS

Lots of our data comes in the form of shapefiles, for this I can use

Can I use ArcGIS to do this??? Is this true? Ziggis? I don't want to get anywhere near SDE if I don't have to, so putting it on top of PostgreSQL is not what I want.

What about loading ArcGIS data using fwtools?

Use a geoprocessing Python script - read features using a cursor and load them into postgis

Exporting data from PostGIS

pgsql2shp

Resources

An article of interest

http://www.bostongis.com/PrinterFriendly.aspx?content_name=sqlserver2008_postgis_mysql_compare

Other

574 post(s)

1. 23-Mar-09 20:06

I suppose I could share a few things that I have found out through trial and error. I am certainly no expert.

I am running on windows server 2003 32 bit, so if you are running 64 bit or *nix versions, the experiences could be very different.

1. Make sure you have the latest version of postgres / postGIS. In another thread you mentioned that you have only just installed it, so I would think that this is the case. Version Postgres 8.3 has a nice easy windows installer that makes it a piece of cake to install. The application Stack Builder picks up the latest postGIS version, so there is no need to worry too much about installing postGIS seperately.

2. I found that installing the Tuning wizard that can be found in the Application Stack Builder helps to configure the memory setting quite well. For my purposes, I didn't need to tweak any of the memory setting into the postgresql.conf file.

3. If you are planning on doing spatial queries on any datasets that you export to postgres, then make sure you have spatial indexes on the geometry field. When you export from manifold using the Postgres type, then these indexes are created automatically. I would suggest reading the postgis user manual, it is pretty comprehensive, and fairly easy to follow. It will give you a good idea of what is possible with spatial queries. It is available http://postgis.refractions.net/download/postgis-1.3.5.pdf

4. The postGIS email list is very helpful. See http://postgis.refractions.net/mailman/listinfo/postgis-users to join.

5. If you are doing joins between tables in the database, then you will have to manually set up the indexes. If you are using pgAdmin III (which I like), then before you run queries, you can see if your indexes are being used by using explain.

--sql (for use within postgresql (or in the database manager console in manifold)

explain select * from test limit 1000;

This will tell you what type of scan is being done. In the above case it is always going to be a sequential scan, due to the nature of the query. If you are doing a query like:

select * from "PARCEL_MAPPING" INNER JOIN "100K index" ON st_intersects("PARCEL_MAPPING".geometry, "100K index".geometry)

where "100K index"."Sheet_Name" = 'DANYO'

There is an explain button in the query editor in pgAdmin III which shows this in a graphic form so you can see pretty quickly if you aren't taking advantage of your indexes.

6. Also as mentioned above, it is a good idea to do vaccuum analyzes often as these help indexes perform better.

7. If you are using large drawings that have lots of drawing objects, then make use of the AOI windowing when linking the drawings. Also see http://forum.manifold.net/forum/t64537.18 for a handy tool to help out.

8. Change the extents() aggregate function in postgres. Manifold uses this when linking drawings (for which reason I don't know). It is VERY slow on large datasets. If you use the tool mentioned above, you don't need the extents function at all. All I did was rename the function so I could still use it if I had to in code and postGIS, but Manifold wasn't automatically trying to use it.

--sql (in postgres)

DROP AGGREGATE extent(geometry);

CREATE AGGREGATE _extent(geometry) (

 SFUNC=public.st_combine_bbox,

 STYPE=box2d

);

That is about all I can think of for the moment, but I am sure others have more tips.

Hope this helps

James