Building GDAL on Linux: Difference between revisions

Latest revision as of 06:48, 7 February 2019

See also my newer PostGIS page. I have GDAL build instructions there, too.

WARNING: 06 Feb 2019 these instructions to build from git break right now because they updated to using PROJ 6 and this build uses PROJ 5 I will fix this soon.

Using a Docker version

There are various builds on the Docker Hub, for example, https://hub.docker.com/r/geodata/gdal It's a quick way to try out GDAL without having to install anything on your computer.

To download,

docker pull geodata/gdal

For version information,

docker run geodata/gdal

This one says it's GDAL 2.3.0dev. I am interested in vector support for PDF at the moment. The output says it has "PDF -raster,vector- (rw+vs): Geospatial PDF"

To test it with one of my files on the local hard drive,

docker run -v $(pwd):/data geodata/gdal gdalinfo project.pdf

docker run -v $(pwd):/data geodata/gdal ogrinfo project.pdf INFO: Open of `project.pdf'

     using driver `PDF' successful.

Metadata:

 CREATION_DATE=D:20190110101503+05
 CREATOR=Esri ArcMap 10.3.0.4284
 NEATLINE=POLYGON ((74.24014 23.54477,74.24014 30.73475,87.49497 30.73475,87.49497 23.54477,74.24014 23.54477))

1: Layers_Other 2: Layers_Export_Output 3: Layers_ganges_river_t47 4: Layers_state

Now I want to pull some data out of the PDF and put it into another vector format, a shapefile for example:

docker run -v $(pwd):/data geodata/gdal ogr2ogr -f "ESRI Shapefile" ganges.shp project.pdf Layers_ganges_river_t47

List all the layers in the PDF

gdalinfo CityMasterPlan.pdf -mdd LAYERS

This is not working yet...

ogr2ogr -f "ESRI Shapefile" buildings.shp CityMasterPlan.pdf -oo GDAL_PDF_LAYERS="LAYER_037_NAME=ELU_PD_02_A1_pdf.Layers.2"

Build prerequisites

sudo apt-get install subversion git
sudo apt-get install swig python-dev
sudo apt-get install cmake-qt-gui make bison flex gcc g++ libtool
sudo apt-get install lib1g-dev libcurl4-gnutls-dev liblcms2-dev liblcms2-utils

sudo apt-get install libpng12-dev libtiff4-dev
sudo apt-get install libspatialite-dev

You might also need, depending on how you install PostGIS

sudo apt-get install libpq-dev

OGDI support for VMAP

http://www.gdal.org/ogr/drv_ogdi.html

sudo apt-get install ogdi-bin libogdi3.2-dev

JPEG2000

Options for JPEG2000 support include

LizardTech MrSID SDK (--with-jp2mrsid) (costs money to use encoder)
Jasper (--with-jasper)
Kakadu (commercial license "if you have to ask...") http://www.kakadusoftware.com/
OpenJPEG (--with-openjpeg)

Which is best?? Choose one

OpenJPEG

sudo apt-get -y install libopenjpeg-dev openjpeg-tools

Jasper

You want the Jasper from the remotesensing site, not the official one. It has support for something called GeoJP2

ftp://ftp.remotesensing.org/pub/gdal/

Building gdal on my 64-bit Debian system required this

cd jasper src dir
export CFLAGS="-fPIC -m64"
./configure
make; sudo make install
unset CFLAGS
sudo ldconfig

Non-free support

ESRI file geodatabases

NOTE 1: You won't be able to access FGDBs created before ArcGIS 10.0
NOTE 2: To get ESRI FGDB support requires gdal 1.9.0 or better

I want to be able to script loading data from file geodatabases into postgis, as described in the GDAL page. Refer to the GDAL FGDB driver page

Agree to their terms and then download the file gdb API from the ESRI site.

Here you will find build instructions.

I put the lib*so files in /usr/local/lib

cd FileGDB_API
export LD_LIBRARY_PATH=`pwd`/lib
cd samples
make
sudo cp ../lib/* /usr/local/lib
sudo ldconfig

Lizardtech MrSID raster files

Go to the Lizardtech developer site, sign up, and download the DSDK. Unpack it in a convenient place then tell gdal configure where to find it. You don't need to install anything unless you want to use their utilities.

2019-Jan-24 GDAL build

I generally build GDAL from sources so that I can add the non-standard parts such as MRSID support. (Regarding Geospatial PDF, see http://www.gdal.org/frmt_pdf.html -- there are links to further documentation and sample GeoPDF's at the bottom of this page.)

GEOS

libgeos is not optional. I prefer to build the latest as it is updated far more often than the repostory version.

When I had the free commercial MrSID tools installed, I had some trouble because the GDAL build kept seeing the bundled libraries. MrSID uses GEOS, too, but an outdated version. Make sure your environment is not set to expose the old library to the whole system.

wget http://download.osgeo.org/geos/geos-3.7.1.tar.bz2
tar xjf geos*bz2
cd geos*
./configure --enable-python
make -j 8
sudo make install
cd ..

Supporting actors

You can install these if you need support for other less common formats.

PDF support

Poppler is for PDF support, see also PoDoFo

# UBUNTU and current Debian -- First install required prerequisite packages
sudo apt-get -y install libproj-dev libpoppler-dev libpoppler-private-dev

MrSID support

I just tell GDAL where to find the library in the MrSID directory.

Spatialite

sudo apt-get install libsqlite3-dev
wget http://www.gaia-gis.it/gaia-sins/libspatialite-sources/libspatialite-4.3.0.tar.gz
tar xzvf libspatialite*gz
cd libspatialite-*
./configure --enable-freexl=no
make -j 8
sudo make install

Building GDAL (finally!)

I grab the source code directly from the github repository.

 git clone https://github.com/OSGeo/gdal

I put together a script called CONFIG.gdal with one of the following configure lines in it.

On good days I use this, it has ESRI and MrSID installed.

./configure --with-python --with-poppler=yes \
--with-geos --with-geotiff --with-jpeg --with-png --with-expat --with-libkml --with-xerces-c \
--with-mrsid=$HOME/src/GIS/MrSID/Raster_DSDK \
--with-fgdb=$HOME/src/GIS/FileGDB_API \
--with-openjpeg --with-geos --with-pg \
--with-curl \
--with-spatialite \
--with-expat

On bad days 03-04-13 it won't build right now so I cut a bunch of unneeded stuff out and finally got it to build with this. I don't know exactly why it won't build with all the formats above but I don't really care enough to investigate because what I am doing is trying to get to the point where I can build QGIS from source. I know ogdi and openjpeg were causing problems for me. I had to set to "no" to stop them from being included. Leaving it off the list was not enough.

./configure --with-python --with-poppler=no \
--with-geos --with-geotiff --with-jpeg --with-png \
--with-mrsid=$HOME/src/GIS/MrSID/Raster_DSDK --with-fgdb=$HOME/src/GIS/FileGDB_API \
--with-openjpeg=no --with-pg --with-ogdi=no --with-curl

OR on my public server I use this, where I don't need as much support for different file formats

./configure --with-python --with-poppler=yes \
--with-openjpeg --with-geos \
--with-ogdi

OR at Alsea Geospatial I used this

./configure --with-python --with-poppler --with-pg --with-curl --with-openjpeg \
 --with-geos --with-geotiff --with-jpeg --with-png --with-expat --with-libkml --with-xerces-c \
 --with-mrsid=$HOME/src/GIS/MrSID/Raster_DSDK \
 --with-java=/opt/oracle/jdk1.7.0_09 --with-fgdb=$HOME/src/GIS/FileGDB_API \
 --with-mdb

After configuring successfully (all the formats you need say YES) then do the build and install. Take a break, it takes a long time even giving it 10 CPU's.

make -j 10
sudo make install

Does it work? Quick test

% gdalinfo 1501ANC4815_geo.pdf 
Driver: PDF/Geospatial PDF
Files: 1501ANC4815_geo.pdf
Size is 4470, 3375
Coordinate System is:
PROJCS["UTM Zone 48, Northern Hemisphere",
   GEOGCS["Indian 1960",
       DATUM["Indian_1960",
           SPHEROID["Everest 1830 (1937 Adjustment)",6377276.345,300.8017,
               AUTHORITY["EPSG","7015"]],
           TOWGS84[198,881,317,0,0,0,0],
           AUTHORITY["EPSG","6131"]],
       PRIMEM["Greenwich",0,
           AUTHORITY["EPSG","8901"]],
       UNIT["degree",0.0174532925199433,
           AUTHORITY["EPSG","9122"]],
       AUTHORITY["EPSG","4131"]],
   PROJECTION["Transverse_Mercator"],
   PARAMETER["latitude_of_origin",0],
   PARAMETER["central_meridian",105],
   PARAMETER["scale_factor",0.9996],
   PARAMETER["false_easting",500000],
   PARAMETER["false_northing",0],
   UNIT["Meter",1]]
GeoTransform =
 429999.053008, 42.34642896, 0.04051872
 999725.914982, 0.04124303999999999, -42.332688
Metadata:
 NEATLINE=POLYGON ((445035.80158801417565 999277.556482305750251,444993.235997625160962 991902.349403560161591,444864.738057852664497 884201.712978136376478,452159.580137342331 884208.817744017927907,585964.54743435990531 884289.527591207530349,614300.221590773784555 884383.269830735283904,614190.235319046885706 999293.477391553344205,605175.140901734353974 999218.552309990860522,553978.789515793323517 999300.979716640315019,505958.457250406034291 999353.427943945163861,451172.635888226970565 999382.750743734533899,445035.722449889173731 999360.237513555795886,445035.80158801417565 999277.556482305750251,445035.80158801417565  999277.556482305750251,445035.80158801417565 999277.556482305750251))
Corner Coordinates:
Upper Left  (  429999.053,  999725.915) (104d21'46.72"E,  9d 2'39.27"N)
Lower Left  (  430135.804,  856853.093) (104d21'58.76"E,  7d45' 6.78"N)
Upper Right (  619287.590,  999910.271) (106d 5' 7.80"E,  9d 2'41.48"N)
Lower Right (  619424.341,  857037.449) (106d 4'59.36"E,  7d45' 9.52"N)
Center      (  524711.697,  928381.682) (105d13'28.20"E,  8d23'57.68"N)
Band 1 Block=4470x1 Type=Byte, ColorInterp=Red
Band 2 Block=4470x1 Type=Byte, ColorInterp=Green
Band 3 Block=4470x1 Type=Byte, ColorInterp=Blue

JOY! It works!

To see if it built all required formats, for real, you can check:

% gdalinfo --formats
Supported Formats:
 VRT (rw+v): Virtual Raster
 GTiff (rw+v): GeoTIFF
 NITF (rw+v): National Imagery Transmission Format
 RPFTOC (rov): Raster Product Format TOC format
 HFA (rw+v): Erdas Imagine Images (.img)
 SAR_CEOS (rov): CEOS SAR Image
 CEOS (rov): CEOS Image
 JAXAPALSAR (ro): JAXA PALSAR Product Reader (Level 1.1/1.5)
 GFF (rov): Ground-based SAR Applications Testbed File Format (.gff)
 ELAS (rw+): ELAS
 AIG (rov): Arc/Info Binary Grid
 AAIGrid (rwv): Arc/Info ASCII Grid
 SDTS (rov): SDTS Raster
 DTED (rwv): DTED Elevation Raster
 PNG (rwv): Portable Network Graphics
 JPEG (rwv): JPEG JFIF
 MEM (rw+): In Memory Raster
 JDEM (ro): Japanese DEM (.mem)
 GIF (rwv): Graphics Interchange Format (.gif)
 BIGGIF (rov): Graphics Interchange Format (.gif)
 ESAT (ro): Envisat Image Format
 BSB (rov): Maptech BSB Nautical Charts
 XPM (rwv): X11 PixMap Format
 BMP (rw+v): MS Windows Device Independent Bitmap
 DIMAP (rov): SPOT DIMAP
 AirSAR (ro): AirSAR Polarimetric Image
 RS2 (ro): RadarSat 2 XML Product
 PCIDSK (rw+v): PCIDSK Database File
 PCRaster (rw): PCRaster Raster File
 ILWIS (rw+v): ILWIS Raster Map
 SGI (rw+): SGI Image File Format 1.0
 SRTMHGT (rwv): SRTMHGT File Format
 Leveller (rw+): Leveller heightfield
 Terragen (rw+): Terragen heightfield
 ISIS3 (rov): USGS Astrogeology ISIS cube (Version 3)
 ISIS2 (rov): USGS Astrogeology ISIS cube (Version 2)
 PDS (rov): NASA Planetary Data System
 TIL (ro): EarthWatch .TIL
 ERS (rw+): ERMapper .ers Labelled
 JPEG2000 (rwv): JPEG-2000 part 1 (ISO/IEC 15444-1)
 L1B (ro): NOAA Polar Orbiter Level 1b Data Set
 FIT (rwv): FIT Image
 GRIB (rov): GRIdded Binary (.grb)
 MrSID (ro): Multi-resolution Seamless Image Database (MrSID)
 JP2MrSID (ro): MrSID JPEG2000
 RMF (rw+): Raster Matrix Format
 WCS (ro): OGC Web Coverage Service
 WMS (ro): OGC Web Map Service
 MSGN (ro): EUMETSAT Archive native (.nat)
 RST (rw+): Idrisi Raster A.1
 INGR (rw+v): Intergraph Raster
 GSAG (rw): Golden Software ASCII Grid (.grd)
 GSBG (rw+): Golden Software Binary Grid (.grd)
 GS7BG (ro): Golden Software 7 Binary Grid (.grd)
 COSAR (ro): COSAR Annotated Binary Matrix (TerraSAR-X)
 TSX (ro): TerraSAR-X Product
 COASP (ro): DRDC COASP SAR Processor Raster
 R (rwv): R Object Data Store
 PNM (rw+): Portable Pixmap Format (netpbm)
 DOQ1 (ro): USGS DOQ (Old Style)
 DOQ2 (ro): USGS DOQ (New Style)
 ENVI (rw+v): ENVI .hdr Labelled
 EHdr (rw+v): ESRI .hdr Labelled
 GenBin (ro): Generic Binary (.hdr Labelled)
 PAux (rw+): PCI .aux Labelled
 MFF (rw+): Vexcel MFF Raster
 MFF2 (rw+): Vexcel MFF2 (HKV) Raster
 FujiBAS (ro): Fuji BAS Scanner Image
 GSC (ro): GSC Geogrid
 FAST (ro): EOSAT FAST Format
 BT (rw+): VTP .bt (Binary Terrain) 1.3 Format
 LAN (rov): Erdas .LAN/.GIS
 CPG (ro): Convair PolGASP
 IDA (rw+): Image Data and Analysis
 NDF (ro): NLAPS Data Format
 EIR (rov): Erdas Imagine Raw
 DIPEx (ro): DIPEx
 LCP (rov): FARSITE v.4 Landscape File (.lcp)
 GTX (rw+v): NOAA Vertical Datum .GTX
 LOSLAS (rov): NADCON .los/.las Datum Grid Shift
 NTv2 (rw+v): NTv2 Datum Grid Shift
 RIK (ro): Swedish Grid RIK (.rik)
 USGSDEM (rw): USGS Optional ASCII DEM (and CDED)
 GXF (ro): GeoSoft Grid Exchange Format
 HTTP (ro): HTTP Fetching Wrapper
 NWT_GRD (ro): Northwood Numeric Grid Format .grd/.tab
 NWT_GRC (ro): Northwood Classified Grid Format .grc/.tab
 ADRG (rw+v): ARC Digitized Raster Graphics
 SRP (rov): Standard Raster Product (ASRP/USRP)
 BLX (rw): Magellan topo (.blx)
 Rasterlite (rw): Rasterlite
 PostGISRaster (ro): PostGIS Raster driver
 SAGA (rw+v): SAGA GIS Binary Grid (.sdat)
 KMLSUPEROVERLAY (rwv): Kml Super Overlay
 XYZ (rwv): ASCII Gridded XYZ
 HF2 (rwv): HF2/HFZ heightfield raster
 OZI (rov): OZI

...all the right stuff!

Now back to the real work of Mapping Vietnam or building Xastir.