Coastal LiDAR: Difference between revisions

From Wildsong
Jump to navigationJump to search
Brian Wilson (talk | contribs)
mNo edit summary
Brian Wilson (talk | contribs)
mNo edit summary
Line 1: Line 1:
== Santa Clara county ==
For this project I want to closely examine a reach of road that goes into the hills to see if it can accommodate addition of a footpath.
I downloaded data that was collected by the Santa Clara Water District in 2006. The area of interest is small, covered by two tiles.
''What coordinate system is this data in?'' See https://coast.noaa.gov/htdata/lidar1_z/geoid12a/data/4870/
From metadata I think it's Cal III NAD83 and the vertical unit is in feet.
The data is in LAZ format.
First thing I want to do is generate simple first return and last return surfaces.
The files are already classified, see the metadata:
"This data set is an LAZ (compressed LAS) format file containing LIDAR point cloud data. LAS format files, raw LiDAR data in its native format, classified bare-earth LiDAR DEM  and photogrammetrically derived breaklines generated from LiDAR Intensity stereo-pairs.  Breakline, Top of Bank, and contour files in ESRI personal geodatabase format, Microstation V8 .dgn format, and AutoCAD 2004 formats for the San Jose Phase 3 project of Santa Clara County, Ca. This project arrived with only unclassified data. NOAAs Office for Coastal Management performed an automated classification using lasground.  Although class 1 and class 2 are available, there was no QA/QC on the points after lasground was performed."
'''Generating surfaces from raw data''' For ArcGIS this means "terrains". For QGIS it means DEM rasters.
I am starting with terrains, which means first I need to get the data out of the LAZ files and into ArcGIS.
I installed the lastools into ArcGIS already, let's see what I have in there.
http://www.spatialguru.com/lidar-data-to-raster-file-with-open-source-gdal-tool/
http://www.cs.unc.edu/~isenburg/lastools/
See the README files for details on each tool.
In the order I might use them in my tool chain:
'''lasmerge''' : Merge my two tiles into one LAS file. lasmerge -i *.las -o out.las
'''lasclip''' : Clip the data to a polygon. Most of one of the tiles is outside my study area, so I should get rid of that data.
'''lasground''' : tool for bare earth extraction (class=2 : ground, class=1 not ground)
'''las2dem''' : Make a TIN then make a raster from the TIN.
This is interesting too.
'''lastrack''' : Generates profile along a GPS track.
Setting step size will help remove buildings. Default is 5m, try -town or -city or -metro as options.
Vertical unit can be meters (the default) or else try "-feet".
For steep hills try "-fine" or "-extra_fine"
<pre>
lasinfo (160710) report for D:\GISData\CA\SantaClara\LAH_merged.laz
reading 'D:\GISData\CA\SantaClara\LAH_merged.laz' with 13366138 points
reporting all LAS header entries:
  file signature:            'LASF'
  file source ID:            0
  global_encoding:            0
  project ID GUID data 1-4:  00000000-0000-0000-0000-000000000000
  version major.minor:        1.1
  system identifier:          'LAStools (c) by rapidlasso GmbH'
  generating software:        'lasmerge (version 160710)'
  file creation day/year:    1/1970
  header size:                227
  offset to point data:      387
  number var. length records: 2
  point data format:          1
  point data record length:  28
  number of point records:    13366138
  number of points by return: 11562914 1709526 92895 803 0
  scale factor x y z:        0.0000001 0.0000001 0.001
  offset x y z:              0 0 0
  min x y z:                  -122.1056002 37.3255269 -795.944
  max x y z:                  -122.0878199 37.3532182 971.289
variable length header record 1 of 2:
  reserved            43707
  user ID              'NOAA_CSC'
  record ID            129
  length after header  2
  description          'Sort Order'
variable length header record 2 of 2:
  reserved            43707
  user ID              'LASF_Projection'
  record ID            34735
  length after header  48
  description          'Projection Parameters'
    GeoKeyDirectoryTag version 1.1.0 number of keys 5
      key 1024 tiff_tag_location 0 count 1 value_offset 2 - GTModelTypeGeoKey: ModelTypeGeographic
      key 2048 tiff_tag_location 0 count 1 value_offset 4269 - GeographicTypeGeoKey: GCS_NAD83
      key 2054 tiff_tag_location 0 count 1 value_offset 9102 - GeogAngularUnitsGeoKey: Angular_Degree
      key 4096 tiff_tag_location 0 count 1 value_offset 5103 - VerticalCSTypeGeoKey: VertCS_North_American_Vertical_Datum_1988
      key 4099 tiff_tag_location 0 count 1 value_offset 9001 - VerticalUnitsGeoKey: Linear_Meter
the header is followed by 2 user-defined bytes
LASzip compression (version 2.4r2 c2 50000): POINT10 2 GPSTIME11 2
reporting minimum and maximum for all LAS point record entries ...
  X          -1221056002 -1220878199
  Y          373255269  373532182
  Z            -795944    971289
  intensity          1      5100
  return_number      1          4
  number_of_returns  1          4
  edge_of_flight_line 0          0
  scan_direction_flag 0          0
  classification      1          2
  scan_angle_rank    0          0
  user_data          0          0
  point_source_ID  294        770
  gps_time 661.683328 602614.538400
number of first returns:        11562914
number of intermediate returns: 93700
number of last returns:        11563016
number of single returns:      9853492
overview over number of returns of given pulse: 9853492 3233170 276260 3216 0 0 0
histogram of classification of points:
        9643729  unclassified (1)
        3722409  ground (2)
Success. lasinfo done.
Completed script lasinfo...
Succeeded at Mon Jul 18 12:11:15 2016 (Elapsed Time: 6.35 seconds)
</pre>
== Gold Beach ==
== Gold Beach ==



Revision as of 19:25, 18 July 2016

Santa Clara county

For this project I want to closely examine a reach of road that goes into the hills to see if it can accommodate addition of a footpath.

I downloaded data that was collected by the Santa Clara Water District in 2006. The area of interest is small, covered by two tiles.

What coordinate system is this data in? See https://coast.noaa.gov/htdata/lidar1_z/geoid12a/data/4870/

From metadata I think it's Cal III NAD83 and the vertical unit is in feet.

The data is in LAZ format.

First thing I want to do is generate simple first return and last return surfaces.

The files are already classified, see the metadata:

"This data set is an LAZ (compressed LAS) format file containing LIDAR point cloud data. LAS format files, raw LiDAR data in its native format, classified bare-earth LiDAR DEM and photogrammetrically derived breaklines generated from LiDAR Intensity stereo-pairs. Breakline, Top of Bank, and contour files in ESRI personal geodatabase format, Microstation V8 .dgn format, and AutoCAD 2004 formats for the San Jose Phase 3 project of Santa Clara County, Ca. This project arrived with only unclassified data. NOAAs Office for Coastal Management performed an automated classification using lasground. Although class 1 and class 2 are available, there was no QA/QC on the points after lasground was performed."


Generating surfaces from raw data For ArcGIS this means "terrains". For QGIS it means DEM rasters. I am starting with terrains, which means first I need to get the data out of the LAZ files and into ArcGIS. I installed the lastools into ArcGIS already, let's see what I have in there.

http://www.spatialguru.com/lidar-data-to-raster-file-with-open-source-gdal-tool/

http://www.cs.unc.edu/~isenburg/lastools/ See the README files for details on each tool.

In the order I might use them in my tool chain:

lasmerge : Merge my two tiles into one LAS file. lasmerge -i *.las -o out.las

lasclip : Clip the data to a polygon. Most of one of the tiles is outside my study area, so I should get rid of that data.

lasground : tool for bare earth extraction (class=2 : ground, class=1 not ground)

las2dem : Make a TIN then make a raster from the TIN.

This is interesting too.

lastrack : Generates profile along a GPS track.

Setting step size will help remove buildings. Default is 5m, try -town or -city or -metro as options.

Vertical unit can be meters (the default) or else try "-feet".

For steep hills try "-fine" or "-extra_fine"

lasinfo (160710) report for D:\GISData\CA\SantaClara\LAH_merged.laz
reading 'D:\GISData\CA\SantaClara\LAH_merged.laz' with 13366138 points
reporting all LAS header entries:
  file signature:             'LASF'
  file source ID:             0
  global_encoding:            0
  project ID GUID data 1-4:   00000000-0000-0000-0000-000000000000
  version major.minor:        1.1
  system identifier:          'LAStools (c) by rapidlasso GmbH'
  generating software:        'lasmerge (version 160710)'
  file creation day/year:     1/1970
  header size:                227
  offset to point data:       387
  number var. length records: 2
  point data format:          1
  point data record length:   28
  number of point records:    13366138
  number of points by return: 11562914 1709526 92895 803 0
  scale factor x y z:         0.0000001 0.0000001 0.001
  offset x y z:               0 0 0
  min x y z:                  -122.1056002 37.3255269 -795.944
  max x y z:                  -122.0878199 37.3532182 971.289
variable length header record 1 of 2:
  reserved             43707
  user ID              'NOAA_CSC'
  record ID            129
  length after header  2
  description          'Sort Order'
variable length header record 2 of 2:
  reserved             43707
  user ID              'LASF_Projection'
  record ID            34735
  length after header  48
  description          'Projection Parameters'
    GeoKeyDirectoryTag version 1.1.0 number of keys 5
      key 1024 tiff_tag_location 0 count 1 value_offset 2 - GTModelTypeGeoKey: ModelTypeGeographic
      key 2048 tiff_tag_location 0 count 1 value_offset 4269 - GeographicTypeGeoKey: GCS_NAD83
      key 2054 tiff_tag_location 0 count 1 value_offset 9102 - GeogAngularUnitsGeoKey: Angular_Degree
      key 4096 tiff_tag_location 0 count 1 value_offset 5103 - VerticalCSTypeGeoKey: VertCS_North_American_Vertical_Datum_1988
      key 4099 tiff_tag_location 0 count 1 value_offset 9001 - VerticalUnitsGeoKey: Linear_Meter
the header is followed by 2 user-defined bytes
LASzip compression (version 2.4r2 c2 50000): POINT10 2 GPSTIME11 2
reporting minimum and maximum for all LAS point record entries ...
  X          -1221056002 -1220878199
  Y           373255269  373532182
  Z             -795944     971289
  intensity           1       5100
  return_number       1          4
  number_of_returns   1          4
  edge_of_flight_line 0          0
  scan_direction_flag 0          0
  classification      1          2
  scan_angle_rank     0          0
  user_data           0          0
  point_source_ID   294        770
  gps_time 661.683328 602614.538400
number of first returns:        11562914
number of intermediate returns: 93700
number of last returns:         11563016
number of single returns:       9853492
overview over number of returns of given pulse: 9853492 3233170 276260 3216 0 0 0
histogram of classification of points:
         9643729  unclassified (1)
         3722409  ground (2)

Success. lasinfo done.
Completed script lasinfo...
Succeeded at Mon Jul 18 12:11:15 2016 (Elapsed Time: 6.35 seconds)


Gold Beach

20-Apr-2011 Going to Gold Beach for Easter weekend, so naturally I am looking at Curry county data once again.

I have a nice NAIP 2009 photo of the area now. 1/2 meter 2009 county level files are available as zipped files via FTP from Oregon Explorer

Working with LiDAR obtained from the Oregon Coastal Atlas

Holes in LiDAR data

I have created a contour and a hillshade, which is great, but the LiDAR is full of holes! I think since it's a coastal dataset (NOAA) they don't care about the areas up a bit from the beach.

I am thinking I can fill the holes with 10m DEM data, so I found an example on how to do this and will look at it this evening.

http://blogs.esri.com/Support/blogs/mappingcenter/archive/2009/06/16/Filling-and-clipping-a-raster.aspx