Coastal LiDAR: Difference between revisions

From Wildsong
Jump to navigationJump to search
← Older edit
VisualWikitext
Brian Wilson (talk | contribs)
Bureaucrats, restrict, Administrators, uploaders
11,540 edits
mNo edit summary
Brian Wilson (talk | contribs)
Bureaucrats, restrict, Administrators, uploaders
11,540 edits
mNo edit summary
 
(6 intermediate revisions by the same user not shown)
Line 1: Line 1:
This is called the "Coastal" LiDAR page because I generally go to [https://coast.noaa.gov/dataviewer/ NOAA's coastal data viewer] to find my LiDAR.
Since I live near the coast it's generally the best place for me to go.
Currently I use the data for (1) my own education (2) TrailPeople projects.
For TrailPeople mostly I have been downloaded DEM's as a short cut for getting accurate elevation data.
I am trying to develop a good workflow for processing the LAS point data directly.
== Lincoln county, Oregon ==
Feb 2017-
I am looking for a house in Lincoln county and have decided to commit to a full GIS approach, so I am collecting all the data I can find to build up an analysis. Then I can look at properties from afar and be better informed.
Workflow: My intent is to go full open source on this project.
This will include LiDAR so that I can check on slope and aspect to help evaluate solar access. I want PV so I need sun.
== Santa Clara county ==
== Santa Clara county ==
2016- TrailPeople


For this project I want to closely examine a reach of road that goes into the hills to see if it can accommodate addition of a footpath.
For this project I want to closely examine a reach of road that goes into the hills to see if it can accommodate addition of a footpath.


I downloaded data that was collected by the Santa Clara Water District in 2006. The area of interest is small, covered by two tiles.
Workflow: On this project I did use LAZ point files but I used ESRI tools.


''What coordinate system is this data in?'' See https://coast.noaa.gov/htdata/lidar1_z/geoid12a/data/4870/
I downloaded data that was collected by the Santa Clara Water District in 2006. The area of interest is small, covered by 4 tiles.
See https://coast.noaa.gov/htdata/lidar1_z/geoid12a/data/4870/
The data is in LAZ format.


From metadata I think it's Cal III NAD83 and the vertical unit is in feet.
''What coordinate system is this data in?'' It's in WGS84 and the vertical unit is 1 meter.
 
The data is in LAZ format.  


First thing I want to do is generate simple first return and last return surfaces.
First thing I want to do is generate simple first return and last return surfaces.
Line 16: Line 34:


"This data set is an LAZ (compressed LAS) format file containing LIDAR point cloud data. LAS format files, raw LiDAR data in its native format, classified bare-earth LiDAR DEM  and photogrammetrically derived breaklines generated from LiDAR Intensity stereo-pairs.  Breakline, Top of Bank, and contour files in ESRI personal geodatabase format, Microstation V8 .dgn format, and AutoCAD 2004 formats for the San Jose Phase 3 project of Santa Clara County, Ca. This project arrived with only unclassified data. NOAAs Office for Coastal Management performed an automated classification using lasground.  Although class 1 and class 2 are available, there was no QA/QC on the points after lasground was performed."
"This data set is an LAZ (compressed LAS) format file containing LIDAR point cloud data. LAS format files, raw LiDAR data in its native format, classified bare-earth LiDAR DEM  and photogrammetrically derived breaklines generated from LiDAR Intensity stereo-pairs.  Breakline, Top of Bank, and contour files in ESRI personal geodatabase format, Microstation V8 .dgn format, and AutoCAD 2004 formats for the San Jose Phase 3 project of Santa Clara County, Ca. This project arrived with only unclassified data. NOAAs Office for Coastal Management performed an automated classification using lasground.  Although class 1 and class 2 are available, there was no QA/QC on the points after lasground was performed."


'''Generating surfaces from raw data''' For ArcGIS this means "terrains". For QGIS it means DEM rasters.
'''Generating surfaces from raw data''' For ArcGIS this means "terrains". For QGIS it means DEM rasters.
I am starting with terrains, which means first I need to get the data out of the LAZ files and into ArcGIS.
I am starting with terrains, which means first I need to get the data out of the LAZ files and into ArcGIS.
I installed the lastools into ArcGIS already, let's see what I have in there.


http://www.spatialguru.com/lidar-data-to-raster-file-with-open-source-gdal-tool/
http://www.spatialguru.com/lidar-data-to-raster-file-with-open-source-gdal-tool/


http://www.cs.unc.edu/~isenburg/lastools/
http://www.laszip.org/
See the README files for details on each tool.


In the order I might use them in my tool chain:
'''Problem encountered:''' One of the tiles was misclassified; vegetation (tall trees) and buildings ended up in the "bare earth" category resulting in some crazy contour lines.


'''lasmerge''' : Merge my two tiles into one LAS file. lasmerge -i *.las -o out.las
== Gold Beach ==
 
'''lasclip''' : Clip the data to a polygon. Most of one of the tiles is outside my study area, so I should get rid of that data.
 
'''lasground''' : tool for bare earth extraction (class=2 : ground, class=1 not ground)
 
'''las2dem''' : Make a TIN then make a raster from the TIN.
 
This is interesting too.


'''lastrack''' : Generates profile along a GPS track.
''This project goes back a ways now, and I recall the big problem was holes in the data that probably no longer exist. The holes became the interesting problem but if I were to try again, I'd probably be able to get 2009 or later data with 100% coverage of the area of interest.''
 
Setting step size will help remove buildings. Default is 5m, try -town or -city or -metro as options.
 
Vertical unit can be meters (the default) or else try "-feet".
 
For steep hills try "-fine" or "-extra_fine"
 
<pre>
lasinfo (160710) report for D:\GISData\CA\SantaClara\LAH_merged.laz
reading 'D:\GISData\CA\SantaClara\LAH_merged.laz' with 13366138 points
reporting all LAS header entries:
  file signature:            'LASF'
  file source ID:            0
  global_encoding:            0
  project ID GUID data 1-4:  00000000-0000-0000-0000-000000000000
  version major.minor:        1.1
  system identifier:          'LAStools (c) by rapidlasso GmbH'
  generating software:        'lasmerge (version 160710)'
  file creation day/year:    1/1970
  header size:                227
  offset to point data:      387
  number var. length records: 2
  point data format:          1
  point data record length:  28
  number of point records:    13366138
  number of points by return: 11562914 1709526 92895 803 0
  scale factor x y z:        0.0000001 0.0000001 0.001
  offset x y z:              0 0 0
  min x y z:                  -122.1056002 37.3255269 -795.944
  max x y z:                  -122.0878199 37.3532182 971.289
variable length header record 1 of 2:
  reserved            43707
  user ID              'NOAA_CSC'
  record ID            129
  length after header  2
  description          'Sort Order'
variable length header record 2 of 2:
  reserved            43707
  user ID              'LASF_Projection'
  record ID            34735
  length after header  48
  description          'Projection Parameters'
    GeoKeyDirectoryTag version 1.1.0 number of keys 5
      key 1024 tiff_tag_location 0 count 1 value_offset 2 - GTModelTypeGeoKey: ModelTypeGeographic
      key 2048 tiff_tag_location 0 count 1 value_offset 4269 - GeographicTypeGeoKey: GCS_NAD83
      key 2054 tiff_tag_location 0 count 1 value_offset 9102 - GeogAngularUnitsGeoKey: Angular_Degree
      key 4096 tiff_tag_location 0 count 1 value_offset 5103 - VerticalCSTypeGeoKey: VertCS_North_American_Vertical_Datum_1988
      key 4099 tiff_tag_location 0 count 1 value_offset 9001 - VerticalUnitsGeoKey: Linear_Meter
the header is followed by 2 user-defined bytes
LASzip compression (version 2.4r2 c2 50000): POINT10 2 GPSTIME11 2
reporting minimum and maximum for all LAS point record entries ...
  X          -1221056002 -1220878199
  Y          373255269  373532182
  Z            -795944    971289
  intensity          1      5100
  return_number      1          4
  number_of_returns  1          4
  edge_of_flight_line 0          0
  scan_direction_flag 0          0
  classification      1          2
  scan_angle_rank    0          0
  user_data          0          0
  point_source_ID  294        770
  gps_time 661.683328 602614.538400
number of first returns:        11562914
number of intermediate returns: 93700
number of last returns:        11563016
number of single returns:      9853492
overview over number of returns of given pulse: 9853492 3233170 276260 3216 0 0 0
histogram of classification of points:
        9643729  unclassified (1)
        3722409  ground (2)
 
Success. lasinfo done.
Completed script lasinfo...
Succeeded at Mon Jul 18 12:11:15 2016 (Elapsed Time: 6.35 seconds)
</pre>
 
 
 
== Gold Beach ==


20-Apr-2011 Going to Gold Beach for Easter weekend, so naturally I am looking at Curry county data once again.
20-Apr-2011 Going to Gold Beach for Easter weekend, so naturally I am looking at Curry county data once again.
Line 135: Line 61:


http://blogs.esri.com/Support/blogs/mappingcenter/archive/2009/06/16/Filling-and-clipping-a-raster.aspx
http://blogs.esri.com/Support/blogs/mappingcenter/archive/2009/06/16/Filling-and-clipping-a-raster.aspx
[[Category: GIS]]

Latest revision as of 20:44, 27 February 2017

This is called the "Coastal" LiDAR page because I generally go to NOAA's coastal data viewer to find my LiDAR. Since I live near the coast it's generally the best place for me to go.

Currently I use the data for (1) my own education (2) TrailPeople projects. For TrailPeople mostly I have been downloaded DEM's as a short cut for getting accurate elevation data. I am trying to develop a good workflow for processing the LAS point data directly.

Lincoln county, Oregon

Feb 2017- I am looking for a house in Lincoln county and have decided to commit to a full GIS approach, so I am collecting all the data I can find to build up an analysis. Then I can look at properties from afar and be better informed.

Workflow: My intent is to go full open source on this project.

This will include LiDAR so that I can check on slope and aspect to help evaluate solar access. I want PV so I need sun.

Santa Clara county

2016- TrailPeople

For this project I want to closely examine a reach of road that goes into the hills to see if it can accommodate addition of a footpath.

Workflow: On this project I did use LAZ point files but I used ESRI tools.

I downloaded data that was collected by the Santa Clara Water District in 2006. The area of interest is small, covered by 4 tiles. See https://coast.noaa.gov/htdata/lidar1_z/geoid12a/data/4870/ The data is in LAZ format.

What coordinate system is this data in? It's in WGS84 and the vertical unit is 1 meter.

First thing I want to do is generate simple first return and last return surfaces.

The files are already classified, see the metadata:

"This data set is an LAZ (compressed LAS) format file containing LIDAR point cloud data. LAS format files, raw LiDAR data in its native format, classified bare-earth LiDAR DEM and photogrammetrically derived breaklines generated from LiDAR Intensity stereo-pairs. Breakline, Top of Bank, and contour files in ESRI personal geodatabase format, Microstation V8 .dgn format, and AutoCAD 2004 formats for the San Jose Phase 3 project of Santa Clara County, Ca. This project arrived with only unclassified data. NOAAs Office for Coastal Management performed an automated classification using lasground. Although class 1 and class 2 are available, there was no QA/QC on the points after lasground was performed."

Generating surfaces from raw data For ArcGIS this means "terrains". For QGIS it means DEM rasters. I am starting with terrains, which means first I need to get the data out of the LAZ files and into ArcGIS.

http://www.spatialguru.com/lidar-data-to-raster-file-with-open-source-gdal-tool/

http://www.laszip.org/

Problem encountered: One of the tiles was misclassified; vegetation (tall trees) and buildings ended up in the "bare earth" category resulting in some crazy contour lines.

Gold Beach

This project goes back a ways now, and I recall the big problem was holes in the data that probably no longer exist. The holes became the interesting problem but if I were to try again, I'd probably be able to get 2009 or later data with 100% coverage of the area of interest.

20-Apr-2011 Going to Gold Beach for Easter weekend, so naturally I am looking at Curry county data once again.

I have a nice NAIP 2009 photo of the area now. 1/2 meter 2009 county level files are available as zipped files via FTP from Oregon Explorer

Working with LiDAR obtained from the Oregon Coastal Atlas

Holes in LiDAR data

I have created a contour and a hillshade, which is great, but the LiDAR is full of holes! I think since it's a coastal dataset (NOAA) they don't care about the areas up a bit from the beach.

I am thinking I can fill the holes with 10m DEM data, so I found an example on how to do this and will look at it this evening.

http://blogs.esri.com/Support/blogs/mappingcenter/archive/2009/06/16/Filling-and-clipping-a-raster.aspx

Retrieved from "https://wiki.wildsong.biz//index.php?title=Coastal_LiDAR&oldid=10158"