Soil Data Access. The Soil Data Access The Soil Data Access facility is a suite of web services and...

47
Soil Data Access

Transcript of Soil Data Access. The Soil Data Access The Soil Data Access facility is a suite of web services and...

Page 1: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Soil Data Access

Page 2: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

The Soil Data Accesshttp://sdmdataaccess.nrcs.usda.gov

• The Soil Data Access facility is a suite of web services and applications

• Designed to meet user needs not met by other applications– Web based spatial and tabular queries– Services for other applications to

interface with

Page 3: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

A Procedure in Many Steps

• Definition of the area of interest• Conversion to appropriate format• Submitting a request to the utility• Receiving map unit information• Submitting an SQL request for data• Receiving a report!

Page 4: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

NOTE

• Even though SDA allows a larger spatial extent than Web Soil Survey, there is still a limit!

• The total extent defined by any spatial filter cannot exceed 10,100,000,000 square meters (approximately 3,900 square miles or 2,500,000 acres).

Page 5: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Soil Data Access

Web Services

Page 6: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Want mukey information on a specific point?

Instructions for this are on the HELP menu of the web site. For more details see the Help menu. Using the Web Service you can modify coordinates to retrieve data on a specific point.

Change the -97.85,38.88 coordinates in these URLs to fit your point.

• POINT SPATIAL TAGhttp://sdmdataaccess.nrcs.usda.gov/Spatial/SDMNAD83Geographic.wfs?service=WFS&version=1.0.0&outputformat=GML2&typename=MapunitPoly&request=GetFeature&Filter=<Filter><DWithin><PropertyName>Geometry</PropertyName><gml:Point><gml:coordinates>-97.85,38.88</gml:coordinates></gml:Point><Distance units='m'>0</Distance></DWithin></Filter>

• Enter this URL string into a browser address and save the resulting file as an XML and import into Access

Page 7: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

POINT SPATIAL TAG

Enter the WMS in the address of a web page. Save the resulting web page as an XML file to import into Access or read the requested information from the web page.

http://sdmdataaccess.nrcs.usda.gov/Spatial/SDMNAD83Geographic.wfs?service=WFS&version=1.0.0&outputformat=GML2&typename=MapunitPoly&re

quest=GetFeature&Filter=<Filter><DWithin><PropertyName>Geometry</PropertyName><gml:Point><gml:coordinates>-97.85,38.88</

gml:coordinates></gml:Point><Distance units='m'>0</Distance></DWithin></Filter>

Page 8: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

POINT SPATIAL TAG• Results of XML file imported into Access provides the

area symbol, musym and mukey for a point for further querying

Page 9: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Want attribute information on a specific point?

Using the Web Service you can modify coordinates to retrieve additional soil properties and interpretation data on a specific point. Change the -97.85,38.88 coordinates in these URLs to fit your point.

• POINT WITH DATAhttp://sdmdataaccess.nrcs.usda.gov/Spatial/SDMNAD83Geographic.wfs?service=WFS&version=1.0.0&outputformat=GML2&typename=MapunitPolyExtended&request=GetFeature&Filter=%3CFilter%3E%3CDWithin%3E%3CPropertyName%3EGeometry%3C/PropertyName%3E%3Cgml:Point%3E%3Cgml:coordinates%3E-97.85,38.88%3C/gml:coordinates%3E%3C/gml:Point%3E%3CDistance%20units='m'%3E0%3C/Distance%3E%3C/DWithin%3E%3C/Filter%3E

• This URL will provide an XML file containing “muaggatt” data based on the coordinates. Save the resulting file as an XML file and import it into Access.

Page 10: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

POINT WITH DATA Results

This script includes information from the “muaggatt” table. Save the resulting web page as an XML file to import into Access or read the requested information from the web page.

http://sdmdataaccess.nrcs.usda.gov/Spatial/SDMNAD83Geographic.wfs?service=WFS&version=1.0.0&outputformat=GML2&typename=MapunitPolyExtended&request=GetFeature&Filter=%3CFilter%3E%3CDWithin%3E%3CPropertyName%3EGeometry%3C/PropertyName%3E%3Cgml:Point%3E%3Cgml:coordinates%3E-97.85,38.88%3C/gml:coordinates%3E%3C/gml:Point%3E%3CDistance%20units='m'%3E0%3C/Distance%3E%3C/DWithin%3E%3C/Filter%3E

Page 11: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

POINT WITH DATA Results• Results of XML file imported into Access

provides the “muaggatt” data for a point

Page 12: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

What about spatial data in Arc?•Add a “WMS Server” link

Page 13: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Spatial Data WMS

• Enter this URL, click on “Get Layers” then OK• http://sdmdataaccess.nrcs.usda.gov/Spatial/SDM.wms?

Page 14: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Spatial Data WMS

• Survey boundary data appears only as you begin to zoom in, first as survey areas

Page 15: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Spatial Data WMS

Then data appears as you zoom in further, finally as polygons with symbols

Page 16: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Bringing in the NAIP and SDA

Then turn off the Area Polygon and Polygon Transparent layer and include the NAIP WPS (http://gis1.ftw.nrcs.usda.gov/arcgis/services) as background

Page 17: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Soil Data Access

Using ArcGIS to create a specific Area of Interest

Page 18: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Defining the Area of Interest

For this example we’ll assume we want a report on an 85,000 acre watershed that spans five soil survey areas and two states as part of a cooperative watershed assessment.

Page 19: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Converting the Area of Interest

The next series of steps take place in ArcMap in order to set the right map projection and to export the data to a format that can be read by the utility. Make sure that you use the boundary file.

Page 20: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Reproject the Data

Whether the data are in shape file or coverage form, they still need to be projected, if not already there, to an Albers Equal Area Conic projection.

Page 21: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Reproject the Data

The Project utility for shape files is in the ArcMap Toolbox is under:

– Data Management Tools

• Projections and Transformations

– Feature» Project

The correct projection for Soil Data Access is:

– Projected Coordinate Systems• Continental

– North America» USA Contiguous Albers Equal Area Conic.prj

Page 22: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Converting a shape file to a coverage

•If your data is already in a coverage you can skip this step and proceed to the section on converting to ASCII format

Page 23: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Convert to a CoverageThe tool to convert a shape file to a coverage is in the Toolbox under:

– Conversion Tools

• To Coverage– Feature Class

to Coverage

Just the input shape file and the name of a coverage are required with no other parameters needed. Then add the coverage to your data view.

Page 24: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Converting to ASCII Format

The Ungenerate utility for coverages is in the ArcMap Toolbox under:

– Coverage Tools• Conversion

– From Coverage» Ungenerate

•Set “Input Coverage” to the coverage or the latest coverage that came from a reprojected shape file•Set the “Output Generate File” to a text file name•Set “Feature Type” to Line•Set “Numeric Format” to Fixed

Page 25: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Editing the ASCII File

The resultant text file appears as the example but much longer depending on the number of coordinate pairs in the polygon perimeter.

Page 26: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Editing the ASCII FileThe Soil Data Access Utility does not function well for a polygon perimeter with more than somewhere around 50 coordinate pairs. This is an identified bug which will eventually be addressed.

Note: This particular watershed had 1307 coordinate pairs in its perimeter and, while it displayed in the proper place, did not process through to receive a list of MUKEY’s. This watershed had to be generalized down to 50 coordinate pairs (vertices) before it would parse properly.

The perimeter line may need to be edited from the ARC command prompt using the GENERALIZE command before the UNGENERATE command is used. This can be kind of a long process going from GENERALIZE to UNGENERATE and back again until the resultant UNGENERATE file contains less than 50 coordinate pairs.

Page 27: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Editing the ASCII File

Once you have your file with approximately 50 coordinate pairs. Place it back into arc map using the generate command located at Coverage tools > to coverage > generate. Check to make sure that the data is close to the original file before moving forward.

Page 28: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Editing the ASCII File

The ASCII file created previously from the UNGENERATE needs to be in WKT format. WKT stands for Well-Known-Text, one of the languages that the SDA Utility reads. Editing can be done in a variety of ways, including Word and Excel, but the resultant format needs to be in this format with the word “Polygon” at the start followed by the coordinate pairs and the proper parentheses.

Page 29: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Editing the ASCII File

•Editing the file in Excel

– Open Excel– Import the .TXT file created by UNGENERATE

• It is a text file so select “Delimited” with “Space” as the delimiter then hit “Finish”

• This will create a three column file

– Delete all rows except those with coordinate pairs

• There should be a number at the beginning of each line segment and the word “END” after the coordinate pairs for that line segment along with one extra line with “END” at the bottom of the file

Page 30: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Editing the ASCII File

•Editing the file in Excel

– Place cursor in the first row, fourth column– Type =CONCATENATE(A1,” “,B1,”,”) and copy down for

all the rows of data• This will format the data with the X coordinate, a space, the Y coordinate,

and a comma – the base format WKT reads

– The entire column of formatted text will be highlighted so hit CTRL-C to copy it to the scrap or save the file for future use.

– The first and last coordinate pairs must be the same – think of it like closing a survey circuit – check to make sure

Page 31: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Editing the ASCII File•Editing the file in Word

– Paste the concatenated column of numbers and text into Word

– The column is in table format and must be converted to text

• At the bottom of the column is a paste options icon which will give a drop down choice list – select “Keep Text Only”

– Now each coordinate pair is on a separate line followed by a carriage return. These extra returns must be removed.

• Starting at the top of the column select “Edit” from the main menu bar then “Find and Replace” from the drop down box

• In the “Find what” box type: ^p (finds all carriage returns)• In the “Replace with” box: Leave blank• Press the “Replace All” button. All of the carriage returns are

removed except the last one and the file is in “WKT” format

Page 32: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Using Soil Data AccessUp until now this tutorial has focused on developing the spatial data file in Arc.

Now it’s time to use the Soil Data Access Web site to retrieve soils information.

http://sdmdataaccess.nrcs.usda.gov/

Page 33: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Using Soil Data AccessStep 1 – Select “Submit a custom request for soil spatial data”

Page 34: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Using Soil Data Access

Step 2 – Select “My Area Of Interest”

Page 35: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Using Soil Data AccessStep 3 – Hit the blue “toggle” word

Step 4 – See next slide

Page 36: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Using Soil Data Access

Step 4 – Entering in the polygon information

– In the text box, type in caps POLYGON ((

– If you previously copied the column of coordinates and text in Excel into the scrap, do a CTRL-V to paste it next, otherwise go highlight the column of full text, CTRL-C, then CTRL-V here to paste them in

– At the end of the text box, backspace over the last comma and type in “))” to close the expression

– Then hit the “Activate My WKT Shape” button

Page 37: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Using Soil Data AccessThe map now shows the Area of Interest and

the soil survey areas that it intersects

Page 38: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Using Soil Data AccessStep 5 – Hit the “MUKEYs -> SQL

query” buttonIf you are unable to

click on the mukeys -> SQL button zoom in

closer to the map and it should come

up.

Page 39: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Using Soil Data AccessStep 6 – Click OK

Note how the coordinates have been changed to

geographic coordinates, the size

of this URL line is what affects the

functionality of the query.

If no window pops up disable your pop up

blocker and try again

Page 40: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Using Soil Data AccessStep 7 – Copy the

MUKEY's for your AOI

The text now lists all of the unique MUKEY's for your AOI. SQL can be

added in this window and parsed or,

perhaps a better approach, the

line can be copied into Word

for full SQL development.

Page 41: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Using Soil Data Access

Step 8 – Create an

SQL

There are two main parts to

the SQL …

One is the list of the MUKEY’s

to query for.

The other is the query for the selected

MUKEY’s

('153456', '153445', '153464', '153501', '153449', '153455', '153452', '153497', '153401', '153349', '153453', '153441', '84334', '84361', '84376', '84373', '84377', '84348', '84343', '84318', '84374', '84360', '84365', '84345', '84371', '84310', '84311', '84346', '84344', '84332', '84357', '84358', '84333', '84336', '84362', '84331', '84364', '84337', '84335', '84372', '84338', '84321', '84340', '84328', '84356', '84363', '84354', '84370', '84380', '84325', '84347', '84323', '84353', '84359', '84322', '84366', '84349', '84342', '84339', '84317', '84319', '84369', '84368', '84314', '84329', '84367', '79560', '79533', '79520', '79531', '79512', '68512', '68483', '68533', '68491', '68486', '68492', '68556', '68563', '68551', '68570', '68564', '68536', '68488', '68493', '68534', '68514', '68484', '68559', '68591', '68562')

SELECTl.areasymbol, l.areaname,,musym, muname, mu.mukey,comppct_r, compname, tfact, wei, drainagecl, weg, slope_l,slope_hFROM sacatalog sacINNER JOIN legend l ON l.areasymbol = sac.areasymbolINNER JOIN mapunit mu ON mu.lkey = l.lkeyAND mu.mukey IN(LIST MUKEY’s HERE FROM THE SPATIAL QUERY)LEFT OUTER JOIN component c ON c.mukey = mu.mukeyORDER BY l.areaname, museq, comppct_r DESC, compname

Page 42: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Using Soil Data Access

Step 9 – Put it all together

SELECT

l.areasymbol, l.areaname, musym, muname, mu.mukey,comppct_r, compname, tfact, wei, drainagecl, weg, slope_l,slope_h

FROM sacatalog sac

INNER JOIN legend l ON l.areasymbol = sac.areasymbol

INNER JOIN mapunit mu ON mu.lkey = l.lkey

AND mu.mukey IN

('153456', '153445', '153464', '153501', '153449', '153455', '153452', '153497', '153401', '153349', '153453', '153441', '84334', '84361', '84376', '84373', '84377', '84348', '84343', '84318', '84374', '84360', '84365', '84345', '84371', '84310', '84311', '84346', '84344', '84332', '84357', '84358', '84333', '84336', '84362', '84331', '84364', '84337', '84335', '84372', '84338', '84321', '84340', '84328', '84356', '84363', '84354', '84370', '84380', '84325', '84347', '84323', '84353', '84359', '84322', '84366', '84349', '84342', '84339', '84317', '84319', '84369', '84368', '84314', '84329', '84367', '79560', '79533', '79520', '79531', '79512', '68512', '68483', '68533', '68491', '68486', '68492', '68556', '68563', '68551', '68570', '68564', '68536', '68488', '68493', '68534', '68514', '68484', '68559', '68591', '68562')

LEFT OUTER JOIN component c ON c.mukey = mu.mukey

ORDER BY l.areaname, museq, comppct_r DESC, compname

The text that you enter into the website should look similar to this.Select the type of return you want and press the submit query button

Page 43: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Using Soil Data Access

Step 10 – View the results!

The previous query results in a new web page popping up with this report.

Page 44: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Using Soil Data AccessThe SQL can be edited to give a more formatted

report.

The “as” comments in the “Select”

portion of the new SQL gives

the column headers a new name rather

than the cryptic NASIS one.

SELECTl.areasymbol as Survey, l.areaname as "Survey_Name", mu.mukey as Mukey, musym as Symbol, muname as "Map_Unit_Name", compname as Component, comppct_r as "Component_Percent", hzname as Horizon, hzdept_r as "Upper_Depth", hzdepb_r As "Lower_Depth", kwfact as "K_Factor", awc_r as "AWC_inches_per_inch", ph1to1h2o_r as pH, round(([ksat_l]*0.1417),1) as "Perm_Low", round(([ksat_h]*0.1417),1) as "Perm_High"FROM sacatalog sacINNER JOIN legend l ON l.areasymbol = sac.areasymbolINNER JOIN mapunit mu ON mu.lkey = l.lkeyAND mu.mukey IN('153456', '153445', '153464', '153501', '153449', '153455', '153452', '153497', '153401', '153349', '153453', '153441', '84334', '84361', '84376', '84373', '84377', '84348', '84343', '84318', '84374', '84360', '84365', '84345', '84371', '84310', '84311', '84346', '84344', '84332', '84357', '84358', '84333', '84336', '84362', '84331', '84364', '84337', '84335', '84372', '84338', '84321', '84340', '84328', '84356', '84363', '84354', '84370', '84380', '84325', '84347', '84323', '84353', '84359', '84322', '84366', '84349', '84342', '84339', '84317', '84319', '84369', '84368', '84314', '84329', '84367', '79560', '79533', '79520', '79531', '79512', '68512', '68483', '68533', '68491', '68486', '68492', '68556', '68563', '68551', '68570', '68564', '68536', '68488', '68493', '68534', '68514', '68484', '68559', '68591', '68562')LEFT OUTER JOIN component c ON c.mukey = mu.mukeyLEFT OUTER JOIN chorizon ch ON ch.cokey = c.cokeyORDER BY l.areaname, museq, comppct_r DESC, compname, hzdept_r

Note the inclusion of a formula also into

the SQL query!

Page 45: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Using Soil Data Access

The new formatted

report looks like this.

Page 46: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Process Step ReviewObtain a boundary

Convert to coverage

UNGENERATE

Edit in Excel and Word to WKT format

Submit polygon to obtain MUKEY’s

View report

Develop an SQL query for

data

Insert list of

MUKEY’s into SQL

query

Project to Albers

Page 47: Soil Data Access. The Soil Data Access  The Soil Data Access facility is a suite of web services and applications Designed.

Finale!

•The actual process of going from a map layer to a report takes about 30 minutes once the user is familiar with the steps