Video: Open GIS Data Portals

Open GIS data portals are becoming more common, and can contain lots of geospatial data, but is it a case of too much of a good thing?

What is a Portal?
An internet site providing access or links to other sites. In the case of open GIS data, the file may appear on the same page, link out to another website (or portal) or require more digging.  In addition, some GIS portals have links to web map services.

Search is the thing...
Open data portals house more and more data and therefore can become more difficult to search over time. Creating complex searches is sometimes not available and can slow the discovery process.

What are they made of?
Open data portals leverage free and open, propriety, or hybrid data and/or GIS platforms.

Here one minute, gone the next...
While often data can be downloaded directly, some sites link to external or partner data sources. Some data may seem accessible and near-at-hand but actually require more searching and digging.

Syndication
In some cases, this is the result of syndicated data sets. For example, on federal data portals, local and state governments can have their data sets metadata appear in search results and link out to the resource.  In others, this can be overused, frustrating, or lead to broken links or missing resources. Sometimes going directly to the source can be easier than navigating a much larger data portal.

Examples
We will look at several open data portals for geospatial data:

• Data.gov
• ESRI's Open Data
• National Weather Service GIS Data Portal
• NYC Open Data

Also check out http://dataportals.org/ for a neat interactive map of open data portals worldwide.

Looking for an open data portal?  Check out the interactive and searchable map on dataportals.org

Video: Free and Open Source GIS Reference Data

After last week's post looking at the landscape of free and open source GIS software, the next logical step is looking at open GIS data. This video looks at reference data. I am still deciding on what to do the next video on...the choices are a) Open data portals, b) novel data types, social media (TwitteR and Instagram), or c) remote sensing.  Let me know what you would like to see in the comments below.

Topics covered in the video include:
Census: https://www.census.gov/geo/maps-data/data/tiger.html
DataFerrett: http://dataferrett.census.gov/
Summary File 1 DVD: https://www.census.gov/mp/www/cat/decennial_census_2010/summary_file_1_1.html
Natural Earth: http://www.naturalearthdata.com/
OpenStreetMap: https://www.openstreetmap.org/
GeoNames: http://www.geonames.org/
US Board on Geographic Names:http://geonames.usgs.gov/domestic/index.html
Maryland Department of Planning/ACS: http://www.mdp.state.md.us/msdc/S7_ACS.shtml

ESRI's Open Data vs.Data.gov

ESRI recently announced its open data website (http://opendata.arcgis.com/), which was in beta in mid-2014, so I decided it was a good time to take a look.  More and more open data is being published in a variety of places, by different organizations.

I thought it was only fair to make some comparisons. Because ESRI's user base and audiences are large, I decided the best comparison is Data.gov.  The only drawback is that Data.gov has been in existence longer.

Of course, open data is important regardless of the platform.  Moreover, open data can be imported into any free and open source GIS.

"[Open data's] impacts include... cost savings, efficiency, fuel for business, improved civic services, informed policy, performance planning, research and scientific discoveries, transparency and accountability, and increased public participation in the democratic dialogue." - Data.gov

One big difference between the sites is ESRI's contains lots of data from States vs. federal (state, and local) data found in Data.gov.  ESRI touts that it works with more than 380,000 organizations across the globe, so more open data is on its way!

A table comparing ESRI's Open Data vs. Data.gov

One big advantage of ESRI's Open Data page is being able to view geographic data in your browser immediately and even see some attribute data.

Data.gov's metrics pages are really neat including one on data sets published by agency by month: . Most data sets come from NOAA and USGS, which can be accessed in several different ways.   Obviously, a lot of the pages linked from Data.gov either use ESRI formats or are driven by ESRI products.  For developers it is also important to note that Data.gov has challenges/competitions. So be sure to check their website and social media! 

Data.gov helps you get started with browsing categories.

Currently, Data.gov has more data sets but it will be interesting to see how much ESRI can catch-up in the months to come.  It is a win-win situation for any data scientist or GIS analyst.  As open data sites get larger, they can become harder to search and navigate.  In sum, both sites will have to keep innovating to help bring out the best in open data and analysis.