Earlier this month Eric Gagstatter wrote a great little article for
Geospatial Solutions Monthly titled
"Nightmare on GIS Street: Accuracy, Datums, and Geospatial Data". Anybody who's worked in the GIS field for more than a week has experienced the kind of issues Eric discusses. Simply put, it is a rare event when data pulled from multiple sources fits together with any semblance of accuracy or precision.
For a small scale project (let's say 1:20,000 or smaller) data fit is less important - at those smaller scales 'eyeball close' is often good enough. The problem we face is that with modern GIS software the user is not stuck with a fixed scale like they were when everything was based on paper maps. We live in the era of Google Earth, the era of high resolution satellite imagery, where everybody expects to be able to read the address number on their mailbox from space. This new found ability to zoom to any scale with just the scroll of a mouse wheel has highlighted a problem that the general public and, to be frank, many Geospatial and civil engineering professionals, were not aware of:
the data doesn't fit.
Eric highlights the most important factor impacting this issue - the emergence of high precision GPS-based field data. In the past 10 years or so GPS data, that data collected by survey grade or SBAS* augmented GPS units, has dramatically exposed the errors embedded in decades of historical geospatial data.
It's not that this old data was collected badly - most of it was collected to established standards using the best resources and techniques available at the time. In the old days it was paper maps, scaled aerial photos, compass headings, pace counts (or odometer readings for really long distances) and field notebooks. Mapping grade accuracy was the accepted norm. When you were using 1:24,000 USGS topo sheets as your project base an error of +/- 24 meters (the approximate National Map Accuracy Standard for those map sheets) was good enough. Formal surveys were expensive and time consuming, and only done if there was a strong business justification - usually to establish legal boundary definitions, accurately map out small project areas, or precisely position critical features.
Today a Geospatial professional can collect data with handheld GPS units that easily achieves accuracies of +/- 15
feet with just SBAS augmentation, and centimeter level accuracies with survey-grade RTK (real time kinematic) equipment. Accuracy has improved by several orders of magnitude and the cost of acquiring that data had dropped dramatically.
While Eric focuses on the issues of datums and datum transformations, my experience is a little different. I work at a major airport that has terabytes of historical CAD data and a warehouse full of historical project plans on paper, mylar or linen that go back to the early 1940s. Virtually all of this data is referenced to a local grid system that was first established as a construction grid back in 1948. At the time this grid was established it was never formally defined in reference to the local State Plane coordinate system. In fact, the surveyors who laid it out committed the cardinal sin of not establishing a central meridian that is oriented to true north. The entire grid is rotated a few degrees off of true north and that angle of rotation was never defined when the grid was established. For years this was not a problem. The airport was happy to exist as an 'island', floating on the face of the earth within its own little grid system. However, when the airport started to expand dramatically in the 1960s the engineers realized they needed to start tying into properly defined coordinate systems like State Plane. USGS and USC&GS survey control was extended onto the airport and several monuments were defined in both the local grid system and State Plane. This allowed project engineers and surveyors to 'extend' State Plane control onto their project sites if required, but all design and construction work was continued in the local grid system. To this point all design work was done using old manual drafting methods, so the levels of error inherent in these processes were acceptable for the time.
In the 1980s CAD (computer aided design and drafting) systems started to be used on more and more projects at the airport. Since our local grid is a simple x,y grid based on distances in feet measured from an origin point it was easy to lay out in CAD. No need to worry about that pesky rotation. Or, for that matter, grid-to-ground mismatches over long distances (like say on a 9,000' runway). But very soon some serious folks with serious money, like the Federal government, began asking for airport data in a 'real' coordinate system like State Plane. A number of attempts were made to try to define the local grid as a true spatial coordinate system (with a tie to a known coordinate system, an origin point and a rotation and scale factor) but with no success. As a result some very sloppy work-arounds were developed, most using a 'local fit' method - an engineer or CAD technician would snap local project data from one coordinate system to known good data in the other coordinate system; building corners, grid tics, manholes, whatever they could find. The problem was that a lot of 'known good' data turned out to be not so good. Errors propagated and started to become uncontrollable. The engineering staff worked around this by referencing in local project data (for example, a new taxiway segment) against a small subset of the overall CAD basemap for the airport. This method tended to keep the the coordinate system shift error within acceptable limits for the small project area, but when the data was added to the larger CAD basemap grid shift errors of up to 15' were common.
When my Geospatial group came on board in 2007 the coordinate system transformation issue quickly became one of our biggest headaches. We were faced with creating an enterprise geospatial database from scratch using this legacy CAD data. We desperately needed a proper spatial definition for this local grid system, something that would work in both our CAD and GIS systems. Our engineering staff was happy to dump the issue in our lap. In fact, when I interviewed for the job one of the senior engineers told me that if I was hired the first thing he wanted me to do was to
"fix this damned State Plane thing."
As we started talking with the engineering staff about the problem it became apparent they all had an institutional distrust of State Plane, or any spatial coordinate system for that matter. They placed the entire blame for the data fit issues on 'inaccuracies' in the State Plane system - inaccuracies they couldn't articulate. In their minds all data prepared in their local grid system was golden. After all, the local grid system was known. It was proven. It was simple. They had built an entire world-class airport on it. This goofy State Plane thing just got everybody confused and besides, when they did move their CAD data to State Plane it 'got shifted all around' and didn't work anymore. It might fit OK at one corner of the airport, but didn't fit too well at the other.
We eventually got the grid transformation issue solved. We attacked it from several directions and ended up with a very accurate local grid system projection file for use in both AutoCAD and ArcGIS, and a best-fit definition for use in
Blue Marble (for bulk coordinate point conversions). All of these definitions are based on the same survey data so errors are consistent and controllable from system to system. We can hold transformation errors to about 0.2' across the airport property. And yet our engineering staff still retained a latent distrust of State Plane-based data. The old institutional bias remained. The perception that ran deep is that the old 'known' CAD data in the local coordinate system is somehow better, more accurate, than any newly collected GIS data. There is a natural distrust of geospatial data; few civil engineers understand what geospatial data is, how it differs from CAD data and how geospatial data can be incorporated into planning and design projects. If the data file doesn't have a .dwg at the end of it they don't like it.
We decided to approach the perception issue from two directions. The first was a current high resolution, high accuracy orthophoto of the airport. Using our newly developed projection file we were able to reproject the aerial from State Plane to the local grid system for use in AutoCAD. For the first time ever the engineers and CAD staff had a single continuous coverage aerial image in their grid system that could be used as a base for project planning and drawing development. Next, we acquired RTK-based data collectors that are capable of centimeter level accuracy. We launched on an aggressive project to collect photo identifiable data - manholes, paint markings, slab joints, airfield lights - and provide the data in both grid systems as a tool to check current and historical data against. From this we created a 'trusted' CAD file, one the engineering group verified using their own sources. Ever so slowly some of the doubters started to come around. Once they started matching their legacy data against these new sources and saw the problems for themselves they began to do more aggressive data checks and not take CAD data, old or new, at face value.
Yet we continued to have perception problems. The old-line engineering staff retained a deeply embedded distrust of GIS data in State Plane and our insistence that all legacy data to be closely checked and adjusted if necessary. Their reasoning actually sounded pretty good -
"We spent decades building a world class airport with this CAD data and it all came together. How can the data be wrong?"
Our GIS group didn't really have a good response until some of the long time CAD staff complained that
"it's impossible to get as-builts around here." Our antennae went up and we started to do some digging on the issue. Very quickly the problem revealed itself. Our engineering staff rarely received true as-builts from the contractors that do construction on the airport. The as-built delivery requirement is written into most contracts but is rarely enforced. Contractors would regularly walk away from the as-built requirement and eat the contract penalty because they were too busy on other airport projects or the cost of developing the as-builts exceeded the monetary penalty. If a contractor did deliver what they labeled as 'as-built' drawings they were seldom, if ever, checked for accuracy and completeness by the project manager. The data was accepted at face value and often recycled for use on the next project. Errors in spatial accuracy or attributes (pipe sizes, slab thicknesses, etc.) were unknowingly propagated from project to project as the project planners and designers used the same inaccurate data over and over again. Down the line some errors became so glaringly obvious (like a stormwater line flowing uphill) that the engineering staff would hire engineering firms to go to the field and conduct existing condition surveys. It was not unusual for the airport to hire the same firm that originally delivered the bad data to go back out and field verify what they should have originally delivered years before in the project as-builts!
But this only addresses half of the question. The fact remains that this airport got built, and got built pretty darned well. Was it all built on sloppy CAD data and it's just a happy accident that everything fits? Well, once we understood the as-built issue the rest of the story fell into place. The engineering staff at this airport only does planning and initial design. The final design work and construction drawings are done by contracted engineering firms. Construction drawings are based on a number of sources - initial design, existing condition surveys and final design plans. Construction drawings are what the project engineers and tradesmen take to the field to actually build against These are the drawings that get marked up as modifications are done in the field and it's these drawings that should be used to generate the as-builts. These engineering firms do a very good job of making sure everything fits within the designated project space, and any ties to existing systems - utility lines, roadways, buildings, etc. - are adjusted for in the final design or in the field. But we are back to the old as-built issue. Much of what was actually constructed in the field never makes it back into the airport's master CAD drawing.
So the reality is that the airport got built, but the airport doesn't have a complete and accurate record of what got built.
But I do get the sense that we are over the hump. In the last two years we've noticed an improvement in the consistency of the spatial accuracy of the CAD data being delivered. We still find a good number of attribute data issues (stormwater manholes labeled as sewer manholes, that sort of thing), but as far as spatial accuracy things seem to be greatly improved. I put it down to our engineering staff's increased use of known good data as a quality control check, increased emphasis on as-built delivery, a willingness to let us be part of the quality control check process, increased dialog between the CAD and GIS analysts and an increased dependence on our RTK data collectors to do quick field verification. In fact, our engineering staff is now the #1 hands-on user of our RTK systems. The GIS group also has tight relationships with many of the major construction contractors doing work at the airport and we provide the coordinate system definition files and verified base data for use in project planning. We also offer ourselves up as the data conversion experts and will help contractors verify that their data has been properly moved from one grid system to the other. Over time our insistence on spatial accuracy has 'leaked into' the engineering business processes and workflows here at the airport.
We've shifted the paradigm just a bit and the momentum is in our favor. Geospatial engineering 1, bad data 0. That's the way it should be.
Brian
*SBAS = Space Based Augmentation System. There are several SBAS systems in use around the world. These are satellites in geosynchronous orbit that transmit correction data for the US GPS satellite constellation. The Wide Area Augmentation System (WAAS) is a set of satellites transmitting correction data over the US and the eastern Pacific and is maintained by the FAA and DOT. If your GPS unit can receive and use these signals they will roughly double the accuracy of the position fix your unit provides. You can read more about WAAS and SBAS on Wikipedia.
