Spy Satellites Declassified

A KH-9 Hexagon Imagery Satellite.  The thing's as big
as a Greyhound bus!

On 17 September 2011 the US declassified the KH series of satellites and their mission information.

Guess now I can tell my wife what I was doing for most of those 23 years I was in the Army.

What's not discussed in the story, and I won't go into too much detail until I know for sure it's OK to discuss it in full, is the contribution these satellites made to the DoD's world-wide mapping program.  Suffice to say, without these birds we would not have been able to accurately map the vast territories of the Soviet Union, Eastern Europe, China and all the other hostile places we thought we might have to go fight in.

More to follow...  Maybe.

Brian

The Software I Hate To Love

In the Geospatial Engineering world there is one Big Dog software developer and a pack of miniature chihuahuas snapping at its heels.  The Big Dog is ESRI, developers of the ArcGIS suite of software products.

ESRI dominates the GIS (geospatial information systems) software field in the same way Microsoft dominates the computer operating system field - there are competitors but nobody even comes close to the market share that ESRI developed and has held for decades.

But unlike Microsoft, ESRI didn't get to where it is by being predatory and imposing crushing licensing agreements on its clients.  ESRI got it's market share the old fashioned way - by simply being the best product in the market for the target consumer group.  ArcGIS is the software product that moved the traditional discipline of topography out of the paper map and overlay era and into the computer-based, analysis driven discipline of Geospatial Engineering.

ESRI was started by Jack Dangermond, someone I refer to as a "Birkenstock wearin', Volvo drivin', granola crunchin' hippie."  In the late 1960s and early 70s, building on pioneer work that had been done on early GIS concepts and development in Canada (where the discipline of GIS got its start), Dangermond created a land cover analysis program called ArcInfo and released it as a commercial product in the early 1980s.

Early versions of ArcInfo were hindered by limited computer processing, storage and graphics capability.  Geospatial analysis is very much a visual discipline - you're making maps, after all.  Early desktop hardware simply didn't have the capability and capacity to bring the full visual mapping experience to the user.  Up through the mid 1990s only expensive Unix workstations could handle that level of processing.  This all changed around 1995 when desktop computing power started increasing exponentially with each new processor design while at the same time hardware prices dropped like a brick.  Almost overnight inexpensive desktop computers appeared that could easily handle the processing and graphics demands a software package like ArcInfo placed on them.  I was working as a GIS program manager for the US Army when this hardware revolution hit the field and watched as in less than two years inexpensive desktop PCs caught up with and then quickly surpassed the processing power of the Unix-based Sun, Silicon Graphics and HP  systems we had been relying on.  What also helped was Microsoft's release of WindowsNT at about the same time.  Finally we had a serious network-ready enterprise operating system running on high capacity hardware that didn't make our budget guys weep every time we said we needed to do an upgrade.

ArcInfo is the flagship product of the ESRI line and is extremely powerful software.  But in the 1980s ESRI realized that not everyone needed the processing power of ArcInfo (nor could they afford the nausea-inducing cost of an ArcInfo software license).  ESRI introduced a lightweight version of ArcInfo that included most of the visualization capability of the high end package but left out the heavyweight analysis and data development functionality.  They named it ArcView.  It was priced right - something small organizations and even individuals serious about GIS could afford (if I remember correctly the GSA schedule price for a single ArcView license ran around $600 in 2000).  The vast majority of today's GIS professionals cut their teeth on ArcView.

But ESRI's real contribution to the GIS profession is the development of data types that both support complex spatial analysis and can be shared across different software platforms.  It is Dangermond's vision that GIS-based mapping and analysis solutions should not be a stovepipe, but a shared resource.  This drove ESRI to develop the concept of the geodatabase.  A geodatabase is a collection of data in a standard relational database management system (RDBMS) like Oracle or SQL Server, but the data has very unique spatial values (location in x, y and z coordinates) assigned to it.  This means that GIS software can leverage the spatial values to relate the data in a location context and other RDBMS-based software systems can easily share their information with the geodatabase.   The geodatabase only needs to store GIS-unique features and can pull and do analysis against associated data in another database.

ESRI also developed a version of the geodatabase that does not require a high powered relational database management system as it's foundation.  About a decade ago ESRI introduced the concept of a file-based geodatabase designed for use by small organizations or groups.  The file geodatabase is a simple to create yet powerful and extremely flexible data format that brings most of the power of the relational database and complex data analysis to the desktop machine and the individual user.

But what does the future hold?  ESRI realized long ago that the Internet was the map content delivery vehicle of the future.  Paper maps were headed to obsolescence and what Jack Dangermond describes as the 'rich web map' would quickly become the geospatial data visualization and analysis tool of the future.  He's right, but only very recently has web technology started to catch up with his vision.

For the better part of a decade it was possible to hire professional web developers to create some very nice web mapping applications built on ESRIs early web technology called ArcIMS.  The problem was that those applications were difficult to develop, difficult to maintain, and required a lot of heavy weight back-end web and database server technology.  Only large enterprises and governments could support the hardware, software, development and maintenance costs.  ESRI's web solutions were very much limited by the immature web development technologies available at the time.  It is ESRI's vision that even the average geospatial professional working for a small business or local government should be able to develop, launch and maintain high quality web maps that bring value to the organization they support.  ESRI started laying the groundwork for this vision back with their ArcGIS 9 series of software releases and the development of things like ArcGIS Server and the concept of Map Services.  Two years ago they released ArcGIS 10 that brought a lot of maturity to the concept of integrated and streamlined web mapping using the Microsoft Silverlight and Adobe Flex web development environments, and the launch of ArcGIS Online with its peek into the future concept of 'cloud services' for hosting GIS data, services and web maps.

At it's recent worldwide user's conference ESRI announced the pending release of ArcGIS 10.1 with better integrated and streamlined web development tools.  But ESRI also announced two new developments that are generating a lot of interest.  The first is the announcement that ESRI has partnered with Amazon.com to host robust, enterprise-level cloud services for GIS web mapping, data hosting and application development.  The idea is that an enterprise purchases an ArcGIS Server software license, passes that license over to Amazon and Amazon stands up and maintains the necessary database and web development environment for the enterprise.  This is a huge development because it can free the GIS group supporting the enterprise from the often onerous and restrictive shackles placed on it by their local IT department.

The other announcement was the pending release of the ArcGIS Online Organizational Account program.  The Organizational Account program appears to be targeted as smaller enterprises and groups that don't have the money or need to purchase full-up cloud services like those offered by Amazon.  Under the Organizational Account concept an organization will be able to purchase data and web hosting services from ESRI on a subscription basis.  It is still a 'cloud' model, but on a smaller, more tailorable scale that should allow small organizations to enjoy most of the capabilities of a full-up ArcGIS Server implementation.

The last good thing I need to discuss is another little-known program released this year - the concept of ArcGIS for home or personal use.  ESRI's software licensing fees have escalated to the point that the geospatial professional simply can't afford a copy to use to keep his or her skills sharp.  I noted above that the GSA price for an ArcView license used to run about $600 - a bearable cost if you were serious about GIS.  However, the cost for an ArcView license now hovers around $1,600, far too much for even the serious home user.  This year ESRI announced the ArcGIS for Home Use program.  Anyone can purchase a 1-year license of ArcView for $100, a very reasonable price.  Not only does this $100 include on-line software training and support, but you also get a very extensive suite of add-on modules like 3D Analyst, Spatial Analyst and Geostatistical Analyst.  The total value of the software you get for your $100 subscription comes to over $10,000.  One hell of a deal.  Of course there are restrictions attached to this deal.  The intent of the home use program is just that - you can only use it at home.  You can also only use it for personal development/training purposes or non-profit use.  Still, like I said, it's one hell of a deal.

__________________________________________________________

Now, it's not all rainbows and unicorns when it comes to ArcGIS and ESRI's position in the GIS world.  All this GIS goodness is of little use unless it's leveraged in an environment with clearly defined professional standards.  Nor can you allow a professional discipline to be defined by a software application or be inexorably joined to a piece of software.  This is where ESRI's has failed the geospatial community, and they have failed in ways they can't even visualize from where they sit.

Here's the reality: geospatial engineering is the discipline, the term geospatial information systems - GIS - merely describes the tools geospatial professionals use to do their job.  Where ESRI has failed is in using its industry position and influence to help clearly delineate the difference between the two.  As a result, far too many engineering professionals view geospatial professionals as little more than button pushing software monkeys, one step up from data entry clerks.

Part of the culture Jack Dangermond has fostered and progressed through ESRI is the idea that GIS is for everyone and nobody owns it.  What he is effectively saying is that GIS is the discipline; the tools and the software drive the field, not the other way around.

While community ownership is a noble goal, ESRI's dominance of the field gives lie to that very philosophy.  Effectively, ESRI 'owns' GIS; it is by far the world's largest GIS software developer.  It has either developed or successfully implemented most of the recognized spatial analysis processes in use today.  It's data management features have driven the development of most of the spatial data standards in use today.  The vast majority of geospatial professionals worldwide learned their trade using ArcGIS.

What is lacking, however, is a clear and recognized definition of just what a geospatial professional is.  Dangermond is correct when he claims it's not his role to define what a geospatial professional should be - that is the job of the geospatial field and industry as a whole.  But Dangermond has been the biggest catalyst in the geospatial world for the last 30 years.  He and the resources he commands through ESRI have been in the best position to cajole and coerce the private sector, academia and the government to establish the roles, practices and responsibilities that define Geospatial Engineering as a formal discipline.  He should have been the single biggest champion of the concept of Geospatial Engineering as a professional discipline.  Instead he's been pretty much silent on the whole issue.

It is only in the last few years that the US Department of Labor developed a formal competency model for GIS (GIS, not Geospatial Engineering), and the GIS Professional Certification program is just starting to get its feet on the ground (after a disastrous grandfathering period that allowed perhaps hundreds of clearly unqualified individuals to get a GISP certificate and do damage to the reputation of the geospatial profession that may take years to overcome).  Great, but all this should have happened 20 years ago.

What this means is that Geospatial Engineering is not respected as a professional discipline.  I can tell you from long personal experience that geospatial professionals are looked down upon by other disciplines such as civil engineering and surveying, in large part because there are no testable and enforced standards that define us as a 'profession'.  Guess what - they are right!

_________________________________________________________

Many readers are probably asking themselves "Huh?  What's he getting at here?"  I guess I'd ask the same question myself if I didn't understand the background issues.

I've been a topographer and geospatial engineer for over 30 years.  A few months back I laid out my initial arguments in a post titled In Praise of the Old Topographer.  In that post I made the argument that Geospatial Engineering is just a logical continuation of the older and much respected profession of Topographer.  I also outlined my argument that geospatial information systems, including ArcGIS, are merely the tools that the Geospatial Engineer uses to do his or her job.

With this post my goal was to identify one of the main culprits that is keeping Geospatial Engineering from fully maturing into a recognized profession, a profession with it's own standards, roles and responsibilities.

ArcGIS is that culprit.  On the one hand we have extraordinarily capable software that is almost single handedly responsible for bringing the discipline into the computer age and is poised to bring it fully into the age of  world wide web.  On the other hand, ArcGIS and it's parent company ESRI are almost single handedly responsible for holding the discipline back and keeping it from taking it's rightful place as a profession on par with other engineering disciplines.

For these reasons ArcGIS is the software I hate to love.

Ohio Is Such a Mess

"On the road above the Bell Company's dock, Pennsylvania Route 68 invisibly changes to Ohio Route 38, and trees half hide some signs by the roadside.  The place could hardly be more anonymous.  Even someone familiar with the historical significance of this particular spot, who has traveled several thousand miles to find it, and whose eyes are flickering wildly from the narrow blacktop to the grassy verge between the road and river, can drive a couple of hundred yards past it before hitting the brakes.


The language of the signs is equally undemonstrative.  A stone marker carries a plaque headed "The Point of Beginning" that reads "1112 feet south of this spot was the point of beginning for surveying the public lands of the United States.  There on September 30th, 1785, Thomas Hutchins, first Geographer of the United States, began the Geographer's Line of the Seven Ranges."


There is nothing to suggest that it was here that the United States began to take physical shape, nothing to indicate that from here a grid was laid out across the land that would stretch west to the Pacific Ocean, and north to Canada, and south to the Mexican border, and would cover more than three million square miles, and would create a structure of land ownership unique in history..."


                                                                         - Andro Linklater, "Measuring America" (2002)


In his wonderful book 'Measuring America', author Andro Linklater explains in detail just how it is that the concept of property ownership, and in particular the ownership of land, is the cornerstone of the American republic.  America was founded on the concept of property rights, and there is no greater realization of that concept than the idea that the common man can buy, hold and own land and that he, his family and his descendants will prosper and profit from the ownership and improvement of land.  The land does not belong to a government or a sovereign, but to the people.  It was a radical concept in 1776 and it is still very much a unique concept in the world today.

At the end of the Revolutionary War the weak federal government was cash poor but increasingly land rich.  Under the Articles of Confederation the federal government had no authority to raise revenue through taxation - that power was still retained by the individual states.  But the states were defaulting on their obligations to provide funding for the federal government.  The federal Army had not been paid for months and was on the brink of mutiny.  We had no navy to speak of.  Revolutionary War veterans were holding IOUs from the Continental Congress that were about to come due and our overseas creditors were demanding payment.  In desperation the federal government turned to the only asset it had available - land.

The Treaty of Paris that ended the Revolutionary War gave the new American nation control of a large tract of land west of the Ohio River in what is today southeastern Ohio.  This was really the only tangible asset the federal government owned that was not already claimed by one of the 13 states.  Almost in desperation, the Congress of the Confederation  hit on the idea of land sales as a way to support the struggling federal government.  The idea was simple - divide up the land and sell it for a dollar an acre.  Cash only, no credit!

But how to divide it?  This new nation needed a land measurement and inventory system that was logical, easy to implement and resulted in land parcels that could be easily and quickly sold.  The resulting system, codified in the Land Ordinance of 1785, gave us what we know today as the township and range land survey system.  Conceptually is was simple - divide the land into six miles square sections (townships), then subdivide each township into one mile square sections, then further into quarter sections.  The initial unit of sale was a quarter section of 640 acres.

But where to start?  The Congress of the Confederation set up a committee to study the issue and appointed Thomas Hutchins, a noted military engineer and surveyor, as Geographer of the United States.  It was decided to start the land survey at the point where Pennsylvania's northwestern boundary intersects the Ohio River.  This point became the Point of Beginning for all public land surveys in the United States.

So, on a blustery day in late September, 1785, Thomas Hutchins and his survey party walked down to the banks of the Ohio River, drove a stake in the ground, set their survey instruments up and began to lay out what became known as the Seven Ranges region of Ohio.

From this Point of Beginning Thomas Hutchins set in place the land survey system that would ultimately encompass 75% of the land mass of the United States, clearly establish and define private land ownership and set the stage for the explosive westward expansion of the US in the 19th century.  On September 30th, 1785 Thomas Hutchins literally drove the stake that established the geographic fabric upon which the United States was built.

View Larger Map

Ohio was to be the proving grounds for the township and range survey system.  Like a lot of first tries at anything problems cropped up, adjustments were made and shortcuts were taken.  Part of the problem stemmed from the fact that much of the land in what we today call Ohio was subject to prior claim.  Large areas of  northern Ohio were ceded to Native Americans under various treaties.  Connecticut claimed a large region stretching from present day Sandusky, Ohio east to the Pennsylvania border.  Virginia claimed a large tract in the south to use to compensate her veterans.  Other bits and pieces here and there were set aside.  Ohio was a patchwork quilt of land claims, set-asides, treaty lands and private holdings.

Ohio Land Claims - 1800's

But very quickly another series of problems popped up.  Congress was pressured by speculators to sell large chunks of land.  Congress saw this as a way to generate quick cash - sell land at a slight discount for immediate payment and let the speculators carry the cost of the land surveys.  The land speculators saw it as a road to riches - if they could sell fast.  But before any land could be sold it had to be surveyed and the surveys registered.  That meant the surveys needed to be done fast.  Accuracy be damned!

In the 18th century anyone with rudimentary math skills and who could afford a surveyor's compass and chain could call themselves a surveyor, and many did.  Since surveyors at the time were paid by the mile the faster they worked the more they got paid.  This meant the surveys were sloppy and niceties like calculating the local differences between true north and magnetic north were either not done as often as required or simply not done at all.

As a result, a lot of Ohio's township and range section lines take off at odd angles and don't quite form square parcels.  Eventually the errors accumulated and corrections had to be made.  Often it was the simple expedient of offsetting a north-south range line at the start of the next township line.  Since roads in Ohio tended to follow the township and range section boundaries this led to the quirky (and often dangerous) tendency of country roads ending at a T-intersections for no apparent reason, then picking up again about 100 feet east or west of the end point.  These little jogs are a modern reflection of the corrections the surveyors were forced to build into their work over 200 years ago.

Other times the errors were so extreme that there was really no way to correct them and the government was just forced to incorporate the errors into the public record as-is:

The intersection of surveys for the Symmes Purchase, Virginia Military Reserve
and standard Public Land Survey areas.  There are about three different
interpretations of true north indicated by these township and range layouts!

So there you have it.  Ohio is a darned mess.  But a fascinating mess that leaves us the physical traces of the birth of the survey system that made westward expansion possible.

Brian