The State of Mobile GIS Software

Over the past six months or so I've been doing a lot of casual testing of the various mobile GIS platforms available on the market today.  Right now is an ideal time to discuss the offerings because just in the past week we've had an update to a key application in this arena (Collector for ArcGIS), we are on the verge of having an interesting new hardware player enter the market (Garmin's soon to be released Monterra handheld GPS) and several vendors are dropping serious hints about where they see their products headed in 2014.

I was ramping up to do a lengthy blog post on this when I dropped by Alex Mahrou's always interesting RockyMountainGeo GIS blog and was surprised to see he had already done all my work for me.  Back in October Alex did a great overview of the current offerings in a posting titled Enterprise Mobile GIS Software Functionality.  All I can do is add minor updates to some of his information and add a few of my own observations.

I like the switchboard analogy!

The single biggest update is the newest version of Collector for ArcGIS (version 10.2) that was released last week for the iOS and Android platforms.  This version addresses one of the two biggest complaints about earlier versions of Collector - polyline and polygon data collection.  It also offers an improved user interface and well thought out workflows.  While the Android version still has some rough edges, the iOS version is a polished, smoothly functioning app that reflects ESRI's mature experience in developing for Apple's mobile operating system.  It is a very good app.

Where ESRI seems to be unnecessarily holding back is off-line data collection and editing, and data synchronization.  As Alex notes, ESRI informally promised that this feature would 'absolutely, positively' be incorporated into Collector before the end of 2013.  It now looks like we'll have to wait until sometime in early 2014, when ESRI plans for a significant overall upgrade to Collector, perhaps better positioning it within their enterprise software offerings.  In my opinion ESRI missed the ball on this one.  Incorporating off-line data storage and editing in the iOS and Android operating systems isn't hard to do; Trimble had it available almost six months ago in their initial release of TerraFlex.  I understand there are other issues at play here - background map data caching and the incorporation of operational layers (both something Trimble's offering lacks), but ESRI still could have incorporated basic off-line functionality in this new release and just built on it for the upcoming major release.

Trimble's TerraFlex is an app I tested back in June and was initially very impressed.  Where most of ESRI's mobile offerings (Collector, ArcGIS App, ArcGIS for Windows Mobile) require some expensive back-end infrastructure - ArcGIS Online, Portal or ArcGIS for Server - TerraFlex offers a far simpler mobile solution paradigm.  Everything is cloud based and single fee.  You pay your money and you get everything TerraFlex has to offer, and all for a relatively paltry price as compared to ESRI's mobile solutions in the same marketplace.  Of course, this easier to use solution comes at a cost (pun intended) - what the initial release of TerraFlex didn't offer was pretty extensive; no background map caching, no data editing either on the device or in the desktop interface, no operational layers, and some very limited data export options.  On the other hand, what TerraFlex does offer is pretty impressive given the price: off line data storage and sync, mature and stable apps not just on the iOS and Android platforms, but Trimble also had an app available for the Windows Embedded Handheld OS right out of the gate.  Trimble wasn't about to leave out the thousands of Trimble customers running their Juno handhelds who are still stuck with a dying Windows OS.  Kudos to Trimble on this.

Trimble indicates many of these shortcomings will be addressed in 2014, and Trimble seems poised to leverage what they do best - allow TerraFlex to incorporate high precision GNSS positions (including RTK-based solutions) into the data collection stream.  This could turn TerraFlex from a mere mapping grade data collector into a serious high precision data collection tool.

In his blog post Alex discusses Fulcrum.  To be honest, this is an application I've known about but have not had a chance to test.  Looks like I'll have to take it for a spin sometime soon.

So as 2013 draws to a close where are we at with mobile GIS solutions?  The best analogy I can think of is that of a ballplayer with a lot of potential who's just been called up to the majors.  His batting stats are getting better with each game, but he still has problems connecting with the ball.  The potential is there, he just needs more time. So it is with mobile GIS apps.  Most are still somewhat of a 'swing and a miss', but they are getting close to smacking the ball out of the park.  Whether it's off-line data collection with ESRI's offerings or TerraFlex's incorporation of cached maps, in-app editing or incorporation of high precision position feeds, 2014 is starting to look like the field will really mature and we'll get closer to the full promise of mobile GIS.

It'll be an interesting year!

Brian

  

ESRI Southeast User Conference

I just got back from the 2013 ESRI Southeast User Conference in Jacksonville, FL.  Here's my overview of all the key topics presented or discussed at the conference:

ArcGIS Online, ArcGIS Online, ArcGIS Online, Cloud, ArcGIS Online, ArcGIS Online,ArcGIS Online, Hosted Services,ArcGIS Online, ArcGIS Online, Mobile with ArcGIS Online, Cloud, ArcGIS Online, ArcGIS for Desktop, ArcGIS Online, ArcGIS Online, ArcGIS Online, Cloud, Geoprocessing In the Cloud, ArcGIS Online, ArcGIS Online, ArcGIS for Server, ArcGIS Online.

And to wrap it all up: ArcGIS Online.

ESRI, you are sounding like a broken record.

Busting Brush with ArcGIS for Windows Mobile

Huh?  What?

OK, at work we are testing a new software package - ArcGIS for Windows Mobile.  The name is a mashup of two software package names - ArcGIS (ESRI) and Windows Mobile (Microsoft).  Yes, it is a cumbersome name.  Really, really cumbersome.

It is nothing more than a lite version of ESRI's ArcGIS software designed specifically to run on the Windows Mobile OS.  Never heard of Windows Mobile?  Don't worry, you haven't missed much.  It's an operating system that saw its widest use on mobile phones.  Notice I didn't say 'smart' phones, because Windows Mobile was (and still is) an awful operating system that made every piece of hardware it touched dumber.  Same for its users.  I'm sure when Steve Jobs was yelling at his software engineers during the early stages of Apple's iOS development he forced them to use Windows Mobile phones so they clearly understood what  iOS would not end up looking like.

Microsoft has moved on and introduced their new Windows Phone OS that is based on the Windows 8 platform.  However, their Windows Mobile OS hangs on in a few interesting places.  It's used a lot in 'embedded' applications, computers running inside of other things that don't look like computers.  For example, the Microsoft Sync system that controls just about everything in my new Ford F-150 is a version of Windows Mobile.

Another place Windows Mobile has achieved a lot of market penetration is in the surveying and GPS-based data collection market.  These are highly customized hardware systems that are more than mobile phones but less that full-fledged computers, robust devices dedicated to a narrow set of field data collection tasks.  Virtually all manufacturers of surveying and GPS-based field data collection systems use Windows Mobile - there's really nothing else available today that meets the need.

Two of the devices in this picture run on Windows Mobile.
The rest are easy to use.

Because of this ESRI still develops a lot of software to run on the Windows Mobile OS (now up to version 6.5 and renamed Windows Embedded Handheld).  ArcPad, ESRI's flagship field data collection package, has been running on the Windows Mobile platform for almost a decade.  A few years back ESRI released a version of its server software (called, naturally, ArcGIS for Server) that allowed the user to develop GIS applications that run on Windows Mobile handhelds and consume map services hosted on a local ArcGIS for Server instance.  We tested this at work but came away with the impression that the whole system required more care and feeding than we were able to provide.  In addition, our IT department was never willing to cooperate and provide a way to authenticate these mobile devices (Trimble Junos) on our organization's domain so they could 'see' our internal GIS software servers.

Instead, we've spent the last year developing mobile GIS applications to run on Apple iOS devices - iPads and iPhones - and leveraging the new hosted map service concept available through ESRI's ArcGIS Online cloud service.  The entire system works amazingly well and our users like the idea of collecting GIS data using an iPad (plus they can play Angry Birds in their down time).  But as we tested and developed in this environment we quickly bumped up against another roadblock - again, our IT department.  They repeatedly refuse to approve the purchase of iPads for the user base.  No real explanation - they say no just because they can.  We were stuck and desperate for an alternative.

This all changed in late February with the release of ArcGIS for Windows Mobile 10.1.1.  The #1 change with this new version is the ability of the software to connect to an ArcGIS Online subscription account and use a hosted (cloud) feature service as a data layer.  We no longer are forced to use our local instance of ArcGIS for Server as the data source.  Since the data is hosted in the cloud all we need is a wi-fi connection and an ArcGIS Online subscription account to get to our data.  The need to have our mobile devices authenticate on our organization's network is gone.  Our dependency on our local IT department is severed.

The other big benefit that ArcGIS for Windows Mobile brings is the ability to do disconnected editing when there is no wi-fi signal available.  This was always a concern with the iOS devices, which require an 'always on' internet connection when being used to collect data.  ArcGIS for Windows Mobile works differently in that the application places a copy of the map layer's database directly on the mobile device when it is first downloaded from ArcGIS Online.  In the field a wi-fi connection to the internet is not necessary - all the newly collected data and edits are stored on the mobile device.  When the user get back under wi-fi coverage they can do a synchronization of this new data with the master database stored in the cloud on ArcGIS Online.  The new data is pushed up to the master database via a wi-fi connection to the internet - any wi-fi connection to the internet; in the office, in Starbucks, in McDonalds, wherever they can get a signal.  Simple, slick and robust.

But why Trimble Junos?  While our IT department balks at the purchase of iPads, they have no problem with us purchasing Junos even though the current generation of Junos cost about twice as much as an iPad!.  IT views them as dedicated field survey devices and allows us to buy as many as our budget can support.  Over the past two years we've purchased seven of them so we have plenty of hardware available to put into the hands of our users.

The Trimble fairy barfed on my desk

So like any good Geospatial weenie I figured I needed to test my applications before unleashing them on the unsuspecting public.  To acquaint myself with the workflows embedded in ArcGIS for Windows Mobile I set up a project to collect data during one of my favorite activities - hiking.  Then it was time to take ArcGIS for Windows Mobile to the wilds of suburban Atlanta for the ultimate test: can a befuddled 56 year old make sense of this mobile thing and actually collect useful data?

Everything I need to survive - water, food, first aid kit and
ArcGIS for Windows Mobile on a Trimble Juno

So how did everything work?  Pretty darned good.  In fact, better than I expected.  The simplified workflows in the ArcGIS for Windows Mobile interface make collecting data almost foolproof.  I only had a chance to capture about two miles of trail information and some points of interest, but it was enough to convince me this mobile GIS interface will work just fine for most of our user base.  Of course it's an imperfect world, and so is this application.  I'd love the ability to collect photos while remaining in the GPS data streaming mode, and being the GPS geek that I am I'd like a better GPS performance interface (similar to what you get with ArcPad or Trimble's TerraSync), but I also understand this package is designed for simplified data collection by non-GIS personnel, so I can live with the lack of GPS performance data.

Yellow SO clashes with my woods gear!

Of course all this simplification also serves as a straightjacket.  What you give up with the ArcGIS for Windows Mobile interface is the ability to make on-the-fly changes to your project - add new data types, change symbology, adjust GPS performance parameters, do complex searches, buffers, etc.  It's a trade-off;  reduced complexity =  fewer options.  It's a trade-off my organization can live with.

ArcGIS for Windows Mobile up close and personal.
A simple interface that works well on devices with small screens.

The REAL elephant in the room is the overall cost of this capability.  ArcGIS for Windows Mobile is an enterprise level solution for enterprise level projects.  As configured this project relies on an ArcGIS Online subscription account as the data hosting platform and ArcGIS Mobile deployment licenses tied to a very expensive ArcGIS enterprise license.  This equates to about $55,000 in licensing costs (toss in another $1,200 or so for the Trimble Juno).

Obviously this is not for the little guy.  But it should be, and it can be!  Let's say you are a Geospatial geek like me.  Right now, today, if you participate in the ArcGIS for Home Use Program you get one free user license for  ArcGIS for Windows Mobile.  The software package includes a toolset that allows you to stage all your data on your local computer instead of on ArcGIS for Server or in the ArcGIS Online cloud.  You don't get to wirelessly update your data - you have to connect the device to your computer via USB to do the synchronization process - but the rest of the workflows are the same.  A great (and cheap) way to test this capability for yourself.  You just have to go find a compatible Windows Mobile device (check eBay, there's plenty for sale out there).

Looking into the future I see ESRI opening the ArcGIS Online subscription program up to market segments that don't need and can't afford to buy into the service at the current enterprise-like levels.  It's almost an inevitability.  There's a lot of emerging competition in the cloud GIS services arena, and companies leveraging some of the better Open Source GIS tools will start to provide low cost cloud hosting services in direct competition with ESRI.  Of course ESRI has the market share and clout, but the 'cloud' is a huge space, there's a huge potential market, and ESRI can't control it all.  Price competition will soon have its inevitable impact.  Plus, ESRI is pushing 'the cloud' like it's the Second Coming and at some point they will realize their service availability will have to mesh with their message.  As they move more and more capability to the cloud ESRI will have to start offering low cost services for the little guy.

Maybe not this year, maybe not next year, but soon.

In the meantime use what's available and get moving on Mobile!

Brian

The Software I Hate To Love

In the Geospatial Engineering world there is one Big Dog software developer and a pack of miniature chihuahuas snapping at its heels.  The Big Dog is ESRI, developers of the ArcGIS suite of software products.

ESRI dominates the GIS (geospatial information systems) software field in the same way Microsoft dominates the computer operating system field - there are competitors but nobody even comes close to the market share that ESRI developed and has held for decades.

But unlike Microsoft, ESRI didn't get to where it is by being predatory and imposing crushing licensing agreements on its clients.  ESRI got it's market share the old fashioned way - by simply being the best product in the market for the target consumer group.  ArcGIS is the software product that moved the traditional discipline of topography out of the paper map and overlay era and into the computer-based, analysis driven discipline of Geospatial Engineering.

ESRI was started by Jack Dangermond, someone I refer to as a "Birkenstock wearin', Volvo drivin', granola crunchin' hippie."  In the late 1960s and early 70s, building on pioneer work that had been done on early GIS concepts and development in Canada (where the discipline of GIS got its start), Dangermond created a land cover analysis program called ArcInfo and released it as a commercial product in the early 1980s.

Early versions of ArcInfo were hindered by limited computer processing, storage and graphics capability.  Geospatial analysis is very much a visual discipline - you're making maps, after all.  Early desktop hardware simply didn't have the capability and capacity to bring the full visual mapping experience to the user.  Up through the mid 1990s only expensive Unix workstations could handle that level of processing.  This all changed around 1995 when desktop computing power started increasing exponentially with each new processor design while at the same time hardware prices dropped like a brick.  Almost overnight inexpensive desktop computers appeared that could easily handle the processing and graphics demands a software package like ArcInfo placed on them.  I was working as a GIS program manager for the US Army when this hardware revolution hit the field and watched as in less than two years inexpensive desktop PCs caught up with and then quickly surpassed the processing power of the Unix-based Sun, Silicon Graphics and HP  systems we had been relying on.  What also helped was Microsoft's release of WindowsNT at about the same time.  Finally we had a serious network-ready enterprise operating system running on high capacity hardware that didn't make our budget guys weep every time we said we needed to do an upgrade.

ArcInfo is the flagship product of the ESRI line and is extremely powerful software.  But in the 1980s ESRI realized that not everyone needed the processing power of ArcInfo (nor could they afford the nausea-inducing cost of an ArcInfo software license).  ESRI introduced a lightweight version of ArcInfo that included most of the visualization capability of the high end package but left out the heavyweight analysis and data development functionality.  They named it ArcView.  It was priced right - something small organizations and even individuals serious about GIS could afford (if I remember correctly the GSA schedule price for a single ArcView license ran around $600 in 2000).  The vast majority of today's GIS professionals cut their teeth on ArcView.

But ESRI's real contribution to the GIS profession is the development of data types that both support complex spatial analysis and can be shared across different software platforms.  It is Dangermond's vision that GIS-based mapping and analysis solutions should not be a stovepipe, but a shared resource.  This drove ESRI to develop the concept of the geodatabase.  A geodatabase is a collection of data in a standard relational database management system (RDBMS) like Oracle or SQL Server, but the data has very unique spatial values (location in x, y and z coordinates) assigned to it.  This means that GIS software can leverage the spatial values to relate the data in a location context and other RDBMS-based software systems can easily share their information with the geodatabase.   The geodatabase only needs to store GIS-unique features and can pull and do analysis against associated data in another database.

ESRI also developed a version of the geodatabase that does not require a high powered relational database management system as it's foundation.  About a decade ago ESRI introduced the concept of a file-based geodatabase designed for use by small organizations or groups.  The file geodatabase is a simple to create yet powerful and extremely flexible data format that brings most of the power of the relational database and complex data analysis to the desktop machine and the individual user.

But what does the future hold?  ESRI realized long ago that the Internet was the map content delivery vehicle of the future.  Paper maps were headed to obsolescence and what Jack Dangermond describes as the 'rich web map' would quickly become the geospatial data visualization and analysis tool of the future.  He's right, but only very recently has web technology started to catch up with his vision.

For the better part of a decade it was possible to hire professional web developers to create some very nice web mapping applications built on ESRIs early web technology called ArcIMS.  The problem was that those applications were difficult to develop, difficult to maintain, and required a lot of heavy weight back-end web and database server technology.  Only large enterprises and governments could support the hardware, software, development and maintenance costs.  ESRI's web solutions were very much limited by the immature web development technologies available at the time.  It is ESRI's vision that even the average geospatial professional working for a small business or local government should be able to develop, launch and maintain high quality web maps that bring value to the organization they support.  ESRI started laying the groundwork for this vision back with their ArcGIS 9 series of software releases and the development of things like ArcGIS Server and the concept of Map Services.  Two years ago they released ArcGIS 10 that brought a lot of maturity to the concept of integrated and streamlined web mapping using the Microsoft Silverlight and Adobe Flex web development environments, and the launch of ArcGIS Online with its peek into the future concept of 'cloud services' for hosting GIS data, services and web maps.

At it's recent worldwide user's conference ESRI announced the pending release of ArcGIS 10.1 with better integrated and streamlined web development tools.  But ESRI also announced two new developments that are generating a lot of interest.  The first is the announcement that ESRI has partnered with Amazon.com to host robust, enterprise-level cloud services for GIS web mapping, data hosting and application development.  The idea is that an enterprise purchases an ArcGIS Server software license, passes that license over to Amazon and Amazon stands up and maintains the necessary database and web development environment for the enterprise.  This is a huge development because it can free the GIS group supporting the enterprise from the often onerous and restrictive shackles placed on it by their local IT department.

The other announcement was the pending release of the ArcGIS Online Organizational Account program.  The Organizational Account program appears to be targeted as smaller enterprises and groups that don't have the money or need to purchase full-up cloud services like those offered by Amazon.  Under the Organizational Account concept an organization will be able to purchase data and web hosting services from ESRI on a subscription basis.  It is still a 'cloud' model, but on a smaller, more tailorable scale that should allow small organizations to enjoy most of the capabilities of a full-up ArcGIS Server implementation.

The last good thing I need to discuss is another little-known program released this year - the concept of ArcGIS for home or personal use.  ESRI's software licensing fees have escalated to the point that the geospatial professional simply can't afford a copy to use to keep his or her skills sharp.  I noted above that the GSA price for an ArcView license used to run about $600 - a bearable cost if you were serious about GIS.  However, the cost for an ArcView license now hovers around $1,600, far too much for even the serious home user.  This year ESRI announced the ArcGIS for Home Use program.  Anyone can purchase a 1-year license of ArcView for $100, a very reasonable price.  Not only does this $100 include on-line software training and support, but you also get a very extensive suite of add-on modules like 3D Analyst, Spatial Analyst and Geostatistical Analyst.  The total value of the software you get for your $100 subscription comes to over $10,000.  One hell of a deal.  Of course there are restrictions attached to this deal.  The intent of the home use program is just that - you can only use it at home.  You can also only use it for personal development/training purposes or non-profit use.  Still, like I said, it's one hell of a deal.

__________________________________________________________

Now, it's not all rainbows and unicorns when it comes to ArcGIS and ESRI's position in the GIS world.  All this GIS goodness is of little use unless it's leveraged in an environment with clearly defined professional standards.  Nor can you allow a professional discipline to be defined by a software application or be inexorably joined to a piece of software.  This is where ESRI's has failed the geospatial community, and they have failed in ways they can't even visualize from where they sit.

Here's the reality: geospatial engineering is the discipline, the term geospatial information systems - GIS - merely describes the tools geospatial professionals use to do their job.  Where ESRI has failed is in using its industry position and influence to help clearly delineate the difference between the two.  As a result, far too many engineering professionals view geospatial professionals as little more than button pushing software monkeys, one step up from data entry clerks.

Part of the culture Jack Dangermond has fostered and progressed through ESRI is the idea that GIS is for everyone and nobody owns it.  What he is effectively saying is that GIS is the discipline; the tools and the software drive the field, not the other way around.

While community ownership is a noble goal, ESRI's dominance of the field gives lie to that very philosophy.  Effectively, ESRI 'owns' GIS; it is by far the world's largest GIS software developer.  It has either developed or successfully implemented most of the recognized spatial analysis processes in use today.  It's data management features have driven the development of most of the spatial data standards in use today.  The vast majority of geospatial professionals worldwide learned their trade using ArcGIS.

What is lacking, however, is a clear and recognized definition of just what a geospatial professional is.  Dangermond is correct when he claims it's not his role to define what a geospatial professional should be - that is the job of the geospatial field and industry as a whole.  But Dangermond has been the biggest catalyst in the geospatial world for the last 30 years.  He and the resources he commands through ESRI have been in the best position to cajole and coerce the private sector, academia and the government to establish the roles, practices and responsibilities that define Geospatial Engineering as a formal discipline.  He should have been the single biggest champion of the concept of Geospatial Engineering as a professional discipline.  Instead he's been pretty much silent on the whole issue.

It is only in the last few years that the US Department of Labor developed a formal competency model for GIS (GIS, not Geospatial Engineering), and the GIS Professional Certification program is just starting to get its feet on the ground (after a disastrous grandfathering period that allowed perhaps hundreds of clearly unqualified individuals to get a GISP certificate and do damage to the reputation of the geospatial profession that may take years to overcome).  Great, but all this should have happened 20 years ago.

What this means is that Geospatial Engineering is not respected as a professional discipline.  I can tell you from long personal experience that geospatial professionals are looked down upon by other disciplines such as civil engineering and surveying, in large part because there are no testable and enforced standards that define us as a 'profession'.  Guess what - they are right!

_________________________________________________________

Many readers are probably asking themselves "Huh?  What's he getting at here?"  I guess I'd ask the same question myself if I didn't understand the background issues.

I've been a topographer and geospatial engineer for over 30 years.  A few months back I laid out my initial arguments in a post titled In Praise of the Old Topographer.  In that post I made the argument that Geospatial Engineering is just a logical continuation of the older and much respected profession of Topographer.  I also outlined my argument that geospatial information systems, including ArcGIS, are merely the tools that the Geospatial Engineer uses to do his or her job.

With this post my goal was to identify one of the main culprits that is keeping Geospatial Engineering from fully maturing into a recognized profession, a profession with it's own standards, roles and responsibilities.

ArcGIS is that culprit.  On the one hand we have extraordinarily capable software that is almost single handedly responsible for bringing the discipline into the computer age and is poised to bring it fully into the age of  world wide web.  On the other hand, ArcGIS and it's parent company ESRI are almost single handedly responsible for holding the discipline back and keeping it from taking it's rightful place as a profession on par with other engineering disciplines.

For these reasons ArcGIS is the software I hate to love.