Drones, swarms, artificial intelligence and virtual reality now rule

This e-newsletter marks the sun setting on the career of GeoIntelligence Insider columnist and contributing editor Art Kalinski, long active in geospatial circles. Art is handing the journalistic reins to William Tewelow, a former intelligence specialist with the U.S. Navy. He also served a special assignment at the U.S. Department of Transportation and is currently  a manager with the U.S. Federal Aviation Administration (FAA). The following discussion muses about the direction of our geospatial tradecraft in the coming years.

Tewelow’s Geointelligence column will alternate monthly with Mike Jones’ Defense PNT column. Every issue of this newsletter also brings you the latest news on both expanding fronts, hardware and software, in the defense and security realm.

Passing the torch

A conversation between retiring GeoIntelligence editor Art Kalinski, GISP, and his successor, William Tewelow, GISP.

Counter-drone technology research and operation efforts are helping to prepare for the potential threat of drone swarms. This Lockheed Martin concept is explored in “Technology That Counters Drone Swarms.”  
(Image: Lockheed Martin)

Art: A factor of life is that you just can’t predict with certainty where technology is headed or what the issues will be. Several weeks ago I attended a UAV (unmanned aerial vehicle) users group that seemed like a glimpse of the Wild West. I thought the key discussion topics would be the latest technology which seems to change almost weekly. That was not the case. The key discussions were the same as the early days of GIS: legal liability, jurisdictions, data ownership, administrative issues and so on, as the FAA changes the rules and regulations. I began to see that there will be more employment opportunities for lawyers than techies and pilots.

Although most of the attendees at the conference were geospatial/first responder users, I’ve learned of a growing community of non-traditional users such as power line inspectors and even tower painters. Yes, a tower painting company using tethered UAVs to paint in dangerous locations such as bridges, tanks and towers. At first I thought that would be difficult but by using a tethered system, heavy batteries are eliminated and bigger UAVs will permit heavier hoses, etc. Although the FAA had little interest in tethered balloons and UAVs, that could change as the size, altitude and potential crash radius expands.

This is just one example of how our technology is moving in many unplanned directions, everything from UAVs with new ways of collecting imagery to more exotic topics such as artificial intelligence, machine learning, augmented reality, virtual reality (VR) and gamification. The entire trade craft is evolving in ways I don’t think anybody could have predicted just a few years ago, much less a decade ago. It will be interesting to see where it goes.

William: You mentioned several new technologies mixing things up. Small sats are also making a significant impact. They are game changers. Imagine the traditional 2D satellite imagery base map. With the tech we have now 3D base maps are possible and have been for a long time with DEMS. Now, small sat imagery can be overlaid atop the 3D basemaps providing fresh imagery with periodicities of 24 hours or less, including multi-spectral. I’ve even seen small sat video overlays. That’s game changing.

Art: Not only small sats but the work being done with drone swarms is an exciting new area. If data needs to be collected in a hostile area or if flight clearance in a busy air corridor was limited, a swarm of maybe 100 drones could capture a wide swath of data at high resolutions, with multiple spectrums in very short order.

Virtual reality and augmented reality are two areas which have always intrigued me and they are coming to fruition. When I started out in GIS, I told people working for me to do the best we could building our regional database because at some point we would have construction workers wearing goggles that will augment the reality of what they are seeing with GIS data. That day is here and people are finally starting to do that. So, the entire concept of AR (augmented reality) and VR I find extremely exciting.

William: Yes, absolutely, in fact I believe they used drone swarms in the hostage situation in Algiers in 2014. And you are right. VR and AR use for first responders is especially critical. Say you’re a disaster relief person at night in a smoke-filled building. Night vision and thermal vision reach a practical limit. However, combining the vision devices with AR you may be able to navigate through the environment well enough. AR shows at least what should be there so accurate and timely data is critical. I can see this is where things are headed. Additionally, more mundane tasks such as design and modifications of structures could be enhanced. Work is already being done using AR and VR to facilitate the design process.

Art: VR started as fun and games but to me serious use VR is where the new and exciting action will be for geospatial practitioners. Virtual trainers I saw recently included small arms and artillery trainers for the military to a less exciting but equally impressive virtual trainer to teach arc welding and even spray painting a car. Significantly more critical was a VR surgical trainer where the muscle learning and hand eye coordination required is significant. It’s obviously quicker, easier and cheaper to practice virtually than on a real human or real equipment and the VR environment is just one short step away from surgical robots operating in remote locations.

William: Right, and you are touching on gamification, where VR can be used to train toward certain results. Take the military, for example, gamification can save all manner of resources – bullets, people, buildings the environment. Additionally, users can also scale up or down. The VR environment doesn’t just mean roads, maps or mountains. You can get down to a micro level.

Art: It’s interesting you mention that. One aha! moment I had years ago was at the first Esri User Conference in San Diego. I went through the map gallery. A lot of examples of different uses for GIS were on display and one person had created a map of the human circulatory system using ArcView Network Analyst. Looking at it I thought, of course, whether its blood vessels or interstate highways, it’s a network; so, this individual was using a geospatial tool to map the entire blood vessel network.

William: We are now in the realm of nano technology, where things are small enough to work on blood cells. Imagine steering straight towards a cancer cell by pinpointing a location in relation to the body.

Art: It’s almost like the movie “Fantastic Voyage” except in a virtual environment.

William: Yes. Being able to put the nano technology right where it needs to be means entering an age of focused medicine so that a pill releasing nano particulates doesn’t impact the whole body just the area needed. That’s another discussion in itself. Of course, all of that will have to be controlled, mapped and understood in terms of what it’s doing and where it’s going in relation to the body.

Art: One area that is very concerning to me is cyber. All of this stuff we’ve talked about is very vulnerable whether stealing data or doing damage to a society. Thwarting those threats is getting into machine learning, AI and other topics bordering science fiction.

William: Yes, and you are getting into artificial intelligence. IBM, Google, Amazon, Facebook, Microsoft and our three letter agencies are all investing into the world of Geospatial AI. I am not sure where GeoAI will end up but we can already see some hints. The amount of geospatial data is overwhelming and imagine the explosion happening with IoT. How will all that be managed? GeoAI will do pattern recognition so in effect GeoAI is assisted intelligence.

As an example let’s consider a high profile event. There are 1000’s of parameters that need to be tracked such as surveillance cameras, social media feeds, a suite of sensors, etc. Identifying critical patterns is crucial so Geospatial Assisted Intelligence can monitor the parameters of the event then notify the analyst when a triggering event occurs. That’s where the technology is going where it has to go.

Art: One has to have a truly dizzying intellect to keep track of all aspects of our technology. At the last NGA (National Geospatial-Intelligence Agency) conference, Director Cardillo talked extensively about open systems and open sources which are growing directions for the agency. I believe it’s good to get away from everything being “inside the tent” and taking advantage of the extensive capability in the broad geospatial community. The stated goal is to be able to develop needed geospatial tools, in an unclassified environment, not in months or weeks but literally in minutes to provide those tools to analysts in the field. Things have come so far from the day I was just happy to get a pen plotter to print a simple map.

William: It really is mind boggling. I was there when full motion video began and that was dynamic 2D imagery but then it made the jump to 3D dynamic imagery and my circuits fried. I still have a hard time comprehending all the data crunching that goes into it and it is now almost standard. The future is amazing. That is the direction I want to continue to explore.

Editor’s Note  from Art: Since we are approaching Mother’s Day and Memorial Day, I’d like to call your attention to my best column addressing mothers and the military.