Data infrastructure optimization software
Data integration and quality software
Data availability and security software
Cloud solutions

Big Data Meets the Little Drone

When Amazon’s Jeff Bezos surprised 60 Minutes reporter Charlie Rose with his concept of using drones to deliver Amazon goods, he didn’t highlight Amazon’s other big business — Amazon Web Services (AWS).

He should have. Drones like the ones Bezos showed Rose may one day add even more to the Amazon bottom line by adding to the already substantial revenues of AWS.

An Amazon Future Capability: Drone-Delivered Packages

Born of a 1992 DARPA Dream

Products like those from the company Parrot represent a generation of small, capable drones originally envisioned as mobile microrobots in a farsighted 1992 DARPA workshop.

The Parrot senseFly claims to have, along with “live mission monitoring” and “rapid image processing,” onboard artificial intelligence that allows its autopilot software to control its own flight – even in the case of communication loss. The senseFly smart sensors are claimed to facilitate building maps and 3D models down to 5cm precision. Using a swarm of senseFly eBee drones, the company was able to use its own Pix4D software and 3,000 eBee images to create a 3D model of the Matterhorn. Missions can be planned, monitored and adjusted mid-flight with 3D maps using senseFly’s eMotion 2 software.

All this processing entails a steady diet of data.

Buzz into a Big Data Barn

Applications for drones, beyond the military unmanned aerial vehicle (UAV) systems that brought the sector to life over the last two decades, include agriculture, traffic management, wildlife management, meteorology, disaster relief and environmental compliance (via LiveScience).

Further commercial adoption of civilian drones in the US is tied to the FAA Modernization and Reform Act of 2012. So far, the FAA only allows limited use of US airspace for drones in a very few locations.

Because agricultural lands are primarily rural, UAV operations in USA agricultural airspace could see earliest FAA approval. Agriculture uses could include crop spraying, seeding, weeding, herding, disease monitoring and biological pest control. The Association of Unmanned Vehicle Systems International commissioned a March 2013 report that estimated the economic impact of UAV technology in the US to be around $82B between 2015 and 2025. Industry analyst Dan Kara goes further:

The combination of Big Data analytics and drone technology provides a powerful new tool for precision agriculture. The concept is in its infancy, and there is much work to do, including obtaining regulatory approval for drone flyovers.

Using drones to control insect populations – pests, pest predators or pest-fighting bacteria – is new, but Japanese farmers have used drones for cropdusting for two decades. With that success in mind, some research has explored this possible ag-future.

Cornell’s Elson Shields is working in aerobiology, where drones can be used to measure the air above and between crops to detect anomalies, measure pathogens and optimize the use of pesticides. Shields, Donald Aylor and Virginia Tech’s David Schmale collaborated on a USDA-sponsored project which includes a collection of nine drones equipped with collection plates. When flown over potato fields in Virginia, the drones could look for the presence of the oomycete P.infestans – the microorganism responsible for a 1996 outbreak of potato blight in eastern Virginia.

Networked UAVs

Data volume will be amplified when drones are used in tandem with one another. The cancelled Raytheon SilentEyes project was envisioned as a small drone that is launched from a larger drone, such as the Predator UAV. The tube-shaped slave drone (only 18″ x 3″) could contain a video camera, jammer payload and even a small warhead if desired.

Similarly, the aerobiology experiments conducted by Techy, Schmale and Woolsey demonstrated that “coordinating the flight of two or more UAVs ensures that the vehicles enter, sample, and exit a spore plume at consistent times.”

Still more exotic: miniaturized, coordinated swarm drones could perform monitoring or very small materials distribution tasks on a scale not currently seen.

Big Data Gulp or Glut?

Even without the expected onslaught of civilian drone data, military applications demonstrate that a data glut of unmanageable proportions is upon drone data consumers. In 2009, the Air Force shot 24 years’ worth of video over Iraq and Afghanistan. That was before updated Reaper drones were deployed which could record 10 video feeds simultaneously, and later versions of the drone are capable of 65 feeds.

Little Drones: Lightweight, Low Cost, Safer

While software to intelligently reason directly from video feeds is still in a research phase, drone data is generally being saved until that technology becomes available. For now, data is generally relayed to ground stations for handling, but UAV improvements include increased onboard processing, encryption, compression and fusion. These improvements could increase data velocity, but in the meantime, data is likely to be stored and processed in resources like Amazon’s EMR, using tools like Syncsort’s Ironcluster Hadoop ETL.

As the Parrot systems demonstrate, commercial drone systems are not wildly different from their military cousins. They too will be equipped with high performance video and other sensors that need geospatial processing and interpretation.

This demand will likely create a subspecies of Syncsort-like ETL applications which enable these specialized data streams to be integrated with traditional structured enterprise data stores.

Another way that little drones will have a big data impact.

Mapping of the Paris Air Show by Parrot (2013)