Data infrastructure optimization, availability & security software
Data integration & quality software
The Next Wave of technology & innovation

C4ISR Was Big Data Before They Called it That

Been There, Done That

If any software engineers have earned the right to pooh-pooh the fuss over Big Data, it would those who have labored in Command, Control Communications and Computers Intelligence Surveillance and Reconnaissance (C4I) for the U.S. defense industry. On the other hand, these engineers may be more likely to mutter, “Been there. Tried that” – instead of the usual form of the expression.

Technocrats within the military have been working to make each Service arm more “net-centric” for at least two decades. The problem of exploding data volumes, driven by image and sensor streams, was recognized in the mid-80’s – long before the term “Big Data” came into its own. For architects who believe that Big Data began with Hadoop, a quick look at C4ISR could reveal lessons learned and models for successful implementation.

Variety is not Victory

In addition to data volumes, the variety of data has been a C4ISR challenge from the beginning. Data may arrive on specialized secure military channels, or it may be so-called open intelligence gathered over the internet or media reports. The sheer diversity of platforms – airborne, satellite, submarine, surface ship, soldier-borne – is one cause. Another is limited channel bandwidth, especially for secure communications. Another is organizational complexity. The U.S. Navy, for instance, operates both land, sea and air assets. Just as business intelligence must be tailored for particular user communities, is has not proved easy to provide C4ISR across these diverse Navy realms in a form that is suitable for use by a particular commander.

FLIR Systems Star SAFIRE 380-HD electro-optical/infrared (EO/IR) sensor

FLIR Systems’ Star SAFIRE 380-HD electro-optical/infrared (EO/IR) sensor is used by Boeing Defense, Space & Security as its baseline EO/IR sensor supporting various Maritime Surveillance Aircraft (MSA) missions such as search and rescue, anti-piracy patrol, and coastal and border security.

Aeronix 802.16 LAW-ANR (Advanced Network Radio)

Aeronix 802.16 LAW-ANR (Advanced Network Radio)

IoT Advocates? Listen Up.

As we pointed out in this column on another occasion, the Internet of Things (IoT) will make Big Data bigger. IoT evangelists may want to study what has happened in C4ISR sensor technology. Resolution, speed, data rates and variety of sensor types have all accelerated, and form factors have shrunk. But the news isn’t all good.

“Sensors have become prolific,” says Allan Cantle, CTO of Interconnect Systems Inc. and president of Nallatech. “Invariably, the bandwidth of the communication channels is not keeping pace with these increasing amounts of data. This challenge has created the need to either process the data locally at the sensor or store the data for retrieval and postprocessing later after a mission is complete.”

Cantle advocates “Near Sensor Computing” – i.e., processing near the edge, including onboard security for both manned and unmanned ships and vehicles. As we pointed out earlier this year in our Syncsort Kate Matsudaira interview, compression is likely to play a large role in addressing some of these concerns.

<i>USS Zumwalt</i> Raytheon-built Ruggedized Server modules

USS Zumwalt Raytheon-built Ruggedized Server modules: 16 self-contained shipboard mini data centers with 176 racks and weighing 18 tons

C4ISR scalability is an omnipresent worry. While press reports have discussed the possibility of floating Google data centers, the U.S. Navy is already tasked with fielding ocean-going data centers. These data centers must connect with land and submarine servers and devices, as well as a diversity of onboard subsystems. Information must be sorted, cleansed and aggregated for use by Zumwalt commanders and specialists, but it must also serve the needs of larger organizations like carrier or expeditionary strike groups.

In addition to security challenges, military C4ISR architects must live within power, durability and size constraints. Lockheed-Martin’s Automated Logistics Information Systems, for example, was designed in concert with other aspects of the F-35 Joint Strike fighter jet, rather than separately. In the latest version, to be delivered in Fall 2014, a “portable memory device” is employed to speed up downloading of fault codes and logistics support information from the aircraft. While it still takes 15 minutes to do so, at the end of each F-35 flight, this is 1/3 the time it took in the initial release.

What You S/A?

“Situational Awareness” or “S/A,” the oft-cited holy grail for military intelligence, includes data from many sources. Consider just one of these – geospatial information. Support for geospatial intelligence (or “GEOINT”) includes imagery from national military satellite systems, commercial satellites, coalition partners, and unmanned vehicles (“drones”). Some of these platforms also provide a range of sensor data, such as ultraviolet imaging, radar, motion imaging, and real time target tracking. Data from GEOINT sources must be fused with other data sources in order to make sense of the data streams.

A Bryan Harris is Director of Research and Development for Cyber Analytics, SAS, put it, “Every day, defense and intelligence decision-makers are faced with the difficult task of finding actionable data in a constantly-expanding sea of information. As Big Data increasingly permeates organizations worldwide, competitive advantage comes from the ability to analyze that data quickly to close analytic and cognitive loops.”

JIE Whiz

The Department of Defense (DoD) is building what it calls a Joint Information Environment (JIE). JIE is a single global network that includes everything from email to cloud computing. And C4ISR.

“Fusing data across the DOD’s enterprise to make it more readily accessible and actionable to military leadership remains a massive work in progress. Business intelligence tools will help improve real-time monitoring of the system and improve the DOD Information Network, but a variety of operational [issues] remain unresolved and will need to be improved as the JIE expands,” Defense Information Systems Agency (DISA) chief of staff Brigadier General Frederick A. Henry said.

A further example is provided by DISA’s Cybersecurity Situational Awareness Analytic Cloud. The cloud compiles data coming in from sensors and Internet access points throughout DoD’s networks and data centers, fusing that information to monitor for any issues, events or anomalous behavior.

A final DoD Big Data example comes from computer security. Big Defense contractor ($29B market cap, 63,000 employees) Raytheon’s SureView offering is a security endpoint and monitoring tool which Dan Velez, Raytheon Director of Insider Threat Programs, says “also aggregates data from multiple sources including Big Data analytics solutions.”

Under the Big Data Radar

“DoD is placing a big bet on Big Data” said Zachary Lemnios, U.S. Assistant Secretary of Defense for Research and Engineering.

How much Big Data C4ISR is already going on in the defense sector? Consider that Northrop Grumman ($25B market cap, 65,000 employees) and General Dynamics ($40B, 96,000 employees) together have twice as many employees as Apple ($600B cap). They have far more years of experience than Apple’s teams working on wearables and expanding iCloud.

C4ISR activities are not limited to the military. The agency responsible for improving inter-agency coordination across the government after 9/11 is the Information Sharing Environment (ISE) office. In its latest report to Congress, the ISE describes efforts to bring all major domestic law enforcement entities into a national network of “fusion centers.”

Just how much Big Data is in these fusion centers is unclear. A report by PR Watch in 2013 took note of 2i-MIND OpenMIND, SAS Memex deployments, though the report also makes clear some of the funding and bureaucratic problems that weaken attempts to improve interoperability and broaden coverage across affected areas.

ISE Communities of Interest

Communities of Interest Identified in the 2014 ISE Report to Congress

Learn from the Past

Since C4ISR practitioners have been working Big Data longest, what advice do they have for the rest of us? One answer from Jon “Doc” Kimminau, Air Force analysis mission technical adviser for the deputy chief of staff for ISR, may surprise civilian app designers lured by the prospect of using Big Data for situation awareness.

“These patterns and trends can richly inform analysis of the current situation,” Kimminau says. “[But] Big Data’s strength is not identification of the current situation; it is discovering and understanding past activities and events. To the extent that the understanding of the past – previously not recognized – informs appreciation of current events, it supports and assists situational awareness.”

The stakes with C4ISR can also be Big, as the unplanned heroics in the 2013 film Lone Survivor demonstrated. Lapses in real time C4ISR resilience and reliability can prove fatal.

Related Posts