Cloudera’s DHM: SOA for Big Data?
Calling it a holy grail might be an exaggeration, but the idea of an enterprise “hub” or “bus” has been kicking around for a couple of decades now. It can probably be retraced to the earliest ideas about enterprise architecture written by John Zachman in the 1980’s. But The Open Group’s Len Fehskens goes back further, calling it “. . . another round of realization that things ought to be modularized.”
So what makes a hub a hub? What is different about a Big Data enterprise hub?
About the Hub (Bub)
Ask Cloudera, and they have a simple response: “The foundation of an enterprise data hub is, of course, Apache Hadoop.”
Well, not just Hadoop. Then Cloudera added their management tool, Impala for processing SQL over Hadoop, Spark, HBase, extensions for DR and backup, Apache Sentry to improve Hadoop security, and then Navigator for metadata management.
Cloudera believes all this value-adding labor adds up to more than the sum of its parts.
Is that sum what in an earlier stage of enterprise architecture widget-making was called the Service Oriented Architecture (SoA) approach?
Enterprise Hubs Likened to Data Freight Services
After all, messaging middleware isn’t unique. For example, Apache Mule is an existing tool that implemented a widely discussed Enterprise Integration pattern presented by Gregor Hohpe. Apache ActiveMQ provides Mule with additional glueworks. In the Apache open source world, others have embraced Camel. IBM’s WebSphere has an ESB, as does Oracle. Even .NET isn’t left out as Microsoft has an ESB for Azure.
SOA as a concept seems to be alive and well, though in the downward side of Gartner’s hype curve. In a paper on business service modeling, a 2012 paper proposed using SOA as the basis for business service architecture (Zikie, et al. 2012). This was largely the approach taken by a joint Boeing-GMU project undertaken a decade earlier (Sommer et al., 2002).
Deprecate XML if you must use JSON instead, and you have the glue that many cloud services are using today.
Enterprise Service Bus (ESB) was described by Ueno and Tatsubori as simply “a bus which delivers messages from service requesters to service providers.” Is Cloudera up to something different, or does Ecclesiastes 1:9 apply (“. . . there is no new thing under the sun”).
New Things Under the Sun
David Sprott, in “Death of Enterprise Architecture” http://bit.ly/1gljpEi proposed a new term, “Smart Ecosystem Architecture.” In his Copernican way of seeing things, the enterprise should not see itself as the center of the universe – creating and consuming its own services in isolation. Instead, each enterprise is part of a larger ecosystem consisting of multiple service producers and consumers. He defines Service Factory as a Service (SFaaS) — increasingly delivered “in an MDA/MDD stack.
David Sprott is betting on a future for Service Factory as a service.
Whether Sprott has this exactly right probably resolves around one’s favored toolbox. But if the stack deployed by startups is any indication, an ecosystem around Cloudera’s products – supplemented by specialized cloud services like MailChimp or RunMyProcess – can spin up quite a few apps that would take much longer to create using other methods.
Analytics as the Killer App
Gartner’s Philip Allega wrote:
It is clear to Gartner that the role of Enterprise Architecture is not ‘dead.’ It has instead just found its seat at the IT leadership table.
Of course there is no one killer app, as MySpace and WordPerfect demonstrate. But in the cauldron of Big Data software, analytics is as good a choice for a killer app as any. Cloudera believes:
The real benefit comes when you can increasingly bring new analytic workloads to the EDH, reducing the need to invest in and move large volumes of data between platforms just to ask new questions.
Syncsort’s DMX-h joins a circle of Cloudera enablers by SAS, Revolution Analytics and Splunk. They have weaponized the killer app that is analytics.
Lock and load!
Bo, D., Kun, D., Xiaoyi, Z., Oct. 2008. A high performance enterprise service bus platform for complex event processing. In: Grid and Cooperative Computing, 2008. GCC ’08. Seventh International Conference on. pp. 577-582. URL http://dx.doi.org/10.1109/GCC.2008.66
Daigneau, R., Nov. 2011. Service Design Patterns: Fundamental Design Solutions for SOAP/WSDL and RESTful Web Services, 1st Edition. Addison-Wesley Professional. URL http://bit.ly/PtQp3L.
Hohpe, G., Woolf, B., 2003. Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA. URL http://bit.ly/1lWPm9T.
Sommer, R. A., Gulledge, T. R., Bailey, D., Mar. 2002. The n-tier hub technology. SIGMOD Rec. 31 (1), 18-23. URL http://doi.acm.org/10.1145/507338.507343
Ueno, K., Tatsubori, M., Sep. 2006. Early capacity testing of an enterprise service bus. In: Web Services, 2006. ICWS ’06. International Conference on. pp. 709-716. URL http://dx.doi.org/10.1109/ICWS.2006.57.
Zikie, F. A., Dico, A. S., Debela, D. M., 2012. Business service modeling using SOA: A core component of business architecture. In: Proceedings of the International Conference on Management of Emergent Digital EcoSystems. MEDES ’12. ACM, New York, NY, USA, pp. 181-182. URL http://doi.acm.org/10.1145/2457276.2457309.
Mark Underwood writes about knowledge engineering, Big Data security and privacy.