Barbecue & Big Data: Dickey’s Journey into Analytics in the Cloud
Dickey’s Barbecue approached us in 2013 with a common problem faced by growing mid-market companies: build a world class analytics platform without the price tag typically associated with enterprise hardware and software.
Like most organizations its size, there was an incredible dependency on spreadsheets. Analysts would wake up early in the morning and pull raw data from antiquated systems and set out to refresh their spreadmarts in preparation for daily operations. Sales data came from reports delivered to an email inbox. Same store sales came from the spreadsheets on the network drive. Inventory was self reported at varying time periods across the franchisee system. Purchase data had to be manually pulled from vendor websites. Guest satisfaction results were copied from web page output. Dickey’s made it work with 250 locations, but their growth to 500 required a better solution.
A scatter chart comparing sales with catering purchase. Illustrates possible compliance issues with stores either purchasing off program or not using the POS to capture catering sales.
We had the usual challenges to solve:
- Define the analytics roadmap
- Define and deploy technical infrastructure
- Install and configure software
- Build data marts and analytic applications
- Deploy, train, and support
Thanks to great technology partnerships with Syncsort, Amazon Web Services, and Yellowfin, we were able to work out incredibly cost effective technical infrastructure that allowed us to scale with the growth of the business.
Our infrastructure consists of the following:
- Job control: 1 x t1.micro
- DMX: 1 x m3.xlarge
- Additional high IOPS volumes for concurrent processing
- BI: 1 x m3.xlarge
- Redshift: 2 node dw2.large
- Metadata: 1 x db.m1.small
We developed software agents to monitor the 500 point of sale systems across the organization and send data to our platform in near real time. Syncsort DMX allowed us to build a unique ingest and egress framework for ELT on Amazon Redshift that resulted in near instant data loads.
The platform is truly impressive and was exciting to build, but I didn’t anticipate how it would become the catalyst for change in the organization. The analytics provides insights across subject areas without analysts spending hours integrating data. For example, we developed reports to show correlations between training compliance, guest complaints, and how this impacts sales. We can show healthy vs. unhealthy COGS models and how operators need to pull the right levers to maximize profitability.
New metrics that were previously impossible to calculate are readily available. By integrating Holt Winters and ARIMA models into our DMX loads, we can pretty accurately predict daily sales at a store level.
Business units began flooding the queue with new requests for reports. The platform became the keystone of all business applications. We are actually influencing upstream application designs to ensure compatibility with our platform.