Bringing trusted insights through modern data architecture

Technology

AWS
Qlik Replicate
Tableau

Industry

A leading insurance company markets preneed and individual life and annuities, as well as group markets products. The company sought to improve its processes and add value but found that future initiatives hinged on building a foundation for data management. 

After assessing several cloud providers, the company chose AWS to implement their new Modern Data Architecture (MDA). The MDA solution provides a single source of truth for all reporting, addresses their business needs, instills stakeholder trust in the data, saves users from doing countless hours of manual work, and will enable future initiatives to find new revenue streams and optimize operations.

The opportunity

Given the siloed nature of the current data environment, the company’s technologists spent a lot of effort developing and maintaining reports. To drive meaningful insights, they needed to evolve from the manual, ad hoc nature of reporting to something that was less time-consuming to maintain. In addition to complicated infrastructure, the company hosted many of its reports in Excel, creating an unstable foundation to build out enhanced analytics in the future. Continuing to use a segregated data structure across multiple environments would prevent them from creating the data-driven culture they aspire to achieve. By centralizing their data, the team would be able to leverage a single source of truth for reporting, reduce the time spent accessing data, and enhance overall data governance. To alleviate the challenges of the current environment, they sought to identify the right cloud platform to host and unify their data.

Why Inspire11?

Inspire11 has deep, multi-dimensional expertise and a proven track record of helping clients identify transformational opportunities. Through a strategic assessment, we identified many opportunities within the company’s distribution channels to drive revenue, improve operational inefficiencies, and subsequently cut costs. These initiatives hinged on proper data management, and getting this data into the cloud was the first step in enabling their digital capabilities. In addition to Inspire11’s extensive experience designing and implementing modern cloud data architecture designs for other partners, the company opted to continue to partner with Inspire11 through the implementation of their modern cloud data architecture because of the knowledge gained and trust built during the digital strategy assessment.

What we did

Inspire11 mapped the current state of the company’s data environment, highlighting ways the three sources of data were causing a large volume of manual work as users attempted to build reports with a blend of all three. With this context, we proposed a future state data architecture. Our proposal included three layers: a Staging and Data Lake layer, an ODS and Sandbox layer, and an EDW layer.

The implementation leveraged four accounts, an AWS networking account and three workflow sub-accounts, Dev, QA, and Prod. From the client’s on-prem environment, data flows through the AWS networking account, through a Palo Alto network firewall, into the workflow sub-accounts. Data is directed into an S3 bucket that is used as a data lake. A Qlik Compose tool that is hosted on a VRM cluster is used to design the warehouse. From there, the data is ready to be loaded into AWS Glue and is transformed and cleaned using Python Spark jobs. The data finally lands in a Redshift cluster where an additional set of transformations occur before the data is ready to be consumed.

Built into the MDA environment is a set of robust data operations capabilities such as versioning, pull requests, and tracked changes that bring the best pieces of DevOps to the data world. A CI/CD pipeline enables teams to develop in the cloud as well as audit their processes. The company now has visibility into their development ETL process that did not exist prior to working with Inspire11.