Meet Virtusa at Microsoft Ignite 2024 | Schedule a meeting here

success story

A Fortune 500 health service provider increased 30% in productivity with Azure Databricks

The Challenge

The client, a Fortune 500 health service provider, was overwhelmed by unstructured data coming from over 12 data sources, 7+ lines of business (LOBs) and applications, and more than 47,000 job configurations. This included 90+ Ab Initio projects and 2,000+ Ab Initio graphs, which, combined with 10+ reporting and downstream applications, significantly contributed to complexity. The reliance on multiple data sources resulted in high fixed hardware and licensing costs from tools like Ab Initio and Netezza. Moreover, the client faced long development cycles due to the intricate nature of their existing platform, alongside persistent data integrity and quality issues.

The Solution

Virtusa designed a robust, modern data platform architecture using Azure Databricks to simplify the client's data ecosystem.

Key improvements included:

  • Leveraging Azure Data Factory (ADF): Connected the client’s on-premises servers to efficiently extract data from multiple sources.
  • Implementing Databricks Delta Tables: Achieved high-speed data processing for ingestion and transformation, ensuring faster data workflows.
  • Optimized Storage: Introduced automatic data compression using Delta Tables, reducing storage costs and improving overall data efficiency.
  • Metadata-Driven Ingestion Framework: Developed a custom, metadata-driven framework for streamlined data ingestion and transformation, leveraging the power of Databricks and PySpark.
  • Utilizing PySpark and PolyBase: Enhanced transformation processes and accelerated data loading into Azure Synapse (Data Warehouse).
A Fortune 500 health service provider increased 30% in productivity with Azure Databricks
The Benefit

Post modernization, the client experienced improved overall agility and scalability of their data platform.

Benefits included:

Faster analytics and insights for business

Improved data quality and integrity

Significant reduction in IT costs

Unity Catalog Migration Studio™

Learn more about Unity Catalog Migration Studio™ powered by the Databricks Data Intelligence Platform

Learn more Click to Learn more
Related content