A cloud data platform for healthcare analytics

Context

The organisation provides data and analytics on healthcare and life-sciences markets. Its products depend on large, continually updated datasets, and it was replacing its core data and analytics platform with a new generation built for cloud scale and machine-learning use.

Challenge

• Data was spread across legacy systems that were difficult to extend and costly to change.
• Analytics products and data science work needed consistent, well-modelled data rather than one-off extracts.
• Machine-learning use required pipelines and data structures designed for training and inference, not only for reporting.
• Migration had to happen while existing products kept running.

What was done

Bexley’s founder was the architectural lead and technical product owner for the new core platform:

• Designed and led delivery of a modular, Snowflake-based data platform, engineered hands-on, including ML-ready pipelines and data structures that support inference.
• Co-designed and implemented key components of a distributed ETL and ML-enablement platform built on Apache Airflow, Java and Python.
• Delivered a React and Java user-experience platform for advanced analytics and insights.
• Architected and executed large-scale data migrations from legacy systems to AWS and Snowflake.
• Built and managed multi-shore data engineering and platform development teams.

Architecture and approach

The platform was designed as modules with clear boundaries, so ingestion, transformation, storage and serving could evolve independently. Orchestration ran on Airflow, processing on AWS services such as EKS, EMR and Glue, and analytical storage on Snowflake, with Redshift and Athena used where they suited the workload. GraphQL gave products built on the platform a consistent interface. Migration was staged, moving datasets and consumers in increments rather than through a single cutover.

How this applies to your organisation

If your reporting, analytics or AI plans are held back by fragmented or legacy data, the same approach applies at any scale: agree the core data model, build observable pipelines, and migrate in stages while existing services keep running. See Data Architecture & Engineering.