Applied AI on a lakehouse for assurance analytics

Context

The organisation provides assurance and risk services, including an analytics platform that supports environmental, social and governance (ESG) assurance. Several products draw on the same underlying data, and the platform was adding applied AI for search, summarisation and optimisation insights.

Challenge

• Data for several products needed a single, governed foundation rather than separate stores.
• AI features had to be grounded in the organisation’s own data, with retrieval that could be traced back to sources.
• Master data was held in a legacy system that needed a clear replacement strategy.
• Architecture decisions had to be aligned across data, product and engineering leadership.

What was done

Bexley’s founder is the lead and hands-on architect for the platform:

• Designed the enterprise-wide data architecture, built hands-on reference pipelines and proof-of-concept services, and led the transition to a unified Azure and Databricks lakehouse.
• Built governed ingestion and transformation pipelines for the platform’s products.
• Introduced semantic-retrieval AI components using embeddings and vector search to support search and summarisation.
• Defined the strategy for replacing a legacy system with Dynamics CRM-based master data management.
• Supported microservices adoption with Azure Kubernetes Service, Event Hubs and SQL operational stores.
• Authored C4 models, data flow diagrams and architectural standards, working closely with data, architecture and product leadership.

Architecture and approach

The lakehouse on Databricks and Delta Lake provides one governed store for analytical and AI workloads. Event streaming through Event Hubs and Kafka connects operational services to it. AI features use retrieval over the organisation’s own content, so answers can be traced back to their sources and evaluated, rather than relying on a model’s general knowledge.

How this applies to your organisation

AI earns its place when it sits on dependable data and fits a real workflow. Start with the data foundation and a focused, evaluated prototype, then integrate what proves itself. See AI and Data Architecture & Engineering.