Christopher VALLOTDiscuss your project
FREN
All projects

Automating and improving the reliability of BI data flows

Automating, consolidating and checking business data flows before they are used for decision-making.

Professional experienceBI data flows

An assignment from my professional background. No internal or confidential data is disclosed.

Abstract illustration of the main data flow — no internal or confidential data is shown.

Context

Within a BI team, data from health and personal protection insurance flows needed to be integrated, consolidated and reconciled.

  • Business data flows
  • Data extracted from multiple sources
  • Consolidated and reconstructed data
  • Check and reconciliation results

Problem and challenge

The data needed to be reconciled and checked despite the variety of flows and integration processes.

Objectives

  • Automate integration processes.
  • Extract, transform and consolidate data.
  • Check Data Quality and reconcile sources.
  • Improve the reliability of extraction and loading for BI data flows.

Contribution

I developed SAS processing, automated data reconstruction with Python and integration with PowerShell, and set up consistency checks and reconciliation with Power BI.

  • Developing integration processes in SAS.
  • Automating with Python and PowerShell.
  • Checking Data Quality and reconciling data with Power BI.
  • Consolidating and reconstructing data.

Outcome

More repeatable integration processes, with consolidated and checked data before it is loaded into BI flows.

Technologies

  • SAS
  • Python
  • SQL
  • PowerShell
  • Power BI

A similar challenge?

Facing a similar challenge? Let’s talk.

Let’s discuss your data, processes and expected outcome to define an engagement that fits your context.

Discuss your project