Turning Deployment Metadata Into a Platform Service

The first version of a platform often carries its own source of truth in the place where the work happens. For this healthcare analytics platform, that place was the deployment pipeline.

Each customer tenant had a hard-coded JSON configuration file in the codebase. The files contained the variables needed to deploy the tenant through Azure DevOps and Terraform: storage accounts, function apps, Databricks workspaces, key vaults, application monitoring, regional resources, central resources, and the surrounding metadata required to stitch the platform together.

The system worked. That was part of the problem. It worked well enough to keep going, but not well enough to scale cleanly.

Every new tenant or configuration change required a file change, a pull request, review, and a pipeline run. Platform tools outside the deployment process had no clean way to understand tenant state. If someone wanted to know what was deployed, where a tenant was in the deployment process, or which branch and pipeline run produced the current state, they had to inspect deployment logs or read the JSON directly.

At roughly 70 tenant environments, that stopped being a configuration pattern and started becoming platform drag.

Making Metadata Queryable

The solution was to move the platform’s operational metadata behind an API.

Instead of treating JSON files as the durable interface, we built a strongly typed ASP.NET Core metadata service backed by MongoDB and a graph database. The graph represented the customer application, application tenant, central resources, and regional resources as related platform concepts rather than disconnected file fragments. MongoDB stored deployment sets, templates, feature flags, and the backing data for permissions management.

That data model was the hardest technical part of the project. The old configuration files had grown around deployment needs, not around a clean domain model. We had to decide which data belonged to the tenant as a whole, which data belonged to central services, and which data needed to remain separately addressable by region.

The goal was not just to store the same values somewhere else. The goal was to make the platform state explicit, validated, and usable by more than one pipeline.

From Files To Deployment Sets

The deployment pipeline changed from reading per-tenant JSON files to calling the metadata service for a deployment set.

A deployment set combined the tenant metadata, feature flags, and deployment-specific values the pipeline needed to run Terraform. The pipeline could then use that deployment set as its variable source throughout the release. It could also write status back to the service: deploying, available, release branch, and a link to the most recent pipeline run.

That feedback loop mattered. The platform interface could now show tenant managers where a tenant was in the deployment process. Teams could produce tenant-level, regional, and organization-wide views without scraping logs or reading source files. The deployment system was no longer the only place where platform state could be understood.

The metadata service also gave us a better validation boundary. Data transfer objects had to conform to the C# application contract. Required fields were required. Types were types. The platform no longer depended on convention and careful file editing as its primary guardrails.

The Action Layer

Alongside the metadata service, we also delivered an action service: a general interface for operational tasks around the platform.

That service handled work that did not belong directly inside Terraform but still needed to be part of the platform lifecycle: interacting with secret stores, managing Databricks permissions, creating service principals, building Delta shares, and creating AI/BI dashboards.

Authentication used Entra ID bearer tokens, with multiple roles enforced through a shared app registration. Each service ran in its own Azure App Service with supporting resources such as application monitoring, storage accounts, and key vaults.

This split kept the metadata service focused on platform state while giving the platform a controlled place to perform actions around that state.

Getting It Into Production

The organizational challenge was not convincing people the work mattered. The need was obvious. The challenge was getting a busy development effort across the line and into production as a supported service.

My role was to help move the platform from initial development into real production use. I handled most of the infrastructure work, contributed to the application implementation, and became the primary engineer responsible for getting the service over the finish line. After launch, I supported the platform and owned much of the compliance and auditability posture around it.

Rollout happened by ring: development tenants first, then integration, then production. That let us validate the model and pipeline behavior before moving the highest-risk environments.

We also designed for the ordinary failure modes of cloud infrastructure. API failures and Azure drift had to be visible and recoverable. Deployment sets were retained for 90 days, then archived to blob storage so we could answer the audit question that always matters: what was deployed, and when?

What Changed

The biggest practical change was speed.

Creating or updating a tenant no longer required a pull request against a static configuration file. Engineers could interact with the platform through an API. The interface team could display tenant state directly in the product experience. Troubleshooting had a better starting point because deployment status, release branch, and pipeline run links were part of the platform metadata.

The template capability also gave teams a lightweight way to store named, base64-encoded JSON templates without building their own service. That made it easier to standardize application details and feature configuration around the metadata platform instead of creating another scattered set of files.

This project did not magically eliminate every deployment mistake. What it did was remove a major source of friction. It turned configuration from a pull-request-bound artifact into a typed, auditable, queryable platform capability.

That is the lesson I took from the work: once metadata becomes operationally important, it deserves the same engineering care as the systems that consume it.