How I Built a Config Framework That Scales Across 15+ Microservices

How I built a configuration system that mirrors .NET appsettings structure while enabling team-wide consistency across microservices.

The Configuration Challenge

When managing multiple microservices with Pulumi, you quickly run into configuration sprawl. Each service has different ways of organizing settings, secrets are scattered, and onboarding new team members becomes a documentation nightmare.

After deploying 15+ microservices, I developed a structured approach that solves these problems through careful YAML organization and dynamic parsing.

The Core Design: Mirroring .NET Configuration Structure

YAML Structure That Mirrors appsettings.json

Instead of arbitrary configuration, I designed the YAML to mirror the familiar .NET configuration structure:

# Pulumi.dev.yaml - Mirrors your appsettings.json structure
ServiceName:ApiAppSettings:
  ExternalServices:
    PaymentAPI:
      BaseUrl: https://api.payments.com
      Timeout: 30
      RetryAttempts: 3
  Features:
    EnableCaching: true
    EnableLogging: false
  Business:
    Currency: USD
    MaxFileSize: 10485760

ServiceName:FnAppSettings:
  Values:  # Maps directly to local.settings.json "Values" section
    ProcessingBatchSize: 10
    RetryAttempts: 3
  ConnectionStrings:  # Maps to ConnectionStrings section
    EventGridEndpoint: https://events.azure.net/api/events

Common Infrastructure Patterns

Every service shares the same foundational structure:

# These sections are identical across all microservices
ServiceName:Tags:
  Environment: "Development"
  Owner: "DevTeam" 
  Project: "ServiceName"
  ManagedBy: "Pulumi"

ServiceName:PlanSku:
  Capacity: 1
  Family: B
  Name: B1
  Size: B1
  Tier: Basic

ServiceName:DockerSettings:
  DockerRegistryUrl: myregistry.azurecr.io
  DockerRegistryUserName: myregistry
  DockerApiImageName: service-api
  DockerApiImageTag: latest

Why this matters: New developers see a payment service config and instantly understand the user service config. Same sections, same patterns, same structure.

The Modular Architecture

1. DeploymentConfigs Record – The Configuration Hub

public record DeploymentConfigs
{
    // Basic infrastructure (same across all services)
    public string Location { get; init; }
    public string Environment { get; init; }
    public Dictionary<string, string> CommonTags { get; init; }
    public Dictionary<string, object> PlanSku { get; init; }
    public Dictionary<string, string> DockerSettings { get; init; }
    
    // Service-specific app settings (dynamic structure)
    public Dictionary<string, object> ApiAppSettings { get; init; }
    public Dictionary<string, object> FnAppSettings { get; init; }
    
    // Specialized secret handling
    public SecretAccess Secrets { get; init; }

    public DeploymentConfigs(Config config)
    {
        // Standard configs - same parsing for every service
        Location = config.Require("Location");
        CommonTags = config.RequireObject<Dictionary<string, string>>("Tags");
        
        // Dynamic configs - handle any nested structure
        var apiSettingsRaw = config.RequireObject<JsonElement>("ApiAppSettings");
        ApiAppSettings = ConfigParser.ConvertJsonElementToDictionary(apiSettingsRaw);
        
        // Specialized secret handling
        Secrets = new SecretAccess(config);
    }
}

2. SecretAccess Class – Functional Secret Management

Rather than just storing secrets, this class provides functionality:

public class SecretAccess
{
    private readonly Config _config;

    // Direct secret access
    public Output<string> SqlPassword => _config.RequireSecret("SqlPassword");
    public Output<string> DockerRegistryPassword => _config.RequireSecret("DockerRegistryPassword");

    // Functional secret handling - builds connection strings dynamically
    public Output<string> BuildSqlConnectionString(string database)
    {
        var databaseConfig = _config.RequireObject<Dictionary<string, string>>("Database");
        var server = databaseConfig["SqlServer"];
        var userId = databaseConfig["SqlUserId"];

        return SqlPassword.Apply(pwd =>
            $"server=tcp:{server};User ID={userId};Password={pwd};database={database}");
    }

    public Output<string> BuildBlobConnectionString(string accountName, Output<string> accountKey)
    {
        return accountKey.Apply(key =>
            $"DefaultEndpointsProtocol=https;AccountName={accountName};AccountKey={key};EndpointSuffix=core.windows.net");
    }
}

Key insight: Secrets aren’t just values – they’re building blocks for dynamic configuration assembly.

3. ConfigParser – Dynamic YAML to Dictionary Conversion

The magic happens in the parser. Pulumi gives us a JsonElement, but we need flexible dictionaries:

public static Dictionary<string, object> ConvertJsonElementToDictionary(JsonElement element)
{
    var dictionary = new Dictionary<string, object>();

    foreach (var property in element.EnumerateObject())
    {
        switch (property.Value.ValueKind)
        {
            case JsonValueKind.Object:
                // Recursively handle nested objects
                dictionary[property.Name] = ConvertJsonElementToDictionary(property.Value);
                break;
            case JsonValueKind.Array:
                dictionary[property.Name] = ConvertJsonElementToArray(property.Value);
                break;
            case JsonValueKind.String:
                dictionary[property.Name] = property.Value.GetString() ?? string.Empty;
                break;
            // Handle numbers, booleans, nulls...
        }
    }
    return dictionary;
}

The payoff: Add any nested YAML structure without touching C# code:

# Add this to YAML...
ServiceName:ApiAppSettings:
  NewFeature:
    ComplexNesting:
      DeepValue: "works automatically"
      AnotherLevel:
        EvenDeeper: true


// ...and access it immediately in C#
if (apiSettings.TryGetValue("NewFeature", out var newFeatureObj) && 
    newFeatureObj is Dictionary<string, object> newFeatureDict)
{
    // Dynamic access to any depth
    ConfigureNewFeature(newFeatureDict);
}

Stack Organization: The Assembly Line

Organized Resource Creation

The main stack follows a clear pattern:

public class ContainerizedStack : Pulumi.Stack
{
    public ContainerizedStack()
    {
        var config = new Config("ServiceName");
        var deploymentConfigs = new DeploymentConfigs(config);

        // 1. Foundation
        var resourceGroup = CreateResourceGroup(deploymentConfigs);

        // 2. Core Services  
        var apiAppService = CreateApiAppService(deploymentConfigs, resourceGroup);
        var functionApp = CreateFunctionApp(deploymentConfigs, resourceGroup);

        // 3. Supporting Services
        var signalRService = CreateSignalRService(deploymentConfigs, resourceGroup);
        var eventGridTopic = CreateEventGridTopic(deploymentConfigs, resourceGroup);

        // 4. Outputs
        this.ApiUrl = apiAppService.DefaultHostName.Apply(hostname => $"https://{hostname}");
        this.FunctionUrl = functionApp.DefaultHostName.Apply(hostname => $"https://{hostname}");
    }
}

Modular App Settings Assembly

Each resource type has its own settings assembly pattern:

private static NameValuePairArgs[] GetApiAppSettings(DeploymentConfigs config, Component appInsights)
{
    var settings = new List<NameValuePairArgs>
    {
        // Standard settings (same for every service)
        new() { Name = "WEBSITE_RUN_FROM_PACKAGE", Value = "1" },
        new() { Name = "APPLICATIONINSIGHTS_CONNECTION_STRING", Value = appInsights.ConnectionString },
    };

    // Dynamic settings assembly
    AddExternalServiceSettings(settings, config);
    AddFeatureFlagSettings(settings, config);
    AddBusinessSettings(settings, config);

    return settings.ToArray();
}

private static void AddExternalServiceSettings(List<NameValuePairArgs> settings, DeploymentConfigs config)
{
    if (config.ApiAppSettings.TryGetValue("ExternalServices", out var servicesObj) && 
        servicesObj is Dictionary<string, object> servicesDict)
    {
        foreach (var service in servicesDict)
        {
            if (service.Value is Dictionary<string, object> serviceConfig)
            {
                foreach (var setting in serviceConfig)
                {
                    // Dynamic setting name: ExternalServices__PaymentAPI__BaseUrl
                    var settingName = $"ExternalServices__{service.Key}__{setting.Key}";
                    settings.Add(new() { Name = settingName, Value = setting.Value?.ToString() ?? "" });
                }
            }
        }
    }
}

Real-World Benefits

Configuration Consistency

# Payment Service
PaymentService:ApiAppSettings:
  ExternalServices:
    BankAPI: { BaseUrl: "...", Timeout: 30 }

# User Service  
UserService:ApiAppSettings:
  ExternalServices:
    AuthAPI: { BaseUrl: "...", Timeout: 30 }

# Same structure, different content

Zero Code Changes for New Config

# Add this to any service YAML
ServiceName:ApiAppSettings:
  Monitoring:
    EnableDetailedLogging: true
    LogLevel: "Information"
    CustomMetrics:
      TrackUserActions: true
      TrackPerformance: false

The parser handles it automatically. No C# compilation required.

Functional Secret Management

// Instead of storing complete connection strings in config:
connectionStrings.Add(new ConnStringInfoArgs
{
    Name = "DefaultConnection",
    ConnectionString = secrets.BuildSqlConnectionString("MyDatabase"),  // Built at runtime
    Type = ConnectionStringType.SQLAzure
});

Implementation in Action

Service Creation Workflow

1. Copy template → Update service name in files
2. Customize YAML → Add service-specific sections
3. Run secrets script → Set encrypted values
4. Deploy → pulumi up

Team Consistency Results

• New developer onboarding: 2 hours instead of 2 days
• Configuration reviews: Instant pattern recognition
• Debugging: Same structure across all services
• Infrastructure updates: Change YAML, redeploy

The Technical Foundation

This approach combines:

• Structured YAML that mirrors familiar .NET patterns
• Dynamic parsing that adapts to any configuration structure
• Functional secret management that builds configurations at runtime
• Modular stack organization that separates concerns cleanly

The result is infrastructure code that scales with your team rather than fighting against it.

Key Implementation Files

├── Pulumi.dev.yaml          # Structured configuration (mirrors appsettings.json)
├── DeploymentConfigs.cs     # Configuration hub and parser integration
├── SecretAccess.cs          # Functional secret management  
├── ConfigParser.cs          # Dynamic YAML→Dictionary conversion
├── ContainerizedStack.cs    # Organized resource assembly
└── add-secrets.ps1         # Standardized secret setup

Conclusion

Configuration management in infrastructure code doesn’t have to be chaotic. By mirroring familiar .NET patterns, parsing configurations dynamically, and organizing functionality modularly, you can build systems that grow with your team instead of slowing them down.

The patterns shown here have been battle-tested across 15+ production microservices. They work because they solve real team problems with thoughtful technical design.

Repository: pulumi-azure-infrastructure-template

The code speaks for itself. The patterns scale with your team.