Skip to content

MCP Servers Deployment: Security & Networking Architecture

Related: ADR-039: MCP Servers Container Apps Deployment
Status: Approved
Date: 2025-01-15


Security Architecture Overview

Network Topology

┌──────────────────────────────────────────────────────────────────────────┐
│                           Public Internet                                │
└────────────────────────────────┬─────────────────────────────────────────┘
                                 │ HTTPS (Port 443)
                                 │ ✅ Public access allowed
┌──────────────────────────────────────────────────────────────────────────┐
│                     Azure Front Door / APIM (Future)                     │
│                    Rate Limiting, WAF, DDoS Protection                   │
└────────────────────────────────┬─────────────────────────────────────────┘
                                 │ HTTPS (Filtered)
┌────────────────────────────────┴─────────────────────────────────────────┐
│                   Azure Container Apps Environment                        │
│                    VNet: 10.0.0.0/16 (East US 2)                         │
│                                                                           │
│  ┌────────────────── External Subnet (10.0.0.0/23) ──────────────────┐  │
│  │                                                                     │  │
│  │  ┌──────────────────────────────────────────────────┐             │  │
│  │  │          UI Container App                        │             │  │
│  │  │  - Port: 8080                                    │             │  │
│  │  │  - Ingress: External                             │             │  │
│  │  │  - Public DNS: *.azurecontainerapps.io          │             │  │
│  │  │  - Azure-managed TLS certificate                │             │  │
│  │  │  - Attack Surface: ✅ Exposed (required)        │             │  │
│  │  └──────────────────────────────────────────────────┘             │  │
│  │                                                                     │  │
│  └─────────────────────────────────────────────────────────────────────┘  │
│                                 │                                          │
│                                 │ HTTPS (Internal only)                   │
│                                 │ Azure Internal DNS                      │
│                                 ▼                                          │
│  ┌────────────────── Internal Subnet (10.0.0.0/23) ──────────────────┐  │
│  │                                                                     │  │
│  │  ┌──────────────────────────────────────────────────┐             │  │
│  │  │          API Container App                       │             │  │
│  │  │  - Port: 8000                                    │             │  │
│  │  │  - Ingress: Internal                             │             │  │
│  │  │  - Internal DNS: *.internal.*.azurecontainerapps │             │  │
│  │  │  - No public access                              │             │  │
│  │  │  - Attack Surface: ❌ Not exposed                │             │  │
│  │  └──────────────────────────────────────────────────┘             │  │
│  │                                                                     │  │
│  │                         │                                           │  │
│  │                         │ HTTPS (Internal only)                    │  │
│  │                         │ Service-to-service                       │  │
│  │                         ▼                                           │  │
│  │  ┌─────────────────────────────────────────────────────────────┐  │  │
│  │  │              MCP Servers                                    │  │  │
│  │  │                                                             │  │  │
│  │  │  ┌────────────────┐  ┌────────────────┐  ┌──────────────┐ │  │  │
│  │  │  │   MCP Server 1 │  │   MCP Server 2 │  │ MCP Server 3 │ │  │  │
│  │  │  │  Verification  │  │   Documents    │  │  Financial   │ │  │  │
│  │  │  │   Port: 8010   │  │   Port: 8011   │  │  Port: 8012  │ │  │  │
│  │  │  │  Internal only │  │  Internal only │  │ Internal only│ │  │  │
│  │  │  │  ❌ No public   │  │  ❌ No public   │  │ ❌ No public  │ │  │  │
│  │  │  └────────────────┘  └────────────────┘  └──────────────┘ │  │  │
│  │  └─────────────────────────────────────────────────────────────┘  │  │
│  │                                                                     │  │
│  └─────────────────────────────────────────────────────────────────────┘  │
│                                                                           │
└───────────────────────────────────────────────────────────────────────────┘

Traffic Flow Analysis

User Request Path

1. User in Browser
   │ HTTPS (443)
   │ TLS 1.3, Strong Ciphers
2. Azure Edge (Front Door/CDN)
   │ ✅ SSL Termination
   │ ✅ DDoS Protection
   │ ✅ WAF Filtering
   │ HTTPS
3. UI Container App (External Ingress)
   │ ✅ Public FQDN: ldfdev-ui.azurecontainerapps.io
   │ ✅ React SPA (Static Content)
   │ ✅ Client-side routing
   │ HTTPS (Internal)
   │ Via Azure Internal DNS
4. API Container App (Internal Ingress)
   │ ✅ Internal FQDN: ldfdev-api.internal.{domain}
   │ ❌ No public access
   │ ✅ FastAPI Backend
   │ HTTPS (Internal)
   │ Service Discovery
5. MCP Server Container Apps (Internal Ingress)
   │ ✅ Internal FQDNs only
   │ ❌ No public access
   │ ❌ No external DNS records
   │ ✅ Tool execution
6. Response back through chain

Attack Surface Analysis

┌─────────────────────────────────────────────────────────────────┐
│                    EXTERNAL ATTACK SURFACE                      │
└─────────────────────────────────────────────────────────────────┘

Component: UI Container App
├─ Exposure: Public Internet
├─ Attack Vectors:
│  ├─ XSS (Cross-Site Scripting)        Mitigation: CSP Headers
│  ├─ CSRF (Cross-Site Request Forgery) Mitigation: SameSite Cookies
│  ├─ DDoS (Denial of Service)          Mitigation: Azure Front Door
│  └─ Malicious Content Injection       Mitigation: Input sanitization
└─ Risk Level: 🟡 MEDIUM (Public-facing but static content)

┌─────────────────────────────────────────────────────────────────┐
│                    INTERNAL ATTACK SURFACE                      │
└─────────────────────────────────────────────────────────────────┘

Component: API Container App
├─ Exposure: Internal VNet only
├─ Attack Vectors:
│  ├─ SQL Injection                     Mitigation: Parameterized queries
│  ├─ API Abuse                         Mitigation: Rate limiting
│  ├─ Unauthorized Access               Mitigation: JWT validation
│  └─ Data Exfiltration                 Mitigation: RBAC + Monitoring
└─ Risk Level: 🟢 LOW (Internal + Authentication)

Component: MCP Server 1 (Verification)
├─ Exposure: Internal VNet only
├─ Attack Vectors:
│  ├─ Direct Tool Access                Mitigation: Internal ingress
│  ├─ Resource Exhaustion               Mitigation: Rate limiting
│  └─ Data Leakage                      Mitigation: PII masking in logs
└─ Risk Level: 🟢 LOW (Internal only)

Component: MCP Server 2 (Documents)
├─ Exposure: Internal VNet only
├─ Attack Vectors:
│  ├─ Malicious Document Upload         Mitigation: File type validation
│  ├─ XXE (XML External Entity)         Mitigation: Disable external entities
│  └─ Buffer Overflow                   Mitigation: Secure parsing libraries
└─ Risk Level: 🟢 LOW (Internal only)

Component: MCP Server 3 (Financial)
├─ Exposure: Internal VNet only
├─ Attack Vectors:
│  ├─ Calculation Manipulation          Mitigation: Input validation
│  ├─ Integer Overflow                  Mitigation: Decimal types
│  └─ Precision Attacks                 Mitigation: Fixed-point math
└─ Risk Level: 🟢 LOW (Internal only)

Security Controls

Network Security

1. Ingress Control

External Ingress (UI only):

ingress: {
  external: true              // Public access allowed
  targetPort: 8080
  exposedPort: 443            // HTTPS only
  transport: 'http'
  traffic: [
    {
      weight: 100
      latestRevision: true
    }
  ]
  customDomains: []           // Can add custom domain later
}

Internal Ingress (API + MCP Servers):

ingress: {
  external: false             // ✅ CRITICAL: No public access
  targetPort: 8000            // Or 8010, 8011, 8012
  transport: 'http'
  allowInsecure: false        // ✅ Enforce HTTPS
}

Result: - UI: Public FQDN created, Azure-managed certificate - API + MCP: Internal FQDN only, no public DNS, no public IP

2. Network Security Groups (NSGs)

Container Apps Subnet NSG:

Inbound Rules:
──────────────────────────────────────────────────────────
Priority  Name                Source       Destination  Action
──────────────────────────────────────────────────────────
100       AllowAzureLoadBalancer  AzureLoadBalancer  *  Allow
110       AllowVNetInbound     VirtualNetwork    VNet     Allow
120       DenyAllInbound       *                 *        Deny

Outbound Rules:
──────────────────────────────────────────────────────────
100       AllowVNetOutbound    VNet        VirtualNetwork Allow
110       AllowInternetOutbound VNet       Internet      Allow
120       AllowAzureOutbound   VNet        AzureCloud    Allow

Effect: - ✅ Allow internal VNet traffic - ✅ Allow Azure services (ACR, monitoring) - ✅ Allow internet outbound (for dependencies) - ❌ Block all external inbound (except load balancer)

Current: Using internal ingress (sufficient for MVP)

Future: Private Link for additional security

┌─────────────────┐         ┌──────────────────┐
│  API Container  │─────────│  Private         │
│  App            │         │  Endpoint        │
└─────────────────┘         └──────┬───────────┘
                                   │ Private IP
                            ┌──────────────────┐
                            │  Azure OpenAI    │
                            │  (No public IP)  │
                            └──────────────────┘

Benefits: - No data traverses public internet - Additional network isolation - Compliance requirement (some industries)

Not needed for MVP: Internal ingress sufficient

Authentication & Authorization

Current State (MVP)

Service-to-Service:

UI ──────────────────────▶ API ──────────────────▶ MCP Servers
    No auth (same VNet)        Managed Identity       Managed Identity

Azure Resources:

Container Apps ─────────────▶ ACR (Pull images)
               Managed Identity    AcrPull role

Container Apps ─────────────▶ Log Analytics (Send logs)
               Managed Identity    Monitoring Metrics Publisher

Container Apps ─────────────▶ App Insights (Send telemetry)
               Connection String    (from secrets)

Future State (Post-MVP)

User Authentication:

User ──────────────────────▶ UI ──────────────────▶ API
    Entra ID (Azure AD)        JWT Token           JWT Validation

Implementation:

// UI: Entra ID auth
import { PublicClientApplication } from "@azure/msal-browser";

const msalConfig = {
  auth: {
    clientId: process.env.AZURE_CLIENT_ID,
    authority: `https://login.microsoftonline.com/${process.env.AZURE_TENANT_ID}`,
    redirectUri: window.location.origin
  }
};

const msalInstance = new PublicClientApplication(msalConfig);

// Get token
const tokenResponse = await msalInstance.acquireTokenSilent({
  scopes: ["api://loan-defenders-api/user_impersonation"]
});

// Call API with token
fetch(`${API_URL}/sessions`, {
  headers: {
    'Authorization': `Bearer ${tokenResponse.accessToken}`
  }
});

# API: JWT validation
from fastapi import Depends, HTTPException
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
from jose import jwt, JWTError

security = HTTPBearer()

async def verify_token(credentials: HTTPAuthorizationCredentials = Depends(security)):
    try:
        payload = jwt.decode(
            credentials.credentials,
            key=public_key,
            algorithms=["RS256"],
            audience="api://loan-defenders-api"
        )
        return payload
    except JWTError:
        raise HTTPException(status_code=401, detail="Invalid token")

@app.get("/sessions", dependencies=[Depends(verify_token)])
async def get_sessions():
    # Protected endpoint
    pass

Data Protection

1. Data in Transit

All traffic encrypted:

User ──TLS 1.3──▶ UI ──TLS 1.3──▶ API ──TLS 1.3──▶ MCP Servers
      (443)          (Internal)       (Internal)

Azure-managed certificates: - External: *.azurecontainerapps.io (auto-renewed) - Internal: Azure internal CA (auto-managed) - Custom domains: Can add via Let's Encrypt or Azure Certificates

Cipher suites (Azure default): - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 - No weak ciphers (RC4, DES, MD5 disabled)

2. Data at Rest

Container App Configuration: - Secrets stored in Container App secrets (encrypted by Azure) - Environment variables (non-sensitive) in plain text - No Key Vault for MVP (per ADR-032)

Logs & Monitoring: - Application Insights: Encrypted at rest - Log Analytics: Encrypted at rest - Azure Storage (if used): Encryption enabled by default

Future: Customer-managed keys (CMK) for additional control

3. PII Protection

Logging Best Practices:

# ❌ BAD: Logging PII
logger.info(f"Processing application for {ssn}")

# ✅ GOOD: Masked PII
logger.info(f"Processing application for applicant_id {applicant_id[:8]}***")

# ✅ BETTER: Structured logging with PII filter
logger.info(
    "Processing application",
    extra={
        "applicant_id": applicant_id,
        "pii_masked": True
    }
)

MCP Server Implementation:

# All MCP servers already mask PII in logs
logger.info(
    "Credit report request",
    extra={
        "applicant_id": applicant_id[:8] + "***",  # Masked
        "request_id": request_id
    }
)

Secrets Management

Current Implementation

Container App Secrets:

resource mcpServer 'Microsoft.App/containerApps@2024-03-01' = {
  properties: {
    configuration: {
      secrets: [
        {
          name: 'appinsights-connection-string'
          value: appInsightsConnectionString
        }
      ]
    }
    template: {
      containers: [
        {
          env: [
            {
              name: 'APPLICATIONINSIGHTS_CONNECTION_STRING'
              secretRef: 'appinsights-connection-string'
            }
          ]
        }
      ]
    }
  }
}

Benefits: - Encrypted at rest by Azure - Not visible in Container App configuration - Automatic rotation possible (manual process) - No Key Vault needed (simpler for MVP)

Future: Key Vault Integration

When to add Key Vault: - Multiple secrets (>5 per service) - Secret rotation requirements - Compliance mandates - Secrets shared across services

Implementation (future):

// Key Vault reference
secrets: [
  {
    name: 'db-connection-string'
    keyVaultUrl: '${keyVault.properties.vaultUri}secrets/db-connection-string'
    identity: 'system'
  }
]


Monitoring & Compliance

Security Monitoring

1. Application Insights

Tracked Events: - All HTTP requests (status, duration, headers) - Exceptions and errors - Custom events (loan application submitted, etc.) - Dependencies (MCP tool calls, Azure OpenAI calls)

Security-Relevant Metrics:

// Failed authentication attempts
requests
| where resultCode == 401 or resultCode == 403
| summarize count() by client_IP, bin(timestamp, 5m)
| where count_ > 10  // Alert on >10 failures in 5 minutes

// Unusual MCP tool usage
dependencies
| where target contains "mcp"
| summarize count() by name, bin(timestamp, 1h)
| where count_ > 1000  // Alert on >1000 calls per hour

// Response time degradation
requests
| where name contains "mcp"
| summarize avg(duration), max(duration) by bin(timestamp, 5m)
| where avg_duration > 500  // Alert if average >500ms

2. Log Analytics

Container Logs:

// Suspicious patterns in MCP server logs
ContainerAppConsoleLogs_CL
| where ContainerAppName_s contains "mcp"
| where Log_s contains "error" or Log_s contains "failed"
| project TimeGenerated, ContainerAppName_s, Log_s
| order by TimeGenerated desc

Security Auditing:

// Track all MCP tool calls
ContainerAppConsoleLogs_CL
| where Log_s contains "tool_call"
| extend tool = extract('tool":"([^"]+)"', 1, Log_s)
| extend applicant_id = extract('applicant_id":"([^"]+)"', 1, Log_s)
| project TimeGenerated, tool, applicant_id, Log_s

3. Azure Monitor Alerts

Recommended Alerts: 1. High Error Rate: >5% of requests failing 2. Latency Spike: p95 latency >500ms for 5 minutes 3. Health Check Failures: Any health check failing 4. Unexpected Scaling: Replicas >3 (may indicate DDoS) 5. Unusual Traffic: >1000 requests/minute (abnormal for MVP)

Alert Configuration:

resource alert 'Microsoft.Insights/metricAlerts@2018-03-01' = {
  name: 'mcp-high-error-rate'
  properties: {
    criteria: {
      allOf: [
        {
          criterionType: 'StaticThresholdCriterion'
          metricName: 'Requests'
          operator: 'GreaterThan'
          threshold: 5
          timeAggregation: 'Average'
          dimensions: [
            {
              name: 'StatusCode'
              operator: 'Include'
              values: ['5xx']
            }
          ]
        }
      ]
    }
    evaluationFrequency: 'PT1M'
    windowSize: 'PT5M'
    severity: 2
    actions: [
      {
        actionGroupId: alertActionGroup.id
      }
    ]
  }
}

Compliance Considerations

GDPR / CCPA

Requirements Met: - ✅ Data minimization (only collect necessary PII) - ✅ Purpose limitation (PII used only for loan processing) - ✅ Data encryption (in transit and at rest) - ✅ Audit logging (all data access logged) - ⚠️ Right to erasure (need to implement data deletion)

Future Work: - Data retention policies (delete applications after X months) - User consent management - Data portability (export user data) - Privacy impact assessment

SOC 2 / ISO 27001

Security Controls: - ✅ Access control (managed identities, RBAC) - ✅ Encryption (TLS, at-rest) - ✅ Monitoring (comprehensive logging) - ✅ Incident response (alerts configured) - ⚠️ Change management (need formal process)

Audit Evidence: - Deployment logs (GitHub Actions) - Access logs (Azure AD audit logs) - Change history (Git commits) - Security scanning (Dependabot, CodeQL)

PCI DSS (if handling payment info)

Current State: Not handling payment info (out of scope)

If needed in future: - Level 4 compliance (lowest level) - Encrypted transmission required ✅ (already have) - No storage of CVV/PAN ✅ (not storing) - Secure access controls ✅ (managed identities)


Cost Security

Resource Quotas

Prevent runaway costs:

resource mcpServer 'Microsoft.App/containerApps@2024-03-01' = {
  properties: {
    template: {
      scale: {
        maxReplicas: 3              // ✅ Hard cap on scaling
      }
      containers: [
        {
          resources: {
            cpu: json('0.5')        // ✅ Max 0.5 vCPU per container
            memory: '1Gi'           // ✅ Max 1 GB per container
          }
        }
      ]
    }
  }
}

Cost alerts:

resource budgetAlert 'Microsoft.Consumption/budgets@2021-10-01' = {
  name: 'mcp-servers-budget'
  properties: {
    amount: 200                     // $200/month budget
    category: 'Cost'
    timeGrain: 'Monthly'
    timePeriod: {
      startDate: '2025-01-01'
    }
    notifications: {
      '80percent': {
        enabled: true
        operator: 'GreaterThan'
        threshold: 80
        contactEmails: ['team@example.com']
      }
      '100percent': {
        enabled: true
        operator: 'GreaterThan'
        threshold: 100
        contactEmails: ['team@example.com']
      }
    }
  }
}

Cost Anomaly Detection

Azure Cost Management: - Daily cost review (automated) - Anomaly detection (AI-powered) - Budget forecasting (based on trends)

Expected costs (dev): - Normal: $40-55/month - Alert threshold: >$80/month - Investigation trigger: >$100/month


Disaster Recovery

Backup Strategy

Stateless Services (API, UI, MCP servers): - No backups needed (images in ACR) - Can redeploy from Bicep templates - RTO: 5-10 minutes (redeploy time) - RPO: 0 (no data loss)

Configuration Backup: - Bicep templates in Git (version controlled) - Environment variables documented - Deployment scripts automated

High Availability

Multi-Replica Deployment (production):

scale: {
  minReplicas: 2                   // ✅ Always 2+ instances
  maxReplicas: 10                  // Can scale to 10
}

Benefits: - Zero-downtime deployments (rolling updates) - Fault tolerance (1 replica fails, others continue) - Load distribution (requests spread across replicas)

Availability: - SLA: 99.95% uptime (Azure Container Apps) - Actual: 99.99%+ expected (with 2+ replicas)

Regional Failover (Future)

Current: Single region (East US 2)

Future Multi-Region:

┌──────────────────┐        ┌──────────────────┐
│  East US 2       │        │  West US 2       │
│  (Primary)       │◄──────▶│  (Secondary)     │
│  - All services  │        │  - All services  │
└──────────────────┘        └──────────────────┘
         │                           │
         └───────────┬───────────────┘
              ┌──────┴──────┐
              │ Traffic     │
              │ Manager     │
              │ (failover)  │
              └─────────────┘

Cost: 2x infrastructure ($400-500/month)
Benefit: 99.99%+ uptime guarantee
Recommendation: Not needed for MVP


Security Checklist

Pre-Deployment

  • All MCP servers configured with external: false
  • No secrets in environment variables (use secretRef)
  • Health check endpoints implemented
  • PII masking in logs verified
  • RBAC roles configured correctly
  • NSG rules reviewed and approved

Post-Deployment

  • Verify external access blocked (test from internet)
  • Verify internal access working (test from API)
  • Review Application Insights logs for errors
  • Confirm health checks passing
  • Test auto-scaling behavior
  • Set up cost alerts
  • Configure security monitoring alerts

Ongoing

  • Weekly: Review security logs for anomalies
  • Monthly: Review access logs and permissions
  • Quarterly: Security assessment and penetration testing
  • Annually: Compliance audit and certification renewal

References


Security Level: 🟢 HIGH - Production-ready with internal endpoints
Compliance Status: ✅ GDPR/CCPA-compliant (with minor enhancements needed)
Next Review: 2025-02-15 (30 days post-deployment)