Skip to content

Conversation

@haflidif
Copy link

@haflidif haflidif commented Dec 2, 2025

Features

Container App Jobs for Self-Hosted Agents (Azure DevOps)

  • Event-driven Azure DevOps agents using Container App Jobs with KEDA scaling
  • BYO mode integration with existing VNet, ACR, and managed identity resources
  • Private networking with managed identity authentication (no PAT in environment)
  • Separate image configuration for ACA vs ACI deployments
  • Structured naming convention with location abbreviations for 32-char limit compliance

Identity Module

  • Dedicated module for managing user-assigned managed identities
  • Federated credential configuration for OIDC authentication
  • Azure DevOps service principal entitlement for agent identity
  • Role assignment outputs for seamless integration

Configuration

Enable Container App Jobs:

use_self_hosted_agents: true
use_container_app_jobs: true  # Azure DevOps only
use_private_networking: true

Variables Added

Container App Jobs:

  • agent_container_app_image_tag - Image tag for ACA (default: "221742d")
  • agent_container_app_image_folder - Image folder for ACA (default: "azure-devops-agent-aca")
  • container_app_environment_name - Override environment name
  • container_app_job_name - Override job name
  • container_app_job_placeholder_name - Override placeholder job name
  • container_app_infrastructure_resource_group_name - Override infrastructure RG name

Naming:

  • service_name - Service identifier for structured naming
  • environment_name - Environment identifier for structured naming

Technical Details

  • Uses Azure/avm-ptn-cicd-agents-and-runners/azurerm v0.5 pattern module
  • Memory configuration: stored as number, formatted to "4Gi" string at API boundary
  • Naming pattern: {prefix}-{service}-{env}-{location_short}-{num}
  • Location abbreviation via substr(location, 0, 3) for name length compliance
  • Container App Environment, Job, and Placeholder Job all configurable

Files Changed

New modules:

  • modules/identities/ - Identity management module with federated credentials
  • modules/azure/container_app_jobs.tf - Container App Jobs deployment

Modified:

  • Azure DevOps agent identity permissions
  • Conditional image selection (ACA vs ACI)
  • Private networking configuration for Container Apps
  • Resource naming with location abbreviations

haflidif and others added 14 commits November 26, 2025 14:49
- Add resource group creation for bootstrap identities
- Add user-assigned managed identity resources (map-based)
- Add federated identity credential support for workload identity federation
- Support for plan, apply, and future agent identities
- Add comprehensive module documentation

BREAKING CHANGE: Identity resources moved to separate module for better separation of concerns
- Change from creating identities to accepting them as inputs
- Replace user_assigned_managed_identities with managed_identity_ids map
- Replace federated_credentials with managed_identity_client_ids map
- Add managed_identity_principal_ids for role assignments
- Remove identity resource group creation
- Delete managed_identity.tf (moved to identities module)
- Update role assignments to use passed-in principal IDs
- Update outputs to pass through identity client IDs

BREAKING CHANGE: Module no longer creates identities internally
- Add module.identities instantiation with Azure DevOps federated credentials
- Configure federated credentials with api://AzureADTokenExchange audience
- Update module.azure to consume identity outputs
- Update module.azure_devops to use identity client IDs
- Add moved blocks for backwards-compatible state migration
- Add tags variable support for identity resources

Moved blocks ensure existing deployments upgrade without resource recreation:
- module.azure.azurerm_resource_group.identity → module.identities.azurerm_resource_group.identity
- module.azure.azurerm_user_assigned_identity.alz → module.identities.azurerm_user_assigned_identity.identities
- module.azure.azurerm_federated_identity_credential.alz → module.identities.azurerm_federated_identity_credential.credentials
- Add module.identities instantiation with GitHub Actions federated credentials
- Configure federated credentials for GitHub workload identity federation
- Update module.azure to consume identity outputs
- Update module.github to use identity client IDs
- Add moved blocks for backwards-compatible state migration
- Add tags variable support for identity resources

Moved blocks ensure existing deployments upgrade without resource recreation:
- module.azure.azurerm_resource_group.identity → module.identities.azurerm_resource_group.identity
- module.azure.azurerm_user_assigned_identity.alz → module.identities.azurerm_user_assigned_identity.identities
- module.azure.azurerm_federated_identity_credential.alz → module.identities.azurerm_federated_identity_credential.credentials
- Add module.identities with conditional count parameter
- Conditionally create identities based on create_bootstrap_resources_in_azure
- Update module.azure to consume identity outputs with [0] indexing
- Add moved blocks for backwards-compatible state migration
- Add tags variable support for identity resources

Local orchestration pattern differs from azuredevops/github due to optional Azure resource creation.

Moved blocks ensure existing deployments upgrade without resource recreation:
- module.azure[0].azurerm_resource_group.identity → module.identities[0].azurerm_resource_group.identity
- module.azure[0].azurerm_user_assigned_identity.alz → module.identities[0].azurerm_user_assigned_identity.identities
- module.azure[0].azurerm_federated_identity_credential.alz → module.identities[0].azurerm_federated_identity_credential.credentials
…y_principal_ids

- Replace var.user_assigned_managed_identities with var.managed_identity_principal_ids
- Replace azurerm_user_assigned_identity.alz[each.key].principal_id with each.value
- Fixes role assignments for storage container and storage reader roles
- Completes azure module refactoring for identity consumption
This commit adds support for Azure Container App Jobs as an alternative
to Container Instances for running self-hosted Azure DevOps agents.

Changes:
- Add container_app_jobs module integration with AVM pattern module v0.5
- Add use_container_app_jobs variable to enable/disable Container App Jobs
- Add Container App subnet configuration (10.0.4.0/23)
- Add agent_container_cpu and agent_container_memory variables (defaults: 2 cores, 4Gi)
- Add managed identity support for agent authentication (no PAT token required)
- Add Container App Environment with Log Analytics workspace integration
- Update resource dependencies and role assignments for BYO mode

Fixes in this amendment:
- Remove duplicate agent_pool_name variable declaration (line 274)
- Correct AVM module parameters to match v0.5.0 interface
- Add required parameters: location, postfix, version_control_system_organization, version_control_system_type
- Add compute_types for azure_container_app
- Use flat identity structure (user_assigned_managed_identity_id/client_id/principal_id)
- Use correct BYO parameters (virtual_network_name, container_registry_name)
- Add Container App configuration (cpu, memory, execution counts, polling interval)
- Use UAMI authentication with null PAT token
- Add random_string resource for postfix requirement

Co-authored-by: GitHub Copilot <noreply@github.com>
…eters

- Add virtual_network_id parameter required for private DNS zone link
- Use container_app_subnet_id instead of incorrectly named virtual_network_subnet_id
- Required for Container App Environment with private networking
- Use custom_container_registry_login_server instead of container_registry_name
- Add container_registry_private_endpoint_subnet_id for private networking
- Enable container_registry_private_dns_zone_creation_enabled
- Remove double 'Gi' suffix from container_app_container_memory (already in variable)
- Fixes null registry_login_server error in container-app-job module
…zone duplicate

- Add time_sleep resource for 30s UAMI propagation delay
- Add azuredevops_service_principal_entitlement to register UAMI as service principal
- Change group membership to use entitlement descriptor instead of principal_id
- Fix duplicate DNS zone link by using existing container_registry DNS zone
- Set container_registry_private_dns_zone_creation_enabled = false

Fixes two deployment errors:
1. UAMI not registered in Azure DevOps (controller not found error)
2. Duplicate private DNS zone virtual network link error

Pattern based on AVM module example:
https://github.com/Azure/terraform-azurerm-avm-ptn-cicd-agents-and-runners/blob/main/examples/azure_devops_container_app_uami/main.tf
Add support for deploying Azure DevOps agents using Azure Container App Jobs
as an alternative to Azure Container Instances. Container App Jobs provide
event-driven scaling with KEDA and better integration with Container Apps
infrastructure.

Key features:
- BYO (Bring Your Own) mode integration with existing infrastructure
- Custom container image support with configurable repository and tags
- Managed identity authentication for ACR access
- Private networking support with VNet integration
- Structured naming convention following bootstrap patterns

Changes:
- Add container_app_jobs.tf module for Container App Jobs deployment
- Integrate with AVM pattern module (avm-ptn-cicd-agents-and-runners v0.5)
- Add variables for Container App naming (environment, job, placeholder)
- Configure memory formatting (4 -> 4Gi) for Container Apps API
- Add service_name and environment_name variables for naming
- Add separate image configuration for ACA vs ACI deployments
- Use location abbreviation for Container App Jobs naming (32 char limit)

Resources created:
- Container App Environment (cae-*)
- Container App Job (caj-*)
- Container App Job Placeholder (caj-*-ph)
- Infrastructure Resource Group (rg-*-ca-infra)

New variables:
- agent_container_app_image_tag: Image tag for Container App Jobs (default: 221742d)
- agent_container_app_image_folder: Dockerfile folder for ACA (default: azure-devops-agent-aca)
- container_app_environment_name: Name for Container App Environment
- container_app_job_name: Name for Container App Job
- container_app_job_placeholder_name: Name for placeholder job
- container_app_infrastructure_resource_group_name: Name for infra RG
@haflidif haflidif changed the title Add Container App Jobs and Identity Module feat: Add Container App Jobs and Identity Module Dec 2, 2025
Resolved conflicts and integrated new file_manipulation module from main.
All Container App Jobs features preserved:
- Identity module with federated credentials
- Container App Jobs support with BYO mode
- Separate ACA/ACI image configuration
- Structured naming with location abbreviations

New from main:
- file_manipulation module for bicep-classic support
- Updated CI/CD pipelines (bicep-classic variants)
- Cleanup scripts improvements
- Module refactoring (removed template_architecture_definition)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants