Chaos to Order: Structuring IaC repository Like a Boss! 👑

With this post I want to share my new GitHub repository -> Starter Kit for an Azure IaC repository . This starter kit is based on best practices, personal experiences, and guidelines for creating and organizing code and resources tailored for Azure infrastructure deployment. While the example itself is based on a combination of Azure DevOps, Bicep and PowerShell, the concepts are applicable to any IaC language and tool combination.

Why a starter kit?

The goal of this repository is to help engineers reduce their start-up time by providing the necessary structure. The goal of this post is to elaborate on the repository structure and provide some background information.

The starter kit

The starter kit is composed of 5 top level folders with 📂.vscode folder which contains my configuration for Visual Studio Code. The 📜extensions.json file contains the extensions needed by most Microsoft Azure IaC projects. The most notable config in the 📜settings.json is the tab size, PowerShell code formatting and the horizontal editor rulers set to 80, 100 and 120 characters. These ruler settings represent the Guidance for contributing to Microsoft Docs and the PSScriptAnalyzer rule AvoidLongLines . Like all other parts of the starter kit, feel free to change settings to your liking. In addition to the top level folders the starter kit also has a generic 📜.gitignore and the main 📜.README.md which contains the description for this kit. Obviously the 📜LICENSE file is not part of the starter kit.
Each folder is described in a separate chapter of this blog post, enjoy the read!

The main tree view of the Starter Kit for an Azure IaC repository is shown below.

📦azure-iac-repo-structure-starter-kit
 ┣ 📂.vscode
 ┣ 📂docs
 ┣ 📂pipeline-gallery
 ┣ 📂pipelines
 ┣ 📂src
 ┣ 📂tools
 ┣ 📜.gitignore
 ┣ 📜LICENSE
 ┗ 📜README.md

Docs

Moving trough the top level folders, the first one is the 📂docs folder. This is the folder to keep all the technical documentation considered relevant for the IaC files and scripts. Following hte everything as code mantra the documentation must be written in Markdown. Each markdown file should cover a single topic and should be named accordingly. Since each project tends to have it’s own documentation requirements, the documentation folder is not further structured into subfolders.

Pipeline gallery

The pipeline-gallery folder is the place to keep all the pipeline templates. Usage of pipeline templates encourages reusability by bootstrapping new pipelines. In addition the use of pipeline templates positively impacts the maintainability and testing of pipelines as each template cna be tested in isolation. As the concept of this folder is based on the DRY principle the added value of this folder depends on the reusability scale of the pipelines templates. If your needs require the reuse the templates across multiple repositories opting for a specific repository with an Artifacts feed might be a more suited option.

For consistency reasons the folder is structured into two subfolders which are based on the pipeline type. With the 📂jobs folder for all the job templates and the 📂steps folder for all the step templates. Populated with files, it looks something like this:

📦pipeline-gallery
 ┣ 📂jobs
 ┃ ┣ 📜deploySolution.yml
 ┃ ┣ 📜deployInfrastructure.yml
 ┃ ┣ 📜staticAnalysis.yml
 ┃ ┗ 📜codeQualityValidation.yml
 ┣ 📂steps
 ┃ ┣ 📜buildSolution.yml
 ┃ ┣ 📜staticAnalysis.yml
 ┃ ┣ 📜publishSolutionArtifacts.yml
 ┃ ┗ 📜buildBicep.yml
 ┗ 📜readme.md

Pipelines

The pipelines folder is the place to keep all the pipeline definitions. Each pipeline definition is placed inside a subfolder which is named after the pipeline. Depending on the design of the pipeline, there are two options to choose from:

• A single multistage 📜.yml file that contains all the stages or environments.
• Multitude of 📜.yml files to represent each stage or environment.

And finally a 📜readme.md that contains technical documentation elaborating on the specifics of the respective pipeline is required. Populated with files, it looks something like this:

📦pipelines
 ┣ 📂 management
 ┃ ┣ 📜 managementStructure.yml
 ┃ ┗ 📜 readme.md
 ┣ 📂 workload
 ┃ ┣ 📜 deploy-workload.yml
 ┃ ┗ 📜 readme.md
 ┣ 📂 microservice-deploy
 ┃ ┣ 📜 deploy-microservice.yml
 ┃ ┗ 📜 readme.md
 ┗ 📜 readme.md

Src

The src folder is the place to keep all the source code. The heart of the repository and the folder with the deepest structure. Due to the depth of this folder each subfolder will be described in a separate chapter. But first lets cover the top level folders.

• 📂iac (Infrastructure as Code) directory contains a structured folder hierarchy for managing Infrastructure as Code templates, such as ARM, Terraform, and Bicep.\
• 📂policies directory is home to all policy and initiative definitions.\
• 📂params (Parameters) directory is intended to store non-sensitive configuration and parameter files.
• 📂scripts directory is organized with a folder structure designed for the effective management of (PowerShell) scripts.

The folder structure looks like this:

📦src
 ┣ 📂iac
 ┣ 📂params
 ┣ 📂policies
 ┣ 📂scripts
 ┗ 📜readme.md

IaC

The target folder for storing Infrastructure as Code files. Based on my personal experience I ended up dividing all IaC files into two categories, namely: 📂az-modules and 📂az-controllers. Where the 📂az-modules are the building blocks of the infrastructure and 📂az-controllers are the orchestrators of the modules.

The folder structure looks like this:

📦iac
 ┣ 📂az-controllers
 ┣ 📂az-modules
 ┗ 📜readme.md

Controllers

As mentioned this folder contains IaC files that are purposed to as orchestrators. Meaning IaC files which are intended to utilize one or more modules to deploy resources. Each controller is placed in a similar named folder. The controller folder contains two subfolders: 📂samples and 📂test.

• 📂samples folder contains a script which can be used to deploy the controller from a local machine (without the use of a pipeline).
• 📂test folder contains all the test related materials. Like unit tests, scenario specific parameter files, and a 📜readme.md file that describes the controller and its usage.

The module file itself is typically named equally to the folder but this is based on personal preference. Populated with files, it looks something like this:

📦az-controllers
 ┣ 📂defender-plans
 ┃ ┣ 📂samples
 ┃ ┃ ┗ 📜localDeploy.ps1
 ┃ ┣ 📂test
 ┃ ┃ ┣ 📜happy-flow.params.json
 ┃ ┃ ┣ 📜invalid-setting.params.json
 ┃ ┃ ┗ 📜unhappy-flow.params.json
 ┃ ┣ 📜defender-plans.bicep
 ┃ ┗ 📜readme.md
 ┣ 📂subscription-vending
 ┃ ┣ 📂samples
 ┃ ┃ ┗ 📜localDeploy.ps1
 ┃ ┣ 📂test
 ┃ ┃ ┣ 📜happy-flow.params.json
 ┃ ┃ ┣ 📜invalid-setting.params.json
 ┃ ┃ ┗ 📜unhappy-flow.params.json
 ┃ ┣ 📜subscription-vending.bicep
 ┃ ┗ 📜readme.md
 ┗ 📜readme.md

Modules

As mentioned this folder contains IaC files that are used as a unit of code reuse. A module file can contain a single resource or set of resources, and can be called from controllers. This allows deployments to be constructed from small, manageable components, and to reuse common pieces.

For the purposes of traceability and discoverability, each module is organized in a folder structure that aligns with the Azure Resource Provider structure. E.g. Microsoft.Storage/storageaccounts. An additional advantage of this approach is that the required Azure Resource Providers are immediately visible. Furthermore, automation can be established to manage any drift in the providers, based on the modules.\

Like the controller the module folder contains two subfolders: 📂samples and 📂test.

• 📂samples folder contains a script which can be used to deploy the controller from a local machine (without the use of a pipeline).
• 📂test folder contains all the test related materials. Like unit tests, scenario specific parameter files, and a 📜readme.md file that describes the controller and its usage.

The module itself is typically named 📜deploy.bicep but this is based on personal preference. Populated with files, it looks something like this:

📦az-modules
 ┣ 📂Microsoft.KeyVault
 ┃ ┗ 📂vaults
 ┃   ┣ 📂samples
 ┃   ┃ ┗ 📜localDeploy.ps1
 ┃   ┣ 📂test
 ┃   ┃ ┣ 📜happy-flow.params.json
 ┃   ┃ ┣ 📜invalid-setting.params.json
 ┃   ┃ ┗ 📜unhappy-flow.params.json
 ┃   ┣ 📜deploy.bicep
 ┃   ┗ 📜readme.md
 ┣ 📂Microsoft.Network
 ┃ ┣ 📂dnszones
 ┃ ┃ ┣ 📂samples
 ┃ ┃ ┃ ┗ 📜localDeploy.ps1
 ┃ ┃ ┣ 📂test
 ┃ ┃ ┃ ┣ 📜happy-flow.params.json
 ┃ ┃ ┃ ┣ 📜invalid-setting.params.json
 ┃ ┃ ┃ ┗ 📜unhappy-flow.params.json
 ┃ ┃ ┣ 📜deploy.bicep
 ┃ ┃ ┗ 📜readme.md
 ┃ ┗ 📂virtualnetworks
 ┃   ┣ 📂samples
 ┃   ┃ ┗ 📜localDeploy.ps1
 ┃   ┣ 📂test
 ┃   ┃ ┣ 📜happy-flow.params.json
 ┃   ┃ ┣ 📜invalid-setting.params.json
 ┃   ┃ ┗ 📜unhappy-flow.params.json
 ┃   ┣ 📜deploy.bicep
 ┃   ┗ 📜readme.md
 ┣ 📂Microsoft.Storage
 ┃ ┗ 📂storageaccounts
 ┃   ┣ 📂samples
 ┃   ┃ ┗ 📜localDeploy.ps1
 ┃   ┣ 📂test
 ┃   ┃ ┣ 📜happy-flow.params.json
 ┃   ┃ ┣ 📜invalid-setting.params.json
 ┃   ┃ ┗ 📜unhappy-flow.params.json
 ┃   ┣ 📜deploy.bicep
 ┃   ┗ 📜readme.md
 ┗ 📜readme.md

Params

The purpose of this folder is to provide a centralized location for all parameter files, making them easier to manage and maintain.

Policies

In some cases this is a optional folder as not all projects require the use of policies. Depending on the number of policies and the complexity of their design, the policies folder can be structured split into two subfolders: 📂initiatives and 📂policies.

Scripts

A centralized location for all scripts, making them easier to manage and maintain. The folder is organized based on the PoshCode - PowerShellPracticeAndStyle . Similar to the IaC folder this folder is organized into 📂controllers and 📂functions. In this case the 📂functions are the building blocks and the 📂controllers are the orchestrators. Each controller script should be placed in a separate file. The file name should represent the purpose of the controller script. Each function should be placed in a separate file. The file name should be the same as the function name.

The folder structure looks like this:

📦scripts
 ┣ 📂controllers
 ┃ ┣ 📜roleDefinitionsForWorkloads.ps1
 ┃ ┣ 📜subscriptionVending.ps1
 ┃ ┣ 📜policyDefinitions.ps1
 ┃ ┗ 📜roleAssignmentsForWorkloads.ps1
 ┣ 📂functions
 ┃ ┣ 📜New-AzAdApp.ps1
 ┃ ┣ 📜Set-AzRoleAssignment.ps1
 ┃ ┣ 📜New-AdoAuthenticationToken.ps1
 ┃ ┗ 📜Confirm-AdoGroupMembership.ps1
 ┗ 📜readme.md

Tools

This is the main directory for any and all utility files. The utilities are categorized into three subdirectories:

• 📂helpers directory is purposed for utility scripts which are essential for development and maintenance anything inside the 📂src folder. For example, a helper to establish local debugging.
• 📂private directory is designated for private and personal scripts used during the debugging and building processes. For example, private configuration files like I explained in an earlier post /posts/2023/dont-spill-the-beans-keep-your-secrets-secure/ .
• 📂references directory serves as a storage for unused but potentially valuable files. For example, a script that has been decommissioned from the main source but is yet valuable.

The folder structure looks like this:

📦tools
 ┣ 📂helpers
 ┣ 📂private
 ┣ 📂references
 ┗ 📜readme.md

Wrapping up

I hope you found the explanation of the Azure iac repo structure starter kit educational and the starter kit useful. If you are interested in the reference material used to make this post, please visit the following links.

• Starter Kit for an Azure IaC repository
• Azure Resource Provider
• Contributing to PowerShell documentation
• PSScriptAnalyzer
• SOLID principle
• DRY principle

As always, a big thanks for reading this post. If you liked it, don’t be shy and have a look at my other posts .