Packer is a great tool to build images. This blog is about using YAML to build your Packer Azure DevOps Pipeline. When using YAML, it is very easy to replicate your pipeline to other Azure DevOps organizations. So, what is the architecture about?

Firstly, let’s start with this high level design of a packer build:

Packer YAML Azure DevOps Pipeline - High level design

The packer build runs basically in 3 “main steps”.

1. The windows based agent which is connected to your Azure DevOps environment starts the build. Packer is downloaded alongside the templates from the DevOps GIT repository.

2. Packer runs on the “Windows Based Agent” and start to create a seperate resource group where the VM is build. Firstly, the VM is created. After that, the provisioning process starts. Windows Updates and applications are installed.

3. The last step is that the VM is sysprepped and generalized to output a managed image in a seperate resourcegroup. After that, the resource group with the VM and the resources needed is deleted.

In addition, a forth step could be that the managed image is uploaded to a Azure Compute Gallery. I will write a part 2 to this blog which features adding the shared image gallery to this build.

Prerequisites

You need to have your Azure DevOps organization and Azure tenant setup. I have written a blog on how to do so. This is the link.

After that, you are ready to start this blog!

Add Packer to Azure DevOps

Firstly, we need to add Packer to Azure DevOps in order to use this feature.

Log on to Azure DevOps.

Next, go to Organization settings:

Packer YAML Azure DevOps Pipeline - Organization Settings

After that, go to extensions:

Packer YAML Azure DevOps Pipeline - DevOps extensions

Browse the marketplace and search for Packer:

Packer YAML Azure DevOps Pipeline - Marketplace
Packer YAML Azure DevOps Pipeline - Marketplace Packer

Click on “Get it free”:

Packer YAML Azure DevOps Pipeline - Packer get

Select your organization and click install:

Packer YAML Azure DevOps Pipeline - Packer install
Packer YAML Azure DevOps Pipeline - Packer installation finished

Packer is now available in your organization:

Packer YAML Azure DevOps Pipeline - Packe in DevOps portal

Create the Key Vault

Firstly, we need to make sure that the “Service Principal Application ID” and the “Service Principal Secret” are available in your Azure Key Vault.

If you don’t have an Azure Key Vault, we first need to create one.

Go to the Azure Portal and click on create a resource:

Packer YAML Azure DevOps Pipeline - Create Key Vault

After that, search for “Key vault” and create one:

Packer YAML Azure DevOps Pipeline - Create Key Vault 2
Packer YAML Azure DevOps Pipeline - Create Key Vault 3

Make sure your Service Principal may access the Key vault:

Packer YAML Azure DevOps Pipeline - Create Key Vault 4
Packer YAML Azure DevOps Pipeline - Create Key Vault 5
Packer YAML Azure DevOps Pipeline - Create Key Vault 6

I am allowing all networks. If you limit this option, make sure your Azure DevOps environment and Agent can reach the keyvault.

After this, go to review and create.

Lastly, create the Key Vault.

Go to your keyvault and go to “Secrets”:

Add your Service Principal App ID and Service Principal Secret:

Go back to Azure DevOps. Go to Pipelines and to Library:

Create a new variable group:

Call it Azure Key Vault and add the variables from the newly created Azure Key Vault:

Now you can use the variables in your pipelines.

Create the Packer Pipeline

Variable Groups

Firstly, we need another variable group. Go back pipelines and to library:

This image has an empty alt attribute; its file name is image-27.png

Create a new variable group:

This image has an empty alt attribute; its file name is image-28.png

Add these variables to the group:

ARM_Subscription_ID –> the ID of the subscription that you want to deploy in. (The service connection must also be mapped to this subscription.

Location –> This is the location of the Resource Group for the Managed Image.

resourcegroupname –> This is the name of the Resource Group for the Managed Image.

ServiceConnectionName –> This is the name of the Service Connection used.

Template Files

After that, we add the packer files to Azure DevOps. I put these files in Github account. You can find them here:

1. Windows 10 Packer template file.

2. Windows 10 Packer variable file.

3. Windows 10 Packer YAML file.

4. ARM template for creating a resource group.

Put these file in your GIT repo in a folder called “Packer”. Furthemore, if you put the files here. you don’t have to change the paths in the YAML file later.


In addition, I have also displayed the YAML file here. I need to explain a few variables.

trigger: none

pool:
  name: 'AGENT POOL NAME'

variables:
- group: 'PackerVariables'
- group: 'YOUR VARIABLE GROUP (KEYVAULT)'

stages:
- stage: CreateResourceGroup
  jobs:
  - job: CreateResourceGroup
    steps:
    - task: AzureResourceManagerTemplateDeployment@3
      inputs:
        deploymentScope: 'Resource Group'
        azureResourceManagerConnection: '$(ServiceConnectionName)'
        subscriptionId: '$(ARM_Subscription_ID)'
        action: 'Create Or Update Resource Group'
        resourceGroupName: '$(resourcegroupname)'
        location: '$(Location)'
        templateLocation: 'Linked artifact'
        csmFile: 'Packer/EmptyResourceGroup.json'
        deploymentMode: 'Incremental'

- stage: RunPacker
  jobs:
  - job: CreateImage
    steps:
    - task: riezebosch.Packer.PackerTool.PackerTool@0
      displayName: 'Use Packer Latest'

    - task: riezebosch.Packer.Packer.Packer@1
      displayName: 'Packer version'
      inputs:
        azureSubscription: '$(ServiceConnectionName)'
        templatePath: 'Packer/Windows10AVDImage.pkr.hcl'
        command: version

    - task: riezebosch.Packer.Packer.Packer@1
      displayName: 'Packer init'
      inputs:
        azureSubscription: '$(ServiceConnectionName)'
        templatePath: 'Packer/Windows10AVDImage.pkr.hcl'
        command: init

    - task: riezebosch.Packer.Packer.Packer@1
      displayName: 'Packer validate'
      inputs:
        azureSubscription: '$(ServiceConnectionName)'
        templatePath: 'Packer/Windows10AVDImage.pkr.hcl'
        command: validate
        variables-file: 'Packer/Windows10AVDVariables.pkr.hcl'

    - task: riezebosch.Packer.Packer.Packer@1
      displayName: 'Packer build'
      inputs:
        azureSubscription: '$(ServiceConnectionName)'
        templatePath: 'Packer/Windows10AVDImage.pkr.hcl'
        variables-file: 'Packer/Windows10AVDVariables.pkr.hcl'

Firstly, you need to change these variables in the YAML file:

‘AGENT POOL NAME’ –> Name of the pool of agents created in the preparation part.

Don’t change the name of the ‘PackerVariables’ group.

‘YOUR VARIABLE GROUP (KEYVAULT)’ –> If you followed this blog, this ‘AzureKeyVault’. If you did not, please enter your own Key Vault name.

Furthermore, if you did not put the files in the folder “Packer” please change these variables:

For the ResourceGroup creation:

And foreach Packer task:

After that, we edit the Packer Variables files. The file looks like this:

Make sure you edit the following variables:

client_id –> Insert he variable from the AzureKeyVault variable group.
client_secret –> Insert he variable from the AzureKeyVault variable group.
virtual_network_name –> Name of your Virtual Network in Azure.
virtual_network_resource_group_name –> Name of your Virtual Network ResourceGroup Name.
virtual_network_subnet_name –> Name of the subnet in Azure.

In addition, by default the private_virtual_network_with_public_ip is set to false. Make sure that your agent can make a WinRM connection to the private IP address of the VM that is created by Packer. If you are not sure, set this to “true”.

Create Pipeline

Lastly, it is finally time to create the Packer Pipeline itself. Go Pipelines and select Pipelines:

After that, create a new pipeline:

In addition, if you don’t have any pipeline it will ask you to create your first pipeline.

Lastly, click on save:

And that’s how your create a Packer YAML pipeline in Azure Devops!

Packer YAML pipeline example run

Click on Run Pipeline:

Firstly, the resource group task runs:

After that, the packer build starts:

WinRM connects, downloads and install Windows Updates:

Reboot & Sysprep:



Lastly, the managed image:

References

Packer documentation

23 thoughts on “Packer YAML Azure DevOps Pipeline Pt. 1 – Create the Packer YAML Pipeline”
  1. Hi Niels
    I love the work you do and your blog

    I have few questions
    1. Service Principal App ID and Service Principal Secret: the Principal App ID is the App registrations that we did? but where did we set a Service Principal Secret?
    2.
    virtual_network_name –> Name of your Virtual Network in Azure.
    virtual_network_resource_group_name –> Name of your Virtual Network ResourceGroup Name.
    virtual_network_subnet_name –> Name of the subnet in Azure.
    should we make a new RG , Network add the valuse as there where no step to make new RG etc?

    3. when i hit Click on Run Pipeline:
    i get
    An error occurred while loading the YAML build pipeline. Variable group was not found or is not authorized for use. For authorization details, refer to https://aka.ms/yamlauthz.
    i thikn Point 1 i did worng ?

    1. Hi Raf,

      Thanks for the compliment!

      To answer your questions:

      1. The secret was created in the preparation blog. But this blog will show you how to do so (Authentication, Option 2) https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/active-directory/develop/howto-create-service-principal-portal.md

      2. I would advise you to create a separate Resource Group and create the Virtual Network there. Then add the values of the virtual network to your variable file.

      3. Go to Pipelines > Library > Click on your variable group > in the top bar you have the option for “Pipeline Permissions” make sure that your Pipeline is added here.

      Thanks,

      Niels

  2. Hi Niels

    for point 3 i did add pPermissions but still fail but when i go to”Stages to run” i see error
    Unable to load the pipeline’s stages.”: i did use your ymail so not sure about the error

    thx for help

    1. Hi Raf,

      Did you change the YAML file to your environment accordingly? The error states that the YAML syntax is incorrect.

      Thanks,
      Niels

  3. Hi,
    Here is a typical question from our customers – why would NOT you use Azure native tool , AIB , and sticking using Packer .. ?
    What is the reasoning ? Or what are benefits using Packer for typical AVD image deployment ( for example) ..
    How would have you answered such a question?
    (PS customization tasks to be used within a template – only PowerShell/cmd)
    TIA

    1. Hi Maxim,

      Thanks for your question. Packer is multicloud and it also can be used on-premises. That way I always can use the same template for all images for all clouds and infrastructures. If you only use Azure you could use Azure Image Builder also. It is very much like packer.

      Thanks,

      Niels

  4. Hi Niels,
    Excellent job and post.
    My question is about to update an existing image in a shared Gallery, Which parameter, variable i need to change or add?

    1. Hi Javier,

      Thanks! In part 2 you need to change these parameters:

      -SharedImageGalleryName ”NKO_SharedImageGallery” -SharedImageGalleryRG ”RG_WE_AVD_SharedImageGallery” -SharedImageGalleryDefinitionName ”AVDImage”

      If you have any more questions, just let me know. 🙂

      Niels

  5. Hi Niels,
    I was trying to update an image from a Imagegallery and i can´t.
    Reviewing the files ” Windows10AVDImage.pkr.hcl” and “Windows10AVDVariables.pkr.hcl” . What´s “custom_managed_image_name” in a image gallery or it´s necessary to use another parameter?
    Case:
    21H2_avd is a VM image definition in a Azure compute Gallery” in n the resource group ‘rg_packer”
    When build the pipeline, appear the next error;
    ” azure-arm.windowsvm: Cannot find an image named ’21H2_avd’ in the resource group ‘rg_packer”
    Any idea to resolve this issue
    Thanks
    Javier

    1. Hi Javier,

      Did you complete the second part of this series? If you did not, please do so and try again.

      Kind Regards,

      Niels

      1. Hi Niels,
        Thanks for your answered
        I have create the image, upload programs, execute powershell and upload it a Gallery, but always with Public IP, It´s possible to create an Image without Public IP?
        Thanks
        Javier

        1. Hi Javier,

          It is possible. Please search for “we edit the Packer Variables files” in the post and that’s where I describe how to do so.

          Kind Regards,

          Niels

  6. hi Niels, are you using azure devops linux agent or windows agent ?
    my pipeline failing on packer version task trying to load packer config – it is 100% related to windows update plugin

    ##[debug]templatePath=/home/vsts/work/1/s/Packer Build/packer_BYOD.pkr.hcl
    ##[debug]check path : /home/vsts/work/1/s/Packer Build/packer_BYOD.pkr.hcl
    ##[debug]/opt/hostedtoolcache/packer/1.8.0/x64/packer arg: /home/vsts/work/1/s/Packer Build/packer_BYOD.pkr.hcl
    ##[debug]exec tool: /opt/hostedtoolcache/packer/1.8.0/x64/packer
    ##[debug]arguments:
    ##[debug] version
    ##[debug] -var
    ##[debug] client_id=***
    ##[debug] -var
    ##[debug] client_secret=***
    ##[debug] -var
    ##[debug] subscription_id=0a6b8caa-6c92-46a4-8436-5f284082c02b
    ##[debug] -var
    ##[debug] tenant_id=56be28a2-156c-41ee-9970-91a4eceaa43a
    ##[debug] /home/vsts/work/1/s/Packer Build/packer_BYOD.pkr.hcl
    /opt/hostedtoolcache/packer/1.8.0/x64/packer version -var client_id=*** -var client_secret=*** -var subscription_id=0a6b8caa-6c92-46a4-8436-5f284082c02b -var tenant_id=56be28a2-156c-41ee-9970-91a4eceaa43a /home/vsts/work/1/s/Packer Build/packer_BYOD.pkr.hcl
    Error loading configuration:

    exit status 2
    ##[debug]Exit code 1 received from tool ‘/opt/hostedtoolcache/packer/1.8.0/x64/packer’
    ##[debug]STDIO streams have closed for tool ‘/opt/hostedtoolcache/packer/1.8.0/x64/packer’
    ##[debug]task result: Failed
    ##[error]Error: The process ‘/opt/hostedtoolcache/packer/1.8.0/x64/packer’ failed with exit code 1
    ##[debug]Processed: ##vso[task.issue type=error;]Error: The process ‘/opt/hostedtoolcache/packer/1.8.0/x64/packer’ failed with exit code 1
    ##[debug]Processed: ##vso[task.complete result=Failed;]Error: The process ‘/opt/hostedtoolcache/packer/1.8.0/x64/packer’ failed with exit code 1
    Finishing: Packer version

    1. Hi Maxim,

      Thanks for your message. I am using a Windows Based Agent. I think that is the problem indeed.

      You could look at windows based agent in a container.

      Kind Regards,

      Niels

Leave a Reply

Your email address will not be published.