Automation Pipelines

Automation pipelines are a set of automated processes that allow you to build, test, and deploy your code in a consistent and repeatable manner. They are essential for ensuring that your code is always in a deployable state and that any changes made to the code are thoroughly tested before being deployed to production.

We hear this in the context of software development, which isn’t what you are normally used to discussing with network operations. Yet many of these concepts are applicable to network operations as well. In the context of Network as Code with Nexus Dashboard, we utilize the underlying toolbase to automate the deployment of configuration. Yet the important element is that network changes are integrated into this software pipeline such that changes are integrated with validation and other software practices.

What is an Automation Pipeline?

An automation pipeline is a series of automated steps that take your code from development to production. It typically includes the following stages:

1. Source Control: The code is stored in a version control system (VCS) such as Git. This allows for tracking changes, collaboration, and rollback if necessary.
2. Build: The code is built into a deployable artifact. This can include compiling code, packaging files, and preparing the environment for deployment.
3. Test: Automated tests are run to ensure that the code is functioning as expected. This can include unit tests, integration tests, and end-to-end tests.
4. Deploy: The code is deployed to the target environment, such as a production server or a network device.

We can make the connection between these high level software concepts and network operations by looking at the following table:

When you observe this in the context of software development, developers commit code into branches that are processed via software linting and other tools to ensure the code matches expected standards. Once peer review approves the code this is then moved to control branches like develop for further testing and integration. Finally code is then merged, compiled and tested to established standards.

The advantage of this for network configuration changes is the ability to integrate what we have mentioned before: Semantic validation, pre and post testing, and automated deployment. Network engineers simply codify intended state and the software pipeline is executed transparently to ensure all the required steps are completed to push the configuration correctly to the network.

Pipelines and Network as Code for Nexus Dashboard

For Network as Code with Nexus Dashboard, we combine the different stages of execution in Ansible to create different stages based on the Ansible roles.

Complete Pipeline Flow Overview:

Validation Stage (pre-change validation)

The validation stage is the first and most critical step in our pipeline, serving as a gatekeeper to ensure that only valid configurations proceed to deployment. This stage leverages the Ansible validate role, which performs comprehensive checks on the data model before any changes are applied to the network infrastructure.

What happens during validation:

1. Data Model Assembly: The validation process reads all files in the host_vars directory and creates a single, unified data model in memory for processing.

2. Schema Compliance: The data model is validated against the predefined schema to ensure all required fields are present and data types are correct. This includes checking for mandatory fields like switch serial numbers and proper IP address formats.

3. Semantic Validation: Beyond basic schema compliance, the validation includes semantic checks that ensure the configuration makes logical sense from a networking perspective. This might include verifying that VLAN ranges don’t overlap, IP subnets are properly configured, or that fabric parameters are consistent across devices.

4. Custom Rules Engine: The validation stage can incorporate custom rules that enforce network-specific policies. These rules are written in Python and can range from simple naming convention enforcement to complex interconnectivity validation that prevents common configuration mistakes. These rules are available as part of the VXLAN as Code service offering.

Validation Flow

Branch Strategy Integration: We can implement validation across different Git branching strategies to provide early feedback to network engineers. When a network operator commits changes to a feature branch, the pipeline automatically runs validation-only jobs, allowing for rapid iteration and error correction before the changes are merged into the main branch.

Benefits of Early Validation:

• Fast Feedback Loop: Engineers receive immediate feedback on their changes without waiting for full deployment cycles
• Reduced Risk: Invalid configurations are caught before they can impact the network
• Collaborative Development: Multiple engineers can work on different aspects of the network configuration with confidence that their changes will be validated consistently

Deployment Stage

The deployment stage transforms validated configuration data into actual network state through a carefully orchestrated series of Ansible roles. This stage represents the critical transition from intent (as expressed in the data model) to reality (as configured on network devices via Nexus Dashboard).

Deployment Process Flow:

1. Create Role Execution: The Ansible create role processes the validated data model and generates all necessary templates and variable parameters required for VXLAN fabric deployment. This role converts the declarative data model into the specific configuration templates that the Nexus Dashboard (ND) requires.

2. Fabric State Creation: The create role establishes the fabric state within ND, preparing all configuration objects, policies, and templates that will be applied to the managed devices.

3. Deploy Role Execution: The deploy role takes the fabric state created by the create role and applies it to the actual network devices managed by ND. This ensures that switches receive their intended configuration and are brought into the desired state.

4. Declarative Consistency: A key advantage of this approach is that the deployment process automatically handles both additions and removals. If configuration elements are removed from the data model, the pipeline ensures they are also removed from the network devices, maintaining perfect synchronization between declared intent and actual network state.

Safety Mechanisms:

• Dependency Validation: The deploy stage has explicit dependencies on successful validation, ensuring that only verified configurations are applied
• Deployment Control Options: The pipeline can be configured for fully automated deployment on merge to production branches, or with manual approval gates for additional human oversight. This flexibility allows organizations to choose the appropriate level of automation based on their risk tolerance and branching strategy

Deployment Process Flow:

Testing Stage (post-deployment validation)

The testing stage provides critical verification that the deployed configuration matches the intended state and that the network is functioning as expected. This stage uses the nac-test tool (previously known as iac-test) to perform comprehensive post-deployment validation.

Note: The current testing capabilities focus primarily on configuration accuracy validation. The testing ensures that what was deployed matches the intended configuration defined in the data model.

What Testing Validates:

1. Configuration Accuracy: Verifies that the configuration deployed to Nexus Dashboard exactly matches what was defined in the data model. This includes fabric settings, device configurations, and policy applications. The testing stage checks for discrepancies between the intended configuration and the actual state on the devices.

Testing Process:

• Template-Based Testing: The nac-test tool uses Jinja2 templates that define the expected state based on the data model. These templates generate specific test cases that are executed against the actual network state.

• Human-Readable Output: Test results are generated in HTML report format for easy human review and analysis.

• Comprehensive Reporting: Failed tests provide detailed information about what was expected versus what was found, making troubleshooting efficient and precise.

Testing Process Overview:

Integration with Pipeline:

• Artifact Collection: All test results are stored as pipeline artifacts, providing a permanent record of network state validation
• Failure Handling: Test failures provide immediate feedback on deployment issues and can trigger alerts to notify about the configuration discrepancies
• Change Documentation: Test reports serve as evidence for change documentation and operational processes

Benefits of Automated Testing:

• Confidence in Changes: Engineers can be confident that deployed changes work as intended
• Rapid Issue Detection: Problems are identified immediately after deployment, not during business hours when users report issues
• Documentation: Test results provide clear documentation of what was tested and aid in the troubleshooting process

Notification Stage (optional enhancement)

Notifications can also be configured as a final pipeline stage that provides stakeholders with real-time updates about deployment status and results. This stage can send deployment summaries to collaboration platforms like Cisco Webex Teams, Microsoft Teams, automatically distribute test reports and deployment artifacts to relevant stakeholders, and trigger alerts to on-call teams when critical deployments fail. Common integrations include sending adaptive cards with pipeline summaries, distributing comprehensive reports via email, or automatically creating ServiceNow change records with deployment results. This ensures that network operations teams stay informed about infrastructure changes without actively monitoring the pipeline interface.

Pipeline Benefits and Best Practices

Implementing automation pipelines for Network as Code with Nexus Dashboard provides significant advantages over traditional network change management approaches:

Key Benefits

Risk Reduction: Every change is validated multiple times - first against schema and rules, then through deployment testing. This multi-layered approach dramatically reduces the risk of network outages or misconfigurations.

Consistency: All network changes follow the same standardized process, eliminating the variability that comes from manual processes or individual engineer preferences.

Auditability: Every change is tracked in version control, with complete visibility into who made what changes, when they were made, and what testing was performed.

Scalability: As network complexity grows, the pipeline approach scales much better than manual processes, allowing teams to manage larger and more complex network infrastructures.

Traditional vs Pipeline Approach Comparison:

Example: CI/CD Pipeline for Network as Code Nexus Dashboard

---
image: danischm/nac:0.1.5
stages:
  - setup
  - validate
  - deploy
  - test

variables:
  ND_HOST:
    description: "Cisco ND HOST"
  ND_DOMAIN:
    description: "Cisco ND Domain"
  ND_USERNAME:
    description: "Cisco ND Username"
  ND_PASSWORD:
    description: "Cisco ND Password"
  DC_VXLAN_SCHEMA:
    description: "Path to the schema file"
  DC_VXLAN_RULES:
    description: "Path to the rules file"
  NDFC_SW_USERNAME:
    description: "Cisco NDFC Switch Username"
  NDFC_SW_PASSWORD:
    description: "Cisco NDFC Switch Password"

cache:
  paths:
    - .pip_cache/

setup:
  stage: setup
  rules:
    - if: $CI_COMMIT_TAG == null
  script:
    - echo "Setting up environment..."
    - pip install --cache-dir .pip_cache -r requirements.txt --force-reinstall
    - ansible-galaxy collection install -U -f -p ./collections/ansible_collections/ -r requirements.yaml
  artifacts:
    paths:
      - collections/ansible_collections/

validate:
  stage: validate
  dependencies:
    - setup
  needs:
    - setup
  rules:
    - if: $CI_COMMIT_TAG == null
  script:
    - set -o pipefail && ansible-playbook -i inventory.yaml ndfc_validate.yaml |& tee nac_fabric1_validate_output.txt
  artifacts:
    paths:
      - nac_fabric1_validate_output.txt
      - collections/ansible_collections/
  cache: []

deploy:
  stage: deploy
  dependencies:
    - setup
    - validate
  needs:
    - setup
    - validate
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  script:
    - pip install --cache-dir .pip_cache -r requirements.txt
    - set -o pipefail && ansible-playbook -i inventory.yaml vxlan.yaml |& tee nac_fabric1_deploy_output.txt
  artifacts:
    when: always
    paths:
      - nac_fabric1_deploy_output.txt
  cache: []

test-integration:
  stage: test
  dependencies:
    - setup
    - deploy
  needs:
    - setup
    - deploy
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  script:
    - set -o pipefail && nac-test -d ./host_vars/nac-fabric1 -d ./group_vars/ndfc/defaults.yaml -t ./tests/templates -o ./tests/results/nac-fabric1 |& tee test_output.txt
  artifacts:
    when: always
    paths:
      - tests/results/nac-fabric1/*.html
      - tests/results/nac-fabric1/xunit.xml
      - test_output.txt
    reports:
      junit: tests/results/nac-fabric1/xunit.xml

Note: The pipeline example above uses the current nac-test tool name. However, you may still encounter references to iac-test in older documentation or legacy systems, as this was the previous name for the same tool. When implementing new pipelines, always use nac-test for the most up-to-date tooling.

Pipeline Explained

image section

image: danischm/nac:0.1.5

• Purpose: Specifies the Docker image used for all jobs in the pipeline. This image includes all the necessary tools (such as Ansible and nac-test) required to run the pipeline jobs. Using a pre-built image ensures consistency and avoids issues with missing dependencies.

variables section

variables:
  ND_HOST:
    description: "Cisco ND HOST"
  ND_DOMAIN:
    description: "Cisco ND Domain"
  ND_USERNAME:
    description: "Cisco ND Username"
  ND_PASSWORD:
    description: "Cisco ND Password"
  DC_VXLAN_SCHEMA:
    description: "Path to the schema file"
  DC_VXLAN_RULES:
    description: "Path to the rules file"
  NDFC_SW_USERNAME:
    description: "Cisco NDFC Switch Username"
  NDFC_SW_PASSWORD:
    description: "Cisco NDFC Switch Password"

• Purpose: Declares environment variables used throughout the pipeline. These variables provide credentials and file paths to the playbooks and scripts, making the pipeline flexible and secure. You can set these variable keys and values in your GitLab project settings. We will see how to do this in the next section.

  • ND_HOST, ND_DOMAIN, ND_USERNAME, ND_PASSWORD: Connection details for the Cisco Nexus Dashboard.
  • DC_VXLAN_SCHEMA, DC_VXLAN_RULES: Paths to the schema and rules files used for validation.
  • NDFC_SW_USERNAME, NDFC_SW_PASSWORD: Credentials for the network switches.

cache section

cache:
  paths:
    - .pip_cache/

• Purpose: Defines global caching configuration to improve pipeline performance by caching pip packages between job runs. This reduces the time needed to install Python dependencies in subsequent pipeline executions.

stages section

stages:
  - setup
  - validate
  - deploy
  - test
  - notify  # Optional notification stage

• Purpose: Defines the sequence of stages in the pipeline. Each job is assigned to one of these stages, and the pipeline executes them in this order:
  • setup: Installs dependencies and prepares the environment for subsequent stages.
  • validate: Performs semantic and syntactic validation of the configuration for errors before deployment.
  • deploy: Applies the validated configuration to the network.
  • test: Runs the integration tests on the deployed configuration.
  • notify: (Optional) Sends notifications to stakeholders about deployment results.

setup stage

setup:
  stage: setup
  rules:
    - if: $CI_COMMIT_TAG == null
  script:
    - echo "Setting up environment..."
    - pip install --cache-dir .pip_cache -r requirements.txt --force-reinstall
    - ansible-galaxy collection install -U -f -p ./collections/ansible_collections/ -r requirements.yaml
  artifacts:
    paths:
      - collections/ansible_collections/

• Purpose: Prepares the environment by installing Python dependencies and Ansible collections. This stage ensures all necessary tools and libraries are available for subsequent stages. The artifacts preserve the installed collections for use in other jobs.
• Key points:
  • Uses caching to speed up dependency installation.
  • Creates artifacts to share installed collections across pipeline jobs.
  • Only runs when the commit is not a tag.

validate stage

validate:
  stage: validate
  dependencies:
    - setup
  needs:
    - setup
  rules:
    - if: $CI_COMMIT_TAG == null
  script:
    - set -o pipefail && ansible-playbook -i inventory.yaml ndfc_validate.yaml |& tee nac_fabric1_validate_output.txt
  artifacts:
    paths:
      - nac_fabric1_validate_output.txt
      - collections/ansible_collections/
  cache: []

• Purpose: Runs the Ansible playbook ndfc_validate.yaml to check the configuration for errors before any deployment. The output is saved as an artifact for review. This job depends on the setup stage and only runs if the commit is not a tag.
• Key points:
  • Depends on successful setup stage completion.
  • Ensures only valid configurations move forward.
  • Artifacts help with troubleshooting validation failures.

deploy stage

deploy:
  stage: deploy
  dependencies:
    - setup
    - validate
  needs:
    - setup
    - validate
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  script:
    - pip install --cache-dir .pip_cache -r requirements.txt
    - set -o pipefail && ansible-playbook -i inventory.yaml vxlan.yaml |& tee nac_fabric1_deploy_output.txt
  artifacts:
    when: always
    paths:
      - nac_fabric1_deploy_output.txt
  cache: []

• Purpose: Deploys the validated configuration to the network using the vxlan.yaml playbook. This job only runs on the default branch and depends on both setup and validation stages. Note that manual approval has been removed, allowing for fully automated deployment.
• Key points:
  • Depends on successful setup and validation.
  • Reinstalls dependencies to ensure consistency.
  • Automatic deployment on main branch (no manual trigger required).
  • Saves deployment output as artifacts for review.

test-integration stage

test-integration:
  stage: test
  dependencies:
    - setup
    - deploy
  needs:
    - setup
    - deploy
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  script:
    - set -o pipefail && nac-test -d ./host_vars/nac-fabric1 -d ./group_vars/ndfc/defaults.yaml -t ./tests/templates -o ./tests/results/nac-fabric1 |& tee test_output.txt
  artifacts:
    when: always
    paths:
      - tests/results/nac-fabric1/*.html
      - tests/results/nac-fabric1/xunit.xml
      - test_output.txt
    reports:
      junit: tests/results/nac-fabric1/xunit.xml

• Purpose: Runs integration tests on the deployed configuration using the nac-test tool. The results are saved as artifacts (HTML and JUnit XML reports) for review and CI visibility. Uses the current tool name nac-test instead of the legacy iac-test.
• Key points:
  • Depends on successful setup and deployment.
  • Tests against the specific fabric configuration (nac-fabric1).
  • Ensures the deployed configuration works as intended.
  • Artifacts provide detailed test results for troubleshooting and compliance.

Summary

Automation pipelines represent a fundamental shift in how network changes are managed, bringing the reliability and consistency of software development practices to network operations. By implementing the multi-stage pipeline approach (validate, deploy, test) with Network as Code for Nexus Dashboard, organizations can significantly reduce the risk of network outages while improving the speed and consistency of network changes.

The combination of schema validation, automated deployment through Ansible roles, and post-deployment testing creates a robust framework that ensures network configurations are both syntactically correct and operationally sound. This foundation prepares network engineering teams to embrace Infrastructure as Code practices while maintaining the reliability and security that network operations demand.

In the next section, we will implement a working automation pipeline for Network as Code with Nexus Dashboard, putting these concepts into practice with hands-on configuration and deployment.