NaC Testing

In any discussion of Infrastructure as Code, you will find some form of testing. In any modern Development Operations (DevOps) run environment, testing is integrated at different development stages. First you have the unit tests, embeded into code that is being developed and then integration and system testing.

For Network as Code, we provide to aspects of testing. First to ensure that the structure and syntax of the data model is correct, and second post change validation to ensure that the configuration and operational state of the network is correct after a change has been applied.

Overview

For Network as Code there are two distinct tools that we utilize to achieve our testing objectives:

yaml-lint	nac-validate	nac-test
This tool is used to validate the syntax and structure of YAML files. It checks for errors in YAML files and ensures that they are well-formed. In combination with other tools like `pre-commit` you could ensure that changes to the YAML are structured correctly (in YAML format) before even getting posted into the GIT repository.	This tool is used to validate the data model syntax and structure. It checks for errors in the data model and ensures that it is ready for deployment. This element performs specific validation that could pass YAML lint but could be an invalid configuration and not allowed by schema.	This tool is used to validate the configuration and operational state of the network after a change has been applied. It checks that the configuration is correct and that the operational state of the network matches the expected state.

Note : nac-validate and nac-test were previously known as iac-validate and iac-test

The combination of these two tools provides the network operator with the combination of tools that avoid common errors from hitting production changes. These tools can be both used inside an automation pipeline or as standalone tools. The following diagram described what it would look like from a software lifecycle perspective.

Operators can be creating changes in the data model locally and performing checks against the work they are doing. Once completed, then when they push these changes into a central repository, the automation pipeline will perform the same validation checks and tests before accepting the changes.

Overview of testing

Let’s take a look at each of these tools in more detail.

nac-validate

The nac-validate tooling focuses on pre-change validation. The best way to “correlate” this to a software development process is to think of it as a linter. It checks the syntax and structure of the data model. To accomplish this, it uses a YAML schema to corroborate that data model. When working with code linting is a common practice to ensure that the code is syntactically correct and follows best practices. For Network as Code this facility allows the operators to validate their work while they are making changes to the data model.

NaC Validate

In addition to the semantic validation, nac-validate also can run rules to perform compliance checks. This is something that is used extensively by the Network as Code for Nexus Dashboard to ensure that the data model is compliant with the requirements of the Nexus Dashboard. The validate role utilizes these tools to exten

Usage: nac-validate [OPTIONS] PATHS...

 A CLI tool to perform syntactic and semantic validation of YAML files.


╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ *    paths      PATHS...  List of paths pointing to YAML files or directories. [default: None] [required]                                                           │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --verbosity   -v      [DEBUG|INFO|WARNING|ERROR|CRITICAL]  Verbosity level. [env var: NAC_VALIDATE_VERBOSITY] [default: WARNING]                                    │
│ --schema      -s      FILE                                 Path to schema file. [env var: NAC_VALIDATE_SCHEMA] [default: .schema.yaml]                              │
│ --rules       -r      DIRECTORY                            Path to directory with semantic validation rules. [env var: NAC_VALIDATE_RULES] [default: .rules]        │
│ --output      -o      FILE                                 Write merged content from YAML files to a new YAML file. [env var: NAC_VALIDATE_OUTPUT] [default: None]  │
│ --non-strict                                               Accept unexpected elements in YAML files. [env var: NAC_VALIDATE_NON_STRICT]                             │
│ --version                                                  Display version number.                                                                                  │
│ --help                                                     Show this message and exit.                                                                              │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

The Schema file

To be able to do the semantic validation in nac-validate, there is a schema file that defines the structure of the data model. The complete schema file is available as part of Services as Code.

The schema file is built utilizing Yamale, a YAML schema validator. Yamale is a Python library that allows Network as Code to perform this validation. Our tool, nac-validate, uses this library to validate the data in the model after some pre-processing steps.

To help you understand the schema file, the example repository contains a minimal schema that is provided for demonstration purposes. This schema file is located in the schema directory of the example repository.

Note : Due to the limited scope of the example schema file, you will not enable the execution of the schema from the validate role. In normal operation, once the full schema file is available, you specify the environment variable DC_VXLAN_SCHEMA with the path to the full schema.

Step1 : Run nac-validate local

In this step you will run the nac-validate tool locally to validate the data model. The schema file is already included in the example repository.

cd ~/network-as-code/nac-nd
nac-validate -s schema/schema.yaml host_vars/nac-fabric1

When you run this command, it should result in an errored condition. This is because the example schema file is not complete (example version). To make it work with the limited schema file in this example repository, you need to add the --non-strict option to the command:

nac-validate -s schema/schema.yaml --non-strict host_vars/nac-fabric1 -v DEBUG

Which should result in output similar to:

INFO - Loading schema
INFO - No rules found
INFO - Validate file: global.nac.yaml
INFO - Validate file: underlay.nac.yaml
INFO - Validate file: topology_switches.nac.yaml
INFO - Validate file: networks.nac.yaml
INFO - Validate file: vrfs.nac.yaml

nac-test

In the same vein as nac-validate, the nac-test tool is used to perform post-change validation. This tool is used to validate that the configuration on Nexus Dashboard matches the configuration that is declared in the data model. As part of this example repository, you will find a set of tests against the fabric type.

NaC Test

The primary focus of nac-test is to do a 1:1 verification that the intended configuration matches what is deployed. For this reason, to accomplish this task it is normally executed after the deployment of the configuration to Nexus Dashboard

 Usage: nac-test [OPTIONS]

 A CLI tool to render and execute Robot Framework tests using Jinja templating.


╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ *  --data         -d      PATH                                 Path to data YAML files. [env var: NAC_TEST_DATA]             │
│                                                                [default: None]          [required]                           │
│ *  --templates    -t      DIRECTORY                            Path to test templates. [env var: NAC_TEST_TEMPLATES]         │
│                                                                [default: None]         [required]                            │
│ *  --output       -o      DIRECTORY                            Path to output directory. [env var: NAC_TEST_OUTPUT]          │
│                                                                [default: None]           [required]                          │
│    --filters      -f      DIRECTORY                            Path to Jinja filters. [env var: NAC_TEST_FILTERS]            │
│                                                                [default: None]                                               │
│    --tests                DIRECTORY                            Path to Jinja tests. [env var: NAC_TEST_TESTS]                │
│                                                                [default: None]                                               │
│    --include      -i      TEXT                                 Selects the test cases by tag (include).                      │
│                                                                [env var: NAC_TEST_INCLUDE]                                   │
│    --exclude      -e      TEXT                                 Selects the test cases by tag (exclude).                      │
│                                                                [env var: NAC_TEST_EXCLUDE]                                   │
│    --render-only                                               Only render tests without executing them.                     │
│                                                                [env var: NAC_TEST_RENDER_ONLY]                               │
│    --dry-run                                                   Dry run flag. See robot dry run mode.                         │
│                                                                [env var: NAC_TEST_DRY_RUN]                                   │
│    --verbosity    -v      [DEBUG|INFO|WARNING|ERROR|CRITICAL]  Verbosity level.   [env var: NAC_VALIDATE_VERBOSITY]          │
│                                                                [default: WARNING]                                            │
│    --version                                                   Display version number.                                       │
│    --help                                                      Show this message and exit.                                   │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Step2: Create results directory

To enable the execution of the nac-test tool, you need to create a results directory where the test results will be stored. This directory is specified in the --output option of the command.

cd ~/network-as-code/nac-nd
mkdir -p tests/results/nd

Step3: Run nac-test local

cd ~/network-as-code/nac-nd
nac-test -d ./host_vars/nac-fabric1 -d ./group_vars/ndfc/defaults.yaml -t ./tests/templates  -o ./tests/results/nd

Which will generate results that should be similarr to the following output:

Robot Framework remote server at 127.0.0.1:8270 started.
Storing .pabotsuitenames file
2025-07-11 10:57:29.020572 [PID:347524] [0] [ID:0] EXECUTING Nd.Config.Fabric.Fabric
2025-07-11 10:57:30.225227 [PID:347524] [0] [ID:0] PASSED Nd.Config.Fabric.Fabric in 1.2 seconds
7 tests, 7 passed, 0 failed, 0 skipped.
===================================================
Output:  /home/dcloud/network-as-code/nac-nd/tests/results/nd/output.xml
XUnit:   /home/dcloud/network-as-code/nac-nd/tests/results/nd/xunit.xml
Log:     /home/dcloud/network-as-code/nac-nd/tests/results/nd/log.html
Report:  /home/dcloud/network-as-code/nac-nd/tests/results/nd/report.html
Stopping PabotLib process
Robot Framework remote server at 127.0.0.1:8270 stopped.
PabotLib process stopped
Total testing: 1.20 seconds
Elapsed time:  1.60 seconds

We have enabled an HTTP server to view the results easily. Robot generates reports in HTML, XML.

http://198.18.133.101/network-as-code/nac-nd/tests/results/nd/report.html