Scaling

Applies to nac-fmc >= 0.1.3

When you start with a small nac-fmc configuration — a handful of network objects, a few access policies, and a limited number of rules — everything works smoothly. Terraform plans are fast, apply times are short, and the overall structure is easy to reason about.

As the configuration grows, scaling problems may arise:

• Performance Degradation - Terraform must refresh the state of every managed resource on each run. With tens of objects this is negligible, but with thousands of them, this time increases. A growing configuration means Terraform has more work to do during both the planning and apply phases. A change to a single access rule can still trigger a full re-evaluation of all resources.
• State File Size - All managed resources are tracked in the Terraform state file. As the number of objects increases, the state file grows accordingly. Large state files increase the time needed to read, lock, and write state.

For more information about Terraform state refer to the documentation.

Multistate configuration for Network as Code for FMC

With a multistate approach, the configuration can be split into multiple directories, where for each of them, a separate state is maintained. Once the configuration is applied, the module will create a database file that can be used by other instances of the nac-fmc module to get information about objects they may need.

Folder structure

Here is an example split of the configuration files.

• Directorydata/

  • Directoryobjects/

    • network_objects.nac.yaml
  • Directoryaccess_control_policy_Krakow/

    • acp_krakow.nac.yaml
  • Directoryaccess_control_policy_Warsaw/

    • acp_warsaw.nac.yaml
  • Directorydevices/

    • ftd_krakow.nac.yaml
    • ftd_warsaw.nac.yaml
  • Directorydeploy/

    • deploy.nac.yaml
• Directoryobjects/

  • main.tf
  • terraform.tfstate
• Directoryaccess_control_policy_Krakow/

  • main.tf
  • terraform.tfstate
• Directoryaccess_control_policy_Warsaw/

  • main.tf
  • terraform.tfstate
• Directorydevices/

  • main.tf
  • terraform.tfstate
• Directorydeploy/

  • main.tf
  • terraform.tfstate

In this scenario, configuration is split into five separate stages:

1. Object management
2. Access Control Policy for Krakow site
3. Access Control Policy for Warsaw site
4. Device management for both sites
5. Trigger deployment

The following applies:

• Objects created in one stage can be used in subsequent stages.
• Configuration split is arbitrary, however a single object cannot be defined multiple times.
• It is possible to skip evaluation of stages for which YAML definitions have not changed.
• Deploy should be executed as the last one, so that device always has most recent configuration. Deploy is always attempted (terraform plan for this stage is never empty).

Important: Manual Dependency Management Required

When splitting Terraform state across multiple folders, you lose Terraform’s native dependency resolution and automatic operation ordering. You become responsible for managing the correct order of operations in your deployment workflow.

This means:

• You must manually ensure dependencies are deployed in the correct sequence
• Your CI/CD pipeline must handle the orchestration of multiple Terraform runs
• Cross-state dependencies cannot be automatically resolved by Terraform
• You need to share created database files between Terraform runs
• You cannot parallelize Terraform runs, as FMC does not allow parallel changes

Plan your deployment workflow carefully and implement proper sequencing in your automation pipelines.

Configuration example

Objects

data/objects/network_objects.nac.yaml

fmc:
  domains:
    - name: Global
      objects:
        hosts:
          - name: Krakow_Host1
            ip: 10.10.10.1
          - name: Krakow_Host2
            ip: 10.10.10.2
          - name: Warsaw_Host1
            ip: 10.10.20.3
          - name: Warsaw_Host2
            ip: 10.10.20.4

objects/main.tf

module "fmc" {
  source  = "netascode/nac-fmc/fmc"
  version = "0.1.3"

  yaml_directories   = ["../data/objects"]
  # Write object data
  write_objects_file = "../data_objects/objects.yaml"
}

After a successful Terraform apply operation, a new file is created. This file contains information for other nac-fmc modules to reference objects.

data_objects/objects.yaml

"data":
  "fmc":
    "devices": {}
    "integrations": {}
    "objects":
      "hosts":
        "Global:Krakow_Host1":
          "id": "005056B0-7B7F-0ed3-0000-004295120977"
          "type": "Host"
        "Global:Krakow_Host2":
          "id": "005056B0-7B7F-0ed3-0000-004295120995"
          "type": "Host"
        "Global:Warsaw_Host1":
          "id": "005056B0-7B7F-0ed3-0000-004295121013"
          "type": "Host"
        "Global:Warsaw_Host2":
          "id": "005056B0-7B7F-0ed3-0000-004295121031"
          "type": "Host"
    "policies": {}
    "vpns": {}

Access Control Policy Krakow

data/access_control_policy_Krakow/acp_krakow.nac.yaml

fmc:
  domains:
    - name: Global
      policies:
        access_control_policies:
          - name: Krakow_ACP
            default_action: BLOCK
            access_rules:
              - name: Allow_Krakow_to_Warsaw
                action: ALLOW
                source_network_objects:
                  - Krakow_Host1
                  - Krakow_Host2
                destination_network_objects:
                  - Warsaw_Host1
                  - Warsaw_Host2

access_control_policy_Krakow/main.tf

module "fmc" {
  source  = "netascode/nac-fmc/fmc"
  version = "0.1.3"

  yaml_directories = [
    "../data/access_control_policy_Krakow",
    # Include object information from other nac-fmc instances
    "../data_objects"
  ]

  # Write object data
  write_objects_file = "../data_objects/access_control_policy_Krakow.yaml"
}

After a successful Terraform apply operation, a new file is created.

data_objects/access_control_policy_Krakow.yaml

"data":
  "fmc":
    "devices": {}
    "integrations": {}
    "objects": {}
    "policies":
      "access_control_policies":
        "Global:Krakow_ACP":
          "domain": "Global"
          "id": "005056B0-7B7F-0ed3-0000-004295121080"
          "name": "Krakow_ACP"
          "type": "AccessPolicy"
    "vpns": {}

The similar logic follows through the remaining configuration files, where to each subsequent run of nac-fmc you can inject one or more outputs from previous runs.