Merging YAML

When dealing with large configurations, it becomes increasingly important to keep the configuration organized and modular. In order to achieve this, the configuration can be divided into multiple YAML files and folders. There is a specific logic applied to merge the content of multiple YAML files into a single configuration.

In general, the merging process follows these rules:

• The merging process is recursive, meaning that it will traverse through all nested dictionaries and lists.
• The merging process will deep merge content of nested dictionaries. This means that if the same key is present in multiple files, the values will be combined into a single dictionary.
• The merging process will overwrite the values of the same keys if the value is a primitive type (string, number, boolean). Therefore it should be avoided to use the same key in different files in such cases.
• The merging process will append the values of the same keys if the value is a list. This means that if the same key is present in multiple files, the values will be combined into a single list.
• If it is a list of dictionaries, the merging process will combine (deep merge) items, if the values of all matching keys are the same. Only keys with primitive values (string, number, boolean) will be considered. If both items have extra keys with primitive values that the other does not have, they will not be merged.

The merging process will deep merge two dictionaries with the same name, combining the keys of both dictionaries.

dict:
  key1: value1

dict:
  key2: value2

dict:
  key1: value1
  key2: value2

The merging process will also deep merge two nested dictionaries, combining the keys of both dictionaries.

dict:
  name: a
  nested_dict:
    key1: value1
    key2: value2

dict:
  name: a
  nested_dict:
    key3: value3

dict:
  name: a
  nested_dict:
    key1: value1
    key2: value2
    key3: value3

The merging process will combine two lists of primitive values (strings, numbers, booleans) into a single list. The order of the values will be preserved, and duplicate values will be retained.

list:
  - value1
  - value1
  - value2

list:
  - value1

list:
  - value1
  - value1
  - value2
  - value1

The merging process will combine two lists of dictionaries with the same name into a single list. The merging process will deep merge the dictionaries, combining the keys of both dictionaries. The merging process will only consider the keys with primitive values (string, number, boolean) for merging and will not merge items if both of them have extra keys with primitive values that the other does not have.

list:
  - name: a
    key1: value1
    dict:
      key1: value1

list:
  - name: a
    dict:
      key2: value2

list:
    - name: a
      key1: value1
      dict:
        key1: value1
        key2: value2

Lists of dictionaries (list) will be merged by the value of all matching keys, while only considering primitive values. In this example the only matching attribute with primitive values is name and only one item has an extra key (key1) with a primitive value, so the two list items will be deep merged into a single item.

list:
  - name: a
    key1: value1
    dict:
      key1: value1

list:
  - name: a
    key2: value2
    dict:
      key2: value2

list:
    - name: a
      key1: value1
      dict:
        key1: value1
    - name: a
      key2: value2
      dict:
        key2: value2

In this example the only matching attribute with primitive values is name and both itema have an extra key (key1 and key2) with a primitive value, so the two list items will not be deep merged into a single item.