Merging YAML
When dealing with large configurations, it becomes increasingly important to keep the configuration organized and modular. In order to achieve this, the configuration can be divided into multiple YAML files and folders. There is a specific logic applied to merge the content of multiple YAML files into a single configuration.
In general, the merging process follows these rules:
- The merging process is recursive, meaning that it will traverse through all nested dictionaries and lists.
- The merging process will deep merge content of nested dictionaries. This means that if the same key is present in multiple files, the values will be combined into a single dictionary.
- The merging process will overwrite the values of the same keys if the value is a primitive type (string, number, boolean). Therefore it should be avoided to use the same key in different files in such cases.
- The merging process will append the values of the same keys if the value is a list. This means that if the same key is present in multiple files, the values will be combined into a single list.
- If it is a list of dictionaries, the merging process will combine (deep merge) items, if the values of all matching keys are the same. Only keys with primitive values (string, number, boolean) will be considered. If both items have extra keys with primitive values that the other does not have, they will not be merged.
Merging Dictionaries
Merging Flat Dictionaries
The merging process will deep merge two dictionaries with the same name, combining the keys of both dictionaries.
File 1
dict: key1: value1
File 2
dict: key2: value2
Result
dict: key1: value1 key2: value2
Merging Nested Dictionaries
The merging process will also deep merge two nested dictionaries, combining the keys of both dictionaries.
File 1
dict: name: a nested_dict: key1: value1 key2: value2
File 2
dict: name: a nested_dict: key3: value3
Result
dict: name: a nested_dict: key1: value1 key2: value2 key3: value3
Merging Lists
Merging Lists of Primitive Values
The merging process will combine two lists of primitive values (strings, numbers, booleans) into a single list. The order of the values will be preserved, and duplicate values will be retained.
File 1
list: - value1 - value1 - value2
File 2
list: - value1
Result
list: - value1 - value1 - value2 - value1
Merging Lists of Dictionaries
The merging process will combine two lists of dictionaries with the same name into a single list. The merging process will deep merge the dictionaries, combining the keys of both dictionaries. The merging process will only consider the keys with primitive values (string, number, boolean) for merging and will not merge items if both of them have extra keys with primitive values that the other does not have.
File 1
list: - name: a key1: value1 dict: key1: value1
File 2
list: - name: a dict: key2: value2
Result
list: - name: a key1: value1 dict: key1: value1 key2: value2
Lists of dictionaries (list
) will be merged by the value of all matching keys, while only considering primitive values. In this example the only matching attribute with primitive values is name
and only one item has an extra key (key1
) with a primitive value, so the two list
items will be deep merged into a single item.
File 1
list: - name: a key1: value1 dict: key1: value1
File 2
list: - name: a key2: value2 dict: key2: value2
Result
list: - name: a key1: value1 dict: key1: value1 - name: a key2: value2 dict: key2: value2
In this example the only matching attribute with primitive values is name
and both itema have an extra key (key1
and key2
) with a primitive value, so the two list
items will not be deep merged into a single item.