cliconfig.dict_routines
Routines to manipulate nested and flat dictionaries (and mix of both).
Used by process_routines and config_routines.
- merge_flat(dict1, dict2, *, allow_new_keys=True)
Flatten then merge dict2 into dict1. The result is flat.
Work even if dict1 and dict2 have a mix of nested and flat dictionaries. For instance like this:
dict1 = {'a.b': 1, 'a': {'c': 2}, 'a.d': {'e.f': 3}}
- Parameters:
- Raises:
ValueError – If allow_new_keys is False and dict2 has new keys that are not in dict1.
ValueError – If there are conflicting keys when flatten one of the dicts. See last example. You may consider calling
clean_pre_flat()on the input dicts in that case.
- Returns:
flat_dict – The flat dict (all keys are at the root and separated by dots).
- Return type:
Dict[str, Any]
Examples
>>> merge_dict({'a.b': 1, 'a': {'c': 2}}, {'c': 3}, allow_new_keys=True) {'a.b': 1, 'a.c': 2, 'c': 3} >>> merge_dict({'a.b': 1, 'a': {'c': 2}}, {'c': 3}, allow_new_keys=False) ValueError: New parameter found 'c' that is not in the original dict. >>> merge_dict({'a.b': 1, 'a': {'b': 1}}, {'c': 3}, allow_new_keys=True) ValueError: duplicated key 'a.b'. The above exception was the direct cause of the following exception: ValueError: You may consider calling 'clean_pre_flat' on dict 1 before merging.
- merge_flat_paths(dict_or_path1, dict_or_path2, *, allow_new_keys=True)
Flatten then merge two dict eventually loaded from yaml file paths.
Similar to
merge_flat()but allow passing the paths of dicts as inputs. It merges the second dict into the first one.- Parameters:
- Raises:
ValueError – If allow_new_keys is False and dict2 has new keys that are not in dict1.
ValueError – If there are conflicting keys when flatten one of the dicts. See last example. You may consider calling
clean_pre_flat()on the input dicts in that case.
- Returns:
flat_dict – The flat dict (all keys are at the root and separated by dots).
- Return type:
Dict[str, Any]
- flatten(in_dict)
Flatten dict then return it (flat keys are built with dots).
Work even if in_dict is a mix of nested and flat dictionaries. For instance like this:
>>> flatten({'a.b': {'c': 1}, 'a': {'b.d': 2}, 'a.e': {'f.g': 3}}) {'a.b.c': 1, 'a.b.d': 2, 'a.e.f.g': 3}
- Parameters:
in_dict (Dict[str, Any]) – The dict to flatten. It can be nested, already flat or a mix of both.
- Raises:
ValueError – If dict has some conflicting keys (like
{'a.b': <x>, 'a': {'b': <y>}}).- Returns:
flat_dict – The flattened dict.
- Return type:
Dict[str, Any]
Note
Nested empty dict are ignored even if they are conflicting (see last example).
Examples
>>> flatten({'a.b': 1, 'a': {'c': 2}, 'd': 3}) {'a.b': 1, 'a.c': 2, 'd': 3} >>> flatten({'a.b': {'c': 1}, 'a': {'b.d': 2}, 'a.e': {'f.g': 3}}) {'a.b.c': 1, 'a.b.d': 2, 'a.e.f.g': 3} >>> flatten({'a.b': 1, 'a': {'b': 1}}) ValueError: duplicated key 'a.b' >>> flatten({'a.b': 1, 'a': {'c': {}}, 'a.c': 3}) {'a.b': 1, 'a.c': 3}
- unflatten(flat_dict)
Unflatten a flat dict then return it.
- Parameters:
flat_dict (Dict[str, Any]) – The dict to unflatten. Must be a fully flat dict (depth of 1 with keys separated by dots).
- Raises:
ValueError – If flat_dict is not flat and then found conflicts.
- Returns:
unflat_dict – The output nested dict.
- Return type:
Dict[str, Any]
Examples
>>> unflatten({'a.b': 1, 'a.c': 2, 'c': 3}) {'a': {'b': 1, 'c': 2}, 'c': 3} >>> unflatten({'a.b': 1, 'a': {'c': 2}}) ValueError: duplicated key 'a' The dict must be flatten before calling unflatten function.
- clean_pre_flat(in_dict, priority)
Remove keys in input dict that may cause conflicts when flattening.
- Parameters:
in_dict (Dict[str, Any]) – The dict to clean. It must be the union of a fully flat dict (no nested dict i values) and a fully nested dict (without dots in keys). See warning section below.
priority (str) – One of ‘flat’ or ‘unflat’, ‘error’. If ‘flat’, keys with dots at the root like
{'a.b': ...}(flat keys) have priority over nested keys like{'a': {'b': ...}}when there are conflicts. If ‘unflat’, nested keys have priority over flat keys when there are conflicts. If ‘error’, raise an error when there are conflicts.
- Raises:
ValueError – If priority is not one of ‘flat’, ‘unflat’ or ‘error’.
- Returns:
The cleansed dict.
- Return type:
Dict[str, Any]
Warning
No flat key can contain a dict. Then, dicts like
{'a.b': {'c': 1}}are not supported.All the keys that contain dots (the flat keys) must be at the root. Then, dicts like
{a: {'b.c': 1}}are not supported.To summarize, the dict must contain only fully flat dicts and/or fully nested dicts.
Examples
>>> clean_pre_flat({'a.b': 1, 'a': {'b': 2}, 'c': 3}, priority='flat') {'a.b': 1, 'c': 3} >>> clean_pre_flat({'a.b': 1, 'a': {'b': 2}, 'c': 3}, priority='unflat') {'a': {'b': 2}, 'c': 3} >>> clean_pre_flat({'a.b': 1, 'a': {'b': 2}, 'c': 3}, priority='error') ValueError: duplicated key 'a.b'
- save_dict(in_dict, path)
Save a dict to a yaml file (with yaml.dump).
- load_dict(path)
Load dict from a yaml file path.
Support multiple files in the same document and yaml tags.
- Parameters:
path (str) – The path to the file to load the dict.
- Returns:
out_dict – The nested (unflatten) loaded dict.
- Return type:
Dict[str, Any]
Note
If multiple yaml files are in the same document, they are merged from the first to the last.
To use multiple yaml tags, separate them with “@”. E.g.
!tag1@tag2.You can combine any number of yaml and cliconfig tags together.
- show_dict(in_dict, start_indent=0)
Show the input dict in a pretty way.
The config dict is automatically unflattened before printing.