1. Messages
  2. Thoughts on layering of multiple config files

Re: Thoughts on layering of multiple config files

Nathan Rusch <natha...@...>
 

Hi Ben,

I did see your responses to Hugh's thread after he pointed me to it. However, once again, the discussion seems to be centered around the idea of nesting or referencing config files, whereas I'm mainly interested in being able to layer config files together for very specific overrides. I'll try to sketch my thoughts out a little more clearly:
  • Base configuration lives at `/path/to/base.config` (this could be facility-wide or show-specific; that part isn't important).
  • Set up OCIO environment var in the same way you might set up $PATH, except with OCIO configs
    • export OCIO=/path/to/sparse/override.config:/path/to/base.config
  • OCIO creates a Config object from the environment by doing the following:
    1. Load the configuration file found at the *back* of the $OCIO variable (in the above case, the "base config")
    2. Working from back to front, load each config definition onto the Config instance that was created in step 1.
    3. Only sanity-check the resulting Config instance after all paths have been loaded.
In this example, the file `/path/to/sparse/override.config` would be able to, say, replace a single ColorSpace definition with a more complex set of transforms or add a Look by *only* containing the desired additional or override definitions (and, say, the ocio_profile_version key). If the sparse file defined a ColorSpace with a name that matched the name of a ColorSpace in the "base config," the sparse file's definition would replace it purely based on the order in which the config files appear on the $OCIO variable.

This would allow for overrides without requiring any preprocessing of the YAML data (to, say, resolve includes), any syntactical changes to the configuration files, or any alterations to any *existing* configurations. We use this same kind of thing to manage stored preferences/recent settings for pipeline operators at different contextual levels (show, shot, element, action, user). It allows each type of preference to be as sparse as possible to avoid writing out the same data many times, and for a given application or scenario, the order in which these contextual preferences "squash" each other can change (in the case that duplicate preference keys are possible).

The main scenario I can see for deploying this type of config "patching" is when the crazy requests start rolling in from production near the end of a show, when it is often critical to be nimble. You are correct that even this type of extension would give anyone more wiggle room to break things, and that it wouldn't provide a way to remove things from the final Config. However, I don't really see this layering idea being the standard way of doing things, but simply an added dimension of flexibility.

There is currently nothing stopping anyone from manually breaking an existing configuration with a text editor, or writing a malformed definition from scratch, and it isn't OCIO's responsibility to prevent them from doing this. In that same spirit, it wouldn't be OCIO's responsibility to prevent someone from trying to combine two configuration files that don't result in a valid Config object.

Hope this helps clarify my thoughts a little more, and how they differ from the idea of referencing or including configuration files between each other.

-Nathan



On Thursday, June 27, 2013 8:09:13 AM UTC-7, dbr/Ben wrote:

Jeremy's reply to https://groups.google.com/forum/#!topic/ocio-dev/okBHXi1ibnM sums up most of my concerns about this feature nicely..

Merging two arbitrary YAML configs seems like a direction in which madness lies.. There's lots of cases where the behaviour would be non-obvious, e.g:
* parent defines and uses a 'linear' space, child also defines a 'linear' space with, say, different allocation - what happens?
* Does a file-reference in the parent search the child search_path?
* If there's a base config, can I not inherit a specific colorspace?
* ..and a bunch more I haven't fully thought through (like "would you need to control the order in which views are merged?", "in the child config, could you remove a colorspace from the parent?")


Limiting the scope a bit might make things clearer, e.g something like

colorspaces:
  - !<IncludeColorspaces> {src: "/path/to/shared/base.ocio", include: [srgb, "*log"], exclude: ['badlog']}
  - !<Colorspace>
     name: ...

Where this would just include the colorspace definitions, but ignore anything like the search_path from the included file. If the included config requires LUT's in a certain location, this path would be added to "search_path" as usual.


That said, I don't really understand the benefit of this.. While the current "one config.ocio per project" setup involves some duplication of colorspace definitions etc, I don't see this as a downside.. Either a colorspace never changes (in which case, copying-and-pasting is little hassle), or the colorspace needs changed (in which case, it's useful being able to update new projects without fear of breaking current/finished projects)

It'd be good to have a few clear explanations of workflows where the nested configs would be beneficial (beneficial over alternative approaches like copy-pasting-and-modifying, or Python-generated configs)
- Ben
Join {ocio-dev@lists.aswf.io to automatically receive all group messages.