|
Re: Review: Updated cmake_minimum_required to 2.8
Malcolm Humphreys <malcolmh...@...>
LGTM
toggle quoted message
Show quoted text
On 27/10/2010, at 9:14 AM, Jeremy Selan wrote:
We already required cmake 2.8 (for ExternalProject usage), now this is explicit. |
|
|
|
Re: Review: nuke plugin crash fix
Malcolm Humphreys <malcolmh...@...>
LGTM
toggle quoted message
Show quoted text
On 27/10/2010, at 9:14 AM, Jeremy Selan wrote:
Nuke plugins wont crash if the specified $OCIO is invalid. |
|
|
|
Re: Review: Inital pass at searchpath impl
Malcolm Humphreys <malcolmh...@...>
We've just started testing the current version of OCIO at Imageworks,Can you elaborate a little on what was confusing. * An OCIO profile is not just a single configuration file in isolation+1 to a self contained file * Consider, what would the OCIO world look like if we 'locked down' aWhat about per shot balance grades and look / creative grades, I would cry if I needed to create a zip ocio profile per shot to do this. Then re-syncing these profiles would be a nightmare. * With this model, the "single file" profile + luts representationLet me describe how I would like to see OCIO setup on a show. * central show OCIO profile describing - working/reference space - display devices - colorspace definitions - luts (relative to this profile) * show/seq/shot (cdl) balance grades as separate xml on disk * show/seq/shot (cdl/lut/spline) look / creative grades as separate files on disk I could see this setup being forked and lock at different stages of production (prepro/onset, trailer, theatrical). Onset when tech and human resources are limited, being able to setup this up with minimal hassle would be ideal. For simplicity sake you had some cdl balance grades layout like this. /job/config/ocio/balance_approved.xml /job/myseq/config/ocio/balance_approved.xml /job/myseq/someshot/config/ocio/balance_approved.xml /job/myseq/othershot/config/ocio/balance_approved.xml Your search path would be in the central config "/job/$SEQ/$SHOT/config/ocio:/job/$SEQ/config/ocio:/job/config/ocio" Changing your $SEQ and $SHOT would allow switching between grades. This is not to say that you might merge this into a more involved system when in house, maybe without using env vars. eg. a Transform plugin which picks up the approved grade from a database. I think it's important to support both ways of working. But you do want to make it possible to lock down which files are loaded relative to the profile and which use the searchpath. I agree this could causes issues. I would look at having some kind of prefix to filenames to mark them as 'only look in the resource path' this could be by prefixing the paths with an '@' eg. !<FileTransform> {src: @/lg10.spi1d} this would resolve to only paths in the resource path anything without an '@' would look through the search path (in order). * resourcepath - is relative to the profile, either embedded or on disk * searchpath - is a ':' ordered list of directories to search for file references * file references - are absolute paths / '@' in the resourcepath / or somewhere in the searchpath Want to know what's inside your .ocio.zip config? Easy, just unzip it-1 for fixed directory layout, I think it's important that this is configurable. I can't say how different places will want to setup their pipe. Configurable decision points should be left configurable ( with sensible defaults and examples ;) ). I think it's important for a process to validate the packaging of a profile. I was hoping for something like: Package up a profile $> ociopacakge source.ocio packaged.ocio This would package up all the absolute and resourcepaths into a packaged ocio profile. Package up a shot profile $> ociopackage --resolve-searchpath source.ocio packaged.ocio This would resolve the searchpath and package up any found files and these would be given '@' references in the packaged.ocio Unpack a profile $> ociopackage -x packaged.ocio ~/tmp/ I'm a lot more in favour of a single file with embedded ascii or binary blobs at the bottom. It's quite easy for someone unzip a dir remove a lut and re-zip it and circumvent any validation process. I also I think people will be checking these into version control systems (aka easy setup publish system) being able to open profiles in a text editor / web browser and reading the header would be really nice rather than having to unzip to check stuff out. I'm happy to code up a prototype of this in a branch if people are keen to see that idea. On a slighly different topic, I think it will be preferable whenCan you elaborate on this? I don't completely get what you mean, does the '@' in file reference help distinguish between local and global? .malcolm
|
|
|
|
Re: Reference !<Colorspace>
Malcolm Humphreys <malcolmh...@...>
Mostly I see this is for notation, right now profiles infer that all to_reference and from_reference entires go to/from the same reference space. But I don't need a reference space in 'roles:' or even in 'colorspaces:' for ocio to work. This really isn't a problem if all your reference spaces are the same for all profiles across jobs. It gets just a little murky when you open the possibilities of different connection spaces in different profiles.
Well thats another option, enforce scene_linear as the reference space for all profiles (I would recommend using that as the working space anyhow). I'm just not sure if that will make it tricker for uptake at smaller shops which would be mostly working in some type of display linear pipe.
I would remove it till it is needed.
Thats not so good if people are sending around profiles with out docs.. 'hmm this profile seems funky but I don't know why, oh and the docs are out of date..' moments.
I agree I think having it as a normal colorspace is good, just if there was a specific tag at the top of the profile which told which colorspace was the references space. We could also barf when that colorspace isn't defined in the 'colorspaces:' section. The use case I keep thinking about is dealing with two profiles with different reference spaces, but maybe this is just over complicating things for where we are right now.
This was actually: reference:to:Transform()
Depending on your thoughts about the reference space we might want some other naming. I was wondering could we rename the <GroupTransform> children param to something like 'items', 'transforms'. (maybe I'm too pedantic parent -> children relationships look like a trees while lists, items etc have an order) .malcolm |
|
|
|
Re: Reference !<Colorspace>
Jeremy Selan <jeremy...@...>
Rod,
Answering your short email with a long essay... (just had a coffee) My concern is when/where we want to have different primaries onI understand your concern. If we wanted to solve this today -- leveraging OCIO -- the simplest approach would be to use naming conventions to denote differences between shows. Colorspaces would only have identical names if they were cross-show compatible, and all other 'show specific' spaces would be uniquely named at a facility level. Say, for example, you had Show A which used rec709 primaries, and Show B which used P3 primaries. You could define 2 profiles: Show A OCIO Profile: (r709 primaries) -colorspace: lin_709 transform: none -colorspace: lin_p3 transform to_reference : p3_to_r709.mtx Show B OCIO Profile: (p3 primaries) -colorspace: lin_p3 transform: none -colorspace: lin_709 transform to_reference: r709_to_p3.mtx Then, when you encountered an image on disk, as long as you could uniquely identify it as belonging to either lin_709, or lin_p3 it could be loaded, and work sensibly on either show. This has minimal ambiguity, and doesnt rely on any 'dynamic' conversions. If you load an image on Show A, it uses Show A's profile. Load it on Show B, it uses Show B's profile. If, during production on "B", you encounter an unforseen color issue you can tweak B profile, and show A will stay locked off in its own sandbox. Ideal interoperability ruined? Yes. But do both shows deliver? Yes. :) ... Load it on show C? (which doesnt know about lin_p3 or lin_709)? It wont know about that color space, and will throw an error rather than silently guessing at an answer. Also a good thing. Of course, this approach relies on your facility creating inoperable profiles (and not introducing ambiguous naming), but it's already in your power to do so. The fleshed out Imageworks OCIO profiles we'll be releasing later this week (hopefully) are NOT the only way of working. We would of course presume that any large facility would use prefer to used a completely customized workflow. But, our hope is that our profiles may be useful at small facilities where they just want a reasonable answer right out of the box. We would be delighted if other studios or vfx shops distributed their OCIO profiles, and would love to roll them in as default examples as well! What better way to learn about alternate approaches to color management? Rod- if you're allowed to say publicly - how many sets of color primaries are we discussing here? Is it a handful of predefined options, or do you foresee a situation where you'll have dozens of sets of primaries (and/or white points) being used on a single show? How are you managing this problem currently? As we've discussed before, if you truly won't know about what sets of primaries will be allowed until runtime, or if we're talking about more than a handful of primaries, then the only sensible option is to use a dynamic colorspace plugin API. (which will let you create transforms on the fly.) I am still very enthusiastic about adding this, though being realistic on time, I don't believe we'll be able to get to it before the new year. At SPI we do have more than one set of primaries in use, but very rarely within the same show. When we do need to interchange image assets between shows, we often create a custom colorspace suited for that purpose. This flexibility / explicitness is often preferable to doing the "right" thing automatically, as "right" often depends on context. A simple example: Say you have a texture asset (diffuse color?) from our P3 primary show, and you want to re-use it on a 709 show. What conversion is most appropriate? If you want to preserve the absolute look (such as for a low saturation blue skydome), probably a matrix transform is justified. But, what if you want to use it as a source texture, where it will undergo further image manipulation? The P3->709 conversion has a substantial potential to introduce clipping, if it's going to undergo further color correction anyways, loading it in 'raw' may be preferable. I'll ignore related topic of white point adaptation, but I could forsee some circumstances where you would want to preserve the look of the original white point, and other circumstances where one may prefer to have equal code values in p3 map to equal code values in r709. Â So the reference space is probably a particular RGBI am curious... it sounds like you're grasping for some sort of system where the reference colorspaces are tagged with enough information so that OCIO is able to build up transforms between unrelated profiles automatically. Am I correct here? What sort of information would be required to be tagged on the reference space to allow this? Unfortunately, in VFX work it's quite common to use a set of primaries that are only defined implicitly (in that they're a 1D transformation away from a log scan). If I were to link up one of these VFX workflows with a colorimetricly 'pure' CG profile, what should happen? If a 1d-linearized log scan *is* the only definition of the reference colorspace, what could the OCIO profile to be tagged with to allow you to undo it? A film stock? Note: The Academy's work with IIF is an attempt to answer this question, and the complexities they have encountered with the IDTs + RRTs very much demonstrate how hard this issue is to solve with a "one size fits all" approach. Hell, it's required a new film scanning spec! Wrapping this whole complex issue into "naming" is a rather tidy solution. Color management is in many ways a huge can of worms, and the question of whether OCIO will -- or should -- succeed is very much still up in the air. It's my strong belief that to give OCIO 1.0 its best chance at becoming a success, it's preferable to keep it as close as possible in philosophy to the workflow we've validated at SPI these last years. We've had successful visual effect shows and successful animated features that have relied on completely different approaches to color. Different configurations. Same library. Same plugins. This approach is what we want to share with OCIO. From my perspective, there's minimal amount of SPI-ness left in OCIO at this point. And the major conceptual assumptions we've made are probably best left in there, lest we screw up the good in pursuit of perfect. I have no misconceptions that OCIO will solve 100% of the color problems out there, but if it's flexible enough to let the majority of facilities (both big and small) plug into it, then it will at least be a step in the right direction. You are allowed to strongly disagree. I like learning new things. :) -- Jeremy |
|
|
|
Re: Reference !<Colorspace>
Rod Bogart <bog...@...>
My concern is when/where we want to have different primaries on
toggle quoted message
Show quoted text
different shows. So the reference space is probably a particular RGB space (rather than XYZ), and it is not obvious what that is by examining a to_linear function. In the SPI examples, there was never more than one set of primaries in use, but that seems unrealistic for the future. Right? RGB On Mon, Oct 25, 2010 at 10:17 PM, Jeremy Selan <jeremy...@...> wrote:
I'm not sure what promoting the reference colorspace to something |
|
|
|
Review: OCIO namespace update
Jeremy Selan <jeremy...@...>
OCIO namespace #define is now configured using the CMAKE
configure_file mojo. This makes namespace errors far less likely, as it's baked into the header that's a part of make_install. commits: d00581c9373806b85416115a58488d57a12be04a |
|
|
|
Review: Updated cmake_minimum_required to 2.8
Jeremy Selan <jeremy...@...>
We already required cmake 2.8 (for ExternalProject usage), now this is explicit.
commits: 949855cca45431a6e813ac3b6f3688de0799f379 |
|
|
|
Review: nuke plugin crash fix
Jeremy Selan <jeremy...@...>
Nuke plugins wont crash if the specified $OCIO is invalid.
This issue is related to calling the nuke IOP::error method during construction, but in the meantime we can at least prevent a crash by not calling the function. -- Jeremy commits: ecb3b0e41e0bbf5973cf16bf4d39ca0de557155b |
|
|
|
Re: Review: Inital pass at searchpath impl
Jeremy Selan <jeremy...@...>
I've checked in this patch to the master branch. The code itself
toggle quoted message
Show quoted text
looks good; whether we choose to backtrack from that API direction in the upcoming weeks can/should be a separate issue from this particular checkin. -- Jeremy On Mon, Oct 25, 2010 at 10:55 PM, Jeremy Selan <jeremy...@...> wrote:
Malcolm, |
|
|
|
Re: Review: Inital pass at searchpath impl
Jeremy Selan <jeremy...@...>
Malcolm,
toggle quoted message
Show quoted text
I wish I had known you were thinking about writing this particular piece of code; I probably would have asked you to hold off on it for a little while... (This is my mistake, I do realize that this task was listed in the OCIO TODO file). We've just started testing the current version of OCIO at Imageworks, and already we've run into some major confusion regarding lut paths. This past week's experiences (with Joseph playing the kind role of guinea pig) illuminates this discussion, and makes me think that some backtracking may be in order on this topic. * An OCIO profile is not just a single configuration file in isolation - it's a config AND a set of associated luts. Keeping these together all times (and conceptually thinking of them as inseparable) is probably the only way to avoid ambiguity and errors. An OCIO config is not useful if it gets separated from its luts, and vice-versa. In fact, if it's possible to separate the two, it WILL happen, and only bad will come of it. It's hit us once this week, I'd like to prevent anyone else from ever having to track down this type of issue. * I'd like for an OCIO profile to be able to be distributed, and used, either as a single self contained file or as a working directory. * Consider, what would the OCIO world look like if we 'locked down' a fixed directory layout that all profiles had to conform to? Fixed OCIO profile (in directory form) rootdir/ config.txt filetransforms/ a.lut, b.lut, etc. Point $OCIO at the root directory, (not the config,txt file), and everything "just works". All search paths, resource paths, go away. A bit inconvenient? Perhaps. But unambiguous and simple? Definitely. * With this model, the "single file" profile + luts representation becomes trivial: we would just support pointing $OCIO at a compressed version of the .ocio directory layout (zip, tgz, etc). Want to know what's inside your .ocio.zip config? Easy, just unzip it using the normal os tools. You can also unzip a profile, drag and drop a new lut internally, and then re-zip it for distribution. Super easy, and no lut transcoding needed. But what would the API look like if we went with this approach? It would be more directory / file based, and less stream centric. Would the "write" call write a directory instead? (and copy all referenced luts into a fixed local lut dir?) Probably. As foreign as this idea sounds, putting the OCIO configuration on rails and locking down a fixed directory layout may really help to minimize errors and confusion going forward. On a slighly different topic, I think it will be preferable when possible to minimize the use of envvars. Consider the case where the 'global config' (created with $OCIO) is not used. (where config objects are instead manually initialized). Any options specified with envvars, such as search paths, will by definition be 'global' and will have to be worked around (unset) in the the other profiles. -- Jeremy ---------- Forwarded message ---------- From: Malcolm Humphreys <malcolmh...@...> Date: Mon, Oct 25, 2010 at 3:29 AM Subject: Re: [ocio-dev] Review: Inital pass at searchpath impl To: ocio...@... - fixed a bug with abs searchpath entries - empty '::' searchpaths now resolve to the profile cwd http://github.com/malcolmhumphreys/OpenColorIO/commit/5d922362259e86073592481d428926de1054414d .malcolm On 25/10/2010, at 6:59 PM, Malcolm Humphreys wrote:
- removed the EnvExpand recusion limits as this didn't work in the production env which |
|
|
|
Re: Reference !<Colorspace>
Jeremy Selan <jeremy...@...>
I'm not sure what promoting the reference colorspace to something
special - either in the API or the format - buys us. Can you provide details on how this would make things simpler (or any other benefits you see)? Coming clean, I have to admit that in our internal SPI Color library (the predecessor to OCIO) we actually never exposed a function call to query the name of the reference colorspace in 7 years of use. There was always a call to query scene linear, but the fact that this also happened to be the internal profile connection space was never relevant to end users. My thought in adding it to OCIO was for completeness sake when spec'ing out the ROLE listings, but until it has an obvious use case perhaps we should remove ROLE_REFERENCE? It can always be added back in, when needed. (This is probably the API corollary to premature optimization, where you speculatively add a feature that only adds to confusion and in practice is never used). I do believe that understanding what a particular color configuration uses as its connection space is critical, but in my opinion this is best solved textually at a configuration level (i.e. config docs). Thoughts? In terms of your specific suggestions, I think I would prefer to have the reference color space just be another colorspace, with no particular distinction. While it doesn't have a transform block, it does have all the other colorspace markup, so it really does feel 'colorspace-like'. I could also foresee a situation where someone was using the connection space implicitly, and it wasn't intended to be either named or used by the end user. (We havent done this, but I see no reason to disallow it). Addressing the extra nesting added to your .ocio transform block, your version doesn't strike me as any more readable. old: to_reference.children new: to.reference.children Isn't redundant nesting bad? Perhaps there is a better name for the "to_reference" and "from_reference" values? I'm totally open to revising that enum name. -- Jeremy |
|
|
|
Re: Review: added new yaml-cpp with YAML::Newline feature
Jeremy Selan <jeremy...@...>
Thanks,
LGTM On Mon, Oct 25, 2010 at 3:05 PM, Malcolm Humphreys <malcolmh...@...> wrote: On 26/10/2010, at 5:14 AM, Jeremy Selan wrote:Is there a reason to not remove yaml-cpp-0.2.5* from ext?No reason was just waiting to get the tick of approval, this is now removed. |
|
|
|
Re: Review: converted configs/testing/config.ocio to YAML
Jeremy Selan <jeremy...@...>
Looks good to me. I'll commit this after I finish working through the
search path checkins (tomorrow, hopefully). -- Jeremy On Mon, Oct 25, 2010 at 3:00 PM, Malcolm Humphreys <malcolmh...@...> wrote: - converted configs/testing/config.ocio to YAML |
|
|
|
Re: Reference !<Colorspace>
Malcolm Humphreys <malcolmh...@...>
Is it just a role? seems like each colorspace needs it to convert between each other. Roles main purpose is to have abstract names / pointers for colorspaces. The reference space could also be a role but it is also a little more special and required field for a vaild profile where I see roles as a optional workflow UI feature.
The ascii versions should still be legible. I was hoping to take a look at creating something which would take a ocio profile and create an embedded version. The basic idea would be to create a section at the bottom of the profile with all the luts / files that are contained in some !!binary tags. We could either update the src="foo" in the !<FileTransform> to references these embedded files or adapt the search path mechanism. It would be nice to get some ideas on how you envisioned this stuff working, before I get carried away.
I think that is fine idea, what about something like. --snip-- ocio_profile_version: 1 resource_path: luts strictparsing: false luma: [0.2126, 0.7152, 0.0722] reference: lnh roles: scene_linear: lnh compositing_log: lgh colorspaces: - !<ColorSpace> name: lnh ... - !<ColorSpace> name: lgh .. to_reference: !<FileTransform> {src: lgf.spi1d, interpolation: linear} --snip-- .malcolm
|
|
|
|
Re: Reference !<Colorspace>
Joseph Slomka <jsl...@...>
Malcolm,
I don't see that the reference space should be separate in the profile. It's important, but it is just a role. Where the reference sits in the xml or yaml should matter much. Although quite a bit of effort is going into making the configuration files readable one of the goals of ocio, so I thought, was to not manually edit these files. I would imagine that there will be binary versions of configurations files not long after luts can be fully embedded. As a personal implementation preference I would only want to implement the definition of the reference as a pointer to another colorspace. Just my 2 cents. -Joseph ________________________________________ From: ocio...@... [ocio...@...] On Behalf Of Malcolm Humphreys [malcolmh...@...] Sent: Monday, October 25, 2010 5:48 AM To: OpenColorIO Developers Subject: [ocio-dev] Reference !<Colorspace> Do you think the 'reference' colorspace should in a separate section in the profile? This would be similar to the PCS in the icc world. It could be an extra line 'reference: !<ColorSpace> ...' before the colorspaces in the profile. The 'reference' name could be reserved so that colorspace -> reference transforms could work the same. I can see this working as roles with ROLE_REFERENCE but it really feels like a more central OCIO concept than roles and should be separated. Also looking at the serialization with to_referance and from_referance, I'm thinking it would be easy to read if it was structured like this? what do you think? --from-- to_reference: !<GroupTransform> children: - !<FileTransform> {src: "", interpolation: unknown} - !<CineonLogToLinTransform> max_aim_density: [2.046, 2.046, 2.046] neg_gamma: [0.6, 0.6, 0.6] neg_gray_reference: [0.434995, 0.434995, 0.434995] linear_gray_reference: [0.18, 0.18, 0.18] from_reference: !<GroupTransform> children: - !<ExponentTransform> {value: [1, 1, 1, 1]} --from-- --to-- reference: to: !<GroupTransform> children: - !<FileTransform> {src: "", interpolation: unknown} - !<CineonLogToLinTransform> max_aim_density: [2.046, 2.046, 2.046] neg_gamma: [0.6, 0.6, 0.6] neg_gray_reference: [0.434995, 0.434995, 0.434995] linear_gray_reference: [0.18, 0.18, 0.18] from: !<GroupTransform> children: - !<ExponentTransform> {value: [1, 1, 1, 1]} --to-- .malcolm |
|
|
|
Re: Review: added new yaml-cpp with YAML::Newline feature
Malcolm Humphreys <malcolmh...@...>
On 26/10/2010, at 5:14 AM, Jeremy Selan wrote:
Is there a reason to not remove yaml-cpp-0.2.5* from ext?No reason was just waiting to get the tick of approval, this is now removed. http://github.com/malcolmhumphreys/OpenColorIO/commit/26aa9bfc1baa74796b355de0b2ec476960b1b481 .malcolm
|
|
|
|
Review: converted configs/testing/config.ocio to YAML
Malcolm Humphreys <malcolmh...@...>
- converted configs/testing/config.ocio to YAML
- added OpenColorIOTest to the ctest run ie 'make test & make test_verbose' - some test output was being written to /mcp changed this to /tmp for now http://github.com/malcolmhumphreys/OpenColorIO/commit/c8000fa44fbc4fa62feeebb2f505c115883dad6d .malcolm |
|
|
|
Re: Review: added new yaml-cpp with YAML::Newline feature
Jeremy Selan <jeremy...@...>
Is there a reason to not remove yaml-cpp-0.2.5* from ext?
Other than that, looks good to me. -- Jeremy On Mon, Oct 25, 2010 at 5:22 AM, Malcolm Humphreys <malcolmh...@...> wrote: - added yaml-cpp r443 which contains the new YAML::Newline emitter manipulator |
|
|
|
Re: Review: Inital pass at GCC visibility
Jeremy Selan <jeremy...@...>
Looks good to me.
(I tested this, as well as all the related post-email updates). -- Jeremy On Thu, Oct 21, 2010 at 1:44 PM, Malcolm Humphreys <malcolmh...@...> wrote: - Inital pass at GCC visibility http://gcc.gnu.org/wiki/Visibility |
|
|