combining a File-Based LUT Transform and a Display Transform?


Paul Miller <pa...@...>
 

I saw in the Mari OCIO demo video they were able to load up a LUT and combine it with other display controls (gain/exposure). I can see how the LUT can be loaded using the FileTransform but how is it possible to combine that with a DisplayTransform with the other controls for gain/gamma and channel swizzling?

Are you guys doing your own gain/gamma/swizzling when using the File-based Transform?


Joseph Slomka <joseph...@...>
 

Paul,

It's simple enough as the display transforma are just specified as colorspaces.

I suspect that the swizzling is being done outside of ocio. You can just apply the standard image manipulation once it's loaded. So you can load a file, linearize it, apply exposure in linear, then move it to display space, and apply addition transforms, for gain offset gamma.

Or did I miss your question?

-Joseph  


On Thu, Dec 15, 2011 at 10:41 AM, Paul Miller <pa...@...> wrote:
I saw in the Mari OCIO demo video they were able to load up a LUT and combine it with other display controls (gain/exposure). I can see how the LUT can be loaded using the FileTransform but how is it possible to combine that with a DisplayTransform with the other controls for gain/gamma and channel swizzling?

Are you guys doing your own gain/gamma/swizzling when using the File-based Transform?


Jeremy Selan <jeremy...@...>
 

I've just add the Mari source for the OCIO integration to the repo (with permission from the Foundry, of course) ;)

https://github.com/imageworks/OpenColorIO/tree/master/src/mari/1.4v1

As far as I can tell,  they do not do any special channel swizzle handling when in 'simple LUT mode'. (my terminology).

I think this will be a common integration request in the future:  "to utilize OCIO for the convenience of DisplayTransform + Lut loading, but to optionally allow the user to not worry about creating a full OCIO config."

I think it would be easy to provide a new option to DisplayTransform to allow for this possibility - let me look into it further.

-- Jeremy


On Thu, Dec 15, 2011 at 10:34 AM, Joseph Slomka <joseph...@...> wrote:
Paul,

It's simple enough as the display transforma are just specified as colorspaces.

I suspect that the swizzling is being done outside of ocio. You can just apply the standard image manipulation once it's loaded. So you can load a file, linearize it, apply exposure in linear, then move it to display space, and apply addition transforms, for gain offset gamma.

Or did I miss your question?

-Joseph  


On Thu, Dec 15, 2011 at 10:41 AM, Paul Miller <pa...@...> wrote:
I saw in the Mari OCIO demo video they were able to load up a LUT and combine it with other display controls (gain/exposure). I can see how the LUT can be loaded using the FileTransform but how is it possible to combine that with a DisplayTransform with the other controls for gain/gamma and channel swizzling?

Are you guys doing your own gain/gamma/swizzling when using the File-based Transform?



Jeremy Selan <jeremy...@...>
 

New thought - one workaround for the moment would be to dynamically add a new colorspace for the user specified lut.

This is definitely not ideal (I'd much rather extend DisplayTransform), but this will work in the existing codebase.

Psuedocode:

# When the user lut is selected...
config = OCIO.GetCurrentConfig().createEditableCopy()
colorspace = OCIO.ColorSpace(name='user')
t = OCIO.FileTransform('/absolute/path/to/yourlut.lut', interpolation=OCIO.Constants.INTERP_LINEAR)
colorspace.setTransform(t, OCIO.Constants.COLORSPACE_DIR_FROM_REFERENCE)
colorspace.addColorSpace(colorspace)   # this will overwrite 'user', if it already exists.

# Add a fake  user display / user view.  (Ugh)
config.addDisplay('user', 'user', 'user', '')

OCIO.SetCurrentConfig(config)



# And then when the display transform is used, just do it normally.
Specify as inputColorSpace ROLE_REFERENCE.   And then specify display / view as 'user' / 'user'.  The lut will be applied.



On Thu, Dec 15, 2011 at 11:05 AM, Jeremy Selan <jeremy...@...> wrote:
I've just add the Mari source for the OCIO integration to the repo (with permission from the Foundry, of course) ;)

https://github.com/imageworks/OpenColorIO/tree/master/src/mari/1.4v1

As far as I can tell,  they do not do any special channel swizzle handling when in 'simple LUT mode'. (my terminology).

I think this will be a common integration request in the future:  "to utilize OCIO for the convenience of DisplayTransform + Lut loading, but to optionally allow the user to not worry about creating a full OCIO config."

I think it would be easy to provide a new option to DisplayTransform to allow for this possibility - let me look into it further.

-- Jeremy


On Thu, Dec 15, 2011 at 10:34 AM, Joseph Slomka <joseph...@...> wrote:
Paul,

It's simple enough as the display transforma are just specified as colorspaces.

I suspect that the swizzling is being done outside of ocio. You can just apply the standard image manipulation once it's loaded. So you can load a file, linearize it, apply exposure in linear, then move it to display space, and apply addition transforms, for gain offset gamma.

Or did I miss your question?

-Joseph  


On Thu, Dec 15, 2011 at 10:41 AM, Paul Miller <pa...@...> wrote:
I saw in the Mari OCIO demo video they were able to load up a LUT and combine it with other display controls (gain/exposure). I can see how the LUT can be loaded using the FileTransform but how is it possible to combine that with a DisplayTransform with the other controls for gain/gamma and channel swizzling?

Are you guys doing your own gain/gamma/swizzling when using the File-based Transform?




Paul Miller <pa...@...>
 

On 12/15/2011 1:05 PM, Jeremy Selan wrote:
I've just add the Mari source for the OCIO integration to the repo (with
permission from the Foundry, of course) ;)

https://github.com/imageworks/OpenColorIO/tree/master/src/mari/1.4v1

As far as I can tell, they do not do any special channel swizzle
handling when in 'simple LUT mode'. (my terminology).
Before I dive into this - does "simple LUT mode" bypass the usual DisplayTransform or is there a way to combine the FileTransform (LUT) with the DisplayTransform? In other words, does anything get disabled UI-wise when a "simple LUT" is selected?


Paul Miller <pa...@...>
 

On 12/15/2011 1:15 PM, Jeremy Selan wrote:
New thought - one workaround for the moment would be to dynamically add
a new colorspace for the user specified lut.

This is definitely not ideal (I'd much rather extend DisplayTransform),
but this will work in the existing codebase.

Psuedocode:
...

# Add a fake user display / user view. (Ugh)
...

Ugh is right. Maybe I'll just wait until there is way to combine them logically in the API. :-)


Jeremy Selan <jeremy...@...>
 

Oh!

One other alternative implementation.

When in 'Simple Lut' mode, instead of using the displaytransform, just use a group transform and add the normal view components in the canonical order.

g = GroupTransform()
g.push_back(linearGainTransform)
g.push_back(channelSwizzleTransform)
g.push_back(FileTransform(src...))
g.push_back(gamma/bp/wpTransform)

config.getProcessor(g)

-- Jeremy


On Thu, Dec 15, 2011 at 11:21 AM, Paul Miller <pa...@...> wrote:
On 12/15/2011 1:15 PM, Jeremy Selan wrote:
New thought - one workaround for the moment would be to dynamically add
a new colorspace for the user specified lut.

This is definitely not ideal (I'd much rather extend DisplayTransform),
but this will work in the existing codebase.

Psuedocode:

...


# Add a fake  user display / user view.  (Ugh)

...

Ugh is right.  Maybe I'll just wait until there is way to combine them logically in the API. :-)


Paul Miller <pa...@...>
 

On 12/15/2011 1:28 PM, Jeremy Selan wrote:
Oh!

One other alternative implementation.

When in 'Simple Lut' mode, instead of using the displaytransform, just
use a group transform and add the normal view components in the
canonical order.
Oh - I think that is EXACTLY what I was looking for! (wasn't aware the GroupTransform could do this). Thanks!


Paul Miller <pa...@...>
 

On 12/15/2011 1:28 PM, Jeremy Selan wrote:
One other alternative implementation.

When in 'Simple Lut' mode, instead of using the displaytransform, just
use a group transform and add the normal view components in the
canonical order.

g = GroupTransform()
g.push_back(linearGainTransform)
g.push_back(channelSwizzleTransform)
g.push_back(FileTransform(src...))
g.push_back(gamma/bp/wpTransform)
This makes sense conceptually, but those operations boil down to a lot of different transforms that are hidden inside the DisplayTransform, each calling some (internal?) functions to build things recursively.

I've borrowed my existing DisplayTransform setup code from ociodisplay. Can you be more specific on how to set up the individual sub-transforms and add them to the Group in the context of what ociodisplay does?


Paul Miller <pa...@...>
 

On 12/15/2011 4:47 PM, Paul Miller wrote:
On 12/15/2011 1:28 PM, Jeremy Selan wrote:
One other alternative implementation.

When in 'Simple Lut' mode, instead of using the displaytransform, just
use a group transform and add the normal view components in the
canonical order.

g = GroupTransform()
g.push_back(linearGainTransform)
g.push_back(channelSwizzleTransform)
g.push_back(FileTransform(src...))
g.push_back(gamma/bp/wpTransform)
This makes sense conceptually, but those operations boil down to a lot
of different transforms that are hidden inside the DisplayTransform,
each calling some (internal?) functions to build things recursivel
Ah, this was easier than it looked. Note I skipped the display, view, and input color-space options when a simple LUT is selected. Is that the preferred way to do it?


Jeremy Selan <jeremy...@...>
 

> Ah, this was easier than it looked. Note I skipped the display, view, and input color-space options when a simple LUT is selected. Is that the preferred way to do it?

Yup.  The input colorspace, view, and display are just used to look up which display transform to use, and your user-specified LUT conceptually is performing that operation.  So the sequence I gave you is exactly what you'd get internally, if I added this feature to the DisplayTransform natively.

-- Jeremy


Paul Miller <pa...@...>
 

On 12/16/2011 10:46 AM, Jeremy Selan wrote:
> Ah, this was easier than it looked. Note I skipped the display, view,
and input color-space options when a simple LUT is selected. Is that the
preferred way to do it?

Yup. The input colorspace, view, and display are just used to look up
which display transform to use, and your user-specified LUT conceptually
is performing that operation. So the sequence I gave you is exactly
what you'd get internally, if I added this feature to the
DisplayTransform natively.
Cool. It was "easy" once I realized the FileTransform replaced the display/view/colorspace stuff. Here is the code in C++:

if (fileTransform)
{
OCIO::GroupTransformRcPtr group = OCIO::GroupTransform::Create();
group->push_back(gainTransform);
group->push_back(swizzle);
group->push_back(fileTransform);
group->push_back(expTransform);

processor = config->getProcessor(group);
}
else
{
OCIO::DisplayTransformRcPtr transform = OCIO::DisplayTransform::Create();
transform->setInputColorSpaceName(colorspace);
transform->setDisplay(display);
transform->setView(view);
transform->setLinearCC(gainTransform);

processor = config->getProcessor(transform);
}


Paul Miller <pa...@...>
 

else
{
OCIO::DisplayTransformRcPtr transform = OCIO::DisplayTransform::Create();
transform->setInputColorSpaceName(colorspace);
transform->setDisplay(display);
transform->setView(view);
transform->setLinearCC(gainTransform);

processor = config->getProcessor(transform);
}
Note I left out the expTransform from the second case by accident. There should be:

transform->setDisplayCC(expTransform);

Added after the setLinearCC line.


dbr/Ben <dbr....@...>
 

On 17/12/2011, at 3:46 AM, Paul Miller wrote:
if (fileTransform)
{
   OCIO::GroupTransformRcPtr group = OCIO::GroupTransform::Create();
   group->push_back(gainTransform);
   group->push_back(swizzle);
   group->push_back(fileTransform);
   group->push_back(expTransform);

   processor = config->getProcessor(group);
}

Was going through old emails and noticed - the swizzle should probably go after the fileTransform, in case the file has channel-crosstalk (e.g desaturating RGB would be very different to desaturating the swizzled "RRR")


Jeremy Selan <jeremy...@...>
 

This order (view channel matrix *before* 3d view transform) was intentional, and mimics what's going on under the hood in DisplayTransform:
https://github.com/imageworks/OpenColorIO/blob/master/src/core/DisplayTransform.cpp#L292

The rationale for applying the channel view matrix *before* the viewing transform is that, based on my understanding, most users actually want to inspect the imagery prior to cross-talk being introduced.

I.e., when you press 'R' to view 'red', my mental model is the artist wants to inspect the data that is in the source image's 'red channel', (rather than what happens to be driving the red output on the viewing display).  Consider looking at grain on a plate.  If you press 'R', you're probably concerned with just looking at the data in that channel, and are not concerned with the cross-talk in the viewing lut. Is there a time where it is useful to do the opposite?

This also has the added benefit that the output image is stable independent of output display.  I.e., one were to do the channel swizzling post view transform, pressing 'R' on a DLP vs. srgb display would yield very different images.  But doing channel swizzling before makes for a visually equivalent output. (such as in the above grain example).

The downside is that in this approach you can have a color cast to the 'grayscale' output of 'r,g,b,l' modes (if the viewing transform doesnt map gray to gray), but this is not typically a concern in practice. (and is often desired anyways in some cases)

Thoughts?

-- Jeremy

On Sun, Jan 29, 2012 at 4:11 AM, dbr/Ben <dbr....@...> wrote:
On 17/12/2011, at 3:46 AM, Paul Miller wrote:
if (fileTransform)
{
   OCIO::GroupTransformRcPtr group = OCIO::GroupTransform::Create();
   group->push_back(gainTransform);
   group->push_back(swizzle);
   group->push_back(fileTransform);
   group->push_back(expTransform);

   processor = config->getProcessor(group);
}

Was going through old emails and noticed - the swizzle should probably go after the fileTransform, in case the file has channel-crosstalk (e.g desaturating RGB would be very different to desaturating the swizzled "RRR")


Brendan Bolles <bre...@...>
 

On Jan 30, 2012, at 12:19 PM, Jeremy Selan wrote:

The downside is that in this approach you can have a color cast to the 'grayscale' output of 'r,g,b,l' modes (if the viewing transform doesnt map gray to gray), but this is not typically a concern in practice. (and is often desired anyways in some cases)

This is the thing that would seem the weirdest to me - looking at the red channel and seeing it have a color cast. I'm not sure why that would ever be "desired". The film look in the spi-vfx config makes a cast. I'm pretty sure no other program does that.

At the same time, I agree that when you want to look at just the red channel, you probably want to see the image data pre-LUT. Maybe a really smart implementation would do lin->log->red or lin->sRGB->red instead of lin->film->red when just the red channel was being viewed.


Brendan


Jeremy Selan <jeremy...@...>
 

On Wed, Feb 1, 2012 at 10:29 AM, Brendan Bolles <bre...@...> wrote:
On Jan 30, 2012, at 12:19 PM, Jeremy Selan wrote:

> The downside is that in this approach you can have a color cast to the 'grayscale' output of 'r,g,b,l' modes (if the viewing transform doesnt map gray to gray), but this is not typically a concern in practice. (and is often desired anyways in some cases)


This is the thing that would seem the weirdest to me - looking at the red channel and seeing it have a color cast.  I'm not sure why that would ever be "desired".  The film look in the spi-vfx config makes a cast.  I'm pretty sure no other program does that.

Depends on the intent of the color cast.

Say you're using a film emulation lut where the bright neutrals get mapped to a warm color, and the dark neutrals get mapped to a cool color. Having your 'luma' mode to this too may look a bit odd.

But say your output device is a DCI P3 projector, where equal code values map to a very greenish white point.   Your output 3D lut may instead be mapping all values equally to a more pleasing white point, in which case if we '1d-ified' the luma mode all of a sudden you'd jump between the color and luma modes and the image would all of a sudden become greenish.


Consider a grayscale image, where you're toggling between luma mode and color mode.  Would the artist expect the image to change or not?  In the current implementation, they would appear identical.


At the same time, I agree that when you want to look at just the red channel, you probably want to see the image data pre-LUT.  Maybe a really smart implementation would do lin->log->red or lin->sRGB->red instead of lin->film->red when just the red channel was being viewed.

In the old SPI library (the internal predecessor to OCIO) when viewing individual channels we dynamically computed a version of the 3D viewing lut that didnt have any crosstalk. (which makes input equal code values always map to output equal code values) but this always felt a bit too magical for my tastes, and also looked weird on the DLP.. (greenish)


-- Jeremy

 

Brendan