On May 5, 2:30 pm, brunobignose <bruno.j....@...>
wrote:

What is you internal representation?
 - we know you want to present this as a black box but we would like
to know some technical details

My discussion of the internals as a 'black' box are meant to be from
the perspective of a simple client application, not from folks
interested in color management or to 'authoring' apps. (Authoring apps
being applications that can generate / introspect into color
configurations).

 - what are the scope of the manipulations, are they all 3D ? Mixed 1D and 3D?

The internal transformations currently include 1D luts / (normal + log
allocation), matrix operations, gamma, 3D lut. They can be arbitrarily
chained and used in the forward or inverse directions (with the
exception of 3d luts, where it is left to use the user to provide an
inverse). We also are adding a group transform, which will be useful
in the API.

 - how do you represent
       - transforms
       - display devices
       - looks

Please see the file config_v0.ocs, you will see examples of all 3.

ColorSpaces are ordered lists of atomic transforms (1d luts, mtx,
gamma, 3d lut, etc). They can be used in any order as many times as
necessary.
DisplayDevices are just labels that define additional blocks of
transforms.
"Looks" are currently a bit in flux. If a look is constant across a
show (even if there are multiple looks), it is typically rolled into
the DisplayDevice block as this is more convenient. Examples of this
would be show-wide 'Warm Look', or "DI Gamut check', etc. If a look
is shot-varying we can use either a single 3dlut or asc color
correction. (However, before release i expect this will get more
general and allow for an arbitrary transform type).

What third party LUTs do you manage?

Currently the list is small, just some internal formats and .3dl
files. But, we hope to support all commercial ascii lut formats
before 1.0

 - any open source ones?

What open source lut formats are you referring to?

 - what happens if a user doesn't have a license for a proprietary lut
library?

It is currently not obvious how we will add support for proprietary
(encrypted) lut formats. Example, truelight. Some companies allow for
the export of ascii luts, but others dont. One workaround would be to
provide a lut plugin interface, so 3rd party color management
companies could write OCS plugins that do the appropriate licence
checks before running a lut transform. A plugin API is probably
outside the scope of a 1.0 release, but could be added to 1.1 without
effecting binary compatibility.

How does one go about generating...
 - LUTs?

Any ways you want. This is not a problem we're looking to solve for
1.0, and is treading deep into philosophical / 'secret sauce' issues.
Internally at SPI, we have a bunch of raw measurement data (such as
film stock profiles, camera exposure sweeps, etc) that guide this
practice. You could make a whole class on this, I'm sure. There are
also a bunch of different philosophies on how this should be done,
many of which are equally valid. My gut feel is that if this project
succeeds, we'll end up with a whole mailing list devoted to this
topic.

Please read the color space comments in this file,
config_v0.ocs

it provides a bit more info.

 - display profiles?

Same as above.

 - looks?

These are typically delivered by DI houses, and are director / DP
guided. If a user is not provided a look by their client, they
probably dont have to worry about one.

 - anything else in there...

Remember that we'll provide a few good default configurations, but I
expect all major facilities to define their own. And, we shouldnt
dictate how they do this. The default configurations wont suck
either, they're really what we use as the starting point for films.

Are you looking at making it extensible?

Not sure what you mean here. Anyone can add or create a custom
colorspace configuration. Anyone can take one of the posted
configurations and add new color spaces or devices to it. And the
whole library will be open source so the internal details wont be
secret. So,... "yes?"

How does this all play with ICC proiles?

We hope to add the ability to export ICC profiles, for lut preview in
tools such as photoshop. Other than that, they are not closely
related. We currently have no intent to add support for reading ICC
profiles.

What level of thread safety is there?

The library is fully thread safe. (and efficient in a multi-thread
context).

I am currently adding the color configuration mutable API, and am
grappling with a few corner issues such as ' what if one thread
manipulates the color configuration while another is processing
pixels?' but this is a mental correctness issue, not a crashing issue.

Invertibility,
 - how guaranteed is that for the general colour transforms?

Color configurations specify whether each operation is invertible or
not. An 'auto' mode is provided for types that are easy to perfectly
invert, but we dont handle 'non-obvious' inversion cases. (i.e., 3d
luts).

Note that our 1D lut processing is really pretty clever (clever in a
good way), and guaranteed to have perfect invertibility. For
example, you can specify a 1d lut that goes from a low dynamic range
allocation to an arbitrary high-dynamic range allocation, and still
get perfect invertibility. We also have a validation script that
prints out info about the current color configuration, this has a step
thats walks through each colorspace and validates perfect
invertibility (flagging otherwise).

 - can you flag invertible transforms in any way?

Yes.

Distributing colour management XML and LUTS
 - have you thought of how to pack XML + LUTS + whatever into a single
file for distribution?

Yes, we're starting to think about it. Internally a directory works
great but we're open to a single file as well. zip or tgz sound
great, we have no preference. I'd probably do a timing check and pick
whichever is faster to decompress live.

 - zip/tgz/voodoo?

OpenGL fragment shaders
 - how do we/you manage any LUTs that need to go along with the
shaders you create?

Only a single 3d lut is needed client side for the shader. The client
is responsible for uploading to the card as necessary, though we
provide a cacheid (cheap to compute) so it can be recomputed /
uploaded as infrequently as possible.

 - have you thought about OpenCL code generation?

Not yet. Are you interested in opengl?

 - we want to be able to supply our own names for objects in generated
code so as to avoid name collision, eg: the function name,

Yes, this is already part of the base api. please see the header.

Note that sidefx has already pointed out that we're going to have to
have finer granularity in our specification of glsl profiles (better
glsl versions), this has yet to be added but will soon.

On going support
 - who is responsible for support?
 - how do we manage folding in fixes?

It's an open source project, so at a minimum I'm responsible for
support. But my hope is that before 1.0 we develop a critical mass
community of those interested who can all work on this 'as developer
equals'. (multiple people having 'root' checkin privileges). I always
get suspicious of open source projects where one person approves all
checkins. My hope is that a few other reputable color folks will step
up and we can all be responsible for validating new code checkins,
etc. Of course, i'll be reading every line of code, but having more
folks on board helps prevent 'hit by bus' scenarios. (or more likely,
'caught in a hard production crunch-time' scenario).

Support on the artist / usage side will be open to a much broader
community, just as nuke-users is.

Are you shipping it with any vanilla XFMs that you have created at
Sony?
 - and not just log/lin
 - device profiles?

What's an XFM? A configuration I presume? (im now calling them .ocs
files)

Yes, we're gonna ship all input device profiles we've got, not just
log/lin. The only camera we've never characterized is RED, so we'd
have to leave that to someone else. (Or could adapt the nuke one for
use in our environment).

For device profiles the only ones we ever use are srgb, r709, and p3
dlp. These will be included.

thanks

Bruno NIcoletti

Oh, and one thing to keep in mind; the current version of this library
already works great in Nuke. :)

We've exposed 3 new nodes: ColorSpaceConvert / LogConvert /
DisplayTransform. The 1st two are used in the comp graph natively,
and the final one is used in the Input Process group to do filmlook
conversions. We also have a custom read / write nodes that uses our
color processing instead of the nuke-native processing. All of these
plugins are planned on being open-sourced as well, so right off the
bat we'll have a pretty good workflow example. (And clean example code
on how to do plugin client integration).

Obviously, native application support is the direction we're looking
to head in the long term, but I wanted to clarify that direct
manufacturer support is by no means required for Open Color Space to
be useful at launch.

-- Jeremy

Hi Jeremy,

thanks for the reply. A few more questions....

 - how do you represent
       - transforms
       - display devices
       - looks

Please see the file config_v0.ocs, you will see examples of all 3.

These reference various .spiXXX files, are you going to open up the
format of those files?

 - any open source ones?

What open source lut formats are you referring to?

Little CMS was the one mumbled about here. Not sure how important this
is, need to get back to our folks.

Are you looking at making it extensible?

Not sure what you mean here.  Anyone can add or create a custom
colorspace configuration.  Anyone can take one of the posted
configurations and add new color spaces or devices to it.  And the
whole library will be open source so the internal details wont be
secret.  So,... "yes?"

By some sort of plugin method, you mentioned this in your reply with
respect to proprietary CMS systems.

What level of thread safety is there?

The library is fully thread safe. (and efficient in a multi-thread
context).

I am currently adding the color configuration mutable API, and am
grappling with a few corner issues such as ' what if one thread
manipulates the color configuration while another is processing
pixels?' but this is a mental correctness issue, not a crashing issue.

I'd simply say you couldn't process while manipulating the the configuration.

 - have you thought about OpenCL code generation?

Not yet. Are you interested in opengl?

We are playing with OpenCL and it might be useful to have that at some
point. Not a high priority for us at the moment.

Are you shipping it with any vanilla XFMs that you have created at
Sony?
 - and not just log/lin
 - device profiles?

What's an XFM?  A configuration I presume? (im now calling them .ocs
files)

Yes, we're gonna ship all input device profiles we've got, not just
log/lin.  The only camera we've never characterized is RED, so we'd
have to leave that to someone else. (Or could adapt the nuke one for
use in our environment).

For device profiles the only ones we ever use are srgb, r709, and p3
dlp. These will be included.

Sorry, 'XFM' is my bizarre short hand for 'transform', but that still
answers it.



I suppose what we are missing are use cases of the library in
practice. I have a pretty good guess as to how it would work at big
film houses, film houses collaborating and film houses sharing work
with smaller shops, but clear descriptions of that would be great.

I guess my major concern is that if The Foundry is using this as the
basis of a cross product CMS, we still need to make it work for the
small shops in isolation, and they need tools to do at least the basic
stuff that the big boys have in-house specialist do. So display
calibration profiles and so on. Have you had any thoughts as to that?


--
Bruno Nicoletti

Sorry for the delay!

Progress on the library is going well, I hope to have a code drop
ready in the next few weeks.

To answer your latest few questions...

These reference various .spiXXX files, are you going to open up the
format of those files?

Yes and no. We're going to open up the formats in the sense that the
library will have a built-in reader support (The format details are
incredibly uninteresting; just another brain-dead ascii text format).
However, we're not going to push for anyone else to adopt them.
Internally, once we get support for commercial lut formats we'll
probably try to retire the spi formats.

I suppose what we are missing are use cases of the library in
practice. I have a pretty good guess as to how it would work at big
film houses, film houses collaborating and film houses sharing work
with smaller shops, but clear descriptions of that would be great.

I guess my major concern is that if The Foundry is using this as the
basis of a cross product CMS, we still need to make it work for the
small shops in isolation, and they need tools to do at least the basic
stuff that the big boys have in-house specialist do. So display
calibration profiles and so on. Have you had any thoughts as to that?

I'm not really familiar with how small houses (where someone is tasked
only part time) deal with the issue of color management. Maybe we
would ask some contacts at smaller houses for more detail about their
'off the shelf' color pipelines? I think this route would
particularly be worth the effort, as the small houses (who dont have
someone tasked full time to color) would have the most to gain in
having this library succeed.

-- Jeremy