Hello!

I'm at the stage where I would like to get a survey of lut file
formats (1d, 3d, matrix, etc) that folks actually use in the wild.

If you commonly use a lut format at your facility, or define one in
your software package, I would hugely appreciate it if you could
upload example files and/or spec documents so I can begin work on the
importers.

Even if it's a proprietary format, I'd love to take a peek so I can
get a sense of the scope of formats out in the wild.  (I need to make
sure the internal API is rich enough to encompass current use cases).

Many formats have been mentioned previously, including:
   - Truelight ASCII .cub 3D lut
   - Assimilate Scratch  (Arri /Kodak .3dl)
   - Iridas Framecycler  (Iridas .cube)
   - Autodesk            (MESH3D, .lut, etc.)

For the majority of these, I do not have either example files or
specifications. Please help! :)

Also, does anyone know if the majority of lut formats identifiable by
their extension?  Are there common extension conflicts?  Ideally, I'd
like to try and have format reader registered based on file extension,
and only if that fails give each lut loader a chance to read it.
(Similar to how reader plugins work in nuke).

(Note that I'm not assuming 1 file == 1 lut.  (We will support readers
where 1 file can generate multiple transforms, such as a 1d shaper lut
-> 3d lut -> 1d shaper lut.))

Hi Jeremy,

these are the ones we support in Nuke...

.vf     Nuke native vector field
vfz    gzip compressed .vf files
.cub(e)    Truelight 1D or 3D LUT
.cube    Iridas LUT
.csp    Cinespace LUT
.3dl    Autodesk/Scratch LUT
.blut    Houdini mplay blut

I can get you example files

b

On 21 May 2010 22:35, Jeremy Selan <jeremy...@gmail.com> wrote:

Hello!

I'm at the stage where I would like to get a survey of lut file
formats (1d, 3d, matrix, etc) that folks actually use in the wild.

If you commonly use a lut format at your facility, or define one in
your software package, I would hugely appreciate it if you could
upload example files and/or spec documents so I can begin work on the
importers.

Even if it's a proprietary format, I'd love to take a peek so I can
get a sense of the scope of formats out in the wild.  (I need to make
sure the internal API is rich enough to encompass current use cases).

Many formats have been mentioned previously, including:
   - Truelight ASCII .cub 3D lut
   - Assimilate Scratch  (Arri /Kodak .3dl)
   - Iridas Framecycler  (Iridas .cube)
   - Autodesk            (MESH3D, .lut, etc.)

For the majority of these, I do not have either example files or
specifications. Please help! :)

Also, does anyone know if the majority of lut formats identifiable by
their extension?  Are there common extension conflicts?  Ideally, I'd
like to try and have format reader registered based on file extension,
and only if that fails give each lut loader a chance to read it.
(Similar to how reader plugins work in nuke).

(Note that I'm not assuming 1 file == 1 lut.  (We will support readers
where 1 file can generate multiple transforms, such as a 1d shaper lut
-> 3d lut -> 1d shaper lut.))

--
Bruno Nicoletti

I can send the specs for the Houdini .lut (ASCII) and .blut (bin)
formats, if you need that. They're also in the HDK documentation, if
you have access to that.

Neat.  So just to clarify, the big idea is that any app that uses your color library properly will, by extension, understand ALL of these luts (whichever happens to be available/used at the facility using the app)?

Just how automated can this be?  I'm a little unclear on how much extra homework a display program needs to do.  Will your library... um... just find the lut specs in whatever standard places they are stored and know how to correct images for proper display on that device?  Are there standard places for the luts to be stored?  Will an app need various user options/dialogs to set this up?  Is this too much of a newbie question?

On May 25, 2010, at 2:51 PM, Jeremy Selan wrote:

Blake was kind enough to a google doc spreadsheet of lut formats:
http://spreadsheets.google.com/ccc?key=0Avn2cZhY9jaYdF9qTHYteU1MUDYtd...

I'll try to edit it to keep it up to date.  (what formats we support,
are planning to support, have examples of, etc). And eventually it'll
probably turn into part of the ocs website.

--
Larry Gritz
l....@imageworks.com

Exactly.   Clients apps will automatically have support for all lut
formats.  (It's about time!)

To an end user, this should feel pretty automated.  Apps shouldn't
need ebd user options / dialogs, all of the lut-location mojo is
internal to the ColorConfiguration file.

Specifying which configuration to use is controlled via a single
environment variable (which points at an ocs file). (Explicit
initialization is also allowed, but this will probably be less
common).

The internals of the .ocs file are still in flux, but the rough idea
is that there will be two flavors.  One flavor will be a human-
readable xml file, of which one element lut dir.  The alternative will
be a binary ocs 'package', where the lut dir is self contained in the
file. (Think tgz of a config / lut dir).

In the super common newbie-case, downloading a single .ocs binary
file, and pointing to it with OCS_CONFIG will be all that's needed.

-- Jeremy

On May 25, 3:09 pm, Larry Gritz <l....@imageworks.com> wrote:

Neat.  So just to clarify, the big idea is that any app that uses your color library properly will, by extension, understand ALL of these luts (whichever happens to be available/used at the facility using the app)?

Just how automated can this be?  I'm a little unclear on how much extra homework a display program needs to do.  Will your library... um... just find the lut specs in whatever standard places they are stored and know how to correct images for proper display on that device?  Are there standard places for the luts to be stored?  Will an app need various user options/dialogs to set this up?  Is this too much of a newbie question?

On May 25, 2010, at 2:51 PM, Jeremy Selan wrote:

Blake was kind enough to a google doc spreadsheet of lut formats:
http://spreadsheets.google.com/ccc?key=0Avn2cZhY9jaYdF9qTHYteU1MUDYtd...

I'll try to edit it to keep it up to date.  (what formats we support,
are planning to support, have examples of, etc). And eventually it'll
probably turn into part of the ocs website.

--
Larry Gritz
l....@imageworks.com

Cool, thanks for the info.   So it sounds like when we want to write a .3dl
exporter, we'll need to specify which flavor of .3dl to write.  However, on
the import side it still sounds like we'll be able to get away with a
generalized import, presuming we're a bit clever with the bit-depth
interpretation.  Would you agree?

-- Jeremy

On Wed, May 26, 2010 at 7:08 PM, bsloan <bsl...@gmail.com> wrote:

Regarding 3dl --

The lore I've been fed is that Mitch Bogdonowicz formerly of Kodak
wrote (invented? burped?) the 3dl "specification" when he was at Kodak
as a quick and dirty solution of 3D LUT ASCII serialization.

Various interpretations have since been adopted (supported?)  by
Arri,  Autodesk, Assimilate and others.

Yes, the initial 1D monochrome shaper array is in some cases in a
different bit depth from the 3D rgb array. What's more, various
applications choke if the header comment fields (those preceded with
a"#") are not 'just so'  (eg. truelight's tl-utils).

I guess the point is that there isn't just one spec for it. Maybe the
way to deal with 3dl is to treat different interpretations as
different formats:

*arri.3dl
*autodesk.3dl
*scratch.3dl

ugly -r- us.

:)

-blake

This one will actually build on other people's machines...

please see INSTALL file for instructions on how to build the Nuke
example (posted here as well):

Make a build directory, cd into it:

mkdir -p build
cd build

Configure cmake, pointing at your nuke include dir:

cmake -D NUKE:FILEPATH=/usr/local/nuke6.0.3/include/ -D CMAKE_BUILD_TYPE=Release ../

Build all targets:

make

You will now see the built .so files:

ls

Set the environment variable pointing at which configuration to use:
(Point this to your install)

setenv OCS /net/homedirs/jeremys/git/Color/configs/spivfx/config.ocs

Launch nuke (the same version you used for the include files), then
load the plugin (this can also go in your init.py)

nuke.load('/net/homedirs/jeremys/git/Color//build/libNukeColorSpaceConversion.so')
nuke.menu('Nodes').addMenu('Color').addCommand('OCSColorSpaceConversion', 'nuke.createNode("OCSColorSpaceConversion")')

Connect an image to the OCSColorSpaceConversion node.
Our 'spivfx' config has only a few example colorspaces, this one
demonstrates a conversion between scene linear and a film-log
colorspace.

looking much nicer. However, I have The Fear as you are using C++ TR1
for smart pointer magic.

The use of tr1 shared_ptr is only a stub.  When we get multi-platform
stuff sorted out that section will definitely be replaced with
platform / compiler specific #ifdefs. Any shared_ptr that provides a
dynamic_cast will suffice. (tr1 does work on osx already though  :) )

First a justification of shared_ptrs in general...
We've spend a long time thinking about object ownership and object
lifetimes, and the choice to expose a smart pointer was not arrived at
lightly.  I do not see a graceful alternative.  If people are
interested we should probably discuss this further to bring everyone
on board.  (Or to prove me wrong!)  The quick summary is that in the
context of multi-threaded apps, where there exists a 'global' config
people can get/set, you need to have some form of reference counting
to assure that configs aren't destroyed while still in use.   I would
have ended up just re inventing a shared_ptr / intrusive_ptr, so it
seemed expedient to just use a real one.  It will also allow for a
more simper python use model, as objects can now be created on either
side of the fence (C++ or python) and passed back and forth without
concern for object lifetime.  (Keeping a reference to the python
object will keep the C++ object alive, which is very desirable). This
will particularly be useful in python UIs that make use of the mutable
API. (To create and edit configs 'live')

One related thing of note - all exposed objects which use shared-ptrs
have private constructors; the only way to create them is with static
factory functions.   Example: ColorSpaceRcPtr ColorSpace::Create() .
This is done so that the shared pointer is always created with a
custom object deallocator.  Our hope is this will work solve the
windows dll memory management issue.  (And we hope to verify this
soon).

OCS::ImageDesc
The init functions could definitely be constructors. My concerns is
that it would then rely on type signatures to maintain uniqueness, and
this has hosed me in the past. (Particularly where default values /
int / bools are involved).  You can accidentally create a change which
is binary compatible, but not source compatible.

I really like having super explicit names such as initRGBA,
initSingleBuffer, initMultiBuffer.  This will also let us add new
convenience constructors should they be desired without worrying about
compatibility.   (Consider 'initRGB', which would have the same type
signature as initRGBA.

But I agree the current init functions aren't ideal.  Does anyone have ideas?


Soon enough we will have a live share of the GIT repo, and which point
everyone can start contributing... ;)

-- Jeremy


On Fri, Jun 4, 2010 at 4:09 AM, Bruno Nicoletti
<bruno.j....@googlemail.com> wrote:

Hi Jeremy,

looking much nicer. However, I have The Fear as you are using C++ TR1
for smart pointer magic. We are on Visual Studio 2005 for Windows and
there is no TR1 for VS5 (we won't be changing in the near future, long
story). A #ifndef around SharedPtr and DynamicPtrCast would help, as
we could redirect this to boost.

Another minor quibble, should not the OCS::ImageDesc have ctors rather
than the initRGBA/initSingleBuffer/initMultiBuffer methods? Similarly
for other classes?

b

On 4 June 2010 00:44, Jeremy Selan <jeremy...@gmail.com> wrote:

This one will actually build on other people's machines...

please see INSTALL file for instructions on how to build the Nuke
example (posted here as well):

Make a build directory, cd into it:

mkdir -p build
cd build

Configure cmake, pointing at your nuke include dir:

cmake -D NUKE:FILEPATH=/usr/local/nuke6.0.3/include/ -D CMAKE_BUILD_TYPE=Release ../

Build all targets:

make

You will now see the built .so files:

ls

Set the environment variable pointing at which configuration to use:
(Point this to your install)

setenv OCS /net/homedirs/jeremys/git/Color/configs/spivfx/config.ocs

Launch nuke (the same version you used for the include files), then
load the plugin (this can also go in your init.py)

nuke.load('/net/homedirs/jeremys/git/Color//build/libNukeColorSpaceConversion.so')
nuke.menu('Nodes').addMenu('Color').addCommand('OCSColorSpaceConversion', 'nuke.createNode("OCSColorSpaceConversion")')

Connect an image to the OCSColorSpaceConversion node.
Our 'spivfx' config has only a few example colorspaces, this one
demonstrates a conversion between scene linear and a film-log
colorspace.

--
Bruno Nicoletti