I've used 10bit DPX for this purpose in the past (when I thought I'd
invented 2D 3D LUTs).

8 or 16 bit PPM may be a good compromise. It won't please everyone but
implementation requires so little code -- and the advantage of *not*
having to depend on openimageio may outweigh the disadvantage of
whining developers :).

There's some precedent for this -- NVidia's early Cg SDK provided the
source for a lightweight PPM IO library.

Who knows? This may be one case where OCS can influence the standard.

-blake


On Jun 8, 11:28 am, Larry Gritz <l....@imageworks.com> wrote:

I wouldn't burden OCS with the image-reading functionality.  You'll never make everybody happy without supporting a bzillion formats.  Why not just allow the lut to be specified as a big array and call it a day, and let apps that use OCS be responsible for reading the image files (or having its own dependency on an image-reading library) in order to pass the array?

On Jun 8, 2010, at 11:10 AM, Naty Hoffman wrote:

There are some subtleties with layout and mappings, so let me know
when it's time to implement the import/export logic and I'll write up
a spec.

As for file formats, people use different ones - TIFF and TGA are
common.  There are enough tools to support converting between 2D
formats that I don't know if it's worth adding complex library
dependencies to Open Color Space just for this. Does OCS have any 2D
image file formats supported at the moment? PPM (the RGB version of
PBM) could perhaps work - how widely is it supported by common image
viewing utilities, OS thumbnails, etc.?

Naty

Sent from my iPhone

On Jun 8, 2010, at 10:05 AM, Jeremy Selan <jeremy...@gmail.com>
wrote:

Yah, the "lut as image" approach should be really easy to support,
I'll add that to the short list.  My only concern would be which
fileformats to support.  I'd still like to keep our dependencies as
low as possible though.  Naty, when folks use this approach in
production, which file formats are actually used to store the pixel
data?
If it were a trivial ascii format (such as pbm,pnm, etc) that would be
ideal... ;)

-- Jeremy

On Jun 7, 7:04 pm, "Nathaniel Hoffman" <n....@io.com> wrote:

Jeremy,

The 2D LUT format I describe below is starting to become a bit more
standardized in the game industry - it is being used by two major
middleware companies (Crytek and Epic - see here:http://
udn.epicgames.com/Three/ColorGrading.html). So it might make sense
to support it after all...

Thanks,

Naty

Jeremy,

There are two LUT formats I know of that are used in game
development.

One is a 2D image format (any lossless image format will do -
we've used
BMP, TGA and PNG) with a 2D representation of the LUT where the
planes
along the 3rd axis have been placed next to each other. So a
32x32x32 LUT
would turn into a 1024x32 2D image. A common usage is for an
identity LUT
in this format to be placed next to an ungraded screenshot, both
manipulated in Photoshop or some other color manipulation package,
and
then the colorized LUT is extracted.

The other format is a DDS (Microsoft DirectDraw Surface) file with
a 3D
texture in it, typically uncompressed. This is usually loaded
directly
into the game engine.

These are both a bit ad-hoc and not really standardized, so I
don't know
if they are relevant for OCS. If they are, let me know and I can
try to
work up something more like an actual spec for each of these.

Thanks,

Naty

Hello!

I'm at the stage where I would like to get a survey of lut file
formats (1d, 3d, matrix, etc) that folks actually use in the wild.

If you commonly use a lut format at your facility, or define one in
your software package, I would hugely appreciate it if you could
upload example files and/or spec documents so I can begin work on
the
importers.

Even if it's a proprietary format, I'd love to take a peek so I can
get a sense of the scope of formats out in the wild.  (I need to
make
sure the internal API is rich enough to encompass current use
cases).

Many formats have been mentioned previously, including:
   - Truelight ASCII .cub 3D lut
   - Assimilate Scratch  (Arri /Kodak .3dl)
   - Iridas Framecycler  (Iridas .cube)
   - Autodesk            (MESH3D, .lut, etc.)

For the majority of these, I do not have either example files or
specifications. Please help! :)

Also, does anyone know if the majority of lut formats
identifiable by
their extension?  Are there common extension conflicts?  Ideally,
I'd
like to try and have format reader registered based on file
extension,
and only if that fails give each lut loader a chance to read it.
(Similar to how reader plugins work in nuke).

(Note that I'm not assuming 1 file == 1 lut.  (We will support
readers
where 1 file can generate multiple transforms, such as a 1d
shaper lut
-> 3d lut -> 1d shaper lut.))

--
Larry Gritz
l....@imageworks.com

I wouldn't burden OCS with the image-reading functionality.  You'll never make everybody happy without supporting a bzillion formats.  Why not just allow the lut to be specified as a big array and call it a day, and let apps that use OCS be responsible for reading the image files (or having its own dependency on an image-reading library) in order to pass the array?

On Jun 8, 2010, at 11:10 AM, Naty Hoffman wrote:

There are some subtleties with layout and mappings, so let me know  
when it's time to implement the import/export logic and I'll write up  
a spec.

As for file formats, people use different ones - TIFF and TGA are  
common.  There are enough tools to support converting between 2D  
formats that I don't know if it's worth adding complex library  
dependencies to Open Color Space just for this. Does OCS have any 2D  
image file formats supported at the moment? PPM (the RGB version of  
PBM) could perhaps work - how widely is it supported by common image  
viewing utilities, OS thumbnails, etc.?

Naty

Sent from my iPhone

On Jun 8, 2010, at 10:05 AM, Jeremy Selan <jeremy...@gmail.com>  
wrote:

Yah, the "lut as image" approach should be really easy to support,
I'll add that to the short list.  My only concern would be which
fileformats to support.  I'd still like to keep our dependencies as
low as possible though.  Naty, when folks use this approach in
production, which file formats are actually used to store the pixel
data?
If it were a trivial ascii format (such as pbm,pnm, etc) that would be
ideal... ;)

-- Jeremy

On Jun 7, 7:04 pm, "Nathaniel Hoffman" <n....@io.com> wrote:

Jeremy,

The 2D LUT format I describe below is starting to become a bit more
standardized in the game industry - it is being used by two major
middleware companies (Crytek and Epic - see here:http://
udn.epicgames.com/Three/ColorGrading.html). So it might make sense
to support it after all...

Thanks,

Naty

Jeremy,

There are two LUT formats I know of that are used in game  
development.

One is a 2D image format (any lossless image format will do -  
we've used
BMP, TGA and PNG) with a 2D representation of the LUT where the  
planes
along the 3rd axis have been placed next to each other. So a  
32x32x32 LUT
would turn into a 1024x32 2D image. A common usage is for an  
identity LUT
in this format to be placed next to an ungraded screenshot, both
manipulated in Photoshop or some other color manipulation package,  
and
then the colorized LUT is extracted.

The other format is a DDS (Microsoft DirectDraw Surface) file with  
a 3D
texture in it, typically uncompressed. This is usually loaded  
directly
into the game engine.

These are both a bit ad-hoc and not really standardized, so I  
don't know
if they are relevant for OCS. If they are, let me know and I can  
try to
work up something more like an actual spec for each of these.

Thanks,

Naty

Hello!

I'm at the stage where I would like to get a survey of lut file
formats (1d, 3d, matrix, etc) that folks actually use in the wild.

If you commonly use a lut format at your facility, or define one in
your software package, I would hugely appreciate it if you could
upload example files and/or spec documents so I can begin work on  
the
importers.

Even if it's a proprietary format, I'd love to take a peek so I can
get a sense of the scope of formats out in the wild.  (I need to  
make
sure the internal API is rich enough to encompass current use  
cases).

Many formats have been mentioned previously, including:
   - Truelight ASCII .cub 3D lut
   - Assimilate Scratch  (Arri /Kodak .3dl)
   - Iridas Framecycler  (Iridas .cube)
   - Autodesk            (MESH3D, .lut, etc.)

For the majority of these, I do not have either example files or
specifications. Please help! :)

Also, does anyone know if the majority of lut formats  
identifiable by
their extension?  Are there common extension conflicts?  Ideally,  
I'd
like to try and have format reader registered based on file  
extension,
and only if that fails give each lut loader a chance to read it.
(Similar to how reader plugins work in nuke).

(Note that I'm not assuming 1 file == 1 lut.  (We will support  
readers
where 1 file can generate multiple transforms, such as a 1d  
shaper lut
-> 3d lut -> 1d shaper lut.))

--
Larry Gritz
l....@imageworks.com

Yah, the "lut as image" approach should be really easy to support,
I'll add that to the short list. My only concern would be which
fileformats to support. I'd still like to keep our dependencies as
low as possible though. Naty, when folks use this approach in
production, which file formats are actually used to store the pixel
data?
If it were a trivial ascii format (such as pbm,pnm, etc) that would be
ideal... ;)

-- Jeremy

On Jun 7, 7:04 pm, "Nathaniel Hoffman" <n....@io.com> wrote:

Jeremy,

The 2D LUT format I describe below is starting to become a bit more
standardized in the game industry - it is being used by two major
middleware companies (Crytek and Epic - see here:http:// udn.epicgames.com/Three/ColorGrading.html). So it might make sense
to support it after all...

Thanks,

Naty

Jeremy,

There are two LUT formats I know of that are used in game development.

One is a 2D image format (any lossless image format will do - we've used
BMP, TGA and PNG) with a 2D representation of the LUT where the planes
along the 3rd axis have been placed next to each other. So a 32x32x32 LUT
would turn into a 1024x32 2D image. A common usage is for an identity LUT
in this format to be placed next to an ungraded screenshot, both
manipulated in Photoshop or some other color manipulation package, and
then the colorized LUT is extracted.

The other format is a DDS (Microsoft DirectDraw Surface) file with a 3D
texture in it, typically uncompressed. This is usually loaded directly
into the game engine.

These are both a bit ad-hoc and not really standardized, so I don't know
if they are relevant for OCS. If they are, let me know and I can try to
work up something more like an actual spec for each of these.

Thanks,

Naty

Hello!

I'm at the stage where I would like to get a survey of lut file
formats (1d, 3d, matrix, etc) that folks actually use in the wild.

If you commonly use a lut format at your facility, or define one in
your software package, I would hugely appreciate it if you could
upload example files and/or spec documents so I can begin work on the
importers.

Even if it's a proprietary format, I'd love to take a peek so I can
get a sense of the scope of formats out in the wild. (I need to make
sure the internal API is rich enough to encompass current use cases).

Many formats have been mentioned previously, including:
- Truelight ASCII .cub 3D lut
- Assimilate Scratch (Arri /Kodak .3dl)
- Iridas Framecycler (Iridas .cube)
- Autodesk (MESH3D, .lut, etc.)

For the majority of these, I do not have either example files or
specifications. Please help! :)

Also, does anyone know if the majority of lut formats identifiable by
their extension? Are there common extension conflicts? Ideally, I'd
like to try and have format reader registered based on file extension,
and only if that fails give each lut loader a chance to read it.
(Similar to how reader plugins work in nuke).

(Note that I'm not assuming 1 file == 1 lut. (We will support readers
where 1 file can generate multiple transforms, such as a 1d shaper lut
-> 3d lut -> 1d shaper lut.))

Version 0.5.6 (June 8 2010):
    * PyConfig stub implementation
    * Dropped ImageDesc.init; added PlanarImageDesc / PackedImageDesc
    * Dropped tr1::shared_ptr; added boost::shared_ptr

Jeremy,

The 2D LUT format I describe below is starting to become a bit more
standardized in the game industry - it is being used by two major
middleware companies (Crytek and Epic - see here:http://udn.epicgames.com/Three/ColorGrading.html). So it might make sense
to support it after all...

Thanks,

Naty

Jeremy,

There are two LUT formats I know of that are used in game development.

One is a 2D image format (any lossless image format will do - we've used
BMP, TGA and PNG) with a 2D representation of the LUT where the planes
along the 3rd axis have been placed next to each other. So a 32x32x32 LUT
would turn into a 1024x32 2D image. A common usage is for an identity LUT
in this format to be placed next to an ungraded screenshot, both
manipulated in Photoshop or some other color manipulation package, and
then the colorized LUT is extracted.

The other format is a DDS (Microsoft DirectDraw Surface) file with a 3D
texture in it, typically uncompressed. This is usually loaded directly
into the game engine.

These are both a bit ad-hoc and not really standardized, so I don't know
if they are relevant for OCS. If they are, let me know and I can try to
work up something more like an actual spec for each of these.

Thanks,

Naty

Hello!

I'm at the stage where I would like to get a survey of lut file
formats (1d, 3d, matrix, etc) that folks actually use in the wild.

If you commonly use a lut format at your facility, or define one in
your software package, I would hugely appreciate it if you could
upload example files and/or spec documents so I can begin work on the
importers.

Even if it's a proprietary format, I'd love to take a peek so I can
get a sense of the scope of formats out in the wild.  (I need to make
sure the internal API is rich enough to encompass current use cases).

Many formats have been mentioned previously, including:
    - Truelight ASCII .cub 3D lut
    - Assimilate Scratch  (Arri /Kodak .3dl)
    - Iridas Framecycler  (Iridas .cube)
    - Autodesk            (MESH3D, .lut, etc.)

For the majority of these, I do not have either example files or
specifications. Please help! :)

Also, does anyone know if the majority of lut formats identifiable by
their extension?  Are there common extension conflicts?  Ideally, I'd
like to try and have format reader registered based on file extension,
and only if that fails give each lut loader a chance to read it.
(Similar to how reader plugins work in nuke).

(Note that I'm not assuming 1 file == 1 lut.  (We will support readers
where 1 file can generate multiple transforms, such as a 1d shaper lut
-> 3d lut -> 1d shaper lut.))

Jeremy,

There are two LUT formats I know of that are used in game development.

One is a 2D image format (any lossless image format will do - we've used
BMP, TGA and PNG) with a 2D representation of the LUT where the planes
along the 3rd axis have been placed next to each other. So a 32x32x32 LUT
would turn into a 1024x32 2D image. A common usage is for an identity LUT
in this format to be placed next to an ungraded screenshot, both
manipulated in Photoshop or some other color manipulation package, and
then the colorized LUT is extracted.

The other format is a DDS (Microsoft DirectDraw Surface) file with a 3D
texture in it, typically uncompressed. This is usually loaded directly
into the game engine.

These are both a bit ad-hoc and not really standardized, so I don't know
if they are relevant for OCS. If they are, let me know and I can try to
work up something more like an actual spec for each of these.

Thanks,

Naty

Hello!

I'm at the stage where I would like to get a survey of lut file
formats (1d, 3d, matrix, etc) that folks actually use in the wild.

If you commonly use a lut format at your facility, or define one in
your software package, I would hugely appreciate it if you could
upload example files and/or spec documents so I can begin work on the
importers.

Even if it's a proprietary format, I'd love to take a peek so I can
get a sense of the scope of formats out in the wild. (I need to make
sure the internal API is rich enough to encompass current use cases).

Many formats have been mentioned previously, including:
- Truelight ASCII .cub 3D lut
- Assimilate Scratch (Arri /Kodak .3dl)
- Iridas Framecycler (Iridas .cube)
- Autodesk (MESH3D, .lut, etc.)

For the majority of these, I do not have either example files or
specifications. Please help! :)

Also, does anyone know if the majority of lut formats identifiable by
their extension? Are there common extension conflicts? Ideally, I'd
like to try and have format reader registered based on file extension,
and only if that fails give each lut loader a chance to read it.
(Similar to how reader plugins work in nuke).

(Note that I'm not assuming 1 file == 1 lut. (We will support readers
where 1 file can generate multiple transforms, such as a 1d shaper lut
-> 3d lut -> 1d shaper lut.))

Oh, and to clarify,

Is your fear related to the specific include of the TR1 shared_ptr, or
to shared_ptrs in public headers in general?

-- Jeremy

looking much nicer. However, I have The Fear as you are using C++ TR1
for smart pointer magic.

The use of tr1 shared_ptr is only a stub.  When we get multi-platform
stuff sorted out that section will definitely be replaced with
platform / compiler specific #ifdefs. Any shared_ptr that provides a
dynamic_cast will suffice. (tr1 does work on osx already though  :) )

First a justification of shared_ptrs in general...
We've spend a long time thinking about object ownership and object
lifetimes, and the choice to expose a smart pointer was not arrived at
lightly.  I do not see a graceful alternative.  If people are
interested we should probably discuss this further to bring everyone
on board.  (Or to prove me wrong!)  The quick summary is that in the
context of multi-threaded apps, where there exists a 'global' config
people can get/set, you need to have some form of reference counting
to assure that configs aren't destroyed while still in use.   I would
have ended up just re inventing a shared_ptr / intrusive_ptr, so it
seemed expedient to just use a real one.  It will also allow for a
more simper python use model, as objects can now be created on either
side of the fence (C++ or python) and passed back and forth without
concern for object lifetime.  (Keeping a reference to the python
object will keep the C++ object alive, which is very desirable). This
will particularly be useful in python UIs that make use of the mutable
API. (To create and edit configs 'live')

One related thing of note - all exposed objects which use shared-ptrs
have private constructors; the only way to create them is with static
factory functions.   Example: ColorSpaceRcPtr ColorSpace::Create() .
This is done so that the shared pointer is always created with a
custom object deallocator.  Our hope is this will work solve the
windows dll memory management issue.  (And we hope to verify this
soon).

OCS::ImageDesc
The init functions could definitely be constructors. My concerns is
that it would then rely on type signatures to maintain uniqueness, and
this has hosed me in the past. (Particularly where default values /
int / bools are involved).  You can accidentally create a change which
is binary compatible, but not source compatible.

I really like having super explicit names such as initRGBA,
initSingleBuffer, initMultiBuffer.  This will also let us add new
convenience constructors should they be desired without worrying about
compatibility.   (Consider 'initRGB', which would have the same type
signature as initRGBA.

But I agree the current init functions aren't ideal.  Does anyone have ideas?


Soon enough we will have a live share of the GIT repo, and which point
everyone can start contributing... ;)

-- Jeremy


On Fri, Jun 4, 2010 at 4:09 AM, Bruno Nicoletti
<bruno.j....@googlemail.com> wrote:

Hi Jeremy,

looking much nicer. However, I have The Fear as you are using C++ TR1
for smart pointer magic. We are on Visual Studio 2005 for Windows and
there is no TR1 for VS5 (we won't be changing in the near future, long
story). A #ifndef around SharedPtr and DynamicPtrCast would help, as
we could redirect this to boost.

Another minor quibble, should not the OCS::ImageDesc have ctors rather
than the initRGBA/initSingleBuffer/initMultiBuffer methods? Similarly
for other classes?

b

On 4 June 2010 00:44, Jeremy Selan <jeremy...@gmail.com> wrote:

This one will actually build on other people's machines...

please see INSTALL file for instructions on how to build the Nuke
example (posted here as well):

Make a build directory, cd into it:

mkdir -p build
cd build

Configure cmake, pointing at your nuke include dir:

cmake -D NUKE:FILEPATH=/usr/local/nuke6.0.3/include/ -D CMAKE_BUILD_TYPE=Release ../

Build all targets:

make

You will now see the built .so files:

ls

Set the environment variable pointing at which configuration to use:
(Point this to your install)

setenv OCS /net/homedirs/jeremys/git/Color/configs/spivfx/config.ocs

Launch nuke (the same version you used for the include files), then
load the plugin (this can also go in your init.py)

nuke.load('/net/homedirs/jeremys/git/Color//build/libNukeColorSpaceConversion.so')
nuke.menu('Nodes').addMenu('Color').addCommand('OCSColorSpaceConversion', 'nuke.createNode("OCSColorSpaceConversion")')

Connect an image to the OCSColorSpaceConversion node.
Our 'spivfx' config has only a few example colorspaces, this one
demonstrates a conversion between scene linear and a film-log
colorspace.

--
Bruno Nicoletti

looking much nicer. However, I have The Fear as you are using C++ TR1
for smart pointer magic.

This one will actually build on other people's machines...

please see INSTALL file for instructions on how to build the Nuke
example (posted here as well):

Make a build directory, cd into it:

mkdir -p build
cd build

Configure cmake, pointing at your nuke include dir:

cmake -D NUKE:FILEPATH=/usr/local/nuke6.0.3/include/ -D CMAKE_BUILD_TYPE=Release ../

Build all targets:

make

You will now see the built .so files:

ls

Set the environment variable pointing at which configuration to use:
(Point this to your install)

setenv OCS /net/homedirs/jeremys/git/Color/configs/spivfx/config.ocs

Launch nuke (the same version you used for the include files), then
load the plugin (this can also go in your init.py)

nuke.load('/net/homedirs/jeremys/git/Color//build/libNukeColorSpaceConversion.so')
nuke.menu('Nodes').addMenu('Color').addCommand('OCSColorSpaceConversion', 'nuke.createNode("OCSColorSpaceConversion")')

Connect an image to the OCSColorSpaceConversion node.
Our 'spivfx' config has only a few example colorspaces, this one
demonstrates a conversion between scene linear and a film-log
colorspace.

Cool, thanks for the info.   So it sounds like when we want to write a .3dl
exporter, we'll need to specify which flavor of .3dl to write.  However, on
the import side it still sounds like we'll be able to get away with a
generalized import, presuming we're a bit clever with the bit-depth
interpretation.  Would you agree?

-- Jeremy

On Wed, May 26, 2010 at 7:08 PM, bsloan <bsl...@gmail.com> wrote:

Regarding 3dl --

The lore I've been fed is that Mitch Bogdonowicz formerly of Kodak
wrote (invented? burped?) the 3dl "specification" when he was at Kodak
as a quick and dirty solution of 3D LUT ASCII serialization.

Various interpretations have since been adopted (supported?)  by
Arri,  Autodesk, Assimilate and others.

Yes, the initial 1D monochrome shaper array is in some cases in a
different bit depth from the 3D rgb array. What's more, various
applications choke if the header comment fields (those preceded with
a"#") are not 'just so'  (eg. truelight's tl-utils).

I guess the point is that there isn't just one spec for it. Maybe the
way to deal with 3dl is to treat different interpretations as
different formats:

*arri.3dl
*autodesk.3dl
*scratch.3dl

ugly -r- us.

:)

-blake