Date   

Re: OCIO CUDA

Paul Miller <pa...@...>
 

If we've processed a whole frame on the GPU, it would kill our
performance to copy it back to the CPU to run a colorspace transform.
Copying to an OpenGL texture to use the GLSL path would also hurt
performance, though not as badly. So a CUDA path would be very useful to
me.
Can't you just bind your target framebuffer as a texture? No copy required and everything stays on the GPU.


Re: OCIO CUDA

Dithermaster <dither...@...>
 

There isn't any way to share data between OpenCL and CUDA
Not entirely true. If you base them both on an OpenGL context, you can create OpenCL images from OpenGL textures, and you can access OpenGL textures from CUDA as well.

///d@


On Fri, Mar 30, 2012 at 2:58 PM, Nathan Weston <elb...@...> wrote:
On 03/30/2012 03:02 PM, Jeremy Selan wrote:
Excellent, looking forward to seeing what you do.

You mention CUDA (historically) doesn't support dynamic compilation of
kernels?  That would imply an implementation that looks more like a
fixed function processing path, unfortunately.  The downside being that
the results would be even less accurate (potentially) then either our
current CPU or GPU pathways.

Recall that in OCIO, all of the color transforms are dynamically loaded
at runtime, so at library compile-time there's no way to know processing
will be required for a given color transform.  (You roughly know the
building blocks, but not how they will be chained together).

Hmm, I didn't know that. I don't yet know anything at all about OCIO under the hood. If there's any documentation of this kind of stuff it would be very helpful.

When you say the color transforms are loaded at runtime, are you talking about additional code (e.g. a .so that implements a particular transform)? If it's simply reading LUTs or other datafiles then a CUDA implementation might be able to work off the same data. CUDA supports most of C++ on the device side so it's quite a bit more flexible than GLSL.


Even our current GLSL codepath, which leverages a single 3dlut, tries to
do as much as possible in the fragment shader at runtime.  (It's
pipeline looks like [GLSL CODE + 3DLUT + GLSL CODE], with as much done
in code as possible).

What type of CUDA application are you writing? Are you looking for OCIO
in CUDA for performance reasons?  Are you looking for OCIO to match the
quality of the GPU?  Perhaps we can come up with an alternate
implementation approach, or decide that it's better to just target
recent CUDA versions.

I have to be slightly cagey about this for the moment since I work on a commercial product and we haven't yet announced our plans regarding OCIO. But broadly, we have an image processing application which has both CPU and CUDA codepaths depending on the available hardware. All of this is already implemented, so we're pretty much wedded to CUDA for the time being.

If we've processed a whole frame on the GPU, it would kill our performance to copy it back to the CPU to run a colorspace transform. Copying to an OpenGL texture to use the GLSL path would also hurt performance, though not as badly. So a CUDA path would be very useful to me.


On first glance, it appears that OpenCL may support dynamic compilation,
and thus be easier to match the CPU 1:1.  Can anyone with OpenCL
experience chime in?

clCreateProgramWithSource(...), etc.

I haven't done any OpenCL programming, but have paid close attention to it over the years. It does support dynamic compilation. As Larry mentioned, it's somewhat lacking in language features compared to CUDA (e.g. templates and other C++ stuff).

There isn't any way to share data between OpenCL and CUDA, so an OpenCL path wouldn't be of much use to me. Which, as you can imagine, somewhat reduces my motivation to write one. :)

On Thu, Mar 29, 2012 at 11:11 AM, Nathan Weston <elb...@...
<mailto:elb...@...>> wrote:

   Cool. If all goes well, I can hopefully find time to work on this
   over the next couple of months.

   CUDA historically hasn't supported dynamic generation/compilation of
   kernels. I believe it's possible with newer versions of the
   compiler, but only with the lower-level driver API. A
   statically-compiled kernel is probably a better bet, which would
   tend to point toward a more analytical approach along the lines of
   your CPU codepath.

   I've had pretty good luck in the past sharing code between C++ and
   CUDA in order to implement parallel code paths that produce the same
   results. The only snags are
     1) the shared code has to go into header files
     2) Virtual functions require CUDA 4.0 and Fermi hardware

   If possible I'd like to support older cards and CUDA toolkits, which
   means no virtual functions.

   Is there any documentation of the OCIO internals to help me get my
   bearings?


   On Wednesday, March 28, 2012 6:47:58 PM UTC-4, Jeremy Selan wrote:

       I don't think anyone has looked implementing a CUDA pathway, but
       I'm very open to such ideas. Someone did ask about an OpenCL
       implementation recently, but I believe it's still in the concept
       stage.

       A few thoughts on the concept...

       Our current GPU implementation does not attempt to match the CPU
       implementation, by design.  The CPU codepath does the full
       analytical color operations per pixels, while the GPU GLSL/Cg
       implementation relies on a combination of analytical shader text
       code generation, along with a 3d lut sampling.   For color
       operations which can be done in simple shader text (such as math
       ops), these all happen in the glsl shader. But if the user
       references multiple 3d luts for example, it's all baked into a
       single 3d lut.

       I was always hoping that, if we ever implemented a CUDA or
       OpenCL pathway, it would be more akin to the GPU code path and
       do more analytically.  Im not sure if this is possible, but I
       think it's a nice ideal for a 'compute' context.

       Another nicety of the current implementation is that even though
       we support gpu(s), OpenColorIO doesnt actually link to libGL,
       etc.   The 'GPU API' conceptually only deals with POD types,
       returning the float * 3dlut, and the const char * shader text.

       My hope would be that, if possible, a CUDA / OpenCL wouldn't
       impose any new linking requirements on the core library, but
       would instead support new code paths using simple data types.

       -- Jeremy


       On Wed, Mar 28, 2012 at 12:56 PM, Nathan Weston
       <elb...@... <mailto:elb...@...>> wrote:

           I'm currently integrating OpenColorIO into an application
           that uses CUDA for GPU processing. In order to use OCIO's
           shader path, we'd need to copy our images over to OpenGL
           textures and back again. If OCIO had a CUDA path, it would
           be cleaner and faster.

           Has anyone looked into implementing such a thing? If I were
           to implement it myself, is there any interest in including
           it in OCIO?





Re: OCIO CUDA

Nathan Weston <elb...@...>
 

That was my recollection as well, but I can't find any documentation or examples of this. Maybe I'm missing something. Do you remember any details?

All I can turn up are some 3rd-party tools which run the nvcc compiler as a subprocesses. nvcc is part of the CUDA SDK, but I don't think it's shipped with the driver or runtime libraries that would normally be installed on end-user machines, so this kind of approach could get complicated when it comes to deployment.

On 03/30/2012 03:11 PM, Larry Gritz wrote:
IIRC, although it's not quite as easy as OpenCL, these days Cuda can
dynamically compile kernels. (OpenCL was able to do that all along.)

The other advantage of Cuda is that it's really C++ with a couple minor
additions, which may make porting our existing code a lot easier, and
also use all your favorite C++ features such as classes and templates.
OpenCL is its own thing (though very C like).

The disadvantage is, of course, less HW and vendor independence.


On Mar 30, 2012, at 12:02 PM, Jeremy Selan wrote:

Excellent, looking forward to seeing what you do.

You mention CUDA (historically) doesn't support dynamic compilation of
kernels? That would imply an implementation that looks more like a
fixed function processing path, unfortunately. The downside being that
the results would be even less accurate (potentially) then either our
current CPU or GPU pathways.

Recall that in OCIO, all of the color transforms are dynamically
loaded at runtime, so at library compile-time there's no way to know
processing will be required for a given color transform. (You roughly
know the building blocks, but not how they will be chained together).

Even our current GLSL codepath, which leverages a single 3dlut, tries
to do as much as possible in the fragment shader at runtime. (It's
pipeline looks like [GLSL CODE + 3DLUT + GLSL CODE], with as much done
in code as possible).

What type of CUDA application are you writing? Are you looking for
OCIO in CUDA for performance reasons? Are you looking for OCIO to
match the quality of the GPU? Perhaps we can come up with an alternate
implementation approach, or decide that it's better to just target
recent CUDA versions.

On first glance, it appears that OpenCL may support dynamic
compilation, and thus be easier to match the CPU 1:1. Can anyone with
OpenCL experience chime in?

clCreateProgramWithSource(...), etc.

-- Jeremy

On Thu, Mar 29, 2012 at 11:11 AM, Nathan Weston <elb...@...
<mailto:elb...@...>> wrote:

Cool. If all goes well, I can hopefully find time to work on this
over the next couple of months.

CUDA historically hasn't supported dynamic generation/compilation
of kernels. I believe it's possible with newer versions of the
compiler, but only with the lower-level driver API. A
statically-compiled kernel is probably a better bet, which would
tend to point toward a more analytical approach along the lines of
your CPU codepath.

I've had pretty good luck in the past sharing code between C++ and
CUDA in order to implement parallel code paths that produce the
same results. The only snags are
1) the shared code has to go into header files
2) Virtual functions require CUDA 4.0 and Fermi hardware

If possible I'd like to support older cards and CUDA toolkits,
which means no virtual functions.

Is there any documentation of the OCIO internals to help me get my
bearings?


On Wednesday, March 28, 2012 6:47:58 PM UTC-4, Jeremy Selan wrote:

I don't think anyone has looked implementing a CUDA pathway,
but I'm very open to such ideas. Someone did ask about an
OpenCL implementation recently, but I believe it's still in
the concept stage.

A few thoughts on the concept...

Our current GPU implementation does not attempt to match the
CPU implementation, by design. The CPU codepath does the full
analytical color operations per pixels, while the GPU GLSL/Cg
implementation relies on a combination of analytical shader
text code generation, along with a 3d lut sampling. For color
operations which can be done in simple shader text (such as
math ops), these all happen in the glsl shader. But if the
user references multiple 3d luts for example, it's all baked
into a single 3d lut.

I was always hoping that, if we ever implemented a CUDA or
OpenCL pathway, it would be more akin to the GPU code path and
do more analytically. Im not sure if this is possible, but I
think it's a nice ideal for a 'compute' context.

Another nicety of the current implementation is that even
though we support gpu(s), OpenColorIO doesnt actually link to
libGL, etc. The 'GPU API' conceptually only deals with POD
types, returning the float * 3dlut, and the const char *
shader text.

My hope would be that, if possible, a CUDA / OpenCL wouldn't
impose any new linking requirements on the core library, but
would instead support new code paths using simple data types.

-- Jeremy


On Wed, Mar 28, 2012 at 12:56 PM, Nathan Weston
<elb...@... <mailto:elb...@...>> wrote:

I'm currently integrating OpenColorIO into an application
that uses CUDA for GPU processing. In order to use OCIO's
shader path, we'd need to copy our images over to OpenGL
textures and back again. If OCIO had a CUDA path, it would
be cleaner and faster.

Has anyone looked into implementing such a thing? If I
were to implement it myself, is there any interest in
including it in OCIO?


--
Larry Gritz
l...@... <mailto:l...@...>



Re: OCIO CUDA

Nathan Weston <elb...@...>
 

On 03/30/2012 03:02 PM, Jeremy Selan wrote:
Excellent, looking forward to seeing what you do.

You mention CUDA (historically) doesn't support dynamic compilation of
kernels? That would imply an implementation that looks more like a
fixed function processing path, unfortunately. The downside being that
the results would be even less accurate (potentially) then either our
current CPU or GPU pathways.

Recall that in OCIO, all of the color transforms are dynamically loaded
at runtime, so at library compile-time there's no way to know processing
will be required for a given color transform. (You roughly know the
building blocks, but not how they will be chained together).
Hmm, I didn't know that. I don't yet know anything at all about OCIO under the hood. If there's any documentation of this kind of stuff it would be very helpful.

When you say the color transforms are loaded at runtime, are you talking about additional code (e.g. a .so that implements a particular transform)? If it's simply reading LUTs or other datafiles then a CUDA implementation might be able to work off the same data. CUDA supports most of C++ on the device side so it's quite a bit more flexible than GLSL.

Even our current GLSL codepath, which leverages a single 3dlut, tries to
do as much as possible in the fragment shader at runtime. (It's
pipeline looks like [GLSL CODE + 3DLUT + GLSL CODE], with as much done
in code as possible).

What type of CUDA application are you writing? Are you looking for OCIO
in CUDA for performance reasons? Are you looking for OCIO to match the
quality of the GPU? Perhaps we can come up with an alternate
implementation approach, or decide that it's better to just target
recent CUDA versions.
I have to be slightly cagey about this for the moment since I work on a commercial product and we haven't yet announced our plans regarding OCIO. But broadly, we have an image processing application which has both CPU and CUDA codepaths depending on the available hardware. All of this is already implemented, so we're pretty much wedded to CUDA for the time being.

If we've processed a whole frame on the GPU, it would kill our performance to copy it back to the CPU to run a colorspace transform. Copying to an OpenGL texture to use the GLSL path would also hurt performance, though not as badly. So a CUDA path would be very useful to me.

On first glance, it appears that OpenCL may support dynamic compilation,
and thus be easier to match the CPU 1:1. Can anyone with OpenCL
experience chime in?

clCreateProgramWithSource(...), etc.
I haven't done any OpenCL programming, but have paid close attention to it over the years. It does support dynamic compilation. As Larry mentioned, it's somewhat lacking in language features compared to CUDA (e.g. templates and other C++ stuff).

There isn't any way to share data between OpenCL and CUDA, so an OpenCL path wouldn't be of much use to me. Which, as you can imagine, somewhat reduces my motivation to write one. :)

On Thu, Mar 29, 2012 at 11:11 AM, Nathan Weston <elb...@...
<mailto:elb...@...>> wrote:

Cool. If all goes well, I can hopefully find time to work on this
over the next couple of months.

CUDA historically hasn't supported dynamic generation/compilation of
kernels. I believe it's possible with newer versions of the
compiler, but only with the lower-level driver API. A
statically-compiled kernel is probably a better bet, which would
tend to point toward a more analytical approach along the lines of
your CPU codepath.

I've had pretty good luck in the past sharing code between C++ and
CUDA in order to implement parallel code paths that produce the same
results. The only snags are
1) the shared code has to go into header files
2) Virtual functions require CUDA 4.0 and Fermi hardware

If possible I'd like to support older cards and CUDA toolkits, which
means no virtual functions.

Is there any documentation of the OCIO internals to help me get my
bearings?


On Wednesday, March 28, 2012 6:47:58 PM UTC-4, Jeremy Selan wrote:

I don't think anyone has looked implementing a CUDA pathway, but
I'm very open to such ideas. Someone did ask about an OpenCL
implementation recently, but I believe it's still in the concept
stage.

A few thoughts on the concept...

Our current GPU implementation does not attempt to match the CPU
implementation, by design. The CPU codepath does the full
analytical color operations per pixels, while the GPU GLSL/Cg
implementation relies on a combination of analytical shader text
code generation, along with a 3d lut sampling. For color
operations which can be done in simple shader text (such as math
ops), these all happen in the glsl shader. But if the user
references multiple 3d luts for example, it's all baked into a
single 3d lut.

I was always hoping that, if we ever implemented a CUDA or
OpenCL pathway, it would be more akin to the GPU code path and
do more analytically. Im not sure if this is possible, but I
think it's a nice ideal for a 'compute' context.

Another nicety of the current implementation is that even though
we support gpu(s), OpenColorIO doesnt actually link to libGL,
etc. The 'GPU API' conceptually only deals with POD types,
returning the float * 3dlut, and the const char * shader text.

My hope would be that, if possible, a CUDA / OpenCL wouldn't
impose any new linking requirements on the core library, but
would instead support new code paths using simple data types.

-- Jeremy


On Wed, Mar 28, 2012 at 12:56 PM, Nathan Weston
<elb...@... <mailto:elb...@...>> wrote:

I'm currently integrating OpenColorIO into an application
that uses CUDA for GPU processing. In order to use OCIO's
shader path, we'd need to copy our images over to OpenGL
textures and back again. If OCIO had a CUDA path, it would
be cleaner and faster.

Has anyone looked into implementing such a thing? If I were
to implement it myself, is there any interest in including
it in OCIO?



Re: OCIO CUDA

Larry Gritz <l...@...>
 

IIRC, although it's not quite as easy as OpenCL, these days Cuda can dynamically compile kernels.  (OpenCL was able to do that all along.)

The other advantage of Cuda is that it's really C++ with a couple minor additions, which may make porting our existing code a lot easier, and also use all your favorite C++ features such as classes and templates.  OpenCL is its own thing (though very C like).

The disadvantage is, of course, less HW and vendor independence.


On Mar 30, 2012, at 12:02 PM, Jeremy Selan wrote:

Excellent, looking forward to seeing what you do.

You mention CUDA (historically) doesn't support dynamic compilation of kernels?  That would imply an implementation that looks more like a fixed function processing path, unfortunately.  The downside being that the results would be even less accurate (potentially) then either our current CPU or GPU pathways.

Recall that in OCIO, all of the color transforms are dynamically loaded at runtime, so at library compile-time there's no way to know processing will be required for a given color transform.  (You roughly know the building blocks, but not how they will be chained together).

Even our current GLSL codepath, which leverages a single 3dlut, tries to do as much as possible in the fragment shader at runtime.  (It's pipeline looks like [GLSL CODE + 3DLUT + GLSL CODE], with as much done in code as possible).

What type of CUDA application are you writing? Are you looking for OCIO in CUDA for performance reasons?  Are you looking for OCIO to match the quality of the GPU?  Perhaps we can come up with an alternate implementation approach, or decide that it's better to just target recent CUDA versions.

On first glance, it appears that OpenCL may support dynamic compilation, and thus be easier to match the CPU 1:1.  Can anyone with OpenCL experience chime in?

clCreateProgramWithSource(...), etc.

-- Jeremy

On Thu, Mar 29, 2012 at 11:11 AM, Nathan Weston <elb...@...> wrote:
Cool. If all goes well, I can hopefully find time to work on this over the next couple of months.

CUDA historically hasn't supported dynamic generation/compilation of kernels. I believe it's possible with newer versions of the compiler, but only with the lower-level driver API. A statically-compiled kernel is probably a better bet, which would tend to point toward a more analytical approach along the lines of your CPU codepath.

I've had pretty good luck in the past sharing code between C++ and CUDA in order to implement parallel code paths that produce the same results. The only snags are
 1) the shared code has to go into header files
 2) Virtual functions require CUDA 4.0 and Fermi hardware

If possible I'd like to support older cards and CUDA toolkits, which means no virtual functions.

Is there any documentation of the OCIO internals to help me get my bearings?


On Wednesday, March 28, 2012 6:47:58 PM UTC-4, Jeremy Selan wrote:
I don't think anyone has looked implementing a CUDA pathway, but I'm very open to such ideas. Someone did ask about an OpenCL implementation recently, but I believe it's still in the concept stage.

A few thoughts on the concept...

Our current GPU implementation does not attempt to match the CPU implementation, by design.  The CPU codepath does the full analytical color operations per pixels, while the GPU GLSL/Cg implementation relies on a combination of analytical shader text code generation, along with a 3d lut sampling.   For color operations which can be done in simple shader text (such as math ops), these all happen in the glsl shader. But if the user references multiple 3d luts for example, it's all baked into a single 3d lut.

I was always hoping that, if we ever implemented a CUDA or OpenCL pathway, it would be more akin to the GPU code path and do more analytically.  Im not sure if this is possible, but I think it's a nice ideal for a 'compute' context.

Another nicety of the current implementation is that even though we support gpu(s), OpenColorIO doesnt actually link to libGL, etc.   The 'GPU API' conceptually only deals with POD types, returning the float * 3dlut, and the const char * shader text.

My hope would be that, if possible, a CUDA / OpenCL wouldn't impose any new linking requirements on the core library, but would instead support new code paths using simple data types.

-- Jeremy


On Wed, Mar 28, 2012 at 12:56 PM, Nathan Weston <elb...@...> wrote:
I'm currently integrating OpenColorIO into an application that uses CUDA for GPU processing. In order to use OCIO's shader path, we'd need to copy our images over to OpenGL textures and back again. If OCIO had a CUDA path, it would be cleaner and faster.

Has anyone looked into implementing such a thing? If I were to implement it myself, is there any interest in including it in OCIO?



--
Larry Gritz




Re: OCIO CUDA

Jeremy Selan <jeremy...@...>
 

Excellent, looking forward to seeing what you do.

You mention CUDA (historically) doesn't support dynamic compilation of kernels?  That would imply an implementation that looks more like a fixed function processing path, unfortunately.  The downside being that the results would be even less accurate (potentially) then either our current CPU or GPU pathways.

Recall that in OCIO, all of the color transforms are dynamically loaded at runtime, so at library compile-time there's no way to know processing will be required for a given color transform.  (You roughly know the building blocks, but not how they will be chained together).

Even our current GLSL codepath, which leverages a single 3dlut, tries to do as much as possible in the fragment shader at runtime.  (It's pipeline looks like [GLSL CODE + 3DLUT + GLSL CODE], with as much done in code as possible).

What type of CUDA application are you writing? Are you looking for OCIO in CUDA for performance reasons?  Are you looking for OCIO to match the quality of the GPU?  Perhaps we can come up with an alternate implementation approach, or decide that it's better to just target recent CUDA versions.

On first glance, it appears that OpenCL may support dynamic compilation, and thus be easier to match the CPU 1:1.  Can anyone with OpenCL experience chime in?

clCreateProgramWithSource(...), etc.

-- Jeremy

On Thu, Mar 29, 2012 at 11:11 AM, Nathan Weston <elb...@...> wrote:
Cool. If all goes well, I can hopefully find time to work on this over the next couple of months.

CUDA historically hasn't supported dynamic generation/compilation of kernels. I believe it's possible with newer versions of the compiler, but only with the lower-level driver API. A statically-compiled kernel is probably a better bet, which would tend to point toward a more analytical approach along the lines of your CPU codepath.

I've had pretty good luck in the past sharing code between C++ and CUDA in order to implement parallel code paths that produce the same results. The only snags are
 1) the shared code has to go into header files
 2) Virtual functions require CUDA 4.0 and Fermi hardware

If possible I'd like to support older cards and CUDA toolkits, which means no virtual functions.

Is there any documentation of the OCIO internals to help me get my bearings?


On Wednesday, March 28, 2012 6:47:58 PM UTC-4, Jeremy Selan wrote:
I don't think anyone has looked implementing a CUDA pathway, but I'm very open to such ideas. Someone did ask about an OpenCL implementation recently, but I believe it's still in the concept stage.

A few thoughts on the concept...

Our current GPU implementation does not attempt to match the CPU implementation, by design.  The CPU codepath does the full analytical color operations per pixels, while the GPU GLSL/Cg implementation relies on a combination of analytical shader text code generation, along with a 3d lut sampling.   For color operations which can be done in simple shader text (such as math ops), these all happen in the glsl shader. But if the user references multiple 3d luts for example, it's all baked into a single 3d lut.

I was always hoping that, if we ever implemented a CUDA or OpenCL pathway, it would be more akin to the GPU code path and do more analytically.  Im not sure if this is possible, but I think it's a nice ideal for a 'compute' context.

Another nicety of the current implementation is that even though we support gpu(s), OpenColorIO doesnt actually link to libGL, etc.   The 'GPU API' conceptually only deals with POD types, returning the float * 3dlut, and the const char * shader text.

My hope would be that, if possible, a CUDA / OpenCL wouldn't impose any new linking requirements on the core library, but would instead support new code paths using simple data types.

-- Jeremy


On Wed, Mar 28, 2012 at 12:56 PM, Nathan Weston <elb...@...> wrote:
I'm currently integrating OpenColorIO into an application that uses CUDA for GPU processing. In order to use OCIO's shader path, we'd need to copy our images over to OpenGL textures and back again. If OCIO had a CUDA path, it would be cleaner and faster.

Has anyone looked into implementing such a thing? If I were to implement it myself, is there any interest in including it in OCIO?



review: pyglue install location and link flag defaults

dbr/Ben <dbr....@...>
 


Re: OCIO CUDA

Nathan Weston <elb...@...>
 

Cool. If all goes well, I can hopefully find time to work on this over the next couple of months.

CUDA historically hasn't supported dynamic generation/compilation of kernels. I believe it's possible with newer versions of the compiler, but only with the lower-level driver API. A statically-compiled kernel is probably a better bet, which would tend to point toward a more analytical approach along the lines of your CPU codepath.

I've had pretty good luck in the past sharing code between C++ and CUDA in order to implement parallel code paths that produce the same results. The only snags are
 1) the shared code has to go into header files
 2) Virtual functions require CUDA 4.0 and Fermi hardware

If possible I'd like to support older cards and CUDA toolkits, which means no virtual functions.

Is there any documentation of the OCIO internals to help me get my bearings?


On Wednesday, March 28, 2012 6:47:58 PM UTC-4, Jeremy Selan wrote:
I don't think anyone has looked implementing a CUDA pathway, but I'm very open to such ideas. Someone did ask about an OpenCL implementation recently, but I believe it's still in the concept stage.

A few thoughts on the concept...

Our current GPU implementation does not attempt to match the CPU implementation, by design.  The CPU codepath does the full analytical color operations per pixels, while the GPU GLSL/Cg implementation relies on a combination of analytical shader text code generation, along with a 3d lut sampling.   For color operations which can be done in simple shader text (such as math ops), these all happen in the glsl shader. But if the user references multiple 3d luts for example, it's all baked into a single 3d lut.

I was always hoping that, if we ever implemented a CUDA or OpenCL pathway, it would be more akin to the GPU code path and do more analytically.  Im not sure if this is possible, but I think it's a nice ideal for a 'compute' context.

Another nicety of the current implementation is that even though we support gpu(s), OpenColorIO doesnt actually link to libGL, etc.   The 'GPU API' conceptually only deals with POD types, returning the float * 3dlut, and the const char * shader text.

My hope would be that, if possible, a CUDA / OpenCL wouldn't impose any new linking requirements on the core library, but would instead support new code paths using simple data types.

-- Jeremy


On Wed, Mar 28, 2012 at 12:56 PM, Nathan Weston <elb...@...> wrote:
I'm currently integrating OpenColorIO into an application that uses CUDA for GPU processing. In order to use OCIO's shader path, we'd need to copy our images over to OpenGL textures and back again. If OCIO had a CUDA path, it would be cleaner and faster.

Has anyone looked into implementing such a thing? If I were to implement it myself, is there any interest in including it in OCIO?


Re: OCIO CUDA

Jeremy Selan <jeremy...@...>
 

I don't think anyone has looked implementing a CUDA pathway, but I'm very open to such ideas. Someone did ask about an OpenCL implementation recently, but I believe it's still in the concept stage.

A few thoughts on the concept...

Our current GPU implementation does not attempt to match the CPU implementation, by design.  The CPU codepath does the full analytical color operations per pixels, while the GPU GLSL/Cg implementation relies on a combination of analytical shader text code generation, along with a 3d lut sampling.   For color operations which can be done in simple shader text (such as math ops), these all happen in the glsl shader. But if the user references multiple 3d luts for example, it's all baked into a single 3d lut.

I was always hoping that, if we ever implemented a CUDA or OpenCL pathway, it would be more akin to the GPU code path and do more analytically.  Im not sure if this is possible, but I think it's a nice ideal for a 'compute' context.

Another nicety of the current implementation is that even though we support gpu(s), OpenColorIO doesnt actually link to libGL, etc.   The 'GPU API' conceptually only deals with POD types, returning the float * 3dlut, and the const char * shader text.

My hope would be that, if possible, a CUDA / OpenCL wouldn't impose any new linking requirements on the core library, but would instead support new code paths using simple data types.

-- Jeremy


On Wed, Mar 28, 2012 at 12:56 PM, Nathan Weston <elb...@...> wrote:

I'm currently integrating OpenColorIO into an application that uses CUDA for GPU processing. In order to use OCIO's shader path, we'd need to copy our images over to OpenGL textures and back again. If OCIO had a CUDA path, it would be cleaner and faster.

Has anyone looked into implementing such a thing? If I were to implement it myself, is there any interest in including it in OCIO?


OCIO CUDA

Nathan Weston <elb...@...>
 

I'm currently integrating OpenColorIO into an application that uses CUDA for GPU processing. In order to use OCIO's shader path, we'd need to copy our images over to OpenGL textures and back again. If OCIO had a CUDA path, it would be cleaner and faster.

Has anyone looked into implementing such a thing? If I were to implement it myself, is there any interest in including it in OCIO?


Review: pyglue: config.clearLooks() was incorrectly named clearLook()

Jeremy Selan <jeremy...@...>
 


Review: cccid can be specified as name or index

Jeremy Selan <jeremy...@...>
 


Re: ASC .ccc files

Malcolm Humphreys <malcolmh...@...>
 


On 26/03/2012, at 9:50 PM, Jeremy Selan wrote:

I started implementing the index option, and now I'm wondering why to require the '#' character at all?

How about just allowing plain integers as cccids?
cccid=0
cccid=1
etc.

In my current working implementation, I first see if the cccid string is defined in the file, and if so I obey it.  But if it's not a valid id, I see if it's a valid integer, and if so I load that index.  (Note that an empty cccid string always fails so let's exclude that case).

Any objections or downsides I'm not seeing?

Nope sounds fine with me.


-- Jeremy


On Mon, Mar 26, 2012 at 9:57 AM, Jeremy Selan <jeremy...@...> wrote:
Thanks, we'll go with #INT then.

https://github.com/imageworks/OpenColorIO/issues/244

-- Jeremy


On Sat, Mar 24, 2012 at 1:41 PM, Malcolm Humphreys <malcolmh...@...> wrote:

On 24/03/2012, at 5:37 PM, Jeremy Selan wrote:

Hm...  good points.

Ok - so let's drop the idea of using a default cccid.  Too dangerous.

But I'm starting to really like being able to also specify a <ColorCorrection> by index.   Such as cccid = 'idx:0' to pick the 0th cc, or even ccid = '#0'.  This way one is still explicit about which cc to use, but it also allows the use .ccc files where the internal cccids are omitted.

Good idea or bad idea?
Which syntax?  'idx:INT' or '#INT'?

#INT


-- Jeremy

On Sat, Mar 24, 2012 at 1:46 AM, Malcolm Humphreys <malcolmh...@...> wrote:
This does feel a bit dangerous to me where a .ccc file might be resolved in multiple locations. What happens when you have a global .ccc file and a few sequence / shot ones. It wouldn't be uncommon to easily fallback to the global .ccc which you might want to have multiple id for different grades.

Maybe some kind of config option to enable this and/or a default id to search for?

I would hope once we implement the plugin API I would hope that we could write a (reference) database transform to store most of this in. Leaving .ccc as a serialisation format of this data.

.malcolm

On 24/03/2012, at 2:53 AM, dbr/Ben wrote:

>> always load the first color correction, when an empty id
> If the .ccc only contains a single ColorCorrection, and no cccid is specified, using the first/only CC seems reasonable
>
> Alternatively, would being able to specify a numeric index help this? E.g cccid: "idx:0"
>
> On 24/03/2012, at 5:15, Jeremy Selan <jeremy...@...> wrote:
>
>> Right now, when you use OCIO FileTransform to point at a .ccc file (a color correction collection) you must also specify the id of which to use.
>>
>> Example .ccc file:
>> <ColorCorrectionCollection>
>>        <ColorCorrection id="ab101">
>>            <SOPNode ... />
>>            <SatNode ... />
>>        </ColorCorrection>
>>        <ColorCorrection id="ab102">
>>            <SOPNode ... />
>>            <SatNode ... />
>>        </ColorCorrection>
>> </ColorCorrectionCollection>
>>
>> The cccid is explicitly required because all .ccc files may contain multiple transforms.
>> This is why - at SPI - we invented the .cc format, which is just a <ColorCorrection> element from a .ccc file in isolation.
>>
>> But sometimes you're handed a .ccc file that contains only a single correction, and it's annoying that ocio cant handle it without the user picking the right cccid.  And, sometimes the "id" is omitted!
>>
>> I'd like to solve this.
>>
>> What would people think, of allowing ocio to always load the first color correction, when an empty id is specified? Too scary? Too error prone?
>>
>> I'm not sure what the right answer is, but I think we're too strict in loading 'ccc' files, and I'm interested in options...
>>
>> -- Jeremy







Re: ASC .ccc files

Jeremy Selan <jeremy...@...>
 

I started implementing the index option, and now I'm wondering why to require the '#' character at all?

How about just allowing plain integers as cccids?
cccid=0
cccid=1
etc.

In my current working implementation, I first see if the cccid string is defined in the file, and if so I obey it.  But if it's not a valid id, I see if it's a valid integer, and if so I load that index.  (Note that an empty cccid string always fails so let's exclude that case).

Any objections or downsides I'm not seeing?

-- Jeremy


On Mon, Mar 26, 2012 at 9:57 AM, Jeremy Selan <jeremy...@...> wrote:
Thanks, we'll go with #INT then.

https://github.com/imageworks/OpenColorIO/issues/244

-- Jeremy


On Sat, Mar 24, 2012 at 1:41 PM, Malcolm Humphreys <malcolmh...@...> wrote:

On 24/03/2012, at 5:37 PM, Jeremy Selan wrote:

Hm...  good points.

Ok - so let's drop the idea of using a default cccid.  Too dangerous.

But I'm starting to really like being able to also specify a <ColorCorrection> by index.   Such as cccid = 'idx:0' to pick the 0th cc, or even ccid = '#0'.  This way one is still explicit about which cc to use, but it also allows the use .ccc files where the internal cccids are omitted.

Good idea or bad idea?
Which syntax?  'idx:INT' or '#INT'?

#INT


-- Jeremy

On Sat, Mar 24, 2012 at 1:46 AM, Malcolm Humphreys <malcolmh...@...> wrote:
This does feel a bit dangerous to me where a .ccc file might be resolved in multiple locations. What happens when you have a global .ccc file and a few sequence / shot ones. It wouldn't be uncommon to easily fallback to the global .ccc which you might want to have multiple id for different grades.

Maybe some kind of config option to enable this and/or a default id to search for?

I would hope once we implement the plugin API I would hope that we could write a (reference) database transform to store most of this in. Leaving .ccc as a serialisation format of this data.

.malcolm

On 24/03/2012, at 2:53 AM, dbr/Ben wrote:

>> always load the first color correction, when an empty id
> If the .ccc only contains a single ColorCorrection, and no cccid is specified, using the first/only CC seems reasonable
>
> Alternatively, would being able to specify a numeric index help this? E.g cccid: "idx:0"
>
> On 24/03/2012, at 5:15, Jeremy Selan <jeremy...@...> wrote:
>
>> Right now, when you use OCIO FileTransform to point at a .ccc file (a color correction collection) you must also specify the id of which to use.
>>
>> Example .ccc file:
>> <ColorCorrectionCollection>
>>        <ColorCorrection id="ab101">
>>            <SOPNode ... />
>>            <SatNode ... />
>>        </ColorCorrection>
>>        <ColorCorrection id="ab102">
>>            <SOPNode ... />
>>            <SatNode ... />
>>        </ColorCorrection>
>> </ColorCorrectionCollection>
>>
>> The cccid is explicitly required because all .ccc files may contain multiple transforms.
>> This is why - at SPI - we invented the .cc format, which is just a <ColorCorrection> element from a .ccc file in isolation.
>>
>> But sometimes you're handed a .ccc file that contains only a single correction, and it's annoying that ocio cant handle it without the user picking the right cccid.  And, sometimes the "id" is omitted!
>>
>> I'd like to solve this.
>>
>> What would people think, of allowing ocio to always load the first color correction, when an empty id is specified? Too scary? Too error prone?
>>
>> I'm not sure what the right answer is, but I think we're too strict in loading 'ccc' files, and I'm interested in options...
>>
>> -- Jeremy






Re: photoshop 6 beta?

Jeremy Selan <jeremy...@...>
 

Thanks for the details. That's great to know.   The forums you point at are fun reads too. ;)

I'm wondering if 3d lut adjustment layers are pulled internally into the .psd file, or are 'live' external references?  (or maybe both options exist, similar to icc profiles?)

It's neat that they got .look file support it there.  (And it makes sense given Adobe's acqusition of Iridas).  Probably a good ocio format to add in the near term.

I'll have to download the beta. I'm also interested in confirming our lut application matches theirs...

-- Jeremy


On Sat, Mar 24, 2012 at 8:30 AM, dbr/Ben <dbr....@...> wrote:
Installed it today. After like 20 minutes of looking around, found where the LUT's are loaded… You can add a "Color Lookup" adjustment layer:


..or "Image > Adjustments > Color Lookup" and bake it into a layer.

Couldn't see any way to apply it purely as a viewing LUT, which is a shame (adjustment layers make picking colours tedious, and you have to disable it before exporting), but.. I could very well be missing something.

Not sure about a full list of formats, but the "Adobe Photoshop CS6/Presets/3DLUTs/" directory contains a bunch of .3dl files (two variants, one with "MESH" line, one without), .look files (IRIDAS format), and a .cube file

Think this is a representative sampling of the included LUT's:


It can also apply "Device Link" and "Abstract" ICC profiles

About the only mention I could find of the 3D LUT support (aside from copying-and-pastes of the feature-list) was
In my searching, I also came across this amusing thread (well, "amusing" in a frown-inducting way)
Reminds me of the Photoshop vs OpenEXR alpha/premult debate (oh, it's the same Adobe employee, which makes it a bit less worrying :P)

Anyway, hardly OCIO integration, but seems like a step forward. I'll play about more with it later!
- Ben

On 24/03/2012, at 11:50 AM, Jeremy Selan wrote:

Has anyone tried the photoshop 6 beta yet?  One of the line items is '3dlut adjustment', and I was curious what this meant. Couldn't find the docs online (yet).  Maybe you can now select a 3dlut from disk, and bake it into an image? I'm curious what formats they support...

-- Jeremy



Re: ASC .ccc files

Jeremy Selan <jeremy...@...>
 

Thanks, we'll go with #INT then.

https://github.com/imageworks/OpenColorIO/issues/244

-- Jeremy


On Sat, Mar 24, 2012 at 1:41 PM, Malcolm Humphreys <malcolmh...@...> wrote:

On 24/03/2012, at 5:37 PM, Jeremy Selan wrote:

Hm...  good points.

Ok - so let's drop the idea of using a default cccid.  Too dangerous.

But I'm starting to really like being able to also specify a <ColorCorrection> by index.   Such as cccid = 'idx:0' to pick the 0th cc, or even ccid = '#0'.  This way one is still explicit about which cc to use, but it also allows the use .ccc files where the internal cccids are omitted.

Good idea or bad idea?
Which syntax?  'idx:INT' or '#INT'?

#INT


-- Jeremy

On Sat, Mar 24, 2012 at 1:46 AM, Malcolm Humphreys <malcolmh...@...> wrote:
This does feel a bit dangerous to me where a .ccc file might be resolved in multiple locations. What happens when you have a global .ccc file and a few sequence / shot ones. It wouldn't be uncommon to easily fallback to the global .ccc which you might want to have multiple id for different grades.

Maybe some kind of config option to enable this and/or a default id to search for?

I would hope once we implement the plugin API I would hope that we could write a (reference) database transform to store most of this in. Leaving .ccc as a serialisation format of this data.

.malcolm

On 24/03/2012, at 2:53 AM, dbr/Ben wrote:

>> always load the first color correction, when an empty id
> If the .ccc only contains a single ColorCorrection, and no cccid is specified, using the first/only CC seems reasonable
>
> Alternatively, would being able to specify a numeric index help this? E.g cccid: "idx:0"
>
> On 24/03/2012, at 5:15, Jeremy Selan <jeremy...@...> wrote:
>
>> Right now, when you use OCIO FileTransform to point at a .ccc file (a color correction collection) you must also specify the id of which to use.
>>
>> Example .ccc file:
>> <ColorCorrectionCollection>
>>        <ColorCorrection id="ab101">
>>            <SOPNode ... />
>>            <SatNode ... />
>>        </ColorCorrection>
>>        <ColorCorrection id="ab102">
>>            <SOPNode ... />
>>            <SatNode ... />
>>        </ColorCorrection>
>> </ColorCorrectionCollection>
>>
>> The cccid is explicitly required because all .ccc files may contain multiple transforms.
>> This is why - at SPI - we invented the .cc format, which is just a <ColorCorrection> element from a .ccc file in isolation.
>>
>> But sometimes you're handed a .ccc file that contains only a single correction, and it's annoying that ocio cant handle it without the user picking the right cccid.  And, sometimes the "id" is omitted!
>>
>> I'd like to solve this.
>>
>> What would people think, of allowing ocio to always load the first color correction, when an empty id is specified? Too scary? Too error prone?
>>
>> I'm not sure what the right answer is, but I think we're too strict in loading 'ccc' files, and I'm interested in options...
>>
>> -- Jeremy





Re: ASC .ccc files

Malcolm Humphreys <malcolmh...@...>
 


On 24/03/2012, at 5:37 PM, Jeremy Selan wrote:

Hm...  good points.

Ok - so let's drop the idea of using a default cccid.  Too dangerous.

But I'm starting to really like being able to also specify a <ColorCorrection> by index.   Such as cccid = 'idx:0' to pick the 0th cc, or even ccid = '#0'.  This way one is still explicit about which cc to use, but it also allows the use .ccc files where the internal cccids are omitted.

Good idea or bad idea?
Which syntax?  'idx:INT' or '#INT'?

#INT


-- Jeremy

On Sat, Mar 24, 2012 at 1:46 AM, Malcolm Humphreys <malcolmh...@...> wrote:
This does feel a bit dangerous to me where a .ccc file might be resolved in multiple locations. What happens when you have a global .ccc file and a few sequence / shot ones. It wouldn't be uncommon to easily fallback to the global .ccc which you might want to have multiple id for different grades.

Maybe some kind of config option to enable this and/or a default id to search for?

I would hope once we implement the plugin API I would hope that we could write a (reference) database transform to store most of this in. Leaving .ccc as a serialisation format of this data.

.malcolm

On 24/03/2012, at 2:53 AM, dbr/Ben wrote:

>> always load the first color correction, when an empty id
> If the .ccc only contains a single ColorCorrection, and no cccid is specified, using the first/only CC seems reasonable
>
> Alternatively, would being able to specify a numeric index help this? E.g cccid: "idx:0"
>
> On 24/03/2012, at 5:15, Jeremy Selan <jeremy...@...> wrote:
>
>> Right now, when you use OCIO FileTransform to point at a .ccc file (a color correction collection) you must also specify the id of which to use.
>>
>> Example .ccc file:
>> <ColorCorrectionCollection>
>>        <ColorCorrection id="ab101">
>>            <SOPNode ... />
>>            <SatNode ... />
>>        </ColorCorrection>
>>        <ColorCorrection id="ab102">
>>            <SOPNode ... />
>>            <SatNode ... />
>>        </ColorCorrection>
>> </ColorCorrectionCollection>
>>
>> The cccid is explicitly required because all .ccc files may contain multiple transforms.
>> This is why - at SPI - we invented the .cc format, which is just a <ColorCorrection> element from a .ccc file in isolation.
>>
>> But sometimes you're handed a .ccc file that contains only a single correction, and it's annoying that ocio cant handle it without the user picking the right cccid.  And, sometimes the "id" is omitted!
>>
>> I'd like to solve this.
>>
>> What would people think, of allowing ocio to always load the first color correction, when an empty id is specified? Too scary? Too error prone?
>>
>> I'm not sure what the right answer is, but I think we're too strict in loading 'ccc' files, and I'm interested in options...
>>
>> -- Jeremy




Re: ASC .ccc files

Jeremy Selan <jeremy...@...>
 

Hm...  good points.

Ok - so let's drop the idea of using a default cccid.  Too dangerous.

But I'm starting to really like being able to also specify a <ColorCorrection> by index.   Such as cccid = 'idx:0' to pick the 0th cc, or even ccid = '#0'.  This way one is still explicit about which cc to use, but it also allows the use .ccc files where the internal cccids are omitted.

Good idea or bad idea?
Which syntax?  'idx:INT' or '#INT'?

-- Jeremy

On Sat, Mar 24, 2012 at 1:46 AM, Malcolm Humphreys <malcolmh...@...> wrote:
This does feel a bit dangerous to me where a .ccc file might be resolved in multiple locations. What happens when you have a global .ccc file and a few sequence / shot ones. It wouldn't be uncommon to easily fallback to the global .ccc which you might want to have multiple id for different grades.

Maybe some kind of config option to enable this and/or a default id to search for?

I would hope once we implement the plugin API I would hope that we could write a (reference) database transform to store most of this in. Leaving .ccc as a serialisation format of this data.

.malcolm

On 24/03/2012, at 2:53 AM, dbr/Ben wrote:

>> always load the first color correction, when an empty id
> If the .ccc only contains a single ColorCorrection, and no cccid is specified, using the first/only CC seems reasonable
>
> Alternatively, would being able to specify a numeric index help this? E.g cccid: "idx:0"
>
> On 24/03/2012, at 5:15, Jeremy Selan <jeremy...@...> wrote:
>
>> Right now, when you use OCIO FileTransform to point at a .ccc file (a color correction collection) you must also specify the id of which to use.
>>
>> Example .ccc file:
>> <ColorCorrectionCollection>
>>        <ColorCorrection id="ab101">
>>            <SOPNode ... />
>>            <SatNode ... />
>>        </ColorCorrection>
>>        <ColorCorrection id="ab102">
>>            <SOPNode ... />
>>            <SatNode ... />
>>        </ColorCorrection>
>> </ColorCorrectionCollection>
>>
>> The cccid is explicitly required because all .ccc files may contain multiple transforms.
>> This is why - at SPI - we invented the .cc format, which is just a <ColorCorrection> element from a .ccc file in isolation.
>>
>> But sometimes you're handed a .ccc file that contains only a single correction, and it's annoying that ocio cant handle it without the user picking the right cccid.  And, sometimes the "id" is omitted!
>>
>> I'd like to solve this.
>>
>> What would people think, of allowing ocio to always load the first color correction, when an empty id is specified? Too scary? Too error prone?
>>
>> I'm not sure what the right answer is, but I think we're too strict in loading 'ccc' files, and I'm interested in options...
>>
>> -- Jeremy



Re: photoshop 6 beta?

Deke Kincaid <dekek...@...>
 

From your screenshot someone at adobe has a sense of humor been reading the teal and orange blog.  :)


On Mar 24, 2012, at 8:30, dbr/Ben <dbr....@...> wrote:

Installed it today. After like 20 minutes of looking around, found where the LUT's are loaded… You can add a "Color Lookup" adjustment layer:


..or "Image > Adjustments > Color Lookup" and bake it into a layer.

Couldn't see any way to apply it purely as a viewing LUT, which is a shame (adjustment layers make picking colours tedious, and you have to disable it before exporting), but.. I could very well be missing something.

Not sure about a full list of formats, but the "Adobe Photoshop CS6/Presets/3DLUTs/" directory contains a bunch of .3dl files (two variants, one with "MESH" line, one without), .look files (IRIDAS format), and a .cube file

Think this is a representative sampling of the included LUT's:


It can also apply "Device Link" and "Abstract" ICC profiles

About the only mention I could find of the 3D LUT support (aside from copying-and-pastes of the feature-list) was
In my searching, I also came across this amusing thread (well, "amusing" in a frown-inducting way)
Reminds me of the Photoshop vs OpenEXR alpha/premult debate (oh, it's the same Adobe employee, which makes it a bit less worrying :P)

Anyway, hardly OCIO integration, but seems like a step forward. I'll play about more with it later!
- Ben

On 24/03/2012, at 11:50 AM, Jeremy Selan wrote:

Has anyone tried the photoshop 6 beta yet?  One of the line items is '3dlut adjustment', and I was curious what this meant. Couldn't find the docs online (yet).  Maybe you can now select a 3dlut from disk, and bake it into an image? I'm curious what formats they support...

-- Jeremy


Re: photoshop 6 beta?

dbr/Ben <dbr....@...>
 

Installed it today. After like 20 minutes of looking around, found where the LUT's are loaded… You can add a "Color Lookup" adjustment layer:


..or "Image > Adjustments > Color Lookup" and bake it into a layer.

Couldn't see any way to apply it purely as a viewing LUT, which is a shame (adjustment layers make picking colours tedious, and you have to disable it before exporting), but.. I could very well be missing something.

Not sure about a full list of formats, but the "Adobe Photoshop CS6/Presets/3DLUTs/" directory contains a bunch of .3dl files (two variants, one with "MESH" line, one without), .look files (IRIDAS format), and a .cube file

Think this is a representative sampling of the included LUT's:


It can also apply "Device Link" and "Abstract" ICC profiles

About the only mention I could find of the 3D LUT support (aside from copying-and-pastes of the feature-list) was
In my searching, I also came across this amusing thread (well, "amusing" in a frown-inducting way)
Reminds me of the Photoshop vs OpenEXR alpha/premult debate (oh, it's the same Adobe employee, which makes it a bit less worrying :P)

Anyway, hardly OCIO integration, but seems like a step forward. I'll play about more with it later!
- Ben

On 24/03/2012, at 11:50 AM, Jeremy Selan wrote:

Has anyone tried the photoshop 6 beta yet?  One of the line items is '3dlut adjustment', and I was curious what this meant. Couldn't find the docs online (yet).  Maybe you can now select a 3dlut from disk, and bake it into an image? I'm curious what formats they support...

-- Jeremy

1341 - 1360 of 2243