Render Software Metric


Arya Mahini <amahini@...>
 

Hello,

 

I wanted to ask about the functionality and features in OpenCue.

 

  1. Does OpenCue feature a tool so that we can configure slaves to transfer files between other slaves. I wanted to ask this because there would be bandwidth and drive controller limitations if the data for jobs all had to come from a single network drive or from the master controller’s drives. Is something like that possible and easily accomplished?

 

  1. Is there an advanced Load Balancing option such that if a slave becomes idle in the middle of a job, and another slave is still rendering part of the same task, a certain number of frames to be rendered from the active slave can be transferred to the idle slave so the idle slave can do work and not remain idle.

 

  1. If OpenCue does not have inherent support for the item above, can it be developed?

 

  1. Is there render node remote control from the master?

 

 

Thanks.

 

Arya

 


Brian Cipriano
 

Hi Arya! Answers inline --

On Tue, Feb 4, 2020 at 6:48 PM Arya Mahini <amahini@...> wrote:

Hello,

 

I wanted to ask about the functionality and features in OpenCue.

 

  1. Does OpenCue feature a tool so that we can configure slaves to transfer files between other slaves. I wanted to ask this because there would be bandwidth and drive controller limitations if the data for jobs all had to come from a single network drive or from the master controller’s drives. Is something like that possible and easily accomplished?

OpenCue doesn't have anything like this built-in. It currently doesn't really do any file management -- it assumes that when a job starts that the worker node will either have everything it needs already or will take care of transferring files itself.

OpenCue supports sending arbitrary commands to workers, so the best way to do this now would probably be to wrap renders in a wrapper script that will do any file transfer you need to happen before the render starts. It's certainly possible though I'd imagine the sort of peer-to-peer file transfer you're talking about would be a fairly complex task to manage. 

 
  1. Is there an advanced Load Balancing option such that if a slave becomes idle in the middle of a job, and another slave is still rendering part of the same task, a certain number of frames to be rendered from the active slave can be transferred to the idle slave so the idle slave can do work and not remain idle.
Historically the standard way of running OpenCue is to only send a single frame to a worker at a time, so in that sense yes, frames will be evenly distributed among workers, and as workers become idle they will be sent additional frames to render.

We recently added support for chunking, i.e. sending a frame range to a worker instead of a single frame. If you use that method, once a worker starts a chunk (for example "render frames 11-20") the scheduler will not redistribute those particular frames to other workers unless some failure occurs. However if a worker becomes idle and there are other different chunks waiting to be rendered, those will be distributed to the worker.

  

  1. If OpenCue does not have inherent support for the item above, can it be developed?
If I'm understanding your question correctly I believe it does, though maybe you mean something slightly different than what I described above. 
 

  1. Is there render node remote control from the master?

It depends on what functionality you're looking for, but yes, that mechanism exists.

The client-side tools (i.e. the python API / cuegui / cueadmin) can initiate these actions, and that all routes through the master which takes action as necessary.

For example you can "lock" a host to stop any new work from being scheduled to it.
  

 

Thanks.

 

Arya