Date   
What does 'mcp' stand for?

Donal McMullan
 

It seems to be a temp or scratch directory on the render nodes that by default (on Linux) is expected to be at /mcp

Re: New Repository Location

Donal McMullan
 

(belatedly) thanks John.

Re: New Repository Location

John Mertic
 

Head to https://corporate.lfcla.com to kick the process off. You can refer to the guide at https://github.com/swinslow/cla-tool-docs/blob/master/walkthroughs/3-Corporate-Contributor-first-for-company.md for details on the entire workflow.

Thank you,

John Mertic
Director of Program Management - Linux Foundation
ASWF, ODPi, and Open Mainframe Project
Schedule time with me at https://calendly.com/jmertic



On Tue, Aug 13, 2019 at 10:48 AM Donal McMullan <donal.mcmullan@...> wrote:

Brian I need to download a corporate CLA for a new employer. Where would I find that now?

Re: New Repository Location

Donal McMullan
 

Brian I need to download a corporate CLA for a new employer. Where would I find that now?

OpenCue at SIGGRAPH

Brian Cipriano
 

Hi all,

The OpenCue team is at SIGGRAPH and we'd love to hear from you!

The OpenCue Birds of a Feather session is Tuesday at 10am as part of the ASWF's Open Source day. You find more info on that session over on the ASWF Open Source Day website.

On Wednesday the Steering Committee will be holding a regular meeting, which is open to the public. You can find more info about this meeting on the OpenCue blog.

Hope to see you there!
- brian

Re: OpenEXR repo moving to ASWF organization

Sharif Salah
 

On Mon, Jul 15, 2019 at 12:10 PM Brian Cipriano <cipriano@...> wrote:

- We briefly had a problem with some of our opencue.io presubmit checks disappearing... +Sharif Salah did you end up figuring out what happened there?

For the presubmits: I think it was mainly just issues related to reauthorizing and reconfiguring various services we'd enabled for the repository. I don't recall why, but we needed to reauthorize our Netlify integration, which resulted in losing some configuration. If you've got any third party services, such as Travis CI, then it's a good idea to make notes about any settings you can't export or won't be able to refer to after you reauthorize.

thanks,
Sharif

--
Google
------------
Google UK Limited
Registered Office: Belgrave House, 76 Buckingham Palace Road, London SW1W 9TQ
Registered in England Number: 3977902

Re: OpenEXR repo moving to ASWF organization

Sharif Salah
 

On Mon, Jul 15, 2019 at 12:10 PM Brian Cipriano <cipriano@...> wrote:

- We briefly had a problem with some of our opencue.io presubmit checks disappearing... +Sharif Salah did you end up figuring out what happened there?

For the presubmits: I think it was mainly just issues related to reauthorizing and reconfiguring various services we'd enabled for the repository. I don't recall why, but we needed to reauthorize our Netlify integration, which resulted in losing some configuration. If you've got any third party services, such as Travis CI, then it's a good idea to make notes about any settings you can't export or won't be able to refer to after you reauthorize.

thanks,
Sharif

--
Google
------------
Google UK Limited
Registered Office: Belgrave House, 76 Buckingham Palace Road, London SW1W 9TQ
Registered in England Number: 3977902

Re: OpenEXR repo moving to ASWF organization

Brian Cipriano
 

From the OpenCue side, the migration went quite smoothly for us -- as you said everything got migrated over and in most cases it was like nothing had changed at all.

A few minor things to keep in mind...
- Once the transfer happens, it's likely the LF CLA presubmit check will be in effect, so make sure you're ready for that.
- The person initiating the transfer needs to have both Owner permissions on the source repo as well as permission to create repositories in the destination org.
- We briefly had a problem with some of our opencue.io presubmit checks disappearing... +Sharif Salah did you end up figuring out what happened there?

Thanks,
- brian

On Mon, Jul 15, 2019 at 9:05 AM Cary Phillips <cary@...> wrote:
All,

With the adoption of OpenEXR by the ASWF, the GitHub repo at https://github.com/openexr/openexr will be moving to its new home in the ASWF organization, at https://github.com/AcademySoftwareFoundation/openexr. This should be mostly transparent. https://github.com/openexr/openexr will redirect to the new location, and all Issues, PR's, and the entire project history should be unaffected, and forks of the repo should update automatically.

Before we make the move, we're interested to hear from anyone from OpenVDB and OpenCue, which have already moved. Did the move cause any problems for anyone? Were there any gotchas we should anticipate? 

Thanks in advance for any feedback or suggestions.

- Cary

--
Cary Phillips | R&D Supervisor | ILM | San Francisco

OpenEXR repo moving to ASWF organization

Cary Phillips <cary@...>
 

All,

With the adoption of OpenEXR by the ASWF, the GitHub repo at https://github.com/openexr/openexr will be moving to its new home in the ASWF organization, at https://github.com/AcademySoftwareFoundation/openexr. This should be mostly transparent. https://github.com/openexr/openexr will redirect to the new location, and all Issues, PR's, and the entire project history should be unaffected, and forks of the repo should update automatically.

Before we make the move, we're interested to hear from anyone from OpenVDB and OpenCue, which have already moved. Did the move cause any problems for anyone? Were there any gotchas we should anticipate? 

Thanks in advance for any feedback or suggestions.

- Cary

--
Cary Phillips | R&D Supervisor | ILM | San Francisco

Join us for Beers of a Feather at SIGGRAPH

Emily Olin
 

Hi all - please join us for the ASWF Beers of a Feather at SIGGRAPH. The invite is below. Also, check out the Open Source Day we've put together at SIGGRAPH - a full day of open source project BoFs! 

Updated Event: OpenCue TSC (F2F) - Wednesday, 31 July 2019 #cal-invite

opencue-dev@lists.aswf.io Calendar <opencue-dev@...>
 

OpenCue TSC (F2F)

When:
Wednesday, 31 July 2019
10:00am to 11:00am
(GMT-07:00) America/Los Angeles

Where:
JW Marriott, Olympic 3

Organizer:
jmertic@...

Description:
-::~:~::~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~::~:~::-
Please do not edit this section of the description.

View your event at https://www.google.com/calendar/event?action=VIEW&eid=NjcxZjQ0dmYzMzBlZ3FuOTgxcGMwZ21mYm0gb3BlbmN1ZS1kZXZAbGlzdHMuYXN3Zi5pbw&tok=Mjcjam1lcnRpY0BsaW51eGZvdW5kYXRpb24ub3JnMmNkZjg4MzNiNzIwYzM1ZWZiY2Q3ZjY0YjY1OGQ2OGIwZjRmZGZmOA&ctz=America%2FNew_York&hl=en&es=1.
-::~:~::~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~::~:~::-

Re: lets remove some stuff!

Brian Cipriano
 

This sounds like a great idea!


On Wed, Jun 5, 2019 at 3:59 PM Matt Chambers via Lists.Aswf.Io <mvchambers=me.com@...> wrote:


6. ActiveMQ / JMS

Remove it for now but but bring it back as an external plugin. With spring, as long as you package the jar properly, you can load load external plugins at runtime. These plugins would have to be packaged as Spring Boot starters, which is basically just a file in src/main/resources that tells Spring where to search for beans.

Sounds interesting though I'm a little less clear on the details here. It sounds like it will simplify the Cuebot config by letting us ditch some of the "enable this feature" config values we recently added to help Cuebot work better in new environments.


ActiveMQ is hard coded as a dependency to OpenCue, however ActiveMQ is just one of the message bus systems in use in the M/E industry.  I actually think RabbitMQ is quite popular as well.

Maybe we should explore  using the local pub/sub system that is part of Google Guava to feed potentially any external message bus.  Then we just have plugins for Google PubSub, ActiveMQ, RabbitMQ, etc.  The plugins would be separate stand-alone jars available on the docker image that can be activated with an environment variable, something like that.

-Matt

Reviewing chained changes

Brian Cipriano
 

Hi Matt, all,

Big thanks for all the work you've been doing so far. Lots of good stuff in these PRs you've sent.

Listing them here for easy reference:
https://github.com/AcademySoftwareFoundation/OpenCue/pull/354

It looks like these changes are "chained", i.e. each is dependent on the one that came before it. Having done this before myself I certainly get why it's done this way :)  But it does make each change hard to review on its own, since each PR also contains the changelist from the ones before it.

So we should probably follow a process like:
- Start with the first PR, iterate on it until it's ready to go.
- Merge it to master.
- Move to the second PR, merge master into that branch, resolve conflicts as needed.
- Review second PR until it's ready to go.
- And so on.

Does that make sense? If anyone has suggestions for better ways to do this, either on the PR-creation side or the review side, I'm all ears.

Thanks,
- brian

New Repository Location

Brian Cipriano
 

Hi all,

As part of OpenCue's transfer to the Academy Software Foundation, the OpenCue Github repo can now be found at a new URL:


If you're running OpenCue directly from a code checkout, you'll want to update your checkout to use the new URL.

$ git remote set-url origin git@...:AcademySoftwareFoundation/OpenCue.git

(Note a couple things about this command:

1. The name of your Git remote might be different. Here I'm using "origin" but if you have your own fork it might be named "upstream" or something like that.

2. This assumes you're using SSH to check out from Github and not HTTP.)

Also, the ASWF requires any code contributors to have signed the OpenCue CLA. Any Pull Requests to the OpenCue repo will trigger an automatic check to make sure the author (or the author's company) has this on file. If not, the ASWF CLA tool will post a message to the Pull Request walking you through the process.

Let us know if you have any questions or run into any problems!

Thanks,
- brian

Re: lets remove some stuff!

Matt Chambers
 



6. ActiveMQ / JMS

Remove it for now but but bring it back as an external plugin. With spring, as long as you package the jar properly, you can load load external plugins at runtime. These plugins would have to be packaged as Spring Boot starters, which is basically just a file in src/main/resources that tells Spring where to search for beans.

Sounds interesting though I'm a little less clear on the details here. It sounds like it will simplify the Cuebot config by letting us ditch some of the "enable this feature" config values we recently added to help Cuebot work better in new environments.


ActiveMQ is hard coded as a dependency to OpenCue, however ActiveMQ is just one of the message bus systems in use in the M/E industry.  I actually think RabbitMQ is quite popular as well.

Maybe we should explore  using the local pub/sub system that is part of Google Guava to feed potentially any external message bus.  Then we just have plugins for Google PubSub, ActiveMQ, RabbitMQ, etc.  The plugins would be separate stand-alone jars available on the docker image that can be activated with an environment variable, something like that.

-Matt

Invitation: OpenCue TSC (F2F) @ Wed Jul 31, 2019 7pm - 8pm (EDT) (opencue-dev@lists.aswf.io)

John Mertic
 

You have been invited to the following event.

OpenCue TSC (F2F)

When
Wed Jul 31, 2019 7pm – 8pm Eastern Time - New York
Where
JW Marriott, Olympic 3 (map)
Calendar
opencue-dev@...
Who
jmertic@... - organizer
Greg Denton
Larry Gritz
greg.gewickey@...
mvchambers@...
tac@...
estrauss@...
Benjamin Dines
eolin@...
kthurston@...
james.l.jeffers@...
Brian Cipriano
opencue-dev@...

Going (opencue-dev@...)?   Yes - Maybe - No    more options »

Invitation from Google Calendar

You are receiving this courtesy email at the account opencue-dev@... because you are an attendee of this event.

To stop receiving future updates for this event, decline this event. Alternatively you can sign up for a Google account at https://www.google.com/calendar/ and control your notification settings for your entire calendar.

Forwarding this invitation could allow any recipient to send a response to the organizer and be added to the guest list, or invite others regardless of their own invitation status, or to modify your RSVP. Learn More.

Re: dev environment setup

Donal McMullan
 

Thanks lads - docker compose does seem like the lowest friction way to get someone started.

Re: lets remove some stuff!

Brian Cipriano
 

Thanks Matt! Good stuff. Will address points inline, and others feel free to chime in if you have some opinion on any of these.

On Fri, May 31, 2019 at 6:41 AM Matt Chambers via Lists.Aswf.Io <mvchambers=me.com@...> wrote:

I have OpenCue building and running with OpenJDK11, SpringBoot 2.1.5, Gradle 5.4.1 locally and within Docker!  Was easier than I thought it would be.

That's great! Feel free to send a Pull Request if you're ready, or you can also send a Draft Pull Request if you're just looking for general feedback before it's ready for real review.
 

I was thinking that, before we look at the scheduler/dispatcher, it might be beneficial to simplify/cleanup as much as possible.

1. Track-It stuff.

Can the track-it data source and just about everything associated with it be removed for now in favor of a future external prioritization interface? The scheduler could hit an external service to grab similar data and we’ll publish a REST interface people would implement in order to build a prioritization service.  We won’t even need to store it in tables anymore, so that stuff can go as well.

2.  Can we just delete everything related to Oracle?

Postgres is totally fine for this (better IMHO), and has no problems with a NetApp or any decent NFS implementation as backend storage.  If people need enterprise support to help out with disaster recovery on-prem, there is the Postgres based Enterprise DB.  Backups and HA are easily configurable for Postgres in Google Cloud SQL as well.

I'd be happy to ditch both of these things. We should collect Sony's opinion first.


3. All the XML files

Burn them with fire and switch to annotation based configuration.

Also agreed. It's pretty hard to find docs online that reference the XML formats -- switching to annotations should decrease barrier to entry.
 

4. Logging stuff.

Switch to LogBack which, will be a giant pull request where the import for org.apache.log4j.Logger changes to something else, though that something is API compatible.  Logback is the default for spring boot, much easier to configure stuff like log rotation, external logging aggregators, etc.

We have a somewhat related Issue in https://github.com/imageworks/OpenCue/issues/72.

In particular we're interested in providing the option to send logs to external log storage (e.g. Google Cloud Logging). The primary reason is, it would be one step closer to removing the need to have a common mount point that all components (RQD/CueGUI/etc) have access to. Setting up such a mount point can be fairly challenging in a hybrid on-prem/cloud environment.

But there are other benefits to doing this as well -- external logging services typically provide nice tools for analyzing logs, for example.

It looks like there's a Logback plugin for Google Cloud Logging at least, so I think this could work.
 

5. Java boiler plate.

I think all the getters/setters can all be removed form the service/dao beans as is right now and IoC will still be able to wire up the app.  We can also move to constructor injection which lets you finalize all those dependencies as well.

Hm, I seem to remember that the getters/setters were required under the current setup? If I'm wrong about that I don't have a problem ditching them.
 

6. ActiveMQ / JMS

Remove it for now but but bring it back as an external plugin. With spring, as long as you package the jar properly, you can load load external plugins at runtime. These plugins would have to be packaged as Spring Boot starters, which is basically just a file in src/main/resources that tells Spring where to search for beans.

Sounds interesting though I'm a little less clear on the details here. It sounds like it will simplify the Cuebot config by letting us ditch some of the "enable this feature" config values we recently added to help Cuebot work better in new environments.
 

I can submit a bunch of pulls for these 6 things plus the JDK11/SpringBoot 2.1.5 stuff which is mainly the gradle build file and Docker file.

-Matt


lets remove some stuff!

Matt Chambers
 

I have OpenCue building and running with OpenJDK11, SpringBoot 2.1.5, Gradle 5.4.1 locally and within Docker! Was easier than I thought it would be.

I was thinking that, before we look at the scheduler/dispatcher, it might be beneficial to simplify/cleanup as much as possible.

1. Track-It stuff.

Can the track-it data source and just about everything associated with it be removed for now in favor of a future external prioritization interface? The scheduler could hit an external service to grab similar data and we’ll publish a REST interface people would implement in order to build a prioritization service. We won’t even need to store it in tables anymore, so that stuff can go as well.

2. Can we just delete everything related to Oracle?

Postgres is totally fine for this (better IMHO), and has no problems with a NetApp or any decent NFS implementation as backend storage. If people need enterprise support to help out with disaster recovery on-prem, there is the Postgres based Enterprise DB. Backups and HA are easily configurable for Postgres in Google Cloud SQL as well.

3. All the XML files

Burn them with fire and switch to annotation based configuration.

4. Logging stuff.

Switch to LogBack which, will be a giant pull request where the import for org.apache.log4j.Logger changes to something else, though that something is API compatible. Logback is the default for spring boot, much easier to configure stuff like log rotation, external logging aggregators, etc.

5. Java boiler plate.

I think all the getters/setters can all be removed form the service/dao beans as is right now and IoC will still be able to wire up the app. We can also move to constructor injection which lets you finalize all those dependencies as well.

6. ActiveMQ / JMS

Remove it for now but but bring it back as an external plugin. With spring, as long as you package the jar properly, you can load load external plugins at runtime. These plugins would have to be packaged as Spring Boot starters, which is basically just a file in src/main/resources that tells Spring where to search for beans.

I can submit a bunch of pulls for these 6 things plus the JDK11/SpringBoot 2.1.5 stuff which is mainly the gradle build file and Docker file.

-Matt

Re: dev environment setup

Matt Chambers
 

Hi Donal!

I try to run everything through docker.

We could make a docker-compose file that runs Postgres, a Cuebot, and one or two RQDs.  

Something like this (I left a lot out) at the root of the project.  With this file, you wouldn’t even need to build anything to get a simple queue running.  It would pull the latest dev images from docker-hub, or use your local images.

postgres:
    image: “postgres:9.6.9”
    ports:
        - “5432:5432”
    environment:
        POSTGRES_PASSWORD: cuebot
        POSTGRES_USER: cuebot
        POSTGRES_DB: cuebot
        PG_DATA: /var/lib/postgresql/data/pgdata

cuebot:
   links:
      - “postgres”
   image: cuebot
   build:
       context: cuebot
       cache_from:
            - OpenCue/cuebot:latest 
            - cuebot
   ports:
       - 8080:8080
   volumes:
       - “/cuebot/target:/cuebot/lib”

rqd1:
   build:
       context: rqd

In this case, you would do:

docker-compose build
docker-compose up

This way if someone just want to do UI work, they can quickly spin up a cluster.

-Matt


On May 29, 2019, at 6:54 PM, Brian Cipriano via Lists.Aswf.Io <cipriano=google.com@...> wrote:

Hi Donal,

Thanks for sharing this!

How do you set up your development environments?

My personal setup depends on what I'm working on.

I run a local Postgres instance, installed via Homebrew.

If I'm doing work on Cuebot, I've got IntelliJ set up with a Run Configuration to run CuebotApplication, with some Program arguments to point it to my local postgres instance.

If I'm working on any of the Python components, my process looks like:
- Run Cuebot either a) via IntelliJ as above or b) (more typically) via Docker
- Activate whichever virtualenv I plan to use
- Execute the Python code directly from my local checkout, using the CUEBOT_HOSTS env var to point it to my local Cuebot instance.
- I use PyCharm locally and will sometimes set up Run configurations in there to execute the module in question, which lets me use things like PyCharm's built-in debugger.

Happy to go into more detail on any of this if desired.

- brian


On Sun, May 26, 2019 at 6:46 AM Donal McMullan <donal.mcmullan@...> wrote:
I added some scripts to help myself get started with development on the repo:

Specifically I wanted some bootstraps to create a development environment for new users, and then to launch the applications directly from that environment without any intermediate step (like running installers, etc).

How do you set up your development environments?

Thanks

DJM