Re: ASWF CI Infrastructure: VFX Reference Platform Dependencies
Daniel Heckenberg
Thanks for those of you answered the poll. There's definitely enough interest to kick off the working group. As you all probably realised, I made an error in my mental timezone juggling between Australian DST, US PDT and EDT (what's 2000 miles and 4 hours?!) so I've updated the poll to the correct time (same as the fortnightly TAC, but offset on alternate weeks) as originally planned. A meeting invite and calendar update will come through on the TAC list shortly. Thanks, Daniel |
|
|
|
Re: OpenVDB Jenkins builds broken due to Boost Python not found
Dan Bailey
Looking at the version of Boost that's being pulled in (1.58), that's actually older than the the version being specified by VFX Reference Platform for 2017 which is 1.61 (https://www.vfxplatform.com/).
I think we should ideally try and stick with the versions coming from the reference platform if possible rather than getting whatever is coming in with the Linux distro. More than likely this will mean a bit more work building from source and adding Boost headers/libraries for these specific versions into Nexus and pulling them down though. Likewise the compiler is GCC 5.4.0 which isn't the version specified by VFX Reference Platform. I appreciate we're just trying to get things up and running in right now (and we don't even subscribe to this ideology in Travis for OpenVDB), so for expediency I think it makes sense to change CMake in whatever way is needed. Having said that, I would suggest that the Houdini plugins are probably a higher priority to figure out than the Python bindings as I believe most people are exposed to OpenVDB through Houdini these days. |
|
|
|
Re: OpenVDB Jenkins builds broken due to Boost Python not found
Nick Porcino
Boost in general and boost python in particular complicates build environments, maintenance, and compiler migrations. I would like to +1 the suggestion of a move to pybind11. This might be an excellent opportunity for community involvement. - Nick |
|
|
|
Re: ASWF CI Infrastructure: VFX Reference Platform Dependencies
Daniel Heckenberg
As discussed at the last TAC meeting, we'll set up a working group to progress this, and I'd like to encourage involvement from the community. For pragmatic reasons, I'll propose the same timeslot as the TAC meeting, on alternate weeks. That would place our first meeting on Wednesday 31 at 1pm (US/ EDT). Please indicate your interest and ability to attend at this time in the following doodle poll: https://doodle.com/poll/ig8zr6mtszq6r35p Thanks, Daniel |
|
|
|
Re: OpenVDB Jenkins builds broken due to Boost Python not found
Daniel Heckenberg
It's never good to see a broken build, but in this case what serendipity! It provides the perfect motivation for the cmake tools project which Larry has proposed on the tac mailing list, and demonstrates the kinds of ambiguities that we might reduce by defining a working set of vfxplatform builds to use as dependencies. But back to the problem at hand... Can the OpenVDB folks address this, or should an issue be raised? Thanks, Daniel |
|
|
|
Re: OpenVDB Jenkins builds broken due to Boost Python not found
Larry Gritz
Agreed about pybind11. I can't recommend it strongly enough.
Kimball did a good job of explaining what I was getting at, but I'll make a pitch on the TAC list in a new thread as well. -- lg
|
|
|
|
Re: OpenVDB Jenkins builds broken due to Boost Python not found
Kimball Thurston
switch to pybind11? ;-P
This is what Larry was talking about the last meeting, and in a much more concrete manner than I had babbled about in our first meeting.
I think it is a great idea to make a repo that has template cmake macros and a collection of all the Find* things we need for the various projects in ASWF. People can then request those Find* be pushed upstream to cmake, and each project can keep a stash of what the current ones they need are, and rev versions as necessary to get whatever new macros, etc. they want.
Back to your actual question, in OpenEXR, we do a very similar thing to compile for a specific python variant, except don't have a period between when asking for boost:
.... <paraphrasing a bit here>
##### NB: with a period between maj.minor find_package(PythonLibs ${OPENEXR_PYTHON_MAJOR}.${OPENEXR_PYTHON_MINOR} ##### NB: withOUT a period between find_package(Boost COMPONENTS python${OPENEXR_PYTHON_MAJOR}${OPENEXR_PYTHON_MINOR}) if (NOT Boost_PYTHON${OPENEXR_PYTHON_MAJOR}${OPENEXR_PYTHON_MINOR}_FOUND) message(WARNING "requested boost python version not found...") endif() ##### find_package(NumPy)
....
Hope that helps, Kimball From: main@... <main@...> on behalf of Thanh Ha <thanh.ha@...>
Sent: Saturday, October 27, 2018 7:36:02 AM To: main@... Subject: [ASWF] OpenVDB Jenkins builds broken due to Boost Python not found *Originates outside of Weta Digital
Hi Everyone,
Since the move of OpenVDB to ASWF. The Jenkins build is now broken due to CMake not able to find the python boost library, you can see the error message here:
I've traced it down to this commit which changed causing the build to now fail:
https://github.com/AcademySoftwareFoundation/openvdb/commit/566ffa6b802e489011eea8917f0db81d431d47ed
I'm not sure what Linux distro that patch was targetting but on Ubuntu 16.04 the boost python library is located in /usr/include/boost/python/ (with no trailing version number). I guess we need to improve the CMake script to detect if there is a trailing
version or not. Does anyone else have any other recommendations on what we should do here?
Thanks,
Thanh
|
|
|
|
OpenVDB Jenkins builds broken due to Boost Python not found
Hi Everyone, Since the move of OpenVDB to ASWF. The Jenkins build is now broken due to CMake not able to find the python boost library, you can see the error message here: I've traced it down to this commit which changed causing the build to now fail: https://github.com/AcademySoftwareFoundation/openvdb/commit/566ffa6b802e489011eea8917f0db81d431d47ed I'm not sure what Linux distro that patch was targetting but on Ubuntu 16.04 the boost python library is located in /usr/include/boost/python/ (with no trailing version number). I guess we need to improve the CMake script to detect if there is a trailing version or not. Does anyone else have any other recommendations on what we should do here? Thanks, Thanh |
|
|
|
Re: ASWF CI Infrastructure: VFX Reference Platform Dependencies
Daniel Heckenberg
Hi Dan,
Thanks for the supportive feedback!
We have established vendor-approved mechanisms to automate download and build for the Maya SDK and for Houdini Apprentice which additionally allows for running tests. The Foundry have indicated some support but that discussion is still in progress.
Some details, which I encourage others to correct to add to...
The Maya SDK (aka devkit) is directly downloadable.
The Houdini download will require authentication but we plan to use Jenkins credential storage for an ASWF or OpenVDB project account.
Now that the OpenVDB project has been adopted and transferred into the ASWF we should be ready to start augmenting the CI with these components.
Please lead the way, Dan!
Thanks,
Daniel
On Thu, Oct 25, 2018 at 7:08 AM +1100, "Dan Bailey"
<danbailey@...> wrote:
--
Daniel Heckenberg R&D Supervisor - Graphics T: +61 2 9383 4800 (main) D: +61 2 8322 3123 (direct) E: Daniel.Heckenberg@... Building 54 / FSA #19, Fox Studios Australia, 38 Driver Avenue Moore Park, NSW 2021 AUSTRALIA ![]() ![]() ![]() ![]() ![]() www.animallogic.com CONFIDENTIALITY AND PRIVILEGE NOTICE This email is intended only to be read or used by the addressee. It is confidential and may contain privileged information. If you are not the intended recipient, any use, distribution, disclosure or copying of this email is strictly prohibited. Confidentiality and legal privilege attached to this communication are not waived or lost by reason of the mistaken delivery to you. If you have received this email in error, please delete it and notify us immediately by telephone or email. |
|
|
|
Re: ASWF CI Infrastructure: VFX Reference Platform Dependencies
Dan Bailey
Hi Daniel,
Thanks for sending this around, this is a great overview. I know vendor dependencies were discussed in a past meeting, but skimming the meeting notes, I can't seem to find the outcome of those discussions. Wondering what the plan is to build against libraries for Houdini, Maya, Nuke, etc in terms of EULAs and exactly how this will be exposed in Nexus? Regarding CI for OpenVDB specifically, I don't see a problem with sticking to using VFX Reference Platform versions of dependencies. I suggest one of the next steps will be to match the existing build matrix used with Travis CI (https://github.com/AcademySoftwareFoundation/openvdb/blob/master/.travis.yml) and the major component missing in ASWF Jenkins currently is building the Houdini plugins. Thanks, Dan |
|
|
|
ASWF CI Infrastructure: VFX Reference Platform Dependencies
Daniel Heckenberg
Hello everyone, As we start to adopt projects into the ASWF, I would like to turn attention to the CI setup and especially the way that project dependencies are built and managed. There have been many discussions here and elsewhere which I shall ambitiously summarize as, "We should make the ASWF CI dependencies a practical realization of the VFX Reference Platform." This would entail establishing build configurations for the relevant packages to be built from source and storage of the results as reusable artifacts. As all parts of this chain will be open, accessible and reproducible this offers a great opportunity for contributions from the community, and for the configurations and artifacts to be used to facilitate software development and OSS software use. Of course, there are lots of gaps and questions with this, such as "what do we do for non-Linux builds"? I'd like to encourage the discussion here as we embark on the effort. Some links follow with excellent Linux Foundation documentation of the CI infrastructure. Thanks, Daniel Heckenberg ASWF TAC Chair References: ci-management repo here:
CI docs are available here:
https://docs.releng.linuxfoundation.org/en/latest/As a background into the current ASWF CI infrastructure, Thanh Ha gave a great overview back in July: CI Infrastructure Usage Details
|
|
|
|
News Today: New Members & OpenVDB
Emily Olin
ASWF Community:
Today, the Academy Software Foundation announced four new members: Sony Pictures Entertainment/Sony Pictures Imageworks as a Premier member, Warner Bros. as a General Member, and Blender Foundation and Visual Effects Society (VES) as Associate members. We also announced that OpenVDB, developed and maintained by DreamWorks Animation, has been approved as the Foundation's first hosted project.
The press release with additional details is here. Please consider sharing this news via your social channels. Below are some suggested Tweets.
Thanks! .@SonyPictures @WarnerBrosEnt
Click-to-tweet: https://ctt.ac/jvd0t.@AcademySWF gains momentum with four new members and approves OpenVDB by @DWAnimation as the Foundation's first open source project. Read more here: https://bit.ly/2q8q13Q Click-to-Tweet: https://ctt.ac/0I86H |
|
|
|
Re: hello
And past TAC meeting minutes are here: --
Cary Phillips | R&D Supervisor | ILM | San Francisco
|
|
|
|
Re: hello
Thanks for the answers Rob!
Cheers! |
|
|
|
Re: hello
Rob Bredow
Thank you for your interest. Great questions. 1) No, or at least not yet. The VES is doing an excellent job running the VFX Platform and there are a number of other projects that the group has discussed which seem much higher priority to be adopted by the ASWF. Projects like OpenEXR, OpenVDB, OCIO are under active discussion for early adoption. That said, there is a lot of overlap between VES Technology Committee members and ASWF members and the door is open for any collaboration that makes sense for sure. But right now, I personally am just happy both projects are moving forward and staying in close communication with each other. For example, the build infrastructure is based heavily on compatibility with the VFX Platform. 2) Yes. I understand it's an open meeting. Only TAC members can vote, but the calls are open for anyone to join in. Take a look at the list here for meeting announcements. Thanks-- Rob On Mon, Sep 24, 2018 at 9:39 AM <sreenivas9alapati@...> wrote: I am excited to seen the initiative by the community to push towards open source. Congratulations to all the folks involved in setting up ASWF! |
|
|
|
hello
I am excited to seen the initiative by the community to push towards open source. Congratulations to all the folks involved in setting up ASWF!
Two questions I have! * Going forward, VFX platform(http://www.vfxplatform.com/) will be moved under the umbrella of ASWF ? * Are the Wednesday TAC meetings open for all to join ? Thanks! |
|
|
|
Re: Building OpenEXR on Windows
Nick Porcino
Thanks! I was still grepping to find the VS2010 reference. The root Readme.md is correct. On Wed, Sep 19, 2018 at 2:23 PM Cary Phillips <cary@...> wrote:
--
-------------------------------- Nick Porcino @meshula Virtual and augmented production, interactive applications, and robotics, since 1982 |
|
|
|
Re: Building OpenEXR on Windows
I'll update the README. On Wed, Sep 19, 2018 at 12:57 PM Thanh Ha <thanh.ha@...> wrote:
--
Cary Phillips | R&D Supervisor | ILM | San Francisco
|
|
|
|
Re: Building OpenEXR on Windows
Oh yeah that makes sense, I should have thought of that. I copied what was in README and didn't think to try changing the value. I got further now with the updated generator. Will report more on progress once I have something working. Thanks, Thanh On Wed, Sep 19, 2018 at 3:40 PM Nick Porcino <nick.porcino@...> wrote:
|
|
|
|
Re: Building OpenEXR on Windows
Nick Porcino
Hi there, the script I see here: https://jenkins.aswf.io/sandbox/job/openexr-win64-master/configure seems to be incorrectly configured to run a very old Visual Studio, Visual Studio 2010. -G "Visual Studio 10 Win64" .. The Generator expression for CMake needs to specify version and year. We've tested OpenEXR with Visual Studio 2015 and 2017, and I see you're intending to use 2017. So, update the generator with: -G "Visual Studio 15 2017 Win64" HTH! - Nick On Wed, Sep 19, 2018 at 11:57 AM Thanh Ha <thanh.ha@...> wrote:
--
-------------------------------- Nick Porcino @meshula Virtual and augmented production, interactive applications, and robotics, since 1982 |
|
|