BLDMGR-10051: Waf 2.1.6 patched#16
Conversation
This patch makes Fortran source nodes that USE a module depend only on
the .mod node and not on the object file generated from the module
source file. This makes it possible to reduce recompilation when the
module source file changes in a trivial way that doesn't affect the
contents of the .mod file.
In other words, if foo.f has "USE mymod", and mymod is defined in
mymod.f, we turn this dependency graph:
foo.f ---> mymod.mod ---> mymod.f
| ^
| |
\-------> mymod.o -------/
Into this:
foo.f ---> mymod.mod ---> mymod.f
^
|
mymod.o -------/
In practice, when using ifort, to get the benefit described above one
also needs to do two more things:
1) Decorate the fc class with Task.update_outputs. Not done as part of
this patch to leave the decision up to the wscript author.
2) Monkey-patch Utils.h_file() to strip the silly timestamp that ifort
inserts in the .mod files it writes, but that's not included in this
patch because it is an implementation-dependent hack (this patch fixes
an actual bug IMHO). Note: even stripping the timestamp is not perfect
because ifort sometimes inserts other mystery junk into the .mod file,
but at least we can avoid unnecessary recompilation *some* of the time.
If 1) is not done, this patch won't change the current behavior, which
is to have foo.f effectively be dependent on mymod.f because mymod.mod
inherits the signature from mymod.f.
This is schrodinger specific - track only in our build and source directories, which are parents of the waf build and source directories. Allows this to work in a multi-repo context.
Used when waf tasks create intermediate files for build targets outside of the root build node. In other words, when installing files into $SCHRODINGER/internal when the root build node is $SCHRODINGER/mmshare-v*/ etc. This will avoid the issue of waf trying to create files with paths containing '__root__'. This is Schrodinger specific.
We set compilation flags manually and the check method modifies, for example, CPPFLAGS. Conflicts: waflib/Tools/qt5.py
Conflicts: waflib/extras/parallel_debug.py
so that calling `waf build install` will produce two sets of outputs, rather than the `install` command's output overwriting the output from the `build` command.
In combination with reverting ea60e436e94a374fb72a303c1d2c63ffd9dc12ce in mmshare, this should fix BLDMGR-3037
* Cache file parsing (cf. c_preproc.py) * Skip some calls to find_deps by checking self.seen sooner in iter * Use set to keep track of seen modules * Use namedtuple instead of tuple unpacking * Use re.findall instead of scanning line-by-line
and put back the 'seen' check for includes found by find_deps.
lennax
left a comment
lennax
left a comment
There was a problem hiding this comment.
I think as follow-up we'd want to merge upstream_waf-2.1.6 into master, keep this branch as-is, then copy the changes into mmshare?
|
@lennax I just tried to merge 2.1.6 into master and it's a mess. |
|
@pdevlaminck yeah it looks like they rewrote the git history on gitlab. I wonder how easy it would be to make this repo no longer a fork... |
|
@lennax So there is no button you can click: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/detaching-a-fork Since this branch is now pushed to github, can I open a new ticket to update the waf in mmshare with a couple other changes to make the new waf work? |
|
@pdevlaminck yes, sgtm. we can figure out how to fix the master branch at a later date - but keeping the upstream_waf-2.1.6 branch indefinitely also seems fine. |
When waf scans C/C++ source files to find dependencies via the included headers, we intentionally only track non-SOFTWARE_LIB schrodinger dependencies with the `get_header_nodepaths` api. This logic is currently broken for PF-enabled builds since package factory installs all third-party dependencies in the $SCHRODINGER build directory. This broken logic has significantly increased clean build times by up to 100%, so we should fix it.
When waf scans C/C++ source files to find dependencies via the included headers, we intentionally only track non-SOFTWARE_LIB schrodinger dependencies with the get_header_nodepaths api. This logic is currently broken for PF-enabled builds since package factory installs all third-party dependencies in the $SCHRODINGER build directory. This broken logic has significantly increased clean build times by up to 100%, so we should fix it.
6 participants
BLDMGR-10051
Rebased our patches onto the upstream waf repo from: https://gitlab.com/ita1024/waf
One commit for
qt5tool was dropped since Qt6 is now supported with the wafqt5tool.It just needs to be configured before the tool is loaded with
conf.want_qt6 = True.I had to add the upstream waf-2.1.6 branch to the repo as well so that the pull request has the proper base branch to compare against.