-
Notifications
You must be signed in to change notification settings - Fork 7
BLD: use cibuildwheel to create wheel matrix #21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
I'll request a review here for now, as I think this is at a decent point and I should probably just upload to pypi and call it a day (and before I fall down the rabbit hole of conda builds) |
.github/workflows/standard.yml
Outdated
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Build wheels |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the PR here, this ends up building wheels marked at version 0.0.0. Is this expected for the non-tag build?
20 wheels produced in 80 minutes:
pyca_epics-0.0.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl 476 kB
pyca_epics-0.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 449 kB
pyca_epics-0.0.0-cp310-cp310-musllinux_1_2_i686.whl 1,569 kB
pyca_epics-0.0.0-cp310-cp310-musllinux_1_2_x86_64.whl 1,432 kB
pyca_epics-0.0.0-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl 476 kB
pyca_epics-0.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 449 kB
pyca_epics-0.0.0-cp311-cp311-musllinux_1_2_i686.whl 1,569 kB
pyca_epics-0.0.0-cp311-cp311-musllinux_1_2_x86_64.whl 1,432 kB
pyca_epics-0.0.0-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl 477 kB
pyca_epics-0.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 450 kB
pyca_epics-0.0.0-cp312-cp312-musllinux_1_2_i686.whl 1,569 kB
pyca_epics-0.0.0-cp312-cp312-musllinux_1_2_x86_64.whl 1,433 kB
pyca_epics-0.0.0-cp313-cp313-manylinux_2_17_i686.manylinux2014_i686.whl 477 kB
pyca_epics-0.0.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 450 kB
pyca_epics-0.0.0-cp313-cp313-musllinux_1_2_i686.whl 1,569 kB
pyca_epics-0.0.0-cp313-cp313-musllinux_1_2_x86_64.whl 1,433 kB
pyca_epics-0.0.0-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl 470 kB
pyca_epics-0.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl 450 kB
pyca_epics-0.0.0-cp39-cp39-musllinux_1_2_i686.whl 1,569 kB
pyca_epics-0.0.0-cp39-cp39-musllinux_1_2_x86_64.whl 1,432 kB
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't sorted this bit out yet. I saw this on my fork and hoped that it was just missing tags.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like all the other cibuildwheel examples end up either collecting the tag ahead of time or setting it statically. I'll deal with this in a similar way, maybe
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some more digging, setuptools_scm generates the version files on import or build. Turns out we had been using the old default wheel build method (python -m pip wheel), and switching this to python -m build generates those files properly.
|
Once we're happy with the way these builds run, I'll flip on the tag filter and only build wheels on release. The ubuntu builds take on the order of an hour and we probably don't need to be eating that many minutes |
|
I think the builds look good now |
ZLLentz
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe let's switch to tag-only, merge, tag, and give it a try
|
I still need to set up the pypi side of things, I'll run the wheels and upload to testpypi first to confirm. After that I'll get the tokens sorted |
|
Merging this now, we need a clean tag to upload to pypi in any incarnation |
(Please squash this on merge)
Description
Uses
cibuildwheelto compile and repair wheels for various linux and macos platforms. (not windows)Context
pycawas missing from pypi originally because it had a naming conflict. After switching to pyca-epics, it turns out that we also just didn't understand what was necessary for packages with compiled libraries.Long story short: I went through a lot of back and forth here, mostly revolving around trying to get the library files to be recognized properly.
Long story long:
cibuildwheelis a tool used by quite a few other open source projects, and works rather nicely. A key thing to remember is that the ci job (sometimes) builds wheels inside a docker container, so you need to make sure that any environment variables and external dependencies are installed in the right place.Windows was skipped after I ran into an error along the lines of "pthread.h not found" Pthread is a *nix-specific construct, and building for windows seems to require using a different library entirely. I'm not in the mood to rewrite the c-extensions here so I'm going to say we don't support windows.d
The generated and repaired wheels are saved as artifacts, which I should probably hook up for automatic upload to pypi but I'll deal with that once we're more sure this all works.
Testing
I tested these wheels on my local mac and on psbuild-rocky9, followed by
import pyca; print(pyca.__file__)(the ci jobs do this inside each build matrix element, but had to double check