Skip to content

Support chunked transfers in wget #118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
relic-se wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Support chunked transfers in wget #118

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
571239a
Support chunked transfers in wget
relic-se Dec 9, 2025
cf1ec95
Fix encoding check
relic-se Dec 9, 2025
f790b71
Fix formatting errors
relic-se Dec 9, 2025
c30a111
Merge standard and chunked download process to avoid PLR0915
relic-se Dec 9, 2025
dc7ecff
Fix `OutlinedLabel` deprecation error
relic-se Dec 9, 2025
cdbce86
Compatibility with frozen adafruit_display_text module
relic-se Dec 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion adafruit_portalbase/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,11 @@
from adafruit_bitmap_font import bitmap_font
from adafruit_display_text import wrap_text_to_lines
from adafruit_display_text.bitmap_label import Label
from adafruit_display_text.outlined_label import OutlinedLabel
Copy link
Contributor

@FoamyGuy FoamyGuy Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should wait a bit on changing this. Some of the device specific libraries that are use PortalBase are for devices with Display Text frozen in.

Unless users of those devices put the latest Display Text library in the root of CIRCUITPY, they won't have access to the newest version.

I think this should only change here after there is a new core release that includes the updated Display Text frozen in

Copy link
Contributor

@FoamyGuy FoamyGuy Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does unfortunately mean that when this library is in use on devices without frozen Display Text it will be printing that deprecation message.

Maybe we should disable the warning print for now and only turn it back on after that release is out so that updating is possible / easy for all devices.

Copy link
Contributor Author

@relic-se relic-se Dec 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I should be able to catch an ImportError so that it will work in both scenarios. Normally, I wouldn't want to touch this in the PR, but it was throwing an error in addition to the deprecation warning:

ImportError: cannot import name 'OutlinedLabel' from 'adafruit_display_text.outlined_label' (/opt/hostedtoolcache/Python/3.12.12/x64/lib/python3.12/site-packages/adafruit_display_text/outlined_label.py). Did you mean: 'outlined_label'?

Copy link
Contributor Author

@relic-se relic-se Dec 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know if the most recent commit, cdbce86, works for you.

Copy link
Collaborator

@makermelissa makermelissa Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Portal_Base is also a frozen library. Maybe some code that checks if the Bitmap Label has outline_color or outline_size properties might work well enough.


try:
from adafruit_display_text.outlined_label import OutlinedLabel
except ImportError:
OutlinedLabel = Label

__version__ = "0.0.0+auto.0"
__repo__ = "https://github.com/adafruit/Adafruit_CircuitPython_PortalBase.git"
Expand Down
32 changes: 23 additions & 9 deletions adafruit_portalbase/network.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -333,27 +333,41 @@ def wget(self, url, filename, *, chunk_size=12000, headers=None):

if self._debug:
print(response.headers)
if "content-length" in headers:
content_length = int(headers["content-length"])
else:
raise RuntimeError("Content-Length missing from headers")
remaining = content_length

chunked = (
"content-length" not in headers
and "transfer-encoding" in headers
and headers["transfer-encoding"] == "chunked"
)
if "content-length" not in headers and not chunked:
raise RuntimeError("Invalid headers in response")

print("Saving data to ", filename)
stamp = time.monotonic()
with open(filename, "wb") as file:
content_length = int(headers["content-length"]) if "content-length" in headers else 0
remaining = chunk_size if chunked else content_length
for i in response.iter_content(min(remaining, chunk_size)): # huge chunks!
self.neo_status(STATUS_DOWNLOADING)
remaining -= len(i)
if chunked:
content_length += len(i)
else:
remaining -= len(i)
file.write(i)
if self._debug:
print("Read %d bytes, %d remaining" % (content_length - remaining, remaining))
if chunked:
print("Read %d bytes, %d total" % (len(i), content_length))
else:
print(
"Read %d bytes, %d remaining" % (content_length - remaining, remaining)
)
else:
print(".", end="")
if not remaining:
if not chunked and not remaining:
break
self.neo_status(STATUS_FETCHING)
response.close()

response.close()
stamp = time.monotonic() - stamp
print("Created file of %d bytes in %0.1f seconds" % (os.stat(filename)[6], stamp))
self.neo_status(STATUS_OFF)
Expand Down