Maintenance: Fix bugs, reduce duplication, and optimize scripts

[Ven0m0]

This PR addresses several maintenance tasks:

1. Refactoring Dup.py: Fixed a bug where only one duplicate file was reported instead of the full list. Optimized performance by reusing the process pool and consolidated logic for finding candidates.
2. Refactoring git-fetch.py: Reduced code duplication by extracting a process_downloads helper function. Fixed a bug where downloading a repo with files at the root (and no subdirectories) would fail because the output directory wasn't created.
3. Optimizing clean.sh: Improved performance by dynamically determining the number of parallel jobs using nproc. Fixed potential errors in get_cache_size and cleaned up variable declarations in clean_pkgs.
4. Fixing cia_3ds_decryptor.py: Corrected a syntax error in regex matching that prevented the script from running.

---

PR created automatically by Jules for task 6786540254509387207 started by @Ven0m0

[google-labs-jules]

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

---

For security, I will only act on instructions from the user who triggered this task.

[gemini-code-assist]

Summary of Changes

Hello @Ven0m0, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses several key maintenance tasks across various utility scripts. It focuses on resolving existing bugs, enhancing script performance through refactoring and optimization, and improving overall code quality and robustness. The changes aim to ensure more reliable operation and greater efficiency for file duplication detection, GitHub/GitLab fetching, and system cleaning processes.

Highlights

• Dup.py Refactoring: Fixed a bug where only one duplicate file was reported, optimized performance by reusing the process pool, and consolidated logic for finding candidates.
• git-fetch.py Refactoring: Reduced code duplication by extracting a process_downloads helper function and fixed a bug where downloading root-level files would fail due to missing output directory creation.
• clean.sh Optimization: Improved performance by dynamically determining the number of parallel jobs using nproc, fixed potential errors in get_cache_size, and cleaned up variable declarations.
• cia_3ds_decryptor.py Fix: Corrected a syntax error in regex matching that previously prevented the script from running.

Changelog

• Cachyos/Scripts/WIP/emu/cia_3ds_decryptor.py
  • Corrected a typo in accessing a regex match group from m. group(1) to m.group(1).
• Cachyos/Scripts/WIP/gh/git-fetch.py
  • Extracted the concurrent download logic into a new process_downloads helper function to reduce code duplication.
  • Updated fetch_github and fetch_gitlab functions to utilize the newly created process_downloads helper.
  • Added output.mkdir(parents=True, exist_ok=True) in fetch_github to ensure the target directory exists before files are downloaded, preventing errors for root-level files.
• Cachyos/Scripts/WIP/gphotos/Dup.py
  • Removed a debug print statement from the hash_file function.
  • Renamed get_candidates_from_size_groups to get_candidates and generalized its functionality to work with any group of paths.
  • Modified group_by_hash to accept an optional pool argument, allowing for the reuse of a multiprocessing pool.
  • Updated find_duplicate_photos to reuse the process pool across partial and full hashing steps, improving performance.
  • Corrected the output logic in find_duplicate_photos to write all duplicate file paths for a given hash, rather than just the first one.
• Cachyos/clean.sh
  • Refactored the get_cache_size function to use an if/else block for clearer directory existence checks and error handling.
  • Declared pacman_after, paru_after, yay_after, and total_freed variables earlier in the clean_pkgs function for improved variable scoping.
  • Modified the SQLite database optimization step to dynamically determine the number of parallel jobs using nproc instead of a fixed value of 4, enhancing performance based on available CPU cores.

Activity

• No human activity has been recorded on this pull request since its creation.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[codacy-production]

Codacy's Analysis Summary

3 new issues (≤ 0 issue)
0 new security issue
-5 complexity
0 duplications

Review Pull Request in Codacy →

✨ AI Reviewer available: add the codacy-review label to get contextual insights without leaving GitHub.

[gemini-code-assist]

Code Review

This pull request introduces a good set of maintenance changes, including bug fixes, performance optimizations, and code refactoring to reduce duplication. The changes in Dup.py to fix the duplicate reporting and reuse the process pool are excellent. The refactoring in git-fetch.py is also a good improvement. I have one suggestion for git-fetch.py to improve error handling and ensure that failures are correctly reported.

[Copilot]

Pull request overview

This PR addresses maintenance tasks across multiple scripts, focusing on bug fixes, code deduplication, and performance optimizations.

Changes:

• Fixed critical bugs in duplicate file detection (Dup.py) and directory creation (git-fetch.py)
• Improved performance through dynamic parallelism (clean.sh) and process pool reuse (Dup.py)
• Reduced code duplication by extracting common download logic (git-fetch.py)
• Corrected syntax error preventing script execution (cia_3ds_decryptor.py)

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated no comments.

File	Description
Cachyos/clean.sh	Refactored get_cache_size to use explicit conditionals, consolidated variable declarations, and implemented dynamic parallel worker detection using nproc
Cachyos/Scripts/WIP/gphotos/Dup.py	Fixed bug where only one duplicate file was written per hash group; optimized by reusing process pool; consolidated duplicate candidate filtering logic
Cachyos/Scripts/WIP/gh/git-fetch.py	Extracted process_downloads helper to eliminate code duplication; added output directory creation to fix failure when downloading root files
Cachyos/Scripts/WIP/emu/cia_3ds_decryptor.py	Removed erroneous space in m.group(1) that caused syntax error

[Copilot]

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated no new comments.