Skip to content

Conversation

@btmonier
Copy link
Member

Summary

This PR refactors the orchestration pipeline to use direct Clikt command invocations instead of spawning Gradle subprocesses, along with various bug fixes and improvements to CLI parameter handling.

Features

  • Replaced all ProcessRunner.runCommand("./gradlew", "run", ...) calls with direct Clikt command parsing via CommandName().parse(args). This eliminates the overhead of spawning new Gradle/JVM processes for each pipeline step.
  • Added restoreOrchestratorLogging() to maintain proper log file separation after each step runs.
  • All paths are now resolved to absolute, normalized paths for consistency. This prevents issues with relative path resolution when working directories change.
  • Changed config section parsing from map?.let { ... } to if (configMap.containsKey(...)) pattern. This allows pipeline steps to run with defaults even when config section is empty/null (i.e., just specifying the key enables the step).
  • Added missing parameter features in downsample_gvcf and covert_to_fasta steps.
  • General "QoL" updates to example pipeline YAML file.

Bug Fixes

  • Fixed biokotlin-tools gzip naming (biokotlin-tools automatically compresses output and adds .gz extension)
  • Temporary directories are now always cleaned up if they exist, not just when files were actively used from them.
  • Added cleanup of ALL existing file appenders when setting up a new log file
  • Prevents log bleeding where messages from step $N + 1$ appear in step $N$'s log file
  • Properly stops old appenders before removing them
  • Corrected binary name issue where "biokotlin" was called instead of "biokotlin-tools"

Breaking Changes

  • None

Checklist

  • I have updated the version in build.gradle.kts (REQUIRED - see below)
  • I have tested these changes locally
  • I have added/updated tests for new functionality
  • I have updated documentation (if applicable)
  • Breaking changes are clearly documented above

@btmonier btmonier self-assigned this Dec 17, 2025
@btmonier btmonier changed the title Fix clikt cli General CLI fixes when building clikt executable Dec 17, 2025
convert_to_fasta:
missing_records_as: "asRef" # Optional: Missing records (asN, asRef, asNone)
missing_genotype_as: "asN" # Optional: Missing genotypes (asN, asRef, asNone)
ignore_contig: "__NO_MATCH_" # Optional: Comma-separated patterns to skip (contigs matching will be ignored) (currently needed)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this have 2 underscores at the end so it matches ignore_contig?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

@btmonier btmonier merged commit 762ff09 into main Dec 17, 2025
1 check passed
@btmonier btmonier deleted the fix-clikt-cli branch December 17, 2025 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants