Running tests in parallel #153
Description
Is your feature request related to a problem? Please describe.
I'd like to parallelize the execution of most of my tests in order to speed up their execution.
To be more specific I've a series of tests that heavily bottleneck on IO not only during execution but also in their fixtures. While an asynchronous IO runtime on the Java-side would be best I don't think that's going to be viable for a few years at least, so having multiple threads to nanny the syscalls is the next best thing and the only alternative to projects who are actually bottlenecking on CPU instead.
Describe the solution you'd like
To be able to set metadata to specific deftests as well as ns declarations (which should be equivalent to marking all of its deftests as parallel), to indicate that these tests can become part of a grand pool of tests that can be executed in parallel by a group of workers before the synchronized tests run.
Nice to haves:
- Being able to specify a custom worker cap as a system property, with a sensible default like the system's virtual core count.
- Buffered output when running Poly CLI for extra clarity, meaning results would only be printed once all tests for a given namespace are done in case they're all parallel, or waiting until the beginning of the namespace's synchronous execution phase to print the results of its parallel tests right before executing the synchronous ones. Failures could be shown when they occur and releasing the buffer for its namespace then and there, stopping execution just like poly test currently does for synchronous tests.
Describe alternatives you've considered
- Plug-in API for custom test runners. -- Feels overkill, and likely to introduce more complexity than having a single test runner with even the superset of all popular test runner features.
- Testing-specific CLI alternative to poly. -- It would likely lag behind Polylith if maintained out of tree and cause a lot more of maintenance effort.