Pretrained SuperAnimal Models Support, Multi-Animal support, DLC 3.x Compatibility, and Docker Testing

[maryapp]

Add Pretrained SuperAnimal Models Support, Multi-Animal support, DLC 3.x Compatibility, and Docker Testing

Closes:

• Support multi-animal pretrained models in DataJoint DeepLabCut Element datajoint/element-deeplabcut#141
• Multi-Animal Individual Support in PoseEstimation datajoint/element-deeplabcut#143
• Docker Support for Testing and Development (client) datajoint/element-deeplabcut#144

Summary

This PR introduces full support for DeepLabCut 3.x, multi-animal and SuperAnimal pretrained models, via PretrainedModel registry, unified trained/pretrained inference flows, and complete Docker-based testing infrastructure.

No parallel schema is introduced — pretrained and trained models are handled as two modes of the same workflow.
All existing behavior remains fully backward compatible.

---

What’s New

Pretrained / SuperAnimal Model Support

PretrainedModel Lookup Table

A registry of SuperAnimal pretrained models with fields

Helper methods:

• PretrainedModel.populate_common_models() — auto-register quadruped, topviewmouse, etc.
• PretrainedModel.is_pretrained(...)
• PretrainedModel.add(...) — manual registration.

Integration with Model

• Model.insert_pretrained_model()
  Allows inserting a pretrained model without a DLC project directory or config.yaml.
• Existing Model schema remains unchanged.
• Pretrained models are detected via metadata (e.g., _pretrained_model_name).

---

Unified Inference Path (Trained + Pretrained)

PoseEstimation.make() now includes:

• Trained path: existing do_trained(...)
• Pretrained path: _do_pretrained_inference(...)

_do_pretrained_inference(...) includes:

• Reading metadata from PretrainedModel
• Using deeplabcut.video_inference_superanimal(...) when available
• Falling back to video_inference(...) when needed
• Filtering kwargs with inspect.signature() to match DLC APIs
• Handling pretrained-specific params (e.g. scale, video, detector settings)

Thus trained and pretrained workflows share the same high-level pipeline.

---

🐭 Multi-Animal / DLC 3.x Compatibility

New Part tables:

Table	Type	Description
PoseEstimation.Individual	Part	Tracks individuals (e.g., animal0, animal1)
PoseEstimation.IndividualMapping	Part	Maps body parts to individuals

Additional updates:

• dlc_reader.py now parses DLC 3.x / SuperAnimal output
• Full support for multi-animal scoremaps and identity tracking

Backward compatible:
For single-animal projects these tables remain empty.

---

🐳 Docker Testing Infrastructure

New cross-platform testing environment:

• Dockerfile — based on official DeepLabCut image
• docker-compose.yaml — MySQL + client container
• Makefile with shortcuts:
  • make test-trained
  • make test-pretrained

Ensures reproducible end-to-end testing for both workflows.

---

Functional Smoke Tests

New tests (and workflow demo):

• test_trained_inference.py — trained model pipeline
• test_video_inference.py — pretrained SuperAnimal pipeline

Documentation added:

• CONDA_ENV_SETUP.md
• DOCKER_TEST_README.md
• tests/TESTING_GUIDE.md

Backward Compatibility

• Existing trained-model workflows are unchanged
• No schema migration required
• Single-animal projects behave exactly as before
• Pretrained workflows are opt-in
• Multi-animal tables remain empty unless DLC 3.x data is used

[maryapp]

TODO:

• Investigate the bug where a file appears as fully changed in Git (red/green diff), likely caused by inconsistent line-ending formatting, and implement a proper fix!

[Copilot]

Pull request overview

This PR introduces comprehensive support for DeepLabCut 3.x, pretrained SuperAnimal models, multi-animal tracking, and Docker-based testing infrastructure. The changes enable two distinct workflow modes: trained models (custom models with user-labeled data) and pretrained models (ready-to-use SuperAnimal models from the DLC Model Zoo). The implementation maintains full backward compatibility with existing trained model workflows while adding new capabilities for pretrained inference and multi-animal pose estimation.

Key Changes:

• Added PretrainedModel lookup table registry for SuperAnimal models
• Implemented unified inference pipeline supporting both trained and pretrained workflows
• Added multi-animal support with new Individual and IndividualMapping part tables
• Upgraded to DeepLabCut 3.x (PyTorch) with Python 3.10+ requirement
• Added comprehensive Docker-based testing infrastructure
• Created functional smoke tests for both workflows

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 20 comments.

Show a summary per file

File	Description
tests/test_pretrained_workflow.py	Unit tests for pretrained model registration, insertion, validation, and parameter merging
tests/TESTING_GUIDE.md	Comprehensive testing guide with setup instructions and examples
test_video_inference.py	Functional smoke test for pretrained SuperAnimal inference workflow
test_trained_inference.py	Functional smoke test for complete trained model workflow with mocked training
setup.py	Updated DLC dependencies to version 3.0.0rc13 with Python 3.10+ requirement
environment.yml	New conda environment file with DLC 3.x and Python 3.10
element_deeplabcut/train.py	Enhanced training logic with DLC 3.x compatibility and metadata handling
element_deeplabcut/readers/dlc_reader.py	Extended reader to support DLC 3.x multi-animal format and lenient metadata parsing
element_deeplabcut/model.py	Added PretrainedModel table, insert_pretrained_model() method, multi-animal tables, and unified inference
docs/src/workflows.md	New documentation explaining trained vs pretrained workflow modes
docs/src/testing.md	Testing guide covering functional tests and Docker setup
docs/src/docker.md	Docker setup and usage documentation
docker-compose.yaml	Docker Compose configuration for database and client services
Dockerfile	Dockerfile based on official DeepLabCut image
Makefile	Convenient shortcuts for Docker operations

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.