18311 Robot Learning Final Project

Adding Quadratic Structure to SAC

sac.py contains the baseline sac implementation. sac_lqr.py is the version after adding quadratic structure.

---

Setup

1. Install UV Package Manager

curl -LsSf https://astral.sh/uv/install.sh | sh

Important: Restart your terminal after installation.

2. Clone Repository

git clone https://github.com/jacoblgit/actorcritics.git
cd actorcritics

3. Install Dependencies

# Core dependencies
uv sync

# Development tools (ruff linter/formatter)
uv sync --extra dev

---

Usage

Run Training

uv run python sac_lqr.py --help
uv run python sac_lqr.py --env-id InvertedPendulum-v4 

Loss type

Use max_eigenvalue (corresponding to constraining the max eigen value to stablize runs) with coef 0.001 for combined run

uv run python sac_lqr.py (... other flags) --loss-type max_eigenvalue --loss-coef 0.001

For more details, checkout:

• eigen_losses/README.md
• Wandb Report

View Training Data

uv run tensorboard --logdir runs/

Code Quality

# Check code
uv run ruff check

# Format code
uv run ruff format

Add Dependencies

uv add <package-name>

---

Reference

Environment setup based on Modern Python Practices

Baseline SAC Implementation is CleanRL SAC

Info on SAC: Soft Actor-Critic — Spinning Up documentation

Info on LQR: Ch. 8 - Linear Quadratic Regulators