Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 8 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,24 +18,23 @@ Understanding human behavior requires measuring behavioral actions. Due to its c
- This repository contains the implementation for our preprint on evaluating and training multi-modal large language models for action recognition.
- Our code is built on [LLaVA-NeXT](https://github.com/LLaVA-VL/LLaVA-NeXT), and files in the directory `llavaction/action` are related to our work. We thank the authors of LLaVA-NeXT for making their code publicly available.
- The files in the `/eval`, `/model`, `/serve` and `/train` are directly from [LLaVA-NeXT](https://github.com/LLaVA-VL/LLaVA-NeXT), unless modified and noted below.
- Modified files are:
- - /model/llava_arch.py
- - /model/language_model/llava_qwen.py
- - /train/train.py
- - /train/llava_trainer.py
- - /utils.py
- - A diff can be generated against the commit (79ef45a6d8b89b92d7a8525f077c3a3a9894a87d) of LLaVA-NeXT to see our modifications.
- `/model/llava_arch.py`
- `/model/language_model/llava_qwen.py`
- `/train/train.py`
- `/train/llava_trainer.py`
- `/utils.py`

## Demo
- Currently, we provide code to run video inference in a Jupyter Notebook (which can be run on Google Colaboratory).
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/AdaptiveMotorControlLab/LLaVAction/blob/main/example/llavaction_video_demo.ipynb)
We provide code to run video inference in a Jupyter Notebook (which can be run on Google Colaboratory).


### Installation guide for video inference:
```bash
conda create -n llavaction python=3.10 -y
conda activate llavaction
pip install --upgrade pip # Enable PEP 660 support.
pip install -e .
pip install --pre llavaction
```

- Please see the `/example` directory for a demo notebook.
Expand Down
Loading