CodeGen: SLM (Small Language Model) Driven Coding

What is CodeGen?

Enhancing the replitv3 3B language model's code generation by ongoing diverse language training, employing fine-tuning for accurate pattern prediction in coding paradigms.

• Tools: FastAPI, Docker (with watchtower enabled), GitHub Actions, React JS, HTML, CSS, Vanilla JS,Redis.
• Technologies: OAuth, JWT, Google Sign-In
• Machine Learning Models: Replitv3 3B (3B parameter Causal Language Model focused on CodeInstruction, employing ggml quantization for faster inference)

Key Features 🎯

• ✅ Fast & Efficient: Models are qunatized using ggml to 4bit to provide faster inference on edge device
• ✅ Secure: Made with security in mind, Your data your control.
• ✅ Docker Support: Comes with docker support, which makes it cross-platform and can be run on any os
• ✅ Offline Mode: Works in offline Mode, so that your data is with you only.
• ✅ Redis Chat history: Comes with chat history support so that you can navigate to your queries. [Collaboration with Heta Vyas]

Future ideas & implementation

• ✅ Content Moderation: Controlling the behaviour of model to not provide harmful asnwers & query, and also it should only provide code.
• ✅ Authentication & Security [Collaboration with Heta Vyas]
• ✅ Cloud Storage of Chat history: Provide a cloud storage for authenticated users. [Collaboration with Heta Vyas]

Demo Highlights 🎥

Two main endpoint that server run

• localhost:8000/autocomplete
• localhost:8000/codegen

Setup

Step one with python locally

First create a venv.

python -m venv env && source env/bin/activate

Next step open the environment in powershell of vscode

./env/Scripts/Activate.ps1

Next install the requirements

pip install -r requirements.txt

Next download the model weight

python ./api/download_model.py

Now run the server.py

uvicorn server:app --reload

Run using Docker

docker-compose build

docker-compose up