🎯 Nim — Learn Reinforcement Learning by Playing

A web app where you play Nim against a computer… and watch an AI teach itself to play it, game after game, through reinforcement learning.

Built for the Algorithms 2 course at Polytech Nice Sophia, in partnership with Terra Numérica (a science-outreach centre). The goal: make a notoriously abstract idea — reinforcement learning — concrete and hands-on, including for high-school students.

---

▶️ Play / Try it

🌐 Online — ▶ Play the live demo

Hosted free on Render. The app sleeps after ~15 min of inactivity, so the first visit may take a few seconds to wake up — then it's instant.

💻 Locally, in one command (Docker)

Requirements: Docker + Docker Compose + Make.

make up

Then open http://localhost:3000 — and play. To stop: make down.

📦 Locally without Compose (single image)

Handy for testing the exact image that gets deployed online:

docker build -f Dockerfile.web -t nim-web .
docker run -p 7860:7860 nim-web

Then open http://localhost:7860.

---

✨ What the app does

The interface is organised into four tabs:

Tab	What you do there
🎮 Play	A game of Nim against the computer (or two players). You choose the number of sticks, the allowed moves, and the opponent type (random, optimal, or trained AI).
🧠 Train the AI	Launch a reinforcement-learning run and watch the AI improve live: win rate, exploration, learning curves.
🧊 3D Visualization	A 3D view of how the AI's policy converges over successive games.
📚 Understand the AI	An interactive guide and a hands-on workshop to grasp what reinforcement learning is, with no prerequisites.

---

🧠 How the AI learns (in two minutes)

Nim is a subtraction game: you start from a pile of sticks, each player removes 1 to 3 on their turn, and whoever takes the last stick wins (the "normal" variant). There is a perfect strategy… but here we don't hand it to the AI: it discovers it on its own.

The AI plays thousands of games. In each one:

• it tries moves (sometimes at random — that's exploration);
• when a sequence of moves leads to a win, those moves are rewarded; when it leads to a loss, they are penalised;
• over many games, it reinforces good moves and drops the bad ones.

Little by little, without ever being told the winning rule, the AI starts playing near-optimally. That's the whole educational point: you can see the learning build up. Training uses a curriculum (opponents of increasing difficulty) to speed up and stabilise convergence.

---

🚀 Deploy online for free

The app collapses into a single container (the FastAPI backend serves both the API and the frontend, via Dockerfile.web), which makes it free to host.

Option A — Render (recommended, one click)

The render.yaml file is already set up.

1. Create a free account on render.com (no card needed).
2. New → Blueprint, connect this GitHub repo — Render reads render.yaml.
3. A few minutes later: a public URL https://<name>.onrender.com.
4. Paste that URL into the Live demo section above.

Free plan: the app sleeps after ~15 min of inactivity and wakes on the first request (a few seconds). Perfect for a demo.

Option B — Hugging Face Spaces

Create a Docker Space, upload the repo contents with Dockerfile.web renamed to Dockerfile, and expose port 7860. Free, permanent public URL.

---

🏗️ Architecture

The frontend calls the backend using a relative path (/api), which supports both the dev setup (separate nginx + backend) and the "single image" setup.

Backend (Python / FastAPI)

backend/app/
├── main.py                 # FastAPI routes (game + RL training)
├── serve_all.py            # All-in-one server (API + static, one port)
├── ai/
│   └── rl_trainer.py       # Reinforcement-learning training logic
├── domain/
│   ├── nim/logic.py        # Nim rules, bot move, response format
│   └── rl/curriculum.py    # Curriculum and opponent selection
└── config/rl_config.json   # RL hyperparameters

Frontend (JavaScript, HTML/CSS)

frontend/site/js/
├── app/main.js             # Application bootstrap
├── core/{config,api}.js    # Config + HTTP calls to /api
├── shared/{ui,animation}.js
└── sections/
    ├── play/game.js        # Play tab
    ├── train/{training,metrics}.js
    ├── viz/visualization.js
    └── learn/{guide,rl_activity}.js

---

✅ Tests

pip install -r backend/requirements-dev.txt
pytest backend/tests/test_api.py

---

🛠️ Self-hosted production deployment (with a Docker registry and a server)

Follow these steps as-is after a git clone. Replace <REGISTRY/NAMESPACE> with the URL and namespace of your target registry.

1. Image variables

export IMAGE_REGISTRY=<REGISTRY/NAMESPACE>
export IMAGE_TAG=1.0.0
export BACKEND_IMAGE=${IMAGE_REGISTRY}/nim-game-learning-app-backend:${IMAGE_TAG}
export FRONTEND_IMAGE=${IMAGE_REGISTRY}/nim-game-learning-app-frontend:${IMAGE_TAG}

2. Build & push

make build
make push

3. Prepare deployment artifacts

make package-deploy   # generates dist/deploy/{docker-compose.yaml,.env.example}

4. Copy to the server

scp dist/deploy/docker-compose.yaml user@server:/opt/nim-game-learning-app/
scp dist/deploy/.env.example        user@server:/opt/nim-game-learning-app/.env

5. Adapt the .env on the server

BACKEND_IMAGE=<REGISTRY/NAMESPACE>/nim-game-learning-app-backend:1.0.0
FRONTEND_IMAGE=<REGISTRY/NAMESPACE>/nim-game-learning-app-frontend:1.0.0
FRONTEND_PORT=3000
CORS_ORIGINS=https://<PUBLIC_APP_URL>

6. Launch and verify

cd /opt/nim-game-learning-app
docker compose --env-file .env -f docker-compose.yaml up -d
curl -fsS http://<HOST>:3000/api/health

Notes — The frontend acts as the single public gateway; the backend is not exposed directly (reachable via /api). Logs go to stdout/stderr (docker compose logs). The placeholders in deploy/.env.example are intentional and must be replaced on the target server.

---

👥 Authors

Team project — Arij Lemjid, Maud Marconcini, Rayan Outili. Polytech Nice Sophia · Algorithms 2 · in partnership with Terra Numérica.