Development Setup¶

Setting up fetchly for local development, outside the container.

Requirements¶

Essentia and beat_this (for BPM analysis) are optional in a dev environment; the app runs without them and simply skips analysis.

Clone and install¶

git clone https://github.com/Gill-Bates/fetchly.git
cd fetchly

python3.13 -m venv .venv
source .venv/bin/activate

pip install --upgrade pip
pip install --extra-index-url https://download.pytorch.org/whl/cpu -e ".[dev]"
pip install yt-dlp yt-dlp-ejs

pyproject.toml is the single manifest: it holds the release version, the runtime dependencies, and the dev and docs extras. It replaced VERSION, requirements.txt and docs/requirements-docs.txt — a change to any of those three now happens in one file, and app/utils/version.py reads [project] version straight back out at runtime.

--extra-index-url picks the CPU build of PyTorch (beat_this's dependency); without it pip resolves the CUDA wheels and pulls in gigabytes of GPU code the app never runs. yt-dlp and yt-dlp-ejs stay outside the manifest on purpose, since yt-dlp updates on its own cadence as platforms change.

The runtime dependencies carry no version pins, so this install takes the newest resolvable set — the same rule a release follows. The dev and docs extras are the exception: their tooling never enters an image, and a ruff or mkdocs release that changes its own rules should be adopted in its own commit rather than reddening CI on untouched code.

Need a flat requirements file — for a tool that only speaks -r, or for a scan? Generate it instead of keeping a second copy:

python tools/pyproject-deps.py > requirements.txt          # runtime
python tools/pyproject-deps.py docs > requirements-docs.txt
python tools/pyproject-deps.py dev                          # pytest + ruff

Run it¶

export FETCHLY_SECRET_KEY="$(openssl rand -base64 32)"
export LOG_LEVEL=debug
export UVICORN_RELOAD=1
export UVICORN_RELOAD_EXCLUDES="data,.git"

python run.py

UVICORN_RELOAD=1 restarts on code changes. Exclude data/ and .git/ from the watch — the database and job files change constantly and would otherwise trigger reload loops.

Data lands in data/ relative to the working directory unless DATA_DIR says otherwise.

Running the tests¶

pytest

pytest picks up its configuration from [tool.pytest.ini_options] in pyproject.toml. Tests live in tests/ — one file per module under test, largely mirroring app/. Some exercise real subprocess calls (ffmpeg) and are naturally slower.

JavaScript tests¶

From the repository root:

npm install
npm ci --prefix tools/ui-lint   # once — pulls in Playwright, used by one of the tests
npm test

tests/js/*.test.mjs covers front-end contracts — the CSRF token helper, the confirmation modal, the cookie-paste dialog, safe-redirect handling — as plain Node tests, no browser required. tests/js/ui-lint-devices.test.mjs is the exception: it imports tools/ui-lint/run-ui-lint.mjs, which depends on Playwright, so npm test fails with ERR_MODULE_NOT_FOUND until the tools/ui-lint dependencies are installed.

Note the glob: node --test tests/js/ (the directory form) needs Node 24, so the npm test script spells out node --test "tests/js/*.test.mjs" instead.

Linting¶

Three linters:

npm run lint runs both front-end linters; npm run lint:fix applies what is safely fixable. All three run on every push and pull request via .github/workflows/ci.yml.

A few rules encode project decisions rather than style:

• No native browser dialogs. ESLint's no-alert and no-restricted-globals fail the build on confirm()/alert()/prompt(). Use confirmModal() from app/static/js/confirm.js. The single sanctioned exception is inside confirm.js itself, on the branch where Bootstrap failed to load.
• No 100vh without a 100dvh companion. Stylelint warns, because Safari's collapsing toolbar makes 100vh overflow on iPhone and in iPad Split View.
• -webkit-* prefixes are kept. property-no-vendor-prefix is off: those prefixes are load-bearing on iOS and iPadOS, not legacy.

Browser audit (ui-lint)¶

tools/ui-lint drives the running app through Playwright and reports layout, accessibility, contrast and iOS/iPadOS problems. It needs a live server, so it is a local pre-release check rather than part of the PR gate:

npm run ui-lint:install   # once: installs Playwright + Chromium and WebKit
python run.py &           # the app must be reachable
npm run ui-lint

It audits five device profiles, chosen around the breakpoint in app/static/style.css where the desktop jobs table gives way to the mobile feed (@media (max-width: 1024px)):

Form factor and touch are separate axes in the runner, which is what tablet-wide exists to prove: it renders the desktop table and needs 44px hit areas. Audits are gated on the axis they actually depend on — isMobile for which DOM renders, isTouch for hit areas and Safari's viewport quirks, isPhone for the pixel contracts written against 390px.

Results land in results.json under the output directory the run prints, and the process exits non-zero when any hard failure is found.

Code style¶

• Python: type-annotated, from __future__ import annotations where the codebase uses it, docstrings that explain why a design choice was made, not just what the code does
• No native browser dialogs (confirm/alert/prompt) — use the shared confirmModal() in app/static/js/confirm.js
• New settings keys go through the allow-list in app/db.py (_SETTINGS_DEFAULTS, _SETTINGS_TYPES), never accepted unchecked

Building the Docker image locally¶

DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile -t fetchly:dev .

The Dockerfile is multi-stage; expect a longer build on the first run while the ffmpeg and essentia stages compile. That applies on amd64 as much as on arm64: essentia is built from source on both, so that a multi-arch release ships the same version of it everywhere rather than a PyPI wheel on one architecture and a source build on the other. Upstream publishes no linux/aarch64 wheel at all, which is why there is no version to take instead. A plain docker build compiles master; ESSENTIA_REF selects any commit.

Nothing else is pinned either — not the base image, not ffmpeg, not deno, not yt-dlp, and not the Python dependencies. A local build therefore takes whatever is newest at the moment each layer runs. A release must not, because its two architecture jobs run on separate runners and would each resolve "newest" for themselves, so .github/workflows/docker-build.yml resolves every one of those once — the base image by digest, essentia to a commit, ffmpeg to an immutable dated BtbN release, the dependency set to a resolved constraints.txt — and passes the same values to both builds. Before the manifest is published it compares every version present in the two finished images and refuses to push an index whose halves disagree.

Project layout¶

app/
├── main.py              # FastAPI app, lifespan, background tasks
├── routes/               # One module per route family
├── worker.py             # Download/transcode worker threads
├── analysis_worker.py    # BPM analysis process pool
├── bpm*.py                # Tempo detection cascade
├── governor.py           # Resource sizing and semaphores
├── db.py                 # SQLite schema, settings, queries
├── session.py             # Session cookie signing and validation
├── lalal*.py               # Lalal.ai client and product rules
├── utils/                # Focused single-purpose helpers
├── templates/             # Jinja2 templates
└── static/                # CSS, JS, vendored assets
middleware/
└── csrf.py                # Double-submit CSRF middleware
tests/
├── test_*.py               # pytest, one file per module under test
└── js/*.test.mjs            # front-end contract tests

See Architecture for how these pieces fit together.

Where to next¶