Development Setup¶

Setting up fetchly for local development, outside the container.

Requirements¶

Essentia and beat_this (for BPM analysis) are optional in a dev environment; the app runs without them and simply skips analysis.

Clone and install¶

git clone https://github.com/Gill-Bates/fetchly.git
cd fetchly

python3.13 -m venv .venv
source .venv/bin/activate

pip install --upgrade pip
pip install --extra-index-url https://download.pytorch.org/whl/cpu -e ".[dev]"
pip install yt-dlp yt-dlp-ejs

pyproject.toml is the single manifest: it holds the release version, the runtime dependencies, and the dev and docs extras. It replaced VERSION, requirements.txt and docs/requirements-docs.txt — a change to any of those three now happens in one file, and app/utils/version.py reads [project] version straight back out at runtime.

--extra-index-url picks the CPU build of PyTorch (torch and torchaudio); without it pip resolves the CUDA wheels and pulls in gigabytes of GPU code the app never runs. pip has no notion of index priority, so that flag opens the index for every package in the install — acceptable for a dev environment, but not for a release. uv users get the narrow version for free: pyproject.toml declares that index explicit and binds only torch and torchaudio to it, which is the same separation the release resolution and the Dockerfile use. yt-dlp and yt-dlp-ejs stay outside the manifest on purpose, since yt-dlp updates on its own cadence as platforms change.

The runtime dependencies carry no version pins, so this install takes the newest resolvable set — the same rule a release follows. The dev and docs extras are the exception: their tooling never enters an image, and a ruff or mkdocs release that changes its own rules should be adopted in its own commit rather than reddening CI on untouched code.

Need a flat requirements file — for a tool that only speaks -r, or for a scan? Generate it instead of keeping a second copy:

python tools/pyproject-deps.py > requirements.txt          # runtime
python tools/pyproject-deps.py docs > requirements-docs.txt
python tools/pyproject-deps.py dev                          # pytest + ruff

Run it¶

export FETCHLY_SECRET_KEY="$(openssl rand -base64 32)"
export LOG_LEVEL=debug
export UVICORN_RELOAD=1
export UVICORN_RELOAD_EXCLUDES="data,.git"

python run.py

UVICORN_RELOAD=1 restarts on code changes. Exclude data/ and .git/ from the watch — the database and job files change constantly and would otherwise trigger reload loops.

Data lands in data/ relative to the working directory unless DATA_DIR says otherwise.

Running the tests¶

pytest

pytest picks up its configuration from [tool.pytest.ini_options] in pyproject.toml. Tests live in tests/ — one file per module under test, largely mirroring app/. Some exercise real subprocess calls (ffmpeg) and are naturally slower.

JavaScript tests¶

From the repository root:

npm install
npm ci --prefix tools/ui-lint   # once — pulls in Playwright, used by one of the tests
npm test

tests/js/*.test.mjs covers front-end contracts — the CSRF token helper, the confirmation modal, the cookie-paste dialog, safe-redirect handling — as plain Node tests, no browser required. tests/js/ui-lint-devices.test.mjs is the exception: it imports tools/ui-lint/run-ui-lint.mjs, which depends on Playwright, so npm test fails with ERR_MODULE_NOT_FOUND until the tools/ui-lint dependencies are installed.

Note the glob: node --test tests/js/ (the directory form) needs Node 24, so the npm test script spells out node --test "tests/js/*.test.mjs" instead.

Linting¶

Three linters:

npm run lint runs both front-end linters; npm run lint:fix applies what is safely fixable. All three run on every push and pull request via .github/workflows/ci.yml.

A few rules encode project decisions rather than style:

• No native browser dialogs. ESLint's no-alert and no-restricted-globals fail the build on confirm()/alert()/prompt(). Use confirmModal() from app/static/js/confirm.js. The single sanctioned exception is inside confirm.js itself, on the branch where Bootstrap failed to load.
• No 100vh without a 100dvh companion. Stylelint warns, because Safari's collapsing toolbar makes 100vh overflow on iPhone and in iPad Split View.
• -webkit-* prefixes are kept. property-no-vendor-prefix is off: those prefixes are load-bearing on iOS and iPadOS, not legacy.

Browser audit (ui-lint)¶

tools/ui-lint drives the running app through Playwright and reports layout, accessibility, contrast and iOS/iPadOS problems. It needs a live server, so it is a local pre-release check rather than part of the PR gate:

npm run ui-lint:install   # once: installs Playwright + Chromium, WebKit and Firefox
python run.py &           # the app must be reachable
npm run ui-lint

It audits six device profiles, chosen around the breakpoint in app/static/style.css where the desktop jobs table gives way to the mobile feed — @media (max-width: 1024px) for any pointer, plus (max-width: 1366px) and (pointer: coarse) so every touch iPad up to a 12.9" Pro in landscape gets the feed:

Form factor and touch stay separate axes in the runner. isMobile (which DOM renders) keys off width and touch — a touch iPad past 1024px still gets the feed, a hypothetical non-touch window that wide would not. isTouch (44px hit areas, Safari's viewport quirks) keys off touch alone, and isPhone gates the pixel contracts written against 390px.

desktop-firefox holds the viewport identical to desktop and varies only the engine, so anything it finds is a Gecko difference rather than a layout one. An engine that is not installed is skipped with the reason instead of failing the run; those views report SKIP, never PASS. That tolerance applies only to the default. Once UI_LINT_BROWSERS names an engine, a launch failure of that engine makes the run exit non-zero, because coverage that was asked for did not happen.

Beyond the layout checks, each view runs the axe-core WCAG 2.1 A/AA ruleset, records cumulative layout shift (Chromium only — WebKit and Firefox report unsupported, never a zero), and gets a weighted health score out of 100.

Both health knobs are off by default: the score is new, and turning it into a gate before the baseline is agreed would only teach everyone to set it back to zero.

Results land in results.json under the output directory the run prints, and the process exits non-zero when any hard failure is found or an explicitly named engine could not be launched.

Code style¶

• Python: type-annotated, from __future__ import annotations where the codebase uses it, docstrings that explain why a design choice was made, not just what the code does
• No native browser dialogs (confirm/alert/prompt) — use the shared confirmModal() in app/static/js/confirm.js
• New settings keys go through the allow-list in app/db.py (_SETTINGS_DEFAULTS, _SETTINGS_TYPES), never accepted unchecked

Building the Docker image locally¶

DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile -t fetchly:dev .

The Dockerfile is multi-stage; expect a longer build on the first run while the ffmpeg and essentia stages compile. That applies on amd64 as much as on arm64: essentia is built from source on both, so that a multi-arch release ships the same version of it everywhere rather than a PyPI wheel on one architecture and a source build on the other. Upstream publishes no linux/aarch64 wheel at all, which is why there is no version to take instead. A plain docker build compiles master; ESSENTIA_REF selects any commit.

Nothing else is pinned either — not the base image, not ffmpeg, not deno, not yt-dlp, and not the Python dependencies. A local build therefore takes whatever is newest at the moment each layer runs. A release must not, because its two architecture jobs run on separate runners and would each resolve "newest" for themselves, so .github/workflows/docker-build.yml resolves every one of those once — the base image by digest, essentia to a commit, ffmpeg to an immutable dated BtbN release, the dependency set to a resolved constraints.txt — and passes the same values to both builds. Before the manifest is published it compares every version present in the two finished images and refuses to push an index whose halves disagree.

Project layout¶

app/
├── main.py              # FastAPI app, lifespan, background tasks
├── routes/               # One module per route family
├── worker.py             # Download/transcode worker threads
├── analysis_worker.py    # BPM analysis process pool
├── bpm*.py                # Tempo detection cascade
├── governor.py           # Resource sizing and semaphores
├── db.py                 # SQLite schema, settings, queries
├── session.py             # Session cookie signing and validation
├── lalal*.py               # Lalal.ai client and product rules
├── utils/                # Focused single-purpose helpers
├── templates/             # Jinja2 templates
└── static/                # CSS, JS, vendored assets
middleware/
└── csrf.py                # Double-submit CSRF middleware
tests/
├── test_*.py               # pytest, one file per module under test
└── js/*.test.mjs            # front-end contract tests

See Architecture for how these pieces fit together.

Where to next¶