# istrain — APOCALYPSE EDITION (public build guide) *The complete from-scratch rebuild. This guide assumes you can install Linux and are comfortable in a terminal, but it spells out every railroad-, SDR-, and container-specific step — because those are the parts nobody tells you. If a step feels obvious to you, skip it; it's here for the person behind you who needs the hint.* *You are building a **passive** RF receiver. It only listens. No transmit, no license required in the US to receive, no railroad cooperation needed — the trains announce themselves on public frequencies, and you just have to be set up to hear them.* > **Two ways to use this repo:** > - **As a human:** read top to bottom, run the commands. > - **As an agent** (point Claude Code / an LLM coding agent at this repo): the whole working tree > is here — `scripts/` (the DSP + decoders), `dashboard/` (the web UI + API server), > `docker/` (the container definitions), `config/` (templates). Every script has a header > comment explaining what it does and how it's operated. The design rules that keep it robust > are in §11. Start by reading this file, then `dashboard/serve.py` and `scripts/iq_hop.py`. > > **Getting the code:** `git clone istrain`. One gotcha worth knowing if this repo > is hosted behind Cloudflare (or a similar bot-shield): a plain browser and normal `git`/`curl` > clone fine, but a *headless* HTTP fetch (some scripts, some agent tool-calls) can get a 403 bot > wall. The fix is a normal browser User-Agent on the request, e.g. > `git -c http.userAgent="Mozilla/5.0" clone ` — or just use SSH, which sidesteps it entirely. > (This is a hosting quirk, not a code quirk; a fork on a plain host won't see it.) --- ## 0. What you're building, in one minute A train is a chain of radio transmitters. istrain listens passively to three of them and fuses the result into one question: **is a train blocking my crossing right now?** 1. **Rail VHF voice (160–162 MHz, narrowband FM, unencrypted).** Crews, dispatchers, and trackside **defect detectors** speaking plain English: *"Norfolk Southern detector, milepost 148.9, no defects, total axles 460, speed 52, detector out."* A detector hit on your road channel means a train just passed that milepost — the most direct "train coming" signal there is, in words. 2. **End-of-Train telemetry (457.9375 MHz, 1200-baud FFSK).** The flashing box on the last car (EOT / FRED / marker) transmits its unit ID, **brake-pipe pressure**, and a **motion flag** every few seconds. Decode it and the air tells you: passing, stopped, brakes cut, or leaving. 3. **Head-of-Train (452.9375 MHz).** The locomotive's half of the same conversation. Decode the EOT unit it's addressing and you can **join head to tail** — proof of a whole train, plus warning from the front. Bonus ears: **±12.5 kHz off the EOT/HOT centers** (distributed-power / mid-train repeaters live there) and **NOAA weather radio (162.4–162.55 MHz)** as an always-on, known-good test signal. **The two facts that make this work at ANY North American crossing:** - **457.9375 / 452.9375 MHz is continent-wide.** Every EOT and HOT in North America uses this pair. You change nothing when you move. - **The voice channels are public.** The AAR (Association of American Railroads) channel plan is published per railroad and per subdivision. Only your voice channel(s) change with location. The synthesis: any one signal can be too weak to decode, but a real train lights several bands in the same time window. Co-firing bands = a train, even on a modest antenna. --- ## 1. Hardware — the shopping list | Item | What we run | Notes | |---|---|---| | SDR #1 — voice | **[RTL-SDR Blog V4](https://www.rtl-sdr.com/buy-rtl-sdr-dvb-t-dongles/)** | The V4's front-end filtering genuinely helps at VHF. This is the ears for 160–162 MHz. | | SDR #2 — telemetry | Any RTL-SDR (R820T/R828D-class) | Runs the 452⇄457 hop. Generic sticks drift; the decoder compensates (§8). A TCXO stick is nicer, not required. | | Antennas | ¼-wave verticals to start | **18.3 in (46.5 cm)** whip for 161 MHz; **6.4 in (16.3 cm)** for 457 MHz. Length is physics — get it right and a desk whip works day one. Upgrade: a tuned railroad-band base antenna (DPD TrainTenna, KB9VBR J-Pole) mounted high and far from your computers. Height beats gain; distance from noisy electronics beats both. | | USB | A **powered** hub | Two dongles on one host; a powered hub prevents brownout resets. | | Host | Any always-on 64-bit Linux box that runs Docker | Ours is a TrueNAS SCALE machine. A NUC, a Raspberry Pi 4/5, a Pi-like SBC, or an old desktop all work. 2+ cores, 2+ GB RAM. The whole rig is containers. | | A second computer | Your laptop | For setup, watching waterfalls, and reading the dashboard. | **Starting-from-nothing cost floor:** two dongles + whips ≈ US$70–100. Everything else is software in this repo. --- ## 2. Operating system & base software Any modern 64-bit Linux works. We'll give commands for **Debian/Ubuntu-family** (apt); translate to your package manager as needed. Three viable host shapes: - **A regular Linux box (recommended for a first build)** — Debian 12, Ubuntu 22.04+, Linux Mint, Raspberry Pi OS (64-bit). You control everything; nothing fights you. This guide's default. - **An appliance NAS OS (what we ended on)** — TrueNAS SCALE, Unraid, etc., running Docker/Dockge. More convenient long-term, but the OS resists you (see the §4 DVB note — appliance `/etc` can reset on update). Do your *first* build on a regular box, move to the appliance once it works. - **Bare metal vs. VM:** bare metal is simplest. A VM works but USB passthrough adds a failure mode and caps your sample rate — we ran in VirtualBox for a month and it was fine at 1.024 MS/s, but a Pi or NUC on bare metal is less fuss. Install the base tools: ```bash sudo apt update sudo apt install -y git rtl-sdr librtlsdr-dev build-essential cmake pkg-config \ libusb-1.0-0-dev ffmpeg python3 python3-numpy docker.io docker-compose-plugin ``` - `rtl-sdr` gives you `rtl_test`, `rtl_eeprom`, `rtl_fm`, `rtl_sdr` — the osmocom tools that drive the dongles. - `ffmpeg` is the audio plumbing (and the comms band-pass filter in front of transcription). - `python3` + `numpy` run every DSP and decode script here — **no other Python packages are required** for the core rig (the scripts are deliberately stdlib + numpy only). - Docker runs the whole stack. Optionally add **[Dockge](https://github.com/louislam/dockge)** — a light web UI for compose stacks; it's what we use, but plain `docker compose` is identical. Clone this repo onto the host: ```bash git clone istrain && cd istrain ``` --- ## 3. Know your crossing — 30 minutes of desk research This is the only location-specific work. Do it once. 1. **Identify the railroad and subdivision** at your crossing. Read the signage at the crossing itself (the operating railroad's name is on the signal equipment and the blue emergency-notify sign — that sign also lists a **DOT crossing number** and a phone number). Cross-check on [OpenRailwayMap](https://www.openrailwaymap.org/). 2. **Find the road (dispatch) channel.** Use two of these three and confirm they agree: - [RadioReference — US Railroads DB](https://www.radioreference.com/db/aid/7625) - [Railroad-Frequencies.com](https://www.railroad-frequencies.com/) - your region's railfan forum / site. You want the **AAR channel for the subdivision your crossing sits on**. Note its frequency and AAR channel number. Grab the **neighboring subdivisions' channels too** — pin 4–6 channels in a cluster; RTLSDR-Airband demodulates them all at once inside a single ~1 MHz window if they're within ~500 kHz of each other (most road channels cluster in 160.2–161.6 MHz). 3. **Find your defect detectors.** Railfan detector logs and forums list detector mileposts per subdivision. The detectors within ~10 miles either side of your crossing are your early-warning tripwires — write down their milepost numbers so the transcripts make sense. 4. **Note your NOAA weather frequency** (162.400–162.550, whichever is strongest near you). This is your receive-chain sanity signal forever — if you can hear NOAA, your antenna→dongle→software path works. Write it all down. See `config/istrain.conf.example` — it's our worked example (a Norfolk Southern subdivision in central Ohio); replace the five channel frequencies with yours. --- ## 4. Free the dongles from the TV driver (the step that bites everyone) The Linux kernel grabs RTL-SDR dongles as **DVB television tuners** the instant they're plugged in, and then no SDR software can open them. Blacklist the DVB driver on the **host** (this repo ships the file at `config/blacklist-rtl-sdr.conf`): ```bash sudo cp config/blacklist-rtl-sdr.conf /etc/modprobe.d/blacklist-rtl-sdr.conf sudo modprobe -r dvb_usb_rtl28xxu 2>/dev/null # unload it now lsmod | grep dvb # should print NOTHING ``` Reboot to make it stick. **The symptom of losing this fight later:** rtl_airband hangs at "Allocating zero-copy buffers," or `rtl_test` says the device is busy or open failed. It is *always* the DVB driver reloading — re-run the check above. > ⚠ **On appliance OSes (TrueNAS/Unraid/etc.), `/etc` can be reset by a system update.** Re-verify > this blacklist after every OS update. This is a permanent standing chore, not a one-time step. **Serialize your dongles** so every service can pick the right one by name (index order changes on reboot; serials don't). With only ONE dongle plugged in at a time: ```bash rtl_eeprom -s 1002 # plug in dongle A alone, give it serial 1002 (your voice radio) # unplug, plug in dongle B alone: rtl_eeprom -s 1001 # dongle B = 1001 (your telemetry radio) rtl_test # with both in: lists both by serial; confirms they enumerate ``` (We use `1001`/`1002`; any string works — just match it in the configs.) --- ## 5. The stack — what runs Everything is one Docker Compose stack (`docker/compose.example.yaml` + the Dockerfiles beside it): ``` ▣ airband RTLSDR-Airband (rtl-sdr-blog fork, V4-capable) on dongle 1002 → demodulates your 4–6 pinned voice channels at once, squelch-gated → one mp3 per squelch opening + a line in heard.jsonl ▣ scanhop rtl_tcp + scripts/iq_hop.py on dongle 1001 → retunes in place: 452.9375 (10 s) ⇄ 457.9375 (10 s), forever → scripts/iq_channelize.py splits each dwell into center ±12.5 kHz sub-channels → burst → bot.jsonl / eot.jsonl / midtrain.jsonl + a raw .s16 clip → level.json written continuously (the live meter + "is capture alive" signal) ▣ eot-decode scripts/eot-decode.py → scripts/eot/eot_scan.py : two-pass FFSK decode of EOT clips → unit ID · brake psi · motion flag, enriched back into eot.jsonl ▣ bot-recover scripts/bot-recover.py : head-end frames → addressed-unit → head/tail join ▣ worker scripts/transcribe-worker.py : voice clips >5 KB → ffmpeg comms band-pass → a Whisper server (§7) → transcript sidecars; real speech → transcripts.jsonl ▣ trim scripts/trim-captures.py : prunes voice audio >24 h (the jsonl records are kept forever) ▣ web dashboard/serve.py : stdlib Python, zero frameworks — the dashboard + all /api/* endpoints ``` Copy the compose template and edit the two host-specific things — your stack directory path and your dongle serials (already `1001`/`1002` if you followed §4): ```bash cp docker/compose.example.yaml docker/compose.yaml cp config/istrain.conf.example config/istrain.conf # then edit YOUR channels into it (§3) ``` --- ## 6. Bring-up, step by step 1. **Selftest the DSP before any radio.** These run the whole signal chain on synthetic data — no dongle needed. Green here means the math works on your machine; anything failing after this is radio/USB, not code: ```bash python3 scripts/iq_hop.py --selftest python3 scripts/iq_channelize.py --selftest ``` 2. **Prove the receive chain with NOAA first.** Before you wait for a train, confirm the whole antenna→dongle→software path works on a signal that's always there. Point a dongle at your NOAA frequency: ```bash rtl_fm -d 0 -f 162.550M -M fm -s 12k -g 40 - | aplay -r 12k -f S16_LE ``` Weather robot voice = your chain works end to end. Silence = fix the antenna/gain/frequency before going further. (This is the single most useful debugging move; NOAA is your oscilloscope.) 3. **Edit `config/istrain.conf`** — replace the example channels with your voice frequencies from §3. Keep the shape: one `channels` entry per frequency, `squelch_snr_threshold` starting near the example's value (tune per §8), mp3 outputs into the captures dir, and your dongle serial in the device stanza. 4. **Build and start the stack** (from `docker/`): ```bash docker compose --profile radio --profile radio1001 build # airband image builds first; # scanhop copies rtl_tcp out of it docker compose up -d docker compose ps # all containers Up ``` 5. **Verify capture is alive by DATA, not by "is the process running."** Watch `data/level.json` — it should be rewritten every second or two (that's the hop breathing). Watch `data/captures/` for mp3s. This data-freshness habit is the whole health model (§11). 6. **Open the dashboard:** `http://:3456`. The live meter should bounce; the correlate heatmap fills in 15-minute bins as bands fire. 7. **Wait for a train.** The first EOT decode is unmistakable: a unit ID, a brake pressure, and a motion flag from a machine a mile away, pulled out of static by a US$30 dongle. That's the moment the project becomes real. --- ## 7. Ears (optional, and the best part): transcription Any Whisper-family ASR server works. We run **[faster-whisper](https://github.com/SYSTRAN/faster-whisper) (model `small.en`) with VAD enabled** as its own small container/service, and `scripts/transcribe-worker.py` posts each filtered voice clip to it (set `WHISPER_URL` / `MILL_ASR` env to point at your server; the template default is `http://whisper:9000`). Two rules learned the hard way: - **Band-pass the audio first.** The worker runs an ffmpeg ~300–3400 Hz comms filter before ASR — Whisper hallucinates confident nonsense on raw squelch noise. - **VAD on, always.** Without voice-activity detection, an hour of static becomes an hour of imaginary dispatcher chatter. Only clips with *detected speech* earn a line in `transcripts.jsonl` — the project's durable memory of everything the railroad said out loud near your crossing. --- ## 8. Getting the truth out of cheap hardware - **Generic dongles drift off frequency.** Our EOT decoder (`scripts/eot/eot_scan.py`) is **two-pass**: an exact fast path, then an adaptive pass with fuzzy sync + tone tracking (±800 Hz) that recovers the off-frequency, short-burst emitters an exact decoder throws away as noise. Roughly half our real decodes come from the adaptive pass. If you write your own decoder, plan for drift from the start. - **Squelch sits just above YOUR noise floor, not a textbook number.** Watch a quiet hour's squelch-opening rate. If you're capturing hundreds of empty clips, you're squelching on your own house's RF hash — we measured **~⅔ of desk-whip squelch openings were never radio at all.** Raise `squelch_snr_threshold` until the empties stop; lower it if you're missing weak transmissions. - **Gain does not fix weak.** More gain amplifies the same noise. An A/B at gain 25 vs 40 vs auto on a buried signal sounds identical. The levers that actually work, in order: **antenna height, distance from electronics, a tuned antenna, better coax.** - **Beware standing carriers.** We chased a "mystery mid-train transmitter" for days; it was a **fixed wayside EOT signal booster** (an FCC-licensed repeater) parked on 457.925 that never moves. Card it as furniture and subtract its floor from your correlation logic, or it fakes a band forever. - **Trust the decoder, not quick statistics.** Every shortcut we tried to characterize undecodable bursts by spectral/envelope stats failed its own control test (a measure that couldn't tell modulated from unmodulated on *known-good* clips proves nothing). The decoder is the only instrument that doesn't lie. --- ## 9. Reading a train (the payoff) From decoded EOT frames, per train, the **brake-pipe pressure curve is the story silence can't tell:** | Air curve | Meaning | |---|---| | ~85–90 psi + motion flag set | **Passing** through | | Drops to a held ~55–65, motion 0 | **Dwelling** — brakes set, possibly blocking the crossing | | Bleeds to 0 and holds, head still polling | **Cut & standing** — crew cut the crossing; the road may be OPEN even though a train is "there" | | Recharges toward ~88, motion 0→1 | **Departing** — the roll-out is your event time | **Never read a bleed-to-zero as a departure** — that mistake reports a blocked crossing as clear. This taxonomy came from ~50 ground-truthed passages; yours may differ by railroad and operating pattern, so watch and record before you trust it. --- ## 10. Security — before you point this at the internet The dashboard (`dashboard/serve.py`) is safe to run on your LAN as-is. Before exposing it publicly (a reverse proxy, a Cloudflare tunnel, a port-forward), know these: - **Run it read-only when public.** Set the `SUPERVISOR_SNAPSHOT` env (any path) to mark the node as a **presentation node** — that disables clip deletion and refuses control POSTs. Leave it unset only on a private box you alone reach. - **Leave `PUSH_KEY` unset unless you actually use the VM→server push path.** Unset = the `/api/push` and `/api/push-clip` write endpoints are disabled entirely. If you set a key, treat it as a real secret (env only, never committed) — it can overwrite the data your site displays. - **The control endpoints are speed-bumped, not authenticated.** They require an `X-Ops` header (so blind scanners bounce), but that's a bump, not a password. Real control should stay on your LAN/tailnet, never bare on the public name. Put auth at your proxy if you want remote control. - **`/api/errors/report` is intentionally public** (browser error beacon) — it's rate-limited and length-capped server-side; that's the only endpoint that accepts unauthenticated writes, and it only appends bounded strings to a log. - **Never expose the raw container port to the internet.** Front it with a proxy that terminates TLS and, ideally, gates writes. There are **no credentials, keys, or personal data in this repository.** The frequencies are public; the code is stdlib. Keep it that way in your fork — put secrets in env, not in files. --- ## 11. The design rules that keep it robust (read if you're modifying it) These were bought with outages and false readings. Keep them: - **Every capture entrypoint exits nonzero when its dongle is missing.** The container restart policy (`restart: unless-stopped`) does the waiting — so a replug or a cold reboot self-heals with no human. (We lost a morning to a process that exited *zero* on a missing dongle and sat there looking alive.) - **No startup ordering between containers.** Each retries until its own dependencies appear — impossible to deadlock on boot order. - **Decoders never delete clips.** Capture, decode, and cleanup are separate jobs; a decoder bug can't eat your evidence. - **Health is read from data freshness, never from "is the process up."** `level.json`'s timestamp age, the last capture's age per band, the transcript trickle — a fresh timestamp is the only health signal a radio can't fake. - **Keep one append-only, numbered observations log.** Every "mystery signal" we resolved, we resolved by writing down what we actually *measured* — and the two findings we later retracted taught more than most that stood. Number the entries. Record the retractions. Future-you (or the next forker) is who you're writing for. --- ## 12. What's in this repo ``` APOCALYPSE-EDITION.md ← you are here README.md ← the front door + pointers LICENSE ← GPLv3 ATTRIBUTION.md ← third-party code (PyEOT) + the shoulders this stands on scripts/ ← the signal engine + decoders (stdlib + numpy only) iq_hop.py ← retune-in-place 452⇄457 hop (the mission radio) iq_channelize.py ← splits a dwell into center + ±12.5 kHz sub-channels eot-decode.py, eot/ ← two-pass FFSK EOT decoder (eot_scan.py + vendored PyEOT for BCH) bot-recover.py ← head-end frame recovery + head/tail join transcribe-worker.py ← voice clip → comms filter → Whisper → transcript vumon.py ← the live level meter correlate.py ← the cross-band "why" engine behind the heatmap trim-captures.py ← the 24 h voice-audio janitor scan-hop.py, iq_retune.py, envelope.py, probe-band.py ← the hop wrapper + analysis helpers dashboard/ ← serve.py (the API + static server) + the web UI docker/ ← compose template + Dockerfiles (web/airband/scanhop/worker) config/ ← istrain.conf.example (channels), blacklist-rtl-sdr.conf ``` --- *Built at a Norfolk Southern crossing in central Ohio, 2026, on RTL-SDR Blog V4 hardware, by a human and his AI pair-programmer. The live instance runs at [istrain.jhestyr.net](https://istrain.jhestyr.net). This repository is the whole thing — read it, run it, fork it, and if you build one, tell us what you heard.*