Files

T

istrain ea835b8837 guide: add a 'getting the code' note — the Cloudflare/User-Agent clone gotcha (for humans and agents), with the browser-UA + SSH workarounds

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-07-24 23:36:07 -04:00

22 KiB

Raw Blame History

istrain — APOCALYPSE EDITION (public build guide)

The complete from-scratch rebuild. This guide assumes you can install Linux and are comfortable in a terminal, but it spells out every railroad-, SDR-, and container-specific step — because those are the parts nobody tells you. If a step feels obvious to you, skip it; it's here for the person behind you who needs the hint.

You are building a passive RF receiver. It only listens. No transmit, no license required in the US to receive, no railroad cooperation needed — the trains announce themselves on public frequencies, and you just have to be set up to hear them.

Two ways to use this repo:

As a human: read top to bottom, run the commands.

As an agent (point Claude Code / an LLM coding agent at this repo): the whole working tree is here — scripts/ (the DSP + decoders), dashboard/ (the web UI + API server), docker/ (the container definitions), config/ (templates). Every script has a header comment explaining what it does and how it's operated. The design rules that keep it robust are in §11. Start by reading this file, then dashboard/serve.py and scripts/iq_hop.py.

Getting the code: git clone <this-repo-url> istrain. One gotcha worth knowing if this repo is hosted behind Cloudflare (or a similar bot-shield): a plain browser and normal git/curl clone fine, but a headless HTTP fetch (some scripts, some agent tool-calls) can get a 403 bot wall. The fix is a normal browser User-Agent on the request, e.g. git -c http.userAgent="Mozilla/5.0" clone <url> — or just use SSH, which sidesteps it entirely. (This is a hosting quirk, not a code quirk; a fork on a plain host won't see it.)

0. What you're building, in one minute

A train is a chain of radio transmitters. istrain listens passively to three of them and fuses the result into one question: is a train blocking my crossing right now?

Rail VHF voice (160–162 MHz, narrowband FM, unencrypted). Crews, dispatchers, and trackside defect detectors speaking plain English: "Norfolk Southern detector, milepost 148.9, no defects, total axles 460, speed 52, detector out." A detector hit on your road channel means a train just passed that milepost — the most direct "train coming" signal there is, in words.
End-of-Train telemetry (457.9375 MHz, 1200-baud FFSK). The flashing box on the last car (EOT / FRED / marker) transmits its unit ID, brake-pipe pressure, and a motion flag every few seconds. Decode it and the air tells you: passing, stopped, brakes cut, or leaving.
Head-of-Train (452.9375 MHz). The locomotive's half of the same conversation. Decode the EOT unit it's addressing and you can join head to tail — proof of a whole train, plus warning from the front.

Bonus ears: ±12.5 kHz off the EOT/HOT centers (distributed-power / mid-train repeaters live there) and NOAA weather radio (162.4–162.55 MHz) as an always-on, known-good test signal.

The two facts that make this work at ANY North American crossing:

457.9375 / 452.9375 MHz is continent-wide. Every EOT and HOT in North America uses this pair. You change nothing when you move.
The voice channels are public. The AAR (Association of American Railroads) channel plan is published per railroad and per subdivision. Only your voice channel(s) change with location.

The synthesis: any one signal can be too weak to decode, but a real train lights several bands in the same time window. Co-firing bands = a train, even on a modest antenna.

1. Hardware — the shopping list

Item	What we run	Notes
SDR #1 — voice	RTL-SDR Blog V4	The V4's front-end filtering genuinely helps at VHF. This is the ears for 160–162 MHz.
SDR #2 — telemetry	Any RTL-SDR (R820T/R828D-class)	Runs the 452⇄457 hop. Generic sticks drift; the decoder compensates (§8). A TCXO stick is nicer, not required.
Antennas	¼-wave verticals to start	18.3 in (46.5 cm) whip for 161 MHz; 6.4 in (16.3 cm) for 457 MHz. Length is physics — get it right and a desk whip works day one. Upgrade: a tuned railroad-band base antenna (DPD TrainTenna, KB9VBR J-Pole) mounted high and far from your computers. Height beats gain; distance from noisy electronics beats both.
USB	A powered hub	Two dongles on one host; a powered hub prevents brownout resets.
Host	Any always-on 64-bit Linux box that runs Docker	Ours is a TrueNAS SCALE machine. A NUC, a Raspberry Pi 4/5, a Pi-like SBC, or an old desktop all work. 2+ cores, 2+ GB RAM. The whole rig is containers.
A second computer	Your laptop	For setup, watching waterfalls, and reading the dashboard.

Starting-from-nothing cost floor: two dongles + whips ≈ US$70–100. Everything else is software in this repo.

2. Operating system & base software

Any modern 64-bit Linux works. We'll give commands for Debian/Ubuntu-family (apt); translate to your package manager as needed. Three viable host shapes:

A regular Linux box (recommended for a first build) — Debian 12, Ubuntu 22.04+, Linux Mint, Raspberry Pi OS (64-bit). You control everything; nothing fights you. This guide's default.
An appliance NAS OS (what we ended on) — TrueNAS SCALE, Unraid, etc., running Docker/Dockge. More convenient long-term, but the OS resists you (see the §4 DVB note — appliance /etc can reset on update). Do your first build on a regular box, move to the appliance once it works.
Bare metal vs. VM: bare metal is simplest. A VM works but USB passthrough adds a failure mode and caps your sample rate — we ran in VirtualBox for a month and it was fine at 1.024 MS/s, but a Pi or NUC on bare metal is less fuss.

Install the base tools:

sudo apt update
sudo apt install -y git rtl-sdr librtlsdr-dev build-essential cmake pkg-config \
                    libusb-1.0-0-dev ffmpeg python3 python3-numpy docker.io docker-compose-plugin

rtl-sdr gives you rtl_test, rtl_eeprom, rtl_fm, rtl_sdr — the osmocom tools that drive the dongles.
ffmpeg is the audio plumbing (and the comms band-pass filter in front of transcription).
python3 + numpy run every DSP and decode script here — no other Python packages are required for the core rig (the scripts are deliberately stdlib + numpy only).
Docker runs the whole stack. Optionally add Dockge — a light web UI for compose stacks; it's what we use, but plain docker compose is identical.

Clone this repo onto the host:

git clone <this-repo-url> istrain && cd istrain

3. Know your crossing — 30 minutes of desk research

This is the only location-specific work. Do it once.

Identify the railroad and subdivision at your crossing. Read the signage at the crossing itself (the operating railroad's name is on the signal equipment and the blue emergency-notify sign — that sign also lists a DOT crossing number and a phone number). Cross-check on OpenRailwayMap.
Find the road (dispatch) channel. Use two of these three and confirm they agree:
- RadioReference — US Railroads DB
- Railroad-Frequencies.com
- your region's railfan forum / site. You want the AAR channel for the subdivision your crossing sits on. Note its frequency and AAR channel number. Grab the neighboring subdivisions' channels too — pin 4–6 channels in a cluster; RTLSDR-Airband demodulates them all at once inside a single ~1 MHz window if they're within ~500 kHz of each other (most road channels cluster in 160.2–161.6 MHz).
Find your defect detectors. Railfan detector logs and forums list detector mileposts per subdivision. The detectors within ~10 miles either side of your crossing are your early-warning tripwires — write down their milepost numbers so the transcripts make sense.
Note your NOAA weather frequency (162.400–162.550, whichever is strongest near you). This is your receive-chain sanity signal forever — if you can hear NOAA, your antenna→dongle→software path works.

Write it all down. See config/istrain.conf.example — it's our worked example (a Norfolk Southern subdivision in central Ohio); replace the five channel frequencies with yours.

4. Free the dongles from the TV driver (the step that bites everyone)

The Linux kernel grabs RTL-SDR dongles as DVB television tuners the instant they're plugged in, and then no SDR software can open them. Blacklist the DVB driver on the host (this repo ships the file at config/blacklist-rtl-sdr.conf):

sudo cp config/blacklist-rtl-sdr.conf /etc/modprobe.d/blacklist-rtl-sdr.conf
sudo modprobe -r dvb_usb_rtl28xxu 2>/dev/null   # unload it now
lsmod | grep dvb                                 # should print NOTHING

Reboot to make it stick. The symptom of losing this fight later: rtl_airband hangs at "Allocating zero-copy buffers," or rtl_test says the device is busy or open failed. It is always the DVB driver reloading — re-run the check above.

⚠ On appliance OSes (TrueNAS/Unraid/etc.), /etc can be reset by a system update. Re-verify this blacklist after every OS update. This is a permanent standing chore, not a one-time step.

Serialize your dongles so every service can pick the right one by name (index order changes on reboot; serials don't). With only ONE dongle plugged in at a time:

rtl_eeprom -s 1002    # plug in dongle A alone, give it serial 1002 (your voice radio)
# unplug, plug in dongle B alone:
rtl_eeprom -s 1001    # dongle B = 1001 (your telemetry radio)
rtl_test               # with both in: lists both by serial; confirms they enumerate

(We use 1001/1002; any string works — just match it in the configs.)

5. The stack — what runs

Everything is one Docker Compose stack (docker/compose.example.yaml + the Dockerfiles beside it):

▣ airband   RTLSDR-Airband (rtl-sdr-blog fork, V4-capable) on dongle 1002
            → demodulates your 4–6 pinned voice channels at once, squelch-gated
            → one mp3 per squelch opening + a line in heard.jsonl
▣ scanhop   rtl_tcp + scripts/iq_hop.py on dongle 1001
            → retunes in place: 452.9375 (10 s) ⇄ 457.9375 (10 s), forever
            → scripts/iq_channelize.py splits each dwell into center ±12.5 kHz sub-channels
            → burst → bot.jsonl / eot.jsonl / midtrain.jsonl + a raw .s16 clip
            → level.json written continuously (the live meter + "is capture alive" signal)
▣ eot-decode    scripts/eot-decode.py → scripts/eot/eot_scan.py : two-pass FFSK decode of EOT clips
                → unit ID · brake psi · motion flag, enriched back into eot.jsonl
▣ bot-recover   scripts/bot-recover.py : head-end frames → addressed-unit → head/tail join
▣ worker    scripts/transcribe-worker.py : voice clips >5 KB → ffmpeg comms band-pass
            → a Whisper server (§7) → transcript sidecars; real speech → transcripts.jsonl
▣ trim      scripts/trim-captures.py : prunes voice audio >24 h (the jsonl records are kept forever)
▣ web       dashboard/serve.py : stdlib Python, zero frameworks — the dashboard + all /api/* endpoints

Copy the compose template and edit the two host-specific things — your stack directory path and your dongle serials (already 1001/1002 if you followed §4):

cp docker/compose.example.yaml docker/compose.yaml
cp config/istrain.conf.example config/istrain.conf   # then edit YOUR channels into it (§3)

6. Bring-up, step by step

Selftest the DSP before any radio. These run the whole signal chain on synthetic data — no dongle needed. Green here means the math works on your machine; anything failing after this is radio/USB, not code:
```
python3 scripts/iq_hop.py --selftest
python3 scripts/iq_channelize.py --selftest
```
Prove the receive chain with NOAA first. Before you wait for a train, confirm the whole antenna→dongle→software path works on a signal that's always there. Point a dongle at your NOAA frequency:
```
rtl_fm -d 0 -f 162.550M -M fm -s 12k -g 40 - | aplay -r 12k -f S16_LE
```
Weather robot voice = your chain works end to end. Silence = fix the antenna/gain/frequency before going further. (This is the single most useful debugging move; NOAA is your oscilloscope.)
Edit config/istrain.conf — replace the example channels with your voice frequencies from §3. Keep the shape: one channels entry per frequency, squelch_snr_threshold starting near the example's value (tune per §8), mp3 outputs into the captures dir, and your dongle serial in the device stanza.

Build and start the stack (from docker/):

docker compose --profile radio --profile radio1001 build   # airband image builds first;
                                                           # scanhop copies rtl_tcp out of it
docker compose up -d
docker compose ps                                          # all containers Up

Verify capture is alive by DATA, not by "is the process running." Watch data/level.json — it should be rewritten every second or two (that's the hop breathing). Watch data/captures/ for mp3s. This data-freshness habit is the whole health model (§11).
Open the dashboard: http://<host>:3456. The live meter should bounce; the correlate heatmap fills in 15-minute bins as bands fire.
Wait for a train. The first EOT decode is unmistakable: a unit ID, a brake pressure, and a motion flag from a machine a mile away, pulled out of static by a US$30 dongle. That's the moment the project becomes real.

7. Ears (optional, and the best part): transcription

Any Whisper-family ASR server works. We run faster-whisper (model small.en) with VAD enabled as its own small container/service, and scripts/transcribe-worker.py posts each filtered voice clip to it (set WHISPER_URL / MILL_ASR env to point at your server; the template default is http://whisper:9000).

Two rules learned the hard way:

Band-pass the audio first. The worker runs an ffmpeg ~300–3400 Hz comms filter before ASR — Whisper hallucinates confident nonsense on raw squelch noise.
VAD on, always. Without voice-activity detection, an hour of static becomes an hour of imaginary dispatcher chatter. Only clips with detected speech earn a line in transcripts.jsonl — the project's durable memory of everything the railroad said out loud near your crossing.

8. Getting the truth out of cheap hardware

Generic dongles drift off frequency. Our EOT decoder (scripts/eot/eot_scan.py) is two-pass: an exact fast path, then an adaptive pass with fuzzy sync + tone tracking (±800 Hz) that recovers the off-frequency, short-burst emitters an exact decoder throws away as noise. Roughly half our real decodes come from the adaptive pass. If you write your own decoder, plan for drift from the start.
Squelch sits just above YOUR noise floor, not a textbook number. Watch a quiet hour's squelch-opening rate. If you're capturing hundreds of empty clips, you're squelching on your own house's RF hash — we measured ~⅔ of desk-whip squelch openings were never radio at all. Raise squelch_snr_threshold until the empties stop; lower it if you're missing weak transmissions.
Gain does not fix weak. More gain amplifies the same noise. An A/B at gain 25 vs 40 vs auto on a buried signal sounds identical. The levers that actually work, in order: antenna height, distance from electronics, a tuned antenna, better coax.
Beware standing carriers. We chased a "mystery mid-train transmitter" for days; it was a fixed wayside EOT signal booster (an FCC-licensed repeater) parked on 457.925 that never moves. Card it as furniture and subtract its floor from your correlation logic, or it fakes a band forever.
Trust the decoder, not quick statistics. Every shortcut we tried to characterize undecodable bursts by spectral/envelope stats failed its own control test (a measure that couldn't tell modulated from unmodulated on known-good clips proves nothing). The decoder is the only instrument that doesn't lie.

9. Reading a train (the payoff)

From decoded EOT frames, per train, the brake-pipe pressure curve is the story silence can't tell:

Air curve	Meaning
~85–90 psi + motion flag set	Passing through
Drops to a held ~55–65, motion 0	Dwelling — brakes set, possibly blocking the crossing
Bleeds to 0 and holds, head still polling	Cut & standing — crew cut the crossing; the road may be OPEN even though a train is "there"
Recharges toward ~88, motion 0→1	Departing — the roll-out is your event time

Never read a bleed-to-zero as a departure — that mistake reports a blocked crossing as clear. This taxonomy came from ~50 ground-truthed passages; yours may differ by railroad and operating pattern, so watch and record before you trust it.

10. Security — before you point this at the internet

The dashboard (dashboard/serve.py) is safe to run on your LAN as-is. Before exposing it publicly (a reverse proxy, a Cloudflare tunnel, a port-forward), know these:

Run it read-only when public. Set the SUPERVISOR_SNAPSHOT env (any path) to mark the node as a presentation node — that disables clip deletion and refuses control POSTs. Leave it unset only on a private box you alone reach.
Leave PUSH_KEY unset unless you actually use the VM→server push path. Unset = the /api/push and /api/push-clip write endpoints are disabled entirely. If you set a key, treat it as a real secret (env only, never committed) — it can overwrite the data your site displays.
The control endpoints are speed-bumped, not authenticated. They require an X-Ops header (so blind scanners bounce), but that's a bump, not a password. Real control should stay on your LAN/tailnet, never bare on the public name. Put auth at your proxy if you want remote control.
/api/errors/report is intentionally public (browser error beacon) — it's rate-limited and length-capped server-side; that's the only endpoint that accepts unauthenticated writes, and it only appends bounded strings to a log.
Never expose the raw container port to the internet. Front it with a proxy that terminates TLS and, ideally, gates writes.

There are no credentials, keys, or personal data in this repository. The frequencies are public; the code is stdlib. Keep it that way in your fork — put secrets in env, not in files.

11. The design rules that keep it robust (read if you're modifying it)

These were bought with outages and false readings. Keep them:

Every capture entrypoint exits nonzero when its dongle is missing. The container restart policy (restart: unless-stopped) does the waiting — so a replug or a cold reboot self-heals with no human. (We lost a morning to a process that exited zero on a missing dongle and sat there looking alive.)
No startup ordering between containers. Each retries until its own dependencies appear — impossible to deadlock on boot order.
Decoders never delete clips. Capture, decode, and cleanup are separate jobs; a decoder bug can't eat your evidence.
Health is read from data freshness, never from "is the process up." level.json's timestamp age, the last capture's age per band, the transcript trickle — a fresh timestamp is the only health signal a radio can't fake.
Keep one append-only, numbered observations log. Every "mystery signal" we resolved, we resolved by writing down what we actually measured — and the two findings we later retracted taught more than most that stood. Number the entries. Record the retractions. Future-you (or the next forker) is who you're writing for.

12. What's in this repo

APOCALYPSE-EDITION.md   ← you are here
README.md               ← the front door + pointers
LICENSE                 ← GPLv3
ATTRIBUTION.md          ← third-party code (PyEOT) + the shoulders this stands on
scripts/                ← the signal engine + decoders (stdlib + numpy only)
  iq_hop.py             ← retune-in-place 452⇄457 hop (the mission radio)
  iq_channelize.py      ← splits a dwell into center + ±12.5 kHz sub-channels
  eot-decode.py, eot/   ← two-pass FFSK EOT decoder (eot_scan.py + vendored PyEOT for BCH)
  bot-recover.py        ← head-end frame recovery + head/tail join
  transcribe-worker.py  ← voice clip → comms filter → Whisper → transcript
  vumon.py              ← the live level meter
  correlate.py          ← the cross-band "why" engine behind the heatmap
  trim-captures.py      ← the 24 h voice-audio janitor
  scan-hop.py, iq_retune.py, envelope.py, probe-band.py ← the hop wrapper + analysis helpers
dashboard/              ← serve.py (the API + static server) + the web UI
docker/                 ← compose template + Dockerfiles (web/airband/scanhop/worker)
config/                 ← istrain.conf.example (channels), blacklist-rtl-sdr.conf

Built at a Norfolk Southern crossing in central Ohio, 2026, on RTL-SDR Blog V4 hardware, by a human and his AI pair-programmer. The live instance runs at istrain.jhestyr.net. This repository is the whole thing — read it, run it, fork it, and if you build one, tell us what you heard.

22 KiB Raw Blame History Unescape Escape