HUMAN BLIND VERIFICATION OF MACHINE-LEARNING CATALOGUES

Upload your catalogue

The platform returns, for every event, a confirmability score: the probability that blind human reviewers would confirm it, among events they are willing to decide (calibrated at Ridgecrest). Each score comes with a tier and maps that show where the confirmable events sit; the per-event 90% interval is an addition of this demo, not in the preprint.

Drop a CSV here

or

The file is read by this page only. Nothing is uploaded: scoring runs as JavaScript on your machine.
No file at hand? Download a 5,000-event sample CSV with the standard columns.

1 Upload→2 Score→3 Explore→4 Download

Standard columns

One row per event. Different names are fine: you map them on the next screen.

Column	Meaning
n_pha_total	associated phases (picks)score
n_sta	stations with a pickscore
moveout_rms	RMS travel-time residual, sscore
az_gap	largest azimuthal gap, degscore
sec_az_gap	second-largest gap, degscore
dist_nearest_sta_km	distance to nearest station, kmscore
mag	catalogue magnitudescore
lon, lat, depth_km	location, for the mapsmap
time	origin time (ISO 8601 or epoch s)optional

These are the quantities a location code already reports. Missing values are filled with the Ridgecrest median and flagged.

EXAMPLE CATALOGUE

Scoring catalogue

fault segments show hidden faintly

confirmability p

0 low0.51 high

Show events with score between

CALIBRATE ON YOUR OWN REVIEW · 1 OF 4

Draw the review sample

events seed

How the review works

1	Draw about 200 events, 20 from each tenth of the score ranking (equal-population bins). Each event carries the design weight bin size / events drawn.
2	Cut waveforms for them with cut_review_waveforms.py (ObsPy) from FDSN or your own archive, or supply your own files. The request file has no score and no magnitude, so the reviewer stays blind.
3	Upload the package here and review each event blind: real, uncertain or false.
4	The page reports AUC, precision and recall per threshold for your catalogue, design-weighted, with bootstrap intervals.

Why: the ranking by p transferred to a second region in the preprint, the level did not. About 200 blind verdicts per region re-anchor the level. Everything runs in this tab; no file leaves your machine.

CALIBRATE ON YOUR OWN REVIEW · 2 OF 4

Upload the waveform package

Drop the .zip (or the package folder) here

Package format

stations.csv	network,station,location,lat,lon,elev_m required: distances and guides need it
waveforms/<event_id>.mseed	one file per event, miniSEED 2 (Steim-1, Steim-2, int, float), any number of stations; Z,N,E or Z,1,2; Z only is accepted; at least 20 Hz; raw counts

Build it with cut_review_waveforms.py from the request file, or write your own. Events outside the draw are allowed, but then they get weight 1 and precision is not design-weighted.

Files are read in this tab only. Nothing is uploaded.

Event 1 of 200

wheel zoom · drag pan · double-click reset

Guide. Judge whether the event is supported by coherent multi-station waveforms. Red and green dashed lines are approximate P and S times from a straight-ray model, so real arrivals can be off by a few seconds, more so on distant rows. Rows are the nearest available stations; a weak nearest row alone is not evidence of a false event. A larger event in the window is common in busy sequences.
real: coherent onsets on at least two rows with reasonable moveout. uncertain: possible energy, but overlap, a single reliable row, missing traces or timing ambiguity prevents a reproducible decision. false: no coherent multi-station onset near the guides, or only station-local noise.

Progress

Reviewer

Display

band-passtoHz

VpVskm/s

4-pole Butterworth, zero phase. Each trace scaled to its own 99.5th percentile.

CALIBRATE ON YOUR OWN REVIEW · 4 OF 4

Results

Any threshold

Reliability of p on your events

Points: decided events in the ten design bins (mean p vs weighted share confirmed). Line: Platt recalibration.