Diagnose car faults from audio recordings with calibrated ML
cardiag is a calibrated audio-ML pipeline that triages car faults from sound clips and says uncertain instead of guessing.
Why it matters
Triage automotive mechanical problems by analyzing audio recordings from phone clips or social media, providing calibrated predictions about fault presence, location zones, and likely component failures while explicitly indicating uncertainty when confidence is low.
Outcomes
What it gets done
Scrape and clean automotive fault audio from YouTube and TikTok, isolating mechanical sounds from speech, music, and background noise
Classify whether a car sound indicates a fault versus normal operation with 0.79 AUROC
Identify which zone of the car (6 regions) the fault originates from with 75% top-3 accuracy
Rank likely faulty parts (12+ component families) with 45-65% top-3 accuracy and calibrated uncertainty
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/adam-s-car-diagnosis | bash Overview
Car Diagnosis
cardiag is a calibrated audio-ML pipeline that diagnoses car faults from sound clips, isolating the mechanical audio, embedding it with CLAP, and triaging with linear heads. Use it for a first-pass triage on a car noise from a phone or social clip. It's a proof of concept, not a safety-critical or standalone diagnostic tool.
What it does
cardiag is an end-to-end audio-ML pipeline that diagnoses a car fault from its sound: it scrapes fault-sound clips from YouTube and TikTok, cleans the audio to isolate the mechanical sound from speech, music, and noise through a clean() cascade, embeds the isolated span with a frozen CLAP model, and trains small linear heads on the 512-d embedding to triage the fault. It is explicitly built as a calibrated triage aid rather than a diagnoser - it says "uncertain" instead of bluffing when the audio can't support a call - and ships as both a CLI and a live web app.
When to use - and when NOT to
Use it on social-style or targeted-upload audio - a YouTube/TikTok clip or a phone recording a user deliberately makes - to get a first-pass read on whether something sounds wrong, roughly where in the car (6 zones), and a ranked shortlist of likely parts (12+ families). It is a proof of concept and says so plainly: measured out-of-sample with by-video grouped cross-validation over 1,031 video groups (permutation p = 0.0005), it reaches AUROC 0.79 on fault-vs-normal detection, top-3 zone accuracy of about 75%, and top-3 part accuracy of 45-65% - and one head (knock) was demoted for failing out-of-sample. It is explicitly not a safety-critical or standalone diagnostic tool, and model files are joblib artifacts you should only load if you trust the source. The same cleaning-plus-calibrated-training method reaches 0.93 AUROC on clean engine audio, which is the real ceiling the crude-phone-audio numbers are working against.
Inputs and outputs
Input: an audio clip (wav), your own recordings via cardiag ingest, or a scraped YouTube/TikTok corpus. Output: a calibrated verdict, a car-zone estimate, and a ranked list of likely parts, or an explicit UNCERTAIN when the signal doesn't support a call (add --json to any inference command for machine-readable output).
git clone <this-repo> && cd car-diagnosis
uv venv && source .venv/bin/activate
uv pip install -e ".[scrape,web,dev,viz]" # Python 3.11
cardiag doctor # preflight: what's installed
cardiag train --fixtures # a working model offline in ~2s (no scrape, no 2 GB download)
cardiag diagnose <clip.wav> # verdict + where-in-the-car + ranked parts
cardiag serve --model models # live web app: drop a clip / paste a link, "explain why"
A fresh clone is immediately usable - a small pre-trained model and a synthetic demo clip ship in the repo, so nothing needs to be downloaded or scraped before trying it, and bash scripts/clone_verify.sh verifies the whole pipeline end-to-end in an isolated worktree.
Integrations
Built on a frozen CLAP audio-language model for embedding, with one shared clean() segmentation cascade used identically at training and serving time (spans over ~10s are split into windows so CLAP never silently truncates them), eliminating train/serve skew. Additional CLI verbs cover cardiag triage (calibrated engine-vs-running-gear), cardiag clean (isolate the mechanical sound with no model needed), cardiag inspect (an HTML report of spans, spectrograms, and scores), and cardiag scrape youtube|tiktok for building a training corpus (Reddit scraping is deprecated as too noisy).
Who it's for
Anyone who wants a first-pass, honestly-calibrated read on a car noise from a phone or social-media clip - and researchers interested in the reusable cleaning-plus-honest-training recipe, which the project reports also works on other audio datasets. Open source under the MIT license.
Source README
cardiag
Diagnose a car fault from its sound - a calibrated triage aid that says "uncertain" instead of bluffing
CLAP, step by step · Isolating the Engine Audio · Model Card · Defense
cardiag is an end-to-end audio-ML pipeline. It scrapes fault-sound clips from
YouTube/TikTok, cleans the audio (isolating the mechanical sound from speech, music,
and noise), embeds it with a frozen CLAP model, and trains small linear heads to
triage the fault. It is exposed as a CLI and a live web app.
https://github.com/user-attachments/assets/23dc48e7-5571-4890-9a6c-4552d907dd8a
This is a proof of concept, and honest about what that means. Diagnosing a car fault
from a phone recording is genuinely hard, so cardiag is built as a calibrated
triage aid rather than a diagnoser: it tells you whether something sounds wrong,
roughly where in the car it is, and a ranked shortlist of likely parts. When the
audio won't support a call, it says "uncertain" instead of bluffing.
The real contribution is the cleaning + honest-training recipe, which is reusable
on other audio datasets. The modest accuracy here reflects how hard the problem is
from crude phone audio (we hit the literature ceiling); the same method reaches
0.93 AUROC on clean engine audio. See docs/DEFENSE.md.
Interactive demos
Two pages visualize the first two stages of the pipeline:
- Isolating the Engine Audio - an interactive look at the
clean()cascade pulling a short mechanical span out of noisy YouTube audio (speech, music, road noise). - CLAP, step by step - how the frozen CLAP model turns those spans into the 512-d embedding the linear heads classify.
What it actually achieves
Measured out-of-sample, leakage-safe (by-video grouped CV over 1,031 video groups;
permutation p = 0.0005). These are honest numbers, not a leaderboard.
| Capability | Result | vs. chance |
|---|---|---|
| Is something wrong? (fault/normal) | AUROC 0.79 [0.76, 0.83] | 0.50 |
| Where in the car? (6 zones) | right zone in top-3 ≈ 75% | 2× |
| Which part? (12+ families) | right part in top-3 ≈ 45-65% | 3-4× |
| Knows when it doesn't know | calibrated (ECE ≈ 0.04), returns UNCERTAIN |
- |
Full details, and the one head we demoted for failing out-of-sample (knock), are
in docs/MODEL_CARD.md.
Quickstart: clone to inference
A fresh clone is immediately usable. A small pre-trained model ships in models/,
and a synthetic demo clip is bundled, so nothing needs to be downloaded or scraped.
git clone <this-repo> && cd car-diagnosis
uv venv && source .venv/bin/activate
uv pip install -e ".[scrape,web,dev,viz]" # Python 3.11
cardiag doctor # preflight: what's installed
cardiag train --fixtures # a working model offline in ~2s (no scrape, no 2 GB download)
cardiag diagnose <clip.wav> # verdict + where-in-the-car + ranked parts
cardiag serve --model models # live web app: drop a clip / paste a link, "explain why"
Verify the whole thing end-to-end in an isolated worktree: bash scripts/clone_verify.sh.
How it works
audio ──► clean() cascade ──► CLAP embedding ──► linear heads ──► Diagnosis
(isolate spans) (frozen, 512-d) (fault/region/ (calibrated,
part/knock) UNCERTAIN-aware)
There is one segmentation path. Scraped clips, your own recordings (cardiag ingest, any length), and uploads at inference all flow through the same clean()
cascade that isolates short mechanical spans. Spans over ~10 s are split into windows
so CLAP never silently truncates them. Training and serving share one embedding
contract, so there is no train/serve skew.
Usage
cardiag diagnose clip.wav # full model: verdict + region + ranked parts
cardiag triage clip.wav # calibrated engine-vs-running-gear
cardiag clean clip.wav # isolate the mechanical sound (no model needed)
cardiag inspect clip.wav -o r.html # SEE/HEAR the pipeline: spans, spectrograms, scores
cardiag ingest ./my_audio --kind fault --cause wheel_bearing # bring your own audio
cardiag scrape youtube|tiktok # build a corpus (Reddit is deprecated — too noisy)
cardiag train # train on your corpus
Add --json to any inference command for machine-readable output.
Documentation
- docs/DEFENSE.md - the honest case that a deliberately crude method earns a real triage result.
- docs/MODEL_CARD.md - per-head metrics, intended use, limitations.
- docs/architecture.md - pipeline diagrams.
- docs/scraping-guide.md - start-to-finish corpus building.
Scope & honesty
Valid for social-style / targeted-upload audio (YouTube, TikTok, or a phone clip a
user records deliberately). It is not a safety-critical or standalone diagnostic.
It is a triage assistant that narrows where to look and is honest about its
uncertainty. Model files are joblib artifacts: load only ones you trust.
License: see LICENSE.
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.