The Engineering Behind Football Ads Muter

Sometimes the best engineering projects do not begin with an ambitious vision. They begin with a tiny annoyance that refuses to go away.

I love watching football. What I do not love is what happens every few minutes.

The game builds momentum. The crowd gets louder. The commentators get excited.

Then:

Cut.

A car commercial. A betting ad. An insurance company yelling through my speakers at twice the volume of the match itself.

After one particularly long evening of reaching for the mute button, I started wondering:

Could my computer figure out when the match had ended and the advertisements had begun?

Not by talking to the streaming service. Not through browser extensions. Not using cloud AI.

Just by looking at the pixels on my screen.

That question turned into one of my favorite weekend projects.

The Constraint That Made the Project Interesting

Streaming platforms do not expose a simple “this is an advertisement” signal.

From the browser’s perspective, everything is just video. The repository reflects that constraint directly: Football Ads Muter samples the screen, then fades browser audio down during ads and restores it instantly when football returns.

So instead of trying to detect advertisements, I inverted the problem.

Can I reliably recognize football?

If the answer is yes, then everything else becomes probably not football.

That small change simplified the entire architecture.

flowchart LR
A[Screen Capture<br/>mss] --> B[OpenCV Feature Extraction]
B --> C{{Decision Engine}}
C --> D[Temporal Memory]
C --> E[Local Model]
D --> F[Final Football Decision]
E --> F
F --> G{{Football?}}
G -->|Yes| H[Restore Audio]
G -->|No| I[Fade Browser Audio]

Seeing Football Like a Human

Humans do not consciously identify football frame by frame.

We subconsciously combine dozens of tiny clues:

green pitch
white field markings
scoreboard
camera movement
broadcast layout
crowd shots

The detector copies that philosophy.

Instead of relying on one perfect feature, it combines many imperfect ones.

Green Pitch Detection

The first prototype simply measured how much of the screen looked like grass.

It worked surprisingly well until the camera zoomed into a player’s face.

Lesson learned: no single feature survives real broadcasts.

Scoreboard Detection

Rather than performing OCR, I only asked whether the top of the frame looked like a dense UI overlay.

Edge density alone was enough to identify most scoreboards while remaining extremely fast.

That fits the project’s design: a light, local pipeline built from OpenCV, NumPy, and related tooling.

Motion

Football is rarely static.

Players move. Cameras pan. Crowds celebrate. Replays animate.

Motion does not prove football, but motion plus grass plus scoreboard becomes compelling evidence.

The Biggest Engineering Decision

Initially every frame was classified independently.

That was a mistake.

Football broadcasts constantly cut between:

closeups
replays
celebrations
coaches
crowd reactions

Treating every frame independently caused false mutes during perfectly normal broadcasts.

Instead I introduced temporal memory.

If football was confidently detected a second ago, then a closeup immediately afterward is almost certainly still football.

Sometimes remembering the past is more valuable than improving the classifier.

Detecting Advertisement Breaks

Advertisement breaks are surprisingly abrupt.

The detector compares HSV histograms between frames.

Large scene changes combined with little green, no scoreboard, and no field markings reset the football context immediately.

That gives the system a quick way to respond when the broadcast visibly shifts away from match coverage.

Adding Machine Learning

The heuristic detector already worked well.

But broadcasts differ across countries, leagues, broadcasters, display scaling, and browser setups. The repository supports both a local model workflow and a self-training bootstrap loop, which is exactly what you want when a detector needs to adapt to a specific viewing environment.

Instead of replacing the heuristics, I added a lightweight local model.

Every recording session can become new training data.

flowchart LR
A[Recorded Frames] --> B[Manual Labeling]
B --> C[Feature Dataset]
C --> D[Random Forest Training]
D --> E[Joblib Model]
E --> F[Runtime Inference]

The model acts as another opinion rather than the sole decision-maker.

That hybrid approach proved much more robust.

The README also documents both a starter one-class model and a local model workflow, which matches the project’s experimental, iterative design.

Why Everything Runs Locally

One of my design goals was privacy.

Nothing leaves your computer.

no cloud inference
no uploaded screenshots
no browser plugins
no streaming APIs

The repository explicitly notes that screenshots, logs, and trained model files stay local because they can contain private screen contents.

Everything happens locally using OpenCV and scikit-learn.

Sampling only once per second keeps CPU usage extremely low.

Audio Behavior

False positives are far more annoying than slightly delayed muting.

So the application intentionally behaves conservatively.

Football must be seen at least once.
Multiple consecutive non-football frames are required.
Audio fades smoothly.
Audio restores immediately.

The README exposes that tuning directly with options like --mute-after, --context-grace, and --mute-fade, which matches the user experience goal of conservative muting.

The goal is not perfect classification.

It is making the experience feel invisible.

What the Repository Actually Contains

This project is not just a single script.

The repository includes:

football_ad_muter.py
label_frames.py
self_train.py
train_model.py
train_one_class.py
models/
scripts/

That structure tells the real story: runtime detection, data labeling, self-training, and model training all live together as one iterative system.

That matters because the project is not merely a detector.

It is a workflow for improving the detector over time.

How I Thought About the System

At a high level, the system has three jobs:

Sample the screen.
Decide whether the frame still looks like football.
Control browser audio accordingly.

flowchart TD
S[Screen sample] --> F[Feature extraction]
F --> H[Heuristic decision]
F --> M[Optional local model]
H --> J[Combine evidence]
M --> J
J --> K{Football?}
K -->|Yes| L[Unmute / restore]
K -->|No| N[Fade down audio]

The important part is that the model does not replace the heuristics.

The heuristics provide explainability.

The model provides adaptability.

Together they are much more useful than either approach alone.

Why Not Just Use Deep Learning?

I considered it.

But for this project, deep learning would have been the wrong first move.

The problem is narrow. The input is noisy. The environment changes a lot. And I wanted something I could debug from the terminal when it made a bad decision.

A smaller system was a better fit because it let me inspect:

which visual signals were firing
why the audio changed
what kind of frames caused false positives
how the model behaved on my own browser setup

That made iteration faster and the project much more personal.

The Real Payoff

The best part of this project was not the model.

It was the moment the system started behaving like a tiny, thoughtful assistant.

Football on? Audio stays on.

Ad break? Audio fades away.

Match resumes? Audio comes back instantly.

That is the kind of automation I love most: quiet, local, useful, and just a little bit magical.

Lessons Learned

Frame classification is easier than broadcast understanding.
Temporal context beats increasingly complex models.
Explainable heuristics are invaluable during debugging.
Small, targeted ML models often outperform unnecessarily complex AI.
Constraints usually lead to better engineering.

Future Improvements

deeper scene understanding
automatic broadcaster adaptation
per-league calibration
live confidence visualization
browser-independent audio routing
support for more sports

Final Thoughts

Football Ads Muter is not trying to solve computer vision in general.

It is solving one very specific problem extremely well.

I enjoy projects like this because they sit at the intersection of software engineering, automation, computer vision, and machine learning.

A small annoyance became an excuse to learn, experiment, and build something genuinely useful.

Sometimes the best projects are not the ones with the biggest impact.

They are the ones that make you smile every time they quietly do exactly what you wanted.

Try it yourself

Any feedback / PRs are welcome to improve this project: Pragalbha-Patil/football-ads-muter

Building Football Ads Muter