Heard a ton about Gemma 4 this week so I tried a plugin using it to inject chaos rather than my basic randomized hardcoded setup. It was more aggressive and still gave me good results.

Had myself a bit of a crazy mistake. Accidentally labelled 100hz as 1mhz and the framework still ran well on my 16gb M4.

Decided to run on an RTX6000, 5090 and 1660Ti as well. It runs on everything I’ve thrown it at!

Managed to run it on a B200 and the results were great. Trying to find somewhere I can use a GH200 or GB200 to see if it's like a supercharged M4.

Thanks, that means a lot! I'm really glad with how things have gone so far.

GitHub - tarek-clarke/resilient-rap-framework: A resilient, fault‑tolerant telemetry analytics pipeline designed to validate, benchmark, and stress‑test high‑frequency sensor data streams under real‑w... A resilient, fault‑tolerant telemetry analytics pipeline designed to validate, benchmark, and stress‑test high‑frequency sensor data streams under real‑world failure conditions. Includes chaos test...

When tensor models hit the physical noise floor of the hardware, deterministic circuit breakers and DLQs must step in. Multi-layered defense is mandatory for the edge.

Sanitized repo & architecture details here:
github.com/tarek-clarke...
#DataEngineering #SportsAnalytics

But here is the best part: the percentage of anomalies it missed.

The blind spot was highly concentrated on bit_flip_low events on the ecu_canbus.

Why? A low-magnitude bit-flip on an engine sensor perfectly mimics physical engine vibration.

The most interesting bench mark was Apple's M4. By bypassing the traditional PCIe bus penalty with unified memory, it crushed the edge-tier latency.

The system held a 99.77% detection rate with zero breaker trips, while preserving a SHA-256 tamper-evident audit chain.

I threw the 3.6M packet load (with injected chaos & schema drifts) at 4 distinct architectures to ensure it's fully hardware-agnostic:

NVIDIA H200 (Cloud)

AMD 7900 XT on ROCm (Linux Edge)

Apple M4 (Pit Wall)

i5-12600K (x86 fallback)

A month ago, I introduced the Resilient RAP framework for edge telemetry, quoting a <20ms self-healing latency.

I just moved the pipeline to GPU-accelerated tensor anomaly detection. On a 3.6-million packet F1 stress test? I just hit 0.003 ms.

I JUST got into Github Copilot's student program. Devastated.

This actually bothered me more than I'd like to admit

Good validation at least haha

I did it for a trip last September. Genuinely did not know it wasn’t enforced til now

Finally bit the bullet and installed Ubuntu on a partition on my SSD. ROCm is incredible haha

Haha, I submitted a proposal and we’re now adjusting it to find a niche for publication before I’m admitted

Thanks! It's processing in real time. I don't have access to live telemetry streams but it's running off the OpenF1 API for my test cases. I reached out to a few teams to see if they were interested, but it's great to see it work that well in the meantime!

My Resilient RAP framework uses BERT-based semantic mapping to "self-heal" in <20ms, maintaining 85% accuracy where others drop to 15%.

#DataEngineering #SystemsEngineering #MachineLearning #OpenScience #Python #F1Data

I've also uploaded a preprint outlining my initial findings on engrXiv!
doi.org/10.31224/6466

#DataEngineering #F1 #F1Telemetry #SportsAnalytics #NHLAnalytics #F1Analytics

GitHub - tarek-clarke/resilient-rap-framework: To test the viability of a resilient analytical pipeline for clinical and sports health telemetry To test the viability of a resilient analytical pipeline for clinical and sports health telemetry - tarek-clarke/resilient-rap-framework

I’m developing the Resilient RAP Framework to solve the contract of trust problem in high-velocity data.

-Use cases: F1 & NHL telemetry sensors.
-PhD proposal in progress for TalTech.
-1800+ clones on GitHub: github.com/tarek-clarke...

Let's talk #SportsAnalytics!

🚀 Hello #DataSky!

I’m Tarek, a Senior Economic Data Analyst at Statistics Canada with a Masters in Data Engineering. 📊

I’m currently bridging the gap between economic rigour and resilient infrastructure through my research into "Self-Healing" data pipelines. 🧵👇