This system treats fraud as a real-time problem. If you detect something suspicious after the fact, you’ve built an analytics dashboard, not a fraud pipeline.
The core flow is: transactions land in Kafka, are normalized into a stable schema, enriched with deterministic features (velocity, rolling windows, flags), and scored in Kafka Streams. The output is stored in MongoDB for investigations and queryable history.
Architecture
Kafka topics are partitioned by an explicit key (typically account, user, or card) so ordering and state locality are intentional rather than accidental. Kafka Streams maintains rolling state per key (for example: counts over the last N minutes) and emits enriched/scored events downstream.
Vector similarity as one signal
The project uses similarity scoring to compare events (or entities) against known suspicious patterns. It is not “LLM magic”; it’s a retrieval signal that helps answer: does this look like something we’ve seen before?
The practical constraint is cost. You don’t embed every transaction. Instead, you embed selectively: high-risk events, high-value events, or stable entities (merchant/device/user) and reuse those vectors.
Operational notes
The pipeline is designed to fail clearly: explicit dead-lettering, predictable retries, and idempotent writes. The output includes an audit trail of “why this was scored” so an investigator can justify a decision later.
If you want to extend it, the next best step is adding a small “Run it locally” section (Docker compose for Kafka + app), plus a single diagram showing topics, stream stages, and MongoDB collections.