It’s Not What the Scammer Says, It’s How You Reply: Small Scam Detectors Judge Code-Mixed Fraud Calls by the Victim

Rishi Ahuja 1 , Kumar Prateek 1 , Simranjit Singh 1*

  1. 1 Department of Information Technology, National Institute of Technology Jalandhar

* Corresponding author: Simranjit Singh

When a small model flags a phone call as a scam, is it responding to what the caller asks for, or to how the person on the other end reacts? ScamBranch fixes the caller’s request and varies only the reply to find out.

AAAI-27 Student Abstract

The question

Phone scams in India often mix a local language with English, and small models that read call transcripts are an appealing defence. Datasets such as ICFD-31k generate each call independently, though, so the scam’s content and the callee’s reaction change together.

That leaves a detector free to learn the reaction instead of the request. It could flag careful customers on genuine calls and clear scams whose victims go along with them. ScamBranch asks whether a detector’s verdict follows what the caller asks for, or how the person called replies.

Branching one call four ways

Each call starts from a fixed opening that already contains the caller’s request and all the evidence that decides the label. That opening is frozen and continued four times, with the callee:

  • Cooperating.
  • Asking for clarification.
  • Resisting.
  • Voicing suspicion.

The caller can then repeat the request, add urgency, reassure, or give up, but cannot start a new scam. Because evidence and label are shared across the four branches, any change in verdict comes from the reply alone.

Dataset at a glance

  • 2,643 calls in Romanized Hindi–English, Tamil–English, and Telugu–English, from 2,880 generated.
  • 60 scenario families across banking and payments, jobs and recruitment, and e-commerce and delivery.
  • Every scam has a genuine counterpart built on the same situation.
  • Family-level splits, with 12 held-out test families and 540 test calls, half of them fraud.

What we found

Given only the opening, identical branches receive identical verdicts. Adding the first reply splits them in 30–80% of conversation sets across 14 open models with 270M–1.8B parameters.

The direction depends on the model. For Qwen2.5-1.5B, the strongest zero-shot model, a suspicious customer doubles false alarms on genuine calls, and a cooperative victim hides scams. Hiding the victim’s lines helps only partly, because the caller’s own turns react to the reply.

Fine-tuning Qwen2.5-1.5B with LoRA raises macro-F1 from 0.73 to 0.86, cuts false alarms, and roughly halves the disagreement between branches. Training on branched calls matches unbranched accuracy with more consistent verdicts. Scams where the victim cooperates still have the lowest recall.

What this means

Pooled accuracy hides the failure that matters most. After fine-tuning, the people a warning should protect, victims who comply, are still the ones most likely to be missed.

The calls are synthetic and not yet human-validated, the test set covers 12 families, and the evaluation uses a single prompt with three seeds. The dataset and code will be released.

Cite

Download .bib
ACM
Rishi Ahuja, Kumar Prateek, and Simranjit Singh. 2027. It’s Not What the Scammer Says, It’s How You Reply: Small Scam Detectors Judge Code-Mixed Fraud Calls by the Victim. In AAAI-27 Student Abstract. To appear.
APA
Ahuja, R., Prateek, K., & Singh, S. (in press). It’s Not What the Scammer Says, It’s How You Reply: Small Scam Detectors Judge Code-Mixed Fraud Calls by the Victim. In AAAI-27 Student Abstract.

@inproceedings{ahuja2027scambranch,  title     = {It's Not What the Scammer Says, It's How You Reply: Small Scam Detectors Judge Code-Mixed Fraud Calls by the Victim},  author    = {Ahuja, Rishi and Prateek, Kumar and Singh, Simranjit},  booktitle = {AAAI-27 Student Abstract},  year      = {2027},  note      = {Accepted for presentation}}

More research