---
title: "Discussion on \"A hex literal made LLM code reviewers raise false alarms\" | Hashnode"
description: "Discussion on \"A hex literal made LLM code reviewers raise false alarms\". A secure 3072-bit key written as 0xC00 drew false alarms from LLM code reviewers. Results from 16 open models on 90 snippet pairs."
url: "https://hashnode.com/posts/a-hex-literal-made-llm-code-reviewers-raise-false-alarms/6ae24dc650f6fbe56d4c46ef"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [A hex literal made LLM code reviewers raise false alarms](https://sunnychokshi.hashnode.dev/a-hex-literal-made-llm-code-reviewers-raise-false-alarms?utm_source=hashnode&utm_medium=feed)

A secure 3072-bit key written as 0xC00 drew false alarms from LLM code reviewers. Results from 16 open models on 90 snippet pairs.

Oct 28 · 3 min read · sunnychokshi.hashnode.dev  
Read the article

-   [#cryptography](https://hashnode.com/tag/cryptography)
-   [#llm](https://hashnode.com/tag/llm)
-   [#security](https://hashnode.com/tag/security)
-   [#code-review](https://hashnode.com/tag/code-review)

## Written by

[Sunny Chokshi](https://hashnode.com/@sunnychokshi)

@sunnychokshi · Massachusetts, USA

  
[View profile](https://hashnode.com/@sunnychokshi)

Doctoral student researching LLM security

I am a doctoral student at the University of the Cumberlands. My dissertation looks at the adversarial robustness of large language models in cybersecurity applications. I write short notes on experiments I ran myself, mostly LLM code review and crypto API misuse, with the numbers and a link to the preprint or dataset behind each one. Google Scholar: https://scholar.google.com/citations?user=0dosaOcAAAAJ

## Responses 5

Markdown supported · ⌘↵ to post

[Kartik N V J K](https://hashnode.com/@kartik-nvj)

[Oct 29](https://hashnode.com/posts/a-hex-literal-made-llm-code-reviewers-raise-false-alarms/6ae24dc650f6fbe56d4c46ef/comment/6ae34fcbcabb70c219e8bec1)

With a 3072-bit key written as 0xC00 getting flagged, I'd want to read the reviewer's stated reason next to the flag. If it says the key is too small, it misread the value. If it complains about a magic number, a style note got scored as a security finding. Did the false positives come with reasons you could sort that way?

[Sunny Chokshi](https://hashnode.com/@sunnychokshi)

[Oct 29](https://hashnode.com/posts/a-hex-literal-made-llm-code-reviewers-raise-false-alarms/6ae24dc650f6fbe56d4c46ef/comment/6ae34fcbcabb70c219e8bec1#reply-6ae35316e5d6eb2076a9ec33)

Yes, I read the replies for these. For the two 3072-bit twins (0xC00 and 3 \* 2 \*\* 10), 24 replies flagged them. All 24 decoded the value correctly as 3072. 21 then called it too small, weak or insufficient, two called it non-standard, and one cited the backend. So the models read the value and still judged it wrongly. Neither of your two cases fits exactly. It is a wrong stated reason, not a misread number. Sorting every false positive this way is the check I would add next.

[AI Developer](https://hashnode.com/@lycore)

[Oct 29](https://hashnode.com/posts/a-hex-literal-made-llm-code-reviewers-raise-false-alarms/6ae24dc650f6fbe56d4c46ef/comment/6ae2d58627cf9dfd81c907eb)

Interesting that most of the false-alarm rise is on safe values written unusually, like 0xC00. For the open question at the end, a cheap ablation would be separating normalization from reasoning: pre-resolve constants with a deterministic step (an AST pass that folds literals and follows simple aliases), then hand the reviewer the folded value alongside the original code. If false positives still drop only for the folded variant and not for a prompt that merely says 'normalize values first', that would suggest the model can't do the arithmetic reliably rather than being ignorant of the spelling. Also, since two models flag nearly every snippet, it may be worth reporting results with a threshold or flag-rate-matched comparison so the indirection effect isn't confounded by base flag rates.

[Sunny Chokshi](https://hashnode.com/@sunnychokshi)

[Oct 29](https://hashnode.com/posts/a-hex-literal-made-llm-code-reviewers-raise-false-alarms/6ae24dc650f6fbe56d4c46ef/comment/6ae2d58627cf9dfd81c907eb#reply-6ae35300289414ae0e800634)

Useful design. Folding the literal first and giving the reviewer the folded value next to the original code would separate arithmetic from judgment. In my replies the models already got 3072 right and then judged it wrongly, so I would expect the folded variant to help less than a prompt that only says 3072 bits is fine. I have not tested that. On flag rates, I report balanced accuracy and pair-cluster intervals, and I have not run a flag-rate-matched comparison. I will note that as a limit.

[AI Developer](https://hashnode.com/@lycore)

[Oct 29](https://hashnode.com/posts/a-hex-literal-made-llm-code-reviewers-raise-false-alarms/6ae24dc650f6fbe56d4c46ef/comment/6ae2d58627cf9dfd81c907eb#reply-6ae36211ab46857a53df754b)

That distinction matters: if the models decode 3072 correctly and then misjudge it, the failure is in the security-judgment step, not the arithmetic. It also suggests a third arm could help: give the reviewer the folded value plus an explicit reference (for example, 'RSA 3072 meets current minimum guidance') and check whether the 21 'too small' verdicts flip, which would test whether it's a missing-knowledge issue or a miscalibration one. Noting the flag-rate limit sounds like the right call.
