---
title: "Comment by Ahmet Özel on \"CoCa: Contrastive Captioners Are Image-Text Foundation Models (2022)\n\n\" | Hashnode"
description: "The within-batch negatives deserve attention when annotated class names and noisy alt text share a batch. Two different images with the same class label are not"
url: "https://hashnode.com/posts/coca-contrastive-captioners-are-image-text-foundation-models-2022/6ac8959cc9a15a29d3da4a53/comment/6acb5a5e5ce33115e2100def"
---

[Discussion](https://hashnode.com/posts/coca-contrastive-captioners-are-image-text-foundation-models-2022/6ac8959cc9a15a29d3da4a53)

Comment [CoCa: Contrastive Captioners Are Image-Text Foundation Models (2022)](https://hashnode.com/posts/coca-contrastive-captioners-are-image-text-foundation-models-2022/6ac8959cc9a15a29d3da4a53)

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[Oct 11](https://hashnode.com/posts/coca-contrastive-captioners-are-image-text-foundation-models-2022/6ac8959cc9a15a29d3da4a53/comment/6acb5a5e5ce33115e2100def)

The within-batch negatives deserve attention when annotated class names and noisy alt text share a batch. Two different images with the same class label are not semantically unrelated, yet a contrastive objective can treat their pairings as negatives. Captioning supervision is asking a different question about those same examples.

I would keep that tension in mind when interpreting the loss-weight ablation: more captioning weight may change which distinctions the representation preserves, rather than simply adding finer detail everywhere. Comparing same-class retrieval errors across the loss settings would be a useful complement to the aggregate zero-shot classification result.

[Nick One-Drop](https://hashnode.com/@nickonedrop)

[Oct 11](https://hashnode.com/posts/coca-contrastive-captioners-are-image-text-foundation-models-2022/6ac8959cc9a15a29d3da4a53/comment/6acb5a5e5ce33115e2100def#reply-6acb6a381288af1490975521)

Thanks for pointing this out. I agree—I focused on the authors’ interpretation, but the false-negative issue leaves room for a different explanation.

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[Oct 30](https://hashnode.com/posts/coca-contrastive-captioners-are-image-text-foundation-models-2022/6ac8959cc9a15a29d3da4a53/comment/6acb5a5e5ce33115e2100def#reply-6ae433a24ad0f273f3e70f98)

I would still frame false negatives as a competing hypothesis rather than a conclusion from the ablation alone. The loss weights change several training signals together, so the aggregate score cannot isolate that mechanism.

A useful diagnostic would separate same-class retrieval confusions from different-class errors at each weight setting. If those move differently, it would motivate a more controlled experiment with the batch composition or positive-pair definition, while keeping the original paper result and the proposed explanation clearly distinct.
