---
title: "Discussion on \"CoCa: Contrastive Captioners Are Image-Text Foundation Models (2022)\n\n\" | Hashnode"
description: "Discussion on \"CoCa: Contrastive Captioners Are Image-Text Foundation Models (2022)\n\n\". This is a 2022 paper from Google. For me, the key idea is captured in the following passage and two diagrams.\n\n\n\n\n\n\nTraining combines annotated images with noisy image–text pairs. The class labels in"
url: "https://hashnode.com/posts/coca-contrastive-captioners-are-image-text-foundation-models-2022/6ac8959cc9a15a29d3da4a53"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [CoCa: Contrastive Captioners Are Image-Text Foundation Models (2022)](https://nickonedrop.hashnode.dev/coca-contrastive-captioners-are-image-text-foundation-models-2022?utm_source=hashnode&utm_medium=feed)

This is a 2022 paper from Google. For me, the key idea is captured in the following passage and two diagrams. Training combines annotated images with noisy image–text pairs. The class labels in

Oct 9 · 3 min read · nickonedrop.hashnode.dev  
Read the article

-   [#deeplearning](https://hashnode.com/tag/deeplearning)
-   [#machine-learning](https://hashnode.com/tag/machine-learning)
-   [#computer-vision](https://hashnode.com/tag/computer-vision)
-   [#image-retrieval](https://hashnode.com/tag/image-retrieval)

## Written by

[Nick One-Drop](https://hashnode.com/@nickonedrop)

@nickonedrop

  
[View profile](https://hashnode.com/@nickonedrop)

ML researcher and solo builder behind monore. Creating things that make people feel.

ML researcher and solo builder behind monore. Creating things that make people feel.

## Responses 3

Markdown supported · ⌘↵ to post

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[Oct 11](https://hashnode.com/posts/coca-contrastive-captioners-are-image-text-foundation-models-2022/6ac8959cc9a15a29d3da4a53/comment/6acb5a5e5ce33115e2100def)

The within-batch negatives deserve attention when annotated class names and noisy alt text share a batch. Two different images with the same class label are not semantically unrelated, yet a contrastive objective can treat their pairings as negatives. Captioning supervision is asking a different question about those same examples.

I would keep that tension in mind when interpreting the loss-weight ablation: more captioning weight may change which distinctions the representation preserves, rather than simply adding finer detail everywhere. Comparing same-class retrieval errors across the loss settings would be a useful complement to the aggregate zero-shot classification result.

[Nick One-Drop](https://hashnode.com/@nickonedrop)

[Oct 11](https://hashnode.com/posts/coca-contrastive-captioners-are-image-text-foundation-models-2022/6ac8959cc9a15a29d3da4a53/comment/6acb5a5e5ce33115e2100def#reply-6acb6a381288af1490975521)

Thanks for pointing this out. I agree—I focused on the authors’ interpretation, but the false-negative issue leaves room for a different explanation.

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[Oct 30](https://hashnode.com/posts/coca-contrastive-captioners-are-image-text-foundation-models-2022/6ac8959cc9a15a29d3da4a53/comment/6acb5a5e5ce33115e2100def#reply-6ae433a24ad0f273f3e70f98)

I would still frame false negatives as a competing hypothesis rather than a conclusion from the ablation alone. The loss weights change several training signals together, so the aggregate score cannot isolate that mechanism.

A useful diagnostic would separate same-class retrieval confusions from different-class errors at each weight setting. If those move differently, it would motivate a more controlled experiment with the batch composition or positive-pair definition, while keeping the original paper result and the proposed explanation clearly distinct.
