This is a very recognisable failure. The agent optimises the Dockerfile for the environment it can see, which is usually a local shell, and has no idea the CI runner uses a different base image, cache layout or build arguments.
Two things have worked for us. First, put the CI contract in the repository in plain language: which image the pipeline builds on, which steps must pass, and which files the agent should not touch without a reason. Agents follow that kind of note surprisingly well. Second, make a green pipeline part of the definition of done for any change, so the agent's task is not finished when the file looks right locally but when the real build passes.
That second rule sounds obvious, but it changes how work gets scoped. Each change becomes a small, bounded problem with a verifiable outcome, which is exactly what agents handle best.