Wow, this is really an interesting feature that has been added, but I guess the downside would be loading a different image on a larger viewport on a really slow network.
Actually, it depends on how you want to use it. For example, some people take the picture they want to use and resize it to smaller pictures for smaller devices. That way it's only for the best.
But I get your point, it's something to keep in mind (you can add some JS to check for the network connection quality and change the srcset depending on that, or something like that) 👍🏼