this would be one thing the problem is not the up / down part where the DDL is concerned. it's the transformation parts where you cannot actually go back anymore :) Where you don't change the structure but the values.
Writing a simple migration thing is pretty easy ->
if it's SQL -> execute directly as batch if it's custom -> execute it as the language
you have up or down which can be just files to keep it simple
you order the files with 000 prefixes so they execute in order and this actually suffices most use cases in my experience
Usually applications don't change their SQL infrastructure and if they do they do it after 2-3 years of development and at some point some custom functions was applied to leverage the engine.
But that's just me.
TL;DR;
my experience :) but I'm very interested if someone has a general hook system where you can hook your application into the SQL migration while still being language agnostic maybe with a small wrapper for stdin|stderr