Polynomials, AGI, and Neural Nets

If neural networks, including modern LLMs, are just a series of simple linear regressions whose outputs are modified by a simple activation function to add a bit of non-linearity, and then the output is fed again to another layer and so on, does that...