@jimmie

ML Experiment Log: Squaring as activation layer

Oct 8, 2022 · 4 min read · The purpose of an activation layer is to add some non-linearity to the network. This allows expressing much more complex tasks with our networks. But why do we generally use functions that have a low/zero less than zero component and a nearly 1 comp...

On the fly expansion of linear layers

Sep 27, 2022 · 3 min read · A linear layer takes a set of inputs, and for each node in its layer it sums weights*inputs to calculate the node output. This output is then fed to the next layer (or is the network output if it's the final layer). Neural network architectures are ...

ML Experiment Log - Polynomial Network + MNIST

Sep 8, 2022 · 2 min read · I wanted to try implementing previous work nn + polynomial solvers against a larger problem, in this case MNIST. For anyone hopping in here, I'm exploring adapting neural networks to use polynomial solvers instead of gradient descent. I'm looking sp...

ML Experiment Log - NN + Polynomial Solvers Pt. 2

Sep 5, 2022 · 2 min read · In the previous attempt, a polynomial based neural network was setup and we had some inconsistent results coming back. My assumption at that point was that the correct solutions weren't being found, and maybe the work of the solver needed simplified....

ML Experiment Log - NN + Polynomial Solvers

Sep 3, 2022 · 3 min read · This approach doesn't seem to produce good results. The idea is to create a neural network completely from polynomials (doesn't really require much), differentiate it and solve for critical points. Use the critical points to quickly find the weight ...