Since scripts and compiled languages are not that different at the first 2 steps, I would take a look into the cool compiler which has actual lessons about NFA to DFA transformations, separation of steps, left and right recursive algorithms, the AST and so on.
theory.stanford.edu/~aiken/software/cool/cool.html youtube.com/watch
You can leave the compiler step and just build an interpreter so you stay in the NFA and usually if you take python or PHP you could just create a certain OP/Byte-Code as cache so you could skip the lexer and parser and directly go to the Interpreter after the first two steps have been processed. (caching of prepared code)
You could go for the atomic bursts concept which was common in PHP till version 5.6-ng where the AST was firstly introduced, this is only to give a "popular" language example without AST, but I would go for the AST (abstract syntax tree) if possible. You can have better possibilities to optimize via discrete math (Graph analysis) as well as JIT
a really nice talk about language design esp about a functional recursion is this one: https://www.youtube.com/watch?v=Nlqv6NtBXcA
if you go for the llvm as marco suggested https://www.youtube.com/watch?v=FnGCDLhaxKU this is a nice talk about the C++ compiler not really about language synthesis but still a nice optimization step talk.
In my opinion, first you should start with simple declarative syntax which means you don't need to handle variables which makes a lot of things easier.
so don't go for JS don't go for anything complicated try to implement a calculator first and try to add basic function to it without handling variable in memory.
Understand the history of programming don't just jump into something heavy first try to understand the basics:
those things on their own are huge improvement on your coding skills because you understand how languages do work.