Primarily because they are made of electronic components.
It's not that they can't be designed to understand non-binary digits, it's just that signals are easier to transmit if there are only 2 levels
The signals work in such a manner that a specific level of voltage is meant to depict a 1 and another voltage level indicates a 0.
There are significant drawbacks, if the number of voltage levels for transmitting signals is more than 2, hence binary system is used in all electronic components.