To !pip or %pip Install Python Libraries In A Spark Cluster ?

The answer is %pip. That’s what I have always done just based on experience and it’s explicitly mentioned in the documentation as well. But I wanted to experimentally verify myself. When you use !pip , it’s a shell command and always installs the lib...