To do my web crawling, I started using Selenium,which is a Python module for doing web crawling.
I installed it from my command prompt by doing ‘pip install selenium’, and Selenium was working just fine in PyCharm and Python shell. But when I tried doing ‘import selenium’ in a Jupyter notebook, I kept getting a module not found error.
It turned out to be a Python path issue. In short, Selenium had already been installed, but Jupyter could not import Selenium because it wasn’t pointing to the path where Selenium had been installed.
The usual digging led me to this helpful thread, but in the end I couldn’t get Selenium to work on my Jupyter notebook by following the instructions provided.
Fortunately, an intern told me about Anaconda Navigator, a GUI-based application that could be used to install packages for the (virtual) environment running Jupyter notebook. As long as you have anaconda installed, you just had to run the below command to install Navigator:
So I tried searching for the Selenium package on Anaconda Navigator, but the search returned no results.
After doing some digging, I came across this site that had a piece of code I could run to get Selenium:
conda install -c conda-forge selenium
After running this code in my command prompt, I got selenium to work on Jupyter notebook!
Now I could use Selenium and Chrome Driver in a Jupyter notebook just fine.