To switch the python version in pyspark, set the following environment variables. I was working in an environment with Python2 and Python3. I had to use Python3 in pyspark where the spark was using Python 2 by default.
Python 2 was pointing to –> /usr/bin/python
Python 3 was pointing to –> /usr/bin/python3
To configure pyspark to use python 3, set the following environment variables.
export PYSPARK_PYTHON=/usr/bin/python3
export PYSPARK_DRIVER_PYTHON=/usr/bin/python3
Now type pyspark in the commandline.
All About Tech 
Awesome Amal..
Very Much inspired by this – “Victory goes to the player who makes the next-to-last mistake”
Please keep doing the good work.
Thank you Sai Praneeth.
Thank you for sharing this tip.