1. Python getting setup#
1.1 Setting up on your own machine - what to install:#
In the sessions in Nottingham we have desktop PCs available with this software preinstalled but you’ll want to set up your own machine to work on outside of the sessions or if doing this course remotely.
Install:
1.2 Some setup stuff for VsCode#
Check that vscode has been correctly installed.
Press windows button and type code. Open the folder containing the project files.
VsCode can be used with pretty much any programming language. It relies on installing extensions to increase the functionality whilst keeping its overall size small.
To use VsCode properly you should always open a folder. During the course I will send you a link to project files for that week. Clicking on the link should download the appropriate materials and open them in a folder. Clicking on the 2 documents icon on the sidebar (shown below) will open a collapsing pane that shows all the current files and allows you to create new ones, rename, delete etc.
If setting up VsCode for yourself on your own computer you’ll need to install a few extensions. You can do this by clicking on the 4 squares icon on the side bar. You then need to search for the extension and click the install button. Some of these extensions are essential, but I have also added a few which you may find helpful.
This extension contains python features such as, Intellisense, Linting, Debugging.
This is required to run a Jupyter Notebook inside VsCode.
For this course you will also need github classroom integration. This will enable you to download the project files each week and submit your work.
I also find autodocstring a particularly helpful extension. It will help create skeleton docstrings which you can then fill in.
I also find test explorer useful.
There are a lot of extensions which are worth exploring.
If you are doing these sessions remotely you will also need to install liveshare if you would like support during the sessions.
this video explains how liveshare works and will help you learn how to get a link to enable someone to join the session and help you.
1.2.2 Command Palette#
VsCode is setup so that everything can be accessed quickly through the command palette. To access it type: ctrl + shift + p
Typing words into the bar will enable you to quickly find the setting you want. Two things to check here:
Terminal:Select Default Profile–> SelectCommand Prompt.
If on Mac or Linux you want to select your default terminal.
Python: Select Interpreter–> Allows you to select which python / environment you will use. After we’ve discussed environments below we’ll use this.
1.2 Some setup stuff for VsCode#
Check that vscode has been correctly installed.
Press windows button and type code. Open the folder containing the project files.
VsCode can be used with pretty much any programming language. It relies on installing extensions to increase the functionality whilst keeping its overall size small.
To use VsCode properly you should always open a folder. During the course I will send you a link to project files for that week. Clicking on the link should download the appropriate materials and open them in a folder. Clicking on the 2 documents icon on the sidebar (shown below) will open a collapsing pane that shows all the current files and allows you to create new ones, rename, delete etc.
If setting up VsCode for yourself on your own computer you’ll need to install a few extensions. You can do this by clicking on the 4 squares icon on the side bar. You then need to search for the extension and click the install button. Some of these extensions are essential, but I have also added a few which you may find helpful.
This extension contains python features such as, Intellisense, Linting, Debugging.
This is required to run a Jupyter Notebook inside VsCode.
For this course you will also need github classroom integration. This will enable you to download the project files each week and submit your work.
I also find autodocstring a particularly helpful extension. It will help create skeleton docstrings which you can then fill in.
I also find test explorer useful.
There are a lot of extensions which are worth exploring.
If you are doing these sessions remotely you will also need to install liveshare if you would like support during the sessions.
this video explains how liveshare works and will help you learn how to get a link to enable someone to join the session and help you.
1.2.2 Command Palette#
VsCode is setup so that everything can be accessed quickly through the command palette. To access it type: ctrl + shift + p
Typing words into the bar will enable you to quickly find the setting you want. Two things to check here:
Terminal:Select Default Profile–> SelectCommand Prompt.
If on Mac or Linux you want to select your default terminal.
Python: Select Interpreter–> Allows you to select which python / environment you will use. After we’ve discussed environments below we’ll use this.
1.3 Github Classroom and Weekly Project Files#
(We assume you are working on windows here. Hopefully if on Mac or Linux you can figure out the equivalent steps. Once Anaconda is installed, restart your terminal and you should be able to follow the same steps as below using your regular terminal.)
Setup a github account
Check all software is present and working:
Open an Anaconda terminal by pressing windows key and typing anaconda
Check git is working in terminal : type
gitcheck you get help print outCheck github cli is working in terminal : type
ghcheck you get help print outregister username and email globally by typing the following:
git config --global user.name "YOURGITUSERNAME" git config --global user.email YOUR@email.com git config --global core.editor "code --wait"
Now open vscode again and open a terminal within vscode (Select terminal > new terminal from the menu bar). Check that the steps 2-3 are still working (If not you may not have added the software to your path variable Add program to path variable.)
Within vscode sign in to github by clicking on the github icon on the left hand side
register for github classroom by clicking on the link sent to you via email and associate your github account with your name. If your name is not there let me know!
Download this weeks files from link. You should now be able to see the files in the sidebar. One of the files should be a Jupyter Notebook called T1. python_setup.ipynb which contains the same information as this page. Open it and continue with the instructions there.
1.3.2 Using Github Classroom#
Using github classroom is fairly simple but since it is so fundamental we’re going to walk through the important things you’ll need to do. Github Classroom is essentially a way to synchronise files between a remote webserver and your computer. Each week when you click on the link a personal webspace is made on this remote webserver for you containing that weeks files. Follow the link below to see this page, it should have a folder for the first weeks resources which you can look at online:
When you clicked the link these files were also copied to your computer. So there are two copies of everything: one local on your pc and one remote on the webserver. When you open Vscode the side bar has a few icons. The top one of 2 documents shows you the active files which you can select. There is also one that looks a bit like a cat at the bottom which is the github classroom.
If you open this you should see something like the following (though you may only have week1’s files)
If when you hover the mouse next to the week 1 project files you see a folder clicking on this will open this project. Then when you click on the icon of 2 documents at the top of the side bar you should see all the files for this week and be able to open them.
Once you have a particular weeks resources open you’ll find that hovering no longer produces a folder but a pair of circular arrows indicating that the folder can be synced with the remote server. Click this button when you start and again when you finish working on the problems. Any changes to files you make can be backed up to the remote server by clicking these arrows.
Open the E1. python_setup_exercises.ipynb file and follow the simple exercise to get you comfortable with this process before continuing.
Workflow for using github classroom
Every week click on the link to setup the project files for that week.
Click the sync button
Have a go at the exercises
Press the sync button and check on the Github Classroom page that your answers have uploaded.
If you don’t finish the exercises in class then you can finish them at home on your own computer. Set it up with the instructions above, the first time you need to click the link in the email and it will download the latest version of your files.
We recommend that whenever you sit down at any computer you start by pressing the sync button and then when you have finished you press it again. Forgetting this can lead to issues – if you realise that this is not working speak to us.
1.4 Anaconda#
There is an ecosystem of software tools that support python development which make your life easier and will help prevent conflicts etc highlighted above.
Package managers (pip, conda)
Virtual environments (virtualenv, conda environments)
Anaconda is a piece of software that enables you to isolate your setup for a particular project from the rest of your system and install particular packages easily. It comes in two versions which can be installed from the following links:
Anaconda - Comes with huge number of packages pre-installed but takes up a lot of space (https://docs.anaconda.com/free/anaconda/install/index.html)
Miniconda - The basics, smaller but can install exactly what you want for each project (https://docs.conda.io/en/latest/miniconda.html)
1.4.1 Virtual environments#
Python often comes pre-installed on many operating systems. However tempting, don’t use it!!! Installing python packages directly will create a world of problems for you down the line. Virtual environments are effectively a sealed box around your python installation. They enable you to completely isolate the particular versions of python and installed packages so that they do not influence or conflict with work for other projects.
We will focus on conda environments, similar behaviours are available for virtualenv (https://docs.python.org/3/library/venv.html)
Whilst all of what we describe below is done in a separate terminal it will work just as well inside the terminal in vscode if you’ve set things up right. For now use the external Anaconda prompt (Mac and Linux just open a regular terminal).
When it opens you’ll see (base) at the front of the prompt. (base) is a kind of underlying environment.
1.4.1.1 Creating a new environment#
There are 3 options for creating your own environment. We can either:
create an empty environment called myproject
conda create -n "myproject"
create an environment called myenv specifying the version of python and some packages we would like installed
conda create -n "myenv" python=3.10 numpy matplotlib pandas=1.5.3
use a special setup file called environment.yml which is in the current working directory which specifies all the details of what we want in the environment.
Before we can do this we need our terminal to be in the same directory as the file environment.yml. Currently my command prompt is in my user directory:
I can find the file path by right clicking on the files inside vscode and selecting “reveal in file explorer”. Then copy the filepath.
Finally in the terminal type cd "path\to\file" which you just copied and the command prompt will indicate that you have moved to a different directory which contains the files you can see in vscode.
Then we can create the environment from the file environment.yml:
conda env create -f environment.yml
1.4.1.2 Activating an environment#
Once we have created an environment we must activate it
conda activate myproject
This needs to be done when you open the prompt each time. People often forget and then end up installing things in (base) rather than the correct environment. Double check. If you see your environment in brackets at the start of the prompt you’ve done it correctly.
We can see what environments are available and have previously been setup on a system
conda env list
1.4.1.3 Is this all this necessary?#
Suppose you have (at least) two projects:
myproject uses python 3.10 because scipy 1.35 requires certain dependencies that don’t exist in python 3.11. It also uses scikit-optimise which uses numpy. The latest version of numpy has deprecated (got rid of) np.int in favour of np.int64 and therefore you have to use an older version of numpy.
mysecondproject uses python 3.11 and the latest numpy
This is a trivial example. With virtual environments you can configure two environments setup the correct way and all you do is activate the correct virtual environment to work on each project. If you don’t do this you’d have to keep installing and uninstalling different versions of packages. People who use your code would also need to keep changing, and it all ends up in a horrible mess! This is particularly true if you use smaller packages (ie not just numpy, matplotlib etc). To make things easy I often name my virtual environments with a name related to the project I’m working on, to help identify which environment should be used for each project.
If you need to switch environments either activate a different environment or deactivate the current one to return to base
conda deactivate
1.4.2 Package management#
Once you have activated your environment you can see which packages are installed:
conda list
you can then install additional packages (N.B pkgname is a placeholder for the name of the package you want to install):
conda install numpy
install particular version
conda install matplotlib=3.5.2
In addition to conda there is another package manager called pip. This can also be used to install packages into conda environments. It is generally recommended to try conda first and then if that fails try pip like this:
pip install pkgname
If you need to uninstall either conda or pip packages you can simply type:
conda uninstall pkgname
pip uninstall pkgname
If you want to export the contents of your environment (perhaps so that someone else can set up theirs in an identical way)
conda env export --no-builds > environment.yml
The additional option (–no-builds) is not necessary but help to make sure that the file when exported will work on other machines.
Now delete the environment (it’s important to do this before setting up the new one on the same computer. Normally you would use this file to setup the enviroment on a different computer but trying to setup on the same computer if the environment already exists creates a conflict).
conda env remove -n myproject
Once you have exported your environment you can create a new environment from the file:
conda env create -f myproject.yml
Don’t try and remember all the conda commands. Here is a quick cheat sheet where you can look things up:
If you need to dive a bit deeper then you can look at the conda reference api.
1.5 Running a python program#
1.5.1 Using the command line#
The most basic way to run python is at a command prompt.
Open Anaconda terminal
Activate your myproject environment
Type “python”
The >>> indicates you are using python in an interactive way
print('Hello World')
ctrl + z exits the python mode and returns you to the command prompt.
Use the windows file explorer to create a folder for your python projects in your OneDrive documents folder. Copy the address from the searchbar and use the following command to set the current working directory of the command prompt:
cd "C:\Users\<USERNAME>\OneDrive - The University of Nottingham\Documents\PROJECTS"
then type:
notepad
Write: print(‘Hello World’) inside, save the file as hello.py (N.B .py extension. To do this you will change the file type dropdown to all files and type .py on the end of the filename) and close the file
then at the command prompt:
python hello.py
This is better since it enables us to write an entire python script and run it. It also can be great for running quick scripts without having to open Integrated Development Environments. However, writing code is a lot easier in dedicated IDEs with tools to support the coding.
1.5.2 Jupyter Notebooks#
Jupyter Notebooks provide an interactive approach and are used a lot in data analysis. This is because you can break code into little pieces and quickly test and rerun portions of code. They support python (and a number of other languages) mixed with markup text and are great for exploratory dataanalysis as code, graphs, outputs, notes can be kept together.
At the anaconda prompt, activate your environment:
conda activate myproject
jupyter notebook
These can be run inside an IDE like vscode or can be run independently. To run independently, in a conda environment install jupyter notebook support:
conda install jupyter notebook
At the conda prompt then type:
jupyter notebook
this starts a local webserver and opens the local file directory in a webbrowser.
Navigate to your notebook or create a new one and it opens up the notebook in a browser (Jupyter notebooks have a .ipynb extension):
However, for this project we’ll run the Jupyter Notebook inside VsCode as it has more features. Open the notebook inside vscode. To do this you’ll need to have installed the Jupyter Extension:
As you can see in this notebook format there are different types of cells:
Markdown for writing notes, structuring, displaying
Code cells for running python code
1.5.2.1 Markdown#
(Note to see how to write the markdown that results in each item below you’ll need to look at the Jupyter Notebook. Double clicking on this cell will show you the Markdown code used)
Markdown comes with multiple ways to do things so here’s a quick guide to doing some common things. The top title can be deined with a # and progressively smaller headings just have more #####.
Titles#
Headings#
subheadings#
subsubheadings#
Write in bold or italics.
We can defined code blocks with proper syntax highlighting by adding ``` followed by the language (the one below is python but we can also use console, c, html etc)
print('typesetting some code like this')
Draw horizontal lines to delineate sections by using — at the start of a line
HTML tags generally work although not as readable so use sparingly.
Insert image from a file:
<img src="resources/images/logo.png" alt="Drawing" style="width: 100px;"/>
Insert video from file
<video alt="test" controls style="width: 200px;">
<source src="test.mp4" type="video/mp4">
</video>
If you want to insert an image or video directly from a url (make sure it is url to image or video and not the page in which they are embedded)
Insert image from url
<img src="https://www.ntuclearninghub.com/documents/39367/4216797/Python-Symbol.png/369e410e-a90f-f887-c2dc-61f7ef761476/" alt="Drawing" style="width: 20em;"/>
For video change src attribute in video example.
If you want to do something else you can use html as I have done above or refer to this quick look up guide for Markdown text
1.5.2.2 Code cells#
Type python code in directly
shift + Enterto run that cellResults stay in memory so sequence matters (run the first cell followed by the second cell twice, then the first cell then the second cell and look at the values printed out)
In vscode the jupyter notebook has all the features we describe in the IDEs section
a=1
a+=1
a
2
1.5.3 IDEs#
Integrated development environments provide a whole range of tools that make development easier:
Intellisense
Viewing Documentaton
Debugging
Testing
Git integration
Host of add ons
Whilst there are lots of IDEs the two main leaders in the python world are:
PyCharm (www.jetbrains.com/pycharm) : Can use full featured paid version for free as a student if you register
VsCode (code.visualstudio.com)
For the purposes of this course we will be using VsCode
1.5.4 Running a script in vscode#
Before running a script you need to make sure you have activated the correct environment. You can do this by clicking on the environment name in the bottom left hand corner of the screen and selecting the correct environment. Alternatively use the command paletter ctrl + shift + p and type Python: Select Interpreter and select the correct environment.
In an IDE you will usually create a .py script in the file tab and then run it directly. In vscode you can do this by clicking on the run button at the top of the screen.
You can also run the script from the command line by typing:
python hello.py
1.5.5 Other approaches#
Online coding:
cocalc.com
repl.it
Github codespaces
colab.research.google.com
Explore at your own leisure
1.5.6 Vscode Keyboard Shortcuts#
Sometimes these shortcuts maybe configured differently on different systems. You can see all the keybindings and provide some new ones of your own by going to
File > Preferences > Keyboard Shortcuts. If you are running them on a Mac replace Ctrl with Cmd. Here are a few I use frequently.
Short cut |
Command |
|---|---|
|
Open Command Palette |
|
Indent / unindent selected text |
|
Select all occurences of word |
|
Zoom in or out |
|
Toggle sidebar visibility |
|
Toggle terminal visibility |
In addition to keyboard shortcuts I recommend turning on autosave: File > Autosave. Once logged in to your github account, you can also turn on settings sync to make sure things are setup the same way on any computer you use.
1.6 Importing packages#
Once you have installed a package into your environment you can import it into either a Jupyter Notebook or a script you are writing in the IDE. These imports are usually done in a group at the top of your code. In your files you will find a folder called resources which contains a file called example_function.py. Open this in the editor and look at the contents. Python makes importing so easy. A package is a folder containing python files. Individual .py files are modules. Any .py python file can be imported using its name. There are multiple ways to import and they all mean subtlely different things.
To import everything inside example_import.py we can write:
from resources.example_import import *
Whilst this works and you will find a few big projects (e.g. pyqt) which use this syntax it is not recommended. This is because:
I might have two modules imported and by chance they might have functions that are called the same thing. These would then conflict
It makes code harder to understand. If I have lots of imports how do I know which module example_function() came from?
N.B. This is sometimes referred to as “polluting the namespace”. When we look at functions you’ll see that you could have local and global scopes which effect which variables are accessible in and outside a function. A namespace is somewhat similar and consists of the collection of variables, functions, classes accessible to a particular script.
Below are a few far better ways to import and then how to use the imported functions in your code.
import resources.example_import
import resources.example_import as example_nickname
from resources.example_import import example_function
resources.example_import.example_function()
example_nickname.example_function()
example_function()
This is executing the example function
This is executing the example function
This is executing the example function
As we showed above you can import packages and give them a nickname. You usually do this to make it quick to type whilst clearly identifying which package functions have come from. You could use anything but well known packages such as numpy, pandas etc have standard imports which it’s worth following for clarity
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import scipy
Warning - Problems importing#
If you install packages using conda or pip in your environment then you will be able to import them using their name e.g. import numpy. If you provide the address as we did above relative to where the python file is being run from it will also import them. However, it is worth understanding how python finds packages. If you are looking for a module example_import.py then python searches the current directory and then if it doesn’t find it, it will look in a list of other places that are specified by a path variable. The first time it finds a file with the right name it will stop. You can see this by running:
import sys
print(sys.path) # This is the list of places python searches
import example_import # Note it doesn't find this because it is inside the resources folder and so errors
['C:\\Users\\ppzmis\\AppData\\Local\\miniconda3\\envs\\instructor\\python310.zip', 'C:\\Users\\ppzmis\\AppData\\Local\\miniconda3\\envs\\instructor\\DLLs', 'C:\\Users\\ppzmis\\AppData\\Local\\miniconda3\\envs\\instructor\\lib', 'C:\\Users\\ppzmis\\AppData\\Local\\miniconda3\\envs\\instructor', '', 'C:\\Users\\ppzmis\\AppData\\Local\\miniconda3\\envs\\instructor\\lib\\site-packages', 'C:\\Users\\ppzmis\\AppData\\Local\\miniconda3\\envs\\instructor\\lib\\site-packages\\win32', 'C:\\Users\\ppzmis\\AppData\\Local\\miniconda3\\envs\\instructor\\lib\\site-packages\\win32\\lib', 'C:\\Users\\ppzmis\\AppData\\Local\\miniconda3\\envs\\instructor\\lib\\site-packages\\Pythonwin']
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
Cell In[4], line 3
1 import sys
2 print(sys.path) # This is the list of places python searches
----> 3 import example_import # Note it doesn't find this because it is inside the resources folder and so errors
ModuleNotFoundError: No module named 'example_import'
#If we want it to find a file in a different place
import sys
sys.path.insert(0,'resources') #Add a relative address to the front of the path list
print(sys.path)
import example_import # Now the import works
example_import.example_function()
Warning - version numbers and my code has stopped working!#
Over time there have been important changes and developments to packages, some of which may break your code! If you are doing something important pay attention to package version numbers. conda list or exporting your environment is a good way to make sure you can get old code to work. Version numbers look like this:
pandas 2.0.1
The further left the number, the more significant the changes and the more likely it is to make a difference to your code.