1. Python getting setup#
1.1 Setting up on your own machine - what to install:#
In the sessions in Nottingham we have desktop PCs available with this software preinstalled but you’ll want to set up your own machine to work on outside of the sessions or if doing this course remotely.
Before installing any of these softwares please check whether you already have them installed. Press the window’s key and type their name to check. If you have miniconda or anaconda already installed they can conflict with each other. You should only have one copy of each, and one of miniconda or anaconda installed.
Install:
1.2 Some setup stuff for VsCode#
Check that vscode has been correctly installed.
Press windows button and type code.
VsCode can be used with pretty much any programming language. It relies on installing extensions to increase the functionality whilst keeping its overall size small.
To use VsCode properly you should always open a folder. During the course I will send you a link to project files for that week. We’ll walk through this process later. Clicking on the 2 documents icon on the sidebar (shown below) will open a collapsing pane that shows all the current files and allows you to create new ones, rename, delete etc.
If setting up VsCode for yourself on your own computer you’ll need to install a few extensions. You can do this by clicking on the 4 squares icon on the side bar. You then need to search for the extension and click the install button. Some of these extensions are essential, but I have also added a few which you may find helpful.
Essential extensions:
This extension contains python features such as, Intellisense, Linting, Debugging.
This is required to run a Jupyter Notebook inside VsCode.
Optional extensions:
I also find autodocstring a particularly helpful extension. It will help create skeleton docstrings which you can then fill in.
I also find test explorer useful.
There are a lot of extensions which are worth exploring.
1.2.2 Command Palette#
VsCode is setup so that everything can be accessed quickly through the command palette. To access it type: ctrl + shift + p
Typing words into the bar will enable you to quickly find the setting you want. Two things to check here:
Terminal:Select Default Profile–> SelectCommand Prompt.
If on Mac or Linux you want to select your default terminal.
Python: Select Interpreter–> Allows you to select which python / environment you will use. After we’ve discussed environments below we’ll use this.
1.2 Some setup stuff for VsCode#
Check that vscode has been correctly installed.
Press windows button and type code.
VsCode can be used with pretty much any programming language. It relies on installing extensions to increase the functionality whilst keeping its overall size small.
To use VsCode properly you should always open a folder. During the course I will send you a link to project files for that week. Clicking on the link should download the appropriate materials and open them in a folder. Clicking on the 2 documents icon on the sidebar (shown below) will open a collapsing pane that shows all the current files and allows you to create new ones, rename, delete etc.
If setting up VsCode for yourself on your own computer you’ll need to install a few extensions. You can do this by clicking on the 4 squares icon on the side bar. You then need to search for the extension and click the install button. Some of these extensions are essential, but I have also added a few which you may find helpful.
Essential extensions:
This extension contains python features such as, Intellisense, Linting, Debugging.
This is required to run a Jupyter Notebook inside VsCode.
Optional extensions:
I also find autodocstring a particularly helpful extension. It will help create skeleton docstrings which you can then fill in.
I also find test explorer useful.
There are a lot of extensions which are worth exploring.
1.2.2 Command Palette#
VsCode is setup so that everything can be accessed quickly through the command palette. To access it type: ctrl + shift + p (On a Mac this is cmd + shift + p)
Typing words into the bar will enable you to quickly find the setting you want. Two things to check here:
Terminal:Select Default Profile–> SelectCommand Prompt.
If on Mac or Linux you want to select your default terminal.
Python: Select Interpreter–> Allows you to select which python / environment you will use. After we’ve discussed environments below we’ll use this.
1.3 Github classroom and weekly project files#
1.3.1 First time setup of Github Classroom#
N.B you may need to repeat some of this the first time you use a new computer.
(We assume you are working on windows here. Hopefully if on Mac or Linux you can figure out the equivalent steps. Once Anaconda is installed, restart your terminal and you should be able to follow the same steps as below using your regular terminal.)
Setup a github account
Check all software is present and working:
Open an Anaconda terminal by pressing windows key and typing anaconda
Check git is working in terminal : type
gitcheck you get help print outCheck github cli is working in terminal : type
ghcheck you get help print outregister username and email globally by typing the following:
git config --global user.name "YOURGITUSERNAME" git config --global user.email YOUR@email.com git config --global core.editor "code --wait"
Now open vscode again and open a terminal within vscode
Check that the steps 2-3 are still working (If not you may not have added the software to your path variable Add program to path variable.)
Within vscode sign in to github by clicking on the github icon on the left hand side
register for github classroom by clicking on the link sent to you via email and associate your github account with your name. If your name is not there let me know!
Open your file explorer and somewhere in your Onedrive, create a new folder called
phys4038
1.3.2 Weekly procedure for getting teaching materials#
N.B you only need to do this once per assignment (if you work on a laptop or different computer at home you may need to repeat).
Using github classroom is fairly simple but since it is so fundamental we’re going to walk through the important things you’ll need to do each week. Github Classroom is essentially a way to synchronise files between a remote webserver and your computer. Each week you will be sent a link which will take you to a page like the one below when you click on it. Accept the assignment.
When you accept the link a personal webspace is made on this remote webserver for you containing that weeks files.
Copy the web address of this page and paste it into the address bar of your web browser. If you ever need to find this page again you can do so by going to this link which will have all the weeks assignments listed:
On the webpage you opened you should see a big green button that says Code. Open this and copy the command that appears in the box. It should look something like: https://github.com/mpags-python/wk1-python-setup-mikesmithlab.git but setup with your username and the week you are working on.
On your computer, open the file explorer and go to the folder you created in step 8 of previous section.
Then right click in the folder and select Open in Terminal. For this step it doesn’t matter if this is a command prompt or powershell terminal etc.
Into the terminal type git clone and paste the command you copied from the web page. It should look something like this:
git clone https://github.com/mpags-python/wk1-python-setup-mikesmithlab-1.git
This will copy files from the remote webserver to your local computer. You should now see a new folder in the file explorer with the wk1-python-setup.......
Once you have done this open this new folder in vscode.
Here is a summary you can follow each week:
Click on the link and accept the assignment.
Copy the web address into your web browser.
Click the green
Codebutton and copy the command.Open the file explorer and go to the folder you created in step 8 of previous section.
Right click in the folder and select
Open in Terminal.type
git cloneand paste the command you copied from the web page.Go to vscode and open the folder you just cloned.
1.3.3 Working with the weekly files#
Now that you have the files for this week open in vscode you can start working on them.
Once you have finished working on the files during that session you need to sync the changes you have made with the remote webserver. There is an exercise to guide you through this but there are 3 steps to remember:
Add changes
Commit changes
Sync changes
The first two steps are usually done at the same time. You can think of this as like saving your changes but it applies to any files in the folder you have open in vscode, not just the one you are working on. Open up the git tab (3rd symbol in the sidebar):
To add the changes you’ve made, hover over the top + symbol as shown (there are 2. This one only appears as you hover level with the word Changes and a tool tip says Stage All Changes). Click on it. Now type a descriptive message into the text box and then click the big blue Commit button. This effectively saves the changes you have made to the files in the folder to your local computer.
The third step is to sync the changes you have made to the remote webserver. Once you have committed, this is done by clicking on the Sync Changes button at the top of the git tab. This will push your changes to the remote webserver and pull any changes that have been made by others since you last synced.
Making sure your local files are up to date#
Every time you start work on a computer you need to make sure that your local files are up to date with the remote webserver. This is done by clicking on the Sync Changes symbol at the bottom of the vscode window (2 arrows forming a circle). This will update your local files.
Just remember#
Whenever you sit down at any computer you start by pressing the sync button
when you finish working at any computer
add,commitandsyncagain.
Forgetting this can lead to issues – if you realise that something is not working don’t worry we can recover it. Speak to us.
Open the Exercises 1. python_setup_exercises.ipynb file and follow the simple exercise to get you comfortable with this in section 1.2 with this process before continuing.
Make sure you are comfortable with this process. Only files on Github will be visible to markers, so it is important that you sync your changes each week.
1.4 Anaconda#
There is an ecosystem of software tools that support python development which make your life easier and will help prevent issues.
Virtual environments (virtualenv, conda environments)
Package managers (pip, conda)
Anaconda is a piece of software that enables you to (1) isolate your setup for a particular project from the rest of your system and (2) install particular packages easily. It comes in two versions which can be installed from the following links:
Anaconda - Comes with huge number of packages pre-installed but takes up a lot of space (https://docs.anaconda.com/free/anaconda/install/index.html)
Miniconda - The basics, smaller but can install exactly what you want for each project (https://docs.conda.io/en/latest/miniconda.html)
Before you install Anaconda or Miniconda you should check whether you already have it installed.
Press the window’s key and type anaconda or miniconda. You should only have on copy of either Anaconda or Miniconda installed.
1.4.1 Virtual environments#
Python often comes pre-installed on many operating systems. However tempting, don’t use it!!! Installing python packages directly will create many problems for you later on. Virtual environments are effectively a sealed box around your python installation. They enable you to completely isolate the particular versions of python and installed packages so that they do not influence or conflict with work for other projects.
We will focus on conda environments, similar behaviours are available for virtualenv (https://docs.python.org/3/library/venv.html)
Whilst all of what we describe below is done in a separate terminal it will work just as well inside the terminal in vscode if you’ve set things up right. For now use the external Anaconda prompt (Mac and Linux just open a regular terminal).
When it opens you’ll see (base) at the front of the prompt. (base) is a kind of underlying environment.
1.4.1.1 Creating a new environment#
There are 3 options for creating your own environment. We can either:
create an empty environment called myproject
conda create -n "myproject"
create an environment called myenv specifying the version of python (in this case 3.10) and some packages we would like installed
conda create -n "myenv" python=3.10 numpy matplotlib pandas=1.5.3
use a special setup file called environment.yml which is in the current working directory which specifies all the details of what we want in the environment. This is used when someone else has already created an environment and you want to use it.
Before we can do this we need our terminal to be in the same directory as the file environment.yml. Currently my command prompt is in my user directory:
I can find the file path by right clicking on the files inside vscode and selecting “reveal in file explorer”. Then copy the filepath.
Finally in the terminal type cd "path\to\file" which you just copied and the command prompt will indicate that you have moved to a different directory which contains the files you can see in vscode.
Then we can create the environment from the file phys4038.yml:
conda env create -f phys4038.yml
1.4.1.2 Activating an environment#
Once we have created an environment we must activate it
conda activate myproject
This needs to be done when you open the prompt each time. People often forget and then end up installing things in (base) rather than the correct environment. Double check. If you see your environment in brackets at the start of the prompt you’ve done it correctly.
We can see what environments are available and have previously been setup on a system
conda env list
1.4.1.3 Is this all this necessary?#
Suppose you have (at least) two projects:
myproject uses python 3.10 because scipy 1.35 requires certain dependencies that don’t exist in python 3.11. It also uses scikit-optimise which uses numpy. The latest version of numpy has deprecated (got rid of) np.int in favour of np.int64 and therefore you have to use an older version of numpy.
mysecondproject uses python 3.11 and the latest numpy
This is a trivial example. With virtual environments you can configure two environments setup the correct way and all you do is activate the correct virtual environment to work on each project. If you don’t do this you’d have to keep installing and uninstalling different versions of packages. People who use your code would also need to keep changing, and it all ends up in a horrible mess! This is particularly true if you use smaller packages (ie not just numpy, matplotlib etc). To make things easy I often name my virtual environments with a name related to the project I’m working on, to help identify which environment should be used for each project.
If you need to switch environments either activate a different environment or deactivate the current one to return to base
conda deactivate
1.4.2 Package management#
Once you have activated your environment you can see which packages are installed:
conda list
you can then install additional packages:
conda install numpy
install particular version
conda install matplotlib=3.5.2
In addition to conda there is another package manager called pip. This can also be used to install packages into conda environments. It is generally recommended to try conda first and then if that fails try pip like this:
pip install pkgname
(N.B pkgname is a placeholder for the name of the package you want to install)
If you need to uninstall either conda or pip packages you can simply type:
conda uninstall pkgname
pip uninstall pkgname
If you want to export the contents of your environment (perhaps so that someone else can set up theirs in an identical way)
conda env export --no-builds > environment.yml
The additional option (–no-builds) is not necessary but help to make sure that the file when exported will work on other machines.
Now delete the environment (it’s important to do this before setting up the new one on the same computer. Normally you would use this file to setup the enviroment on a different computer but trying to setup on the same computer if the environment already exists creates a conflict).
conda env remove -n myproject
Once you have exported your environment you can create a new environment from the file:
conda env create -f myproject.yml
Don’t try and remember all the conda commands. Here is a quick cheat sheet where you can look things up:
If you need to dive a bit deeper then you can look at the conda reference api.
1.5 Running a python program#
1.5.1 Using the command line#
The most basic way to run python is at a command prompt.
Open Anaconda terminal
Activate your myproject environment
Type “python”
The >>> indicates you are using python in an interactive way
print('Hello World')
ctrl + z exits the python mode and returns you to the command prompt.
Use the windows file explorer to create a folder for your python projects in your OneDrive documents folder. Copy the address from the searchbar and use the following command to set the current working directory of the command prompt:
cd "C:\Users\<USERNAME>\OneDrive - The University of Nottingham\Documents\PROJECTS"
then type:
notepad
Write: print(‘Hello World’) inside, save the file as hello.py (N.B .py extension. To do this you will change the file type dropdown to all files and type .py on the end of the filename) and close the file
then at the command prompt:
python hello.py
This is better since it enables us to write an entire python script and run it. It also can be great for running quick scripts without having to open Integrated Development Environments. However, writing code is a lot easier in dedicated IDEs with tools to support the coding.
1.5.2 Jupyter Notebooks#
Jupyter Notebooks provide an interactive approach and are used a lot in data analysis. This is because you can break code into little pieces and quickly test and rerun portions of code. They support python (and a number of other languages) mixed with markup text and are great for exploratory dataanalysis as code, graphs, outputs, notes can be kept together.
At the anaconda prompt, activate your environment:
conda activate myproject
jupyter notebook
These can be run inside an IDE like vscode or can be run independently. To run independently, in a conda environment install jupyter notebook support:
conda install jupyter notebook
At the conda prompt then type:
jupyter notebook
this starts a local webserver and opens the local file directory in a webbrowser.
Navigate to your notebook or create a new one and it opens up the notebook in a browser (Jupyter notebooks have a .ipynb extension):
However, for this project we’ll run the Jupyter Notebook inside VsCode as it has more features. Open the notebook inside vscode. To do this you’ll need to have installed the Jupyter Extension:
As you can see in this notebook format there are different types of cells:
Markdown for writing notes, structuring, displaying
Code cells for running python code
1.5.2.1 Markdown#
(Note to see how to write the markdown that results in each item below you’ll need to look at the Jupyter Notebook. Double clicking on this cell will show you the Markdown code used)
Markdown comes with multiple ways to do things so here’s a quick guide to doing some common things. The top title can be deined with a # and progressively smaller headings just have more #####.
Titles#
Headings#
subheadings#
subsubheadings#
Write in bold or italics by bracketing text with ** or * respectively.
We can defined code blocks with proper syntax highlighting by adding ``` followed by the language (the one below is python but we can also use console, c, html etc)
print('typesetting some code like this')
Draw horizontal lines to delineate sections by using — at the start of a line
HTML tags generally work although not as readable so use sparingly.
Insert image from a file:
<img src="resources/images/logo.png" alt="Drawing" style="width: 100px;"/>
Insert video from file
<video alt="test" controls style="width: 200px;">
<source src="test.mp4" type="video/mp4">
</video>
If you want to insert an image or video directly from a url (make sure it is url to image or video and not the page in which they are embedded)
Insert image from url
<img src="https://www.ntuclearninghub.com/documents/39367/4216797/Python-Symbol.png/369e410e-a90f-f887-c2dc-61f7ef761476/" alt="Drawing" style="width: 20em;"/>
For video change src attribute in video example.
If you want to do something else you can use html as I have done above or refer to this quick look up guide for Markdown text
1.5.2.2 Code cells#
To run code in the Jupyter notebook you’ll need to select the environment you setup. In the top right corner of the notebook you should see a button that says Select Kernel. Click on it and the environments on your computer should be displayed for you to choose between.
Code cells enable you to:
Type python code in directly
shift + Enterto run that cellResults stay in memory so sequence matters (run the first cell followed by the second cell twice, then the first cell then the second cell and look at the values printed out)
In vscode the jupyter notebook has all the features we describe in the IDEs section
a=1
a+=1
a
2
1.5.3 IDEs#
Integrated development environments provide a whole range of tools that make development easier:
Intellisense
Viewing Documentaton
Debugging
Testing
Git integration
Host of add ons
Whilst there are lots of IDEs the two main leaders in the python world are:
PyCharm (www.jetbrains.com/pycharm) : Can use full featured paid version for free as a student if you register
VsCode (code.visualstudio.com)
For the purposes of this course we will be using VsCode
1.5.4 Running a script in vscode#
Before running a script you need to make sure you have activated the correct environment. You can do this using the command palette ctrl + shift + p and type Python: Select Interpreter and select the correct environment.
In an IDE you will usually create a .py script in the file tab and then run it directly. In vscode you can do this by clicking on the run button at the top of the screen.
You can also run the script from the command line by typing:
python hello.py
1.5.5 Other approaches#
Online coding:
cocalc.com
repl.it
Github codespaces
colab.research.google.com
Explore at your own leisure
1.5.6 Vscode Keyboard Shortcuts#
Sometimes these shortcuts maybe configured differently on different systems. You can see all the keybindings and provide some new ones of your own by going to
File > Preferences > Keyboard Shortcuts. If you are running them on a Mac replace Ctrl with Cmd. Here are a few I use frequently.
Short cut |
Command |
|---|---|
|
Open Command Palette |
|
Indent / unindent selected text |
|
Select all occurences of word |
|
Zoom in or out |
|
Toggle sidebar visibility |
|
Toggle terminal visibility |
In addition to keyboard shortcuts I recommend turning on autosave: File > Autosave. Once logged in to your github account, you can also turn on settings sync to make sure things are setup the same way on any computer you use.
1.6 Importing packages#
Once you have installed a package into your environment you can import it into either a Jupyter Notebook or a script you are writing in the IDE. These imports are usually done in a group at the top of your code. In your files you will find a folder called resources which contains a file called example_function.py. Open this in the editor and look at the contents. Python makes importing so easy. A package is a folder containing python files. Individual .py files are modules. Any .py python file can be imported using its name. There are multiple ways to import and they all mean subtlely different things.
To import everything inside example_import.py we can write:
from resources.example_import import *
Whilst this works and you will find a few big projects (e.g. pyqt) which use this syntax it is not recommended. This is because:
I might have two modules imported and by chance they might have functions that are called the same thing. These would then conflict
It makes code harder to understand. If I have lots of imports how do I know which module example_function() came from?
N.B. This is sometimes referred to as “polluting the namespace”. When we look at functions you’ll see that you could have local and global scopes which effect which variables are accessible in and outside a function. A namespace is somewhat similar and consists of the collection of variables, functions, classes accessible to a particular script.
Below are a few far better ways to import and then how to use the imported functions in your code.
import resources.example_import
import resources.example_import as example_nickname
from resources.example_import import example_function
resources.example_import.example_function()
example_nickname.example_function()
example_function()
This is executing the example function
This is executing the example function
This is executing the example function
As we showed above you can import packages and give them a nickname. You usually do this to make it quick to type whilst clearly identifying which package functions have come from. You could use anything but well known packages such as numpy, pandas etc have standard imports which it’s worth following for clarity
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import scipy
Warning - Problems importing#
If you install packages using conda or pip in your environment then you will be able to import them using their name e.g. import numpy. If you provide the address as we did above relative to where the python file is being run from it will also import them. However, it is worth understanding how python finds packages. If you are looking for a module example_import.py then python searches the current directory and then if it doesn’t find it, it will look in a list of other places that are specified by a path variable. The first time it finds a file with the right name it will stop. You can see this by running:
import sys
print(sys.path) # This is the list of places python searches
import example_import # Note it doesn't find this because it is inside the resources folder and so errors
['C:\\Users\\mikei\\miniconda3\\envs\\instructor\\python311.zip', 'C:\\Users\\mikei\\miniconda3\\envs\\instructor\\Lib', 'C:\\Users\\mikei\\miniconda3\\envs\\instructor\\DLLs', 'C:\\Users\\mikei\\miniconda3\\envs\\instructor', '', 'C:\\Users\\mikei\\miniconda3\\envs\\instructor\\Lib\\site-packages', 'C:\\Users\\mikei\\miniconda3\\envs\\instructor\\Lib\\site-packages\\win32', 'C:\\Users\\mikei\\miniconda3\\envs\\instructor\\Lib\\site-packages\\win32\\lib', 'C:\\Users\\mikei\\miniconda3\\envs\\instructor\\Lib\\site-packages\\Pythonwin']
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
Cell In[4], line 3
1 import sys
2 print(sys.path) # This is the list of places python searches
----> 3 import example_import # Note it doesn't find this because it is inside the resources folder and so errors
ModuleNotFoundError: No module named 'example_import'
#If we want it to find a file in a different place
import sys
sys.path.insert(0,'resources') #Add a relative address to the front of the path list
print(sys.path)
import example_import # Now the import works
example_import.example_function()
Warning - version numbers and my code has stopped working!#
Over time there have been important changes and developments to packages, some of which may break your code! If you are doing something important pay attention to package version numbers. conda list or exporting your environment is a good way to make sure you can get old code to work. Version numbers look like this:
pandas 2.0.1
The further left the number, the more significant the changes and the more likely it is to make a difference to your code.