6.2 Loading data from a file

6.2 Loading data from a file#

In the previous notebook you used numpy to handle vectors of data, and plotted data with matplotlib. The data came either from calculations in Python or from (short) lists which you entered in the notebook. If you need to work with larger amounts of data that becomes inconvenient. Then it’s useful to load data from a file.

This notebook uses a file called rico.txt.

import numpy as np
from pathlib import Path

rico_dataset_path = Path.cwd().parent / "data/rico.txt"

rico = np.loadtxt(rico_dataset_path)

Note that the file in this case is save in a folder data within the parent directory.

What did we just load?

print('The type of rico is:', type(rico))
print('The number of rows and columns is equal to:',rico.shape)

Let’s see what data it contains.

print(rico[:10,:]) # select first 10 rows, all columns

Now do the following:

  • open the file in a different tab by clicking the download button () in the top right and selecting “rico dataset”.

    Note that locally you would open it in a text editor. For example Notepad (basic, comes with Windows), Notepad++ (nice, free, open source). Microsoft Word might work too, but is not as nice for working with these plain text files.

  • look at the top of the file. You will see two lines starting with # followed by lines of numbers.

  • lines starting with # are considered as comments, and are ignored by Numpy. Those lines describe what is in the columns of the file.

  • the following lines with numbers are loaded into the numpy array. You can compare the first few rows from the file with the array content you printed above.

As you probably know, numbers like 2.0000e+01 are written in scientific notation, this one means \(2 \times 10^1 = 20\)

What does it mean?#

The file rico.txt describe atmospheric conditions (like temperature, humidity, wind speed) as a function of height above the surface. The file is part of the input to the DALES model, for running a specific simulation called RICO (Rain In Cumulus over Ocean, vanZanten et al 2011).

The two first lines of the file give a hint of what the columns mean:

#  input file Profiles - RICO Trade Cu Period (12/16-01/08)
#  height(m)    thl(K)          q_t (kg/kg)      u(m/s)          v(m/s)         TKE_init (m/s)

Some more explanation of the columns, and their number (starting from 0, since that’s how Python numbers them):

0 height    height above the surface (m)
1 thl       liquid water potential temperature (K), a kind of temperature
2 qt        specific humidity (kg/kg) - amount of water, kg of water per kg of total air 
3 u         wind speed (m/s) in the eastward direction
4 v         wind speed (m/s) in the northward direction
5 TKE_init  turbulent kinetic energy (m/s) - a measure of the amount of turbulence

We will revisit this file in one of the exercises, and have a look at working with its contents there.

6.2.1 Surface and color plots#

Next you will learn some more plotting with matplotlib, namely to plot a function of two variables, \(f(x,y)\). Either as a 2D image with colors showing the value, or as a surface in 3D where \(z = f(x,y)\).

First try the folloring example showing np.meshgrid(). Meshgrid is used to create 2D arrays X and Y, with x- and y-coordinates, respectively.

import matplotlib.pyplot as plt
x = np.linspace(0, 3, 4) # [0, 1, 2, 3]
y = np.linspace(0, 4, 5) # [0, 1, 2, 3, 4]

X,Y = np.meshgrid(x,y)

print('X')
print(X)
print()
print('Y')
print(Y)

The repeated rows or columns in the meshgrid output may seem redundant, but they are convenient for building expressions.

Surface plot#

x = np.linspace(-5, 5, 40) # create new coordinate arrays with more points
y = np.linspace(-5, 5, 40) 

X,Y = np.meshgrid(x,y)

# a function to plot
Z = X**2 + Y**2

Z has the same shape as X and Y, and each element of Z is computed from the corresponding elements of X and Y.

ax = plt.axes(projection='3d')

ax.plot_surface(X,Y,Z)

plt.show()

Color plot#

x = np.linspace(-5, 5, 40) # create new coordinate arrays with more points
y = np.linspace(-5, 5, 40) 

X,Y = np.meshgrid(x,y)

# a function to plot
Z = X**2 + Y**2

# optional: set square aspect ratio, meaning that one unit along x and y is equally long
plt.gca().set_aspect('equal')  # gca() stands for get current axis

plt.pcolormesh(X,Y,Z)   # plot Z as function of X and Y, using colors
cbar = plt.colorbar()   # shows the color bar

cbar.set_label("Value of Z")  # put a label on the colourbar so it is understandable

plt.show()

After this Chapter you should be able to:#

  • know the basics of how to use numpy

  • be able to import and perform calculations with datasets

  • plot your datasets with matplotlib

  • plot surface plots with np.meshgrid() and matplotlib

Contents