3.1 Data Structures

In this Section you will tackle a data management problem! In the first module you have learned how to create variables, which is cool. But when you populate a lot of variables, or you want to store & access them within one entity, you need to have a data structure.

There are plenty of them, which differ their use cases and complexity. Today we will tackle some of the standard Python built-in data structures. The most popular of those are: list, tuple and dict.

List

First, the easiest and the most popular data structure in Python: list (which is similar to a typical array you could have seen in a different programming language).

You can create a list in the following ways:

# 1). Creating an empty list, option 1

empty_list1 = []
print(f'Type of my_list1 object: {type(empty_list1)}')
print(f'Contents of my_list1: {empty_list1}')
print('--------------------')

# 2). Creating an empty list, option 2 - using the class constructor

empty_list2 = list()
print(f'Type of my_list2 object: {type(empty_list2)}')
print(f'Contents of my_list2: {empty_list2}')
print('--------------------')

# 3). Creating a list from existing data - option 1

my_var1 = 5
my_var2 = "hello"
my_var3 = 37.5

my_list = [my_var1, my_var2, my_var3]
print(f'Type of my_list3 object: {type(my_list)}')
print(f'Contents of my_list3: {my_list}')
print('--------------------')

# 4). Creating a list from existing data - option 2

cool_rock = "sandstone" # remember that a string is a collection of characters

list_with_letters = list(cool_rock)

print(f'Type of list_with_letters object: {type(list_with_letters)}')
print(f'Contents of list_with_letters: {list_with_letters}')
print('--------------------')

As you can see, in all three cases we created a list, only the method how we did it was slightly different:

• the first method uses the bracket notation,

• the second method uses the class constructor approach.

Both methods also apply to the other data structures, as we will see later on.

Now, we have a list — what can we do with it?

Well… we can access and modify any element of an existing list. In order to access a list element, square brackets [] are used with the index of the element we want to access inside. Sounds easy, but keep in mind that Python has a zero-based indexing (as mentioned in Section 1.2 in Chapter 1).

As a reminder, a zero-based indexing means that the first element has index 0 (not 1), the second element has index 1 (not 2) and the n-th element has index n - 1 (not n)!

# len() function returns the lengths of an iterable (string, list, array, etc)
print(len(my_list))

# We have 3 elements, thus we can access 0th, 1st, and 2nd elements
print(f'First element of my list: {my_list[0]}')
print(f'Second element of my list: {my_list[1]}')
print(f'Last element of my list: {my_list[2]}')

After the element is accessed, it can be used as any variable, the list only provides a convenient storage

summation = my_list[0] + my_list[2]
print(f'Sum of {my_list[0]} and {my_list[2]} is {summation}\n')

Since it is a storage - we can easily alter and swap list elements:

my_list[0] += 7
my_list[1] = "My new element"

print(my_list)

However we can only access data we have - Python will give us an error for the following

my_list[10] = 199

We can also add new elements to a list, or remove them! Adding is realized with the append method and removal of an element uses the del keyword. You can even store a list inside another list - list inception! Will turn out to be useful for matrices, images etc.

# adding a new element to the end of the list
my_list.append("new addition to  my variable collection!")
print(my_list)

# store a list inside a list 
my_list.append(['another list', False, 1 + 2j])
print(my_list)

# let's remove 37.5

del my_list[2]

print(my_list)

Attention

Note the syntax here: mylist.append directly modifies mylist, and we thus don’t need to reassign its output to mylist Hence, my_list = my_list.append(“new addition”) will NOT work!

Lists also have other useful functionalities, as you can see from the official documentation. Since lists are still objects you can try and apply some operations to them as well.

lst1 = [2, 4, False]
lst2 = ['second list', 0, 222]

#what will happen?
lst1 = lst1 + lst2
print(lst1)

lst2 = lst2 * 4
print(lst2)

lst2[3] = 5050
print(lst2)

As you can see, adding lists together concatenates them and multiplying them basically does the same thing (it performs addition several times, just like in real math…).

Additionally, you can also use the in keyword to check the presence of a value inside a list.

print(lst1)

if 222 in lst1:
    print("We found 222 inside lst1")
else:
    print("Nope, it's not there...")

Tuple

If you understood how list works, then you already understand 95% of tuple. Tuples are just like lists, with some small differences.

1. In order to create a tuple you need to use () brackets, commas, or a tuple class constructor.

2. You can change the content of your list, but that is not possible for tuples (just like strings). In progamming terminology, we say that tuples and strings are immutable.

# Creating an empty tuple - a bit useless, since you cannot change it

tupl1 = tuple() # option 1 with the class constructor
print(f'Type of tupl1: {type(tupl1)}')
print(f'Content of tupl1: {tupl1}')

tupl2 = () # option 2 with ()
print(type(tupl2), tupl2)

# Creating a non-empty tuple using brackets

my_var1 = 26.5
my_var2 = 'Oil'
my_var3 = False

my_tuple = (my_var1, my_var2, my_var3, 'some additional stuff', 777)
print(f'my tuple: {my_tuple}')

# Creating a non-empty tuple using commas

comma_tuple = 2, 'hi!', 228
print(f'A comma made tuple: {comma_tuple}')

# now, let's try to access an element
print(f'4th element of my_tuple: {my_tuple[3]}')

# but, can we change it?
my_tuple[3] = 'will I change?'

In the cell above, you get an error, because the tuple cannot be changed. Remove the last two lines in the cell above to fix the error.

You will get an error if you try to modify tuples with methods such as append(). Since tuples are immutable, it has no append() method nor any other methods to alter them

You might think that tuple is a useless class. However, there are some reasons for it to exist:

1. Storing constants & objects which shouldn’t be changed.

2. Saving memory (tuple uses less memory to store the same data than a list).

Run the cell below. There you can see that a list requires more memory than a tuple.

#creating a list and a tuple from the same data

my_name = 'Vasyan'
my_age = 27
is_student = True

a = (my_name, my_age, is_student)
b = [my_name, my_age, is_student]

print(f'size of a = {a.__sizeof__()} bytes') #.__sizeof__() determines the size of a variable in bytes
print(f'size of b = {b.__sizeof__()} bytes')

Dictionary

After seeing lists and tuples, you may think:

“Wow, storing all my variables within another variable is cool and gnarly! But… sometimes it’s boring & inconvenient to access my data by using it’s position within a tuple/list, like when I have dozens of different variables in my tuple/list. Is there a way that I can store my object within a data structure but access it via something meaningful, like a keyword…?”

Don’t worry if you had this exact same thought.. Python had it as well!

Dictionaries are suited especially for that purpose — to each element you want to store, you give it a nickname (i.e., a key) and use that key to access the value you want.

# Creating an empty dictionary - we used () for tuples and and [] for lists. 
# Now, it's time to use {}.

empty_dict1 = {}
print(f'Type of empty_dict1: {type(empty_dict1)}')
print(f'Content of it: {empty_dict1}')

# Creating an empty dictionary - using class constructor
empty_dict2 = dict()
print(f'Type of empty_dict2: {type(empty_dict2)}')
print(f'Content of it: {empty_dict2}')

# Creating a non-empty dictionary - specifying pairs of key:value pattern
my_dict = {
    'name': 'Jarno',
    'color': 'red',
    'year': 2007,
    'is cool': True,
    6: 'it works',
    (2, 22): 'that is a strange key'
}

print(f'Content of my_dict: {my_dict}')

In the last example, you can see that only strings, numbers, or tuples were used as keys. Dictionaries can only use immutable data (or numbers) as keys:

# using mutable structures won't work: 

mutable_key_dict = {
    5: 'lets try',
    True: 'I hope it will run perfectly',
    6.78: 'heh',
    ['No problemo', 'right?']: False  
}

print(mutable_key_dict)

Alright, now it is time to access the data we have managed to store inside my_dict…

# for that we use keys!

print('Some random content of my_dict:')
print(my_dict['name'])
print(my_dict[(2, 22)])

# remember the mutable key dict? Let's make it work by omitting the list item
mutable_key_dict = {
    5: 'lets try',
    True: 'I hope it will run perfectly',
    6.78: 'heh'
}

# You can see that it doesn't give any errors but how do we access the data inside it?
# use keys!
print('Accessing weird dictionary...')
print(mutable_key_dict[True])
print(mutable_key_dict[5])
print(mutable_key_dict[6.78])

# Trying to access something we have and something we don't have
print(f'My favorite year is {my_dict["year"]}')
print(f'My favorite song is {my_dict["song"]}')  

Attention

When accessing keys in f-strings, you cannot repeat the same quote type. That’s why we used “year” and not ‘year’ (and similarly for “song”).

Attention

It is best practice to use mainly strings as keys — the other options are error-prone (remember the floating-point equality testing from Lesson 1), and are only extremely rarely used.

What’s next? Dictionaries are mutable, so let’s go ahead and add some additional data and delete old ones.

print(f'my_dict right now: {my_dict}')

my_dict['new_element'] = 'magenta'
my_dict['weight'] = 27.8
my_dict['extra_dict'] = {'dictionaries': ['can', 'contain', 'dictionaries', 'too']}  # dictionary inception, for very large datasets
del my_dict['year']

print(f'my_dict after some operations: {my_dict}')

You can also print all keys present in the dictionary using the .keys() method, or check whether a certain key exists in a dictionary, as shown below. More operations can be found here.

print(my_dict.keys())

# check if my_dict has a name key
print(f"\nmy_dict has a ['name'] key: {'name' in my_dict}")  #\n is the newline character

Real life example: Analyzing satellite metadata

Metadata is a set of data that describes and gives information about other data. For Sentinel-1, the metadata of the satellite is acquired as an .xml file. It is common for Dictionaries to play an important role in classifying this metadata. One could write a function to read and obtain important information from this metadata and store them in a Dictionary. Some examples of keys for the metadata of Sentinel-1 are:

dict_keys([‘azimuthSteeringRate’, ‘dataDcPolynomial’, ‘dcAzimuthtime’, ‘dcT0’, ‘rangePixelSpacing’, ‘azimuthPixelSpacing’, ‘azimuthFmRatePolynomial’, ‘azimuthFmRateTime’, ‘azimuthFmRateT0’, ‘radarFrequency’, ‘velocity’, ‘velocityTime’, ‘linesPerBurst’, ‘azimuthTimeInterval’, ‘rangeSamplingRate’, ‘slantRangeTime’, ‘samplesPerBurst’, ‘no_burst’])

Slices

The last important thing for this Notebook are slices. Similar to how you can slice a string (shown in Section 1.2, in Chapter 1). This technique allows you to select a subset of data from a list or tuple.

With the : symbol we can then select a slice, for example my_list[start:end] selects all elements with indices [start, end), where the last element is not included.

If we would like to select a slice starting with the first element (i.e, index 0), there is a shorter way, as you can see below.

# let's make a simple list

x = [1, 2, 3, 4, 5, 6, 7]
n = len(x) # len(x) gives the length of x

print(f'The first three elements of x: {x[0:3]}')

# we we can also omit the 0
print(f'The first three elements of x in a shorter way: {x[:3]}')

# instead of counting elements from the beginning, we can count from end
print(f'The last element is {x[6]} or {x[n - 1]} or {x[-1]}')

# thus, we can apply it in slicing our list
print(f'Up till and including the fourth but last element: {x[:-4]}')

# we can also specify a third argument: the step size of our selection
print(f'Only elements with index 1 and 3 {x[1:5:2]}')

Thus, the general slicing call is given by iterable[start:end:step].

Here’s more examples where :

• we omit also the last element if we simply want to consider all elements in the list, and select every second element

• reverse the order of a list

• take a slice from the middle

numbers = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

# Only step size needed if we consider all elements
print('Selecting all even numbers', numbers[::2])

# Starting element and step size needed
print('All odd numbers', numbers[1::2])

# Reversing the list
print('Normal order', numbers)
print('Reversed order', numbers[::-1])

# Selecting middle subset
print('Numbers from 5 to 8:', numbers[5:9])

Copying lists

Copying lists sounds easy but is actually rather tricky. An example:

list_1 = [1, 2, 3]
list_2 = list_1
print(f'Original: {list_1}')
print(f'Copy: {list_2}')

So far so good, both lists do in fact contain the same data, which is what we wanted.

The tricky part shows when we start modifying one of the two lists:

list_2[2] = 5
print(f'Original: {list_1}')
print(f'Copy: {list_2}')

So what just happened? We modified the copy (list_2), and the original also changed?

This happens because by copying the list in the way we did, Python did not copy the whole list, but rather the pointer to where the list is stored in your computers memory (how exactly it does that is not important). This means that both the original and the copy are stored at the same location in your computers memory, and modifying one therefore modifies the other.

The way around this is to include the slice; if you want to copy the full list in a different spot in the computer’s memory, you can do that as shown below.

list_1 = [1, 2, 3]
list_2 = list_1[:] 

print(f'Original: {list_1}')
print(f'Copy: {list_2}\n')

list_2[2] = 5
print(f'Original: {list_1}')
print(f'Copy: {list_2}')

Attention

To properly copy a list, you need to copy with slice notation. Otherwise unexpected stuff will happen.

Additional study material:

• Official Python Documentation - https://docs.python.org/3/tutorial/datastructures.html

• Think Python (2nd ed.) - Chapters 8, 10, 11, 12