Learn Python Series (#12) - Handling Files

What Will I Learn?

You will learn:

Some basic file handling operations,
such as how to get your current working directory using the os module,
how to set a file path,
how to open() files from your file path,
how to use the open() function to read data from, and write data to your files,
by using various mode flags,
with which you can both read and write, via various techniques, your file data,
and finally how to use the with keyword, using an alternative syntax without the need to explicitly close() your previously opened files.

Requirements

A working modern computer running macOS, Windows or Ubuntu
An installed Python 3(.6) distribution, such as (for example) the Anaconda Distribution
The ambition to learn Python programming

Difficulty

Basic

Curriculum (of the `Learn Python Series`):

Learn Python Series (#12) - Handling Files

As promised in the previous episode of the Learn Python Series in this episode we will learn how to handle files: opening files, reading from existing files, creating new files, and writing to files.

Reading the contents of a `.txt` file

Let's begin with creating a simple text file, using your favorite code editor, not Python for now, save it as example.txt in our current working directory including the following content:

Months of the year:
January
February
March
April
May
June
July
August
September
October
November
December

The `open()` function

Usage: open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None)

In Python, in order to open a stored file, you first need to associate that file with a variable and to show Python where that file is located on disk, called the file path. Since you might be following along on Windows, Linux, macOS, your file paths are always different. So to make sure everybody, regardless of which type of computer system you are working on right now, can follow along, let's first import the standard library module os and assign the current working directory to the variable current_dir and concatenate the file name example.txt to it, in order to make sure the new variable file_path contains the correct path string, like so:

import os

current_dir = os.getcwd()
file_path = current_dir + '/example.txt'

Now we will associate the file on disk to the newly created variable f (short for: file) using the open() function. The open() function will return a file object. open() allows for receiving many optional arguments, but as its first argument it requires the file path.

A very important second parameter for open() is the mode parameter, which is set by default to r (for reading a file).

Commonly used mode option arguments are:

'r': open file for reading (default)
'w': open file for writing, but truncating the file (overwrite the current contents)
'a': open file for writing data to the end of its contents
'x': create a new file and write data to it
'r+': open file for first reading and then writing to the same file.

f = open(file_path, 'r')

File reading with `read()`, `readline()` and / or `readlines()`

Now that we have associated the file example.txt stored on disk in our current working directory by opening it, the variable f contains the file object. There are three built-in Python file methods that read data from a file:

read(): returns all file contents
readline(): reads "the next line", and steps through the file contents when calling it multiple times
readlines(): reads all file lines and returns them as a list

Nota bene: if you first use read() and then again try to use readline() or readlines(), an empty result will be returned. If you must use multiple read methods then first open the file again and assign it to a new variable (e.g. f_new).

`read()`

This would be an example of how to use read():

content = f.read()
print(content)

Months of the year:
January
February
March
April
May
June
July
August
September
October
November
December

`readline()`

This would be an example of how to use readlines() (once, or multiple times):

first_line = f.readline()
print(first_line)

second_line = f.readline()
print(second_line)

Months of the year:

January

`readlines()`

And we could have returned all individual content lines in the file example.txt as a list, like so:

lines = f.readlines()
print(lines)

['Months of the year:\n', 'January\n', 'February\n', 'March\n', 'April\n', 'May\n', 'June\n', 'July\n', 'August\n', 'September\n', 'October\n', 'November\n', 'December\n']

Writing content to a new `.txt` file

Let's say we want to store the string value of the variable abba to a newly created file called abba.txt and save that to our current working directory.

Let's first import the os module again, get the current_dir and concatenate our file name abba.txt to it as file_path so we know what to store where on disk.

import os

current_dir = os.getcwd()
file_name = 'abba.txt'
file_path = current_dir + '/' + file_name

abba = '''So I say
Thank you for the music, the songs I'm singing
Thanks for all the joy they're bringing
Who can live without it, I ask in all honesty
What would life be?
Without a song or a dance what are we?
So I say thank you for the music
For giving it to me
'''

We can now simply use the open() function again, this time - since we want to create a new file and write content to it, using the mode=x argument, like so:

f = open(file_path, mode='x')

After executing this line, the file abba.txt is indeed created on disk, but of course no content has been written to it yet.

Using the `write()` method

To do that, we use the write() method and pass in the string abba (that contains the data, the song lyrics) as its argument, like so:

f.write(abba)

Closing the file with `close()`

Now the data has been written to the file, but in order for other programs to access (e.g. read from) the file, we first need to close the Python file object, thereby terminating the connection between the file object and the file stored on disk. Like so:

f.close()

Did it work? Use your file browser, check if the file abba.txt is indeed located in the current working directory, and open it using your favorite editor. Is it there? Are the song lyrics stored inside it? Great!

Appending content to an existing `.txt` file

Using the open() function again, this time using the mode=a flag, we can append new content to the end of an existing file. The a flag first opens the file, doesn't actually read its content, but using the write() method where (again) a string of data is passed as an argument, it appends (= writes to the end of the file) that string of data. Like so:

import os

current_dir = os.getcwd()
file_path = current_dir + '/abba.txt'

some_more_text = '----------\nThis is a famous song by ABBA.\n'

f = open(file_path, 'a')
f.write(some_more_text)
f.close()

Now if you open the file abba.txt in your editor again, if all went well, you should see the contents of the variable some_more_text is indeed added at the end of what was in the file before:

So I say
Thank you for the music, the songs I'm singing
Thanks for all the joy they're bringing
Who can live without it, I ask in all honesty
What would life be?
Without a song or a dance what are we?
So I say thank you for the music
For giving it to me
----------
This is a famous song by ABBA.

Appending to or overwriting parts of an existing `.txt` file

Using open() with the r+ flag enables you to both read the existing file contents, do something with those contents (using for example the built-in string methods we discussed earlier), and also (over)write (some parts of) that same file.

Let's see the behavior of the r+ flag by overwriting (parts of) the first file line with some dummy data, like so:

import os
current_dir = os.getcwd()
file_path = current_dir + '/abba.txt'
dummy_data = 'BlaBlaBlaBlaBla'

f = open(file_path, 'r+')
f.write(dummy_data)
f.close()

If you open your abba.txt file again, its contents are like this:

BlaBlaBlaBlaBlayou for the music, the songs I'm singing
Thanks for all the joy they're bringing
Who can live without it, I ask in all honesty
What would life be?
Without a song or a dance what are we?
So I say thank you for the music
For giving it to me
----------
This is a famous song by ABBA.

Did you notice that the first line now contains the BlaBlaBlaBlaBla string, and that the rest of the original file contents is still there? Ok...

Now we could also use the r+ flag to append the a string to the end of the file. In order to do that, we must first read the existing file contents (so the file pointer is at the end), and then write the appending string to the file.

For example:

import os
current_dir = os.getcwd()
file_path = current_dir + '/abba.txt'
f = open(file_path, 'r+')

appending_string = '\n---------\nAlso a way to append data!\n'
original = f.read()

f.write(appending_string)
f.close()

The result file is now as this:

BlaBlaBlaBlaBlayou for the music, the songs I'm singing
Thanks for all the joy they're bringing
Who can live without it, I ask in all honesty
What would life be?
Without a song or a dance what are we?
So I say thank you for the music
For giving it to me
----------
This is a famous song by ABBA.

---------
Also a way to append data!

Truncating and overwriting an existing `.txt` file

Using the open() function wiht the w flag first truncates the content of the existing file, and then writes the new content passed into the write() function.

import os

current_dir = os.getcwd()
file_path = current_dir + '/abba.txt'

new_content = "This is completely new content.\n"

f = open(file_path, 'w')
f.write(new_content)
f.close()

The ABBA song lyrics are now deleted from the file abba.txt and replaced with the string This is completely new content.\n. Like so:

This is completely new content.

Using `with open() as f:`

When working with file objects, it's oftentimes convenient to use the with keyword. Its syntax is a bit different than what we've been discussing until now, but the advantage of using with is that the file you're working on is automatically closed when you're done with your file procedure.

Please regard the following example:

import os

current_dir = os.getcwd()
file_path = current_dir + '/with.txt'

content = '''In west Philadelphia born and raised
On the playground where I spent most of my days
Chilling out, maxing, relaxing all cool
And all shooting some b-ball outside of the school
When a couple of guys, they were up to no good
Started making trouble in my neighbourhood
I got in one little fight and my mom got scared
And said You're moving with your auntie and uncle in Bel-air
'''

with open(file_path, 'w') as f:
    f.write(content)

If you open the file with.txt as expected, its contents will be:

In west Philadelphia born and raised
On the playground where I spent most of my days
Chilling out, maxing, relaxing all cool
And all shooting some b-ball outside of the school
When a couple of guys, they were up to no good
Started making trouble in my neighbourhood
I got in one little fight and my mom got scared
And said You're moving with your auntie and uncle in Bel-air

What did we learn, hopefully?

We've discussed how to get, using the os module, your current working directory, open() files in it, read data from, and write data to your files. We've discussed various mode flag options of the open() function, how you can use them for various purposes, and we've discussed using the with keyword as an alternative mechanism that doesn't need explicit file closing.

Of course, since this was the first time we read content from and written some other content to a file, using Python, I've deliberately kept the explanations and the examples pretty simple to understand.

But in the next tutorial episodes, we will use the knowledge egarding handling files we learned today and build upon that to use and manipulate some more complex data files.

Thank you for your time!

Posted on Utopian.io - Rewarding Open Source Contributors

Learn Python Series (#12) - Handling Files

Learn Python Series (#12) - Handling Files

What Will I Learn?

Requirements

Difficulty

Curriculum (of the Learn Python Series):

Learn Python Series (#12) - Handling Files

Reading the contents of a .txt file

The open() function

File reading with read(), readline() and / or readlines()

read()

readline()

readlines()

Writing content to a new .txt file

Using the write() method

Closing the file with close()

Appending content to an existing .txt file

Appending to or overwriting parts of an existing .txt file

Truncating and overwriting an existing .txt file

Using with open() as f:

What did we learn, hopefully?

Thank you for your time!

Curriculum (of the `Learn Python Series`):

Reading the contents of a `.txt` file

The `open()` function

File reading with `read()`, `readline()` and / or `readlines()`

`read()`

`readline()`

`readlines()`

Writing content to a new `.txt` file

Using the `write()` method

Closing the file with `close()`

Appending content to an existing `.txt` file

Appending to or overwriting parts of an existing `.txt` file

Truncating and overwriting an existing `.txt` file

Using `with open() as f:`