Python is a great tool for processing data. Some of the most common tasks in programming involve reading, writing, or manipulating data. For this reason, it’s especially useful to know how to handle different file formats which store different types of data.
For example, consider a Python program that checks a list of users for access control. Your list of users may be stored as a text file, allowing you to check access or modify permissions. With Python, being able to open, read from, write to, and close files will help you work with tasks such as this.
This tutorial will briefly describe some of the file formats Python is able to handle. After a brief introduction to those, you’ll learn how to open, read, and write a text file in Python 3. When you’re finished, you’ll be able to handle any plain text file in Python.
For this tutorial, you should have Python 3 installed as well as a local programming environment set up on your computer. If this is not the case, you can get set up by following the appropriate installation and set up guide for your operating system:
Python is very flexible and can handle a number of different file formats with ease, including but not limited to the following:
|Plain text||Plain text files store data that represents only characters (or strings) and excludes any structured metadata|
|CSV||Comma-separated values files use commas (or other delimiters) to structure stored data, allowing data to be saved in a table format|
|HTML||HyperText Markup Language files store structured data intended to be displayed by a browser and is commonly used with websites|
This tutorial will focus on working with plain text files.
Before we can begin working in Python, we need to make sure we have a file to work with. To do this, open your code editor and create a new plain text file called
In the new file, enter a few lines of text listing the days of the week:
Monday Tuesday Wednesday Thursday Friday Saturday Sunday
Next, save your file and make make note of its location. For this example, our user sammy, saved the file here as
/home/sammy/days.txt. This will be very important in later steps, where we open the file in Python.
Now that you have a file to process, you can begin to code.
In your code editor, create a new Python file and name it
To open a file in Python, we first need some way to associate the file on disk with a variable in Python. This process is called opening a file, and the variable called a file handle. We begin by telling Python where the file is. The location of your file is often referred to as the file path —
/home/sammy/days.txt in this example. Create a variable to store this path information.
path = '/home/sammy/days.txt'
Now, you can use Python’s
open() function to open our
days.txt file. The
open() function requires the file path as its first argument. The function also accepts many other parameters. However, most important is the optional
mode parameter. This is an optional string that specifies the mode in which the file is opened. The mode you choose will depend on what you wish to do with the file. Here are some of the available modes:
'r': use for reading from a file
'w': use for writing to a file
'a': use for appending to a file
'r+': use for reading and writing to the same file
In this example, we only want to read from the file, so we will use the
'r' mode. Use the
open() function to open the
days.txt file and assign the resulting file handle to the variable
days_file = open(path, 'r')
Now that you have opened the file, the next step will walk you through reading its contents.
Since our file has been opened, we can now manipulate it (i.e. read from it) through the variable it was assigned to. Python provides three related operations for reading information from a file. Let’s take a moment to understand how each of them work.
The first operation
read() returns the entire contents of the file as a single string. For example:
The result would be:
\n in the above string represents a newline character. When interacting directly with Python (as you might when simply running
python from the command line), it will use
\n to show newlines. You can also use it to represent a new line when you are creating strings of your own.
The second operation
readline() returns the next line of the file, returning the text up to and including the next newline character. More simply put, this operation will read a file line-by-line.
The result would be:
Once you read a line with the
readline operation the current point at which the file will be read will move to the next line. If you were to call this operation again, it would return the next line in the file like so:
The last operation,
readlines() returns a list of the lines in the file, where each item of the list represents a single line.
Output['Monday\n', 'Tuesday\n', 'Wednesday\n', 'Thursday\n', 'Friday\n', 'Saturday\n', 'Sunday\n']
Something to keep in mind when you are reading from files, once a file has been read using one of the read operations, it cannot be read again. For example, if you were to first run
days_file.read() followed by
days_file.readlines() the second operation would return an empty string. Therefore, anytime you wish to read from a file you will have to first open a new file variable or use the
seek() method, which is beyond the scope of this tutorial. If you’d like to learn more, Python has excellent documentation about these methods.
Now that we have read from a file, let’s learn how to write to a new file.
In this step, you will write a new file that includes the title Days of the Week followed by the contents of the first file. First, create a
title = 'Days of the Week\n'
You will also need to store the days of the week in a variable, which we’ll call
days. This code opens the file in read mode, reads the file, and stores the returned output from the read operation in our new variable
days. To make it easier to follow, the code from Step 2 is included.
path = '/home/sammy/days.txt' days_file = open(path, 'r') days = days_file.read()
Now that you have variables for title and days of the week, you can begin writing to your new file. First, specify the location of the file. Again, we will use the directory
/home/sammy/, so our path will be
/home/sammy/new_days.txt. You can then open the new file in write mode, using the
open() function with the
'w' mode specified.
new_path = '/home/sammy/new_days.txt' new_days = open(new_path, 'w')
Note: It’s important to note that, if
new_days.txt already exists before opening the file, its old contents will be overwritten, so be careful when using the
Once the new file is opened, you can add data using the
write() method. This method takes a single string parameter and writes that data to the file. If you want to start a new line in the file, you must explicitly provide the newline character,
\n, which was included when you assigned
'Days of the Week\n' to the
Write the title to the file followed by the days of the week. It may be helpful add in some
new_days.write(title) print(title) new_days.write(days) print(days)
Lastly, whenever you are finished with a file, it’s important to close it.
Closing a file makes sure that the connection between the file on disk and the file handle is finished. Closing files also ensures that other programs are able to access them and keeps your data safe. If you are not using
with statements as described in Step 6, always make sure to close your files. For this example, close all our files using the
Now that the script is finished using the files, it releases the file handles using the
The recommended way to work with files in Python (often called the Pythonic way) is to use a feature of the language called
with statements. These statements are shorthand ways to set up a context in which work is done and, once that context is over, final details are automatically taken care of in order to prevent common errors. In the case of working with files, a
with statement will automatically close the file so that you don’t have those file handles lingering once you are finished with your task.
As with any block in Python such as function definitions,
if statements, or loops,
with statements take the form of a simple statement followed by a
: and a block of indented code. Here is an example of code that opens a file and prints its contents:
with open('/home/sammy/days.txt', 'r') as days_file: days = days_file.read() print(days)
Let’s walk through what this code does step by step. As before, we open the file using Python’s builtin
open() function, passing the file path and mode parameters. However, since we are using the
with statement, rather than assigning the resulting file handle to a variable using
=, we assign it using the
as keyword. This is part of the full
with statement’s syntax:
with action as result: . . .
:, we move to the next line and indent our code, which is how Python organizes blocks of functionality. As before, we have access to the file handle in the
days_file variable, so we can call the
read() method to get all of the contents and
Note how there is no
close() method called, however. This is because as soon as the code leaves this block (that is, as soon as either the next line of code is not indented or the file ends), the
with statement’s context knows to close those files automatically. The benefits to this are not just that you don’t need to remember to close your files every time, but also that all of your logic for dealing with that file in particular is visually and logically distinct within that block. This structure helps keep your code clean and readable.
With this in mind, let’s rewrite our code form before using the
with open(path, 'r') as days_file, open(new_path, 'w') as new_days: days = days_file.read() new_days.write(title) new_days.write(days) print(title) print(days)
Now the code is much more organized. We begin as before by defining some variables: the paths of our two files and the title that we will use for heading up the new file. After that, we begin our
with statement, opening our two files and storing their connections in appropriately named variables. As before, we read the contents of
days_file, then write the title and those contents to
new_days. Finally, we end the block by de-indenting our code in order to print the values of
days that we read.
Note: Unlike some other blocks in Python, variables defined within
with blocks are available outside of that scope.
While there are some cases where you will want to use the
close() method as described earlier on and it is important to know how it works, you will most commonly use
with statements when working with files in Python.
Before you run your code, it’s a good idea to make sure everything seems correct. The final product should be something like this:
path = '/home/sammy/days.txt' new_path = '/home/sammy/new_days.txt' title = 'Days of the week\n' with open(path, 'r') as days_file, open(new_path, 'w') as new_days: days = days_file.read() new_days.write(title) new_days.write(days) print(title) print(days)
After saving your code, open up terminal and run your Python script, like so:
- python files.py
The output will be as follows:
OutputDays of the Week Monday Tuesday Wednesday Thursday Friday Saturday Sunday
Now, double check the code fully worked by opening the new file (
new_days.txt). If all went well, it should contain this:
Days of the Week Monday Tuesday Wednesday Thursday Friday Saturday Sunday
In this tutorial, we went through how to handle and manipulate plain text files in Python 3. Now you can open, read, write, and close files in Python, and you can continue working with your own data in Python. Python provides many other helpful methods when working with input and output, and also provides documentation to learn more.
Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.