Read Files Line-by-Line Like a Python Pro

Level up your file handling skills by mastering iterations over text files.

Photo by Gianluca Cinnante on Unsplash

Reading data from files is a bread-and-butter task in Python programming. While reading an entire file into memory is possible, it’s often better to iterate over files line-by-line. This approach conserves memory and allows you to process large files efficiently.

Opening Files

To read a file, first open it using the open() function. The with statement automatically closes the file after your code block:

with open("data.txt", "r") as file:
    # read file line-by-line here

Line-by-Line Iteration

Iterate over each line in the file using a for loop:

with open("data.txt", "r") as file:
    for line in file:
        print(line)

This will print the full contents of data.txt to the console. Note that line already includes a newline character \n at the end.

Processing Lines

Remove the trailing newline and do other processing in the loop:

with open("data.txt", "r") as file:    
    for line in file:
        line = line.strip() # remove leading/trailing whitespace…