Python file handling: text, CSV and JSON

Variables forget, files remember

Everything in a variable is lost when the program ends. To keep data (marks, settings, a diary) until next time, save it to a file on the disk.

Opening a file: open() and modes

f = open("diary.txt", "w")
f.write("Today I learnt files.\n")
f.close()

Add encoding="utf-8" when the text may contain Hindi or symbols like the rupee sign.

Always use with

A with block closes the file for you, even if an error happens:

with open("diary.txt", "w", encoding="utf-8") as f:
    f.write("Line one\n")
    f.write("Line two\n")
# the file is closed here

write() does not add a new line by itself; put \n at the end of each line.

Reading a text file

with open("diary.txt", encoding="utf-8") as f:
    text = f.read()            # the whole file as one string

with open("diary.txt", encoding="utf-8") as f:
    for line in f:             # one line at a time: best for big files
        print(line.strip())    # strip() removes the '\n' at the end

with open("diary.txt", encoding="utf-8") as f:
    lines = f.readlines()      # a list of lines

Append mode: add without erasing

from datetime import datetime

with open("log.txt", "a", encoding="utf-8") as f:
    f.write(f"{datetime.now():%d-%m-%Y %H:%M} logged in\n")

Each run adds one more line. Using "w" here by mistake would wipe the log every time.

File paths: where is my file?

A plain name like "marks.txt" is looked for in the current working directory, usually the folder you ran the program from. If Python says FileNotFoundError, check where you are running it.

open("data/marks.txt")                 # relative path: a 'data' folder inside the current one
open("C:/Users/Asha/marks.txt")        # absolute path (forward slashes work on Windows)
open(r"C:\Users\Asha\marks.txt")       # or a raw string for backslashes

CSV files: tables that open in Excel

A CSV (comma-separated values) file stores a table as plain text, one row per line.

name,marks
Asha,91
Ravi,78

Writing CSV

import csv

rows = [["Asha", 91], ["Ravi", 78]]
with open("marks.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.writer(f)
    writer.writerow(["name", "marks"])     # header
    writer.writerows(rows)

newline="" stops blank lines appearing between rows on Windows.

Reading CSV with DictReader

DictReader turns each row into a dictionary, using the header row as keys:

with open("marks.csv", encoding="utf-8") as f:
    for row in csv.DictReader(f):
        print(row["name"], int(row["marks"]))

Everything read from a CSV is a string, so convert numbers with int() or float().

JSON files: save dictionaries and lists

JSON stores Python dictionaries and lists as text, keeping their structure. It is the standard format for app settings and for data sent between websites.

import json

settings = {"theme": "dark", "font_size": 14, "subjects": ["Maths", "IT"]}

with open("settings.json", "w", encoding="utf-8") as f:
    json.dump(settings, f, indent=2)       # save

with open("settings.json", encoding="utf-8") as f:
    loaded = json.load(f)                  # load: back to a dict
print(loaded["subjects"])

json.dumps() and json.loads() (with an s) do the same with strings instead of files.

Example: a contact book that remembers

import json, os

FILE = "contacts.json"
contacts = {}
if os.path.exists(FILE):
    with open(FILE, encoding="utf-8") as f:
        contacts = json.load(f)

name = input("Name: ")
contacts[name] = input("Phone: ")

with open(FILE, "w", encoding="utf-8") as f:
    json.dump(contacts, f, indent=2)
print(len(contacts), "contacts saved")

Common mistakes

• Opening with "w" when you meant "a", and losing the old contents.
• Forgetting \n with write(), so everything ends up on one line.
• Doing maths on CSV values without converting them from strings.
• FileNotFoundError because the program runs from a different folder than the file.

Key points

• Open files with with open(name, mode, encoding="utf-8") as f:.
• "r" reads, "w" overwrites, "a" adds to the end.
• Use the csv module for tables and json for dictionaries and lists.
• Loop over a file object to read it line by line.

Practice questions

with open("story.txt", encoding="utf-8") as f:
    print(sum(1 for line in f))

import csv
with open("marks.csv", encoding="utf-8") as f:
    for row in csv.DictReader(f):
        if int(row["marks"]) > 80:
            print(row["name"])