Extract text and tables from PDFs with Python and pdfplumber

PDF was designed to be printed and read, not reworked. Anyone who has tried to copy a table from a bank statement or a supplier price list into a spreadsheet knows this: columns get mixed up, numbers land on the wrong rows, amounts turn into text. When the job has to be repeated every month, a Python script using the pdfplumber library turns an hour of copy and paste into a few seconds.

First question: does the PDF contain text?

There are two very different families of PDF. Digital PDFs are generated by a program (an accounting system, Word, a banking portal) and contain the actual text, with the position of every character. Scanned PDFs, on the other hand, are photographs of pages: inside there is only an image.

The simplest test is to open the file and try to select a word with the mouse. If it highlights, the text is there. If the whole page is selected as a block, it is an image and you first need OCR, optical character recognition. Everything that follows is about digital PDFs.

Installing pdfplumber and extracting text

pip install pdfplumber

pdfplumber reads each page and knows the position of characters, lines and rectangles. A few lines are enough to extract plain text:

import pdfplumber

with pdfplumber.open("price_list.pdf") as pdf:
    for page in pdf.pages:
        print(page.extract_text())

The result follows the reading order fairly well, but on documents laid out in several columns it may mix the blocks up. In those cases it is better to crop the area of interest with page.crop((x0, top, x1, bottom)) and work on that alone.

Extracting tables

The library's strong point is extract_tables(), which returns each table as a list of rows, each row a list of cells:

import csv
import pdfplumber

with pdfplumber.open("statement.pdf") as pdf, \
     open("transactions.csv", "w", newline="", encoding="utf-8") as out:
    writer = csv.writer(out, delimiter=";")
    for page in pdf.pages:
        for table in page.extract_tables():
            for row in table:
                writer.writerow(row)

The semicolon as a separator is a practical choice: Italian and other European versions of Excel open such a CSV straight into the right columns.

When the table has no borders

Extraction works best when the table is drawn with visible lines, because pdfplumber uses those very lines to work out where cells begin and end. Many documents, however, align columns using spaces only. In that case you change strategy by passing some settings:

settings = {
    "vertical_strategy": "text",
    "horizontal_strategy": "text",
}
tables = page.extract_tables(settings)

With the text strategy the library infers the columns from how words line up. It is less precise and must be checked, but on a document that never changes, such as the same supplier's monthly report, once you find the right settings it works consistently. For debugging, page.to_image().debug_tablefinder() is very handy: it produces an image with the detected cells highlighted.

Cleaning the extracted data

Extracted text is almost never ready for calculations. Typical cleaning steps are:

• repeated headers: in a multi-page document the header row appears on every page and must be dropped except for the first;
• empty or None cells: separator rows and subtotals produce empty cells that need filtering;
• numbers in European format: 1.234,56 must become 1234.56 by removing the thousands dot and replacing the comma;
• multi-line descriptions: a long description may break across two table rows; you can often spot it because the second row has no date or amount, and it should be joined to the previous one.

def amount(text):
    return float(text.replace(".", "").replace(",", "."))

Getting the data into Excel

If the final goal is a spreadsheet, the pandas library simplifies the last step: you build a DataFrame from the cleaned rows and save it with to_excel(), which also requires the openpyxl package. The advantage over CSV is that numbers stay numbers and dates can be converted into real dates, ready for filters and pivot tables.

Always check the totals

Automatic extraction can lose a row without raising any error. In my work I always add a final check: I add up the extracted amounts and compare them with the total printed in the document. If they match, the extraction is reliable; if not, I know exactly which page to look at again. It is a two-line check that makes the automation trustworthy.

In short

• First check whether the PDF contains text: scans need OCR.
• pdfplumber extracts text and tables because it knows the position of every element.
• For borderless tables use the text strategy and check the result.
• Clean up repeated headers, empty cells and European-format numbers.
• Always compare the extracted sum with the document's total.