pdf-analysis

1# PDF Analysis
2 
3## Step 0 — Detect PDF type (text vs scanned)
4 
5**Critical first step**: determine whether the PDF has extractable text or is a scanned image.
6Never skip this — using the wrong parser wastes time and produces empty results.
7 
8```python
9import fitz  # PyMuPDF
10 
11def detect_pdf_type(pdf_path, sample_pages=3):
12    """
13    Returns 'text' if PDF has extractable text, 'scanned' if image-based.
14    Checks first N pages (or all if fewer).
15    """
16    doc = fitz.open(pdf_path)
17    total_chars = 0
18    pages_checked = min(sample_pages, len(doc))
19 
20    for i in range(pages_checked):
21        page = doc[i]
22        text = page.get_text("text")
23        total_chars += len(text.strip())
24 
25    doc.close()
26    avg_chars = total_chars / max(pages_checked, 1)
27    pdf_type = 'text' if avg_chars > 50 else 'scanned'
28    print(f"PDF type: {pdf_type} (avg {avg_chars:.0f} chars/page, checked {pages_checked} pages)")
29    return pdf_type
30```
31 
32---
33 
34## Core Method 1: Text PDF — Full Text Extraction (ALL pages)
35 
36```python
37import fitz
38 
39def extract_text_pdf(pdf_path):
40    """Extract text from all pages of a text-based PDF."""
41    doc = fitz.open(pdf_path)
42    total_pages = len(doc)
43    print(f"Total pages: {total_pages}")
44 
45    all_text = []
46    for i, page in enumerate(doc):
47        text = page.get_text("text").strip()
48        if text:
49            all_text.append(f"=== Page {i+1} ===\n{text}")
50        else:
51            print(f"  Page {i+1}: no text (may be image — will caption later)")
52…