li-human

1# li-human
2 
3Two tools live in this folder and they both actually run. Use them. Do not
4eyeball this.
5 
6```bash
7python3 humanize.py draft.txt --report        # clean it, show what changed
8python3 detect.py draft.txt                    # score it, five checks
9python3 detect.py before.txt after.txt         # prove the delta
10```
11 
12Both read `slop.json`, which is the lexicon: 100+ stock words and phrases with
13plain-English replacements, 17 invisible character classes, 11 typographic
14substitutions, and 11 structural tells. It is meant to be edited. If the user
15has a word they always use that the lexicon strips, remove it from the file.
16 
17## What gets fixed automatically
18 
19**1. Invisible characters.** Zero-width spaces and joiners, word joiners,
20soft hyphens, byte-order marks, Unicode tag characters, non-breaking and
21narrow spaces. A keyboard does not produce these. They survive copy-paste,
22they are invisible in every editor, and they are the single most mechanical
23thing in generated text. `humanize.py` deletes every one, including any
24remaining Unicode format character it does not have a name for.
25 
26**2. Typography.** Em dash to comma, en dash to hyphen, curly quotes to
27straight, ellipsis to three dots, bullet character to hyphen. The em dash pass
28is the one that matters: it collapses ` — ` to `, ` and then cleans up the
29double punctuation that leaves behind.
30 
31**3. The slop lexicon.** delve, leverage, robust, seamless, crucial, tapestry,
32…