filler-word-processing

1# Filler Word Processing
2 
3## Annotation Format
4 
5Typical annotation JSON structure:
6```json
7[
8  {"word": "um", "timestamp": 12.5},
9  {"word": "like", "timestamp": 25.3},
10  {"word": "you know", "timestamp": 45.8}
11]
12```
13 
14## Converting Annotations to Cut Segments
15 
16Each filler word annotation marks when the word starts. To remove it, use word-specific durations since different fillers have different lengths:
17 
18```python
19import json
20 
21# Word-specific durations (in seconds)
22WORD_DURATIONS = {
23    "uh": 0.3,
24    "um": 0.4,
25    "hum": 0.6,
26    "hmm": 0.6,
27    "mhm": 0.55,
28    "like": 0.3,
29    "yeah": 0.35,
30    "so": 0.25,
31    "well": 0.35,
32    "okay": 0.4,
33    "basically": 0.55,
34    "you know": 0.55,
35    "i mean": 0.5,
36    "kind of": 0.5,
37    "i guess": 0.5,
38}
39DEFAULT_DURATION = 0.4
40 
41def annotations_to_segments(annotations_file, buffer=0.05):
42    """
43    Convert filler word annotations to (start, end) cut segments.
44 
45    Args:
46        annotations_file: Path to JSON annotations
47        buffer: Small buffer before the word (seconds)
48 
49    Returns:
50        List of (start, end) tuples representing segments to remove
51    """
52    with open(annotations_file) as f:
53        annotations = json.load(f)
54 
55    segments = []
56    for ann in annotations:
57        word = ann.get('word', '').lower().strip()
58        timestamp = ann['timestamp']
59        # Use word-specific duration, fall back to default
60        word_duration = WORD_DURATIONS.get(word, DEFAULT_DURATION)
61…