A statistical sentence segmenter predicts sentence boundaries from learned patterns, offering a practical middle ground between fast regular expressions and a slower full NLP pipeline. section 11.2.2
A focused spaCy configuration follows a simple workflow:
- Inspect the default pipeline, exclude unused components, and enable only the senter component. section 11.2.2
- Compare resulting sentence lists with a regular-expression segmenter to identify disagreements, while remembering that agreement does not prove correctness. section 11.2.2
A faster spaCy pipeline
Enabling only senter reduced processing time for the demonstrated extraction task to about 2.3 seconds, compared with roughly 16 seconds for the full pipeline; regular expressions remained faster but less accurate. section 11.2.2
>>> import time
>>> nlp.enable_pipe('senter')
>>> nlp.pipeline
[('senter', <spacy.pipeline.senter.SentenceRecognizer at 0x...>)]
>>> t0 = time.time(); lines2 = extract_lines(nlp=nlp); t1=time.time()
>>> t1 - t0
2.3...
Run the minimal spaCy pipeline example and inspect its sole senter component alongside the measured runtime. section 11.2.2
