Spacy produces random results during inference even when the random seed is fixed

[jiangtianli91]

How to reproduce the behaviour

Here is the minimal script:

import spacy
import truecase

text = truecase.get_true_case('ball,wanna get your train out ? I wanna play soccer ball . .')
print(text)

spacy.util.fix_random_seed(326)
nlp = spacy.load("en_core_web_sm")
doc = nlp(text)

cleaned = [t.text if t.ent_type_ not in ['GPE', 'PERSON', 'ORG'] else t.ent_type_ for t in doc]
print(cleaned)

Run it multiple times: for i in {1..5}; do python temp.py; done

I get random results:

Ball, wanNA get your train out? I wanNA play Soccer ball . .
['PERSON', ',', 'PERSON', 'get', 'your', 'train', 'out', '?', 'I', 'wanNA', 'play', 'Soccer', 'ball', '.', '.']
Ball, wanNA get your train out? I wanNA play soccer ball . .
['PERSON', ',', 'PERSON', 'get', 'your', 'train', 'out', '?', 'I', 'PERSON', 'play', 'soccer', 'ball', '.', '.']
Ball, wanNA get your train out? I wanNA play Soccer ball . .
['PERSON', ',', 'PERSON', 'get', 'your', 'train', 'out', '?', 'I', 'wanNA', 'play', 'Soccer', 'ball', '.', '.']
Ball, wanNA get your train out? I wanNA play soccer ball . .
['PERSON', ',', 'PERSON', 'get', 'your', 'train', 'out', '?', 'I', 'PERSON', 'play', 'soccer', 'ball', '.', '.']
Ball, wanNA get your train out? I wanNA play soccer ball . .
['PERSON', ',', 'PERSON', 'get', 'your', 'train', 'out', '?', 'I', 'PERSON', 'play', 'soccer', 'ball', '.', '.']

Your Environment

• spaCy version: 3.8.2
• Platform: Linux-4.15.0-159-generic-x86_64-with-glibc2.27
• Python version: 3.12.3
• Pipelines: en_core_web_sm (3.8.0), en_core_web_trf (3.8.0)

[honnibal]

Thanks for the report, this is concerning. I'll look into it. In the meantime if you check whether this problem still occurs in v3.7.x, let me know.