Quantities extraction with quantities="all" causes recursive errors

[percevalw]

@LoickChardon was trying to use eds.quantities, when this error occurred

nlp = edsnlp.blank("eds")
nlp.add_pipe(eds.sentences())
nlp.add_pipe(
    eds.quantities(
        quantities="all",
        use_tables=False
    )
)

doc_iterator = edsnlp.data.from_spark(
    surgical_notes,
    converter="omop",
    doc_attributes=["note_id", "note_text"]
)
docs = nlp.pipe(doc_iterator)
docs = docs.set_processing(show_progress=True)

durations_nlp = edsnlp.data.to_spark(docs, converter=get_entities)

other_unit = 'registry'
repr-error 'maximum recursion depth exceeded' .

│ env/lib/python3.7/site-packages/edsnlp/pipes/misc/quantities/quantities.py:193 in __getattr__ │
│ │
│ 190 │ ╭─────────────────────────── locals ───────────────────────────╮ │
│ 191 │ def __getattr__(self, other_unit: str): │ other_unit = 'registry' │ │
│ 192 │ │ try: │ self = <repr-error 'maximum recursion depth exceeded'> │ │
│ ❱ 193 │ │ │ return self.convert_to(other_unit) ╰──────────────────────────────────────────────────────────────╯ │
│ 194 │ │ except KeyError: │
│ 195 │ │ │ raise AttributeError(f"Unit {other_unit} not found") │
│ 196 │
│ │
│ env/lib/python3.7/site-packages/edsnlp/pipes/misc/quantities/quantities.py:179 in convert_to │
│ │
│ 176 │ │ return self.convert_to(other.unit) <= min((part.value for part in other)) ╭─────────────────────────── locals ───────────────────────────╮ │
│ 177 │ │ other_unit = 'registry' │ │
│ 178 │ def convert_to(self, other_unit: str): │ self = <repr-error 'maximum recursion depth exceeded'> │ │
│ ❱ 179 │ │ self_degrees, self_scale = self.registry.parse_unit(self.unit) ╰──────────────────────────────────────────────────────────────╯ │
│ 180 │ │ other_degrees, other_scale = self.registry.parse_unit(other_unit) │
│ 181 │ │ │
│ 182 │ │ if self_degrees != other_degrees:

How to reproduce the bug

Unclear at this point

Your Environment

• Operating System: Linux
• Python Version Used: 3.7.16
• spaCy Version Used: 3.7.5
• EDS-NLP Version Used: 0.15.0
• Environment Information:

[percevalw]

Hi, @LoickChardon, can you post the get_entities function ? I suspect it might be the cause of this.

[LoickChardon]

Hi @percevalw, here's the get_entities() function I use in my script :

def get_entities(doc):

quantities = doc.spans.get("quantities", [])
duree_operation = doc.spans.get("duree_operation", [])
note_id = doc._.note_id
note_type = doc._.note_type

entities = [
    {
        "note_id": note_id,
        "note_type": note_type,
        "duration_raw": quant.text,
        "duration_context": quant.sent.text,
        "duration_value": str(quant._.value.minute),
        "mention_start_char": duree.start_char,
        "duration_start_char": quant.start_char,
    }
    for duree in duree_operation
    for quant in quantities
    if duree.sent.start_char == quant.sent.start_char
]

if not entities:
    entities = [{
        "note_id": note_id,
        "note_type": note_type,
        "duration_raw": None,
        "duration_context": None,
        "duration_value": None,
        "mention_start_char": None,
        "duration_start_char": None,
    }]

return entities