What's new
Since v1.0.1, we haven't been very active when it comes to releases 😇 However, we worked hard on the application itself 💪
The current release 1.1.0 thus brings many significant improvements, some of which are detailed below, and many important bugfixes.
Important
We recommend that you use this version only for a fresh instance. The data generated by the older version of factgenie is most probably not compatible with the current version and vice versa.
🌈 Annotation interface
- New annotation library (#177, #192, #194, #210): We built our own underlying JS annotation library. That allowed us to introduce better visual style, better colors, "undo" functionality, erase / select mode, overlapping annotations, and more.
demo.mp4
- Refactored sliders (#216): We modified the way the sliders work. Now, they work in a more "traditional" mode, where the users can select continuous values, instead of duplicating the role of select boxes for multi-choice answering.
- One-based indexing (#193) : Human annotators now see the examples indexed from 1 instead of 0, which is more natural.
🖥️ Browse interface
- Public mode (#161): It is now possible to publish collected annotations online without providing access to other app parts.
- Outputs with no inputs (#164): It is now possible to have a dataset with no input data.
- New highlight style (#163, #177): The annotation highlights now follow the new style (see above).
🤖 LLM annotations
- New LLM providers (#189): Basing our API calls on the LiteLLM library, we now support annotations from Ollama, vLLM, OpenAI, Google AI Studio, Anthropic, and Vertex AI.
📈 Analysis
- IAA computation (#181, #207, #217, #218): We provide raw data for computing inter-annotator agreement, along with an accompanying Jupyter notebook and CLI interface. We also allow to compute IAA between individual groups in the same campaign.
🔧 Other
- Better logging (#205): The logs from factgenie are now more colorful and informative:
- Caching (#167): Factgenie is now much faster when handling larger amount of data.
Even though we will release a new version on PyPI, we still recommend installing factgenie as an editable Python project.