Added checkpoint-redis project with tests and code

[yahav-levy]

Feat: Introduce Redis Checkpoint Saver

Type: New Feature

Summary

This pull request introduces a new checkpoint saver implementation using Redis, named RedisSaver. This saver provides a robust and efficient way to persist LangGraph checkpoints, leveraging several Redis best practices to ensure data integrity, performance, and scalability.

Attribution

This implementation incorporates and adapts code from the levivoelz/checkpoint-redis project, which is licensed under the MIT License. The original code has been modified to align with LangGraph's architecture and to integrate additional features such as TTL support and enhanced indexing.

Motivation

For LangGraph applications requiring robust and scalable checkpoint management, a persistent and high-performance backend is essential. The RedisSaver is designed to meet these demands by leveraging Redis as its storage layer.

This RedisSaver provides a dedicated solution for users seeking to integrate LangGraph's powerful checkpointing features with the proven capabilities of Redis.

Detailed Description

The RedisSaver implementation adheres to common Redis best practices and offers the following key features:

• Atomic Operations: Utilizes Redis MULTI / EXEC transactions for critical operations like setting checkpoints and their associated index entries. This ensures that checkpoint data and its corresponding index are written atomically, preventing partial writes and maintaining data consistency.
• Indexed Queries:
  • Checkpoints are indexed in a sorted set (ZSET) by their timestamp (ts). This allows for efficient chronological listing and retrieval of checkpoints, including pagination capabilities (e.g., fetching checkpoints before a certain ID, and limiting the number of results).
  • Writes associated with a checkpoint are also indexed in a sorted set, ordered by their insertion sequence. This allows for reliable retrieval of all writes for a given checkpoint.
• TTL Support: All keys stored in Redis (checkpoints, indexes, writes) support Time-To-Live (TTL). This allows for automatic expiration of old checkpoint data, helping to manage storage and prevent Redis from filling up with stale data. The TTL is configurable.
• Efficient Data Storage:
  • Checkpoint data and metadata are serialized and stored efficiently.
  • Associated pending writes are stored separately and linked to their parent checkpoint, allowing for granular management.
• Clear Key Schema: A well-defined and namespaced key schema (checkpoint:<threadId>:<checkpointNs>:<checkpointId>, writes:<threadId>:<checkpointNs>:<checkpointId>:<taskId>:<idx>, checkpoint_index:<threadId>:<checkpointNs>, writes_index:<threadId>:<checkpointNs>:<checkpointId>) is used for easy debugging and management of data within Redis.
• Comprehensive Unit Tests: The RedisSaver is accompanied by a suite of unit tests (redis-saver.test.ts) covering various scenarios, including:
  • Saving and retrieving checkpoints.
  • Saving and retrieving checkpoints with pending writes.
  • Listing checkpoints with and without filters (limit, before).
  • Handling edge cases like missing thread_id or non-existent checkpoints.

The implementation is organized into:

• redis-saver.ts: Contains the main RedisSaver class, implementing the BaseCheckpointSaver interface.
• checkpoint-redis-repository.ts: Encapsulates the direct Redis interactions, providing an abstraction layer for CRUD operations on checkpoints and writes. This promotes separation of concerns and makes the main saver logic cleaner.
• utils.ts: Includes helper functions for Redis key generation, parsing, and data serialization/deserialization.

How to Test

1. Ensure you have a Redis instance running (e.g., using the provided docker-compose.yml in the libs/checkpoint-redis directory, which can be started with docker-compose up).
2. The existing unit tests in libs/checkpoint-redis/src/tests/redis-saver.test.ts can be run using the standard testing command for the workspace (e.g., yarn test or yarn test:single libs/checkpoint-redis/src/tests/redis-saver.test.ts from within the libs/checkpoint-redis directory).
3. Manual testing can be performed by instantiating RedisSaver in a LangGraph application and configuring it as the checkpointer.

Example Usage:

import { Redis } from "ioredis";
import { RedisSaver } from "@langchain/langgraph-checkpoint-redis"; // Adjust import path as per final package structure

// ... assuming a LangGraph setup ...

const redisConnection = new Redis({ port: 6379 }); // Or your Redis connection details
const checkpointSaver = new RedisSaver({ connection: redisConnection, ttlSeconds: 86400 }); // Optional: 1 day TTL

const graph = new StateGraph({
  channels: {}, // your channels
  schema: YourStateType, // your state type
})
  // ... add nodes and edges ...
  .compile({ checkpointer: checkpointSaver });

// ... then use the graph with thread_id for checkpointing ...
// await graph.invoke({ ... }, { configurable: { thread_id: "my-thread-1" } });

This contribution aims to provide a high-quality, production-grade checkpointing solution for LangGraph users who prefer or require Redis. I'm open to feedback and further improvements!