Can I use the Faiss vector store for Next.js ? If so, how?

[Elindo586]

From the line of code below, I am trying to get the path to the restaurant.txt file which is located under pages/api/restaurant.txt... then I run the code and I expect to see the vector files to be returned, but I can't find them anywhere, and when I run this filenode pages/api/indexes.js I get no errors, but I get nothing as well... what could be wrong?

from pages/api/indexes.js

const { OpenAIEmbeddings } = require("langchain/embeddings/openai");
const { FaissStore } = require("langchain/vectorstores/faiss");
const { CharacterTextSplitter } = require("langchain/text_splitter");
const { TextLoader } = require("langchain/document_loaders/fs/text");

const indexes = async (req, res) => {
  const loader = new TextLoader("../../pages/api/restaurant.txt");
  console.log(loader);
  const docs = await loader.load();
  const splitter = new CharacterTextSplitter({
    chunkSize: 200,
    chunkOverlap: 50,
  });
  const documents = await splitter.splitDocuments(docs);
  console.log(documents);

  const embeddings = new OpenAIEmbeddings();
  const vectorStore = await FaissStore.fromDocuments(documents, embeddings);
  await vectorStore.save("./");

  res.status(200).json({ ok: ok });
};

module.exports = { indexes };

[dosubot[bot]]

🤖

Hello,

Yes, you can use the Faiss vector store with Next.js. The issue you're facing might be due to the relative path you're using to load the restaurant.txt file. In Next.js, the root directory is the project root, so you don't need to use ../../ to go up directories.

Here's how you can modify your code:

const loader = new TextLoader("pages/api/restaurant.txt");

Also, when you save the vector store, you're saving it to the root directory ("./"). If you want to save it to a specific directory, you should specify the directory path. For example, if you want to save it in the pages/api directory, you can do:

await vectorStore.save("pages/api");

Here's the complete modified code:

const { OpenAIEmbeddings } = require("langchain/embeddings/openai");
const { FaissStore } = require("langchain/vectorstores/faiss");
const { CharacterTextSplitter } = require("langchain/text_splitter");
const { TextLoader } = require("langchain/document_loaders/fs/text");

const indexes = async (req, res) => {
  const loader = new TextLoader("pages/api/restaurant.txt");
  console.log(loader);
  const docs = await loader.load();
  const splitter = new CharacterTextSplitter({
    chunkSize: 200,
    chunkOverlap: 50,
  });
  const documents = await splitter.splitDocuments(docs);
  console.log(documents);

  const embeddings = new OpenAIEmbeddings();
  const vectorStore = await FaissStore.fromDocuments(documents, embeddings);
  await vectorStore.save("pages/api");

  res.status(200).json({ ok: ok });
};

module.exports = { indexes };

Please try this and let me know if it works for you. If you're still facing issues, please provide more details about the error or unexpected behavior you're experiencing.

Sources

• examples/src/indexes/vector_stores/faiss_saveload.ts
• examples/src/indexes/vector_stores/faiss_fromdocs.ts
• langchain/src/vectorstores/faiss.ts

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[Elindo586]

I tried the paths you suggested and it doesn't seem to read it... I have tried other paths and I can't get it to work.

I am also not sure if Next.js can work with the .txt file extension

[Elindo586]

Needed to npm faiss-node

PLUS..

Integrated this code to another file because NEXT.js wouldn't run the node pages/api/indexes.js command