Skip to content

feat(community): Add a Hyperbrowser package for the js library #7960

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

AHarmlessPyro
Copy link

Description

Hyperbrowser is a platform for running and scaling headless browsers. It lets you launch and manage browser sessions at scale and provides easy to use solutions for any webscraping needs, such as scraping a single page or crawling an entire site.

This PR adds a web document loader, and various tools

  • Web automation tools
    • Scrape page
    • Crawl a website
    • Structured Extraction
  • Hosted Browser Agents
    • Claude Computer Use
    • OpenAI CUA
    • Browser-Use

Integration tests for both the tools and web document loader are also provided. Docs for the same are also provided.

Docs for hyperbrowser are available here

Usage

The document loader and tools both require an API Key. The API Key can be provided either as an argument, or through HYPERBROWSER_API_KEY env var

Issue

None

Dependencies

None

Twitter Handle

@hyperbrowser

Copy link

vercel bot commented Apr 3, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchainjs-docs ❌ Failed (Inspect) Apr 4, 2025 0:07am
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
langchainjs-api-refs ⬜️ Ignored (Inspect) Apr 4, 2025 0:07am

@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. auto:documentation Changes to documentation and examples, like .md, .rst, .ipynb files. Changes to the docs/ folder auto:enhancement A large net-new component, integration, or chain. Use sparingly. The largest features labels Apr 3, 2025
@AHarmlessPyro AHarmlessPyro changed the title feat(hyperbrowser): Add a Hyperbrowser Partner package for the js library feat(community): Add a Hyperbrowser Partner package for the js library Apr 3, 2025
@AHarmlessPyro AHarmlessPyro changed the title feat(community): Add a Hyperbrowser Partner package for the js library feat(community): Add a Hyperbrowser package for the js library Apr 3, 2025
@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels Apr 4, 2025
Copy link
Collaborator

@jacoblee93 jacoblee93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, and apologies for the delay!

Looks fine overall, some really small nits. I'd push them to your branch myself but I don't seem to have access.


| Class | Package | Local | [PY support](https://python.langchain.com/docs/integrations/document_loaders/hyperbrowser/)|
| :---: | :---: | :---: | :---: |
| HyperbrowserLoader | langchain-js | ❌ | ✅ |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Package should be @langchain/community - apologies but I don't think I have access to push to your branch to make changes myself.


| Tool Class | Package | [Python Support](https://python.langchain.com/docs/integrations/tools/hyperbrowser) |
| :---: | :---: | :---: |
| HyperbrowserScrapingTool | langchain-js | ✅ |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above

@benjamincburns
Copy link
Collaborator

Hi @AHarmlessPyro,

I'm really sorry for the back-and-forth on this. There's a couple of things at play here.

It looks like this PR started off life as a partner package, and @jacoblee93 suggested that it should be part of community. The reason for this was that we generally look for partner integration packages to be submitted by the partner in question. This is because we're a small team and there's no way we can maintain the expertise necessary to support every potential integration type.

Historically the default target for submissions from community members was the @langchain/community package. However this is becoming somewhat problematic for a number of reasons:

  • the overall package "weight" is quite heavy (not just the size of the package itself, but the entire dependency tree)
  • all implementations within community share the same dependencies, making it difficult to reliably bump dependency versions for e.g. CVEs, etc.
  • just as with partner packages, the expertise required to maintain the functionality within community is quite broad, and we have a small team

Again, I'm really sorry for the back-and-forth on this, but I think we should not add this to the community package. Instead, I'd recommend one of two courses of action:

  1. We close this PR, and you move this work to a repository that you own and control, and publish it to NPM under your own package. Once that's done, you can open a new PR here to link from our docs to yours as a third party integration. This will help with discoverability.
  2. Assuming you aren't a member of the Hyperbrowser team, you can reach out to them to see if they're willing to help us maintain this. If they are, sweet - we can revert this back to being a partner package. If they aren't, I think option 1 is likely our best bet.

@benjamincburns
Copy link
Collaborator

@AHarmlessPyro my apologies once again - it looks like you are a member of the team at Hyperbrowser? If so, would you mind reaching out to me at [email protected]?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto:documentation Changes to documentation and examples, like .md, .rst, .ipynb files. Changes to the docs/ folder auto:enhancement A large net-new component, integration, or chain. Use sparingly. The largest features size:XXL This PR changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants