Skip to content

feat: docs organized by area with better how to sections #488

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 33 commits into from
Oct 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
59a814d
evaluation almost there
jakerachleff Oct 22, 2024
6141767
evaluation working
jakerachleff Oct 22, 2024
17eec68
observability concepts
jakerachleff Oct 23, 2024
05f9c8c
moved tracing how tos
jakerachleff Oct 23, 2024
316be24
move monitoring how to guides
jakerachleff Oct 23, 2024
4652bad
observability and eval are moved completely
jakerachleff Oct 23, 2024
4bae3f9
prompt engineering complete
jakerachleff Oct 23, 2024
8f5ab12
added administartion section to docs
jakerachleff Oct 23, 2024
1474a19
moved pricing under admin
jakerachleff Oct 23, 2024
81dc4c9
remove indexes
jakerachleff Oct 23, 2024
8163923
add pricing to old docs and freeze
jakerachleff Oct 23, 2024
cd6c7db
link resolution: fixing broken links
jakerachleff Oct 23, 2024
2f77733
more links
jakerachleff Oct 23, 2024
9897147
fix build
jakerachleff Oct 23, 2024
91751da
fix more links
jakerachleff Oct 23, 2024
105356a
more links fixes
jakerachleff Oct 23, 2024
91c0dc3
links hopefully done
jakerachleff Oct 23, 2024
b51fba8
update sidebar to have max depth
jakerachleff Oct 24, 2024
c04aef6
obs ht guides
jakerachleff Oct 24, 2024
358f90c
fix
jakerachleff Oct 24, 2024
f3d2f3a
fix
jakerachleff Oct 24, 2024
245794b
running locally
jakerachleff Oct 24, 2024
9eead8e
fix redirects
jakerachleff Oct 24, 2024
39d6200
fix
jakerachleff Oct 24, 2024
b473613
fix
jakerachleff Oct 24, 2024
68e5960
fix
jakerachleff Oct 24, 2024
fc69cca
add pricing routes
jakerachleff Oct 24, 2024
8aadf30
move tutorials
jakerachleff Oct 24, 2024
04a5cfb
more vercel redirects
jakerachleff Oct 24, 2024
de05002
missing comma
jakerachleff Oct 24, 2024
cab2e53
fix bad redirect
jakerachleff Oct 24, 2024
9ea2d17
missing tutorial
jakerachleff Oct 24, 2024
1de0305
add top level redirect
jakerachleff Oct 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
404 changes: 404 additions & 0 deletions docs/administration/concepts/index.mdx

Large diffs are not rendered by default.

28 changes: 28 additions & 0 deletions docs/administration/how_to_guides/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Administration how-to guides

Step-by-step guides that cover key tasks and operations in LangSmith.

## Organization Management

See the following guides to set up your LangSmith account.

- [Create an account and API key](./how_to_guides/organization_management/create_account_api_key)
- [Set up an organization](./how_to_guides/organization_management/set_up_organization)
- [Create an organization](./how_to_guides/organization_management/set_up_organization#create-an-organization)
- [Manage and navigate workspaces](./how_to_guides/organization_management/set_up_organization#manage-and-navigate-workspaces)
- [Manage users](./how_to_guides/organization_management/set_up_organization#manage-users)
- [Manage your organization using the API](./how_to_guides/organization_management/manage_organization_by_api)
- [Set up a workspace](./how_to_guides/organization_management/set_up_workspace)
- [Create a workspace](./how_to_guides/organization_management/set_up_workspace#create-a-workspace)
- [Manage users](./how_to_guides/organization_management/set_up_workspace#manage-users)
- [Configure workspace settings](./how_to_guides/organization_management/set_up_workspace#configure-workspace-settings)
- [Set up billing](./how_to_guides/organization_management/set_up_billing)
- [Update invoice email, tax id and, business information](./how_to_guides/organization_management/update_business_info)
- [Set up access control (enterprise only)](./how_to_guides/organization_management/set_up_access_control)
- [Create a role](./how_to_guides/organization_management/set_up_access_control#create-a-role)
- [Assign a role to a user](./how_to_guides/organization_management/set_up_access_control#assign-a-role-to-a-user)
- [Set up resource tags](./how_to_guides/organization_management/set_up_resource_tags)
- [Create a tag](./how_to_guides/organization_management/set_up_resource_tags#create-a-tag)
- [Assign a tag to a resource](./how_to_guides/organization_management/set_up_resource_tags#assign-a-tag-to-a-resource)
- [Delete a tag](./how_to_guides/organization_management/set_up_resource_tags#delete-a-tag)
- [Filter resources by tags](./how_to_guides/organization_management/set_up_resource_tags#filter-resources-by-tags)
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@ import { RegionalUrl } from "@site/src/components/RegionalUrls";
To get started with LangSmith, you need to create an account. You can sign up for a free account <RegionalUrl text='here'/>.
We support logging in with Google, GitHub, Discord, and email.

![](../static/create_account.png)
![](./static/create_account.png)

## API keys

LangSmith supports two types of API keys: Service Keys and Personal Access Tokens.
Both types of tokens can be used to authenticate requests to the LangSmith API, but they have different use cases.

Read more about the differences between Service Keys and Personal Access Tokens under [admin concepts](../../concepts/admin/admin.mdx)
Read more about the differences between Service Keys and Personal Access Tokens under [admin concepts](../../concepts)

## Create an API key

Expand All @@ -30,7 +30,7 @@ To create either type of API key head to the <RegionalUrl text='Settings page' s
The API key will be shown only once, so make sure to copy it and store it in a safe place.
:::

![](../static/create_api_key.png)
![](./static/create_api_key.png)

## Configure the SDK

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import React, { useEffect } from "react";
import { useHistory } from "@docusaurus/router";

export default function ChildPage() {
const history = useHistory();

useEffect(() => {
// Redirect to parent page on load
history.push("../how_to_guides");
}, [history]);

return null; // No need to render anything since we're redirecting
}
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,16 @@ LangSmith's API supports programmatic access via API key to all of the actions a

Before diving into this content, it might be helpful to read the following:

- [Conceptual guide on organizations and workspaces](../../concepts/admin)
- [Organization setup how-to guild](../../how_to_guides/setup/set_up_organization.mdx)
- [Conceptual guide on organizations and workspaces](../../concepts)
- [Organization setup how-to guild](./set_up_organization.mdx)

:::

:::note
There are a few limitations that will be lifted soon:

- The LangSmith SDKs do not support these organization management actions yet.
- [Service Keys](../../concepts/admin/admin.mdx#api-keys) don't have access to newly-added workspaces yet (we're adding support soon). We recommend using a PAT of an Organization Admin for now, which by default has the required permissions for these actions.
- [Service Keys](../../concepts#api-keys) don't have access to newly-added workspaces yet (we're adding support soon). We recommend using a PAT of an Organization Admin for now, which by default has the required permissions for these actions.

:::

Expand Down Expand Up @@ -120,7 +120,7 @@ Workspace level:
/>

:::note
These params should be omitted: `read_only` (deprecated), `password` and `full_name` ([basic auth](../../reference/authentication_authorization/authentication_methods.mdx) only)
These params should be omitted: `read_only` (deprecated), `password` and `full_name` ([basic auth](/reference/authentication_authorization/authentication_methods.mdx) only)
:::

## API Keys
Expand All @@ -146,7 +146,7 @@ If the header is not present, operations will default to the workspace the API k
These endpoints are user-scoped and require a logged-in user's JWT, so they should only be executed through the UI.

- `/api-key/current` endpoints: these are related a user's PATs
- `/sso/email-verification/send` (Cloud-only): this endpoint is related to [SAML SSO](../../how_to_guides/setup/set_up_saml_sso.mdx)
- `/sso/email-verification/send` (Cloud-only): this endpoint is related to [SAML SSO](./set_up_saml_sso.mdx)

## Sample Code

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ import { RegionalUrl } from "@site/src/components/RegionalUrls";

:::note
RBAC (Role-Based Access Control) is a feature that is only available to Enterprise customers. If you are interested in this feature, please contact our sales team at [email protected]
Other plans default to using the Admin role for all users. Read more about roles under [admin concepts](../../concepts/admin/admin.mdx)
Other plans default to using the Admin role for all users. Read more about roles under [admin concepts](../../concepts)
:::

:::tip Recommended Reading

Before diving into this content, it might be helpful to read the following:

- [Conceptual guide on organizations and workspaces](../../concepts/admin)
- [Conceptual guide on organizations and workspaces](../../concepts)

:::

Expand All @@ -32,7 +32,7 @@ To create a role, navigate to the `Roles` tab in the `Members and roles` section

Click on the `Create Role` button to create a new role. You should see a form like the one below:

![Create Role](../static/create_role.png)
![Create Role](./static/create_role.png)

Assign permissions for the different LangSmith resources that you want to control access to.

Expand All @@ -42,8 +42,8 @@ Once you have your roles set up, you can assign them to users. To assign a role

Each user will have a `Role` dropdown that you can use to assign a role to them.

![Assign Role](../static/assign_role.png)
![Assign Role](./static/assign_role.png)

You can also invite new users with a given role.

![Invite User](../static/invite_user.png)
![Invite User](./static/invite_user.png)
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ add a credit card on the Plans and Billing page as follows:

### 1. Click `Set up Billing`

![](../static/free_tier_billing_page.png)
![](./static/free_tier_billing_page.png)

### 2. Add your credit card info

Expand All @@ -49,14 +49,14 @@ If you are a startup building with AI, please instead click `Apply Now` on our S
eligible for discounted prices and a generous free, monthly trace allotment.
:::

![](../static/new_org_billing_page.png)
![](./static/new_org_billing_page.png)

### 2. Review your existing members

Before subscribing, LangSmith lets you remove any added users that you would not
like to be charged for.

![](../static/new_org_manage_spend.png)
![](./static/new_org_manage_spend.png)

### 3. Enter your credit card info

Expand All @@ -78,7 +78,7 @@ rate limited to a maximum of 5,000 traces per month.

### 2. Click `Set up Billing`

![](../static/setup_billing_legacy.png)
![](./static/setup_billing_legacy.png)

### 3. Enter your credit card info

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ import { RegionalUrl } from "@site/src/components/RegionalUrls";

Before diving into this content, it might be helpful to read the following:

- [Conceptual guide on organizations and workspaces](../../concepts/admin)
- [Conceptual guide on organizations and workspaces](../../concepts)

:::

:::note
If you're interested in managing your organization and workspaces programmatically, see [this hot-to guide](../../how_to_guides/setup/manage_organization_by_api.mdx).
If you're interested in managing your organization and workspaces programmatically, see [this how-to guide](./manage_organization_by_api.mdx).
:::

## Create an organization
Expand All @@ -25,14 +25,14 @@ When you log in for the first time, a personal organization will be created for
To do this, head to the <RegionalUrl text='Settings page' suffix='/settings'/> and click **Create Organization**.
Shared organizations require a credit card before they can be used. You will need to [set up billing](./set_up_billing) to proceed.

![](../static/create_organization.png)
![](./static/create_organization.png)

## Manage and navigate workspaces

Once you've subscribed to a plan that allows for multiple users per organization, you can [set up workspaces](./set_up_workspace) to collaborate more effectively and isolate LangSmith resources between different groups of users.
To navigate between workspaces and access the resources within each workspace (trace projects, annotation queues, etc.), select the desired workspace from the picker in the top left:

![](../static/select_workspace.png)
![](./static/select_workspace.png)

## Manage users

Expand All @@ -43,7 +43,7 @@ Here you can
- Edit a user's organization role
- Remove users from your organization

![](../static/organization_members_and_roles.png)
![](./static/organization_members_and_roles.png)

Organizations on the Enterprise plan may set up custom workspace roles in the `Roles` tab here. See the [access control setup guide](./set_up_access_control.mdx) for more details.

Expand All @@ -59,4 +59,4 @@ The `Organization User` role is only available in organizations on plans with mu
Custom organization-scoped roles are not available yet.
:::

See [this conceptual guide](../../concepts/admin#organization-roles) for a full list of permissions associated with each role.
See [this conceptual guide](../../concepts#organization-roles) for a full list of permissions associated with each role.
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Before diving into this content, it might be helpful to read the following:

- [Conceptual guide on organizations and workspaces](../../concepts/admin)
- [Conceptual guide on organizations and workspaces](../../concepts)

:::

Expand All @@ -22,7 +22,7 @@ Here, you'll be able to see the existing tag values, grouped by key. Two keys `A

To create a new tag, click on the "New Tag" button. You'll be prompted to enter a key and a value for the tag. Note that you can use an existing key or create a new one.

![](../static/resource_tags/create_tag.png)
![](./static/resource_tags/create_tag.png)

## Assign a tag to a resource

Expand All @@ -34,7 +34,7 @@ You can only tag workspace-scoped resources with resource tags. This includes Tr

You can also assign tags to resources from the resource's detail page. Click on the Resource tags button to open up the tag panel and assign tags.

![](../static/resource_tags/assign_tag.png)
![](./static/resource_tags/assign_tag.png)

To un-assign a tag from a resource, click on the Trash icon next to the tag, both in the tag panel and the resource tag panel.

Expand All @@ -44,7 +44,7 @@ You can delete either a key or a value of a tag from the [workspace settings pag

Note that if you delete a key, all values associated with that key will also be deleted. When you delete a value, you will lose all associations between that value and resources.

![](../static/resource_tags/delete_tag.png)
![](./static/resource_tags/delete_tag.png)

## Filter resources by tags

Expand All @@ -56,4 +56,4 @@ In the homepage, you can see updated counts for resources based on the tags you'

As you navigate through the different product surfaces, you will _only_ see resources that match the tags you've selected. At any time, you can clear the tags to see all resources in the workspace or select different tags to filter by.

![](../static/resource_tags/filter_by_tags.png)
![](./static/resource_tags/filter_by_tags.png)
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Single Sign-On (SSO) functionality is available for Enterprise customers to acce
LangSmith's SSO configuration is built using the SAML (Security Assertion Markup Language) 2.0 standard. SAML 2.0 enables connecting an Identity Provider (IdP) to your organization for an easier, more secure login experience.

:::note
SAML SSO is available for organizations on the [Enterprise plan](../../pricing.mdx). Please [contact sales](https://www.langchain.com/contact-sales) to learn more.
SAML SSO is available for organizations on the [Enterprise plan](../../pricing). Please [contact sales](https://www.langchain.com/contact-sales) to learn more.
:::

## What is SAML SSO?
Expand All @@ -26,7 +26,7 @@ SSO services permit a user to use one set of credentials (for example, a name or

- Your organization must be on an Enterprise plan
- Your Identity Provider (IdP) must support the SAML 2.0 standard
- Only [Organization Admins](../../concepts/admin#organization-roles) can configure SAML SSO
- Only [Organization Admins](../../concepts#organization-roles) can configure SAML SSO

### Initial configuration

Expand Down Expand Up @@ -61,15 +61,15 @@ The URLs are different for the US and EU. Please make sure to select your region
LangSmith supports Just-in-Time provisioning when using SAML SSO. This allows someone signing in via SAML SSO to join the organization and selected workspaces automatically as a member.

:::note
JIT provisioning only runs for new users i.e. users who do not already have access to the organization with the same email address via a [different login method](../../reference/authentication_authorization/authentication_methods.mdx#cloud)
JIT provisioning only runs for new users i.e. users who do not already have access to the organization with the same email address via a [different login method](/reference/authentication_authorization/authentication_methods.mdx#cloud)
:::

## Login methods and access

Once you have completed your configuration of SAML SSO for your organization, users will be able to login via SAML SSO in addition to [other login methods](../../reference/authentication_authorization/authentication_methods.mdx#cloud) such as username/password and Google Authentication.
Once you have completed your configuration of SAML SSO for your organization, users will be able to login via SAML SSO in addition to [other login methods](/reference/authentication_authorization/authentication_methods.mdx#cloud) such as username/password and Google Authentication.

- When logged in via SAML SSO, users can only access the corresponding organization with SAML SSO configured.
- Users with SAML SSO as their only login method do not have [personal organizations](../../concepts/admin/admin.mdx#organizations)
- Users with SAML SSO as their only login method do not have [personal organizations](../../concepts#organizations)
- When logged in via any other method, users can access the organization with SAML SSO configured along with any other organizations they are a part of

## Enforce SAML SSO only
Expand All @@ -93,7 +93,7 @@ If you have issues setting up SAML SSO, please reach out to [[email protected]

Some identity providers retain the original `User ID` through an email change while others do not, so we recommend that you follow these steps to avoid duplicate users in LangSmith:

1. Remove the user from the organization (see [here](../setup/set_up_organization.mdx#manage-users))
1. Remove the user from the organization (see [here](./set_up_organization.mdx#manage-users))
1. Change their email address in the IdP
1. Have them login to LangSmith again via SAML SSO - this will trigger the usual [JIT provisioning](#just-in-time-jit-provisioning) flow with their new email address

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@ import { RegionalUrl } from "@site/src/components/RegionalUrls";

Before diving into this content, it might be helpful to read the following:

- [Conceptual guide on organizations and workspaces](../../concepts/admin)
- [Conceptual guide on organizations and workspaces](../../concepts)

:::

When you log in for the first time, a default [workspace](../../concepts/admin/admin.mdx#workspaces) will be created for you automatically in your [personal organization](./set_up_organization#personal-vs-shared-organizations).
When you log in for the first time, a default [workspace](../../concepts#workspaces) will be created for you automatically in your [personal organization](./set_up_organization#personal-vs-shared-organizations).
Workspaces are often used to separate resources between different teams or business units, ensuring clear trust boundaries between them. Within each workspace, Role-Based Access Control (RBAC) is implemented to manage permissions and access levels, ensuring that users only have access to the resources and settings necessary for their role. Most LangSmith activity happens in the context of a workspace, each of which has its own settings and access controls.

To organize resources _within_ a workspace, you can use [resource tags](./set_up_resource_tags).
Expand All @@ -24,7 +24,7 @@ To organize resources _within_ a workspace, you can use [resource tags](./set_up
To create a new workspace, head to the <RegionalUrl text='Settings page' suffix='/settings'/> `Workspaces` tab in your shared organization and click **Add Workspace**.
Once your workspace has been created, you can manage its members and other configuration by selecting it on this page.

![](../static/create_workspace.png)
![](./static/create_workspace.png)

:::note
Different plans have different limits placed on the number of workspaces that can be used in an organization.
Expand All @@ -44,4 +44,4 @@ Users may also be invited directly to one or more workspaces when they are [invi

Workspace configuration exists in the <RegionalUrl text='workspace settings page' suffix='/settings/workspaces'/> tab. Select the workspace to configure and then the desired configuration sub-tab. The example below shows the `API keys`, and other configuration options including secrets, models, and shared URLs are available here as well.

![](../static/workspace_settings.png)
![](./static/workspace_settings.png)
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Business information, tax id and invoice email can only be updated for the plus

## Update invoice email

![](../static/update_invoice_email.png)
![](./static/update_invoice_email.png)

To update the email address where your invoices are sent, follow these steps:

Expand All @@ -37,7 +37,7 @@ This ensures that all future invoices will be sent to the updated email address.
In certain jurisdictions, LangSmith is required to collect sales tax. If you are a business, providing your Tax ID may qualify you for a sales tax exemption.
:::

![](../static/update_business_info.png)
![](./static/update_business_info.png)

To update your organization's business information, follow these steps:

Expand Down
Loading
Loading