From 4c1f4c9caf739cb9bcbfa92914bb48cd6a13bd37 Mon Sep 17 00:00:00 2001 From: Jake Rachleff Date: Tue, 22 Oct 2024 22:30:52 -0700 Subject: [PATCH] link resolution: fixing broken links --- docs/administration/concepts/index.mdx | 4 +- docs/administration/how_to_guides/index.md | 40 +++++++++---------- .../create_account_api_key.mdx | 2 +- .../manage_organization_by_api.mdx | 4 +- .../set_up_access_control.mdx | 4 +- .../set_up_organization.mdx | 4 +- .../set_up_resource_tags.mdx | 2 +- .../set_up_saml_sso.mdx | 4 +- .../set_up_workspace.mdx | 4 +- .../manage_datasets_in_application.mdx | 4 +- .../evaluation/evaluate_llm_application.mdx | 2 +- .../human_feedback/attach_user_feedback.mdx | 2 +- .../set_up_feedback_criteria.mdx | 2 +- docs/observability/concepts/index.mdx | 8 ++-- .../filter_traces_in_application.mdx | 2 +- .../tracing/add_metadata_tags.mdx | 2 +- .../tracing/log_traces_to_project.mdx | 2 +- .../tracing/trace_with_langchain.mdx | 2 +- .../data_formats/feedback_data_format.mdx | 2 +- .../data_formats/run_data_format.mdx | 2 +- .../tutorials/Administrators/manage_spend.mdx | 2 +- docs/tutorials/Developers/agents.mdx | 10 ++--- 22 files changed, 55 insertions(+), 55 deletions(-) diff --git a/docs/administration/concepts/index.mdx b/docs/administration/concepts/index.mdx index 16b65c31..2d79981b 100644 --- a/docs/administration/concepts/index.mdx +++ b/docs/administration/concepts/index.mdx @@ -13,7 +13,7 @@ There are a few important differences between your personal organization and sha | Feature | Personal | Shared | | ------------------- | ------------------- | ------------------------------------------------------------ | -| Maximum workspaces | 1 | Variable, depending on plan (see [pricing page](../pricing)) | +| Maximum workspaces | 1 | Variable, depending on plan (see [pricing page](./pricing)) | | Collaboration | Cannot invite users | Can invite users | | Billing: paid plans | Developer plan only | All other plans available | @@ -24,7 +24,7 @@ Workspaces were formerly called Tenants. Some code and APIs may still reference ::: A workspace is a logical grouping of users and resources within an organization. A workspace separates trust boundaries for resources and access control. -Users may have permissions in a workspace that grant them access to the resources in that workspace, including tracing projects, datasets, annotation queues, and prompts. For more details, see the [setup guide](../how_to_guides/organization_management/set_up_workspace.mdx). +Users may have permissions in a workspace that grant them access to the resources in that workspace, including tracing projects, datasets, annotation queues, and prompts. For more details, see the [setup guide](./how_to_guides/organization_management/set_up_workspace). It is recommended to create a separate workspace for each team within your organization. To organize resources even further, you can use [Resource Tags](#resource-tags) to group resources within a workspace. diff --git a/docs/administration/how_to_guides/index.md b/docs/administration/how_to_guides/index.md index d420a024..17be8cf0 100644 --- a/docs/administration/how_to_guides/index.md +++ b/docs/administration/how_to_guides/index.md @@ -6,23 +6,23 @@ Step-by-step guides that cover key tasks and operations in LangSmith. See the following guides to set up your LangSmith account. -- [Create an account and API key](./organization_management/create_account_api_key) -- [Set up an organization](./organization_management/set_up_organization) - - [Create an organization](./organization_management/set_up_organization#create-an-organization) - - [Manage and navigate workspaces](./organization_management/set_up_organization#manage-and-navigate-workspaces) - - [Manage users](./organization_management/set_up_organization#manage-users) - - [Manage your organization using the API](./organization_management/manage_organization_by_api) -- [Set up a workspace](./organization_management/set_up_workspace) - - [Create a workspace](./organization_management/set_up_workspace#create-a-workspace) - - [Manage users](./organization_management/set_up_workspace#manage-users) - - [Configure workspace settings](./organization_management/set_up_workspace#configure-workspace-settings) -- [Set up billing](./organization_management/set_up_billing) -- [Update invoice email, tax id and, business information](./organization_management/update_business_info) -- [Set up access control (enterprise only)](./organization_management/set_up_access_control) - - [Create a role](./organization_management/set_up_access_control#create-a-role) - - [Assign a role to a user](./organization_management/set_up_access_control#assign-a-role-to-a-user) -- [Set up resource tags](./organization_management/set_up_resource_tags) - - [Create a tag](./organization_management/set_up_resource_tags#create-a-tag) - - [Assign a tag to a resource](./organization_management/set_up_resource_tags#assign-a-tag-to-a-resource) - - [Delete a tag](./organization_management/set_up_resource_tags#delete-a-tag) - - [Filter resources by tags](./organization_management/set_up_resource_tags#filter-resources-by-tags) +- [Create an account and API key](./how_to_guides/organization_management/create_account_api_key) +- [Set up an organization](./how_to_guides/organization_management/set_up_organization) + - [Create an organization](./how_to_guides/organization_management/set_up_organization#create-an-organization) + - [Manage and navigate workspaces](./how_to_guides/organization_management/set_up_organization#manage-and-navigate-workspaces) + - [Manage users](./how_to_guides/organization_management/set_up_organization#manage-users) + - [Manage your organization using the API](./how_to_guides/organization_management/manage_organization_by_api) +- [Set up a workspace](./how_to_guides/organization_management/set_up_workspace) + - [Create a workspace](./how_to_guides/organization_management/set_up_workspace#create-a-workspace) + - [Manage users](./how_to_guides/organization_management/set_up_workspace#manage-users) + - [Configure workspace settings](./how_to_guides/organization_management/set_up_workspace#configure-workspace-settings) +- [Set up billing](./how_to_guides/organization_management/set_up_billing) +- [Update invoice email, tax id and, business information](./how_to_guides/organization_management/update_business_info) +- [Set up access control (enterprise only)](./how_to_guides/organization_management/set_up_access_control) + - [Create a role](./how_to_guides/organization_management/set_up_access_control#create-a-role) + - [Assign a role to a user](./how_to_guides/organization_management/set_up_access_control#assign-a-role-to-a-user) +- [Set up resource tags](./how_to_guides/organization_management/set_up_resource_tags) + - [Create a tag](./how_to_guides/organization_management/set_up_resource_tags#create-a-tag) + - [Assign a tag to a resource](./how_to_guides/organization_management/set_up_resource_tags#assign-a-tag-to-a-resource) + - [Delete a tag](./how_to_guides/organization_management/set_up_resource_tags#delete-a-tag) + - [Filter resources by tags](./how_to_guides/organization_management/set_up_resource_tags#filter-resources-by-tags) diff --git a/docs/administration/how_to_guides/organization_management/create_account_api_key.mdx b/docs/administration/how_to_guides/organization_management/create_account_api_key.mdx index 68b1abd4..22c32488 100644 --- a/docs/administration/how_to_guides/organization_management/create_account_api_key.mdx +++ b/docs/administration/how_to_guides/organization_management/create_account_api_key.mdx @@ -18,7 +18,7 @@ We support logging in with Google, GitHub, Discord, and email. LangSmith supports two types of API keys: Service Keys and Personal Access Tokens. Both types of tokens can be used to authenticate requests to the LangSmith API, but they have different use cases. -Read more about the differences between Service Keys and Personal Access Tokens under [admin concepts](../../concepts/index) +Read more about the differences between Service Keys and Personal Access Tokens under [admin concepts](../../concepts) ## Create an API key diff --git a/docs/administration/how_to_guides/organization_management/manage_organization_by_api.mdx b/docs/administration/how_to_guides/organization_management/manage_organization_by_api.mdx index 933e5c36..43b7d9a7 100644 --- a/docs/administration/how_to_guides/organization_management/manage_organization_by_api.mdx +++ b/docs/administration/how_to_guides/organization_management/manage_organization_by_api.mdx @@ -8,7 +8,7 @@ LangSmith's API supports programmatic access via API key to all of the actions a Before diving into this content, it might be helpful to read the following: -- [Conceptual guide on organizations and workspaces](../../concepts/index) +- [Conceptual guide on organizations and workspaces](../../concepts) - [Organization setup how-to guild](.//set_up_organization.mdx) ::: @@ -17,7 +17,7 @@ Before diving into this content, it might be helpful to read the following: There are a few limitations that will be lifted soon: - The LangSmith SDKs do not support these organization management actions yet. -- [Service Keys](../../concepts/index#api-keys) don't have access to newly-added workspaces yet (we're adding support soon). We recommend using a PAT of an Organization Admin for now, which by default has the required permissions for these actions. +- [Service Keys](../../concepts#api-keys) don't have access to newly-added workspaces yet (we're adding support soon). We recommend using a PAT of an Organization Admin for now, which by default has the required permissions for these actions. ::: diff --git a/docs/administration/how_to_guides/organization_management/set_up_access_control.mdx b/docs/administration/how_to_guides/organization_management/set_up_access_control.mdx index 5c2fdf13..31e7876c 100644 --- a/docs/administration/how_to_guides/organization_management/set_up_access_control.mdx +++ b/docs/administration/how_to_guides/organization_management/set_up_access_control.mdx @@ -4,14 +4,14 @@ import { RegionalUrl } from "@site/src/components/RegionalUrls"; :::note RBAC (Role-Based Access Control) is a feature that is only available to Enterprise customers. If you are interested in this feature, please contact our sales team at sales@langchain.dev -Other plans default to using the Admin role for all users. Read more about roles under [admin concepts](../../concepts/index) +Other plans default to using the Admin role for all users. Read more about roles under [admin concepts](../../concepts) ::: :::tip Recommended Reading Before diving into this content, it might be helpful to read the following: -- [Conceptual guide on organizations and workspaces](../../concepts/index) +- [Conceptual guide on organizations and workspaces](../../concepts) ::: diff --git a/docs/administration/how_to_guides/organization_management/set_up_organization.mdx b/docs/administration/how_to_guides/organization_management/set_up_organization.mdx index 19fafc02..8c2c9cb5 100644 --- a/docs/administration/how_to_guides/organization_management/set_up_organization.mdx +++ b/docs/administration/how_to_guides/organization_management/set_up_organization.mdx @@ -10,7 +10,7 @@ import { RegionalUrl } from "@site/src/components/RegionalUrls"; Before diving into this content, it might be helpful to read the following: -- [Conceptual guide on organizations and workspaces](../../concepts/index) +- [Conceptual guide on organizations and workspaces](../../concepts) ::: @@ -59,4 +59,4 @@ The `Organization User` role is only available in organizations on plans with mu Custom organization-scoped roles are not available yet. ::: -See [this conceptual guide](../../concepts/index#organization-roles) for a full list of permissions associated with each role. +See [this conceptual guide](../../concepts#organization-roles) for a full list of permissions associated with each role. diff --git a/docs/administration/how_to_guides/organization_management/set_up_resource_tags.mdx b/docs/administration/how_to_guides/organization_management/set_up_resource_tags.mdx index 705679b2..02237f51 100644 --- a/docs/administration/how_to_guides/organization_management/set_up_resource_tags.mdx +++ b/docs/administration/how_to_guides/organization_management/set_up_resource_tags.mdx @@ -4,7 +4,7 @@ Before diving into this content, it might be helpful to read the following: -- [Conceptual guide on organizations and workspaces](../../concepts/index) +- [Conceptual guide on organizations and workspaces](../../concepts) ::: diff --git a/docs/administration/how_to_guides/organization_management/set_up_saml_sso.mdx b/docs/administration/how_to_guides/organization_management/set_up_saml_sso.mdx index 6ca84581..257600a1 100644 --- a/docs/administration/how_to_guides/organization_management/set_up_saml_sso.mdx +++ b/docs/administration/how_to_guides/organization_management/set_up_saml_sso.mdx @@ -26,7 +26,7 @@ SSO services permit a user to use one set of credentials (for example, a name or - Your organization must be on an Enterprise plan - Your Identity Provider (IdP) must support the SAML 2.0 standard -- Only [Organization Admins](../../concepts/index#organization-roles) can configure SAML SSO +- Only [Organization Admins](../../concepts#organization-roles) can configure SAML SSO ### Initial configuration @@ -69,7 +69,7 @@ JIT provisioning only runs for new users i.e. users who do not already have acce Once you have completed your configuration of SAML SSO for your organization, users will be able to login via SAML SSO in addition to [other login methods](../../../reference/authentication_authorization/authentication_methods.mdx#cloud) such as username/password and Google Authentication. - When logged in via SAML SSO, users can only access the corresponding organization with SAML SSO configured. -- Users with SAML SSO as their only login method do not have [personal organizations](../../concepts/index#organizations) +- Users with SAML SSO as their only login method do not have [personal organizations](../../concepts#organizations) - When logged in via any other method, users can access the organization with SAML SSO configured along with any other organizations they are a part of ## Enforce SAML SSO only diff --git a/docs/administration/how_to_guides/organization_management/set_up_workspace.mdx b/docs/administration/how_to_guides/organization_management/set_up_workspace.mdx index 7c9fe76a..58e23c89 100644 --- a/docs/administration/how_to_guides/organization_management/set_up_workspace.mdx +++ b/docs/administration/how_to_guides/organization_management/set_up_workspace.mdx @@ -10,11 +10,11 @@ import { RegionalUrl } from "@site/src/components/RegionalUrls"; Before diving into this content, it might be helpful to read the following: -- [Conceptual guide on organizations and workspaces](../../concepts/index) +- [Conceptual guide on organizations and workspaces](../../concepts) ::: -When you log in for the first time, a default [workspace](../../concepts/index#workspaces) will be created for you automatically in your [personal organization](./set_up_organization#personal-vs-shared-organizations). +When you log in for the first time, a default [workspace](../../concepts#workspaces) will be created for you automatically in your [personal organization](./set_up_organization#personal-vs-shared-organizations). Workspaces are often used to separate resources between different teams or business units, ensuring clear trust boundaries between them. Within each workspace, Role-Based Access Control (RBAC) is implemented to manage permissions and access levels, ensuring that users only have access to the resources and settings necessary for their role. Most LangSmith activity happens in the context of a workspace, each of which has its own settings and access controls. To organize resources _within_ a workspace, you can use [resource tags](./set_up_resource_tags). diff --git a/docs/evaluation/how_to_guides/datasets/manage_datasets_in_application.mdx b/docs/evaluation/how_to_guides/datasets/manage_datasets_in_application.mdx index a9d69091..d70d99e5 100644 --- a/docs/evaluation/how_to_guides/datasets/manage_datasets_in_application.mdx +++ b/docs/evaluation/how_to_guides/datasets/manage_datasets_in_application.mdx @@ -7,7 +7,7 @@ sidebar_position: 1 :::tip Recommended Reading Before diving into this content, it might be helpful to read the following: -- [Concepts guide on evaluation and datasets](../../evaluation/concepts/index#datasets-and-examples) +- [Concepts guide on evaluation and datasets](../../evaluation/concepts#datasets-and-examples) ::: @@ -19,7 +19,7 @@ To get started, you can create a new datasets by heading to the "Datasets and Te ![](./static/new_dataset.png) -Then, enter the relevant dataset details, including a name, optional description, and dataset type. Please see the [concepts](../../evaluation/concepts/index#datasets-and-examples) for more information on dataset types. For most flexibility, the key-value dataset type is recommended. +Then, enter the relevant dataset details, including a name, optional description, and dataset type. Please see the [concepts](../../evaluation/concepts#datasets-and-examples) for more information on dataset types. For most flexibility, the key-value dataset type is recommended. ![](./static/enter_dataset_details.png) diff --git a/docs/evaluation/how_to_guides/evaluation/evaluate_llm_application.mdx b/docs/evaluation/how_to_guides/evaluation/evaluate_llm_application.mdx index 28d4abe7..64e3b986 100644 --- a/docs/evaluation/how_to_guides/evaluation/evaluate_llm_application.mdx +++ b/docs/evaluation/how_to_guides/evaluation/evaluate_llm_application.mdx @@ -15,7 +15,7 @@ import { :::tip Recommended Reading Before diving into this content, it might be helpful to read the following: -- [Conceptual guide on evaluation](../../evaluation/concepts/index) +- [Conceptual guide on evaluation](../../evaluation/concepts) - [How-to guide on managing datasets](../datasets/manage_datasets_in_application) - [How-to guide on managing datasets programmatically](../datasets/manage_datasets_programmatically) diff --git a/docs/evaluation/how_to_guides/human_feedback/attach_user_feedback.mdx b/docs/evaluation/how_to_guides/human_feedback/attach_user_feedback.mdx index 977fabc6..07843f88 100644 --- a/docs/evaluation/how_to_guides/human_feedback/attach_user_feedback.mdx +++ b/docs/evaluation/how_to_guides/human_feedback/attach_user_feedback.mdx @@ -13,7 +13,7 @@ import { :::tip Recommended Reading Before diving into this content, it might be helpful to read the following: -- [Conceptual guide on tracing and feedback](../../observability/concepts/index) +- [Conceptual guide on tracing and feedback](../../observability/concepts) - [Reference guide on feedback data format](../../reference/data_formats/feedback_data_format) ::: diff --git a/docs/evaluation/how_to_guides/human_feedback/set_up_feedback_criteria.mdx b/docs/evaluation/how_to_guides/human_feedback/set_up_feedback_criteria.mdx index b5ca56e1..645a7964 100644 --- a/docs/evaluation/how_to_guides/human_feedback/set_up_feedback_criteria.mdx +++ b/docs/evaluation/how_to_guides/human_feedback/set_up_feedback_criteria.mdx @@ -9,7 +9,7 @@ import { RegionalUrl } from "@site/src/components/RegionalUrls"; :::tip Recommended Reading Before diving into this content, it might be helpful to read the following: -- [Conceptual guide on tracing and feedback](../../observability/concepts/index) +- [Conceptual guide on tracing and feedback](../../observability/concepts) - [Reference guide on feedback data format](../../reference/data_formats/feedback_data_format) ::: diff --git a/docs/observability/concepts/index.mdx b/docs/observability/concepts/index.mdx index c2c467f0..0a0157b5 100644 --- a/docs/observability/concepts/index.mdx +++ b/docs/observability/concepts/index.mdx @@ -50,10 +50,10 @@ Feedback can currently be continuous or discrete (categorical), and you can reus Collecting feedback on runs can be done in a number of ways: -1. [Sent up along with a trace](../how_to_guides/human_feedback/attach_user_feedback) from the LLM application -2. Generated by a user in the app [inline](../how_to_guides/human_feedback/annotate_traces_inline) or in an [annotation queue](../how_to_guides/human_feedback/annotation_queues) -3. Generated by an automatic evaluator during [offline evaluation](../how_to_guides/evaluation/evaluate_llm_application) -4. Generated by an [online evaluator](../how_to_guides/monitoring/online_evaluations) +1. [Sent up along with a trace](/evaluation/how_to_guides/human_feedback/attach_user_feedback) from the LLM application +2. Generated by a user in the app [inline](/evaluation/how_to_guides/human_feedback/annotate_traces_inline) or in an [annotation queue](../how_to_guides/human_feedback/annotation_queues) +3. Generated by an automatic evaluator during [offline evaluation](/evaluation/how_to_guides/evaluation/evaluate_llm_application) +4. Generated by an [online evaluator](./how_to_guides/monitoring/online_evaluations) To learn more about how feedback is stored in the application, see [this reference guide](../reference/data_formats/feedback_data_format). diff --git a/docs/observability/how_to_guides/monitoring/filter_traces_in_application.mdx b/docs/observability/how_to_guides/monitoring/filter_traces_in_application.mdx index 574f99c4..8d0c0ef7 100644 --- a/docs/observability/how_to_guides/monitoring/filter_traces_in_application.mdx +++ b/docs/observability/how_to_guides/monitoring/filter_traces_in_application.mdx @@ -8,7 +8,7 @@ sidebar_position: 1 Before diving into this content, it might be helpful to read the following to gain familiarity with the concepts mentioned here: -- [Conceptual guide on tracing](../../observability/concepts/index) +- [Conceptual guide on tracing](../../observability/concepts) ::: diff --git a/docs/observability/how_to_guides/tracing/add_metadata_tags.mdx b/docs/observability/how_to_guides/tracing/add_metadata_tags.mdx index 805724ea..e34b5d72 100644 --- a/docs/observability/how_to_guides/tracing/add_metadata_tags.mdx +++ b/docs/observability/how_to_guides/tracing/add_metadata_tags.mdx @@ -15,7 +15,7 @@ LangSmith supports sending arbitrary metadata and tags along with traces. Tags are strings that can be used to categorize or label a trace. Metadata is a dictionary of key-value pairs that can be used to store additional information about a trace. Both are useful for associating additional information with a trace, such as the environment in which it was executed, the user who initiated it, or an internal correlation ID. -For more information on tags and metadata, see the [Concepts](/observability/concepts/index#tags) page. For information on how to query traces and runs by metadata and tags, see the [Filter traces in the application](/how_to_guides/monitoring/filter_traces_in_application) page. +For more information on tags and metadata, see the [Concepts](/observability/concepts#tags) page. For information on how to query traces and runs by metadata and tags, see the [Filter traces in the application](/how_to_guides/monitoring/filter_traces_in_application) page. Add Rule`, and configure the rule as follows: ![](./static/P2SampleTraces.png) Run rules match on runs rather than traces. Runs are single units of work within an LLM application's API handling. Traces -are end to end API calls (learn more about [tracing concepts in LangSmith](../../observability/concepts/index)). This means a trace can +are end to end API calls (learn more about [tracing concepts in LangSmith](../../observability/concepts)). This means a trace can be thought of as a tree of runs making up an API call. When a run rule matches any run within a trace, the trace's full run tree upgrades to be retained for 400 days. diff --git a/docs/tutorials/Developers/agents.mdx b/docs/tutorials/Developers/agents.mdx index 4a91c390..e6338a62 100644 --- a/docs/tutorials/Developers/agents.mdx +++ b/docs/tutorials/Developers/agents.mdx @@ -6,7 +6,7 @@ import { RegionalUrl } from "@site/src/components/RegionalUrls"; # Evaluate an agent -In this tutorial, we will walk through 3 evaluation strategies LLM agents, building on the conceptual points shared in our [evaluation guide](https://docs.smith.langchain.com/evaluation/concepts/index#agents). +In this tutorial, we will walk through 3 evaluation strategies LLM agents, building on the conceptual points shared in our [evaluation guide](https://docs.smith.langchain.com/evaluation/concepts#agents). - `Final Response`: Evaluate the agent's final response. - `Single step`: Evaluate any agent step in isolation (e.g., whether it selects the appropriate tool). @@ -348,7 +348,7 @@ Agent evaluation can focus on at least 3 things: :::tip -See our [evaluation guide](https://docs.smith.langchain.com/evaluation/concepts/index#agents) for more details on Agent evaluation. +See our [evaluation guide](https://docs.smith.langchain.com/evaluation/concepts#agents) for more details on Agent evaluation. ::: @@ -358,7 +358,7 @@ We can evaluate how well an agent does overall on a task. This basically involve :::tip -See the full overview of agent response evaluation in our [conceptual guide](https://docs.smith.langchain.com/evaluation/concepts/index#evaluating-an-agents-final-response). +See the full overview of agent response evaluation in our [conceptual guide](https://docs.smith.langchain.com/evaluation/concepts#evaluating-an-agents-final-response). ::: @@ -456,7 +456,7 @@ Agents generally make multiple actions. While it is useful to evaluate them end- :::tip -See the full overview of single step evaluation in our [conceptual guide](https://docs.smith.langchain.com/evaluation/concepts/index#evaluating-a-single-step-of-an-agent). +See the full overview of single step evaluation in our [conceptual guide](https://docs.smith.langchain.com/evaluation/concepts#evaluating-a-single-step-of-an-agent). ::: @@ -519,7 +519,7 @@ We can check a trajectory of tool calls using [custom evaluators](https://docs.s :::tip -See the full overview of single step evaluation in our [conceptual guide](https://docs.smith.langchain.com/evaluation/concepts/index#evaluating-an-agents-trajectory). +See the full overview of single step evaluation in our [conceptual guide](https://docs.smith.langchain.com/evaluation/concepts#evaluating-an-agents-trajectory). :::