langsmith-observability - Tracing, evaluation, and monitoring for LLM applications
Traces LLM calls, evaluates model outputs against datasets, monitors production metrics, and manages evaluation feedback for AI applications.
Tags
Updated: 2026-09-21Capabilities
Typical Inputs
Typical Outputs
What this skill does
- Trace LLM application calls
- Evaluate model outputs against datasets
- Monitor production system metrics
- Manage datasets and test cases
- Collect run feedback and scores
- Pull prompts from prompt hub
Inputs
- LangSmith API key
- LLM inputs and queries
- Evaluation datasets and reference outputs
- User feedback scores and comments
Outputs
- Hierarchical execution traces and runs
- Evaluation metrics and test reports
- Stored datasets and examples
- Recorded feedback entries
Requirements
- Python environment
- langsmith package version 0.2.0 or higher
- Valid LangSmith API key
- Network connectivity to LangSmith platform