Voltar ao ranking

AgentOps-AI/agentops

Pythonagentops.ai

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

agentagentopsaievalsevaluation-metricsllmanthropicautogencost-estimationcrewaigroqlangchain
Crescimento de estrelas
Estrelas
5.7k
Forks
612
Crescimento semanal
Issues
111
3k4k5k
jul. de 2023jul. de 2024jul. de 2025jul. de 2026
ArtefatosPyPIpip install agentops
README
Observability and DevTool platform for AI Agents

Twitter Discord Dashboard Documentation Chat with Docs


AgentOps helps developers build, evaluate, and monitor AI agents. From prototype to production.

Open Source

The AgentOps app is open source under the MIT license. Explore the code in our app directory.

Key Integrations 🔌

OpenAI Agents SDK CrewAI AG2 (AutoGen) Microsoft
LangChain Camel AI LlamaIndex Cohere
📊 Replay Analytics and Debugging Step-by-step agent execution graphs
💸 LLM Cost Management Track spend with LLM foundation model providers
🤝 Framework Integrations Native Integrations with CrewAI, AG2 (AutoGen), Agno, LangGraph, & more
⚒️ Self-Host Want to run AgentOps on your own cloud? You're covered

Quick Start ⌨️

pip install agentops

Session replays in 2 lines of code

Initialize the AgentOps client and automatically get analytics on all your LLM calls.

Get an API key

import agentops

# Beginning of your program (i.e. main.py, __init__.py)
agentops.init( < INSERT YOUR API KEY HERE >)

...

# End of program
agentops.end_session('Success')

All your sessions can be viewed on the AgentOps dashboard

Self-Hosting

Looking to run the full AgentOps app (Dashboard + API backend) on your machine? Follow the setup guide in app/README.md:

Agent Debugging Agent Metadata Chat Viewer Event Graphs
Session Replays Session Replays
Summary Analytics Summary Analytics Summary Analytics Charts

First class Developer Experience

Add powerful observability to your agents, tools, and functions with as little code as possible: one line at a time.
Refer to our documentation

# Create a session span (root for all other spans)
from agentops.sdk.decorators import session

@session
def my_workflow():
    # Your session code here
    return result
# Create an agent span for tracking agent operations
from agentops.sdk.decorators import agent

@agent
class MyAgent:
    def __init__(self, name):
        self.name = name
        
    # Agent methods here
# Create operation/task spans for tracking specific operations
from agentops.sdk.decorators import operation, task

@operation  # or @task
def process_data(data):
    # Process the data
    return result
# Create workflow spans for tracking multi-operation workflows
from agentops.sdk.decorators import workflow

@workflow
def my_workflow(data):
    # Workflow implementation
    return result
# Nest decorators for proper span hierarchy
from agentops.sdk.decorators import session, agent, operation

@agent
class MyAgent:
    @operation
    def nested_operation(self, message):
        return f"Processed: {message}"
        
    @operation
    def main_operation(self):
        result = self.nested_operation("test message")
        return result

@session
def my_session():
    agent = MyAgent()
    return agent.main_operation()

All decorators support:

  • Input/Output Recording
  • Exception Handling
  • Async/await functions
  • Generator functions
  • Custom attributes and names

Integrations 🦾

OpenAI Agents SDK 🖇️

Build multi-agent systems with tools, handoffs, and guardrails. AgentOps natively integrates with the OpenAI Agents SDKs for both Python and TypeScript.

Python

pip install openai-agents

TypeScript

npm install agentops @openai/agents

CrewAI 🛶

Build Crew agents with observability in just 2 lines of code. Simply set an AGENTOPS_API_KEY in your environment, and your crews will get automatic monitoring on the AgentOps dashboard.

pip install 'crewai[agentops]'

AG2 🤖

With only two lines of code, add full observability and monitoring to AG2 (formerly AutoGen) agents. Set an AGENTOPS_API_KEY in your environment and call agentops.init()

Camel AI 🐪

Track and analyze CAMEL agents with full observability. Set an AGENTOPS_API_KEY in your environment and initialize AgentOps to get started.

Installation
pip install "camel-ai[all]==0.2.11"
pip install agentops
import os
import agentops
from camel.agents import ChatAgent
from camel.messages import BaseMessage
from camel.models import ModelFactory
from camel.types import ModelPlatformType, ModelType

# Initialize AgentOps
agentops.init(os.getenv("AGENTOPS_API_KEY"), tags=["CAMEL Example"])

# Import toolkits after AgentOps init for tracking
from camel.toolkits import SearchToolkit

# Set up the agent with search tools
sys_msg = BaseMessage.make_assistant_message(
    role_name='Tools calling operator',
    content='You are a helpful assistant'
)

# Configure tools and model
tools = [*SearchToolkit().get_tools()]
model = ModelFactory.create(
    model_platform=ModelPlatformType.OPENAI,
    model_type=ModelType.GPT_4O_MINI,
)

# Create and run the agent
camel_agent = ChatAgent(
    system_message=sys_msg,
    model=model,
    tools=tools,
)

response = camel_agent.step("What is AgentOps?")
print(response)

agentops.end_session("Success")

Check out our Camel integration guide for more examples including multi-agent scenarios.

Langchain 🦜🔗

AgentOps works seamlessly with applications built using Langchain. To use the handler, install Langchain as an optional dependency:

Installation
pip install agentops[langchain]

To use the handler, import and set

import os
from langchain.chat_models import ChatOpenAI
from langchain.agents import initialize_agent, AgentType
from agentops.integration.callbacks.langchain import LangchainCallbackHandler

AGENTOPS_API_KEY = os.environ['AGENTOPS_API_KEY']
handler = LangchainCallbackHandler(api_key=AGENTOPS_API_KEY, tags=['Langchain Example'])

llm = ChatOpenAI(openai_api_key=OPENAI_API_KEY,
                 callbacks=[handler],
                 model='gpt-3.5-turbo')

agent = initialize_agent(tools,
                         llm,
                         agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
                         verbose=True,
                         callbacks=[handler], # You must pass in a callback handler to record your agent
                         handle_parsing_errors=True)

Check out the Langchain Examples Notebook for more details including Async handlers.

Cohere ⌨️

First class support for Cohere(>=5.4.0). This is a living integration, should you need any added functionality please message us on Discord!

Installation
pip install cohere
import cohere
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)
co = cohere.Client()

chat = co.chat(
    message="Is it pronounced ceaux-hear or co-hehray?"
)

print(chat)

agentops.end_session('Success')
import cohere
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)

co = cohere.Client()

stream = co.chat_stream(
    message="Write me a haiku about the synergies between Cohere and AgentOps"
)

for event in stream:
    if event.event_type == "text-generation":
        print(event.text, end='')

agentops.end_session('Success')

Anthropic ﹨

Track agents built with the Anthropic Python SDK (>=0.32.0).

Installation
pip install anthropic
import anthropic
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)

client = anthropic.Anthropic(
    # This is the default and can be omitted
    api_key=os.environ.get("ANTHROPIC_API_KEY"),
)

message = client.messages.create(
        max_tokens=1024,
        messages=[
            {
                "role": "user",
                "content": "Tell me a cool fact about AgentOps",
            }
        ],
        model="claude-3-opus-20240229",
    )
print(message.content)

agentops.end_session('Success')

Streaming

import anthropic
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)

client = anthropic.Anthropic(
    # This is the default and can be omitted
    api_key=os.environ.get("ANTHROPIC_API_KEY"),
)

stream = client.messages.create(
    max_tokens=1024,
    model="claude-3-opus-20240229",
    messages=[
        {
            "role": "user",
            "content": "Tell me something cool about streaming agents",
        }
    ],
    stream=True,
)

response = ""
for event in stream:
    if event.type == "content_block_delta":
        response += event.delta.text
    elif event.type == "message_stop":
        print("\n")
        print(response)
        print("\n")

Async

import asyncio
from anthropic import AsyncAnthropic

client = AsyncAnthropic(
    # This is the default and can be omitted
    api_key=os.environ.get("ANTHROPIC_API_KEY"),
)


async def main() -> None:
    message = await client.messages.create(
        max_tokens=1024,
        messages=[
            {
                "role": "user",
                "content": "Tell me something interesting about async agents",
            }
        ],
        model="claude-3-opus-20240229",
    )
    print(message.content)


await main()

Mistral 〽️

Track agents built with the Mistral Python SDK (>=0.32.0).

Installation
pip install mistralai

Sync

from mistralai import Mistral
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)

client = Mistral(
    # This is the default and can be omitted
    api_key=os.environ.get("MISTRAL_API_KEY"),
)

message = client.chat.complete(
        messages=[
            {
                "role": "user",
                "content": "Tell me a cool fact about AgentOps",
            }
        ],
        model="open-mistral-nemo",
    )
print(message.choices[0].message.content)

agentops.end_session('Success')

Streaming

from mistralai import Mistral
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)

client = Mistral(
    # This is the default and can be omitted
    api_key=os.environ.get("MISTRAL_API_KEY"),
)

message = client.chat.stream(
        messages=[
            {
                "role": "user",
                "content": "Tell me something cool about streaming agents",
            }
        ],
        model="open-mistral-nemo",
    )

response = ""
for event in message:
    if event.data.choices[0].finish_reason == "stop":
        print("\n")
        print(response)
        print("\n")
    else:
        response += event.text

agentops.end_session('Success')

Async

import asyncio
from mistralai import Mistral

client = Mistral(
    # This is the default and can be omitted
    api_key=os.environ.get("MISTRAL_API_KEY"),
)


async def main() -> None:
    message = await client.chat.complete_async(
        messages=[
            {
                "role": "user",
                "content": "Tell me something interesting about async agents",
            }
        ],
        model="open-mistral-nemo",
    )
    print(message.choices[0].message.content)


await main()

Async Streaming

import asyncio
from mistralai import Mistral

client = Mistral(
    # This is the default and can be omitted
    api_key=os.environ.get("MISTRAL_API_KEY"),
)


async def main() -> None:
    message = await client.chat.stream_async(
        messages=[
            {
                "role": "user",
                "content": "Tell me something interesting about async streaming agents",
            }
        ],
        model="open-mistral-nemo",
    )

    response = ""
    async for event in message:
        if event.data.choices[0].finish_reason == "stop":
            print("\n")
            print(response)
            print("\n")
        else:
            response += event.text


await main()

CamelAI ﹨

Track agents built with the CamelAI Python SDK (>=0.32.0).

Installation
pip install camel-ai[all]
pip install agentops
#Import Dependencies
import agentops
import os
from getpass import getpass
from dotenv import load_dotenv

#Set Keys
load_dotenv()
openai_api_key = os.getenv("OPENAI_API_KEY") or "<your openai key here>"
agentops_api_key = os.getenv("AGENTOPS_API_KEY") or "<your agentops key here>"


You can find usage examples here!.

LiteLLM 🚅

AgentOps provides support for LiteLLM(>=1.3.1), allowing you to call 100+ LLMs using the same Input/Output Format.

Installation
pip install litellm
# Do not use LiteLLM like this
# from litellm import completion
# ...
# response = completion(model="claude-3", messages=messages)

# Use LiteLLM like this
import litellm
...
response = litellm.completion(model="claude-3", messages=messages)
# or
response = await litellm.acompletion(model="claude-3", messages=messages)

LlamaIndex 🦙

AgentOps works seamlessly with applications built using LlamaIndex, a framework for building context-augmented generative AI applications with LLMs.

Installation
pip install llama-index-instrumentation-agentops

To use the handler, import and set

from llama_index.core import set_global_handler

# NOTE: Feel free to set your AgentOps environment variables (e.g., 'AGENTOPS_API_KEY')
# as outlined in the AgentOps documentation, or pass the equivalent keyword arguments
# anticipated by AgentOps' AOClient as **eval_params in set_global_handler.

set_global_handler("agentops")

Check out the LlamaIndex docs for more details.

Llama Stack 🦙🥞

AgentOps provides support for Llama Stack Python Client(>=0.0.53), allowing you to monitor your Agentic applications.

SwarmZero AI 🐝

Track and analyze SwarmZero agents with full observability. Set an AGENTOPS_API_KEY in your environment and initialize AgentOps to get started.

Installation
pip install swarmzero
pip install agentops
from dotenv import load_dotenv
load_dotenv()

import agentops
agentops.init(<INSERT YOUR API KEY HERE>)

from swarmzero import Agent, Swarm
# ...

Evaluations Roadmap 🧭

Platform Dashboard Evals
✅ Python SDK ✅ Multi-session and Cross-session metrics ✅ Custom eval metrics
🚧 Evaluation builder API ✅ Custom event tag tracking 🔜 Agent scorecards
🚧 Javascript/Typescript SDK (Alpha) ✅ Session replays 🔜 Evaluation playground + leaderboard

Debugging Roadmap 🧭

Performance testing Environments LLM Testing Reasoning and execution testing
✅ Event latency analysis 🔜 Non-stationary environment testing 🔜 LLM non-deterministic function detection 🚧 Infinite loops and recursive thought detection
✅ Agent workflow execution pricing 🔜 Multi-modal environments 🚧 Token limit overflow flags 🔜 Faulty reasoning detection
🚧 Success validators (external) 🔜 Execution containers 🔜 Context limit overflow flags 🔜 Generative code validators
🔜 Agent controllers/skill tests ✅ Honeypot and prompt injection detection (PromptArmor) ✅ API bill tracking 🔜 Error breakpoint analysis
🔜 Information context constraint testing 🔜 Anti-agent roadblocks (i.e. Captchas) 🔜 CI/CD integration checks
🔜 Regression testing ✅ Multi-agent framework visualization

Why AgentOps? 🤔

Without the right tools, AI agents are slow, expensive, and unreliable. Our mission is to bring your agent from prototype to production. Here's why AgentOps stands out:

  • Comprehensive Observability: Track your AI agents' performance, user interactions, and API usage.
  • Real-Time Monitoring: Get instant insights with session replays, metrics, and live monitoring tools.
  • Cost Control: Monitor and manage your spend on LLM and API calls.
  • Failure Detection: Quickly identify and respond to agent failures and multi-agent interaction issues.
  • Tool Usage Statistics: Understand how your agents utilize external tools with detailed analytics.
  • Session-Wide Metrics: Gain a holistic view of your agents' sessions with comprehensive statistics.

AgentOps is designed to make agent observability, testing, and monitoring easy.

Star History

Check out our growth in the community:

Logo
Repository Stars
  geekan / MetaGPT 42787
  run-llama / llama_index 34446
  crewAIInc / crewAI 18287
  camel-ai / camel 5166
  superagent-ai / superagent 5050
  iyaja / llama-fs 4713
  ag2ai / ag2 4240
  BasedHardware / Omi 2723
  MervinPraison / PraisonAI 2007
  AgentOps-AI / Jaiqu 272
  swarmzero / swarmzero 195
  strnad / CrewAI-Studio 134
  alejandro-ao / exa-crewai 55
  tonykipkemboi / youtube_yapper_trapper 47
  sethcoast / cover-letter-builder 27
  bhancockio / chatgpt4o-analysis 19
  breakstring / Agentic_Story_Book_Workflow 14
  MULTI-ON / multion-python 13

Generated using github-dependents-info, by Nicolas Vuillamy

Repositórios relacionados
Snailclimb/JavaGuide

Java 面试 & 后端通用面试指南,覆盖计算机基础、数据库、分布式、高并发、系统设计与 AI 应用开发

JavaScriptnpmApache License 2.0javainterview
javaguide.cn
157.2k46.2k
langgenius/dify

Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.

TypeScriptnpmOtheraigpt
dify.ai
149.7k23.6k
TauricResearch/TradingAgents

TradingAgents: Multi-Agents LLM Financial Trading Framework

PythonPyPIApache License 2.0agentfinance
arxiv.org/pdf/2412.20138
94k18.2k
OpenHands/OpenHands

🙌 OpenHands: AI-Driven Development

PythonPyPIOtheragentartificial-intelligence
openhands.dev
81.6k10.4k
lobehub/lobehub

🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.

TypeScriptnpmOtherchatgptopenai
lobehub.com
80.6k15.7k
bytedance/deer-flow

An open-source long-horizon SuperAgent harness that researches, codes, and creates. With the help of sandboxes, memories, tools, skill, subagents and message gateway, it handles different levels of tasks that could take minutes to hours.

PythonPyPIMIT Licenseagentagentic
deerflow.tech
77.6k10.6k
dair-ai/Prompt-Engineering-Guide

🐙 Guides, papers, lessons, notebooks and resources for prompt engineering, context engineering, RAG, and AI Agents.

MDXMIT Licensedeep-learningprompt-engineering
promptingguide.ai
76.8k8.4k
hiyouga/LlamaFactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

PythonPyPIApache License 2.0fine-tuningllama
llamafactory.readthedocs.io
73.4k9k
shareAI-lab/learn-claude-code

Bash is all you need - A nano claude code–like 「agent harness」, built from 0 to 1

PythonPyPIMIT Licenseagentclaude-code
learn.shareai.run
71.9k11.7k
FoundationAgents/MetaGPT

🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming

PythonPyPIMIT Licenseagentgpt
atoms.dev
69.5k8.9k
unslothai/unsloth

Unsloth is a local UI for training and running Gemma 4, Qwen3.6, DeepSeek, Kimi, GLM and other models.

PythonPyPIApache License 2.0fine-tuningllama
unsloth.ai/docs
68.7k6.2k
datawhalechina/hello-agents

📚 《从零开始构建智能体》——从零开始的智能体原理与实践教程

PythonPyPIOtheragenttutorial
hello-agents.datawhale.cc
67.8k8.4k