在 Hugging Face Spaces 上為 LLM 應用程式追蹤設定 Phoenix 可觀測性儀表板

作者：Andrew Reed

Phoenix 是 Arize AI 推出的一個開源可觀測性庫，專為實驗、評估和故障排除而設計。它能讓 AI 工程師和資料科學家快速視覺化資料、評估效能、追蹤問題並匯出資料以進行改進。

追蹤

資料集

實驗

在本 notebook 中，我們將瞭解如何在 Hugging Face Spaces 上部署 Phoenix 可觀測性儀表板，並配置它來自動追蹤 LLM 呼叫，從而全面瞭解您 LLM 應用程式的內部工作情況。

第 1 步：在 Hugging Face Spaces 上部署 Phoenix

雖然 Phoenix 提供了用於本地開發的 notebook 優先選項，但它也可以透過 Docker 部署為一個獨立的儀表板。一個長期執行的、託管的儀表板是提供對您的 LLM 應用程式行為的集中檢視以及與團隊協作的絕佳方式。Hugging Face Spaces 提供了一種簡單的方式來託管帶可選持久儲存的 ML 應用程式，並且它對自定義 Docker 映象的支援使其成為託管 Phoenix 的絕佳平臺 - 讓我們看看它是如何工作的！

首先，我們將複製這個演示空間

我們可以將空間配置為私有或公開，並且它可以存在於我們的使用者名稱空間或組織名稱空間中。我們可以使用預設的免費層級 CPU，並且重要的是，指定我們希望為空間附加一個永續性磁碟。

點選“Duplicate Space”後，Docker 映象將開始構建。這需要幾分鐘才能完成，然後我們會看到一個空的 Phoenix 儀表板。

第 2 步：配置應用程式追蹤

現在我們有了一個執行中的 Phoenix 儀表板，我們可以配置我們的應用程式以使用 OpenTelemetry TracerProvider 自動追蹤 LLM 呼叫。在此示例中，我們將使用 OpenAI 客戶端庫來檢測我們的應用程式，並追蹤從 `openai` Python 包到執行在 Hugging Face 無伺服器推理 API 上的開放 LLM 的呼叫。

首先，我們需要安裝必要的庫

!pip install -q arize-phoenix arize-phoenix-otel openinference-instrumentation-openai openai huggingface-hub

然後，我們將使用 `huggingface_hub` 庫登入 Hugging Face。這將使我們能夠為我們的 Space 和無伺服器推理 API 生成必要的身份驗證。請確保用於身份驗證的 HF 令牌具有對您 Space 所在組織的正確許可權。

from huggingface_hub import interpreter_login

interpreter_login()

現在，我們可以將 Phoenix 客戶端配置到我們正在執行的 Phoenix 儀表板上

1. 透過以下方式註冊 Phoenix tracer provider
   • 指定我們選擇的 `project_name`
   • 將 `endpoint` 值設定為我們 Space 的主機名（可透過儀表板 UI 的“設定”選項卡找到 - 見下文）
   • 將 `headers` 設定為訪問 Space 所需的 Hugging Face 標頭
2. 檢測我們的應用程式程式碼以使用 OpenAI tracer provider

>>> from phoenix.otel import register
>>> from huggingface_hub.utils import build_hf_headers
>>> from openinference.instrumentation.openai import OpenAIInstrumentor

>>> # 1. Register the Phoenix tracer provider
>>> tracer_provider = register(
...     project_name="test",
...     endpoint="https://andrewrreed-phoenix-arize-observability-demo.hf.space" + "/v1/traces",
...     headers=build_hf_headers(),
... )

>>> # 2. Instrument our application code to use the OpenAI tracer provider
>>> OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

🔭 OpenTelemetry Tracing Details 🔭
|  Phoenix Project: test
|  Span Processor: SimpleSpanProcessor
|  Collector Endpoint: https://andrewrreed-phoenix-arize-observability-demo.hf.space/v1/traces
|  Transport: HTTP
|  Transport Headers: {'user-agent': '****', 'authorization': '****'}
|  
|  Using a default SpanProcessor. `add_span_processor` will overwrite this default.
|  
|  `register` has set this TracerProvider as the global OpenTelemetry default.
|  To disable this behavior, call `register` with `set_global_tracer_provider=False`.

第 3 步：進行呼叫並在 Phoenix 儀表板中檢視追蹤

現在，我們可以呼叫一個 LLM 並在 Phoenix 儀表板中檢視追蹤。我們正在使用 OpenAI 客戶端向 Hugging Face 無伺服器推理 API 發出呼叫，該 API 已被檢測以與 Phoenix 協同工作。在這種情況下，我們使用的是 `meta-llama/Llama-3.1-8B-Instruct` 模型。

>>> from openai import OpenAI
>>> from huggingface_hub import get_token

>>> client = OpenAI(
...     base_url="https://api-inference.huggingface.co/v1/",
...     api_key=get_token(),
... )

>>> messages = [{"role": "user", "content": "What does a llama do for fun?"}]

>>> response = client.chat.completions.create(
...     model="meta-llama/Llama-3.1-8B-Instruct",
...     messages=messages,
...     max_tokens=500,
... )

>>> print(response.choices[0].message.content)

Llamas are intelligent and social animals, and they do have ways to entertain themselves and have fun. While we can't directly ask a llama about its personal preferences, we can observe their natural behaviors and make some educated guesses. Here are some things that might bring a llama joy and excitement:

1. **Socializing**: Llamas are herd animals and they love to interact with each other. They'll often engage in play-fighting, neck-wrestling, and other activities to establish dominance and strengthen social bonds. When llamas have a strong social network, it can make them feel happy and content.
2. **Exploring new environments**: Llamas are naturally curious creatures, and they love to explore new surroundings. They'll often investigate their surroundings, sniffing and investigating new sights, sounds, and smells.
3. **Playing with toys**: While llamas don't need expensive toys, they do enjoy playing with objects that stimulate their natural behaviors. For example, a ball or a toy that mimics a target can be an entertaining way to engage them.
4. **Climbing and jumping**: Llamas are agile and athletic animals, and they enjoy using their limbs to climb and jump over obstacles. Providing a safe and stable area for them to exercise their physical abilities can be a fun and engaging experience.
5. **Browsing and foraging**: Llamas have a natural instinct to graze and browse, and they enjoy searching for tasty plants and shrubs. Providing a variety of plants to munch on can keep them engaged and entertained.
6. **Mentally stimulating activities**: Llamas are intelligent animals, and they can benefit from mentally stimulating activities like problem-solving puzzles or learning new behaviors (like agility training or obedience training).

Some fun activities you can try with a llama include:

* Setting up an obstacle course or agility challenge
* Creating a "scavenger hunt" with treats and toys
* Introducing new toys or objects to stimulate their curiosity
* Providing a variety of plants and shrubs to browse and graze on
* Engaging in interactive games like "follow the leader" or "find the treat"

Remember to always prioritize the llama's safety and well-being, and to consult with a veterinarian or a trained llama handler before attempting any new activities or introducing new toys.

如果我們導航回 Phoenix 儀表板，我們可以看到我們 LLM 呼叫的追蹤已被捕獲並顯示！如果您為空間配置了永續性磁碟，那麼每次重啟空間時，所有的追蹤資訊都將被儲存。

附加內容：使用 CrewAI 追蹤多智慧體應用程式

可觀測性的真正威力來自於能夠追蹤和檢查複雜的 LLM 工作流。在這個例子中，我們將安裝並使用 CrewAI 來追蹤一個多智慧體應用程式。

!pip install -q openinference-instrumentation-crewai crewai crewai-tools

和之前一樣，我們將註冊 Phoenix tracer provider 並檢測應用程式程式碼，但這次我們還將取消對現有 OpenAI tracer provider 的檢測以避免衝突。

>>> from opentelemetry import trace
>>> from openinference.instrumentation.crewai import CrewAIInstrumentor

>>> # 0. Uninstrument existing tracer provider and clear the global tracer provider
>>> OpenAIInstrumentor().uninstrument()
>>> if trace.get_tracer_provider():
...     trace.get_tracer_provider().shutdown()
...     trace._TRACER_PROVIDER = None  # Reset the global tracer provider

>>> # 1. Register the Phoenix tracer provider
>>> tracer_provider = register(
...     project_name="crewai",
...     endpoint="https://andrewrreed-phoenix-arize-observability-demo.hf.space" + "/v1/traces",
...     headers=build_hf_headers(),
... )

>>> # 2. Instrument our application code to use the OpenAI tracer provider
>>> CrewAIInstrumentor().instrument(tracer_provider=tracer_provider)

🔭 OpenTelemetry Tracing Details 🔭
|  Phoenix Project: crewai
|  Span Processor: SimpleSpanProcessor
|  Collector Endpoint: https://andrewrreed-phoenix-arize-observability-demo.hf.space/v1/traces
|  Transport: HTTP
|  Transport Headers: {'user-agent': '****', 'authorization': '****'}
|  
|  Using a default SpanProcessor. `add_span_processor` will overwrite this default.
|  
|  `register` has set this TracerProvider as the global OpenTelemetry default.
|  To disable this behavior, call `register` with `set_global_tracer_provider=False`.

現在，我們將使用 CrewAI 定義一個多智慧體應用程式，以研究並撰寫一篇關於 LLM 應用程式中可觀測性和追蹤重要性的部落格文章。

>>> import os
>>> from huggingface_hub import get_token
>>> from crewai_tools import SerperDevTool
>>> from crewai import LLM, Agent, Task, Crew, Process

>>> # Define our LLM using HF's Serverless Inference API
>>> llm = LLM(
...     model="huggingface/meta-llama/Llama-3.1-8B-Instruct",
...     api_key=get_token(),
...     max_tokens=1024,
... )

>>> # Define a tool for searching the web
>>> os.environ["SERPER_API_KEY"] = "YOUR_SERPER_API_KEY"  # must set this value in your environment
>>> search_tool = SerperDevTool()

>>> # Define your agents with roles and goals
>>> researcher = Agent(
...     role="Researcher",
...     goal="Conduct thorough research on up to date trends around a given topic.",
...     backstory="""You work at a leading tech think tank. You have a knack for dissecting complex data and presenting actionable insights.""",
...     verbose=True,
...     allow_delegation=False,
...     tools=[search_tool],
...     llm=llm,
...     max_iter=1,
... )
... writer = Agent(
...     role="Technical Writer",
...     goal="Craft compelling content on a given topic.",
...     backstory="""You are a technical writer with a knack for crafting engaging and informative content.""",
...     llm=llm,
...     verbose=True,
...     allow_delegation=False,
...     max_iter=1,
... )

>>> # Create tasks for your agents
>>> task1 = Task(
...     description="""Conduct comprehensive research and analysis of the importance of observability and tracing in LLM applications.
...   Identify key trends, breakthrough technologies, and potential industry impacts.""",
...     expected_output="Full analysis report in bullet points",
...     agent=researcher,
... )

>>> task2 = Task(
...     description="""Using the insights provided, develop an engaging blog
...   post that highlights the importance of observability and tracing in LLM applications.
...   Your post should be informative yet accessible, catering to a tech-savvy audience.
...   Make it sound cool, avoid complex words so it doesn't sound like AI.""",
...     expected_output="Blog post of at least 3 paragraphs",
...     agent=writer,
... )

>>> # Instantiate your crew with a sequential process
>>> crew = Crew(
...     agents=[researcher, writer],
...     tasks=[task1, task2],
...     verbose=True,
...     process=Process.sequential,
... )

>>> # Get your crew to work!
>>> result = crew.kickoff()

>>> print("------------ FINAL RESULT ------------")
>>> print(result)

# Agent: Researcher
## Task: Conduct comprehensive research and analysis of the importance of observability and tracing in LLM applications.
  Identify key trends, breakthrough technologies, and potential industry impacts.


# Agent: Researcher
## Using tool: Search the internet
## Tool Input: 
"{\"search_query\": \"importance of observability and tracing in LLM applications\"}"
## Tool Output: 

Search results: Title: LLM Observability: The 5 Key Pillars for Monitoring Large Language ...
Link: https://arize.com/blog-course/large-language-model-monitoring-observability/
Snippet: Why leveraging the five pillars of LLM observability is essential for ensuring performance, reliability, and seamless LLM applications.
---
Title: Observability of LLM Applications: Exploration and Practice from the ...
Link: https://www.alibabacloud.com/blog/observability-of-llm-applications-exploration-and-practice-from-the-perspective-of-trace_601604
Snippet: This article clarifies the technical challenges of observability by analyzing LLM application patterns and different concerns.
---
Title: What is LLM Observability? - The Ultimate LLM Monitoring Guide
Link: https://www.confident-ai.com/blog/what-is-llm-observability-the-ultimate-llm-monitoring-guide
Snippet: Observability tools collect and correlate logs, real-time evaluation metrics, and traces to understand the context of unexpected outputs or ...
---
Title: An Introduction to Observability for LLM-based applications using ...
Link: https://opentelemetry.io/blog/2024/llm-observability/
Snippet: Why Observability Matters for LLM Applications · It's vital to keep track of how often LLMs are being used for usage and cost tracking. · Latency ...
---
Title: Understanding LLM Observability - Key Insights, Best Practices ...
Link: https://signoz.io/blog/llm-observability/
Snippet: LLM Observability is essential for maintaining reliable, accurate, and efficient AI applications. Focus on the five pillars: evaluation, tracing ...
---
Title: LLM Observability Tools: 2024 Comparison - lakeFS
Link: https://lakefs.io/blog/llm-observability-tools/
Snippet: LLM observability is the process that enables monitoring by providing full visibility and tracing in an LLM application system, as well as newer ...
---
Title: From Concept to Production with Observability in LLM Applications
Link: https://hadijaveed.me/2024/03/05/tracing-and-observability-in-llm-applications/
Snippet: Traces are essential to understanding the full “path” a request takes in your application, e.g, prompt, query-expansion, RAG retrieved top-k ...
---
Title: The Importance of LLM Observability: A Technical Deep Dive
Link: https://www.linkedin.com/pulse/importance-llm-observability-technical-deep-dive-patrick-carroll-trlqe
Snippet: LLM observability is crucial for any technical team that wants to maintain and improve the reliability, security, and performance of their AI- ...
---
Title: Observability and Monitoring of LLMs - TheBlue.ai
Link: https://theblue.ai/blog/llm-observability-en/
Snippet: LLM-Observability is crucial to maximize the performance and reliability of Large Language Models (LLMs). By systematically capturing and ...
---


導航回 Phoenix 儀表板後，我們可以在新的“crewai”專案中看到我們多智慧體應用程式的追蹤資訊！